ASMLib is optional software to simplify the use of Automatic Storage Management (ASM) for Oracle databases on Linux.

ASMLib ensures consistent naming of devices across RAC clusters, and also maintains permissions on devices across reboots, a feature that was important until UDEV rules were added to Linux with the 2.5 kernel.

EMC Powerpath is an advanced multi-pathing host based solution that works with EMC arrays to intelligently load balance I/O across all available paths as well as provide fault tolerance by automatically re-routing traffic around failed paths. EMC Powerpath is significantly more powerful and robust that native Linux MPIO.

One reason why many DBAs moved away from ASMLib was that with the RHEL 6.0 release, Oracle withdrew access to ASMLib to customers who did not purchase Oracle Linux support. Oracle has now reversed course on that decision, and with the RHEL 6.4 release ASMLib is again free to download and use for Oracle customers (see My Oracle Support Doc ID 1089399.1)

ASMLib has many advantages but also adds some complications, and the DBA needs to be aware of these, especially when using a multi-pathing solution like EMC Powerpath.

A feature of ASMLib is that devices stamped for ASM are assigned an alias. Each disk will be added to the /dev/oracleasm/disks/ device directory, and presented to ASM using the alias ORCL:diskname.

When Linux sees multiple paths to the same disk or LUN, it creates an entry in the SCSI device table for each path. Therefore a single LUN with two paths may appear to Linux as both /dev/sdg and /dev/sdh.

This is problematic for ASM, since ASM cannot natively handle two or more devices mapping to the same LUN.

Like ASMLib, EMC Powerpath also creates aliases, called pseudo-devices. One pseudo-device is assigned to cover all SCSI devices or paths that map back to the same LUN. EMC Powerpath pseudo-devices will have the naming convention /dev/emcpowerX where X is the device letter.

To see how pseudo-devices map to SCSI devices, use the following command:

$ powermnt display dev=all

This will produce output similar to the following:

Pseudo name=emcpowera
VNX ID=GRUFFDBA [ORACLE_JEDI]
Logical device ID=1234567890ABCDEF [LUN 11 - MYPRODDB- ASM02]
state=alive; policy=CLAROpt; queued-IOs=1
Owner: default=SP B, current=SP B         Array failover mode: 4
==============================================================================
--------------- Host ---------------   - Stor -  -- I/O Path --   -- Stats ---
###  HW Path               I/O Paths    Interf.  Mode     State   Q-IOs Errors
==============================================================================
   2 fnic                   sdas        SP B5    active   alive      1      0
   2 fnic                   sdae        SP A5    active   alive      0      0
   1 fnic                   sdq         SP B4    active   alive      0      0
   1 fnic                   sdc         SP A4    active   alive      0      0

In the above example, the pseudo-device /dev/emcpowera maps to the Linux SCSI devices sdas, sdae, sdq and sdac.

To gain the benefits of EMC Powerpath with ASMLib therefore, we need to stamp the pseudo-device as an ASM disk, not the individual Linux SCSI devices. The ASMLib software must be configured to scan the pseudo-devices and not the SCSI devices, which is achieved by editing the file /etc/sysconfig/oracleasm.

The directives ORACLEASM_SCANORDER and ORACLEASM_SCANEXCLUDE need to be modified as follows:

$ cat /etc/sysconfig/oracleasm

# ORACLEASM_ENABLED: 'true' means to load the driver on boot.
ORACLEASM_ENABLED=true

# ORACLEASM_UID: Default user owning the /dev/oracleasm mount point.
ORACLEASM_UID=grid

# ORACLEASM_GID: Default group owning the /dev/oracleasm mount point.
ORACLEASM_GID=oinstall

# ORACLEASM_SCANBOOT: 'true' means scan for ASM disks on boot.
ORACLEASM_SCANBOOT=true

# ORACLEASM_SCANORDER: Matching patterns to order disk scanning
ORACLEASM_SCANORDER="emcpower"

# ORACLEASM_SCANEXCLUDE: Matching patterns to exclude disks from scan
ORACLEASM_SCANEXCLUDE="sd"

Now when ASMLib starts, it will scan for ASM disks among the Powerpath pseudo-devices instead of the SCSI devices.

Since ASMLib requires a partition table, we can create one on any of the devices covered by the pseudo-device, it will be visible on all devices as well as the pseudo-device.

$ parted /dev/sdc mklabel gpt ; parted /dev/sdc mkpart primary 2048s 100%

Information: Don't forget to update /etc/fstab, if necessary

Information: Don't forget to update /etc/fstab, if necessary

We can now stamp the pseudo-devivce as an ASM disk using ASMLib:

$ oracleasm createdisk DATA01 /dev/emcpowera1
Writing disk header: done
Instantiating disk: done

To ensure that ASM is using the EMC Powerpath pseudo-devices and not the Linux SCSI devices we can verify the mapping using the following Korn shell script, which must be executed as root:

#!/bin/sh

#
# generate a list of ASM disks
#
/etc/init.d/oracleasm querydisk -d `/etc/init.d/oracleasm listdisks -d` | \ 
cut -f2,10,11 -d" " | \ 
perl -pe 's/"(.*)".*\[(.*), *(.*)\]/$1 $2 $3/g;' | \ 
while read v_asmdisk v_minor v_major 
do

#
# generate a list of devices under the oracleasm device 
#
  v_device=`ls -la /dev/oracleasm/disks | grep " $v_minor, *$v_major " | awk '{print $10}'`

#
# match to the scsi device using major and minor device numbers 
#
  v_scsidv=`ls -l /dev/* | grep "$v_minor," | \ 
awk {'print $6 " XASMX "$10'} | \ 
grep "$v_major XASMX" | \
awk {'print $3'}`

#
# report the result
#
  echo "ASM disk $v_asmdisk based on /dev/oracleasm/disks/$v_device [$v_minor, $v_major] maps to $v_scsidv"

done

Executing this on a properly configured system should produce an output similar to this:

ASM disk DISK01 based on /dev/oracleasm/disks/DISK01 [120, 17] maps to /dev/emcpowerb1
ASM disk DISK02 based on /dev/oracleasm/disks/DISK02 [120, 33] maps to /dev/emcpowerc1
ASM disk DISK03 based on /dev/oracleasm/disks/DISK03 [120, 49] maps to /dev/emcpowerd1
ASM disk DISK04 based on /dev/oracleasm/disks/DISK04 [120, 65] maps to /dev/emcpowere1
ASM disk DISK05 based on /dev/oracleasm/disks/DISK05 [120, 81] maps to /dev/emcpowerf1
ASM disk DISK06 based on /dev/oracleasm/disks/DISK06 [120, 97] maps to /dev/emcpowerg1
ASM disk DISK07 based on /dev/oracleasm/disks/DISK07 [120, 113] maps to /dev/emcpowerh1
ASM disk DISK08 based on /dev/oracleasm/disks/DISK08 [120, 129] maps to /dev/emcpoweri1