[Ocfs2-users] multipath ocfs2 start by-label missing

[Herbert van den Bergh]

On 1/21/13 1:19 PM, Heiko L. wrote:
>> It looks like multipath is not started, or these devices have just one path.  What is tthe output of:
>> multipath -v3
>> multipath -l
>
> 6 hours later:
> #  blkid | grep s5r6
> /dev/sdb1: LABEL="s5r6l83p1" UUID="b5f5c10d-30bc-455c-8eef-8f08bf71cf54" TYPE="ocfs2"
> /dev/sdb2: LABEL="s5r6l83p2" UUID="f6559525-c70b-4cc2-880b-42f854ff563b" TYPE="ocfs2"
> /dev/mapper/mpath0-part1: LABEL="s5r6l83p1" UUID="b5f5c10d-30bc-455c-8eef-8f08bf71cf54" TYPE="ocfs2"
> /dev/mapper/mpath0-part2: LABEL="s5r6l83p2" UUID="f6559525-c70b-4cc2-880b-42f854ff563b" TYPE="ocfs2"

What is the relevance of the "6 hours later" comment?  Do you mean that 
it took 6 hours for the mpath devices to show up?  Or did you just take 
a 6 hour break?

>
>
> ----------------------------------------------------------
> # reboot
>
>
> # blkid -c /dev/null
>
> # blkid | grep "s5r6"
> /dev/sdb1: LABEL="s5r6l83p1" UUID="b5f5c10d-30bc-455c-8eef-8f08bf71cf54" TYPE="ocfs2"
> /dev/sdb2: LABEL="s5r6l83p2" UUID="f6559525-c70b-4cc2-880b-42f854ff563b" TYPE="ocfs2"

So for some reason the mpath devices have not shown up yet.  Do you know 
what state multipath was in at this time?  Did multipath -ll show the 
mpath maps?

>
> # /etc/init.d/ocfs2.dis start
> Starting Oracle Cluster File System (OCFS2) mount.ocfs2: Device or resource busy while mounting /dev/sdb1 on /mnt/s5r6l83p1.
> Check 'dmesg' for more information on this error.
> mount.ocfs2: Device or resource busy while mounting /dev/sdb2 on /mnt/s5r6l83p2. Check 'dmesg' for more information on this error.
> Failed

Looks like by now multipath may have locked the sd devices underneath 
its mpath devices.  Again, what's the multipath status at this point?

>
>
> # multipath -ll mpath0
> mpath0 (3600601602f1027001a3260cead1ce111) dm-2 DGC,VRAID
> size=500G features='1 queue_if_no_path' hwhandler='1 emc' wp=rw
> |-+- policy='round-robin 0' prio=1 status=active
> | `- 1:0:0:0 sdb 8:16  active ready running
> `-+- policy='round-robin 0' prio=0 status=enabled
>    `- 1:0:1:0 sde 8:64  active ready running

At this point the mpath devices have been created.  Deleting the sd 
devices from /etc/blkid/blkid.tab and rerunning blkid would hopefully 
show the labels on mpath partitions at this point, and mount by label 
should work.

What you might want to do instead of using labels, which are not 
reliable in the case of multipath LUNs, is to create custom bindings for 
your multipath LUNs, and use the names you give to those multipath 
devices instead of the labels.


>
>    - server has 1hba, san has 2connections (EMC SP)
>
> # multipath -v3 | egrep "sdb|sdc|dm-2" | more
> Jan 21 22:31:20 | sdb: not found in pathvec
> Jan 21 22:31:20 | sdb: mask = 0x1f
> Jan 21 22:31:20 | sdb: dev_t = 8:16
> Jan 21 22:31:20 | sdb: size = 1048576000
> Jan 21 22:31:20 | sdb: subsystem = scsi
> Jan 21 22:31:20 | sdb: vendor = DGC
> Jan 21 22:31:20 | sdb: product = VRAID
> Jan 21 22:31:20 | sdb: rev = 0430
> Jan 21 22:31:20 | sdb: h:b:t:l = 1:0:0:0
> Jan 21 22:31:20 | sdb: tgt_node_name = 0x50060160c460267
> Jan 21 22:31:20 | sdb: serial = CKM00094900180
> Jan 21 22:31:20 | sdb: get_state
> Jan 21 22:31:20 | sdb: path checker = emc_clariion (controller setting)
> Jan 21 22:31:20 | sdb: state = running
> Jan 21 22:31:20 | sdb: state = 3
> Jan 21 22:31:20 | sdb: prio = emc (controller setting)
> Jan 21 22:31:20 | sdb: emc prio = 1
> Jan 21 22:31:20 | sdb: getuid = /lib/udev/scsi_id --whitelisted --device=/dev/%n (controller setting)
> Jan 21 22:31:20 | sdb: uid = 3600601602f1027001a3260cead1ce111 (callout)
> Jan 21 22:31:20 | sdc: not found in pathvec
> Jan 21 22:31:20 | sdc: mask = 0x1f
> Jan 21 22:31:20 | sdc: dev_t = 8:32
> Jan 21 22:31:20 | sdc: size = 2124131200
> Jan 21 22:31:20 | sdc: subsystem = scsi
> Jan 21 22:31:20 | sdc: vendor = DGC
> Jan 21 22:31:20 | sdc: product = VRAID
> Jan 21 22:31:20 | sdc: rev = 0430
> Jan 21 22:31:20 | sdc: h:b:t:l = 1:0:0:1
> Jan 21 22:31:20 | sdc: tgt_node_name = 0x50060160c460267
> Jan 21 22:31:20 | sdc: serial = CKM00094900180
> Jan 21 22:31:20 | sdc: get_state
> Jan 21 22:31:20 | sdc: path checker = emc_clariion (controller setting)
> Jan 21 22:31:20 | sdc: state = running
> Jan 21 22:31:20 | sdc: state = 3
> Jan 21 22:31:20 | sdc: prio = emc (controller setting)
> Jan 21 22:31:20 | sdc: emc prio = 0
> Jan 21 22:31:20 | sdc: getuid = /lib/udev/scsi_id --whitelisted --device=/dev/%n (controller setting)
> Jan 21 22:31:20 | sdc: uid = 3600601602f102700a045a69c22bfdf11 (callout)
> Jan 21 22:31:20 | dm-2: device node name blacklisted
> 3600601602f1027001a3260cead1ce111 1:0:0:0 sdb 8:16  1   undef ready  DGC,VRAID
> 3600601602f102700a045a69c22bfdf11 1:0:0:1 sdc 8:32  0   undef ready  DGC,VRAID

Use the above uid's to create custom bindings.

> regards Heiko
>
Thanks,
Herbert.