[Ocfs2-users] Ocfs2-users Digest, Vol 90, Issue 19

[David Johle]

At 10:18 AM 6/28/2011, ocfs2-users-request at oracle.com wrote:
>Message: 1
>Date: Tue, 28 Jun 2011 10:18:07 -0500
>From: Richard Pickett <richard.pickett at csrtechnologies.com>
>Subject: Re: [Ocfs2-users] multiple cluster doesn't work
>To: S?rgio Surkamp <sergio at gruposinternet.com.br>
>Cc: ocfs2-users at oss.oracle.com
>Message-ID: <BANLkTimvrejSyEhn7nEJYv0OwrD5yENm2A at mail.gmail.com>
>Content-Type: text/plain; charset="iso-8859-1"
>
>That's good news. When I go to format them, do I give them all the same
>name:
>
>mkfs.ocfs2 -b 4k -C 32K -L "*OCFS2Cluster*" -N 2 /dev/*sdb1*
>mkfs.ocfs2 -b 4k -C 32K -L "*OCFS2Cluster*" -N 2 /dev/*sdc1*
>mkfs.ocfs2 -b 4k -C 32K -L "*OCFS2Cluster*" -N 2 /dev/*sdd1*
>
>If "no", then how do I do it?
>
>If "yes", then how does o2cb know the difference between each device when
>they are mounted? (how do the nodes distinguish each device when they are
>communicating to the other nodes in the cluster that they are making a
>change?)


I think you are missing something conceptually about OCFS2.  Perhaps 
I can help out by showing an example of a configuration I have which 
has multiple systems (5), multiple volumes (2), but all within a 
single cluster.

So there are 5 systems, let's call them n1 through n5.
There are 2 devices, which all systems can see as a block device, 
let's call them /dev/d1 & /dev/d2
I will be mounting them at /mnt/v1 and /mnt/v2, but not necessarily 
both on all systems as some only need one.

Here is what the cluster.conf looks like, identical on all 5 nodes:

cluster:
    node_count = 5
    name = foo

node:
    ip_port = 7777
    ip_address = 10.10.10.1
    number = 1
    name = n1
    cluster = live

node:
    ip_port = 7777
    ip_address = 10.10.10.2
    number = 2
    name = n2
    cluster = live

node:
    ip_port = 7777
    ip_address = 10.10.10.3
    number = 3
    name = n3
    cluster = live

node:
    ip_port = 7777
    ip_address = 10.10.10.4
    number = 4
    name = n4
    cluster = live

node:
    ip_port = 7777
    ip_address = 10.10.10.5
    number = 5
    name = n5
    cluster = live



When creating the filesystems, I used a command along the lines of:

mkfs.ocfs2 -b 4k -C 32K -L "OCFS2v1" -N 2 /dev/d1
mkfs.ocfs2 -b 4k -C 32K -L "OCFS2v2" -N 2 /dev/d2

Those only need to be done from one node, of course.

And for easy mounting and automount at boot, /etc/fstab entries would 
be like this:

LABEL=OCFS2v1    /mnt/v1    ocfs2    _netdev,noatime    0 0
LABEL=OCFS2v2    /mnt/v2    ocfs2    _netdev,noatime    0 0


Then you can mount as desired from whatever nodes, no requirement 
that you mount any/all of them in all places:

mkdir /mnt/v1
mount /mnt/v1

mkdir /mnt/v2
mount /mnt/v2


So on my setup, I currently have volume 1 mounted on nodes 2 & 3 & 4 
and volume 2 mounted on nodes 1 & 4 & 5.  This can be seen with 
mounted.ocfs2 commands:


root at n1:~# mounted.ocfs2 -f
Device                FS     Nodes
/dev/d2               ocfs2  n1, n4, n5

root at n3:~# mounted.ocfs2 -f
Device                FS     Nodes
/dev/d1               ocfs2  n2, n3, n4

root at n4:~# mounted.ocfs2 -f
Device                FS     Nodes
/dev/d1               ocfs2  n2, n3, n4
/dev/d2               ocfs2  n1, n4, n5


If I were to go on node 2 and do a "umount /mnt/v1" then run the 
above command again on node 4, I would see:

root at n4:~# mounted.ocfs2 -f
Device                FS     Nodes
/dev/d1               ocfs2  n3, n4
/dev/d2               ocfs2  n1, n4, n5


Also note that the actual block device names may not be exactly the 
same on all systems depending on how it allocates them.  In my case 
I'm actually using dm-multipath over multiple sdX created by iscsi to 
volumes on a SAN.  So on one box it could be /dev/sda & /dev/sdb 
presented as /dev/mapper/mpath0, and on another box that same volume 
could by /dev/sde & /dev/sdf presented as /dev/mapper/mpath3.  And 
that's where the -L labels come in handy -- no matter what the block 
devices area called, I created the filesystem with a label that 
relates to what the purpose of the volume is so I can still just 
mount by label and know that I get the right device mounted at the 
correct mount point.  If you want to see what the currently mounted 
volume labels relate to device wise, you can use:  mounted.ocfs2 -d
Or if it's one that is not mounted, you can use:  tunefs.ocfs2 -Q 
"%V\n" /dev/d1