[Ocfs2-users] 2-node configuration ?

[Sunil Mushran]

Laurent Neiger wrote:
> Hi all,
>
> We're building a 2-node cluster that we'd like to work in 
> active/active mode.
> Since drbd 8.x this feature is possible so we hope to achieve a 
> cluster where
> the 2 nodes would be able to share the load, not only one and the 
> other node
> in stanby mode, waiting for the primary's failure...
>
> In order to have a drbd active/active mode, we need to have a cluster 
> filesystem,
> handling DLM. Our choice went to OCFS-2 for its numerous features.
>
> But we encountered a problem : when we cut off the network link of, 
> let's say
> node2, for simulating a crash, we managed to make node 1 fence node2 
> via drbd
> in order to avoid a split-brain configuration, but node1 then 
> self-fence itself
> apparently due to ocfs2.

If you cut the comm link between two nodes in a 2 node cluster, the 
lower node
number survives and the higher node number fences. So I am not sure why 
node1
fenced. Do you have the netconsole logs? That should tell us something.

> After some researches, we understood this seems to be a "normal" 
> feature :
> in a 2-node cluster, when communication (and so ocfs2 heartbeat) is lost,
> the remaining node has no way to know if it's its peer which is down or
> itself which is away from the cluster, so it self-fences.
>
> Our first idea was so to add a third node, so that the 2 remainning nodes
> still communicate and no ocfs2 self-fence is launched.
>
> But ocfs2 heartbeat, as explained in the FAQ, writes to the heartbeat 
> system file,
> which has to be shared. If we set up a third node with a little ocfs2 
> partition
> and o2cb, it doesn't appear into the cluster, even when declared as 
> third node
> in /etc/ocfs2/cluster.conf (of each node). Because the ocfs2 partition 
> on the
> third node is not shared, so ocfs2 heartbeat is not shared.
>
> If we run ocfs2_hb_ctl -I -d /dev/drbd0 on node0 and node1, we get back
> the same reference for heartbeat, but a different one on node2 (third 
> node).
> And in /var/log/kern.log on node 0, we have
> ...
> Feb 28 11:46:42 maq1 kernel: ocfs2_dlm: Node 1 joins domain 
> FB305B8298D94DCA9F9BF75D0AA09B8D
> Feb 28 11:46:42 maq1 kernel: ocfs2_dlm: Nodes in domain 
> ("FB305B8298D94DCA9F9BF75D0AA09B8D"): 0 1
>
> But nothing about node2...
>
> And we cannot share a common partition as drbd only works with 2 peers...
>
> Would anyone have any hint about how we could solve this issue ?
>
> Is there a way to make a 2-node ocfs2 cluster work
> or must we have at least 3 nodes ?
> But if 3 nodes are required, how to make it work with DRBD ?
> Or in 2-node config., can we block self-fencing (but is it desirable) ?
No, one does not have to have 3 nodes when one only wants 2 nodes.

If I have to speculate I would think that node1 fenced because it could not
complete the disk hb io within the hbtimeout. If all you severed was the 
commlink,
then that should not effect the disk hb traffic. That is unless the 
commlink is
also being used by drbd (via linux ha)....