[Ocfs2-users] OCFS2 Multipath Configuration

[Ulf Zimmermann]

On our 3Par SAN we use O2CB_HEARTBEAT_THRESHOLD of 76 (180 seconds). This is the time needed for the controller to fully recover in case of a crash or software upgrade. Multipath is configured with a polling_interval of 10, no_path_retry of 60. With these settings we are able to survive SAN switch crash (has happened), SAN controller crash (has happened) and SAN controller upgrades.


> -----Original Message-----
> From: ocfs2-users-bounces at oss.oracle.com [mailto:ocfs2-users-
> bounces at oss.oracle.com] On Behalf Of Elliott Perrin
> Sent: Thursday, March 18, 2010 6:58 PM
> To: ocfs2-users at oss.oracle.com
> Subject: Re: [Ocfs2-users] OCFS2 Multipath Configuration
> 
> I have seen node fences occur on all of our OCFS2 systems when the
> O2CB_HEARTBEAT_THRESHOLD is set less than 31 on both FC and iSCSI
> connected systems. In some testing I have done I found that all nodes
> would panic with O2CB_HEARTBEAT_THRESHOLD set to any value less than 15
> and we would see single node fencing with values between 15 and 30.
> With the O2CB_HEARTBEAT_THRESHOLD set to 31 or greater we do not see a
> fence occur when fabric disruption occurs, again on both iSCSI and FC.
> 
> This is with a multipath configuration that includes
> 
> polling_interval "5"
> no_path_retry "5"
> features "1 queue_if_no_path"
> failback "immediate"
> selector "round-robin 0"
> 
> as some of the configuration variables used in the setup.
> 
> As long as the output of mount (not mounted.ocfs2) shows that your file
> system mount is from a /dev/mapper created multipath device then OCFS2
> is using the multipath provided device.
> 
> 
> > -----Original Message-----
> > From: ocfs2-users-bounces at oss.oracle.com [mailto:ocfs2-users-
> > bounces at oss.oracle.com] On Behalf Of Sunil Mushran
> > Sent: Thursday, March 18, 2010 2:11 PM
> > To: David Johle
> > Cc: ocfs2-users at oss.oracle.com
> > Subject: Re: [Ocfs2-users] OCFS2 Multipath Configuration
> >
> > Yeah.. mounted is a bit dumb. In the next release, it will recognize
> > /dev/mapper devices. We still need to teach it to handle multipathing
> > fully.
> >
> > David Johle wrote:
> > > I'm not sure about why mounted.ocfs2 is showing both the dm and the
> > > sd devices for the same volume.  But this could all be very similar
> > > to a problem I've experienced with OCFS2 & mount finding the right
> > > device with multipathing.
> > >
> > > See the following thread for some more insight:
> > > http://oss.oracle.com/pipermail/ocfs2-users/2009-March/003391.html
> > >
> > >
> > >
> > > At 04:12 PM 3/15/2010, you wrote:
> > >
> > >> Date: Wed, 16 Dec 2009 14:20:45 -0500
> > >> From: "Nyburg, Daryl" <Daryl.Nyburg at utoledo.edu>
> > >> Subject: [Ocfs2-users] OCFS2 Multipath Configuration
> > >> To: <ocfs2-users at oss.oracle.com>
> > >>
> > >> Hello All,
> > >>     I am having some problems configuring OCFS2 to use only the
> > >> multipath device name.  We have been doing failover testing with
> our
> > >> ISCI SAN and as soon as we unplug one NIC the following messages
> > appear
> > >> in /var/log/messages and the system reboots.
> > >>
> > >>
> > >> Dec 16 12:39:03 mcprac01 kernel: (56,6):o2hb_write_timeout:172
> > ERROR:
> > >> Heartbeat write timeout to device dm-34 after 120000 milliseconds
> > >> Dec 16 12:39:03 mcprac01 kernel: (56,6):o2hb_stop_all_regions:1967
> > >> ERROR: stopping heartbeat on all active regions.
> > >> Dec 16 12:39:03 mcprac01 kernel: ocfs2 is very sorry to be fencing
> > this
> > >> system by restarting
> > >>
> > >>
> > >> RHEL 5.3 Kernel 2.6.18-128.el5
> > >> OCFS2 Versions
> > >>   ocfs2console-1.4.3-1.el5
> > >>   ocfs2-2.6.18-128.el5-1.4.2-1.el5
> > >>   ocfs2-tools-1.4.3-1.el5
> > >>
> > >> Why does this show all device paths? Is there anyway to tell OCFS2
> > to
> > >> ignore the /dev/sd* devices ?
> > >> $ mounted.ocfs2 -d
> > >> Device                FS     UUID
> > Label
> > >> /dev/sdf1             ocfs2  2eaddbd4-fac6-4c83-a86d-357215730b23
> > >> /dev/dm-25          ocfs2  2eaddbd4-fac6-4c83-a86d-357215730b23
> > >> /dev/sdaj1           ocfs2  2eaddbd4-fac6-4c83-a86d-357215730b23
> > >> /dev/sdd1            ocfs2  199f76a6-280a-46c6-812e-50712170a823
> > >> /dev/dm-32          ocfs2  199f76a6-280a-46c6-812e-50712170a823
> > >> /dev/sdai1           ocfs2  199f76a6-280a-46c6-812e-50712170a823
> > >>
> > >
> > > _______________________________________________
> > > Ocfs2-users mailing list
> > > Ocfs2-users at oss.oracle.com
> > > http://oss.oracle.com/mailman/listinfo/ocfs2-users
> > >
> >
> >
> > _______________________________________________
> > Ocfs2-users mailing list
> > Ocfs2-users at oss.oracle.com
> > http://oss.oracle.com/mailman/listinfo/ocfs2-users
> 
> _______________________________________________
> Ocfs2-users mailing list
> Ocfs2-users at oss.oracle.com
> http://oss.oracle.com/mailman/listinfo/ocfs2-users