[Ocfs2-users] Slow umounts on SLES10 patchlevel 3 ocfs2

[Marc Grimme]

So I now have two figures from two different clusters. Both are quite slow during restarts. Having two filesystems mounted.

Cluster1 (that one that last time took very long):
Clusterlocks held by filesystem..
1788AD39151A4E76997420D62A778E65: 274258 locks
1EFA64C36FD54AB48B734A99E7F45A73: 576842 locks
Clusterresources held by filesystem..
1788AD39151A4E76997420D62A778E65: 214545 resources
1EFA64C36FD54AB48B734A99E7F45A73: 469319 resources

Second cluster (also takes quite long):
Clusterlocks held by filesystem..
1EDBCFF0CAB24D0CAE91CB2DA241E8CA: 717186 locks
585462C2FA5A428D913A3CBDBC77E116: 68 locks
Clusterresources held by filesystem..
1EDBCFF0CAB24D0CAE91CB2DA241E8CA: 587471 resources
585462C2FA5A428D913A3CBDBC77E116: 20 resources


Let me know if you need more information.

Thanks
Marc.
----- "Sunil Mushran" <sunil.mushran at oracle.com> wrote:

> It was designed to run in prod envs.
> 
> On 07/07/2011 12:21 AM, Marc Grimme wrote:
> > Sunil,
> > can I query those figures during runtime of a productive cluster?
> > Or might it influence the availability performance what ever?
> >
> > Thanks for your help.
> > Marc.
> > ----- "Sunil Mushran"<sunil.mushran at oracle.com>  wrote:
> >
> >> umount is a two step process. First the fs frees the inodes. Then
> the
> >> o2dlm takes stock of all active resources and migrates ones that
> are
> >> still in use. This typically takes some time. But I have never
> heard
> >> of it taking 45 mins.
> >>
> >> But I guess it could be if one has a lot of resources. Lets start
> by
> >> getting a count.
> >>
> >> This will dump the number of cluster locks held by the fs.
> >> # for vol in /sys/kernel/debug/ocfs2/*
> >>       do
> >>           count=$(wc -l ${vol}/locking_state | cut -f1 -d' ');
> >>           echo "$(basename ${vol}): ${count} locks" ;
> >>       done;
> >>
> >> This will dump the number of lock resources known to the dlm.
> >> # for vol in /sys/kernel/debug/o2dlm/*
> >>       do
> >>           count=$(grep -c "^NAME:" ${vol}/locking_state);
> >>           echo "$(basename ${vol}): ${count} resources" ;
> >>       done;
> >>
> >> The debugfs needs to be mounted for this to work.
> >> mount -t debugfs none /sys/kernel/debug
> >>
> >> Sunil
> >>
> >> On 07/06/2011 08:20 AM, Marc Grimme wrote:
> >>> Hi,
> >>> we are using a SLES10 Patchlevel 3 with 12 Nodes hosting tomcat
> >> application servers.
> >>> The cluster was running some time (about 200 days) without
> >> problems.
> >>> Recently we needed to shutdown the cluster for maintenance and
> >> experianced very long times for the umount of the filesystem. It
> took
> >> something like 45 minutes each node and filesystem (12 x 45
> minutes
> >> shutdown time).
> >>> As a result the planned downtime had to be extended ;-) .
> >>>
> >>> Is there any tuning option or the like to make those umounts
> faster
> >> or is this something we have to live with?
> >>> Thanks for your help.
> >>> If you need more information let me know.
> >>>
> >>> Marc.
> >>>
> >>> Some info on the configuration:
> >>> ---------------------------X8-----------------------------------
> >>> # /sbin/modinfo ocfs2
> >>> filename:
> >> /lib/modules/2.6.16.60-0.54.5-smp/kernel/fs/ocfs2/ocfs2.ko
> >>> license:        GPL
> >>> author:         Oracle
> >>> version:        1.4.1-1-SLES
> >>> description:    OCFS2 1.4.1-1-SLES Wed Jul 23 18:33:42 UTC 2008
> >> (build f922955d99ef972235bd0c1fc236c5ddbb368611)
> >>> srcversion:     986DD1EE4F5ABD8A44FF925
> >>> depends:        ocfs2_dlm,jbd,ocfs2_nodemanager
> >>> supported:      yes
> >>> vermagic:       2.6.16.60-0.54.5-smp SMP gcc-4.1
> >>> atix at CAS12:~>   /sbin/modinfo ocfs2_dlm
> >>> filename:
> >> /lib/modules/2.6.16.60-0.54.5-smp/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko
> >>> license:        GPL
> >>> author:         Oracle
> >>> version:        1.4.1-1-SLES
> >>> description:    OCFS2 DLM 1.4.1-1-SLES Wed Jul 23 18:33:42 UTC
> 2008
> >> (build f922955d99ef972235bd0c1fc236c5ddbb368611)
> >>> srcversion:     16FE87920EA41CA613E6609
> >>> depends:        ocfs2_nodemanager
> >>> supported:      yes
> >>> vermagic:       2.6.16.60-0.54.5-smp SMP gcc-4.1
> >>> parm:           dlm_purge_interval_ms:int
> >>> parm:           dlm_purge_locks_max:int
> >>> # rpm -qa ocfs2*
> >>> ocfs2-tools-1.4.0-0.9.9
> >>> ocfs2console-1.4.0-0.9.9
> >>> ---------------------------X8-----------------------------------
> >>> The kernel version is 2.6.16.60-0.54.5-smp
> >>>
> >>>
> >>
> ______________________________________________________________________________
> >>> Marc Grimme
> >>>
> >>> E-Mail: grimme at atix.de

-- 
______________________________________________________________________________

Marc Grimme

Tel: +49 89 4523538-14
Fax: +49 89 9901766-0
E-Mail: grimme at atix.de

ATIX Informationstechnologie und Consulting AG | Einsteinstrasse 10 |
85716 Unterschleissheim | www.atix.de

Enterprise Linux einfach online kaufen: www.linux-subscriptions.com

Registergericht: Amtsgericht München, Registernummer: HRB 168930, USt.-Id.:
DE209485962 | Vorstand: Thomas Merz (Vors.), Marc Grimme, Mark Hlawatschek, Jan R. Bergrath |
Vorsitzender des Aufsichtsrats: Dr. Martin Buss