[Ocfs2-users] Experience with NFS exporting ocfs2 filesystems
Eric Jones
ejones at jimmy.harvard.edu
Thu May 4 13:01:10 CDT 2006
Folks,
I wanted to see whether anyone here had any experience with sharing
ocfs2 over NFS. We have 41 nodes accessing an ocfs2 filesystem, one of
them exporting it via NFS to 5 or 10 clients. We've been experiencing a
problem where NFS on the host locks up and all the (NFS) clients start
blocking on I/O to the filesystem (the nfsd processes were in "D"
state). If I log into the NFS server (or any other cluster nodes) the
filesystem is mostly OK, except for the directory containing the
semaphore directories. An ls of that directory hangs and can't be
interrupted with ^C. The only way we've found to recover NFS is to
reboot the host.
We haven't firmly determined the problem is ocfs2 but there are three
things pointing that way:
1) I've never seen this kind of problem serving ext3fs before (weak, I
know)
2) We seem to be able to cause the problem by running a program on our
cluster that uses directories on ocfs2 as semaphores, continually
mkdir'ing, rmdir'ing, and checking existence. A process was also
continuously polling the directories over NFS. If we poll locally
(ocfs2-attached system) the problem does not recur.
3) The underlying ocfs2 filesystem also experiences problems.
Configuration:
40 linux hosts in cluster of i686 architecture running 2.6.13-1.1532_FC4smp
1 linux host in cluster of architecture x86_64 running
2.6.13-1.1532_FC4smp (compiled for x86_64)
x86_64 host is NFS server
all hosts running OCFS2 1.2.1 Wed Apr 26 19:47:10 EDT 2006 (build
bd2f25ba0af9677db3572e3ccd92f739)
I haven't successfully created a complete sysrq-t of the system because
it seems to panic the kernel. If others have had success with this sort
of configuration, please let me know so I can start looking elsewhere.
Thanks,
Eric
--
Eric Jones ejones at jimmy dot harvard dot edu
System Adminstrator Department of Biostatistics & Computational Biology
617-632-2447 Dana-Farber Cancer Institute
More information about the Ocfs2-users
mailing list