[Ocfs2-users] Problems with descriptions.

[Vasyl S. Kostroma]

Hi everyone.
The problem is solved.

As I told before, the situation is much standard.

drbd + ocfs2 + nfs4 is not working, 'cause file descriptors (handlers? a lot 
of different translation of this term. I mean first digit in 
/proc/sys/fs/file-nr) never clear. On my systems this number increased up to 
+10000 every minute, so after some time system used to much RAM. Also system 
was unstable.

I changed mount options of clients to nfs3 and everything working fine.
So the problem with ocfs2+nfs4.

Just for information. Who knows who will need it..



-----Исходное сообщение----- 
From: Thomas.Zimolong at bmi.bund.de
Sent: Monday, June 06, 2011 1:58 PM
To: admin at v-sf.info
Cc: ocfs2-users at oss.oracle.com
Subject: AW: [Ocfs2-users] Problems with descriptions.

Hi!

A bit late, but was out of office a few days...

> I do not have any idea, but the allocated descriptors in
> /proc/sys/fs/file-nr increasing every time while drive accessed.
> So after some time allocated descriptions over max value and all
> processes make error “To much opened files” (something like
> this). I do not see any error messages in log files... Any idea?

I think you have to distinguish two things: file handles and open files 
(file
descriptors).

The difference between file handles and open files is, that a process can
have more open files than file handles. That’s because things like current
working directories, memory mapped library files and executable text files
(i.e. the program itself) are counted as open files but don't use file
handles. On the other hand file handles rarely get unallocated, they rather
get unused and are reused by some new process.

As far as I understood, the difference is that a file handle is part of an
additional layer of the C standard I/O library routines. See more on file
handles and file descriptors in
<<http://en.wikipedia.org/wiki/File_descriptor>>.

Concerning the file handle problem: /proc/sys/fs/file-nr shows a statistic 
of
file handles in three values:
First value is the number of allocated file handles, second the number of
free allocated(!) file handles (with Kernel 2.6 always 0) and the third
number is the system wide maximum of allocatable file handles. The maximum 
is
configured in /etc/sysctl.conf by "fs.file-max" and can be controlled via
sysctl (man 8 sysctl) or via /proc/sys/fs/file-max.

Concerning "too many open files" you might have a problem with the maximum
number of allowed open files per user.
Look at "lsof -u <user> | wc -l" to see how many open files all processes of
<user> have.
Look at "ulimit -n", which shows the number of allowed open files for the
user.
Only root can increase the limits (ok, the user can do this between hard and
soft), so for a user different from root you have to configure this in
/etc/security/limits.conf. See the bash manpage for the builtin "ulimit".

Hope this sheds some light on your problem...

Greetz
Thomas