[Ocfs2-users] mailcluster advice?

[Sunil Mushran]

jmoseley at corp.xanadoo.com wrote:
> I am definitely far from an expert on various file locking schemes, whether
> they be part of the OS, application, or the filesystem layer.  Other than
> standard Linux flock() or application locking, I had always assumed that
> with NFS, the NFS lock daemon kept two different NFS clients from writing
> to the same file inode concurrently.

Not really. The way this works is that the write()s are always local.
As in, two apps running on two nfs clients will write concurrently to their
local buffers. At some point the writes will be flushed to the server. If
the writes are small, it could be delayed to when the user close()s the 
file.
The server will handle each write as it comes. The data in the file will be
the last write that got flushed to the server.

For more read about close-to-open cache consistency at 
http://nfs.sourceforge.net/

The lock daemon allows apps to explicitly use posix locking. The locking
is never implicit.

> In the setup I am wondering about, I would running Postfix and Courier-IMAP
> on two different Linux boxes, each being a node of an OCFS2-mounted
> filesystem.  The applications would be cluster-unaware, but have their own
> file locking capabilities (flock for Postfix and lockfile, a courier
> locking utility).
>
> The question, therefore, is what kind of file locking does the OCFS2
> locking manager provide on it's own, especially in the case of two
> applications (that utilize no file locking of their own) that are
> cluster-unaware running on two different OCFS2 nodes?

The lock manager (dlm) in ocfs2 ensures that the file system metadata
and data is consistent. As in, if one node writes to a file and a tick
later another node reads the same file, it will see that last write. (Don't
expect nfs to do this.) Check out the ocfs2 1.4 user's guide for more
on this.
http://oss.oracle.com/projects/ocfs2/dist/documentation/v1.4/ocfs2-1_4-usersguide.pdf

The only problem is that this consistency may not be enough for data.
Some apps do not expect multiple processes r/w the files at the same
time. The trick is to see how they lock (or serialize) the other processes.
If they use standard fs locking like flock(), then we are fine. OCFS2 
1.4 supports
clustered flock(). If however, they are using some other scheme, like 
O_EXCL open
on a file in /tmp (unshared vol) then it won't work.

> Many folks, including myself, have administered multi-server mail setups
> where each mail server is running sendmail/postfix with a courier-imap/pop3
> backend in an NFS environment with very good success.  That is to say, they
> all mount and read/write to the same NFS mounted filesystem where the mail
> spools are located.  The NFS server can either be a Linux box or a high-end
> Network Appliance.
>
> Another way to put the question is would the above setup work with an OFCS2
> filesystem, or does OCFS2 depend on the applications running on each node
> to be cluster-aware or at least attempt to do their own file locking

While OCFS2 does not expect the app to be cluster aware as such, it expects
the locking scheme to be compatible. Refer to the o_excl example above.