[Ocfs2-users] mailcluster advice?

[jmoseley at corp.xanadoo.com]

Sunil, thanks very much.  This information is *exactly* what I (and
probably others) am looking for.  From other posts, it does appear that
others are using multiple mail servers that access mail spool data from the
same OCFS2 filesystem.  Some of those are surely using cluster-aware
software in an environment like RHCS/GFS, and some are probably using a
setup like I describe below.

At this point, I need to start doing some testing.  I've already got two
nodes that have mounted an OCFS2 filesystem (created with the latest
release), now I just need to have both nodes start reading/writing data,
etc.

Thanks again.


James Moseley




                                                                           
             Sunil Mushran                                                 
             <sunil.mushran at or                                             
             acle.com>                                                  To 
                                       jmoseley at corp.xanadoo.com           
             01/06/2009 02:23                                           cc 
             PM                        ocfs2-users at oss.oracle.com          
                                                                   Subject 
                                       Re: [Ocfs2-users] mailcluster       
                                       advice?                             
                                                                           
                                                                           
                                                                           
                                                                           
                                                                           
                                                                           




jmoseley at corp.xanadoo.com wrote:
> I am definitely far from an expert on various file locking schemes,
whether
> they be part of the OS, application, or the filesystem layer.  Other than
> standard Linux flock() or application locking, I had always assumed that
> with NFS, the NFS lock daemon kept two different NFS clients from writing
> to the same file inode concurrently.

Not really. The way this works is that the write()s are always local.
As in, two apps running on two nfs clients will write concurrently to their
local buffers. At some point the writes will be flushed to the server. If
the writes are small, it could be delayed to when the user close()s the
file.
The server will handle each write as it comes. The data in the file will be
the last write that got flushed to the server.

For more read about close-to-open cache consistency at
http://nfs.sourceforge.net/

The lock daemon allows apps to explicitly use posix locking. The locking
is never implicit.

> In the setup I am wondering about, I would running Postfix and
Courier-IMAP
> on two different Linux boxes, each being a node of an OCFS2-mounted
> filesystem.  The applications would be cluster-unaware, but have their
own
> file locking capabilities (flock for Postfix and lockfile, a courier
> locking utility).
>
> The question, therefore, is what kind of file locking does the OCFS2
> locking manager provide on it's own, especially in the case of two
> applications (that utilize no file locking of their own) that are
> cluster-unaware running on two different OCFS2 nodes?

The lock manager (dlm) in ocfs2 ensures that the file system metadata
and data is consistent. As in, if one node writes to a file and a tick
later another node reads the same file, it will see that last write. (Don't
expect nfs to do this.) Check out the ocfs2 1.4 user's guide for more
on this.
http://oss.oracle.com/projects/ocfs2/dist/documentation/v1.4/ocfs2-1_4-usersguide.pdf


The only problem is that this consistency may not be enough for data.
Some apps do not expect multiple processes r/w the files at the same
time. The trick is to see how they lock (or serialize) the other processes.
If they use standard fs locking like flock(), then we are fine. OCFS2
1.4 supports
clustered flock(). If however, they are using some other scheme, like
O_EXCL open
on a file in /tmp (unshared vol) then it won't work.

> Many folks, including myself, have administered multi-server mail setups
> where each mail server is running sendmail/postfix with a
courier-imap/pop3
> backend in an NFS environment with very good success.  That is to say,
they
> all mount and read/write to the same NFS mounted filesystem where the
mail
> spools are located.  The NFS server can either be a Linux box or a
high-end
> Network Appliance.
>
> Another way to put the question is would the above setup work with an
OFCS2
> filesystem, or does OCFS2 depend on the applications running on each node
> to be cluster-aware or at least attempt to do their own file locking

While OCFS2 does not expect the app to be cluster aware as such, it expects
the locking scheme to be compatible. Refer to the o_excl example above.