[Ocfs2-users] ocfs2 hangs during webserver usage

[Ulf Zimmermann]

> -----Original Message-----
> From: ocfs2-users-bounces at oss.oracle.com [mailto:ocfs2-users-
> bounces at oss.oracle.com] On Behalf Of David Johle
> Sent: 01/28/2009 10:12
> To: jmoseley at corp.xanadoo.com
> Cc: ocfs2-users at oss.oracle.com
> Subject: Re: [Ocfs2-users] ocfs2 hangs during webserver usage
> 
> At 06:32 PM 1/27/2009, jmoseley at corp.xanadoo.com wrote:
> >As others have indicated, I don't think that's going to work very
> well.
> >You've got two different nodes trying to write to the same file
> constantly.
> >I would keep each server's log on a locally mounted file system, or
> simply
> >keep the logs on the OCFS2 filesystem, but have each node write to
> >different log files.
> >
> >Yeah, that makes parsing access_logs slightly more of a problem for
> >producing hit reports, etc, but I think you'll notice performance
> improve.
> 
> 
> Yes, parsing logs is just one good reason for having unified log
> files -- one of the motivations for using OCFS2 even.  If our
> statistics program can handle multiple files, then at least having
> them in a shared directory would be useful.
> 
> Another major area this would affect is web site issue
> troubleshooting which outputs to log files (not the access logs but
> others).  I can only imagine the complexity of having to deal with
> locating specific logging information for a site user who is having
> trouble by going to 5 different nodes to dig through locally stored
> log files.  Or worse yet, trying to correlate actions of multiple
> users who are each hitting different nodes!
> 
> On that note, these other logs are written to by our aplications
> running under Tomcat.  I really am not seeing any similar lags for
> those processes, only from apache.  The only big difference I can see
> between them is the I/O pattern -- apache is usually 1 line per
> request as they are serviced, java web apps are more bursts of
> numerous lines, but not every request.  There is still a non-trivial
> amount of logging happening for these java apps though, so I am
> surprised.  In fact, Tomcat itself is configured to log each request
> with the processing time (used to produce user response time
> statistics), but those shared logs don't seem to be a point of
> contention like the apache access logs.
> 
> For informational purposes, here are some line counts for logs on our
> main web site yesterday:
>    1577860 access log
>       1361 error log
>    4887437 web app log
>     340164 processing time log
>    6806822 total
> 
> So only about 20% of the requests are handled by Tomcat.  The web app
> log actually writes 3x as many lines, but overall it's less data
> (373M vs. 428M) and fewer actual write operations.  This could
> explain why it is not/less prone to these write delays.

1.5 million hits for access log is not that much and you should be able
to use
separate files and then combine it into 1 before processing. The tools
are out
there for that. Another option is to send Apache logs to syslog, which
means you
have now 1 process receiving and writing the logfiles.