[Ocfs2-users] ocfs2 hangs during webserver usage
David Johle
djohle at industrialinfo.com
Wed Jan 28 10:11:37 PST 2009
At 06:32 PM 1/27/2009, jmoseley at corp.xanadoo.com wrote:
>As others have indicated, I don't think that's going to work very well.
>You've got two different nodes trying to write to the same file constantly.
>I would keep each server's log on a locally mounted file system, or simply
>keep the logs on the OCFS2 filesystem, but have each node write to
>different log files.
>
>Yeah, that makes parsing access_logs slightly more of a problem for
>producing hit reports, etc, but I think you'll notice performance improve.
Yes, parsing logs is just one good reason for having unified log
files -- one of the motivations for using OCFS2 even. If our
statistics program can handle multiple files, then at least having
them in a shared directory would be useful.
Another major area this would affect is web site issue
troubleshooting which outputs to log files (not the access logs but
others). I can only imagine the complexity of having to deal with
locating specific logging information for a site user who is having
trouble by going to 5 different nodes to dig through locally stored
log files. Or worse yet, trying to correlate actions of multiple
users who are each hitting different nodes!
On that note, these other logs are written to by our aplications
running under Tomcat. I really am not seeing any similar lags for
those processes, only from apache. The only big difference I can see
between them is the I/O pattern -- apache is usually 1 line per
request as they are serviced, java web apps are more bursts of
numerous lines, but not every request. There is still a non-trivial
amount of logging happening for these java apps though, so I am
surprised. In fact, Tomcat itself is configured to log each request
with the processing time (used to produce user response time
statistics), but those shared logs don't seem to be a point of
contention like the apache access logs.
For informational purposes, here are some line counts for logs on our
main web site yesterday:
1577860 access log
1361 error log
4887437 web app log
340164 processing time log
6806822 total
So only about 20% of the requests are handled by Tomcat. The web app
log actually writes 3x as many lines, but overall it's less data
(373M vs. 428M) and fewer actual write operations. This could
explain why it is not/less prone to these write delays.
I will try changing the logging to use separate files for a day or so
to at least see what effect it has with regards to this issue...stay tuned.
More information about the Ocfs2-users
mailing list