[Ocfs2-users] Ocfs2-users Digest, Vol 95, Issue 11

[David Johle]

Are you by chance writing your Apache access & error log files to the 
shared volume?

I was having an issue much like this about 2 years ago:
http://www.mail-archive.com/ocfs2-users@oss.oracle.com/msg02927.html

I updated to the 1.4.x release and it got better, but eventually it came back.

The final solution (final as in the problem hasn't returned yet) was 
to move the separate access_log and error_log files to local 
disks.  At the start of each day a script combines the prior day's 
logs of each node and places the resulting file back on to the shared 
volume for further use by statistics analyzers/etc.

OCFS2 has proven to be an excellent shared filesystem over the years, 
but the biggest shortcoming I've found is poor performance during 
high concurrent writes.  Of course, managing write contention is one 
of the most "difficult" tasks for shared resources, so it's somewhat expected.



At 12:00 PM 11/16/2011, Andy Herrero wrote:
>Message: 1
>Date: Wed, 16 Nov 2011 12:36:37 +0100
>From: Andy Herrero <andy at internetborder.se>
>Subject: [Ocfs2-users] Hangups
>To: <ocfs2-users at oss.oracle.com>
>Message-ID: <4EC3A045.4060609 at internetborder.se>
>Content-Type: text/plain; charset="ISO-8859-1"
>
>Hi all!
>
>I have a four-node OCFS2-kluster running RHEL 5.7 (x86_64) on 
>ProLiants DL360-G5's
>with 6 GB RAM each. Two webservers running Apache and two MySQL-servers. I've
>shut down OCFS2 on the DB-servers since it's never really been used 
>there, so it's only
>live on the webservers.
>
>OCFS2 is a 14-disk RAID-10 connected via dedicated QLogic iSCSI-NICs 
>via a Gigabit
>switch. The serves are connected to the same switch on a separate VLAN for the
>heartbeat. They also have dedicated Gbps-NICs for frontend and 
>backend traffic.
>
>The problem is that last couple of weeks the write-performance has 
>intermittently
>slowed down to a crawl and right now the system is pretty much 
>unusable. httpd has
>its DocumentRoot on /san and these processes often go into D-state. 
>"iostat -dmx 1"
>often reports ~99 %util and writing anything hangs disturbingly 
>often. My biggest
>problem right now is that I've got very little to work with (that is 
>no juicy kernel-panics
>and such) and the only OCFS-related stuff in the logs look like this:
[snip]