[Ocfs2-users] Too much journaling or not ?
greg at greg.net.au
greg at greg.net.au
Sun Aug 22 05:37:56 PDT 2010
Hi All,
I have stumbled across (via Google) a post on this mailing list in relation
to performance issues with OCFS2.
A little overview of our setup:
3 x Dell Poweredge R200 servers, w/8GB RAM, Dual Gig NIC's running VSphere
4.1 (ESXi w/Enterprise License)
1 x Dell Poweredge MD3000i ISCSI SAN w/15x 1tb SATA drives in RAID6
Each ESXi server runs 2 Gentoo Virtual machines running kernel 2.6.34 with
ocfs2-tools - workload consists of lighttpd, Apache & Squid, with caching
from the SAN to the local vm disks & RAM.
Our problem lies within performance of the OCFS2 volume (which is ~10TB)
over the disks.
The iowait is constantly high (30-40% per server), and even though there are
plenty of inodes and physical disk free, we cannot explain the problem.
dnetwww2 ~ # df -h
Filesystem Size Used Avail Use% Mounted on
rootfs 18G 6.5G 11G 39% /
/dev/sda3 18G 6.5G 11G 39% /
rc-svcdir 1.0M 72K 952K 8% /lib64/rc/init.d
udev 10M 184K 9.9M 2% /dev
shm 2.0G 0 2.0G 0% /dev/shm
/dev/sdb1 247G 29G 206G 13% /cache
/dev/ram0 190M 13M 177M 7% /home/core
/dev/ram1 190M 60M 130M 32% /home/moddb
/dev/ram2 190M 20M 170M 11% /home/desura
/dev/mapper/360024e8000758ab1000007624c1525dc1
9.8T 2.1T 7.7T 22% /home/shared
dnetwww2 ~ # cat /etc/fstab
--snip--
/dev/mapper/360024e8000758ab1000007624c1525dc1 /home/shared ocfs2
commit=15,heartbeat=local,data=writeback,noatime,user_xattr 1 2
dnetwww2 ~ # multipath -ll
360024e8000758ab1000007624c1525dcdm-0 ,
[size=9.7T][features=1 queue_if_no_path][hwhandler=0]
\_ round-robin 0 [prio=6][active]
\_ #:#:#:# sdc 8:32 [active][ready]
\_ #:#:#:# sde 8:64 [active][ready]
\_ round-robin 0 [prio=0][enabled]
\_ #:#:#:# sdd 8:48 [active][ghost]
\_ #:#:#:# sdf 8:80 [active][ghost]
Now if I mount an ext4 formatted lun/partition from the MD3000i (mapped via
iscsi&multipath-tools) I can read/write to it at 125MB/s with no issues. The
ocfs2 mounted volume struggles to sustain 25-30MB/s read/write. :-(
We have spent countless hours working (troubleshooting/debugging) this now
without result. We've even replaced both controllers, switches, network
cards and so on in an attempt to rule out a specific hardware cause, but it
seems to be ocfs2 related.
I've noted there are a number of new ocfs2 patches in Linux 2.6.35 & the yet
to be released 2.6.36 - would like to know if any of these resolve this
issue before we are forced to ditch ocfs2 and go back to NFS.
Cheers,
Greg
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://oss.oracle.com/pipermail/ocfs2-users/attachments/20100822/2040dbdb/attachment.html
More information about the Ocfs2-users
mailing list