<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"><head><meta http-equiv=Content-Type content="text/html; charset=us-ascii"><meta name=Generator content="Microsoft Word 14 (filtered medium)"><style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";
mso-fareast-language:EN-US;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:"Calibri","sans-serif";
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri","sans-serif";
mso-fareast-language:EN-US;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]--></head><body lang=EN-AU link=blue vlink=purple><div class=WordSection1><p class=MsoNormal>Hi All,<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>I have stumbled across (via Google) a post on this mailing list in relation to performance issues with OCFS2.<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>A little overview of our setup:<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>3 x Dell Poweredge R200 servers, w/8GB RAM, Dual Gig NIC’s running VSphere 4.1 (ESXi w/Enterprise License)<o:p></o:p></p><p class=MsoNormal>1 x Dell Poweredge MD3000i ISCSI SAN w/15x 1tb SATA drives in RAID6<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>Each ESXi server runs 2 Gentoo Virtual machines running kernel 2.6.34 with ocfs2-tools – workload consists of lighttpd, Apache & Squid, with caching from the SAN to the local vm disks & RAM.<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>Our problem lies within performance of the OCFS2 volume (which is ~10TB) over the disks.<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>The iowait is constantly high (30-40% per server), and even though there are plenty of inodes and physical disk free, we cannot explain the problem.<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>dnetwww2 ~ # df -h<o:p></o:p></p><p class=MsoNormal>Filesystem Size Used Avail Use% Mounted on<o:p></o:p></p><p class=MsoNormal>rootfs 18G 6.5G 11G 39% /<o:p></o:p></p><p class=MsoNormal>/dev/sda3 18G 6.5G 11G 39% /<o:p></o:p></p><p class=MsoNormal>rc-svcdir 1.0M 72K 952K 8% /lib64/rc/init.d<o:p></o:p></p><p class=MsoNormal>udev 10M 184K 9.9M 2% /dev<o:p></o:p></p><p class=MsoNormal>shm 2.0G 0 2.0G 0% /dev/shm<o:p></o:p></p><p class=MsoNormal>/dev/sdb1 247G 29G 206G 13% /cache<o:p></o:p></p><p class=MsoNormal>/dev/ram0 190M 13M 177M 7% /home/core<o:p></o:p></p><p class=MsoNormal>/dev/ram1 190M 60M 130M 32% /home/moddb<o:p></o:p></p><p class=MsoNormal>/dev/ram2 190M 20M 170M 11% /home/desura<o:p></o:p></p><p class=MsoNormal>/dev/mapper/360024e8000758ab1000007624c1525dc1<o:p></o:p></p><p class=MsoNormal> 9.8T 2.1T 7.7T 22% /home/shared<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>dnetwww2 ~ # cat /etc/fstab<o:p></o:p></p><p class=MsoNormal>--snip--<o:p></o:p></p><p class=MsoNormal>/dev/mapper/360024e8000758ab1000007624c1525dc1 /home/shared ocfs2 commit=15,heartbeat=local,data=writeback,noatime,user_xattr 1 2<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>dnetwww2 ~ # multipath -ll<o:p></o:p></p><p class=MsoNormal>360024e8000758ab1000007624c1525dcdm-0 ,<o:p></o:p></p><p class=MsoNormal>[size=9.7T][features=1 queue_if_no_path][hwhandler=0]<o:p></o:p></p><p class=MsoNormal>\_ round-robin 0 [prio=6][active]<o:p></o:p></p><p class=MsoNormal> \_ #:#:#:# sdc 8:32 [active][ready]<o:p></o:p></p><p class=MsoNormal> \_ #:#:#:# sde 8:64 [active][ready]<o:p></o:p></p><p class=MsoNormal>\_ round-robin 0 [prio=0][enabled]<o:p></o:p></p><p class=MsoNormal> \_ #:#:#:# sdd 8:48 [active][ghost]<o:p></o:p></p><p class=MsoNormal> \_ #:#:#:# sdf 8:80 [active][ghost]<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>Now if I mount an ext4 formatted lun/partition from the MD3000i (mapped via iscsi&multipath-tools) I can read/write to it at 125MB/s with no issues. The ocfs2 mounted volume struggles to sustain 25-30MB/s read/write. :-(<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>We have spent countless hours working (troubleshooting/debugging) this now without result. We’ve even replaced both controllers, switches, network cards and so on in an attempt to rule out a specific hardware cause, but it seems to be ocfs2 related.<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>I’ve noted there are a number of new ocfs2 patches in Linux 2.6.35 & the yet to be released 2.6.36 – would like to know if any of these resolve this issue before we are forced to ditch ocfs2 and go back to NFS.<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>Cheers,<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>Greg<o:p></o:p></p></div></body></html>