[Ocfs2-users] IO performance appears slow

[Nick Geron]

Understood.  Thanks for all your help, Sunil.  Looks like this just isn't a good fit for my environment.

-Nick

From: Sunil Mushran [mailto:sunil.mushran at oracle.com]
Sent: Friday, August 19, 2011 3:54 PM
To: Nick Geron
Cc: ocfs2-users at oss.oracle.com
Subject: Re: [Ocfs2-users] IO performance appears slow

The cluster stack uses the interconnect to negotiate the locks.
That's how it is able to provide data coherency. Other solutions
do not provide that kind of coherency.

If you are referring to interconnect speeds in ms, it is not good.
That unit is typically used for disk access.

On 08/19/2011 01:30 PM, Nick Geron wrote:
Actually those first numbers were from GigE links going out to physical switches and back in.  To optimize the private link, I upgraded the VMs NICs to 10GE (VMXNet3 which is the VMware para virt driver), moved them onto the same host system with a dedicated software switch between them.  The numbers only improved slightly, and got worse on 1 of 100 pings (1ms).

10GE between VMs under the same hypervisor: rtt min/avg/max/mdev = 0.194/0.307/1.003/0.132 ms

What I don't understand is why my OCFS2 cluster suffers so greatly.  There's quite a big difference between wall time of 0.17 seconds to traverse the data on an iSCSI link and the 4 minutes to do the same on OCFS2 with a sub 1ms average latent private interconnect.  For that matter, the whole setup is running on another clustered FS (VMFS3) over the same network to the same SAN.  I guess I'm just a little dumbfounded that OCFS2 is so much more demanding than other clustered FSs and alternative network storage options.

Is the network really the most likely candidate?  If so, is anyone else running OCFS2 from within a VM environment?  Is this technology only worthwhile in the physical world?  Is there a sweet spot for network latency that I should strive for?  The user guide only makes mention of 'low latency' but lacks figures save for heartbeat and timeouts.

-nick
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://oss.oracle.com/pipermail/ocfs2-users/attachments/20110819/934dbf1f/attachment-0001.html