[Ocfs2-users] Concurrent write performance issues with OCFS2

[Erik Schwartz]

I have a two-node RHEL5 cluster that runs the following Linux kernel and
accompanying OCFS2 module packages:

  * kernel-2.6.18-274.17.1.el5
  * ocfs2-2.6.18-274.17.1.el5-1.4.7-1.el5

A 2.5TB LUN is presented to both nodes via DM-Multipath. I have carved
out a single partition (using the entire LUN), and formatted it with OCFS2:

  # mkfs.ocfs2 -N 2 -L 'foofs' -T datafiles /dev/mapper/bams01p1

Finally, the filesystem is mounted to both nodes with the following options:

  # mount | grep bams01
/dev/mapper/bams01p1 on /foofs type ocfs2
(rw,_netdev,noatime,data=writeback,heartbeat=local)

----------

When a single node is writing arbitrary data (i.e. dd(1) with /dev/zero
as input) to a large (say, 10 GB) file in /foofs, I see the expected
performance of ~850 MB/sec.

If both nodes are concurrently writing large files full of zeros to
/foofs, performance drops way down to ~45 MB/s. I experimented with each
node writing to /foofs/test01/ and /foofs/test02/ subdirectories,
respectively, and found that performance increased slightly to a - still
poor - 65 MB/s.

----------

I understand from searching past mailing list threads that the culprit
is likely related to the negotiation of file locks, and waiting for data
to be flushed to journal / disk.

My two questions are:

1. Does this dramatic write performance slowdown sound reasonable and
expected?

2. Are there any OCFS2-level steps I can take to improve this situation?


Thanks -

-- 
Erik Schwartz <schwartz.erik.c at gmail.com> | GPG key 14F1139B