[Ocfs-users] ocfs seek-performance

[Andreas Piesk]

hi list,


i have a little problem with 2-node RAC using OCFS. the application running on this cluster does 
heavily index-based accesses. the data volumes are SAN volumes connected by fibrechannel.

the throughput does not exceed 10mb/s, average is 7-8 mb/s. i've used 'iostat -x' and got rkB/s=8000 
while %util=100% (device was saturated) from kernel's POV.


i did some benchmarking with Bonnie (http://www.textuality.com/bonnie/):

# ./Bonnie -d /tmp/ocfstest/ -s 1024 -m bn1rac001 -o_direct -p 1
Bonnie 1.4: File '/tmp/ocfstest/Bonnie.7166', size: 1073741824, volumes: 1
Using O_DIRECT for block based I/O
Writing with putc()...         done:  10816 kB/s  59.1 %CPU
Rewriting...                   done:  15607 kB/s   2.2 %CPU
Writing intelligently...       done:  18442 kB/s   2.1 %CPU
Reading with getc()...         done:  20908 kB/s  60.6 %CPU
Reading intelligently...       done:  45169 kB/s   4.4 %CPU
Seeker 1...start 'em...done...
               ---Sequential Output (nosync)--- ---Sequential Input-- --Rnd Seek-
               -Per Char- -DIOBlock- -DRewrite- -Per Char- -DIOBlock- --04k (01)-
Machine    MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU   /sec %CPU
bn1rac 1*1024 10816 59.1 18442  2.1 15607  2.2 20908 60.6 45169  4.4   11.3  0.4


# ./Bonnie -d /root/ext3test/ -s 1024 -m bn1rac001 -o_direct -p 1
Bonnie 1.4: File '/tmp/ext3test/Bonnie.7843', size: 1073741824, volumes: 1
Using O_DIRECT for block based I/O
Writing with putc()...         done:  28471 kB/s  91.6 %CPU
Rewriting...                   done:  15014 kB/s   5.1 %CPU
Writing intelligently...       done:  23810 kB/s   7.0 %CPU
Reading with getc()...         done:  13150 kB/s  42.8 %CPU
Reading intelligently...       done:  42893 kB/s   3.6 %CPU
Seeker 1...start 'em...done...
               ---Sequential Output (nosync)--- ---Sequential Input-- --Rnd Seek-
               -Per Char- -DIOBlock- -DRewrite- -Per Char- -DIOBlock- --04k (01)-
Machine    MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU   /sec %CPU
bn1rac 1*1024 28471 91.6 23810  7.0 15014  5.1 13150 42.8 42893  3.6  182.4  1.8


the first run is on ocfs, the second an ext3. both runs used 'O_DIRECT'. the directory '/tmp' is on 
a local hard disk (takes the SAN out of the game).
the overall performance is not too bad. writing on ocfs is somewhat slower but reading is faster.
the random seeks are worst, 11.3 vs. 182.4 /s. i'm not sure what's the reason. Bonnie uses lseek(2) 
to perform the seek operation.

maybe this seek-problem is the reason for the insufficient read performance of 8mb/s on index-bases 
access? is this type of benchmark valid for ocfs? if yes, can someone could repeat it to verify the 
results?

thanks for reading.


Best,

-ap

-- 
Andreas Piesk
Systemadministration E-/M-Commerce
--------------------------------------------------------------
VÖB-ZVD
Bank für Zahlungsverkehrsdienstleistungen GmbH
Godesberger Allee 88

53175 Bonn

Tel.: +49 (0) 228 9377-535
Fax:  +49 (0) 228 9377-596
E-Mail: andreas.piesk at voeb-zvd.de
Internet: http://www.voeb-zvd.de