[Ocfs2-users] OCFS2 performance - disk random access time problem

[Sunil Mushran]

The kernel is old. We fixed this issue in 2.6.30. We have also backported
it to the 1.4 production tree.

The problem was that the inodes being created did not have locality
leading to a directory having inodes that were spaced far apart from
each other. The one place where it really affected performance was "rm".

On 05/30/2010 02:38 AM, Proskurin Kirill wrote:
> Hello.
>
> I plan to use OCFS2 + DRBD for email server.
>
> Problem:
> I use "seeker"  for testing
> http://www.linuxinsight.com/how_fast_is_your_disk.html
>
> And get this:
> Results: 65 seeks/second, 15.23 ms random access time
> Then I do rm of many files - it fals to 10 seeks/second and performance
> is terrible.
>
> What can I do to increase it? What`s wrong?
>
> Below is many info.
>
> What we have:
> Debian lenny 2.6.26-2-amd64
> ocfs2-tools 1.4.1-1
> drbd8-2.6.26-2-amd64  2:8.3.7-1~bpo50+1+2.6.26-21lenny4
> drbd8-source                 2:8.3.7-1~bpo50+1
> drbd8-utils                     2:8.3.7-1~bpo50+1
>
> debugfs.ocfs2 -R "stats" /dev/drdb0
>           Revision: 0.90
>           Mount Count: 0   Max Mount Count: 20
>           State: 0   Errors: 0
>           Check Interval: 0   Last Check: Fri Mar 26 11:54:41 2010
>           Creator OS: 0
>           Feature Compat: 1 BackupSuper
>           Feature Incompat: 16 Sparse
>           Tunefs Incomplete: 0 None
>           Feature RO compat: 1 Unwritten
>           Root Blknum: 17   System Dir Blknum: 18
>           First Cluster Group Blknum: 8
>           Block Size Bits: 12   Cluster Size Bits: 15
>           Max Node Slots: 2
>           Label: ocfs2_drbd0
>           UUID: 75507E99756D4AE986337FD09BC7B2E1
>           Cluster stack: classic o2cb
>           Inode: 2   Mode: 00   Generation: 140519344 (0x86027b0)
>           FS Generation: 140519344 (0x86027b0)
>           Type: Unknown   Attr: 0x0   Flags: Valid System Superblock
>           User: 0 (root)   Group: 0 (root)   Size: 0
>           Links: 0   Clusters: 27476584
>           ctime: 0x4baca081 -- Fri Mar 26 11:54:41 2010
>           atime: 0x0 -- Thu Jan  1 00:00:00 1970
>           mtime: 0x4baca081 -- Fri Mar 26 11:54:41 2010
>           dtime: 0x0 -- Thu Jan  1 00:00:00 1970
>           ctime_nsec: 0x00000000 -- 0
>           atime_nsec: 0x00000000 -- 0
>           mtime_nsec: 0x00000000 -- 0
>           Last Extblk: 0
>           Sub Alloc Slot: Global   Sub Alloc Bit: 65535
>
> /etc/init.d/o2cb status
> Driver for "configfs": Loaded
> Filesystem "configfs": Mounted
> Stack glue driver: Loaded
> Stack plugin "o2cb": Loaded
> Driver for "ocfs2_dlmfs": Loaded
> Filesystem "ocfs2_dlmfs": Mounted
> Checking O2CB cluster ocfs2: Online
> Heartbeat dead threshold = 31
>     Network idle timeout: 15000
>     Network keepalive delay: 2000
>     Network reconnect delay: 2000
> Checking O2CB heartbeat: Active
>
> dmesg | grep -i ocfs2
> [   18.821010] OCFS2 Node Manager 1.5.0
> [   18.826429] OCFS2 DLM 1.5.0
> [   18.829955] ocfs2: Registered cluster interface o2cb
> [   18.849955] OCFS2 DLMFS 1.5.0
> [   18.849955] OCFS2 User DLM kernel interface loaded
> [  304.329154] OCFS2 1.5.0
>
> OCFS2 mount options:
> nodev,noauto,noatime,data=writeback
>
> Nodes connected via 1Gbit\s LAN.
> HDD is SATA Raptor 10k RPM.
>
> Mail storage is 260Gb with 4 million files(maildir)
>
> cat /proc/meminfo
> MemTotal:      4058444 kB
> MemFree:       3014752 kB
> Buffers:        585544 kB
> Cached:          71036 kB
> SwapCached:          0 kB
> Active:         326432 kB
> Inactive:       553292 kB
> SwapTotal:     1951856 kB
> SwapFree:      1951164 kB
> Dirty:             172 kB
> Writeback:           0 kB
> AnonPages:      223136 kB
> Mapped:          15820 kB
> Slab:           100492 kB
> SReclaimable:    89588 kB
> SUnreclaim:      10904 kB
> PageTables:       4632 kB
> NFS_Unstable:        0 kB
> Bounce:              0 kB
> WritebackTmp:        0 kB
> CommitLimit:   3981076 kB
> Committed_AS:   457124 kB
> VmallocTotal: 34359738367 kB
> VmallocUsed:    279904 kB
> VmallocChunk: 34359458171 kB
> HugePages_Total:     0
> HugePages_Free:      0
> HugePages_Rsvd:      0
> HugePages_Surp:      0
> Hugepagesize:     2048 kB
>
>