[Ocfs2-users] Slow I/O on ocfs2 file system
Michal Vyoral
michal.vyoral at chmi.cz
Tue Dec 14 06:52:54 PST 2010
Hello Sunil,
I am sending the required statistics:
Sunil Mushran píše v Po 13. 12. 2010 v 10:40 -0800:
> Can you email me the following:
>
> debugfs.ocfs2 -R "stats" /dev/mapper/3600a0b800067df83000003ba4ba0f5ef
> debugfs.ocfs2 -R "stat
dd-1G" /dev/mapper/3600a0b800067df83000003ba4ba0f5ef
# debugfs.ocfs2 -R "stats" /dev/mapper/3600a0b800067df83000003ba4ba0f5ef
Revision: 0.90
Mount Count: 0 Max Mount Count: 20
State: 0 Errors: 0
Check Interval: 0 Last Check: Mon Apr 12 12:56:37 2010
Creator OS: 0
Feature Compat: 1 BackupSuper
Feature Incompat: 16 Sparse
Tunefs Incomplete: 0 None
Feature RO compat: 1 Unwritten
Root Blknum: 5 System Dir Blknum: 6
First Cluster Group Blknum: 3
Block Size Bits: 12 Cluster Size Bits: 12
Max Node Slots: 4
Label:
UUID: 91AE6EADEEE542FE957C9392E02B5D90
Cluster stack: classic o2cb
Inode: 2 Mode: 00 Generation: 3493325634 (0xd037eb42)
FS Generation: 3493325634 (0xd037eb42)
Type: Unknown Attr: 0x0 Flags: Valid System Superblock
User: 0 (root) Group: 0 (root) Size: 0
Links: 0 Clusters: 26214400
ctime: 0x4bc31885 -- Mon Apr 12 12:56:37 2010
atime: 0x0 -- Thu Jan 1 00:00:00 1970
mtime: 0x4bc31885 -- Mon Apr 12 12:56:37 2010
dtime: 0x0 -- Thu Jan 1 00:00:00 1970
ctime_nsec: 0x00000000 -- 0
atime_nsec: 0x00000000 -- 0
mtime_nsec: 0x00000000 -- 0
Last Extblk: 0
Sub Alloc Slot: Global Sub Alloc Bit: 65535
# debugfs.ocfs2 -R "stat
dd-1G" /dev/mapper/3600a0b800067df83000003ba4ba0f5ef
Inode: 2677885 Mode: 0644 Generation: 508664856 (0x1e519c18)
FS Generation: 3493325634 (0xd037eb42)
Type: Regular Attr: 0x0 Flags: Valid
User: 0 (root) Group: 0 (root) Size: 1073741824
Links: 1 Clusters: 262144
ctime: 0x4cd40119 -- Fri Nov 5 13:05:29 2010
atime: 0x4cf63a8c -- Wed Dec 1 12:07:40 2010
mtime: 0x4cd40119 -- Fri Nov 5 13:05:29 2010
dtime: 0x0 -- Thu Jan 1 00:00:00 1970
ctime_nsec: 0x2938e3a0 -- 691594144
atime_nsec: 0x02d717ad -- 47650733
mtime_nsec: 0x2938e3a0 -- 691594144
Last Extblk: 1375775
Sub Alloc Slot: 0 Sub Alloc Bit: 312
Tree Depth: 2 Count: 243 Next Free Rec: 5
## Offset Clusters Block#
0 0 63504 1375374
1 63504 63504 1375380
2 127008 63504 7630229
3 190512 63504 1375633
4 254016 8128 7630482
SubAlloc Bit: 141 SubAlloc Slot: 0
Blknum: 1375374 Next Leaf: 0
Tree Depth: 1 Count: 252 Next Free Rec: 252
## Offset Clusters Block#
0 0 252 7629957
1 252 252 7629958
2 504 252 7629959
3 756 252 7629960
4 1008 252 7629961
5 1260 252 7629962
6 1512 252 7629963
7 1764 252 7629964
8 2016 252 7629965
9 2268 252 7629966
10 2520 252 7629967
.... (truncated)
> This will tell us the fs params and the file layout.
>
> Next run the same with different blocksizes. Run the same with bs=4K
and
> bs=1M. iflag=direct will bypass the cache.
>
> echo 3 >/proc/sys/vm/drop_caches
> strace -c dd if=/data/verejna/dd-1G of=/dev/null bs=4K
> echo 3 >/proc/sys/vm/drop_caches
> strace -c dd if=/data/verejna/dd-1G of=/dev/null bs=1M
> strace -c dd if=/data/verejna/dd-1G of=/dev/null bs=4K iflag=direct
> strace -c dd if=/data/verejna/dd-1G of=/dev/null bs=1M iflag=direct
>
> echo 3 >/proc/sys/vm/drop_caches
> strace -c dd if=/dev/... of=/dev/null of=/dev/null bs=4K
> echo 3 >/proc/sys/vm/drop_caches
> strace -c dd if=/dev/... of=/dev/null of=/dev/null bs=1M
> strace -c dd if=/dev/... of=/dev/null of=/dev/null bs=4K iflag=direct
> strace -c dd if=/dev/... of=/dev/null of=/dev/null bs=1M iflag=direct
>
# sync; echo 3 >/proc/sys/vm/drop_caches
# strace -c dd if=/data/verejna/dd-1G of=/dev/null bs=4K
262144+0 records in
262144+0 records out
1073741824 bytes (1.1 GB) copied, 391.75 s, 2.7 MB/s
% time seconds usecs/call calls errors syscall
------ ----------- ----------- --------- --------- ----------------
98.55 0.885950 3 262150 read
1.45 0.013016 0 262147 write
0.00 0.000000 0 14 6 open
0.00 0.000000 0 12 1 close
0.00 0.000000 0 6 fstat
0.00 0.000000 0 1 lseek
0.00 0.000000 0 15 mmap
0.00 0.000000 0 4 mprotect
0.00 0.000000 0 2 munmap
0.00 0.000000 0 3 brk
0.00 0.000000 0 6 rt_sigaction
0.00 0.000000 0 1 rt_sigprocmask
0.00 0.000000 0 5 5 access
0.00 0.000000 0 2 dup2
0.00 0.000000 0 1 execve
0.00 0.000000 0 1 getrlimit
0.00 0.000000 0 1 arch_prctl
0.00 0.000000 0 1 futex
0.00 0.000000 0 1 set_tid_address
0.00 0.000000 0 1 set_robust_list
------ ----------- ----------- --------- --------- ----------------
100.00 0.898966 524374 12 total
# echo 3 >/proc/sys/vm/drop_caches
# strace -c dd if=/data/verejna/dd-1G of=/dev/null bs=1M
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 413.33 s, 2.6 MB/s
% time seconds usecs/call calls errors syscall
------ ----------- ----------- --------- --------- ----------------
100.00 1.944121 1887 1030 read
0.00 0.000025 0 1027 write
0.00 0.000000 0 14 6 open
0.00 0.000000 0 12 1 close
0.00 0.000000 0 6 fstat
0.00 0.000000 0 1 lseek
0.00 0.000000 0 16 mmap
0.00 0.000000 0 4 mprotect
0.00 0.000000 0 2 munmap
0.00 0.000000 0 3 brk
0.00 0.000000 0 6 rt_sigaction
0.00 0.000000 0 1 rt_sigprocmask
0.00 0.000000 0 5 5 access
0.00 0.000000 0 2 dup2
0.00 0.000000 0 1 execve
0.00 0.000000 0 1 getrlimit
0.00 0.000000 0 1 arch_prctl
0.00 0.000000 0 1 futex
0.00 0.000000 0 1 set_tid_address
0.00 0.000000 0 1 set_robust_list
------ ----------- ----------- --------- --------- ----------------
100.00 1.944146 2135 12 total
ng-vvv1:/data/verejna# strace -c dd if=/data/verejna/dd-1G of=/dev/null
bs=4K iflag=direct
262144+0 records in
262144+0 records out
1073741824 bytes (1.1 GB) copied, 2289.85 s, 469 kB/s
% time seconds usecs/call calls errors syscall
------ ----------- ----------- --------- --------- ----------------
99.86 10.222044 39 262150 read
0.14 0.014632 0 262147 write
0.00 0.000000 0 14 6 open
0.00 0.000000 0 12 1 close
0.00 0.000000 0 6 fstat
0.00 0.000000 0 1 lseek
0.00 0.000000 0 15 mmap
0.00 0.000000 0 4 mprotect
0.00 0.000000 0 2 munmap
0.00 0.000000 0 3 brk
0.00 0.000000 0 6 rt_sigaction
0.00 0.000000 0 1 rt_sigprocmask
0.00 0.000000 0 5 5 access
0.00 0.000000 0 2 dup2
0.00 0.000000 0 1 execve
0.00 0.000000 0 1 getrlimit
0.00 0.000000 0 1 arch_prctl
0.00 0.000000 0 1 futex
0.00 0.000000 0 1 set_tid_address
0.00 0.000000 0 1 set_robust_list
------ ----------- ----------- --------- --------- ----------------
100.00 10.236676 524374 12 total
ng-vvv1:/data/verejna# strace -c dd if=/data/verejna/dd-1G of=/dev/null
bs=1M iflag=direct
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 361.797 s, 3.0 MB/s
% time seconds usecs/call calls errors syscall
------ ----------- ----------- --------- --------- ----------------
100.00 1.364085 1324 1030 read
0.00 0.000028 0 1027 write
0.00 0.000000 0 14 6 open
0.00 0.000000 0 12 1 close
0.00 0.000000 0 6 fstat
0.00 0.000000 0 1 lseek
0.00 0.000000 0 16 mmap
0.00 0.000000 0 4 mprotect
0.00 0.000000 0 2 munmap
0.00 0.000000 0 3 brk
0.00 0.000000 0 6 rt_sigaction
0.00 0.000000 0 1 rt_sigprocmask
0.00 0.000000 0 5 5 access
0.00 0.000000 0 2 dup2
0.00 0.000000 0 1 execve
0.00 0.000000 0 1 getrlimit
0.00 0.000000 0 1 arch_prctl
0.00 0.000000 0 1 futex
0.00 0.000000 0 1 set_tid_address
0.00 0.000000 0 1 set_robust_list
------ ----------- ----------- --------- --------- ----------------
100.00 1.364113 2135 12 total
ng-vvv1:/data/verejna# echo 3 >/proc/sys/vm/drop_caches
ng-vvv1:/data/verejna# strace -c dd
if=/dev/mapper/3600a0b800067df83000003ba4ba0f5ef of=/dev/null
of=/dev/null bs=4K count=262144
262144+0 records in
262144+0 records out
1073741824 bytes (1.1 GB) copied, 15.5014 s, 69.3 MB/s
% time seconds usecs/call calls errors syscall
------ ----------- ----------- --------- --------- ----------------
81.93 0.033628 0 262149 read
18.01 0.007391 0 262147 write
0.07 0.000028 2 15 mmap
0.00 0.000000 0 14 6 open
0.00 0.000000 0 12 1 close
0.00 0.000000 0 6 fstat
0.00 0.000000 0 1 lseek
0.00 0.000000 0 4 mprotect
0.00 0.000000 0 2 munmap
0.00 0.000000 0 3 brk
0.00 0.000000 0 6 rt_sigaction
0.00 0.000000 0 1 rt_sigprocmask
0.00 0.000000 0 5 5 access
0.00 0.000000 0 2 dup2
0.00 0.000000 0 1 execve
0.00 0.000000 0 1 getrlimit
0.00 0.000000 0 1 arch_prctl
0.00 0.000000 0 1 futex
0.00 0.000000 0 1 set_tid_address
0.00 0.000000 0 1 set_robust_list
------ ----------- ----------- --------- --------- ----------------
100.00 0.041047 524373 12 total
ng-vvv1:/data/verejna# echo 3 >/proc/sys/vm/drop_caches
ng-vvv1:/data/verejna# strace -c dd
if=/dev/mapper/3600a0b800067df83000003ba4ba0f5ef of=/dev/null
of=/dev/null bs=1M count=1k
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 9.24075 s, 116 MB/s
% time seconds usecs/call calls errors syscall
------ ----------- ----------- --------- --------- ----------------
99.58 0.945906 919 1029 read
0.42 0.004000 4000 1 execve
0.00 0.000000 0 1027 write
0.00 0.000000 0 14 6 open
0.00 0.000000 0 12 1 close
0.00 0.000000 0 6 fstat
0.00 0.000000 0 1 lseek
0.00 0.000000 0 16 mmap
0.00 0.000000 0 4 mprotect
0.00 0.000000 0 2 munmap
0.00 0.000000 0 3 brk
0.00 0.000000 0 6 rt_sigaction
0.00 0.000000 0 1 rt_sigprocmask
0.00 0.000000 0 5 5 access
0.00 0.000000 0 2 dup2
0.00 0.000000 0 1 getrlimit
0.00 0.000000 0 1 arch_prctl
0.00 0.000000 0 1 futex
0.00 0.000000 0 1 set_tid_address
0.00 0.000000 0 1 set_robust_list
------ ----------- ----------- --------- --------- ----------------
100.00 0.949906 2134 12 total
ng-vvv1:/data/verejna# strace -c dd
if=/dev/mapper/3600a0b800067df83000003ba4ba0f5ef of=/dev/null
of=/dev/null bs=4K count=262144 iflag=direct
262144+0 records in
262144+0 records out
1073741824 bytes (1.1 GB) copied, 76.6662 s, 14.0 MB/s
% time seconds usecs/call calls errors syscall
------ ----------- ----------- --------- --------- ----------------
94.79 0.238143 1 262149 read
5.21 0.013084 0 262147 write
0.00 0.000000 0 14 6 open
0.00 0.000000 0 12 1 close
0.00 0.000000 0 6 fstat
0.00 0.000000 0 1 lseek
0.00 0.000000 0 15 mmap
0.00 0.000000 0 4 mprotect
0.00 0.000000 0 2 munmap
0.00 0.000000 0 3 brk
0.00 0.000000 0 6 rt_sigaction
0.00 0.000000 0 1 rt_sigprocmask
0.00 0.000000 0 5 5 access
0.00 0.000000 0 2 dup2
0.00 0.000000 0 1 execve
0.00 0.000000 0 1 getrlimit
0.00 0.000000 0 1 arch_prctl
0.00 0.000000 0 1 futex
0.00 0.000000 0 1 set_tid_address
0.00 0.000000 0 1 set_robust_list
------ ----------- ----------- --------- --------- ----------------
100.00 0.251227 524373 12 total
ng-vvv1:/data/verejna# strace -c dd
if=/dev/mapper/3600a0b800067df83000003ba4ba0f5ef of=/dev/null
of=/dev/null bs=1M count=1k iflag=direct
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 4.75594 s, 226 MB/s
% time seconds usecs/call calls errors syscall
------ ----------- ----------- --------- --------- ----------------
99.91 0.128550 125 1029 read
0.09 0.000111 0 1027 write
0.00 0.000000 0 14 6 open
0.00 0.000000 0 12 1 close
0.00 0.000000 0 6 fstat
0.00 0.000000 0 1 lseek
0.00 0.000000 0 16 mmap
0.00 0.000000 0 4 mprotect
0.00 0.000000 0 2 munmap
0.00 0.000000 0 3 brk
0.00 0.000000 0 6 rt_sigaction
0.00 0.000000 0 1 rt_sigprocmask
0.00 0.000000 0 5 5 access
0.00 0.000000 0 2 dup2
0.00 0.000000 0 1 execve
0.00 0.000000 0 1 getrlimit
0.00 0.000000 0 1 arch_prctl
0.00 0.000000 0 1 futex
0.00 0.000000 0 1 set_tid_address
0.00 0.000000 0 1 set_robust_list
------ ----------- ----------- --------- --------- ----------------
100.00 0.128661 2134 12 total
Michal
More information about the Ocfs2-users
mailing list