Here's what I got from debugfs.ocfs2 -R "stats". I have to type it out manually, so I'm only including the "features" lines:<div><br></div><blockquote style="margin:0 0 0 40px;border:none;padding:0px">
<div>Feature Compat: 3 backup-super strict-journal-super</div><div>Feature Incompat: 16208 sparse extended-slotmap inline-data metaecc xattr indexed-dirs refcount discontig-bg</div><div>Feature RO compat: 7 unwritten usrquota grpquota</div>
</blockquote><div><br></div><div>Some other info that may be interesting:</div><div><br></div><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><div>Links: 0 Clusters: 52428544</div></blockquote><div><br><br>
<div class="gmail_quote">On Wed, Feb 1, 2012 at 1:04 PM, Sunil Mushran <span dir="ltr"><<a href="mailto:sunil.mushran@oracle.com">sunil.mushran@oracle.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
debugfs.ocfs2 -R "stats" /dev/mapper/...<br>
I want to see the features enabled.<br>
<br>
The main issue with large metdata is the fsck timing. The recently tagged 1.8 release of the tools has much better fsck performance.<div class="HOEnZb"><div class="h5"><br>
<br>
On 02/01/2012 05:25 AM, Mark Hampton wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
We have an application that has many processing threads writing more<br>
than a billion files ranging from 2KB – 50KB, with 50% under<br>
8KB (currently there are 700 million files). The files are never<br>
deleted or modified – they are written once, and read infrequently. The<br>
files are hashed so that they are evenly distributed across ~1,000,000<br>
subdirectories up to 3 levels deep, with up to 1000 files per<br>
directory. The directories are structured like this:<br>
<br>
0/00/00<br>
<br>
0/00/01<br>
<br>
…<br>
<br>
F/FF/FE<br>
<br>
F/FF/FF<br>
<br>
The files need to be readable and writable across a number of<br>
servers. The NetApp filer we purchased for this project has both NFS and<br>
iSCSI capabilities.<br>
<br>
We first tried doing this via NFS. After writing 700 million files (12<br>
TB) into a single NetApp volume, file-write performance became abysmally<br>
slow. We can't create more than 200 files per second on the NetApp<br>
volume, which is about 20% of our required performance target of 1000<br>
files per second. It appears that most of the file-write time is going<br>
towards stat and inode-create operations.<br>
<br>
So I now I’m trying the same thing with OCFS2 over iSCSI. I created 16<br>
luns on the NetApp. The 16 luns became 16 OCFS2 filesystems with 16<br>
different mount points on our servers.<br>
<br>
With this configuration I was initially able to write ~1800 files per<br>
second. Now that I have completed 100 million files, performance has<br>
dropped to ~1500 files per second.<br>
<br>
I’m using OEL 6.1 (2.6.32-100 kernel) with OCFS2 version 1.6. The<br>
application servers have 128GB of memory. I created my OCFS2<br>
filesystems as follows:<br>
<br>
mkfs.ocfs2 –T mail –b 4k –C 4k –L <my label> --fs-features=indexed-dirs<br>
–fs-feature-level=max-features /dev/mapper/<my device><br>
<br>
And I mount them with these options:<br>
<br>
_netdev,commit=30,noatime,<u></u>localflocks,localalloc=32<br>
<br>
So my questions are these:<br>
<br>
<br>
1) Given a billion files sized 2KB – 50KB, with 50% under 8KB, do I have<br>
the optimal OCFS2 filesystem and mount-point configurations?<br>
<br>
<br>
2) Should I split the files across even more filesystems? Currently I<br>
have them split across 16 OCFS2 filesystems.<br>
<br>
Thanks a billion!<br>
</blockquote>
</div></div></blockquote></div><br></div>