<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<meta name="Generator" content="Microsoft Word 14 (filtered medium)">
<!--[if !mso]><style>v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style><![endif]--><style><!--
/* Font Definitions */
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p.MsoAcetate, li.MsoAcetate, div.MsoAcetate
{mso-style-priority:99;
mso-style-link:"Texte de bulles Car";
margin:0cm;
margin-bottom:.0001pt;
font-size:8.0pt;
font-family:"Tahoma","sans-serif";}
span.hoenzb
{mso-style-name:hoenzb;}
span.EmailStyle18
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
span.TextedebullesCar
{mso-style-name:"Texte de bulles Car";
mso-style-priority:99;
mso-style-link:"Texte de bulles";
font-family:"Tahoma","sans-serif";
mso-fareast-language:FR;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri","sans-serif";
mso-fareast-language:EN-US;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:70.85pt 70.85pt 70.85pt 70.85pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="FR" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Thank you very much for your answers !<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<table class="MsoNormalTable" border="0" cellspacing="0" cellpadding="0" style="border-collapse:collapse">
<tbody>
<tr style="height:87.55pt">
<td width="148" valign="top" style="width:110.9pt;padding:0cm 5.4pt 0cm 5.4pt;height:87.55pt">
<p class="MsoNormal" style="margin-bottom:10.0pt;line-height:115%"><span style="font-size:10.0pt;line-height:115%;font-family:"Calibri","sans-serif";color:#1F497D"><img width="132" height="117" id="Image_x0020_1" src="cid:image001.png@01CDD280.8764A7E0" alt="Description : Description : Description : cid:image001.png@01CD01F3.35091200"></span><span style="font-size:10.0pt;line-height:115%;font-family:"Calibri","sans-serif";color:#1F497D"><o:p></o:p></span></p>
</td>
<td width="242" valign="top" style="width:181.65pt;padding:0cm 5.4pt 0cm 5.4pt;height:87.55pt">
<p class="MsoNormal"><b><span style="font-size:8.0pt;font-family:"Calibri","sans-serif";color:#559796"><o:p> </o:p></span></b></p>
<p class="MsoNormal"><b><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#559796">Amaury FRANCOIS</span></b><span style="font-size:14.0pt;font-family:"Calibri","sans-serif";color:#559796"> •</span><span style="font-size:14.0pt;font-family:"Calibri","sans-serif";color:#F0AB37">
</span><b><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#F0AB37">Ingénieur</span></b><span style="font-family:"Calibri","sans-serif";color:#559796"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Calibri","sans-serif";color:#559796">Mobile +33 (0)6 88 12 62 54<o:p></o:p></span></p>
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><a href="mailto:amaury.francois@digora.com">amaury.francois@digora.com</a></span></b><b><span style="font-size:10.0pt;font-family:"Calibri","sans-serif";color:#559796">
<o:p></o:p></span></b></p>
<p class="MsoNormal"><b><span style="font-size:6.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></b></p>
<p class="MsoNormal"><b><span style="font-size:9.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Siège Social – 66 rue du Marché Gare – 67200 STRASBOURG<o:p></o:p></span></b></p>
<p class="MsoNormal"><span style="font-size:9.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Tél : 0 820 200 217 - +33 (0)3 88 10 49 20
</span><span style="font-size:10.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p></o:p></span></p>
</td>
<td width="251" valign="top" style="width:188.55pt;padding:0cm 5.4pt 0cm 5.4pt;height:87.55pt">
<p class="MsoNormal" style="margin-bottom:10.0pt;line-height:115%"><span style="font-size:10.0pt;line-height:115%;font-family:"Calibri","sans-serif";color:#1F497D"><img border="0" width="237" height="122" id="Image_x0020_2" src="cid:image002.jpg@01CDD280.8764A7E0" alt="Description : test"></span><span style="font-size:10.0pt;line-height:115%;font-family:"Calibri","sans-serif";color:#1F497D"><o:p></o:p></span></p>
</td>
</tr>
</tbody>
</table>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">De :</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> Sunil Mushran [mailto:sunil.mushran@gmail.com]
<br>
<b>Envoyé :</b> mercredi 5 décembre 2012 00:22<br>
<b>À :</b> Erik Schwartz<br>
<b>Cc :</b> Amaury Francois; ocfs2-users@oss.oracle.com<br>
<b>Objet :</b> Re: [Ocfs2-users] "ls" taking ages on a directory containing 900000 files<o:p></o:p></span></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal">1.5 ms per inode. Times 900K files equals 22 mins.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Large dirs are a problem is all file systems. The degree of problem<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">depends on the overhead. An easy solution around is to shard the<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">files into multilevel dirs. Like a 2 level structure of a 1000 files in<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">1000 dirs. Or, a 3 level structure with even fewer files per dir.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Or you could use the other approach suggested. Avoids stat()<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">by disabling color-ls. Or just use plain find.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><o:p> </o:p></p>
<div>
<p class="MsoNormal">On Tue, Dec 4, 2012 at 3:16 PM, Erik Schwartz <<a href="mailto:schwartz.erik.c@gmail.com" target="_blank">schwartz.erik.c@gmail.com</a>> wrote:<o:p></o:p></p>
<p class="MsoNormal">Amaury, you can see in strace output that it's performing a stat on<br>
every file.<br>
<br>
Try simply:<br>
<br>
$ /bin/ls<br>
<br>
My guess is you're using a system where "ls" is aliased to use options<br>
that are more expensive.<br>
<br>
Best regards -<br>
<br>
Erik<o:p></o:p></p>
<div>
<div>
<p class="MsoNormal"><br>
<br>
On 12/4/12 5:12 PM, Amaury Francois wrote:<br>
> The strace looks like this (on all files) :<br>
><br>
><br>
><br>
> 1354662591.755319<br>
> lstat64("TEW_STRESS_TEST_VM.1K_100P_10000F.P069_F01589.txt",<br>
> {st_mode=S_IFREG|0664, st_size=1000, ...}) = 0 <0.001389><br>
><br>
> 1354662591.756775<br>
> lstat64("TEW_STRESS_TEST_VM.1K_100P_10000F.P035_F01592.txt",<br>
> {st_mode=S_IFREG|0664, st_size=1000, ...}) = 0 <0.001532><br>
><br>
> 1354662591.758376<br>
> lstat64("TEW_STRESS_TEST_VM.1K_100P_10000F.P085_F01559.txt",<br>
> {st_mode=S_IFREG|0664, st_size=1000, ...}) = 0 <0.001429><br>
><br>
> 1354662591.759873<br>
> lstat64("TEW_STRESS_TEST_VM.1K_100P_10000F.P027_F01569.txt",<br>
> {st_mode=S_IFREG|0664, st_size=1000, ...}) = 0 <0.001377><br>
><br>
> 1354662591.761317<br>
> lstat64("TEW_STRESS_TEST_VM.1K_100P_10000F.P002_F01581.txt",<br>
> {st_mode=S_IFREG|0664, st_size=1000, ...}) = 0 <0.001420><br>
><br>
> 1354662591.762804<br>
> lstat64("TEW_STRESS_TEST_VM.1K_100P_10000F.P050_F01568.txt",<br>
> {st_mode=S_IFREG|0664, st_size=1000, ...}) = 0 <0.001345><br>
><br>
> 1354662591.764216<br>
> lstat64("TEW_STRESS_TEST_VM.1K_100P_10000F.P089_F01567.txt",<br>
> {st_mode=S_IFREG|0664, st_size=1000, ...}) = 0 <0.001541><br>
><br>
> 1354662591.765828<br>
> lstat64("TEW_STRESS_TEST_VM.1K_100P_10000F.P010_F01594.txt",<br>
> {st_mode=S_IFREG|0664, st_size=1000, ...}) = 0 <0.001358><br>
><br>
> 1354662591.767252<br>
> lstat64("TEW_STRESS_TEST_VM.1K_100P_10000F.P045_F01569.txt",<br>
> {st_mode=S_IFREG|0664, st_size=1000, ...}) = 0 <0.001396><br>
><br>
> 1354662591.768715<br>
> lstat64("TEW_STRESS_TEST_VM.1K_100P_10000F.P036_F01592.txt",<br>
> {st_mode=S_IFREG|0664, st_size=1000, ...}) = 0 <0.002072><br>
><br>
> 1354662591.770854<br>
> lstat64("TEW_STRESS_TEST_VM.1K_100P_10000F.P089_F01568.txt",<br>
> {st_mode=S_IFREG|0664, st_size=1000, ...}) = 0 <0.001722><br>
><br>
> 1354662591.772643<br>
> lstat64("TEW_STRESS_TEST_VM.1K_100P_10000F.P009_F01600.txt",<br>
> {st_mode=S_IFREG|0664, st_size=1000, ...}) = 0 <0.001281><br>
><br>
> 1354662591.773992<br>
> lstat64("TEW_STRESS_TEST_VM.1K_100P_10000F.P022_F01583.txt",<br>
> {st_mode=S_IFREG|0664, st_size=1000, ...}) = 0 <0.001413><br>
><br>
><br>
><br>
> We are using a 32 bits architecture, can it be the cause of the kernel<br>
> not having enough memory ? Any possibility to change this behavior ?<br>
><br>
><br>
><o:p></o:p></p>
</div>
</div>
<div>
<p class="MsoNormal">> Description : Description : Description : <a href="cid:image001.png@01CD01F3.35091200">
cid:image001.png@01CD01F3.35091200</a><br>
><br>
><br>
><o:p></o:p></p>
</div>
<p class="MsoNormal">> * *<br>
><br>
> *Amaury FRANCOIS* • *Ingénieur*<o:p></o:p></p>
<div>
<p class="MsoNormal">><br>
> Mobile <a href="tel:%2B33%20%280%296%2088%20%2012%2062%2054">+33 (0)6 88 12 62 54</a><br>
><o:p></o:p></p>
</div>
<p class="MsoNormal">> *<a href="mailto:amaury.francois@digora.com">amaury.francois@digora.com</a> <mailto:<a href="mailto:amaury.francois@digora.com">amaury.francois@digora.com</a>>***<br>
><br>
> * *<br>
><br>
> *Siège Social – 66 rue du Marché Gare – 67200 STRASBOURG*<o:p></o:p></p>
<div>
<p class="MsoNormal">><br>
> Tél : 0 820 200 217 - <a href="tel:%2B33%20%280%293%2088%2010%2049%2020">+33 (0)3 88 10 49 20</a><br>
><br>
><br>
><o:p></o:p></p>
</div>
<p class="MsoNormal">> Description : test<br>
><br>
><br>
><br>
><br>
><br>
> *De :*Sunil Mushran [mailto:<a href="mailto:sunil.mushran@gmail.com">sunil.mushran@gmail.com</a>]<br>
> *Envoyé :* mardi 4 décembre 2012 18:29<br>
> *À :* Amaury Francois<br>
> *Cc :* <a href="mailto:ocfs2-users@oss.oracle.com">ocfs2-users@oss.oracle.com</a><br>
> *Objet :* Re: [Ocfs2-users] "ls" taking ages on a directory containing<o:p></o:p></p>
<div>
<p class="MsoNormal">> 900000 files<br>
><br>
><br>
><br>
> strace -p PID -ttt -T<br>
><br>
><br>
><br>
> Attach and get some timings. The simplest guess is that the system lacks<br>
> memory to cache all the inodes<br>
><br>
> and thus has to hit disk (and more importantly take cluster locks) for<br>
> the same inode repeatedly. The user<br>
><br>
> guide has a section in NOTES explaining this.<br>
><br>
><br>
><br>
><br>
><br>
> On Tue, Dec 4, 2012 at 8:54 AM, Amaury Francois<o:p></o:p></p>
</div>
<div>
<div>
<p class="MsoNormal">> <<a href="mailto:amaury.francois@digora.com">amaury.francois@digora.com</a> <mailto:<a href="mailto:amaury.francois@digora.com">amaury.francois@digora.com</a>>> wrote:<br>
><br>
> Hello,<br>
><br>
><br>
><br>
> We are running OCFS2 1.8 and on a kernel UEK2. An ls on a directory<br>
> containing approx. 1 million of files is very long (1H). The features<br>
> we have activated on the filesystem are the following :<br>
><br>
><br>
><br>
> [root@pa-oca-app10 ~]# debugfs.ocfs2 -R "stats" /dev/sdb1<br>
><br>
> Revision: 0.90<br>
><br>
> Mount Count: 0 Max Mount Count: 20<br>
><br>
> State: 0 Errors: 0<br>
><br>
> Check Interval: 0 Last Check: Fri Nov 30 19:30:17 2012<br>
><br>
> Creator OS: 0<br>
><br>
> Feature Compat: 3 backup-super strict-journal-super<br>
><br>
> Feature Incompat: 32592 sparse extended-slotmap inline-data<br>
> metaecc xattr indexed-dirs refcount discontig-bg clusterinfo<br>
><br>
> Tunefs Incomplete: 0<br>
><br>
> Feature RO compat: 1 unwritten<br>
><br>
> Root Blknum: 5 System Dir Blknum: 6<br>
><br>
> First Cluster Group Blknum: 3<br>
><br>
> Block Size Bits: 12 Cluster Size Bits: 12<br>
><br>
> Max Node Slots: 8<br>
><br>
> Extended Attributes Inline Size: 256<br>
><br>
> Label: exchange2<br>
><br>
> UUID: 2375EAF4E4954C4ABB984BDE27AC93D5<br>
><br>
> Hash: 2880301520 (0xabade9d0)<br>
><br>
> DX Seeds: 1678175851 1096448356 79406012 (0x6406ee6b 0x415a7964<br>
> 0x04bba3bc)<br>
><br>
> Cluster stack: o2cb<br>
><br>
> Cluster name: appcluster<br>
><br>
> Cluster flags: 1 Globalheartbeat<br>
><br>
> Inode: 2 Mode: 00 Generation: 3567595533 (0xd4a5300d)<br>
><br>
> FS Generation: 3567595533 (0xd4a5300d)<br>
><br>
> CRC32: 0c996202 ECC: 0819<br>
><br>
> Type: Unknown Attr: 0x0 Flags: Valid System Superblock<br>
><br>
> Dynamic Features: (0x0)<br>
><br>
> User: 0 (root) Group: 0 (root) Size: 0<br>
><br>
> Links: 0 Clusters: 5242635<br>
><br>
> ctime: 0x508eac6b 0x0 -- Mon Oct 29 17:18:51.0 2012<br>
><br>
> atime: 0x0 0x0 -- Thu Jan 1 01:00:00.0 1970<br>
><br>
> mtime: 0x508eac6b 0x0 -- Mon Oct 29 17:18:51.0 2012<br>
><br>
> dtime: 0x0 -- Thu Jan 1 01:00:00 1970<br>
><br>
> Refcount Block: 0<br>
><br>
> Last Extblk: 0 Orphan Slot: 0<br>
><br>
> Sub Alloc Slot: Global Sub Alloc Bit: 65535<br>
><br>
><br>
><br>
><br>
><br>
> May inline-data or xattr be the source of the problem ?<br>
><br>
><br>
><br>
> Thank you.<br>
><br>
><br>
><o:p></o:p></p>
</div>
</div>
<div>
<p class="MsoNormal">> Description : Description : Description : <a href="cid:image001.png@01CD01F3.35091200">
cid:image001.png@01CD01F3.35091200</a><br>
><br>
><br>
><o:p></o:p></p>
</div>
<p class="MsoNormal">> * *<br>
><br>
> *Amaury FRANCOIS* • *Ingénieur*<o:p></o:p></p>
<div>
<p class="MsoNormal">><br>
> Mobile <a href="tel:%2B33%20%280%296%2088%20%2012%2062%2054">+33 (0)6 88 12 62 54</a><o:p></o:p></p>
</div>
<p class="MsoNormal">> <<a href="tel:%2B33%20%280%296%2088%C2%A0%2012%2062%2054">tel:%2B33%20%280%296%2088%C2%A0%2012%2062%2054</a>><br>
><br>
> *<a href="mailto:amaury.francois@digora.com">amaury.francois@digora.com</a> <mailto:<a href="mailto:amaury.francois@digora.com">amaury.francois@digora.com</a>>*<br>
><br>
> * *<br>
><br>
> *Siège Social – 66 rue du Marché Gare – 67200 STRASBOURG*<o:p></o:p></p>
<div>
<p class="MsoNormal">><br>
> Tél : 0 820 200 217 - <a href="tel:%2B33%20%280%293%2088%2010%2049%2020">+33 (0)3 88 10 49 20</a><o:p></o:p></p>
</div>
<p class="MsoNormal">> <<a href="tel:%2B33%20%280%293%2088%2010%2049%2020">tel:%2B33%20%280%293%2088%2010%2049%2020</a>><o:p></o:p></p>
<div>
<p class="MsoNormal">><br>
><br>
><br>
> Description : test<br>
><br>
><br>
><br>
><br>
><br>
><br>
> _______________________________________________<br>
> Ocfs2-users mailing list<o:p></o:p></p>
</div>
<p class="MsoNormal">> <a href="mailto:Ocfs2-users@oss.oracle.com">Ocfs2-users@oss.oracle.com</a> <mailto:<a href="mailto:Ocfs2-users@oss.oracle.com">Ocfs2-users@oss.oracle.com</a>><br>
> <a href="https://oss.oracle.com/mailman/listinfo/ocfs2-users" target="_blank">https://oss.oracle.com/mailman/listinfo/ocfs2-users</a><o:p></o:p></p>
<div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt">><br>
><br>
><br>
><br>
><br>
> _______________________________________________<br>
> Ocfs2-users mailing list<br>
> <a href="mailto:Ocfs2-users@oss.oracle.com">Ocfs2-users@oss.oracle.com</a><br>
> <a href="https://oss.oracle.com/mailman/listinfo/ocfs2-users" target="_blank">https://oss.oracle.com/mailman/listinfo/ocfs2-users</a><br>
><br>
<br>
<o:p></o:p></p>
</div>
</div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><span class="hoenzb"><span style="color:#888888">--</span></span><span style="color:#888888"><br>
<span class="hoenzb">Erik Schwartz <<a href="mailto:schwartz.erik.c@gmail.com">schwartz.erik.c@gmail.com</a>> | GPG key 14F1139B</span></span><o:p></o:p></p>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</div>
</body>
</html>