<html>
<head>
<style><!--
.hmmessage P
{
margin:0px;
padding:0px
}
body.hmmessage
{
font-size: 10pt;
font-family:Tahoma
}
--></style></head>
<body class='hmmessage'><div dir='ltr'>
<font size="2"><span style="white-space: nowrap; color: rgb(34, 34, 34); font-family: arial, sans-serif;">Hello,</span><br></font><div><div dir="ltr"><div><font color="#222222" face="arial, sans-serif" size="2"><span style="white-space:nowrap"><br></span></font></div><div><font color="#222222" face="arial, sans-serif" size="2"><span style="white-space:nowrap">I would need help with our OCFS2 (1.8.0) filesystem. We are having problems with it since a couple days. When we write onto it, it hangs.</span></font></div><div><font color="#222222" face="arial, sans-serif" size="2"><span style="white-space:nowrap"><br></span></font></div><div><font color="#222222" face="arial, sans-serif" size="2"><span style="white-space:nowrap">The "hanging pattern" is easily reproductible. If I write a 1GB file on the filesystem, it does the following:</span></font></div><div><font color="#222222" face="arial, sans-serif" size="2"><span style="white-space:nowrap"> - write ~200 MB of data on the disk in 1 second</span></font></div><div><font color="#222222" face="arial, sans-serif" size="2"><span style="white-space:nowrap"> - freeze for about 10 seconds</span></font></div><div><font color="#222222" face="arial, sans-serif" size="2"><span style="white-space:nowrap"> - write ~200 MB of data on the disk in 1 second</span></font></div><div><font color="#222222" face="arial, sans-serif" size="2"><span style="white-space:nowrap"> - freeze for about 10 seconds</span></font></div><div><font color="#222222" face="arial, sans-serif" size="2"><span style="white-space:nowrap"> - write ~200 MB of data on the disk in 1 second</span></font></div><div><font color="#222222" face="arial, sans-serif" size="2"><span style="white-space:nowrap"> - freeze for about 10 seconds</span></font></div><div><font color="#222222" face="arial, sans-serif" size="2"><span style="white-space:nowrap"> (and so on)</span></font></div><div><font color="#222222" face="arial, sans-serif" size="2"><span style="white-space:nowrap"><br></span></font></div><div><font color="#222222" face="arial, sans-serif" size="2"><span style="white-space:nowrap">When the freezes occur:</span></font></div><div><font color="#222222" face="arial, sans-serif" size="2"><span style="white-space:nowrap"> - other writes operations (from other processes) on the same node also freeze</span></font></div><div><font color="#222222" face="arial, sans-serif" size="2"><span style="white-space:nowrap"> - writes operations on other nodes are not affected by the freezes on another node</span></font></div><div><font color="#222222" face="arial, sans-serif" size="2"><span style="white-space:nowrap"> </span></font></div><div><font size="2"><font color="#222222" face="arial, sans-serif"><span style="white-space:nowrap">Read operations (on any cluster node, even the one with frozen writes) don't seem to be affected by the freezes. One sure thing, read operations alone d</span></font><span style="white-space:nowrap;color:rgb(34, 34, 34);font-family:arial, sans-serif">on't cause the filesystem freeze.</span></font></div><div><font color="#222222" face="arial, sans-serif" size="2"><span style="white-space:nowrap"><br></span></font></div><div><font color="#222222" face="arial, sans-serif" size="2"><span style="white-space:nowrap"><div>For info, before the problem began to appear we could sustain 640 MB/s writes without any freeze.</div><div><br></div><div>I tried to mount the filesystem on a single node to avoid issues that could happen with inter-node communications and the problem was still there.</div><div><br></div><div><br></div><div><b><u>Filesystem details</u></b></div><div><ul><li>The filesystem has 18 TB and it is currently 72% full.</li><li>Mount options are the following: rw,nodev,_netdev,noatime,errors=panic,data=writeback,noacl,nouser_xattr,commit=60,heartbeat=local</li><li>All Features: backup-super strict-journal-super sparse extended-slotmap inline-data metaecc indexed-dirs refcount discontig-bg unwritten</li></ul></div><div><br></div><div><br></div><div>There is nothing special in the systems logs beside application errors caused by the freezes.</div><div><br></div><div><br></div><div>Would a fsck.ocfs2 help? How long would it take for 18 TB?</div><div><br></div><div>Is there a flag I can enable in debugfs.ocfs2 to get a better idea of what is happening and why it is freezing like that?</div><div><br></div><div><br></div><div>Any help would be greatly appreciated.</div><div><br></div><div>Thanks in advance,</div><div><br></div><div>Jeff</div></span></font></div> </div></div><style><!--
.ExternalClass .ecxhmmessage P
{padding:0px;}
.ExternalClass body.ecxhmmessage
{font-size:10pt;font-family:Tahoma;}
--></style> </div></body>
</html>