[Ocfs2-users] OCFS2 hanging on writes

[Herbert van den Bergh]

Hello Jeff,

You might want to check what the writer process is waiting on when it's 
frozen.  The wchan column of ps might be enough, but if not, then 
perhaps a kernel stack trace of the process from /proc/<pid>/stack or 
from echo t > /proc/sysrq-trigger .  The latter will show other blocked 
processes as well, which may be helpful in determining the cause of the 
freeze.

Thanks,
Herbert.


On 10/25/2012 06:32 PM, Jeff Paterson wrote:
> Hello,
>
> I would need help with our OCFS2 (1.8.0) filesystem.  We are having 
> problems with it since a couple days.  When we write onto it, it hangs.
>
> The "hanging pattern" is easily reproductible.  If I write a 1GB file 
> on the filesystem, it does the following:
>         - write ~200 MB of data on the disk in 1 second
>         - freeze for about 10 seconds
>         - write ~200 MB of data on the disk in 1 second
>         - freeze for about 10 seconds
>         - write ~200 MB of data on the disk in 1 second
>         - freeze for about 10 seconds
>         (and so on)
>
> When the freezes occur:
>         - other writes operations (from other processes) on the same 
> node also freeze
>         - writes operations on other nodes are not affected by the 
> freezes on another node
> Read operations (on any cluster node, even the one with frozen writes) 
> don't seem to be affected by the freezes.  One sure thing, read 
> operations alone don't cause the filesystem freeze.
>
> For info, before the problem began to appear we could sustain 640 MB/s 
> writes without any freeze.
>
> I tried to mount the filesystem on a single node to avoid issues that 
> could happen with inter-node communications and the problem was still 
> there.
>
>
> *_Filesystem details_*
>
>   * The filesystem has 18 TB and it is currently 72% full.
>   * Mount options are the following:
>     rw,nodev,_netdev,noatime,errors=panic,data=writeback,noacl,nouser_xattr,commit=60,heartbeat=local
>   * All Features: backup-super strict-journal-super sparse
>     extended-slotmap inline-data metaecc indexed-dirs refcount
>     discontig-bg unwritten
>
>
>
> There is nothing special in the systems logs beside application errors 
> caused by the freezes.
>
>
> Would a fsck.ocfs2 help?   How long would it take for 18 TB?
>
> Is there a flag I can enable in debugfs.ocfs2 to get a better idea of 
> what is happening and why it is freezing like that?
>
>
> Any help would be greatly appreciated.
>
> Thanks in advance,
>
> Jeff
>
>
> _______________________________________________
> Ocfs2-users mailing list
> Ocfs2-users at oss.oracle.com
> https://oss.oracle.com/mailman/listinfo/ocfs2-users
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://oss.oracle.com/pipermail/ocfs2-users/attachments/20121025/aa844a3b/attachment.html