[Ocfs2-users] heartbeat write timeout

[Sunil Mushran]

This looks very similar to what we were seeing with cfq.

What type of load are you running when this happens?
What is the full kernel version?
Are there are other messages during that time?
What type of storage is this?

Stephan A. Rickauer wrote:
> Sunil Mushran wrote:
>   
>> Set up netdump/netconsole. We print more messages after the
>> write_timeout which will provide more clues. As the node is panicing,
>> these messages are caught only by the netdump server.
>>     
>
> scheduler = deadline, hb_dead_threshold = 50. Still fencing.
>
> ---snip---
> Heartbeat thread (3) printing last 24 blocking operations (cur=10):
> Heartbeat thread stuck at waiting for write completion stuffing current time
> into that blocker (index 10)
> Index 11: took 0 ms to do submit_bio for write
> Index 12: took 0 ms to do checking slots
> Index 13: took 1673 ms to do waiting for write completion
> Index 14: took 250 ms to do msleep
> Index 15: took 0 ms to do allocating bios for read
> Index 16: took 0 ms to do bio alloc read
> Index 17: took 0 ms to do bio add page read
> Index 18: took 0 ms to do submit_bio for read
> Index 19: took 149 ms to do waiting for read completion
> Index 20: took 0 ms to do bio alloc write
> Index 21: took 0 ms to do bio add page write
> Index 22: took 0 ms to do submit_bio for write
> Index 23: took 0 ms to do checking slots
> Index 0: took 4090 ms to do waiting for write completion
> Index 1: took 0 ms to do allocating bios for read
> Index 2: took 0 ms to do bio alloc read
> Index 3: took 0 ms to do bio add page read
> Index 4: took 0 ms to do submit_bio for read
> Index 5: took 134 ms to do waiting for read completion
> Index 6: took 0 ms to do bio alloc write
> Index 7: took 0 ms to do bio add page write
> Index 8: took 0 ms to do submit_bio for write
> Index 9: took 0 ms to do checking slots
> Index 10: took 120045 ms to do waiting for write completion
> Kernel panic - not syncing: ocfs2 is very sorry to be fencing this system by
> panicing
> ---snip---
>
>   
> ------------------------------------------------------------------------
>
> _______________________________________________
> Ocfs2-users mailing list
> Ocfs2-users at oss.oracle.com
> http://oss.oracle.com/mailman/listinfo/ocfs2-users
>