[Ocfs2-users] OCFS2 Crash

[Srinivas Eeda]

On 08/21/2015 03:10 AM, Martin Lund wrote:
> Hello,
>
> We have a 3 node OCFS2 cluster, using:
>
> Kernel: 3.16.0-0.bpo.4-amd64
> ii  ocfs2-tools                           1.6.4-1+deb7u1                amd64        tools for managing OCFS2 cluster filesystems
>
> Today two of the nodes out of the 3 had some partial OCFS2 related kernel panic (see at the end). After rebooting the first node the FS become available again. Anyone run into error like this?
that most likely could be nodes had a broken network connection. You can 
review the rebooted nodes messages file and see it was reporting any 
connection errors. If that's the case and if you have following mainline 
fixes, you shouldn't run into that kind of problem

  5046f18d5bd9ad7638b32c3b304ff39a74c064df
8e9801dfe37c9e68cdbfcd15988df2187191864e
c43c363def04cdaed0d9e26dae846081f55714e7

>
> [Fri Aug 21 11:17:27 2015] INFO: task df:16329 blocked for more than 120 seconds.
> [Fri Aug 21 11:17:27 2015]       Not tainted 3.16.0-0.bpo.4-amd64 #1
> [Fri Aug 21 11:17:27 2015] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
> [Fri Aug 21 11:17:27 2015] df              D ffff88003fc12ec0     0 16329  16324 0x00000000
> [Fri Aug 21 11:17:27 2015]  ffffffff81818480 0000000000000086 0000000000000000 ffff88003a96a210
> [Fri Aug 21 11:17:27 2015]  0000000000012ec0 ffff88000b43ffd8 0000000000012ec0 ffff88003a96a210
> [Fri Aug 21 11:17:27 2015]  0000000000000002 ffff88000b43fd08 7fffffffffffffff ffff88000b43fd00
> [Fri Aug 21 11:17:27 2015] Call Trace:
> [Fri Aug 21 11:17:27 2015]  [<ffffffff815478cd>] ? schedule_timeout+0x1dd/0x240
> [Fri Aug 21 11:17:27 2015]  [<ffffffff81156f65>] ? __alloc_pages_nodemask+0x165/0xbb0
> [Fri Aug 21 11:17:27 2015]  [<ffffffff8119c86c>] ? alloc_pages_vma+0xac/0x180
> [Fri Aug 21 11:17:27 2015]  [<ffffffff815496ac>] ? wait_for_completion+0xac/0x120
> [Fri Aug 21 11:17:27 2015]  [<ffffffff810a0650>] ? try_to_wake_up+0x310/0x310
> [Fri Aug 21 11:17:27 2015]  [<ffffffffa04871ea>] ? __ocfs2_cluster_lock.isra.36+0x1ba/0x7c0 [ocfs2]
> [Fri Aug 21 11:17:27 2015]  [<ffffffff811c85de>] ? __inode_permission+0x2e/0xd0
> [Fri Aug 21 11:17:27 2015]  [<ffffffff811c8b4b>] ? link_path_walk+0x5b/0x880
> [Fri Aug 21 11:17:27 2015]  [<ffffffffa0488929>] ? ocfs2_inode_lock_full_nested+0x149/0xbb0 [ocfs2]
> [Fri Aug 21 11:17:27 2015]  [<ffffffff811c9b34>] ? filename_lookup+0x34/0xd0
> [Fri Aug 21 11:17:27 2015]  [<ffffffff811c83d4>] ? getname_flags+0xc4/0x1b0
> [Fri Aug 21 11:17:27 2015]  [<ffffffffa04ce2f9>] ? ocfs2_statfs+0x79/0x350 [ocfs2]
> [Fri Aug 21 11:17:27 2015]  [<ffffffff811edbb0>] ? statfs_by_dentry+0xa0/0x140
> [Fri Aug 21 11:17:27 2015]  [<ffffffff811edc6a>] ? vfs_statfs+0x1a/0xa0
> [Fri Aug 21 11:17:27 2015]  [<ffffffff811edd2e>] ? user_statfs+0x3e/0x70
> [Fri Aug 21 11:17:27 2015]  [<ffffffff811eddb2>] ? SYSC_statfs+0x12/0x30
> [Fri Aug 21 11:17:27 2015]  [<ffffffff8154bdcd>] ? system_call_fast_compare_end+0x10/0x15
> [Fri Aug 21 11:17:27 2015] INFO: task df:16336 blocked for more than 120 seconds.
> [Fri Aug 21 11:17:27 2015]       Not tainted 3.16.0-0.bpo.4-amd64 #1
> [Fri Aug 21 11:17:27 2015] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
> [Fri Aug 21 11:17:27 2015] df              D ffff88003fc12ec0     0 16336  16331 0x00000000
> [Fri Aug 21 11:17:27 2015]  ffffffff81818480 0000000000000082 0000000000000000 ffff8800370dcce0
> [Fri Aug 21 11:17:27 2015]  0000000000012ec0 ffff8800239e7fd8 0000000000012ec0 ffff8800370dcce0
> [Fri Aug 21 11:17:27 2015]  0000000000000001 ffff8800239e7d08 7fffffffffffffff ffff8800239e7d00
> [Fri Aug 21 11:17:27 2015] Call Trace:
> [Fri Aug 21 11:17:27 2015]  [<ffffffff815478cd>] ? schedule_timeout+0x1dd/0x240
> [Fri Aug 21 11:17:27 2015]  [<ffffffff81156f65>] ? __alloc_pages_nodemask+0x165/0xbb0
> [Fri Aug 21 11:17:27 2015]  [<ffffffff8119c86c>] ? alloc_pages_vma+0xac/0x180
> [Fri Aug 21 11:17:27 2015]  [<ffffffff815496ac>] ? wait_for_completion+0xac/0x120
> [Fri Aug 21 11:17:27 2015]  [<ffffffff810a0650>] ? try_to_wake_up+0x310/0x310
> [Fri Aug 21 11:17:27 2015]  [<ffffffffa04871ea>] ? __ocfs2_cluster_lock.isra.36+0x1ba/0x7c0 [ocfs2]
> [Fri Aug 21 11:17:27 2015]  [<ffffffff811c85de>] ? __inode_permission+0x2e/0xd0
> [Fri Aug 21 11:17:27 2015]  [<ffffffff811c8b4b>] ? link_path_walk+0x5b/0x880
> [Fri Aug 21 11:17:27 2015]  [<ffffffffa0488929>] ? ocfs2_inode_lock_full_nested+0x149/0xbb0 [ocfs2]
> [Fri Aug 21 11:17:27 2015]  [<ffffffff811c9b34>] ? filename_lookup+0x34/0xd0
> [Fri Aug 21 11:17:27 2015]  [<ffffffff811c83d4>] ? getname_flags+0xc4/0x1b0
> [Fri Aug 21 11:17:27 2015]  [<ffffffffa04ce2f9>] ? ocfs2_statfs+0x79/0x350 [ocfs2]
> [Fri Aug 21 11:17:27 2015]  [<ffffffff811edbb0>] ? statfs_by_dentry+0xa0/0x140
> [Fri Aug 21 11:17:27 2015]  [<ffffffff811edc6a>] ? vfs_statfs+0x1a/0xa0
> [Fri Aug 21 11:17:27 2015]  [<ffffffff811edd2e>] ? user_statfs+0x3e/0x70
> [Fri Aug 21 11:17:27 2015]  [<ffffffff811eddb2>] ? SYSC_statfs+0x12/0x30
> [Fri Aug 21 11:17:27 2015]  [<ffffffff8154bdcd>] ? system_call_fast_compare_end+0x10/0x15
>
>
> Thank you
>
> _______________________________________________
> Ocfs2-users mailing list
> Ocfs2-users at oss.oracle.com
> https://oss.oracle.com/mailman/listinfo/ocfs2-users