[Ocfs2-users] ocfs2 hangs the system
Xavier Diumé
xdiume at gmail.com
Fri Apr 8 03:56:11 PDT 2011
Hello,
Two servers are using iscsi to mount lun's from a SAN. The filesystem used
is ocfs2 with Debian 6, with a multipath configuration.
The two servers are working well together but something hangs one of the
server. Look at dmesg outout:
[431163.609779] o2net: connection to node kvs2 (num 2) at
192.168.0.25:7777has been idle for 30.0 seconds, shutting it down.
[431163.609788] (0,0):o2net_idle_timer:1495 here are some times that might
help debug the situation: (tmr 1302076000.284162 now 1302076030.282028 dr
1302076000.284145 adv 1302076000.284167:1302076000.284168 func
(7e2ba784:504) 1302075956.329407:1302075956.329419)
[431163.609824] o2net: no longer connected to node kvs2 (num 2) at
192.168.0.25:7777
[431166.605867] o2net: accepted connection from node kvs2 (num 2) at
192.168.0.25:7777
[431708.706720] o2net: connection to node kvs2 (num 2) at
192.168.0.25:7777has been idle for 30.0 seconds, shutting it down.
[431708.706727] (0,0):o2net_idle_timer:1495 here are some times that might
help debug the situation: (tmr 1302076546.346741 now 1302076576.346061 dr
1302076546.346724 adv 1302076546.346745:1302076546.346746 func
(4a2f0abe:504) 1302076546.346523:1302076546.346530)
[431708.706751] o2net: no longer connected to node kvs2 (num 2) at
192.168.0.25:7777
[431711.700987] o2net: accepted connection from node kvs2 (num 2) at
192.168.0.25:7777
[545691.126992] o2net: connection to node kvs2 (num 2) at
192.168.0.25:7777has been idle for 30.0 seconds, shutting it down.
[545691.127004] (0,0):o2net_idle_timer:1495 here are some times that might
help debug the situation: (tmr 1302190729.946098 now 1302190759.945976 dr
1302190729.946089 adv 1302190729.946098:1302190729.946099 func
(1ad60b89:504) 1302190721.951800:1302190721.951808)
[545691.127042] o2net: no longer connected to node kvs2 (num 2) at
192.168.0.25:7777
[545691.127065] (4807,14):dlm_send_proxy_ast_msg:456 ERROR: status = -112
[545691.127072] (4621,2):dlm_send_proxy_ast_msg:456 ERROR: status = -112
[545691.127075] (4621,2):dlm_flush_asts:599 ERROR: status = -112
[545691.127167] (4807,14):dlm_flush_asts:599 ERROR: status = -112
[545694.118819] o2net: accepted connection from node kvs2 (num 2) at
192.168.0.25:7777
[545876.330611] INFO: task events/0:51 blocked for more than 120 seconds.
[545876.330649] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables
this message.
[545876.330693] events/0 D ffff88033ac0fa20 0 51 2
0x00000000
[545876.330697] ffffffff814611f0 0000000000000046 0000000000000000
ffffffff811f6e95
[545876.330700] 0000000000000000 0000000000000000 000000000000f9e0
ffff88062e581fd8
[545876.330703] 0000000000015780 0000000000015780 ffff88062e578000
ffff88062e5782f8
[545876.330706] Call Trace:
[545876.330714] [<ffffffff811f6e95>] ? extract_buf+0x91/0x12d
[545876.330718] [<ffffffff812fabfc>] ? schedule_timeout+0x2e/0xdd
[545876.330721] [<ffffffff811971bb>] ? __bitmap_weight+0x3a/0x7e
[545876.330729] [<ffffffff8103a6e1>] ? scale_rt_power+0x1f/0x65
[545876.330732] [<ffffffff812faab5>] ? wait_for_common+0xde/0x15b
[545876.330737] [<ffffffff8104a274>] ? default_wake_function+0x0/0x9
[545876.330755] [<ffffffffa0331c7a>] ? __ocfs2_cluster_lock+0x8a4/0x8c5
[ocfs2]
[545876.330758] [<ffffffff812fa83f>] ? thread_return+0x79/0xe0
[545876.330766] [<ffffffffa0331cb0>] ? ocfs2_cluster_lock+0x15/0x17 [ocfs2]
[545876.330773] [<ffffffffa033344a>] ? ocfs2_orphan_scan_lock+0x5d/0xa8
[ocfs2]
[545876.330780] [<ffffffffa033344a>] ? ocfs2_orphan_scan_lock+0x5d/0xa8
[ocfs2]
[545876.330787] [<ffffffffa0344a96>] ? ocfs2_queue_orphan_scan+0x29/0x126
[ocfs2]
[545876.330790] [<ffffffff812fb175>] ? mutex_lock+0xd/0x31
[545876.330797] [<ffffffffa0344bb8>] ? ocfs2_orphan_scan_work+0x25/0x4d
[ocfs2]
[545876.330800] [<ffffffff810616ff>] ? worker_thread+0x188/0x21d
[545876.330807] [<ffffffffa0344b93>] ? ocfs2_orphan_scan_work+0x0/0x4d
[ocfs2]
[545876.330810] [<ffffffff81064d2a>] ? autoremove_wake_function+0x0/0x2e
[545876.330812] [<ffffffff81061577>] ? worker_thread+0x0/0x21d
[545876.330814] [<ffffffff81064a5d>] ? kthread+0x79/0x81
[545876.330818] [<ffffffff81011baa>] ? child_rip+0xa/0x20
[545876.330820] [<ffffffff810649e4>] ? kthread+0x0/0x81
[545876.330822] [<ffffffff81011ba0>] ? child_rip+0x0/0x20
[545876.330824] INFO: task events/1:52 blocked for more than 120 seconds.
[545876.330852] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables
this message.
[545876.330896] events/1 D 0000000000000000 0 52 2
0x00000000
[545876.330899] ffff88062e463880 0000000000000046 0000000000000000
0000000000000000
[545876.330901] 0000000000000000 0000000000000000 000000000000f9e0
ffff88062e5a3fd8
[545876.330904] 0000000000015780 0000000000015780 ffff88062e578710
ffff88062e578a08
[545876.330906] Call Trace:
[545876.330909] [<ffffffff812fabfc>] ? schedule_timeout+0x2e/0xdd
[545876.330911] [<ffffffff811971bb>] ? __bitmap_weight+0x3a/0x7e
[545876.330914] [<ffffffff8103a6e1>] ? scale_rt_power+0x1f/0x65
[545876.330916] [<ffffffff812faab5>] ? wait_for_common+0xde/0x15b
[545876.330918] [<ffffffff8104a274>] ? default_wake_function+0x0/0x9
[545876.330925] [<ffffffffa0331c7a>] ? __ocfs2_cluster_lock+0x8a4/0x8c5
[ocfs2]
[545876.330928] [<ffffffff812fa83f>] ? thread_return+0x79/0xe0
[545876.330935] [<ffffffffa0331cb0>] ? ocfs2_cluster_lock+0x15/0x17 [ocfs2]
[545876.330942] [<ffffffffa033344a>] ? ocfs2_orphan_scan_lock+0x5d/0xa8
[ocfs2]
[545876.330949] [<ffffffffa033344a>] ? ocfs2_orphan_scan_lock+0x5d/0xa8
[ocfs2]
[545876.330956] [<ffffffffa0344a96>] ? ocfs2_queue_orphan_scan+0x29/0x126
[ocfs2]
[545876.330959] [<ffffffff812fb175>] ? mutex_lock+0xd/0x31
[545876.330966] [<ffffffffa0344bb8>] ? ocfs2_orphan_scan_work+0x25/0x4d
[ocfs2]
[545876.330968] [<ffffffff810616ff>] ? worker_thread+0x188/0x21d
[545876.330975] [<ffffffffa0344b93>] ? ocfs2_orphan_scan_work+0x0/0x4d
[ocfs2]
[545876.330977] [<ffffffff81064d2a>] ? autoremove_wake_function+0x0/0x2e
[545876.330980] [<ffffffff81061577>] ? worker_thread+0x0/0x21d
[545876.330982] [<ffffffff81064a5d>] ? kthread+0x79/0x81
[545876.330984] [<ffffffff81011baa>] ? child_rip+0xa/0x20
[545876.330987] [<ffffffff810649e4>] ? kthread+0x0/0x81
[545876.330990] [<ffffffff81011ba0>] ? child_rip+0x0/0x20
[545996.118082] INFO: task events/0:51 blocked for more than 120 seconds.
[545996.118113] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables
this message.
[545996.118158] events/0 D ffff88033ac0fa20 0 51 2
0x00000000
[545996.118162] ffffffff814611f0 0000000000000046 0000000000000000
ffffffff811f6e95
[545996.118165] 0000000000000000 0000000000000000 000000000000f9e0
ffff88062e581fd8
[545996.118168] 0000000000015780 0000000000015780 ffff88062e578000
ffff88062e5782f8
[545996.118171] Call Trace:
[545996.118179] [<ffffffff811f6e95>] ? extract_buf+0x91/0x12d
[545996.118183] [<ffffffff812fabfc>] ? schedule_timeout+0x2e/0xdd
[545996.118187] [<ffffffff811971bb>] ? __bitmap_weight+0x3a/0x7e
[545996.118191] [<ffffffff8103a6e1>] ? scale_rt_power+0x1f/0x65
[545996.118193] [<ffffffff812faab5>] ? wait_for_common+0xde/0x15b
[545996.118197] [<ffffffff8104a274>] ? default_wake_function+0x0/0x9
[545996.118213] [<ffffffffa0331c7a>] ? __ocfs2_cluster_lock+0x8a4/0x8c5
[ocfs2]
[545996.118216] [<ffffffff812fa83f>] ? thread_return+0x79/0xe0
[545996.118224] [<ffffffffa0331cb0>] ? ocfs2_cluster_lock+0x15/0x17 [ocfs2]
[545996.118232] [<ffffffffa033344a>] ? ocfs2_orphan_scan_lock+0x5d/0xa8
[ocfs2]
[545996.118239] [<ffffffffa033344a>] ? ocfs2_orphan_scan_lock+0x5d/0xa8
[ocfs2]
[545996.118247] [<ffffffffa0344a96>] ? ocfs2_queue_orphan_scan+0x29/0x126
[ocfs2]
[545996.118250] [<ffffffff812fb175>] ? mutex_lock+0xd/0x31
[545996.118257] [<ffffffffa0344bb8>] ? ocfs2_orphan_scan_work+0x25/0x4d
[ocfs2]
[545996.118261] [<ffffffff810616ff>] ? worker_thread+0x188/0x21d
[545996.118269] [<ffffffffa0344b93>] ? ocfs2_orphan_scan_work+0x0/0x4d
[ocfs2]
[545996.118272] [<ffffffff81064d2a>] ? autoremove_wake_function+0x0/0x2e
[545996.118274] [<ffffffff81061577>] ? worker_thread+0x0/0x21d
[545996.118276] [<ffffffff81064a5d>] ? kthread+0x79/0x81
[545996.118280] [<ffffffff81011baa>] ? child_rip+0xa/0x20
[545996.118282] [<ffffffff810649e4>] ? kthread+0x0/0x81
[545996.118284] [<ffffffff81011ba0>] ? child_rip+0x0/0x20
[545996.118286] INFO: task events/1:52 blocked for more than 120 seconds.
[545996.118314] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables
this message.
[545996.118359] events/1 D 0000000000000000 0 52 2
0x00000000
[545996.118362] ffff88062e463880 0000000000000046 0000000000000000
0000000000000000
[545996.118365] 0000000000000000 0000000000000000 000000000000f9e0
ffff88062e5a3fd8
[545996.118368] 0000000000015780 0000000000015780 ffff88062e578710
ffff88062e578a08
[545996.118370] Call Trace:
[545996.118374] [<ffffffff812fabfc>] ? schedule_timeout+0x2e/0xdd
[545996.118376] [<ffffffff811971bb>] ? __bitmap_weight+0x3a/0x7e
[545996.118379] [<ffffffff8103a6e1>] ? scale_rt_power+0x1f/0x65
[545996.118381] [<ffffffff812faab5>] ? wait_for_common+0xde/0x15b
[545996.118384] [<ffffffff8104a274>] ? default_wake_function+0x0/0x9
[545996.118392] [<ffffffffa0331c7a>] ? __ocfs2_cluster_lock+0x8a4/0x8c5
[ocfs2]
[545996.118395] [<ffffffff812fa83f>] ? thread_return+0x79/0xe0
[545996.118402] [<ffffffffa0331cb0>] ? ocfs2_cluster_lock+0x15/0x17 [ocfs2]
[545996.118409] [<ffffffffa033344a>] ? ocfs2_orphan_scan_lock+0x5d/0xa8
[ocfs2]
[545996.118416] [<ffffffffa033344a>] ? ocfs2_orphan_scan_lock+0x5d/0xa8
[ocfs2]
[545996.118425] [<ffffffffa0344a96>] ? ocfs2_queue_orphan_scan+0x29/0x126
[ocfs2]
[545996.118428] [<ffffffff812fb175>] ? mutex_lock+0xd/0x31
[545996.118438] [<ffffffffa0344bb8>] ? ocfs2_orphan_scan_work+0x25/0x4d
[ocfs2]
[545996.118441] [<ffffffff810616ff>] ? worker_thread+0x188/0x21d
[545996.118451] [<ffffffffa0344b93>] ? ocfs2_orphan_scan_work+0x0/0x4d
[ocfs2]
[545996.118454] [<ffffffff81064d2a>] ? autoremove_wake_function+0x0/0x2e
[545996.118458] [<ffffffff81061577>] ? worker_thread+0x0/0x21d
[545996.118461] [<ffffffff81064a5d>] ? kthread+0x79/0x81
[545996.118464] [<ffffffff81011baa>] ? child_rip+0xa/0x20
[545996.118467] [<ffffffff810649e4>] ? kthread+0x0/0x81
[545996.118469] [<ffffffff81011ba0>] ? child_rip+0x0/0x20
[546115.905565] INFO: task events/0:51 blocked for more than 120 seconds.
[546115.905596] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables
this message.
[546115.905643] events/0 D ffff88033ac0fa20 0 51 2
0x00000000
[546115.905648] ffffffff814611f0 0000000000000046 0000000000000000
ffffffff811f6e95
[546115.905651] 0000000000000000 0000000000000000 000000000000f9e0
ffff88062e581fd8
[546115.905655] 0000000000015780 0000000000015780 ffff88062e578000
ffff88062e5782f8
[546115.905659] Call Trace:
[546115.905667] [<ffffffff811f6e95>] ? extract_buf+0x91/0x12d
[546115.905672] [<ffffffff812fabfc>] ? schedule_timeout+0x2e/0xdd
[546115.905675] [<ffffffff811971bb>] ? __bitmap_weight+0x3a/0x7e
[546115.905679] [<ffffffff8103a6e1>] ? scale_rt_power+0x1f/0x65
[546115.905681] [<ffffffff812faab5>] ? wait_for_common+0xde/0x15b
[546115.905685] [<ffffffff8104a274>] ? default_wake_function+0x0/0x9
[546115.905701] [<ffffffffa0331c7a>] ? __ocfs2_cluster_lock+0x8a4/0x8c5
[ocfs2]
[546115.905704] [<ffffffff812fa83f>] ? thread_return+0x79/0xe0
[546115.905711] [<ffffffffa0331cb0>] ? ocfs2_cluster_lock+0x15/0x17 [ocfs2]
[546115.905719] [<ffffffffa033344a>] ? ocfs2_orphan_scan_lock+0x5d/0xa8
[ocfs2]
[546115.905726] [<ffffffffa033344a>] ? ocfs2_orphan_scan_lock+0x5d/0xa8
[ocfs2]
[546115.905733] [<ffffffffa0344a96>] ? ocfs2_queue_orphan_scan+0x29/0x126
[ocfs2]
[546115.905736] [<ffffffff812fb175>] ? mutex_lock+0xd/0x31
[546115.905743] [<ffffffffa0344bb8>] ? ocfs2_orphan_scan_work+0x25/0x4d
[ocfs2]
[546115.905747] [<ffffffff810616ff>] ? worker_thread+0x188/0x21d
[546115.905754] [<ffffffffa0344b93>] ? ocfs2_orphan_scan_work+0x0/0x4d
[ocfs2]
[546115.905757] [<ffffffff81064d2a>] ? autoremove_wake_function+0x0/0x2e
[546115.905759] [<ffffffff81061577>] ? worker_thread+0x0/0x21d
[546115.905761] [<ffffffff81064a5d>] ? kthread+0x79/0x81
[546115.905765] [<ffffffff81011baa>] ? child_rip+0xa/0x20
[546115.905767] [<ffffffff810649e4>] ? kthread+0x0/0x81
[546115.905769] [<ffffffff81011ba0>] ? child_rip+0x0/0x20
[546115.905771] INFO: task events/1:52 blocked for more than 120 seconds.
[546115.905799] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables
this message.
[546115.905843] events/1 D 0000000000000000 0 52 2
0x00000000
[546115.905846] ffff88062e463880 0000000000000046 0000000000000000
0000000000000000
[546115.905848] 0000000000000000 0000000000000000 000000000000f9e0
ffff88062e5a3fd8
[546115.905851] 0000000000015780 0000000000015780 ffff88062e578710
ffff88062e578a08
[546115.905853] Call Trace:
[546115.905856] [<ffffffff812fabfc>] ? schedule_timeout+0x2e/0xdd
[546115.905858] [<ffffffff811971bb>] ? __bitmap_weight+0x3a/0x7e
[546115.905860] [<ffffffff8103a6e1>] ? scale_rt_power+0x1f/0x65
[546115.905862] [<ffffffff812faab5>] ? wait_for_common+0xde/0x15b
[546115.905865] [<ffffffff8104a274>] ? default_wake_function+0x0/0x9
[546115.905872] [<ffffffffa0331c7a>] ? __ocfs2_cluster_lock+0x8a4/0x8c5
[ocfs2]
[546115.905875] [<ffffffff812fa83f>] ? thread_return+0x79/0xe0
[546115.905882] [<ffffffffa0331cb0>] ? ocfs2_cluster_lock+0x15/0x17 [ocfs2]
[546115.905889] [<ffffffffa033344a>] ? ocfs2_orphan_scan_lock+0x5d/0xa8
[ocfs2]
[546115.905896] [<ffffffffa033344a>] ? ocfs2_orphan_scan_lock+0x5d/0xa8
[ocfs2]
[546115.905903] [<ffffffffa0344a96>] ? ocfs2_queue_orphan_scan+0x29/0x126
[ocfs2]
[546115.905905] [<ffffffff812fb175>] ? mutex_lock+0xd/0x31
[546115.905912] [<ffffffffa0344bb8>] ? ocfs2_orphan_scan_work+0x25/0x4d
[ocfs2]
[546115.905915] [<ffffffff810616ff>] ? worker_thread+0x188/0x21d
[546115.905922] [<ffffffffa0344b93>] ? ocfs2_orphan_scan_work+0x0/0x4d
[ocfs2]
[546115.905924] [<ffffffff81064d2a>] ? autoremove_wake_function+0x0/0x2e
[546115.905927] [<ffffffff81061577>] ? worker_thread+0x0/0x21d
[546115.905929] [<ffffffff81064a5d>] ? kthread+0x79/0x81
[546115.905931] [<ffffffff81011baa>] ? child_rip+0xa/0x20
[546115.905933] [<ffffffff810649e4>] ? kthread+0x0/0x81
[546115.905935] [<ffffffff81011ba0>] ? child_rip+0x0/0x20
[546235.692857] INFO: task events/0:51 blocked for more than 120 seconds.
[546235.692890] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables
this message.
[546235.692939] events/0 D ffff88033ac0fa20 0 51 2
0x00000000
[546235.692943] ffffffff814611f0 0000000000000046 0000000000000000
ffffffff811f6e95
[546235.692946] 0000000000000000 0000000000000000 000000000000f9e0
ffff88062e581fd8
[546235.692949] 0000000000015780 0000000000015780 ffff88062e578000
ffff88062e5782f8
[546235.692951] Call Trace:
[546235.692959] [<ffffffff811f6e95>] ? extract_buf+0x91/0x12d
[546235.692964] [<ffffffff812fabfc>] ? schedule_timeout+0x2e/0xdd
[546235.692967] [<ffffffff811971bb>] ? __bitmap_weight+0x3a/0x7e
[546235.692971] [<ffffffff8103a6e1>] ? scale_rt_power+0x1f/0x65
[546235.692973] [<ffffffff812faab5>] ? wait_for_common+0xde/0x15b
[546235.692977] [<ffffffff8104a274>] ? default_wake_function+0x0/0x9
[546235.692994] [<ffffffffa0331c7a>] ? __ocfs2_cluster_lock+0x8a4/0x8c5
[ocfs2]
[546235.692996] [<ffffffff812fa83f>] ? thread_return+0x79/0xe0
[546235.693004] [<ffffffffa0331cb0>] ? ocfs2_cluster_lock+0x15/0x17 [ocfs2]
[546235.693011] [<ffffffffa033344a>] ? ocfs2_orphan_scan_lock+0x5d/0xa8
[ocfs2]
[546235.693018] [<ffffffffa033344a>] ? ocfs2_orphan_scan_lock+0x5d/0xa8
[ocfs2]
[546235.693031] [<ffffffffa0344a96>] ? ocfs2_queue_orphan_scan+0x29/0x126
[ocfs2]
[546235.693034] [<ffffffff812fb175>] ? mutex_lock+0xd/0x31
[546235.693041] [<ffffffffa0344bb8>] ? ocfs2_orphan_scan_work+0x25/0x4d
[ocfs2]
[546235.693045] [<ffffffff810616ff>] ? worker_thread+0x188/0x21d
[546235.693052] [<ffffffffa0344b93>] ? ocfs2_orphan_scan_work+0x0/0x4d
[ocfs2]
[546235.693055] [<ffffffff81064d2a>] ? autoremove_wake_function+0x0/0x2e
[546235.693057] [<ffffffff81061577>] ? worker_thread+0x0/0x21d
[546235.693059] [<ffffffff81064a5d>] ? kthread+0x79/0x81
[546235.693063] [<ffffffff81011baa>] ? child_rip+0xa/0x20
[546235.693066] [<ffffffff810649e4>] ? kthread+0x0/0x81
[546235.693070] [<ffffffff81011ba0>] ? child_rip+0x0/0x20
[546235.693073] INFO: task events/1:52 blocked for more than 120 seconds.
[546235.693102] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables
this message.
[546235.693147] events/1 D 0000000000000000 0 52 2
0x00000000
[546235.693150] ffff88062e463880 0000000000000046 0000000000000000
0000000000000000
[546235.693153] 0000000000000000 0000000000000000 000000000000f9e0
ffff88062e5a3fd8
[546235.693155] 0000000000015780 0000000000015780 ffff88062e578710
ffff88062e578a08
[546235.693158] Call Trace:
[546235.693160] [<ffffffff812fabfc>] ? schedule_timeout+0x2e/0xdd
[546235.693163] [<ffffffff811971bb>] ? __bitmap_weight+0x3a/0x7e
[546235.693166] [<ffffffff8103a6e1>] ? scale_rt_power+0x1f/0x65
[546235.693168] [<ffffffff812faab5>] ? wait_for_common+0xde/0x15b
[546235.693171] [<ffffffff8104a274>] ? default_wake_function+0x0/0x9
[546235.693180] [<ffffffffa0331c7a>] ? __ocfs2_cluster_lock+0x8a4/0x8c5
[ocfs2]
[546235.693183] [<ffffffff812fa83f>] ? thread_return+0x79/0xe0
[546235.693191] [<ffffffffa0331cb0>] ? ocfs2_cluster_lock+0x15/0x17 [ocfs2]
[546235.693199] [<ffffffffa033344a>] ? ocfs2_orphan_scan_lock+0x5d/0xa8
[ocfs2]
[546235.693207] [<ffffffffa033344a>] ? ocfs2_orphan_scan_lock+0x5d/0xa8
[ocfs2]
[546235.693216] [<ffffffffa0344a96>] ? ocfs2_queue_orphan_scan+0x29/0x126
[ocfs2]
[546235.693219] [<ffffffff812fb175>] ? mutex_lock+0xd/0x31
[546235.693228] [<ffffffffa0344bb8>] ? ocfs2_orphan_scan_work+0x25/0x4d
[ocfs2]
[546235.693231] [<ffffffff810616ff>] ? worker_thread+0x188/0x21d
[546235.693241] [<ffffffffa0344b93>] ? ocfs2_orphan_scan_work+0x0/0x4d
[ocfs2]
[546235.693244] [<ffffffff81064d2a>] ? autoremove_wake_function+0x0/0x2e
[546235.693247] [<ffffffff81061577>] ? worker_thread+0x0/0x21d
[546235.693250] [<ffffffff81064a5d>] ? kthread+0x79/0x81
[546235.693253] [<ffffffff81011baa>] ? child_rip+0xa/0x20
[546235.693255] [<ffffffff810649e4>] ? kthread+0x0/0x81
[546235.693257] [<ffffffff81011ba0>] ? child_rip+0x0/0x20
[546355.480513] INFO: task events/0:51 blocked for more than 120 seconds.
[546355.480543] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables
this message.
[546355.480588] events/0 D ffff88033ac0fa20 0 51 2
0x00000000
[546355.480592] ffffffff814611f0 0000000000000046 0000000000000000
ffffffff811f6e95
[546355.480596] 0000000000000000 0000000000000000 000000000000f9e0
ffff88062e581fd8
[546355.480600] 0000000000015780 0000000000015780 ffff88062e578000
ffff88062e5782f8
[546355.480602] Call Trace:
[546355.480610] [<ffffffff811f6e95>] ? extract_buf+0x91/0x12d
[546355.480615] [<ffffffff812fabfc>] ? schedule_timeout+0x2e/0xdd
[546355.480618] [<ffffffff811971bb>] ? __bitmap_weight+0x3a/0x7e
[546355.480622] [<ffffffff8103a6e1>] ? scale_rt_power+0x1f/0x65
[546355.480624] [<ffffffff812faab5>] ? wait_for_common+0xde/0x15b
[546355.480628] [<ffffffff8104a274>] ? default_wake_function+0x0/0x9
[546355.480644] [<ffffffffa0331c7a>] ? __ocfs2_cluster_lock+0x8a4/0x8c5
[ocfs2]
[546355.480647] [<ffffffff812fa83f>] ? thread_return+0x79/0xe0
[546355.480654] [<ffffffffa0331cb0>] ? ocfs2_cluster_lock+0x15/0x17 [ocfs2]
[546355.480662] [<ffffffffa033344a>] ? ocfs2_orphan_scan_lock+0x5d/0xa8
[ocfs2]
[546355.480669] [<ffffffffa033344a>] ? ocfs2_orphan_scan_lock+0x5d/0xa8
[ocfs2]
[546355.480676] [<ffffffffa0344a96>] ? ocfs2_queue_orphan_scan+0x29/0x126
[ocfs2]
[546355.480679] [<ffffffff812fb175>] ? mutex_lock+0xd/0x31
[546355.480686] [<ffffffffa0344bb8>] ? ocfs2_orphan_scan_work+0x25/0x4d
[ocfs2]
[546355.480690] [<ffffffff810616ff>] ? worker_thread+0x188/0x21d
[546355.480697] [<ffffffffa0344b93>] ? ocfs2_orphan_scan_work+0x0/0x4d
[ocfs2]
[546355.480700] [<ffffffff81064d2a>] ? autoremove_wake_function+0x0/0x2e
[546355.480702] [<ffffffff81061577>] ? worker_thread+0x0/0x21d
[546355.480704] [<ffffffff81064a5d>] ? kthread+0x79/0x81
[546355.480708] [<ffffffff81011baa>] ? child_rip+0xa/0x20
[546355.480710] [<ffffffff810649e4>] ? kthread+0x0/0x81
[546355.480712] [<ffffffff81011ba0>] ? child_rip+0x0/0x20
[546355.480714] INFO: task events/1:52 blocked for more than 120 seconds.
[546355.480742] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables
this message.
[546355.480786] events/1 D 0000000000000000 0 52 2
0x00000000
[546355.480789] ffff88062e463880 0000000000000046 0000000000000000
0000000000000000
[546355.480792] 0000000000000000 0000000000000000 000000000000f9e0
ffff88062e5a3fd8
[546355.480794] 0000000000015780 0000000000015780 ffff88062e578710
ffff88062e578a08
[546355.480797] Call Trace:
[546355.480799] [<ffffffff812fabfc>] ? schedule_timeout+0x2e/0xdd
[546355.480801] [<ffffffff811971bb>] ? __bitmap_weight+0x3a/0x7e
[546355.480804] [<ffffffff8103a6e1>] ? scale_rt_power+0x1f/0x65
[546355.480806] [<ffffffff812faab5>] ? wait_for_common+0xde/0x15b
[546355.480808] [<ffffffff8104a274>] ? default_wake_function+0x0/0x9
[546355.480816] [<ffffffffa0331c7a>] ? __ocfs2_cluster_lock+0x8a4/0x8c5
[ocfs2]
[546355.480818] [<ffffffff812fa83f>] ? thread_return+0x79/0xe0
[546355.480825] [<ffffffffa0331cb0>] ? ocfs2_cluster_lock+0x15/0x17 [ocfs2]
[546355.480832] [<ffffffffa033344a>] ? ocfs2_orphan_scan_lock+0x5d/0xa8
[ocfs2]
[546355.480840] [<ffffffffa033344a>] ? ocfs2_orphan_scan_lock+0x5d/0xa8
[ocfs2]
[546355.480850] [<ffffffffa0344a96>] ? ocfs2_queue_orphan_scan+0x29/0x126
[ocfs2]
[546355.480853] [<ffffffff812fb175>] ? mutex_lock+0xd/0x31
[546355.480863] [<ffffffffa0344bb8>] ? ocfs2_orphan_scan_work+0x25/0x4d
[ocfs2]
[546355.480870] [<ffffffff810616ff>] ? worker_thread+0x188/0x21d
[546355.480880] [<ffffffffa0344b93>] ? ocfs2_orphan_scan_work+0x0/0x4d
[ocfs2]
[546355.480883] [<ffffffff81064d2a>] ? autoremove_wake_function+0x0/0x2e
[546355.480887] [<ffffffff81061577>] ? worker_thread+0x0/0x21d
[546355.480889] [<ffffffff81064a5d>] ? kthread+0x79/0x81
[546355.480892] [<ffffffff81011baa>] ? child_rip+0xa/0x20
[546355.480894] [<ffffffff810649e4>] ? kthread+0x0/0x81
[546355.480896] [<ffffffff81011ba0>] ? child_rip+0x0/0x20
I can't access to the server. Not graphical or ssh access is possible.. But
I can execute remote commands via ssh:
server1# ssh root at server2 'dmesg'
I try to umount ocfs2 mounted devices but it was not possible.
Remote reboot started but didn't finished.
Is there any option?
--
Xavier Diumé
http://socaqui.cat
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://oss.oracle.com/pipermail/ocfs2-users/attachments/20110408/793e4110/attachment-0001.html
More information about the Ocfs2-users
mailing list