[Ocfs2-users] Soft lockup problem

[Daniel]

Hello

This appeared in my messages log:

Jul 24 07:27:41 tilesrv2 kernel: BUG: soft lockup detected on CPU#0!
Jul 24 07:27:41 tilesrv2 kernel:
Jul 24 07:27:41 tilesrv2 kernel: Call Trace:
Jul 24 07:27:41 tilesrv2 kernel:  <IRQ>  [<ffffffff800b2ca3>]
softlockup_tick+0xdb/0xed
Jul 24 07:27:41 tilesrv2 kernel:  [<ffffffff80093424>]
update_process_times+0x42/0x68
Jul 24 07:27:41 tilesrv2 kernel:  [<ffffffff80073d99>]
smp_local_timer_interrupt+0x23/0x47
Jul 24 07:27:41 tilesrv2 kernel:  [<ffffffff8007445b>]
smp_apic_timer_interrupt+0x41/0x47
Jul 24 07:27:41 tilesrv2 kernel:  [<ffffffff8005bcc2>]
apic_timer_interrupt+0x66/0x6c
Jul 24 07:27:41 tilesrv2 kernel:  <EOI>  [<ffffffff8006270f>]
.text.lock.spinlock+0x5/0x30
Jul 24 07:27:41 tilesrv2 kernel:  [<ffffffff884da534>]
:ocfs2_dlm:dlm_assert_master_handler+0x93d/0xd3a
Jul 24 07:27:41 tilesrv2 kernel:  [<ffffffff8001c239>] __mod_timer+0xb0/0xbe
Jul 24 07:27:41 tilesrv2 kernel:  [<ffffffff8849f3a7>]
:ocfs2_nodemanager:o2net_rx_until_empty+0x0/0x9ca
Jul 24 07:27:41 tilesrv2 kernel:  [<ffffffff8849dfcf>]
:ocfs2_nodemanager:o2net_process_message+0x3ef/0x58b
Jul 24 07:27:41 tilesrv2 kernel:  [<ffffffff8849fbf6>]
:ocfs2_nodemanager:o2net_rx_until_empty+0x84f/0x9ca
Jul 24 07:27:41 tilesrv2 kernel:  [<ffffffff8849f3a7>]
:ocfs2_nodemanager:o2net_rx_until_empty+0x0/0x9ca
Jul 24 07:27:41 tilesrv2 kernel:  [<ffffffff8004b2cf>]
run_workqueue+0x94/0xe5
Jul 24 07:27:41 tilesrv2 kernel:  [<ffffffff80047c2e>]
worker_thread+0x0/0x122
Jul 24 07:27:41 tilesrv2 kernel:  [<ffffffff8009b4f6>]
keventd_create_kthread+0x0/0x61
Jul 24 07:27:41 tilesrv2 kernel:  [<ffffffff80047d1e>]
worker_thread+0xf0/0x122
Jul 24 07:27:41 tilesrv2 kernel:  [<ffffffff80086c6f>]
default_wake_function+0x0/0xe
Jul 24 07:27:41 tilesrv2 kernel:  [<ffffffff8009b4f6>]
keventd_create_kthread+0x0/0x61
Jul 24 07:27:41 tilesrv2 kernel:  [<ffffffff8009b4f6>]
keventd_create_kthread+0x0/0x61
Jul 24 07:27:41 tilesrv2 kernel:  [<ffffffff80032189>] kthread+0xfe/0x132
Jul 24 07:27:41 tilesrv2 kernel:  [<ffffffff8005bfe5>] child_rip+0xa/0x11
Jul 24 07:27:41 tilesrv2 kernel:  [<ffffffff8009b4f6>]
keventd_create_kthread+0x0/0x61
Jul 24 07:27:41 tilesrv2 kernel:  [<ffffffff8003208b>] kthread+0x0/0x132
Jul 24 07:27:41 tilesrv2 kernel:  [<ffffffff8005bfdb>] child_rip+0x0/0x11
Jul 24 07:27:41 tilesrv2 kernel:
Jul 24 07:27:41 tilesrv2 kernel: BUG: soft lockup detected on CPU#1!
Jul 24 07:27:41 tilesrv2 kernel:
Jul 24 07:27:41 tilesrv2 kernel: Call Trace:
Jul 24 07:27:41 tilesrv2 kernel:  <IRQ>  [<ffffffff800b2ca3>]
softlockup_tick+0xdb/0xed
Jul 24 07:27:41 tilesrv2 kernel:  [<ffffffff80093424>]
update_process_times+0x42/0x68
Jul 24 07:27:41 tilesrv2 kernel:  [<ffffffff80073d99>]
smp_local_timer_interrupt+0x23/0x47
Jul 24 07:27:41 tilesrv2 kernel:  [<ffffffff8007445b>]
smp_apic_timer_interrupt+0x41/0x47
Jul 24 07:27:41 tilesrv2 kernel:  [<ffffffff8005bcc2>]
apic_timer_interrupt+0x66/0x6c
Jul 24 07:27:41 tilesrv2 kernel:  <EOI>  [<ffffffff884d2e44>]
:ocfs2_dlm:__dlm_lookup_lockres_full+0xbe/0x108
Jul 24 07:27:41 tilesrv2 kernel:  [<ffffffff884d2e55>]
:ocfs2_dlm:__dlm_lookup_lockres_full+0xcf/0x108
Jul 24 07:27:41 tilesrv2 kernel:  [<ffffffff884dadaf>]
:ocfs2_dlm:dlm_get_lock_resource+0xcb/0x18e4
Jul 24 07:27:41 tilesrv2 kernel:  [<ffffffff80061552>]
__wait_on_bit+0x60/0x6f
Jul 24 07:27:41 tilesrv2 kernel:  [<ffffffff80014bf0>] sync_buffer+0x0/0x3f
Jul 24 07:27:41 tilesrv2 kernel:  [<ffffffff884e29d8>]
:ocfs2_dlm:dlm_in_recovery+0xd/0x20
Jul 24 07:27:41 tilesrv2 kernel:  [<ffffffff884e6ec1>]
:ocfs2_dlm:dlm_wait_for_recovery+0xa1/0x116
Jul 24 07:27:41 tilesrv2 kernel:  [<ffffffff885ecfdb>]
:ocfs2:ocfs2_inode_ast_func+0x0/0x6da
Jul 24 07:27:41 tilesrv2 kernel:  [<ffffffff884e0424>]
:ocfs2_dlm:dlmlock+0x751/0x1220
Jul 24 07:27:41 tilesrv2 kernel:  [<ffffffff885f677d>]
:ocfs2:ocfs2_populate_inode+0x4d3/0x558
Jul 24 07:27:41 tilesrv2 kernel:  [<ffffffff8002cc23>] wake_up_bit+0x11/0x22
Jul 24 07:27:41 tilesrv2 kernel:  [<ffffffff885e915a>]
:ocfs2:ocfs2_cluster_unlock+0x65/0x2cb
Jul 24 07:27:41 tilesrv2 kernel:  [<ffffffff885e9509>]
:ocfs2:ocfs2_meta_unlock+0x121/0x180
Jul 24 07:27:41 tilesrv2 kernel:  [<ffffffff885ea34a>]
:ocfs2:ocfs2_lock_create+0x137/0x346
Jul 24 07:27:41 tilesrv2 kernel:  [<ffffffff885ed6b5>]
:ocfs2:ocfs2_inode_bast_func+0x0/0x15b
Jul 24 07:27:41 tilesrv2 kernel:  [<ffffffff885ea943>]
:ocfs2:ocfs2_cluster_lock+0x205/0x898
Jul 24 07:27:41 tilesrv2 kernel:  [<ffffffff885e8a3e>]
:ocfs2:ocfs2_status_completion_cb+0x0/0xb
Jul 24 07:27:41 tilesrv2 kernel:  [<ffffffff885eeaf1>]
:ocfs2:ocfs2_meta_lock_full+0x216/0xd35
Jul 24 07:27:41 tilesrv2 kernel:  [<ffffffff885f7f7f>]
:ocfs2:ocfs2_inode_revalidate+0x14f/0x228
Jul 24 07:27:41 tilesrv2 kernel:  [<ffffffff885f2fc6>]
:ocfs2:ocfs2_getattr+0x79/0x159
Jul 24 07:27:41 tilesrv2 kernel:  [<ffffffff8003e713>]
vfs_lstat_fd+0x2f/0x47
Jul 24 07:27:41 tilesrv2 kernel:  [<ffffffff885e68c1>]
:ocfs2:ocfs2_readdir+0x40e/0x426
Jul 24 07:27:41 tilesrv2 kernel:  [<ffffffff800252e1>] filldir+0x0/0xb7
Jul 24 07:27:41 tilesrv2 kernel:  [<ffffffff8002a4da>]
sys_newlstat+0x19/0x31
Jul 24 07:27:41 tilesrv2 kernel:  [<ffffffff8005b261>] tracesys+0x71/0xdc
Jul 24 07:27:41 tilesrv2 kernel:  [<ffffffff8005b2c1>] tracesys+0xd1/0xdc

Dell 1959 2xQuadcore, EMC 3-20, CentOS 5 2.6.18-8.1.8.el5 OCFS2 1.2.6-1

What can cause this? Where do I start looking?

Daniel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://oss.oracle.com/pipermail/ocfs2-users/attachments/20070729/cc7f4e7b/attachment.html