[Ocfs2-users] GPF when mounting second device in same cluster

[Richard Pickett]

Gents,

OK, back to the single cluster, 2-node, w/ 3 devices. Here's my cluster.conf
cluster:
node_count = 2
 name = xport1

node:
ip_port = 7777
ip_address = 10.8.4.81
 number = 1
name = wings
cluster = xport1
node:
ip_port = 7777
ip_address = 10.8.4.82
number = 2
 name = xtc
cluster = xport1

I formatted each device from one node.

I mount the first device on both nodes, all seems well, `mount` reflects the
newly loaded device.

I mount the second device on the first node (just testing), it returns w/out
an error, `mount` doesn't show it mounted, and I get the below in the
syslog, a couple minutes later the box is unresponsive and hast to be
powered off/on to come back up.

How do I start tracking down the cause of this problem?

Jun 28 23:17:06 wings kernel: [ 1016.037931] o2dlm: Nodes in domain
4F1AF6E3CA514B8AB64022249255DD26: 1
Jun 28 23:17:06 wings kernel: [ 1016.040713] general protection fault: 0000
[#1] SMP
Jun 28 23:17:06 wings kernel: [ 1016.040758] last sysfs file:
/sys/devices/pci0000:00/0000:00:11.0/0000:02:06.0/host4/target4:0:0/4:0:0:0/block/sdc/uevent
Jun 28 23:17:06 wings kernel: [ 1016.040827] CPU 0
Jun 28 23:17:06 wings kernel: [ 1016.040839] Modules linked in: ocfs2
quota_tree ocfs2_dlmfs ocfs2_stack_o2cb ocfs2_dlm ocfs2_nodemanager
ocfs2_stackglue configfs acpiphp xt_tcpudp nf_conntrack_ipv4 nf_defrag_ipv4
xt_conntrack nf_conntrack iptable_filter ip_tables x_tables ppdev
vmware_balloon psmouse serio_raw parport_pc lp intel_agp shpchp i2c_piix4
parport floppy mptspi mptscsih e1000 mptbase scsi_transport_spi [last
unloaded: configfs]
Jun 28 23:17:06 wings kernel: [ 1016.041099]
Jun 28 23:17:06 wings kernel: [ 1016.041110] Pid: 542, comm: udevd Not
tainted 2.6.35-28-generic #49-Ubuntu 440BX Desktop Reference Platform/VMware
Virtual Platform
Jun 28 23:17:06 wings kernel: [ 1016.041184] RIP: 0010:[<ffffffff81144164>]
 [<ffffffff81144164>] __kmalloc_track_caller+0x84/0x1b0
Jun 28 23:17:06 wings kernel: [ 1016.041244] RSP: 0018:ffff88007c701cc8
 EFLAGS: 00010086
Jun 28 23:17:06 wings kernel: [ 1016.041278] RAX: ffff880001e128c0 RBX:
00000000000000d0 RCX: ffff88007f72eb40
Jun 28 23:17:06 wings kernel: [ 1016.041322] RDX: 0000000000000000 RSI:
00000000000000d0 RDI: 0000000000000004
Jun 28 23:17:06 wings kernel: [ 1016.041367] RBP: ffff88007c701d08 R08:
0000000000000000 R09: 0000000000000246
Jun 28 23:17:06 wings kernel: [ 1016.041540] R10: 0000000000000000 R11:
0000000000000202 R12: ffffffff81a24d20
Jun 28 23:17:06 wings kernel: [ 1016.041634] R13: 000000000000000f R14:
e2c3f000ff53f000 R15: 00000000000000d0
Jun 28 23:17:06 wings kernel: [ 1016.041743] FS:  00007ff946e737a0(0000)
GS:ffff880001e00000(0000) knlGS:0000000000000000
Jun 28 23:17:06 wings kernel: [ 1016.041869] CS:  0010 DS: 0000 ES: 0000
CR0: 0000000080050033
Jun 28 23:17:06 wings kernel: [ 1016.041943] CR2: 00007ff946e7a000 CR3:
000000007c6fc000 CR4: 00000000000006f0
Jun 28 23:17:06 wings kernel: [ 1016.042046] DR0: 0000000000000000 DR1:
0000000000000000 DR2: 0000000000000000
Jun 28 23:17:06 wings kernel: [ 1016.042149] DR3: 0000000000000000 DR6:
00000000ffff0ff0 DR7: 0000000000000400
Jun 28 23:17:06 wings kernel: [ 1016.042232] Process udevd (pid: 542,
threadinfo ffff88007c700000, task ffff88007c6dadc0)
Jun 28 23:17:06 wings kernel: [ 1016.042358] Stack:
Jun 28 23:17:06 wings kernel: [ 1016.042409]  ffff88007c701d18
ffffffff8115f492 ffff88007c701ce8 0000000000000000
Jun 28 23:17:06 wings kernel: [ 1016.042495] <0> ffff880066e28160
000000000000000f 00000000000000d0 ffff88007a617c30
Jun 28 23:17:06 wings kernel: [ 1016.042623] <0> ffff88007c701d38
ffffffff81118155 00000000ffffffff ffff880066e280c0
Jun 28 23:17:06 wings kernel: [ 1016.042789] Call Trace:
Jun 28 23:17:06 wings kernel: [ 1016.042851]  [<ffffffff8115f492>] ?
vfs_rename+0xc2/0x280
Jun 28 23:17:06 wings kernel: [ 1016.042925]  [<ffffffff81118155>]
kstrdup+0x45/0x80
Jun 28 23:17:06 wings kernel: [ 1016.043006]  [<ffffffff8115f492>]
vfs_rename+0xc2/0x280
Jun 28 23:17:06 wings kernel: [ 1016.043091]  [<ffffffff8115f92d>] ?
__lookup_hash+0x5d/0x160
Jun 28 23:17:06 wings kernel: [ 1016.043179]  [<ffffffff811611c2>]
sys_renameat+0x252/0x280
Jun 28 23:17:06 wings kernel: [ 1016.043267]  [<ffffffff8111ee8e>] ?
free_pgtables+0xce/0x120
Jun 28 23:17:06 wings kernel: [ 1016.043357]  [<ffffffff812b808b>] ?
cpumask_any_but+0x2b/0x40
Jun 28 23:17:06 wings kernel: [ 1016.043448]  [<ffffffff81043862>] ?
flush_tlb_mm+0x42/0xa0
Jun 28 23:17:06 wings kernel: [ 1016.043536]  [<ffffffff811240fe>] ?
remove_vma+0x6e/0x90
Jun 28 23:17:06 wings kernel: [ 1016.043621]  [<ffffffff81126549>] ?
do_munmap+0x309/0x3a0
Jun 28 23:17:06 wings kernel: [ 1016.043709]  [<ffffffff8116120b>]
sys_rename+0x1b/0x20
Jun 28 23:17:06 wings kernel: [ 1016.043800]  [<ffffffff8100a0f2>]
system_call_fastpath+0x16/0x1b
Jun 28 23:17:06 wings kernel: [ 1016.043889] Code: 44 00 00 49 89 c1 fa 66
0f 1f 44 00 00 65 48 8b 14 25 58 eb 00 00 49 8b 04 24 48 8d 04 02 4c 8b 30
4d 85 f6 74 78 49 63 54 24 18 <49> 8b 14 16 48 89 10 4c 89 cf 57 9d 0f 1f 44
00 00 4d 85 f6 75
Jun 28 23:17:06 wings kernel: [ 1016.044228] RIP  [<ffffffff81144164>]
__kmalloc_track_caller+0x84/0x1b0
Jun 28 23:17:06 wings kernel: [ 1016.044324]  RSP <ffff88007c701cc8>
Jun 28 23:17:06 wings kernel: [ 1016.044575] ---[ end trace 93a9f0d37546adc3
]---
Jun 28 23:17:06 wings udevd[396]: worker [542] unexpectedly returned with
status 0x000b
Jun 28 23:17:06 wings udevd[396]: worker [542] failed while handling
'/devices/pci0000:00/0000:00:11.0/0000:02:06.0/host4/target4:0:0/4:0:0:0/block/sdc/sdc1'
Jun 28 23:17:06 wings kernel: [ 1016.067655] JBD: Ignoring recovery
information on journal
Jun 28 23:17:06 wings kernel: [ 1016.088738] general protection fault: 0000
[#2] SMP
Jun 28 23:17:06 wings kernel: [ 1016.088861] last sysfs file:
/sys/devices/pci0000:00/0000:00:11.0/0000:02:06.0/host4/target4:0:0/4:0:0:0/block/sdc/uevent
Jun 28 23:17:06 wings kernel: [ 1016.089022] CPU 0
Jun 28 23:17:06 wings kernel: [ 1016.089051] Modules linked in: ocfs2
quota_tree ocfs2_dlmfs ocfs2_stack_o2cb ocfs2_dlm ocfs2_nodemanager
ocfs2_stackglue configfs acpiphp xt_tcpudp nf_conntrack_ipv4 nf_defrag_ipv4
xt_conntrack nf_conntrack iptable_filter ip_tables x_tables ppdev
vmware_balloon psmouse serio_raw parport_pc lp intel_agp shpchp i2c_piix4
parport floppy mptspi mptscsih e1000 mptbase scsi_transport_spi [last
unloaded: configfs]
Jun 28 23:17:06 wings kernel: [ 1016.090189]
Jun 28 23:17:06 wings kernel: [ 1016.090253] Pid: 1906, comm: mount.ocfs2
Tainted: G      D     2.6.35-28-generic #49-Ubuntu 440BX Desktop Reference
Platform/VMware Virtual Platform
Jun 28 23:17:06 wings kernel: [ 1016.090429] RIP: 0010:[<ffffffff81144318>]
 [<ffffffff81144318>] __kmalloc+0x88/0x1c0
Jun 28 23:17:06 wings kernel: [ 1016.090590] RSP: 0018:ffff88007bfddb68
 EFLAGS: 00010086
Jun 28 23:17:06 wings kernel: [ 1016.090678] RAX: ffff880001e128c0 RBX:
00000000000080d0 RCX: ffffffffa033c6c4
Jun 28 23:17:06 wings kernel: [ 1016.090776] RDX: 0000000000000000 RSI:
00000000000080d0 RDI: 0000000000000004
Jun 28 23:17:06 wings kernel: [ 1016.090875] RBP: ffff88007bfddba8 R08:
e018000000000000 R09: 0000000000000246
Jun 28 23:17:06 wings kernel: [ 1016.090973] R10: ffff880066e5d260 R11:
0000000000000000 R12: ffffffff81a24d20
Jun 28 23:17:06 wings kernel: [ 1016.091072] R13: 000000000000000a R14:
e2c3f000ff53f000 R15: 00000000000080d0
Jun 28 23:17:06 wings kernel: [ 1016.091171] FS:  00007fea25fff720(0000)
GS:ffff880001e00000(0000) knlGS:0000000000000000
Jun 28 23:17:06 wings kernel: [ 1016.091313] CS:  0010 DS: 0000 ES: 0000
CR0: 000000008005003b
Jun 28 23:17:06 wings kernel: [ 1016.091403] CR2: 00007ff94679dec0 CR3:
000000007b4d8000 CR4: 00000000000006f0
Jun 28 23:17:06 wings kernel: [ 1016.091528] DR0: 0000000000000000 DR1:
0000000000000000 DR2: 0000000000000000
Jun 28 23:17:06 wings kernel: [ 1016.091646] DR3: 0000000000000000 DR6:
00000000ffff0ff0 DR7: 0000000000000400
Jun 28 23:17:06 wings kernel: [ 1016.091745] Process mount.ocfs2 (pid: 1906,
threadinfo ffff88007bfdc000, task ffff88007b82db80)
Jun 28 23:17:06 wings kernel: [ 1016.091890] Stack:
Jun 28 23:17:06 wings kernel: [ 1016.091957]  0000000000000000
ffffffffa033c6c4 0000000000000001 0000000000000000
Jun 28 23:17:06 wings kernel: [ 1016.092112] <0> ffff88007bf6e000
ffff88007bf6e000 ffff88007bfddcf8 ffff88007b4a3c00
Jun 28 23:17:06 wings kernel: [ 1016.092338] <0> ffff88007bfddc08
ffffffffa033c6c4 ffff88007b82dfd8 ffff88007b82db80
Jun 28 23:17:06 wings kernel: [ 1016.092622] Call Trace:
Jun 28 23:17:06 wings kernel: [ 1016.092705]  [<ffffffffa033c6c4>] ?
ocfs2_compute_replay_slots+0x64/0x160 [ocfs2]
Jun 28 23:17:06 wings kernel: [ 1016.092853]  [<ffffffffa033c6c4>]
ocfs2_compute_replay_slots+0x64/0x160 [ocfs2]
Jun 28 23:17:06 wings kernel: [ 1016.093000]  [<ffffffffa0364bf5>]
ocfs2_check_volume+0x115/0x650 [ocfs2]
Jun 28 23:17:06 wings kernel: [ 1016.093104]  [<ffffffffa03651ef>] ?
ocfs2_mount_volume+0xbf/0x520 [ocfs2]
Jun 28 23:17:06 wings kernel: [ 1016.093208]  [<ffffffffa0365225>]
ocfs2_mount_volume+0xf5/0x520 [ocfs2]
Jun 28 23:17:06 wings kernel: [ 1016.093312]  [<ffffffffa0368c9b>]
ocfs2_fill_super+0x72b/0xd80 [ocfs2]
Jun 28 23:17:06 wings kernel: [ 1016.093408]  [<ffffffff81156694>]
get_sb_bdev+0x194/0x1d0
Jun 28 23:17:06 wings kernel: [ 1016.093504]  [<ffffffffa0368570>] ?
ocfs2_fill_super+0x0/0xd80 [ocfs2]
Jun 28 23:17:06 wings kernel: [ 1016.093607]  [<ffffffffa03628f8>]
ocfs2_get_sb+0x18/0x20 [ocfs2]
Jun 28 23:17:06 wings kernel: [ 1016.093714]  [<ffffffff81155eaa>]
vfs_kern_mount+0x8a/0x1f0
Jun 28 23:17:06 wings kernel: [ 1016.093821]  [<ffffffff81156082>]
do_kern_mount+0x52/0x130
Jun 28 23:17:06 wings kernel: [ 1016.093928]  [<ffffffff8158b733>] ?
_lock_kernel+0x53/0xa3
Jun 28 23:17:06 wings kernel: [ 1016.094036]  [<ffffffff811717fa>]
do_mount+0x20a/0x260
Jun 28 23:17:06 wings kernel: [ 1016.094140]  [<ffffffff811718e0>]
sys_mount+0x90/0xe0
Jun 28 23:17:06 wings kernel: [ 1016.094244]  [<ffffffff8100a0f2>]
system_call_fastpath+0x16/0x1b
Jun 28 23:17:06 wings kernel: [ 1016.094353] Code: 89 c1 fa 66 0f 1f 44 00
00 65 48 8b 14 25 58 eb 00 00 49 8b 04 24 48 8d 04 02 4c 8b 30 4d 85 f6 0f
84 7c 00 00 00 49 63 54 24 18 <49> 8b 14 16 48 89 10 4c 89 cf 57 9d 0f 1f 44
00 00 4d 85 f6 75
Jun 28 23:17:06 wings kernel: [ 1016.095976] RIP  [<ffffffff81144318>]
__kmalloc+0x88/0x1c0
Jun 28 23:17:06 wings kernel: [ 1016.096103]  RSP <ffff88007bfddb68>
Jun 28 23:17:06 wings kernel: [ 1016.096195] ---[ end trace 93a9f0d37546adc4
]---
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://oss.oracle.com/pipermail/ocfs2-users/attachments/20110628/e82dca57/attachment.html