[Ocfs2-users] GPF when mounting second device in same cluster

Wed Jun 29 15:15:36 PDT 2011

Strange. Both udevd and mount thread encountered issue in memory
allocation routine. I would suggest you ping the kernel vendor. This looks
more than just the fs.

On 06/28/2011 04:36 PM, Richard Pickett wrote:
> Gents,
>
> OK, back to the single cluster, 2-node, w/ 3 devices. Here's my cluster.conf
> cluster:
> node_count = 2
> name = xport1
>
> node:
> ip_port = 7777
> ip_address = 10.8.4.81
> number = 1
> name = wings
> cluster = xport1
> node:
> ip_port = 7777
> ip_address = 10.8.4.82
> number = 2
> name = xtc
> cluster = xport1
>
> I formatted each device from one node.
>
> I mount the first device on both nodes, all seems well, `mount` reflects the newly loaded device.
>
> I mount the second device on the first node (just testing), it returns w/out an error, `mount` doesn't show it mounted, and I get the below in the syslog, a couple minutes later the box is unresponsive and hast to be powered off/on to come back up.
>
> How do I start tracking down the cause of this problem?
>
> Jun 28 23:17:06 wings kernel: [ 1016.037931] o2dlm: Nodes in domain 4F1AF6E3CA514B8AB64022249255DD26: 1
> Jun 28 23:17:06 wings kernel: [ 1016.040713] general protection fault: 0000 [#1] SMP
> Jun 28 23:17:06 wings kernel: [ 1016.040758] last sysfs file: /sys/devices/pci0000:00/0000:00:11.0/0000:02:06.0/host4/target4:0:0/4:0:0:0/block/sdc/uevent
> Jun 28 23:17:06 wings kernel: [ 1016.040827] CPU 0
> Jun 28 23:17:06 wings kernel: [ 1016.040839] Modules linked in: ocfs2 quota_tree ocfs2_dlmfs ocfs2_stack_o2cb ocfs2_dlm ocfs2_nodemanager ocfs2_stackglue configfs acpiphp xt_tcpudp nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack iptable_filter ip_tables x_tables ppdev vmware_balloon psmouse serio_raw parport_pc lp intel_agp shpchp i2c_piix4 parport floppy mptspi mptscsih e1000 mptbase scsi_transport_spi [last unloaded: configfs]
> Jun 28 23:17:06 wings kernel: [ 1016.041099]
> Jun 28 23:17:06 wings kernel: [ 1016.041110] Pid: 542, comm: udevd Not tainted 2.6.35-28-generic #49-Ubuntu 440BX Desktop Reference Platform/VMware Virtual Platform
> Jun 28 23:17:06 wings kernel: [ 1016.041184] RIP: 0010:[<ffffffff81144164>]  [<ffffffff81144164>] __kmalloc_track_caller+0x84/0x1b0
> Jun 28 23:17:06 wings kernel: [ 1016.041244] RSP: 0018:ffff88007c701cc8  EFLAGS: 00010086
> Jun 28 23:17:06 wings kernel: [ 1016.041278] RAX: ffff880001e128c0 RBX: 00000000000000d0 RCX: ffff88007f72eb40
> Jun 28 23:17:06 wings kernel: [ 1016.041322] RDX: 0000000000000000 RSI: 00000000000000d0 RDI: 0000000000000004
> Jun 28 23:17:06 wings kernel: [ 1016.041367] RBP: ffff88007c701d08 R08: 0000000000000000 R09: 0000000000000246
> Jun 28 23:17:06 wings kernel: [ 1016.041540] R10: 0000000000000000 R11: 0000000000000202 R12: ffffffff81a24d20
> Jun 28 23:17:06 wings kernel: [ 1016.041634] R13: 000000000000000f R14: e2c3f000ff53f000 R15: 00000000000000d0
> Jun 28 23:17:06 wings kernel: [ 1016.041743] FS:  00007ff946e737a0(0000) GS:ffff880001e00000(0000) knlGS:0000000000000000
> Jun 28 23:17:06 wings kernel: [ 1016.041869] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> Jun 28 23:17:06 wings kernel: [ 1016.041943] CR2: 00007ff946e7a000 CR3: 000000007c6fc000 CR4: 00000000000006f0
> Jun 28 23:17:06 wings kernel: [ 1016.042046] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> Jun 28 23:17:06 wings kernel: [ 1016.042149] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> Jun 28 23:17:06 wings kernel: [ 1016.042232] Process udevd (pid: 542, threadinfo ffff88007c700000, task ffff88007c6dadc0)
> Jun 28 23:17:06 wings kernel: [ 1016.042358] Stack:
> Jun 28 23:17:06 wings kernel: [ 1016.042409]  ffff88007c701d18 ffffffff8115f492 ffff88007c701ce8 0000000000000000
> Jun 28 23:17:06 wings kernel: [ 1016.042495] <0> ffff880066e28160 000000000000000f 00000000000000d0 ffff88007a617c30
> Jun 28 23:17:06 wings kernel: [ 1016.042623] <0> ffff88007c701d38 ffffffff81118155 00000000ffffffff ffff880066e280c0
> Jun 28 23:17:06 wings kernel: [ 1016.042789] Call Trace:
> Jun 28 23:17:06 wings kernel: [ 1016.042851]  [<ffffffff8115f492>] ? vfs_rename+0xc2/0x280
> Jun 28 23:17:06 wings kernel: [ 1016.042925]  [<ffffffff81118155>] kstrdup+0x45/0x80
> Jun 28 23:17:06 wings kernel: [ 1016.043006]  [<ffffffff8115f492>] vfs_rename+0xc2/0x280
> Jun 28 23:17:06 wings kernel: [ 1016.043091]  [<ffffffff8115f92d>] ? __lookup_hash+0x5d/0x160
> Jun 28 23:17:06 wings kernel: [ 1016.043179]  [<ffffffff811611c2>] sys_renameat+0x252/0x280
> Jun 28 23:17:06 wings kernel: [ 1016.043267]  [<ffffffff8111ee8e>] ? free_pgtables+0xce/0x120
> Jun 28 23:17:06 wings kernel: [ 1016.043357]  [<ffffffff812b808b>] ? cpumask_any_but+0x2b/0x40
> Jun 28 23:17:06 wings kernel: [ 1016.043448]  [<ffffffff81043862>] ? flush_tlb_mm+0x42/0xa0
> Jun 28 23:17:06 wings kernel: [ 1016.043536]  [<ffffffff811240fe>] ? remove_vma+0x6e/0x90
> Jun 28 23:17:06 wings kernel: [ 1016.043621]  [<ffffffff81126549>] ? do_munmap+0x309/0x3a0
> Jun 28 23:17:06 wings kernel: [ 1016.043709]  [<ffffffff8116120b>] sys_rename+0x1b/0x20
> Jun 28 23:17:06 wings kernel: [ 1016.043800]  [<ffffffff8100a0f2>] system_call_fastpath+0x16/0x1b
> Jun 28 23:17:06 wings kernel: [ 1016.043889] Code: 44 00 00 49 89 c1 fa 66 0f 1f 44 00 00 65 48 8b 14 25 58 eb 00 00 49 8b 04 24 48 8d 04 02 4c 8b 30 4d 85 f6 74 78 49 63 54 24 18 <49> 8b 14 16 48 89 10 4c 89 cf 57 9d 0f 1f 44 00 00 4d 85 f6 75
> Jun 28 23:17:06 wings kernel: [ 1016.044228] RIP  [<ffffffff81144164>] __kmalloc_track_caller+0x84/0x1b0
> Jun 28 23:17:06 wings kernel: [ 1016.044324]  RSP <ffff88007c701cc8>
> Jun 28 23:17:06 wings kernel: [ 1016.044575] ---[ end trace 93a9f0d37546adc3 ]---
> Jun 28 23:17:06 wings udevd[396]: worker [542] unexpectedly returned with status 0x000b
> Jun 28 23:17:06 wings udevd[396]: worker [542] failed while handling '/devices/pci0000:00/0000:00:11.0/0000:02:06.0/host4/target4:0:0/4:0:0:0/block/sdc/sdc1'
> Jun 28 23:17:06 wings kernel: [ 1016.067655] JBD: Ignoring recovery information on journal
> Jun 28 23:17:06 wings kernel: [ 1016.088738] general protection fault: 0000 [#2] SMP
> Jun 28 23:17:06 wings kernel: [ 1016.088861] last sysfs file: /sys/devices/pci0000:00/0000:00:11.0/0000:02:06.0/host4/target4:0:0/4:0:0:0/block/sdc/uevent
> Jun 28 23:17:06 wings kernel: [ 1016.089022] CPU 0
> Jun 28 23:17:06 wings kernel: [ 1016.089051] Modules linked in: ocfs2 quota_tree ocfs2_dlmfs ocfs2_stack_o2cb ocfs2_dlm ocfs2_nodemanager ocfs2_stackglue configfs acpiphp xt_tcpudp nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack iptable_filter ip_tables x_tables ppdev vmware_balloon psmouse serio_raw parport_pc lp intel_agp shpchp i2c_piix4 parport floppy mptspi mptscsih e1000 mptbase scsi_transport_spi [last unloaded: configfs]
> Jun 28 23:17:06 wings kernel: [ 1016.090189]
> Jun 28 23:17:06 wings kernel: [ 1016.090253] Pid: 1906, comm: mount.ocfs2 Tainted: G      D     2.6.35-28-generic #49-Ubuntu 440BX Desktop Reference Platform/VMware Virtual Platform
> Jun 28 23:17:06 wings kernel: [ 1016.090429] RIP: 0010:[<ffffffff81144318>]  [<ffffffff81144318>] __kmalloc+0x88/0x1c0
> Jun 28 23:17:06 wings kernel: [ 1016.090590] RSP: 0018:ffff88007bfddb68  EFLAGS: 00010086
> Jun 28 23:17:06 wings kernel: [ 1016.090678] RAX: ffff880001e128c0 RBX: 00000000000080d0 RCX: ffffffffa033c6c4
> Jun 28 23:17:06 wings kernel: [ 1016.090776] RDX: 0000000000000000 RSI: 00000000000080d0 RDI: 0000000000000004
> Jun 28 23:17:06 wings kernel: [ 1016.090875] RBP: ffff88007bfddba8 R08: e018000000000000 R09: 0000000000000246
> Jun 28 23:17:06 wings kernel: [ 1016.090973] R10: ffff880066e5d260 R11: 0000000000000000 R12: ffffffff81a24d20
> Jun 28 23:17:06 wings kernel: [ 1016.091072] R13: 000000000000000a R14: e2c3f000ff53f000 R15: 00000000000080d0
> Jun 28 23:17:06 wings kernel: [ 1016.091171] FS:  00007fea25fff720(0000) GS:ffff880001e00000(0000) knlGS:0000000000000000
> Jun 28 23:17:06 wings kernel: [ 1016.091313] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> Jun 28 23:17:06 wings kernel: [ 1016.091403] CR2: 00007ff94679dec0 CR3: 000000007b4d8000 CR4: 00000000000006f0
> Jun 28 23:17:06 wings kernel: [ 1016.091528] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> Jun 28 23:17:06 wings kernel: [ 1016.091646] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> Jun 28 23:17:06 wings kernel: [ 1016.091745] Process mount.ocfs2 (pid: 1906, threadinfo ffff88007bfdc000, task ffff88007b82db80)
> Jun 28 23:17:06 wings kernel: [ 1016.091890] Stack:
> Jun 28 23:17:06 wings kernel: [ 1016.091957]  0000000000000000 ffffffffa033c6c4 0000000000000001 0000000000000000
> Jun 28 23:17:06 wings kernel: [ 1016.092112] <0> ffff88007bf6e000 ffff88007bf6e000 ffff88007bfddcf8 ffff88007b4a3c00
> Jun 28 23:17:06 wings kernel: [ 1016.092338] <0> ffff88007bfddc08 ffffffffa033c6c4 ffff88007b82dfd8 ffff88007b82db80
> Jun 28 23:17:06 wings kernel: [ 1016.092622] Call Trace:
> Jun 28 23:17:06 wings kernel: [ 1016.092705]  [<ffffffffa033c6c4>] ? ocfs2_compute_replay_slots+0x64/0x160 [ocfs2]
> Jun 28 23:17:06 wings kernel: [ 1016.092853]  [<ffffffffa033c6c4>] ocfs2_compute_replay_slots+0x64/0x160 [ocfs2]
> Jun 28 23:17:06 wings kernel: [ 1016.093000]  [<ffffffffa0364bf5>] ocfs2_check_volume+0x115/0x650 [ocfs2]
> Jun 28 23:17:06 wings kernel: [ 1016.093104]  [<ffffffffa03651ef>] ? ocfs2_mount_volume+0xbf/0x520 [ocfs2]
> Jun 28 23:17:06 wings kernel: [ 1016.093208]  [<ffffffffa0365225>] ocfs2_mount_volume+0xf5/0x520 [ocfs2]
> Jun 28 23:17:06 wings kernel: [ 1016.093312]  [<ffffffffa0368c9b>] ocfs2_fill_super+0x72b/0xd80 [ocfs2]
> Jun 28 23:17:06 wings kernel: [ 1016.093408]  [<ffffffff81156694>] get_sb_bdev+0x194/0x1d0
> Jun 28 23:17:06 wings kernel: [ 1016.093504]  [<ffffffffa0368570>] ? ocfs2_fill_super+0x0/0xd80 [ocfs2]
> Jun 28 23:17:06 wings kernel: [ 1016.093607]  [<ffffffffa03628f8>] ocfs2_get_sb+0x18/0x20 [ocfs2]
> Jun 28 23:17:06 wings kernel: [ 1016.093714]  [<ffffffff81155eaa>] vfs_kern_mount+0x8a/0x1f0
> Jun 28 23:17:06 wings kernel: [ 1016.093821]  [<ffffffff81156082>] do_kern_mount+0x52/0x130
> Jun 28 23:17:06 wings kernel: [ 1016.093928]  [<ffffffff8158b733>] ? _lock_kernel+0x53/0xa3
> Jun 28 23:17:06 wings kernel: [ 1016.094036]  [<ffffffff811717fa>] do_mount+0x20a/0x260
> Jun 28 23:17:06 wings kernel: [ 1016.094140]  [<ffffffff811718e0>] sys_mount+0x90/0xe0
> Jun 28 23:17:06 wings kernel: [ 1016.094244]  [<ffffffff8100a0f2>] system_call_fastpath+0x16/0x1b
> Jun 28 23:17:06 wings kernel: [ 1016.094353] Code: 89 c1 fa 66 0f 1f 44 00 00 65 48 8b 14 25 58 eb 00 00 49 8b 04 24 48 8d 04 02 4c 8b 30 4d 85 f6 0f 84 7c 00 00 00 49 63 54 24 18 <49> 8b 14 16 48 89 10 4c 89 cf 57 9d 0f 1f 44 00 00 4d 85 f6 75
> Jun 28 23:17:06 wings kernel: [ 1016.095976] RIP  [<ffffffff81144318>] __kmalloc+0x88/0x1c0
> Jun 28 23:17:06 wings kernel: [ 1016.096103]  RSP <ffff88007bfddb68>
> Jun 28 23:17:06 wings kernel: [ 1016.096195] ---[ end trace 93a9f0d37546adc4 ]---
>
>
> _______________________________________________
> Ocfs2-users mailing list
> Ocfs2-users at oss.oracle.com
> http://oss.oracle.com/mailman/listinfo/ocfs2-users

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://oss.oracle.com/pipermail/ocfs2-users/attachments/20110629/3027bcee/attachment-0001.html 