<div>Gents,</div><div><br></div><div>OK, back to the single cluster, 2-node, w/ 3 devices. Here's my cluster.conf</div><div><div>cluster:</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>node_count = 2</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>name = xport1</div><div><br></div><div>node:</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>ip_port = 7777</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>ip_address = 10.8.4.81</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>number = 1</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>name = wings</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>cluster = xport1</div>
<div>node:</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>ip_port = 7777</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>ip_address = 10.8.4.82</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>number = 2</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>name = xtc</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>cluster = xport1</div></div><div><br></div><div>I formatted each device from one node.</div>
<div><br></div><div>I mount the first device on both nodes, all seems well, `mount` reflects the newly loaded device.</div><div><br></div><div>I mount the second device on the first node (just testing), it returns w/out an error, `mount` doesn't show it mounted, and I get the below in the syslog, a couple minutes later the box is unresponsive and hast to be powered off/on to come back up.</div>
<div><br></div><div>How do I start tracking down the cause of this problem?</div><div><br></div><div>Jun 28 23:17:06 wings kernel: [ 1016.037931] o2dlm: Nodes in domain 4F1AF6E3CA514B8AB64022249255DD26: 1</div><div>Jun 28 23:17:06 wings kernel: [ 1016.040713] general protection fault: 0000 [#1] SMP</div>
<div>Jun 28 23:17:06 wings kernel: [ 1016.040758] last sysfs file: /sys/devices/pci0000:00/0000:00:11.0/0000:02:06.0/host4/target4:0:0/4:0:0:0/block/sdc/uevent</div><div>Jun 28 23:17:06 wings kernel: [ 1016.040827] CPU 0</div>
<div>Jun 28 23:17:06 wings kernel: [ 1016.040839] Modules linked in: ocfs2 quota_tree ocfs2_dlmfs ocfs2_stack_o2cb ocfs2_dlm ocfs2_nodemanager ocfs2_stackglue configfs acpiphp xt_tcpudp nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack iptable_filter ip_tables x_tables ppdev vmware_balloon psmouse serio_raw parport_pc lp intel_agp shpchp i2c_piix4 parport floppy mptspi mptscsih e1000 mptbase scsi_transport_spi [last unloaded: configfs]</div>
<div>Jun 28 23:17:06 wings kernel: [ 1016.041099]</div><div>Jun 28 23:17:06 wings kernel: [ 1016.041110] Pid: 542, comm: udevd Not tainted 2.6.35-28-generic #49-Ubuntu 440BX Desktop Reference Platform/VMware Virtual Platform</div>
<div>Jun 28 23:17:06 wings kernel: [ 1016.041184] RIP: 0010:[<ffffffff81144164>] [<ffffffff81144164>] __kmalloc_track_caller+0x84/0x1b0</div><div>Jun 28 23:17:06 wings kernel: [ 1016.041244] RSP: 0018:ffff88007c701cc8 EFLAGS: 00010086</div>
<div>Jun 28 23:17:06 wings kernel: [ 1016.041278] RAX: ffff880001e128c0 RBX: 00000000000000d0 RCX: ffff88007f72eb40</div><div>Jun 28 23:17:06 wings kernel: [ 1016.041322] RDX: 0000000000000000 RSI: 00000000000000d0 RDI: 0000000000000004</div>
<div>Jun 28 23:17:06 wings kernel: [ 1016.041367] RBP: ffff88007c701d08 R08: 0000000000000000 R09: 0000000000000246</div><div>Jun 28 23:17:06 wings kernel: [ 1016.041540] R10: 0000000000000000 R11: 0000000000000202 R12: ffffffff81a24d20</div>
<div>Jun 28 23:17:06 wings kernel: [ 1016.041634] R13: 000000000000000f R14: e2c3f000ff53f000 R15: 00000000000000d0</div><div>Jun 28 23:17:06 wings kernel: [ 1016.041743] FS: 00007ff946e737a0(0000) GS:ffff880001e00000(0000) knlGS:0000000000000000</div>
<div>Jun 28 23:17:06 wings kernel: [ 1016.041869] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033</div><div>Jun 28 23:17:06 wings kernel: [ 1016.041943] CR2: 00007ff946e7a000 CR3: 000000007c6fc000 CR4: 00000000000006f0</div>
<div>Jun 28 23:17:06 wings kernel: [ 1016.042046] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000</div><div>Jun 28 23:17:06 wings kernel: [ 1016.042149] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400</div>
<div>Jun 28 23:17:06 wings kernel: [ 1016.042232] Process udevd (pid: 542, threadinfo ffff88007c700000, task ffff88007c6dadc0)</div><div>Jun 28 23:17:06 wings kernel: [ 1016.042358] Stack:</div><div>Jun 28 23:17:06 wings kernel: [ 1016.042409] ffff88007c701d18 ffffffff8115f492 ffff88007c701ce8 0000000000000000</div>
<div>Jun 28 23:17:06 wings kernel: [ 1016.042495] <0> ffff880066e28160 000000000000000f 00000000000000d0 ffff88007a617c30</div><div>Jun 28 23:17:06 wings kernel: [ 1016.042623] <0> ffff88007c701d38 ffffffff81118155 00000000ffffffff ffff880066e280c0</div>
<div>Jun 28 23:17:06 wings kernel: [ 1016.042789] Call Trace:</div><div>Jun 28 23:17:06 wings kernel: [ 1016.042851] [<ffffffff8115f492>] ? vfs_rename+0xc2/0x280</div><div>Jun 28 23:17:06 wings kernel: [ 1016.042925] [<ffffffff81118155>] kstrdup+0x45/0x80</div>
<div>Jun 28 23:17:06 wings kernel: [ 1016.043006] [<ffffffff8115f492>] vfs_rename+0xc2/0x280</div><div>Jun 28 23:17:06 wings kernel: [ 1016.043091] [<ffffffff8115f92d>] ? __lookup_hash+0x5d/0x160</div><div>
Jun 28 23:17:06 wings kernel: [ 1016.043179] [<ffffffff811611c2>] sys_renameat+0x252/0x280</div>
<div>Jun 28 23:17:06 wings kernel: [ 1016.043267] [<ffffffff8111ee8e>] ? free_pgtables+0xce/0x120</div><div>Jun 28 23:17:06 wings kernel: [ 1016.043357] [<ffffffff812b808b>] ? cpumask_any_but+0x2b/0x40</div>
<div>Jun 28 23:17:06 wings kernel: [ 1016.043448] [<ffffffff81043862>] ? flush_tlb_mm+0x42/0xa0</div><div>Jun 28 23:17:06 wings kernel: [ 1016.043536] [<ffffffff811240fe>] ? remove_vma+0x6e/0x90</div><div>Jun 28 23:17:06 wings kernel: [ 1016.043621] [<ffffffff81126549>] ? do_munmap+0x309/0x3a0</div>
<div>Jun 28 23:17:06 wings kernel: [ 1016.043709] [<ffffffff8116120b>] sys_rename+0x1b/0x20</div><div>Jun 28 23:17:06 wings kernel: [ 1016.043800] [<ffffffff8100a0f2>] system_call_fastpath+0x16/0x1b</div><div>
Jun 28 23:17:06 wings kernel: [ 1016.043889] Code: 44 00 00 49 89 c1 fa 66 0f 1f 44 00 00 65 48 8b 14 25 58 eb 00 00 49 8b 04 24 48 8d 04 02 4c 8b 30 4d 85 f6 74 78 49 63 54 24 18 <49> 8b 14 16 48 89 10 4c 89 cf 57 9d 0f 1f 44 00 00 4d 85 f6 75</div>
<div>Jun 28 23:17:06 wings kernel: [ 1016.044228] RIP [<ffffffff81144164>] __kmalloc_track_caller+0x84/0x1b0</div><div>Jun 28 23:17:06 wings kernel: [ 1016.044324] RSP <ffff88007c701cc8></div><div>Jun 28 23:17:06 wings kernel: [ 1016.044575] ---[ end trace 93a9f0d37546adc3 ]---</div>
<div>Jun 28 23:17:06 wings udevd[396]: worker [542] unexpectedly returned with status 0x000b</div><div>Jun 28 23:17:06 wings udevd[396]: worker [542] failed while handling '/devices/pci0000:00/0000:00:11.0/0000:02:06.0/host4/target4:0:0/4:0:0:0/block/sdc/sdc1'</div>
<div>Jun 28 23:17:06 wings kernel: [ 1016.067655] JBD: Ignoring recovery information on journal</div><div>Jun 28 23:17:06 wings kernel: [ 1016.088738] general protection fault: 0000 [#2] SMP</div><div>Jun 28 23:17:06 wings kernel: [ 1016.088861] last sysfs file: /sys/devices/pci0000:00/0000:00:11.0/0000:02:06.0/host4/target4:0:0/4:0:0:0/block/sdc/uevent</div>
<div>Jun 28 23:17:06 wings kernel: [ 1016.089022] CPU 0</div><div>Jun 28 23:17:06 wings kernel: [ 1016.089051] Modules linked in: ocfs2 quota_tree ocfs2_dlmfs ocfs2_stack_o2cb ocfs2_dlm ocfs2_nodemanager ocfs2_stackglue configfs acpiphp xt_tcpudp nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack iptable_filter ip_tables x_tables ppdev vmware_balloon psmouse serio_raw parport_pc lp intel_agp shpchp i2c_piix4 parport floppy mptspi mptscsih e1000 mptbase scsi_transport_spi [last unloaded: configfs]</div>
<div>Jun 28 23:17:06 wings kernel: [ 1016.090189]</div><div>Jun 28 23:17:06 wings kernel: [ 1016.090253] Pid: 1906, comm: mount.ocfs2 Tainted: G D 2.6.35-28-generic #49-Ubuntu 440BX Desktop Reference Platform/VMware Virtual Platform</div>
<div>Jun 28 23:17:06 wings kernel: [ 1016.090429] RIP: 0010:[<ffffffff81144318>] [<ffffffff81144318>] __kmalloc+0x88/0x1c0</div><div>Jun 28 23:17:06 wings kernel: [ 1016.090590] RSP: 0018:ffff88007bfddb68 EFLAGS: 00010086</div>
<div>Jun 28 23:17:06 wings kernel: [ 1016.090678] RAX: ffff880001e128c0 RBX: 00000000000080d0 RCX: ffffffffa033c6c4</div><div>Jun 28 23:17:06 wings kernel: [ 1016.090776] RDX: 0000000000000000 RSI: 00000000000080d0 RDI: 0000000000000004</div>
<div>Jun 28 23:17:06 wings kernel: [ 1016.090875] RBP: ffff88007bfddba8 R08: e018000000000000 R09: 0000000000000246</div><div>Jun 28 23:17:06 wings kernel: [ 1016.090973] R10: ffff880066e5d260 R11: 0000000000000000 R12: ffffffff81a24d20</div>
<div>Jun 28 23:17:06 wings kernel: [ 1016.091072] R13: 000000000000000a R14: e2c3f000ff53f000 R15: 00000000000080d0</div><div>Jun 28 23:17:06 wings kernel: [ 1016.091171] FS: 00007fea25fff720(0000) GS:ffff880001e00000(0000) knlGS:0000000000000000</div>
<div>Jun 28 23:17:06 wings kernel: [ 1016.091313] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b</div><div>Jun 28 23:17:06 wings kernel: [ 1016.091403] CR2: 00007ff94679dec0 CR3: 000000007b4d8000 CR4: 00000000000006f0</div>
<div>Jun 28 23:17:06 wings kernel: [ 1016.091528] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000</div><div>Jun 28 23:17:06 wings kernel: [ 1016.091646] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400</div>
<div>Jun 28 23:17:06 wings kernel: [ 1016.091745] Process mount.ocfs2 (pid: 1906, threadinfo ffff88007bfdc000, task ffff88007b82db80)</div><div>Jun 28 23:17:06 wings kernel: [ 1016.091890] Stack:</div><div>Jun 28 23:17:06 wings kernel: [ 1016.091957] 0000000000000000 ffffffffa033c6c4 0000000000000001 0000000000000000</div>
<div>Jun 28 23:17:06 wings kernel: [ 1016.092112] <0> ffff88007bf6e000 ffff88007bf6e000 ffff88007bfddcf8 ffff88007b4a3c00</div><div>Jun 28 23:17:06 wings kernel: [ 1016.092338] <0> ffff88007bfddc08 ffffffffa033c6c4 ffff88007b82dfd8 ffff88007b82db80</div>
<div>Jun 28 23:17:06 wings kernel: [ 1016.092622] Call Trace:</div><div>Jun 28 23:17:06 wings kernel: [ 1016.092705] [<ffffffffa033c6c4>] ? ocfs2_compute_replay_slots+0x64/0x160 [ocfs2]</div><div>Jun 28 23:17:06 wings kernel: [ 1016.092853] [<ffffffffa033c6c4>] ocfs2_compute_replay_slots+0x64/0x160 [ocfs2]</div>
<div>Jun 28 23:17:06 wings kernel: [ 1016.093000] [<ffffffffa0364bf5>] ocfs2_check_volume+0x115/0x650 [ocfs2]</div><div>Jun 28 23:17:06 wings kernel: [ 1016.093104] [<ffffffffa03651ef>] ? ocfs2_mount_volume+0xbf/0x520 [ocfs2]</div>
<div>Jun 28 23:17:06 wings kernel: [ 1016.093208] [<ffffffffa0365225>] ocfs2_mount_volume+0xf5/0x520 [ocfs2]</div><div>Jun 28 23:17:06 wings kernel: [ 1016.093312] [<ffffffffa0368c9b>] ocfs2_fill_super+0x72b/0xd80 [ocfs2]</div>
<div>Jun 28 23:17:06 wings kernel: [ 1016.093408] [<ffffffff81156694>] get_sb_bdev+0x194/0x1d0</div><div>Jun 28 23:17:06 wings kernel: [ 1016.093504] [<ffffffffa0368570>] ? ocfs2_fill_super+0x0/0xd80 [ocfs2]</div>
<div>Jun 28 23:17:06 wings kernel: [ 1016.093607] [<ffffffffa03628f8>] ocfs2_get_sb+0x18/0x20 [ocfs2]</div><div>Jun 28 23:17:06 wings kernel: [ 1016.093714] [<ffffffff81155eaa>] vfs_kern_mount+0x8a/0x1f0</div>
<div>Jun 28 23:17:06 wings kernel: [ 1016.093821] [<ffffffff81156082>] do_kern_mount+0x52/0x130</div><div>Jun 28 23:17:06 wings kernel: [ 1016.093928] [<ffffffff8158b733>] ? _lock_kernel+0x53/0xa3</div><div>
Jun 28 23:17:06 wings kernel: [ 1016.094036] [<ffffffff811717fa>] do_mount+0x20a/0x260</div><div>Jun 28 23:17:06 wings kernel: [ 1016.094140] [<ffffffff811718e0>] sys_mount+0x90/0xe0</div><div>Jun 28 23:17:06 wings kernel: [ 1016.094244] [<ffffffff8100a0f2>] system_call_fastpath+0x16/0x1b</div>
<div>Jun 28 23:17:06 wings kernel: [ 1016.094353] Code: 89 c1 fa 66 0f 1f 44 00 00 65 48 8b 14 25 58 eb 00 00 49 8b 04 24 48 8d 04 02 4c 8b 30 4d 85 f6 0f 84 7c 00 00 00 49 63 54 24 18 <49> 8b 14 16 48 89 10 4c 89 cf 57 9d 0f 1f 44 00 00 4d 85 f6 75</div>
<div>Jun 28 23:17:06 wings kernel: [ 1016.095976] RIP [<ffffffff81144318>] __kmalloc+0x88/0x1c0</div><div>Jun 28 23:17:06 wings kernel: [ 1016.096103] RSP <ffff88007bfddb68></div><div>Jun 28 23:17:06 wings kernel: [ 1016.096195] ---[ end trace 93a9f0d37546adc4 ]---</div>
<div><br></div>