[Ocfs2-users] Kernel panic: ocfs2 is very sorry to be fencing this system by panicing

Tue Aug 16 03:19:16 CDT 2005

Hello,

Prompt where to search for the decision of a problem, please.

When rebooting host2 - ocfs2 working very good. But by rebooting host1, on host2 in /var/log/messages "o2net_check_quorum:1468 ERROR: fencing this node because it is connected to a half-quorum of 1 out of 2 nodes which doesn't include the lowest active node 0" and "Kernel panic: ocfs2 is very sorry to be fencing this system by panicing".

Part of /var/log/messages with both nodes in attachment.

-------------- next part --------------
Aug 15 10:39:00 host2 exiting on signal 15
Aug 15 10:39:10 host1 kernel: (0,0):o2net_state_change:512 connection to node host2 num 1 at 10.0.1.2:7777 has been idle for 10 seconds, shutting it down.
Aug 15 10:39:10 host1 kernel: (6584,0):o2net_set_nn_state:414 no longer connected to node host2 at 10.0.1.2:7777
Aug 15 10:39:46 host1 kernel: (7913,1):dlm_send_proxy_ast_msg:428 ERROR: status = -107
Aug 15 10:39:46 host1 kernel: (7913,1):dlm_flush_asts:572 ERROR: status = -107
Aug 15 10:52:50 host1 kernel: (7913,3):dlm_send_proxy_ast_msg:428 ERROR: status = -107
Aug 15 10:52:50 host1 kernel: (7913,3):dlm_flush_asts:572 ERROR: status = -107
Aug 15 10:52:50 host1 kernel: (8021,1):dlm_send_proxy_ast_msg:428 ERROR: status = -107
Aug 15 10:52:50 host1 kernel: (8021,1):dlm_flush_asts:572 ERROR: status = -107
Aug 15 10:52:50 host1 kernel: (1340,2):ocfs2_replay_journal:1123 Recovering node 1 from slot 1 on device (8,16)
Aug 15 10:52:50 host1 kernel: (1341,0):ocfs2_replay_journal:1123 Recovering node 1 from slot 1 on device (8,48)
Aug 15 10:52:51 host1 kernel: kjournald starting.  Commit interval 5 seconds
Aug 15 10:52:51 host1 kernel: kjournald starting.  Commit interval 5 seconds
Aug 15 10:55:50 host2 syslogd 1.4.1: restart.
Aug 15 10:55:55 host2 kernel: OCFS2 Node Manager 0.99.15-SLES Mon Jun 27 17:33:57 PDT 2005 (build sles)
Aug 15 10:55:55 host2 kernel: OCFS2 DLM 0.99.15-SLES Mon Jun 27 17:33:57 PDT 2005 (build sles)
Aug 15 10:55:55 host2 kernel: OCFS2 DLMFS 0.99.15-SLES Mon Jun 27 17:33:57 PDT 2005 (build sles)
Aug 15 10:55:55 host2 kernel: OCFS2 User DLM kernel interface loaded
Aug 15 10:56:00 host1 kernel: (6584,0):o2net_set_nn_state:431 accepted connection from node host2 num 1 at 10.0.1.2:7777
Aug 15 10:56:02 host2 kernel: (6638,0):o2net_set_nn_state:431 connected to node host1 num 0 at 10.0.1.1:7777
Aug 15 10:56:04 host1 kernel: (6584,0):__dlm_print_nodes:379 Nodes in my domain ("F710595BBF68481A9D08F42DF6FCA92D"):
Aug 15 10:56:04 host1 kernel: (6584,0):__dlm_print_nodes:383  node 0
Aug 15 10:56:04 host1 kernel: (6584,0):__dlm_print_nodes:383  node 1
Aug 15 10:56:06 host2 kernel: OCFS2 0.99.15-SLES Mon Jun 27 17:33:57 PDT 2005 (build sles)
Aug 15 10:56:06 host2 kernel: (7924,3):ocfs2_initialize_osb:1179 max_slots for this device: 2
Aug 15 10:56:07 host2 kernel: (7924,3):ocfs2_fill_local_node_info:851 I am node 1
Aug 15 10:56:07 host2 kernel: (7924,1):__dlm_print_nodes:379 Nodes in my domain ("F710595BBF68481A9D08F42DF6FCA92D"):
Aug 15 10:56:07 host2 kernel: (7924,1):__dlm_print_nodes:383  node 0
Aug 15 10:56:07 host2 kernel: (7924,1):__dlm_print_nodes:383  node 1
Aug 15 10:56:07 host2 kernel: (7924,1):ocfs2_find_slot:266 taking node slot 1
Aug 15 10:56:07 host2 kernel: kjournald starting.  Commit interval 5 seconds
Aug 15 10:56:07 host2 kernel: ocfs2: Mounting device (8,16) on (node 1, slot 1)
Aug 15 10:56:08 host1 kernel: (6584,0):__dlm_print_nodes:379 Nodes in my domain ("D414ADF107F84066A6B32BBD6F25C55E"):
Aug 15 10:56:08 host1 kernel: (6584,0):__dlm_print_nodes:383  node 0
Aug 15 10:56:08 host1 kernel: (6584,0):__dlm_print_nodes:383  node 1
Aug 15 10:56:11 host2 kernel: (8067,3):ocfs2_initialize_osb:1179 max_slots for this device: 4
Aug 15 10:56:11 host2 kernel: (8067,3):ocfs2_fill_local_node_info:851 I am node 1
Aug 15 10:56:11 host2 kernel: (8067,1):__dlm_print_nodes:379 Nodes in my domain ("D414ADF107F84066A6B32BBD6F25C55E"):
Aug 15 10:56:11 host2 kernel: (8067,1):__dlm_print_nodes:383  node 0
Aug 15 10:56:11 host2 kernel: (8067,1):__dlm_print_nodes:383  node 1
Aug 15 10:56:11 host2 kernel: (8067,1):ocfs2_find_slot:266 taking node slot 1
Aug 15 10:56:11 host2 kernel: kjournald starting.  Commit interval 5 seconds
Aug 15 10:56:11 host2 kernel: ocfs2: Mounting device (8,48) on (node 1, slot 1)
Aug 15 10:56:11 host2 logger: (Oracle CSSD will be run out of init)
Aug 15 10:56:11 host2 logger: (Oracle EVMD will be run out of init)
Aug 15 10:56:11 host2 logger: (Oracle CRSD will be run out of init, set to start boot services)

Aug 15 11:06:17 host1 exiting on signal 15
Aug 15 11:06:29 host2 kernel: (0,0):o2net_state_change:512 connection to node host1 num 0 at 10.0.1.1:7777 has been idle for 10 seconds, shutting it down.
Aug 15 11:06:29 host2 kernel: (6638,0):o2net_set_nn_state:414 no longer connected to node host1 at 10.0.1.1:7777
Aug 15 11:06:47 host2 kernel: (6638,0):o2net_check_quorum:1468 ERROR: fencing this node because it is connected to a half-quorum of 1 out of 2 nodes which doesn't include the lowest active node 0
Aug 15 11:06:47 host2 kernel: (6638,0):o2hb_stop_all_regions:1589 ERROR: stopping heartbeat on all active regions.
Aug 15 11:06:47 host2 kernel: Kernel panic: ocfs2 is very sorry to be fencing this system by panicing
Aug 15 11:06:47 host2 kernel:
Aug 15 11:10:38 host1 syslogd 1.4.1: restart.
Aug 15 11:10:43 host1 kernel: OCFS2 Node Manager 0.99.15-SLES Mon Jun 27 17:33:57 PDT 2005 (build sles)
Aug 15 11:10:43 host1 kernel: OCFS2 DLM 0.99.15-SLES Mon Jun 27 17:33:57 PDT 2005 (build sles)
Aug 15 11:10:43 host1 kernel: OCFS2 DLMFS 0.99.15-SLES Mon Jun 27 17:33:57 PDT 2005 (build sles)
Aug 15 11:10:43 host1 kernel: OCFS2 User DLM kernel interface loaded
Aug 15 11:10:53 host1 kernel: OCFS2 0.99.15-SLES Mon Jun 27 17:33:57 PDT 2005 (build sles)
Aug 15 11:10:53 host1 kernel: (7664,3):ocfs2_initialize_osb:1179 max_slots for this device: 2
Aug 15 11:10:53 host1 kernel: (7664,3):ocfs2_fill_local_node_info:851 I am node 0
Aug 15 11:10:53 host1 kernel: (7664,1):__dlm_print_nodes:379 Nodes in my domain ("F710595BBF68481A9D08F42DF6FCA92D"):
Aug 15 11:10:53 host1 kernel: (7664,1):__dlm_print_nodes:383  node 0
Aug 15 11:10:53 host1 kernel: (7664,1):ocfs2_find_slot:266 taking node slot 0
Aug 15 11:10:53 host1 kernel: kjournald starting.  Commit interval 5 seconds
Aug 15 11:10:53 host1 kernel: ocfs2: Mounting device (8,16) on (node 0, slot 0)
Aug 15 11:10:57 host1 kernel: (7925,3):ocfs2_initialize_osb:1179 max_slots for this device: 4
Aug 15 11:10:57 host1 kernel: (7925,3):ocfs2_fill_local_node_info:851 I am node 0
Aug 15 11:10:57 host1 kernel: (7925,1):__dlm_print_nodes:379 Nodes in my domain ("D414ADF107F84066A6B32BBD6F25C55E"):
Aug 15 11:10:57 host1 kernel: (7925,1):__dlm_print_nodes:383  node 0
Aug 15 11:10:57 host1 kernel: (7925,3):ocfs2_find_slot:266 taking node slot 0
Aug 15 11:10:57 host1 kernel: kjournald starting.  Commit interval 5 seconds
Aug 15 11:10:57 host1 kernel: ocfs2: Mounting device (8,48) on (node 0, slot 0)
Aug 15 11:10:57 host1 logger: (Oracle CSSD will be run out of init)
Aug 15 11:10:57 host1 logger: (Oracle EVMD will be run out of init)
Aug 15 11:10:57 host1 logger: (Oracle CRSD will be run out of init, set to start boot services)
Aug 15 11:14:03 host2 syslogd 1.4.1: restart.
Aug 15 11:14:09 host2 kernel: OCFS2 Node Manager 0.99.15-SLES Mon Jun 27 17:33:57 PDT 2005 (build sles)
Aug 15 11:14:09 host2 kernel: OCFS2 DLM 0.99.15-SLES Mon Jun 27 17:33:57 PDT 2005 (build sles)
Aug 15 11:14:09 host2 kernel: OCFS2 DLMFS 0.99.15-SLES Mon Jun 27 17:33:57 PDT 2005 (build sles)
Aug 15 11:14:09 host2 kernel: OCFS2 User DLM kernel interface loaded
Aug 15 11:14:14 host1 kernel: (6644,0):o2net_set_nn_state:431 accepted connection from node host2 num 1 at 10.0.1.2:7777
Aug 15 11:14:16 host2 kernel: (6643,0):o2net_set_nn_state:431 connected to node host1 num 0 at 10.0.1.1:7777
Aug 15 11:14:18 host1 kernel: (6644,0):__dlm_print_nodes:379 Nodes in my domain ("F710595BBF68481A9D08F42DF6FCA92D"):
Aug 15 11:14:18 host1 kernel: (6644,0):__dlm_print_nodes:383  node 0
Aug 15 11:14:18 host1 kernel: (6644,0):__dlm_print_nodes:383  node 1
Aug 15 11:14:20 host2 kernel: OCFS2 0.99.15-SLES Mon Jun 27 17:33:57 PDT 2005 (build sles)
Aug 15 11:14:20 host2 kernel: (7920,2):ocfs2_initialize_osb:1179 max_slots for this device: 2
Aug 15 11:14:20 host2 kernel: (7920,2):ocfs2_fill_local_node_info:851 I am node 1
Aug 15 11:14:20 host2 kernel: (7920,2):__dlm_print_nodes:379 Nodes in my domain ("F710595BBF68481A9D08F42DF6FCA92D"):
Aug 15 11:14:20 host2 kernel: (7920,2):__dlm_print_nodes:383  node 0
Aug 15 11:14:20 host2 kernel: (7920,2):__dlm_print_nodes:383  node 1
Aug 15 11:14:20 host2 kernel: (7920,2):ocfs2_find_slot:266 taking node slot 1
Aug 15 11:14:20 host2 kernel: kjournald starting.  Commit interval 5 seconds
Aug 15 11:14:20 host2 kernel: ocfs2: Mounting device (8,16) on (node 1, slot 1)
Aug 15 11:14:22 host1 kernel: (6644,0):__dlm_print_nodes:379 Nodes in my domain ("D414ADF107F84066A6B32BBD6F25C55E"):
Aug 15 11:14:22 host1 kernel: (6644,0):__dlm_print_nodes:383  node 0
Aug 15 11:14:22 host1 kernel: (6644,0):__dlm_print_nodes:383  node 1
Aug 15 11:14:24 host2 kernel: (8069,3):ocfs2_initialize_osb:1179 max_slots for this device: 4
Aug 15 11:14:24 host2 kernel: (8069,3):ocfs2_fill_local_node_info:851 I am node 1
Aug 15 11:14:24 host2 kernel: (8069,1):__dlm_print_nodes:379 Nodes in my domain ("D414ADF107F84066A6B32BBD6F25C55E"):
Aug 15 11:14:24 host2 kernel: (8069,1):__dlm_print_nodes:383  node 0
Aug 15 11:14:24 host2 kernel: (8069,1):__dlm_print_nodes:383  node 1
Aug 15 11:14:24 host2 kernel: (8069,1):ocfs2_find_slot:266 taking node slot 1
Aug 15 11:14:24 host2 kernel: kjournald starting.  Commit interval 5 seconds
Aug 15 11:14:24 host2 kernel: ocfs2: Mounting device (8,48) on (node 1, slot 1)
Aug 15 11:14:24 host2 logger: (Oracle CSSD will be run out of init)
Aug 15 11:14:24 host2 logger: (Oracle EVMD will be run out of init)
Aug 15 11:14:24 host2 logger: (Oracle CRSD will be run out of init, set to start boot services)