<HTML><HEAD></HEAD>
<BODY dir=ltr>
<DIV dir=ltr>
<DIV style="FONT-SIZE: 12pt; FONT-FAMILY: 'Calibri'; COLOR: #000000">
<DIV>Ok i got log from kernel panic - how to reproduce:</DIV>
<DIV>Install fresh OEL 6.4</DIV>
<DIV> </DIV>
<DIV>[root@OCFS2-1 ~]# mkfs.ocfs2 -N 2 -L MAIL --fs-feature-level=max-features
/dev/sdb1</DIV>
<DIV>mkfs.ocfs2 1.8.0</DIV>
<DIV>Cluster stack: classic o2cb</DIV>
<DIV>Overwriting existing ocfs2 partition.</DIV>
<DIV>Proceed (y/N): Y</DIV>
<DIV>Label: MAIL</DIV>
<DIV>Features: sparse extended-slotmap backup-super unwritten inline-data
strict-journal-super metaecc xattr indexed-dirs usrquota grpquota refcount
discontig-bg</DIV>
<DIV>Block size: 4096 (12 bits)</DIV>
<DIV>Cluster size: 4096 (12 bits)</DIV>
<DIV>Volume size: 17179836416 (4194296 clusters) (4194296 blocks)</DIV>
<DIV>Cluster groups: 131 (tail covers 1016 clusters, rest cover 32256
clusters)</DIV>
<DIV>Extent allocator size: 12582912 (3 groups)</DIV>
<DIV>Journal size: 107372544</DIV>
<DIV>Node slots: 2</DIV>
<DIV>Creating bitmaps: done</DIV>
<DIV>Initializing superblock: done</DIV>
<DIV>Writing system files: done</DIV>
<DIV>Writing superblock: done</DIV>
<DIV>Writing backup superblock: 2 block(s)</DIV>
<DIV>Formatting Journals: done</DIV>
<DIV>Growing extent allocator: done</DIV>
<DIV>Formatting slot map: done</DIV>
<DIV>Formatting quota files: done</DIV>
<DIV>Writing lost+found: done</DIV>
<DIV>mkfs.ocfs2 successful</DIV>
<DIV> </DIV>
<DIV>[root@OCFS2-1 ~]# cat /etc/fstab</DIV>
<DIV>/dev/sdb1
/home
ocfs2
_netdev,defaults,usrquota,noatime,data=writeback,commit=30
0 0</DIV>
<DIV> </DIV>
<DIV>[root@OCFS2-1 ~]# cat /proc/mounts</DIV>
<DIV>/dev/sdb1 /home ocfs2
rw,seclabel,noatime,_netdev,heartbeat=local,nointr,data=writeback,errors=remount-ro,commit=30,usrquota,coherency=full,user_xattr,acl
0 0</DIV>
<DIV> </DIV>
<DIV>i create and run own terror script on both servers in cluster to check
stability of ocfs2:</DIV>
<DIV>[root@OCFS2-2 ~]# cat terror.sh</DIV>
<DIV>#!/bin/bash</DIV>
<DIV>while true</DIV>
<DIV>do</DIV>
<DIV>rm -rf /home/1</DIV>
<DIV>mkdir /home/1</DIV>
<DIV>cp -r /usr /home/1</DIV>
<DIV>rm -rf /home/1</DIV>
<DIV>done</DIV>
<DIV> </DIV>
<DIV>[root@OCFS2-2 ~]# cat terror2.sh</DIV>
<DIV>#!/bin/bash</DIV>
<DIV>while true</DIV>
<DIV>do</DIV>
<DIV>du -sh /home/1</DIV>
<DIV>find /home/1</DIV>
<DIV>sleep 30</DIV>
<DIV>done;</DIV>
<DIV> </DIV>
<DIV>[root@OCFS2-2 ~]# cat terror3.sh</DIV>
<DIV>#!/bin/bash</DIV>
<DIV>while true</DIV>
<DIV>do</DIV>
<DIV>quota -v root</DIV>
<DIV>sleep 30</DIV>
<DIV>done;</DIV>
<DIV> </DIV>
<DIV>After few minutes get kernel panic this is log:</DIV>
<DIV>Mar 19 11:33:01 (rm,2184,0):ocfs2_unlink:951 ERROR: status = -2</DIV>
<DIV>Mar 19 11:33:01 (rm,2304,3):ocfs2_unlink:951 ERROR: status = -39</DIV>
<DIV>Mar 19 11:33:01 (cp,2312,1):ocfs2_check_dir_for_entry:2119 ERROR: status =
-17</DIV>
<DIV>Mar 19 11:33:01 (cp,2312,1):ocfs2_mknod:459 ERROR: status = -17</DIV>
<DIV>Mar 19 11:33:01 (cp,2312,1):ocfs2_create:629 ERROR: status = –17</DIV>
<DIV>Mar 19 11:33:02 OCFS2-1 kernel: (updatedb,3121,1):o2net_send_tcp_msg:961
ERROR: sendmsg returned -13 instead of 96</DIV>
<DIV>Mar 19 11:33:02 OCFS2-1 kernel:
(updatedb,3121,1):dlm_do_master_request:1326 ERROR: status = -13</DIV>
<DIV>Mar 19 11:33:02 OCFS2-1 kernel:
(updatedb,3121,1):dlm_do_master_request:1327 ERROR: unhandled error!</DIV>
<DIV>Mar 19 11:33:02 OCFS2-1 kernel: ------------[ cut here ]------------</DIV>
<DIV>Mar 19 11:33:02 OCFS2-1 kernel: kernel BUG at
fs/ocfs2/dlm/dlmmaster.c:1328!</DIV>
<DIV> </DIV>
<DIV>Message from syslogd@OCFS2-1 at Mar 19 11:33:02 ...</DIV>
<DIV>kernel:------------[ cut here ]------------</DIV>
<DIV>Mar 19 11:33:02 OCFS2-1 kernel: invalid opcode: 0000 [#1] SMP</DIV>
<DIV>Mar 19 11:33:02 OCFS2-1 kernel: CPU 1</DIV>
<DIV>Mar 19 11:33:02 OCFS2-1 kernel: Modules linked in: ocfs2 autofs4
ocfs2_dlmfs ocfs2_stack_o2cb ocfs2_dlm ocfs2_nodemanager ocfs2_stackglue
configfs sunrpc ipv6 ppdev e1000 microcode vmw_balloon parport_pc parport pcspkr
sg i2c_piix4 i2c_core shpchp ext4 mbcache jbd2 sd_mod crc_t10dif sr_mod cdrom
mptspi mptscsih mptbase scsi_transport_spi pata_acpi ata_generic ata_piix floppy
dm_mirror dm_region_hash dm_log dm_mod [last unloaded: speedstep_lib]</DIV>
<DIV>Mar 19 11:33:02 OCFS2-1 kernel:</DIV>
<DIV> </DIV>
<DIV>Message from syslogd@OCFS2-1 at Mar 19 11:33:02 ...</DIV>
<DIV>kernel:invalid opcode: 0000 [#1] SMP</DIV>
<DIV>Mar 19 11:33:02 OCFS2-1 kernel: Pid: 3121, comm: updatedb Not tainted
2.6.39-400.17.2.el6uek.x86_64 #1 VMware, Inc. VMware Virtual Platform/440BX
Desktop Reference Platform</DIV>
<DIV>Mar 19 11:33:02 OCFS2-1 kernel: RIP: 0010:[<ffffffffa02f9e4b>]
[<ffffffffa02f9e4b>] dlm_do_master_request+0x4bb/0x6f0 [ocfs2_dlm]</DIV>
<DIV>Mar 19 11:33:02 OCFS2-1 kernel: RSP: 0018:ffff8801095015b8 EFLAGS:
00010292</DIV>
<DIV>Mar 19 11:33:02 OCFS2-1 kernel: RAX: 000000000000004a RBX: fffffffffffffff3
RCX: 0000000000000000</DIV>
<DIV>Mar 19 11:33:02 OCFS2-1 kernel: RDX: 0000000000000000 RSI: 0000000000000082
RDI: 0000000000000246</DIV>
<DIV>Mar 19 11:33:02 OCFS2-1 kernel: RBP: ffff8801095016f8 R08: 0000000000000000
R09: ffffffff815649a0</DIV>
<DIV>Mar 19 11:33:02 OCFS2-1 kernel: R10: 656c646e61686e75 R11: 0000000000000001
R12: 0000000000000001</DIV>
<DIV>Mar 19 11:33:02 OCFS2-1 kernel: R13: ffff88006465e540 R14: ffff880133dda800
R15: ffff88006465e5d8</DIV>
<DIV>Mar 19 11:33:02 OCFS2-1 kernel: FS: 00007f4e44beb700(0000)
GS:ffff88013fc80000(0000) knlGS:0000000000000000</DIV>
<DIV>Mar 19 11:33:02 OCFS2-1 kernel: CS: 0010 DS: 0000 ES: 0000 CR0:
0000000080050033</DIV>
<DIV>Mar 19 11:33:02 OCFS2-1 kernel: CR2: 00007f4e44bf4000 CR3: 0000000128445000
CR4: 00000000000006e0</DIV>
<DIV>Mar 19 11:33:02 OCFS2-1 kernel: DR0: 0000000000000000 DR1: 0000000000000000
DR2: 0000000000000000</DIV>
<DIV>Mar 19 11:33:02 OCFS2-1 kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0
DR7: 0000000000000400</DIV>
<DIV>Mar 19 11:33:02 OCFS2-1 kernel: Process updatedb (pid: 3121, threadinfo
ffff880109500000, task ffff880134e7a540)</DIV>
<DIV>Mar 19 11:33:02 OCFS2-1 kernel: Stack:</DIV>
<DIV>Mar 19 11:33:02 OCFS2-1 kernel: fffffffffffffff3 ffffffff8105b3c0
ffff88013707e348 ffff880134e7a540</DIV>
<DIV>Mar 19 11:33:02 OCFS2-1 kernel: ffff880109501638 ffff880109501678
ffff88013707e348 ffff880134e7ab88</DIV>
<DIV>Mar 19 11:33:02 OCFS2-1 kernel: ffff880134e7a540 ffff880134e7ab88
ffff880134e7a540 ffff880134e7ab88</DIV>
<DIV>Mar 19 11:33:02 OCFS2-1 kernel: Call Trace:</DIV>
<DIV>Mar 19 11:33:02 OCFS2-1 kernel: [<ffffffff8105b3c0>] ?
__dequeue_entity+0x30/0x50</DIV>
<DIV>Mar 19 11:33:02 OCFS2-1 kernel: [<ffffffffa02ff19f>]
dlm_get_lock_resource+0x77f/0xe30 [ocfs2_dlm]</DIV>
<DIV>Mar 19 11:33:02 OCFS2-1 kernel: [<ffffffff81116cda>] ?
__rmqueue+0xba/0x4f0</DIV>
<DIV> </DIV>
<DIV>Message from syslogd@OCFS2-1 at Mar 19 11:33:02 ...</DIV>
<DIV>kernel:Stack:</DIV>
<DIV> </DIV>
<DIV>Message from syslogd@OCFS2-1 at Mar 19 11:33:02 ...</DIV>
<DIV>kernel:Call Trace:</DIV>
<DIV>Mar 19 11:33:02 OCFS2-1 kernel: [<ffffffff8115c371>] ?
__kmalloc+0x1f1/0x200</DIV>
<DIV>Mar 19 11:33:02 OCFS2-1 kernel: [<ffffffffa0304334>] ?
dlm_new_lock+0xd4/0x150 [ocfs2_dlm]</DIV>
<DIV>Mar 19 11:33:02 OCFS2-1 kernel: [<ffffffffa0305824>]
dlmlock+0x4b4/0xcc0 [ocfs2_dlm]</DIV>
<DIV>Mar 19 11:33:02 OCFS2-1 kernel: [<ffffffff8136dc20>] ?
scsi_dma_map+0x90/0xc0</DIV>
<DIV>Mar 19 11:33:02 OCFS2-1 kernel: [<ffffffffa016d225>]
o2cb_dlm_lock+0x65/0x90 [ocfs2_stack_o2cb]</DIV>
<DIV>Mar 19 11:33:02 OCFS2-1 kernel: [<ffffffffa016d000>] ?
0xffffffffa016cfff</DIV>
<DIV>Mar 19 11:33:02 OCFS2-1 kernel: [<ffffffffa016d020>] ?
o2dlm_lock_ast_wrapper+0x20/0x20 [ocfs2_stack_o2cb]</DIV>
<DIV>Mar 19 11:33:02 OCFS2-1 kernel: [<ffffffffa0187422>]
ocfs2_dlm_lock+0x42/0x50 [ocfs2_stackglue]</DIV>
<DIV>Mar 19 11:33:02 OCFS2-1 kernel: [<ffffffffa035cdbb>]
__ocfs2_cluster_lock+0x4eb/0x7d0 [ocfs2]</DIV>
<DIV>Mar 19 11:33:02 OCFS2-1 kernel: [<ffffffff81262a0f>] ?
string+0x4f/0xf0</DIV>
<DIV>Mar 19 11:33:02 OCFS2-1 kernel: [<ffffffffa035e8fc>]
ocfs2_open_lock+0xcc/0x1a0 [ocfs2]</DIV>
<DIV>Mar 19 11:33:02 OCFS2-1 kernel: [<ffffffffa036f8d6>] ?
ocfs2_read_locked_inode+0x266/0x5e0 [ocfs2]</DIV>
<DIV>Mar 19 11:33:02 OCFS2-1 kernel: [<ffffffff812026ee>] ?
security_inode_alloc+0x1e/0x20</DIV>
<DIV>Mar 19 11:33:02 OCFS2-1 kernel: [<ffffffffa036f8d6>]
ocfs2_read_locked_inode+0x266/0x5e0 [ocfs2]</DIV>
<DIV>Mar 19 11:33:02 OCFS2-1 kernel: [<ffffffffa036fe03>]
ocfs2_iget+0x1b3/0x2b0 [ocfs2]</DIV>
<DIV>Mar 19 11:33:02 OCFS2-1 kernel: [<ffffffffa03591ab>] ?
ocfs2_lookup_ino_from_name+0x4b/0x60 [ocfs2]</DIV>
<DIV>Mar 19 11:33:02 OCFS2-1 kernel: [<ffffffffa037ad88>]
ocfs2_lookup+0xc8/0x340 [ocfs2]</DIV>
<DIV>Mar 19 11:33:02 OCFS2-1 kernel: [<ffffffff8117c485>]
d_alloc_and_lookup+0x45/0x90</DIV>
<DIV>Mar 19 11:33:02 OCFS2-1 kernel: [<ffffffff81187ce5>] ?
d_lookup+0x35/0x60</DIV>
<DIV>Mar 19 11:33:02 OCFS2-1 kernel: [<ffffffff8117e2d0>]
do_lookup+0x2e0/0x330</DIV>
<DIV>Mar 19 11:33:02 OCFS2-1 kernel: [<ffffffff81180724>]
path_lookupat+0x134/0x680</DIV>
<DIV>Mar 19 11:33:02 OCFS2-1 kernel: [<ffffffff81205a6a>] ?
avc_has_perm_flags+0x7a/0xa0</DIV>
<DIV>Mar 19 11:33:02 OCFS2-1 kernel: [<ffffffff81180ca1>]
do_path_lookup+0x31/0xc0</DIV>
<DIV>Mar 19 11:33:02 OCFS2-1 kernel: [<ffffffff81181979>]
user_path_at_empty+0x59/0xa0</DIV>
<DIV>Mar 19 11:33:02 OCFS2-1 kernel: [<ffffffff81206576>] ?
selinux_inode_getattr+0x36/0x40</DIV>
<DIV>Mar 19 11:33:02 OCFS2-1 kernel: [<ffffffff81176318>] ?
cp_new_stat+0xf8/0x110</DIV>
<DIV>Mar 19 11:33:02 OCFS2-1 kernel: [<ffffffff811819d1>]
user_path_at+0x11/0x20</DIV>
<DIV>Mar 19 11:33:02 OCFS2-1 kernel: [<ffffffff811763fc>]
vfs_fstatat+0x4c/0x90</DIV>
<DIV>Mar 19 11:33:02 OCFS2-1 kernel: [<ffffffff811764ae>]
vfs_lstat+0x1e/0x20</DIV>
<DIV>Mar 19 11:33:02 OCFS2-1 kernel: [<ffffffff811764d4>]
sys_newlstat+0x24/0x50</DIV>
<DIV>Mar 19 11:33:02 OCFS2-1 kernel: [<ffffffff810cfc47>] ?
audit_syscall_entry+0x1d7/0x200</DIV>
<DIV>Mar 19 11:33:02 OCFS2-1 kernel: [<ffffffff810cf98b>] ?
audit_syscall_exit+0x25b/0x290</DIV>
<DIV>Mar 19 11:33:02 OCFS2-1 kernel: [<ffffffff81517082>]
system_call_fastpath+0x16/0x1b</DIV>
<DIV>Mar 19 11:33:02 OCFS2-1 kernel: Code: 74 70 81 fb 00 fe ff ff 74 68 83 fb
e4 74 63 48 b8 40 02 00 00 00 00 00 10 48 85 05 e0 c9 fb ff 74 09 48 85 05 f7 e8
fb ff 74 04 <0f> 0b eb fe 65 48 8b 34 25 00 c4 00 00 8b 96 94 04 00 00 48
63</DIV>
<DIV>Mar 19 11:33:02 OCFS2-1 kernel: RIP [<ffffffffa02f9e4b>]
dlm_do_master_request+0x4bb/0x6f0 [ocfs2_dlm]</DIV>
<DIV>Mar 19 11:33:02 OCFS2-1 kernel: RSP <ffff8801095015b8></DIV>
<DIV>Mar 19 11:33:02 OCFS2-1 kernel: ---[ end trace 28004c7608480bce ]---</DIV>
<DIV>Mar 19 11:33:02 OCFS2-1 kernel: Kernel panic - not syncing: Fatal
exception</DIV>
<DIV> </DIV>
<DIV>Message from syslogd@OCFS2-1 at Mar 19 11:33:02 ...</DIV>
<DIV>kernel:Code: 74 70 81 fb 00 fe ff ff 74 68 83 fb e4 74 63 48 b8 40 02 00 00
00 00 00 10 48 85 05 e0 c9 fb ff 74 09 48 85 05 f7 e8 fb ff 74 04 <0f> 0b
eb fe 65 48 8b 34 25 00 c4 00 00 8b 96 94 04 00 00 48 63</DIV>
<DIV> </DIV>
<DIV>Message from syslogd@OCFS2-1 at Mar 19 11:33:02 ...</DIV>
<DIV>kernel:Kernel panic - not syncing: Fatal exception</DIV>
<DIV>Mar 19 11:33:02 OCFS2-1 kernel: Pid: 3121, comm: updatedb Tainted:
G D
2.6.39-400.17.2.el6uek.x86_64 #1</DIV>
<DIV>Mar 19 11:33:02 OCFS2-1 kernel: Call Trace:</DIV>
<DIV>Mar 19 11:33:02 OCFS2-1 kernel: [<ffffffff8150bd0b>]
panic+0x91/0x1a8</DIV>
<DIV>Mar 19 11:33:02 OCFS2-1 kernel: [<ffffffff8150fe1c>]
oops_end+0xdc/0xf0</DIV>
<DIV>Mar 19 11:33:02 OCFS2-1 kernel: [<ffffffff8101870b>]
die+0x5b/0x90</DIV>
<DIV>Mar 19 11:33:02 OCFS2-1 kernel: [<ffffffff8150f984>]
do_trap+0xc4/0x170</DIV>
<DIV>Mar 19 11:33:02 OCFS2-1 kernel: [<ffffffff81016665>]
do_invalid_op+0x95/0xb0</DIV>
<DIV>Mar 19 11:33:02 OCFS2-1 kernel: [<ffffffffa02f9e4b>] ?
dlm_do_master_request+0x4bb/0x6f0 [ocfs2_dlm]</DIV>
<DIV>Mar 19 11:33:02 OCFS2-1 kernel: [<ffffffff8151803b>]
invalid_op+0x1b/0x20</DIV>
<DIV>Mar 19 11:33:02 OCFS2-1 kernel: [<ffffffffa02f9e4b>] ?
dlm_do_master_request+0x4bb/0x6f0 [ocfs2_dlm]</DIV>
<DIV>Mar 19 11:33:02 OCFS2-1 kernel: [<ffffffffa02f9eea>] ?
dlm_do_master_request+0x55a/0x6f0 [ocfs2_dlm]</DIV>
<DIV>Mar 19 11:33:02 OCFS2-1 kernel: [<ffffffff8105b3c0>] ?
__dequeue_entity+0x30/0x50</DIV>
<DIV>Mar 19 11:33:02 OCFS2-1 kernel: [<ffffffffa02ff19f>]
dlm_get_lock_resource+0x77f/0xe30 [ocfs2_dlm]</DIV>
<DIV>Mar 19 11:33:02 OCFS2-1 kernel: [<ffffffff81116cda>] ?
__rmqueue+0xba/0x4f0</DIV>
<DIV>Mar 19 11:33:02 OCFS2-1 kernel: [<ffffffff8115c371>] ?
__kmalloc+0x1f1/0x200</DIV>
<DIV>Mar 19 11:33:02 OCFS2-1 kernel: [<ffffffffa0304334>] ?
dlm_new_lock+0xd4/0x150 [ocfs2_dlm]</DIV>
<DIV>Mar 19 11:33:02 OCFS2-1 kernel: [<ffffffffa0305824>]
dlmlock+0x4b4/0xcc0 [ocfs2_dlm]</DIV>
<DIV>Mar 19 11:33:02 OCFS2-1 kernel: [<ffffffff8136dc20>] ?
scsi_dma_map+0x90/0xc0</DIV>
<DIV>Mar 19 11:33:02 OCFS2-1 kernel: [<ffffffffa016d225>]
o2cb_dlm_lock+0x65/0x90 [ocfs2_stack_o2cb]</DIV>
<DIV>Mar 19 11:33:02 OCFS2-1 kernel: [<ffffffffa016d000>] ?
0xffffffffa016cfff</DIV>
<DIV>Mar 19 11:33:02 OCFS2-1 kernel: [<ffffffffa016d020>] ?
o2dlm_lock_ast_wrapper+0x20/0x20 [ocfs2_stack_o2cb]</DIV>
<DIV>Mar 19 11:33:02 OCFS2-1 kernel: [<ffffffffa0187422>]
ocfs2_dlm_lock+0x42/0x50 [ocfs2_stackglue]</DIV>
<DIV>Mar 19 11:33:02 OCFS2-1 kernel: [<ffffffffa035cdbb>]
__ocfs2_cluster_lock+0x4eb/0x7d0 [ocfs2]</DIV>
<DIV>Mar 19 11:33:02 OCFS2-1 kernel: [<ffffffff81262a0f>] ?
string+0x4f/0xf0</DIV>
<DIV>Mar 19 11:33:02 OCFS2-1 kernel: [<ffffffffa035e8fc>]
ocfs2_open_lock+0xcc/0x1a0 [ocfs2]</DIV>
<DIV>Mar 19 11:33:02 OCFS2-1 kernel: [<ffffffffa036f8d6>] ?
ocfs2_read_locked_inode+0x266/0x5e0 [ocfs2]</DIV>
<DIV>Mar 19 11:33:02 OCFS2-1 kernel: [<ffffffff812026ee>] ?
security_inode_alloc+0x1e/0x20</DIV>
<DIV>Mar 19 11:33:02 OCFS2-1 kernel: [<ffffffffa036f8d6>]
ocfs2_read_locked_inode+0x266/0x5e0 [ocfs2]</DIV>
<DIV> </DIV></DIV></DIV></BODY></HTML>