[Ocfs-users] ORA-03113 EOF on Comm Channel when using OCFS + LVM + Striping

[John Smiley]

While creating a new Oracle RAC database, I am getting
"ORA-03113 - end-of-file on communication channel"
during the create database or create tablespace phase.
 This only happens if I use OCFS on top of LVM with
striped logical volumes.  Everything works fine if the
LV is not striped.

I have a three-node RAC cluster using a six disk JBOD
connected to each node via a single SCSI bus (daisy
chained).  I would like to use SAME (Stripe And Mirror
Everything) and have been using the striping feature
of LVM to do this.  Here are the configuration
details:

6 36GB SCSI disks as JBOD
1 36GB primary partition on each disk of type 8e (LVM)
1 Volume group consisting of all 6 PVs (total of
~200G) named /dev/oracle
1 LV striped across all 6 PVs with a stripe width of
128k of size 2G (lvcreate -C y -i 6 -I 128 -n demo -L
2g /dev/oracle
OCFS file system built on /dev/oracle/demo with 128k
block size (mkfs.ocfs -b 128 -F -g 502 -u 501 -L /u02
-m /u02 /dev/oracle/demo

oracm and gsd start fine on all nodes, but when I try
to create the database with dbca -datafileDestination
/u02/oradata, it fails with ORA-03113 during database
creation or tablespace creation.  The same error
occurs if I try to create the database manually.

If I use the exact same configuration, but with a
non-striped LV, everything works fine.

I have reproduced these results on RHEL 3 with the
original and errata kernels (2.4.21-4.EL and
2.4.21-4.0.1.EL) and UL 1.0 with the SP2a kernel
(304).  I have even tried upgrading the kernel LVM
modules from the supplied 1.0.5 version to the latest
1.0.8 version from Sistina (patch the kernel with
Sistina's code and re-compile the kernel, install the
new modules and kernel, boot the custom kernel, etc.)

I've tried ocfs 1.0.9-9, 1.0.9-11, and 1.0.9-12.

I have also tried md to create a striped RAID 0 device
instead of using LVM, but got the same results.

Any thoughts on what I'm doing wrong or is there a
problem with using ocfs on striped volumes?

John Smiley
Sr. Database Architect
Sprint Corporation