[Ocfs2-users] ocfs2, redhat 2.6.18-238.x kernel and async i/o hang

[srinivas.eeda at oracle.com]

On 07/28/2011 09:20 AM, Matthew Huff wrote:
> I migrated a Oracle 11gR2 database from one HP RHEL5 box to another. The only difference I can tell is that the older box was running 2.16.18-194 kernel, and the newer box is running 2.6.18.238 kernel (I've tried 2.6.18.238.el5, 238.5.1.el5 and 239.19.1.el5). On the new box, any Oracle instance hangs on startup mounting the files if async io is turned on. Direct I/O works file. The files are all on ocfs2 partitions.
>
> Anyone seen this issue? The databases run fine if the spfile has:
We came to know about this recently and are looking into it. Till it is 
resolved please use the older working kernel.
> *.disk_asynch_io=FALSE
> *.filesystemio_options='directio'
>
> But hang 100% of the time (even after a few hours of waiting) if the parameters are set to:
> *.disk_asynch_io=TRUE
> *.filesystemio_options='setall'
>
>
> Other than having libaio rpm loaded, I don't think there are any configuration or knobs I need to turn.
>
>
> Here are some details:
>
>
> [oracle at hubble lib64]$ ldd $ORACLE_HOME/bin/oracle
>          linux-vdso.so.1 =>   (0x00007fff9b7fc000)
>          libodm11.so =>  /opt/oracle/product/11.2.0/dbhome_1/lib/libodm11.so (0x00002b487959a000)
>          libcell11.so =>  /opt/oracle/product/11.2.0/dbhome_1/lib/libcell11.so (0x00002b487969c000)
>          libskgxp11.so =>  /opt/oracle/product/11.2.0/dbhome_1/lib/libskgxp11.so (0x00002b4879800000)
>          librt.so.1 =>  /lib64/librt.so.1 (0x00000032fa600000)
>          libnnz11.so =>  /opt/oracle/product/11.2.0/dbhome_1/lib/libnnz11.so (0x00002b48799ea000)
>          libclsra11.so =>  /opt/oracle/product/11.2.0/dbhome_1/lib/libclsra11.so (0x00002b4879db7000)
>          libdbcfg11.so =>  /opt/oracle/product/11.2.0/dbhome_1/lib/libdbcfg11.so (0x00002b4879ecf000)
>          libhasgen11.so =>  /opt/oracle/product/11.2.0/dbhome_1/lib/libhasgen11.so (0x00002b4879ff2000)
>          libskgxn2.so =>  /opt/oracle/product/11.2.0/dbhome_1/lib/libskgxn2.so (0x00002b487a7c6000)
>          libocr11.so =>  /opt/oracle/product/11.2.0/dbhome_1/lib/libocr11.so (0x00002b487a8c8000)
>          libocrb11.so =>  /opt/oracle/product/11.2.0/dbhome_1/lib/libocrb11.so (0x00002b487aa6e000)
>          libocrutl11.so =>  /opt/oracle/product/11.2.0/dbhome_1/lib/libocrutl11.so (0x00002b487ac0e000)
>          libaio.so.1 =>  /usr/lib64/libaio.so.1 (0x00002b487ad19000)
>          libdl.so.2 =>  /lib64/libdl.so.2 (0x00000032f9a00000)
>          libm.so.6 =>  /lib64/libm.so.6 (0x00000032f9600000)
>          libpthread.so.0 =>  /lib64/libpthread.so.0 (0x00000032f9e00000)
>          libnsl.so.1 =>  /lib64/libnsl.so.1 (0x00000032fc600000)
>          libc.so.6 =>  /lib64/libc.so.6 (0x00000032f9200000)
>          /lib64/ld-linux-x86-64.so.2 (0x00000032f8e00000)
>
> [oracle at hubble lib64]$ rpm -qa | grep aio
> libaio-0.3.106-5
> libaio-devel-0.3.106-5
> libsane-hpaio-1.6.7-6.el5_6.1
> libaio-0.3.106-5
> libaio-devel-0.3.106-5
>
> [oracle at hubble dbs]$ cd /usr/lib64
> [oracle at hubble lib64]$ ls -lF *aio*
> -rw-r--r-- 1 root root 15870 Dec 21  2009 libaio.a
> lrwxrwxrwx 1 root root    15 Jul 23 15:21 libaio.so ->  libaio.so.1.0.1*
> lrwxrwxrwx 1 root root    15 Jul 22 21:59 libaio.so.1 ->  libaio.so.1.0.1*
> -rwxr-xr-x 1 root root  3768 Dec 21  2009 libaio.so.1.0.0*
> -rwxr-xr-x 1 root root  3768 Dec 21  2009 libaio.so.1.0.1*
> -rw-r--r-- 1 root root 92866 Apr  8 09:14 librtkaio.a
>
> [root at hubble ~]# grep ocfs2 /etc/fstab
> LABEL=oracle_logfiles   /u01                    ocfs2   _netdev,datavolume,nointr
> LABEL=oracle_datafiles  /u02                    ocfs2   _netdev,datavolume,nointr
>
> [root at hubble ~]# grep ocfs2 /etc/mtab
> ocfs2_dlmfs /dlm ocfs2_dlmfs rw 0 0
> /dev/cciss/c0d1 /u01 ocfs2 rw,_netdev,datavolume,nointr,heartbeat=local 0 0
> /dev/cciss/c0d2 /u02 ocfs2 rw,_netdev,datavolume,nointr,heartbeat=local 0 0
>
> [oracle at hubble lib64]$ rpm -qa | grep ocfs
> ocfs2console-1.4.4-1.el5
> ocfs2-2.6.18-238.19.1.el5-1.4.7-1.el5
> ocfs2-tools-1.4.4-1.el5
>
> [root at hubble ~]# uname -a
> Linux hubble.ox.com 2.6.18-238.19.1.el5 #1 SMP Sun Jul 10 08:43:41 EDT 2011 x86_64 x86_64 x86_64 GNU/Linux
>
>
> ----
> Matthew Huff             | 1 Manhattanville Rd
> Director of Operations   | Purchase, NY 10577
> OTA Management LLC       | Phone: 914-460-4039
> aim: matthewbhuff        | Fax:   914-460-4139
>
>
>
> _______________________________________________
> Ocfs2-users mailing list
> Ocfs2-users at oss.oracle.com
> http://oss.oracle.com/mailman/listinfo/ocfs2-users