[Olt-users] Failures in rac-st-asmlib-mempressure due to ASM start failure

[Hayden,Robert]

I am running through the OLT for the first time and have ran into an issue with some rac tests, in particular the rac-st-asmlib-mempressure and 
rac-st-asmlib-memleak2.  Each of these testcases have three sub-tests and the failure may occur in any one of the three sub-tests.  The failures occur after the DBT2 completes and when ASM is restarted in the validation stage.  The ASM start returns an ORA-3113 to the OLT which in turns causes the testcase to fail.

I am looking for advice on whether this is a known issue (maybe seen in bugzilla number 3450) with OLT or if I need to log an SR with Oracle support.  Searching Metalink, I have found a close match with bug 5659909.

Here is the rac_tuning.tlg output from which you can see that the DBT2 test completes, and the ASM startup issue occurs in the validation stage.

INFO:     12:29:25 : f_dbt2execute(oltdbt2-executetest) Invoking the DBT2 rac test
INFO:     12:57:05 : f_dbt2execute For automated distructive test now failure will happen
INFO:     12:57:05 : f_dbt2execute For manual distructive tests manually do the failure part
INFO:     16:57:35 : f_dbt2execute (oltdbt2-executetest) DBT2 rac test completed
INFO:     16:57:35 : f_stopdb(oltdbt2-executetest) Going to stop Database instance on ipht03
INFO:     16:58:03 : f_stopdb(oltdbt2-executetest) Going to stop Database instance on ipht04
INFO:     16:58:29 : f_stop_asm (oltdbt2-asm) Stopping ASM Instance on ipht03
INFO:     16:58:34 : f_stop_asm (oltdbt2-asm) Stopping ASM Instance on ipht04
INFO:     16:58:39 : f_stoplsnr(oltdbt2-executetest) Going to stop Listener on ipht03
INFO:     16:58:45 : f_stoplsnr(oltdbt2-executetest) Going to stop Listener on ipht04
INFO:     16:58:45 : f_stopqa (oltdbt2-qamonitor) Stopping qamonitor on ipht04
INFO:     16:58:47 : f_validate(oltdbt2-results) Validating the run
INFO:     16:58:47 : f_crsctrlmain(oltdbt2-executetest) Executing CRS - start
INFO:     16:58:50 : f_crsctrlmain(oltdbt2-executetest) Completed CRS - start
INFO:     16:58:50 : f_start_asm (oltdbt2-asm) Starting ASM Instance on ipht03
ERROR:    16:58:55: OULT_ERR_50:Unable to start ASM instance:f_start_asm(oltdbt2-executetest) Unable to start ASM instance on node 3
INFO:     16:58:55 : f_stopdb(oltdbt2-executetest) Going to stop Database instance on ipht03
INFO:     16:58:55 : f_stopdb(oltdbt2-executetest) Going to stop Database instance on ipht04
INFO:     16:58:56 : f_stop_asm (oltdbt2-asm) Stopping ASM Instance on ipht03
INFO:     16:58:56 : f_stop_asm (oltdbt2-asm) Stopping ASM Instance on ipht04
INFO:     16:58:56 : f_stop_asm (oltdbt2-asm) Stopping ASM Instance on ipht03
INFO:     16:58:56 : f_stop_asm (oltdbt2-asm) Stopping ASM Instance on ipht04
INFO:     16:58:56 : f_stoplsnr(oltdbt2-executetest) Going to stop Listener on ipht03
INFO:     16:58:56 : f_stoplsnr(oltdbt2-executetest) Going to stop Listener on ipht04
INFO:     16:58:57 : f_clean_devshm(oltdbt2-results) Cleaning up /dev/shm
INFO:     16:58:57 : f_analyzetest(oltdbt2-results) Analyzing the test 
INFO:     16:58:57 : f_analyzelogs(oltdbt2-results) Analyzing logs 
INFO:     16:59:00 : f_master_relieve (oltdbt2-nodesync) Relieving all the RAC nodes


Looking through the log files, I am finding that the LMON process terminates the ASM instance during ASM startup of the first node in a 2 node cluster in the validation stage with the following:

-----------start cut------------
*** 2008-03-11 16:58:50.973
kjxggpoll: change poll time to 600 ms
GES Client Freeze unsuccessful- retrying...
Process 27403(LMS) not frozen

<lines skipped>

  GRANTED_Q: 
   DEFER MSG QUEUE ON LMS0 IS EMPTY
   SEQUENCES: Terminating instance due to freeze timeout (lms0 not frozen)

-------------end cut--------------------

I looked through the SYSLOGs and did not see any issues with regards to memory being exhausted (13 GB free) at the time of the ASM startup.

Any advice would be appreciated.

Thanks
Robert


----------------------------------------------------------------------
CONFIDENTIALITY NOTICE This message and any included attachments are from Cerner Corporation and are intended only for the addressee. The information contained in this message is confidential and may constitute inside or non-public information under international, federal, or state securities laws. Unauthorized forwarding, printing, copying, distribution, or use of such information is strictly prohibited and may be unlawful. If you are not the addressee, please promptly delete this message and notify the sender of the delivery error by e-mail or you may call Cerner's corporate offices in Kansas City, Missouri, U.S.A at (+1) (816)221-1024.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://oss.oracle.com/pipermail/olt-users/attachments/20080312/e83ae4fa/attachment.html