### RUNNING '000-save_sysinfo' ### ========== /proc/buddyinfo: ========== ========== /proc/cgroups: ========== ========== /proc/cmdline: ========== ========== /proc/cpuinfo: ========== ========== /proc/crypto: ========== ========== /proc/devices: ========== ========== /proc/diskstats: ========== ========== /proc/dma: ========== ========== /proc/execdomains: ========== ========== /proc/fb: ========== ========== /proc/filesystems: ========== ========== /proc/hpqci: ========== ========== /proc/interrupts: ========== ========== /proc/iomem: ========== ========== /proc/ioports: ========== ========== /proc/keys: ========== ========== /proc/key-users: ========== ========== /proc/latency_stats: ========== ========== /proc/loadavg: ========== ========== /proc/locks: ========== ========== /proc/mdstat: ========== ========== /proc/meminfo: ========== ========== /proc/misc: ========== ========== /proc/modules: ========== ========== /proc/mounts: ========== ========== /proc/mtrr: ========== ========== /proc/pagetypeinfo: ========== ========== /proc/partitions: ========== ========== /proc/sched_debug: ========== ========== /proc/schedstat: ========== ========== /proc/slabinfo: ========== ========== /proc/softirqs: ========== ========== /proc/stat: ========== ========== /proc/swaps: ========== ========== /proc/timer_list: ========== ========== /proc/timer_stats: ========== ========== /proc/uptime: ========== ========== /proc/version: ========== ========== /proc/vmallocinfo: ========== ========== /proc/vmstat: ========== ========== /proc/zoneinfo: ========== ========== /proc/config.gz: ========== proc files dumped ### RUNNING '010-hpilo_try_build' ### hpilo_try_build: already have hpilof.ko for kernel version 2.6.31 ### RUNNING '020-kernel-tuning.sh' ### run kernel-tuning.sh: change IO scheduler: noop anticipatory [deadline] cfq ### RUNNING '030-power-tuning.sh' ### run power-tuning.sh: kill pcscd: pcscd: no process killed set dirty writeback to 15 seconds: use power-aware CPU scheduler: disable nmi_watchdog: power-tuning.sh fini ### RUNNING '040-install_packages' ### ca-ostest293: Attempting to install packages... Installing ltp-tstres... ltp-tstres not found, attempting to lookup /cruc/packages/ltp-tstres found, attempting to get latest pkg_dir: /cruc/packages Installing 'ltp-tstres' 'ltp-tstres-001.tar.gz' source /cruc/packages/package.fns source /cruc/packages/ltp-tstres/ltp-tstres.fns action_unpack /cruc/packages/ltp-tstres/ltp-tstres-001.tar.gz /usr/builds ltp-tstres-001 >> /cruc/runs/6629/logs/ca-ostest293/ltp-tstres-001.log action_patch action_configure /usr/builds/ltp-tstres-001 action_build action_check action_install ### RUNNING '050-powertop_start' ### ### RUNNING '060-hp_read_power' ### ### RUNNING '070-runtest_oltest_power' ### oltest 0 INFO : BEGIN PACKAGE oltest oltest 0 INFO : Running oltest on SUT: ca-ostest293 Cleaning process containing - ora_ sqlplus make olt-archive-testlogs olt-execute run-test runtest gzip-scp.sh auto.sh shtriage status-listener iostat vmstat mpstat sar test-syncer sleep olt-synctestcases opmn httpd tns stressdb apache lockfile find diagnostic fifo tnslsnr rac-st rac-node rac-dest olt-check-schedule dest- mempressure nfs-hang nfs_hang dbt2 dbt3 oci oracledbt2 oradbt driver lsnr listener attach strace install silent bufferhead fio sar sadc aio_writetest customtest xm-stress xm-dom-test sv-ovs ... Cleaning up ipcs ... Cleaning up tests completed insmod: error inserting '/usr/builds/hpilof-modules/2.6.31/hpilof.ko': -1 File exists hpilof insmod failed Using /cruc/packages/oltest/validate-power.cmd Scheduled command file: validate-power.cmd_ca-ostest293.us.oracle.com! Schedule Log file: /opt/oracle/oltest/olt-schedule/log/validate-power.cmd_ca-ostest293.us.oracle.com.log The suite summary log file (csv format) for all nodes is: /opt/oracle/oltest/olt-schedule/log/olt-summary.csv To get the current the status of scheduled tests: olt-schedule -f -s oltest 0 INFO : using: /usr/bin/olt-schedule -f /cruc/packages/oltest/validate-power.cmd -r oltest 0 INFO : oltest >>> Node : ca-ostest293.us.oracle.com - testsuite "install-silent-oracle-11106" has PASSED >>> Node : ca-ostest293.us.oracle.com - 2 out of 6 testcases in testsuite "ft-aio-dio" have PASSED OLT log results file: cat: /opt/oracle/oltest/olt-schedule/log/validate-power.cmd_ca-ostest293.us.oracle.com.log: No such file or directory ========== --- --- --- Starting Log for (validate-power.cmd_ca-ostest293.us.oracle.com) @ Mon Oct 05 02:42:51 PDT 2009 --- --- --- INFO: (validate-power.cmd_ca-ostest293.us.oracle.com) - Checking if nothing else is running on the remote node ... INFO: (validate-power.cmd_ca-ostest293.us.oracle.com) - [If it hangs here for more than 60secs then tests have been scheduled by someone else using QAMonitor (AE or olt-execute) INFO: (validate-power.cmd_ca-ostest293.us.oracle.com) - I will wait till these tests complete INFO: (validate-power.cmd_ca-ostest293.us.oracle.com) - To find out what is running on the remote box run: olt-get-status -n ca-ostest293.us.oracle.com] INFO: (validate-power.cmd_ca-ostest293.us.oracle.com) - Nothing else is running now. Going to schedule... INFO: (validate-power.cmd_ca-ostest293.us.oracle.com) - Obtain n/w lock: This can hang for a very long time in case of already scheduled long running tests on the node INFO: (validate-power.cmd_ca-ostest293.us.oracle.com) - [If a previous test hung or m/c restarted you may have to clean the lock file manually under /tmp/olt-node_schedule.lck on the remote node] INFO: (validate-power.cmd_ca-ostest293.us.oracle.com) - N/w lock Obtained cat: /make.pid: No such file or directory FORCE_EXECUTE_OPTION = 0 REMOTE_OLT_SCHEDULE_WORKQA_DIR=/home/oracle/work/install-silent-oracle-11106_Oct05_09-02-42-55/olt-schedule_work INFO: (validate-power.cmd_ca-ostest293.us.oracle.com) - Scheduled tests install-silent-oracle-11106 INFO: (validate-power.cmd_ca-ostest293.us.oracle.com) - Getting the pid to poll for test completion INFO: (validate-power.cmd_ca-ostest293.us.oracle.com) - Polling on 2563... echo "Description: Verify if Oracle 11.1.0.6 installs and relinks" Description: Verify if Oracle 11.1.0.6 installs and relinks Executing Test install-silent-oracle-11106 Creating the QA_HOME work dir (/home/oracle/work/install-silent-oracle-11106_Oct05_09-02-42-55/install-silent-oracle-11106_Oct-05-02-42-55-717001908-28081) for test execution start-triage successful Calling /opt/oracle/oltest/olt-schedule/qamonitor/utils/auto.sh /x /home/oracle/work/install-silent-oracle-11106_Oct05_09-02-42-55/install-silent-oracle-11106_Oct-05-02-42-55-717001908-28081 /home/oracle/work/install-silent-oracle-11106_Oct05_09-02-42-55/install-silent-oracle-11106_Oct-05-02-42-55-717001908-28081/results PRODUCT_NAME=oracle:PRODUCT_VERSION=11.1.0.6:UNIX_GROUP_NAME=g502:ORACLE_HOME_NAME=ohome_ora_11106:OUI_TIMEOUT=150:POSTREBOOT=0; 0 tasks running: +++ Task added 1 tasks running: /home/oracle/work/install-silent-oracle-11106_Oct05_09-02-42-55/install-silent-oracle-11106_Oct-05-02-42-55-717001908-28081/results Default runtest not found in path - looking for a custom script - /opt/oracle/oltest/.srchome/si-install/si-install Setting TEST_SCRIPT to /opt/oracle/oltest/.srchome/si-install/si-install with params - /home/oracle/work/install-silent-oracle-11106_Oct05_09-02-42-55/install-silent-oracle-11106_Oct-05-02-42-55-717001908-28081 /home/oracle/work/install-silent-oracle-11106_Oct05_09-02-42-55/install-silent-oracle-11106_Oct-05-02-42-55-717001908-28081/results PRODUCT_NAME=oracle:PRODUCT_VERSION=11.1.0.6:UNIX_GROUP_NAME=g502:ORACLE_HOME_NAME=ohome_ora_11106:OUI_TIMEOUT=150:POSTREBOOT=0 1 tasks running: /home/oracle/work/install-silent-oracle-11106_Oct05_09-02-42-55/install-silent-oracle-11106_Oct-05-02-42-55-717001908-28081/results --- Task removed 0 tasks running: Done runtest /opt/oracle/oltest/olt-schedule/qamonitor/stop-triage: stopped shtriage 0 tasks running: /opt/oracle/oltest/olt-schedule/qamonitor/stop-triage: Copying system logs Warning: Test logs will not be archived. Variable ARCHIVE_TEST_LOG_HOST is not set Enterprise Linux Enterprise Linux Server release 5 (Carthage) Done excuting install-silent-oracle-11106 ; exit 0 FORCE_EXECUTE_OPTION = 0 REMOTE_OLT_SCHEDULE_WORKQA_DIR=/home/oracle/work/ft-aio-dio_Oct05_09-02-48-02/olt-schedule_work INFO: (validate-power.cmd_ca-ostest293.us.oracle.com) - Scheduled tests ft-aio-dio INFO: (validate-power.cmd_ca-ostest293.us.oracle.com) - Getting the pid to poll for test completion INFO: (validate-power.cmd_ca-ostest293.us.oracle.com) - Polling on 7908... echo "Description: Verify combined aio and dio functionality with Oracle ( Can be used to check this across drivers )" Description: Verify combined aio and dio functionality with Oracle ( Can be used to check this across drivers ) make[1]: Entering directory `/home/oracle/work/ft-aio-dio_Oct05_09-02-48-02/olt-schedule_work' Executing Test ft-aio-dio-dbt2_1 Executing Test ft-aio-dio-dbt2_1-attach Configuring the Oracle Home ... Configuring the Oracle Home ... Create the necessary directory structures... Sucessfully created the QA Home - /home/oracle/work/ft-aio-dio_Oct05_09-02-48-02/ft-aio-dio-dbt2_1_Oct-05-02-48-02-112884801-8506 for test oltdbt2 start-triage successful Calling /opt/oracle/oltest/olt-schedule/qamonitor/utils/auto.sh /x /home/oracle/work/ft-aio-dio_Oct05_09-02-48-02/ft-aio-dio-dbt2_1_Oct-05-02-48-02-112884801-8506 /home/oracle/work/ft-aio-dio_Oct05_09-02-48-02/ft-aio-dio-dbt2_1_Oct-05-02-48-02-112884801-8506/work single ORACLE_VERSION=11.1.0.6:DCG=9G:USERS=100:DURATION=300:ODIRECT=true:AIO=true:DBBLOCKSIZE=8192:STORAGETYPE=local:FILESYSTEMLOC=/olt-storage/local:numtests=2:MODE=io:POSTREBOOT=0; 0 tasks running: +++ Task added 1 tasks running: /home/oracle/work/ft-aio-dio_Oct05_09-02-48-02/ft-aio-dio-dbt2_1_Oct-05-02-48-02-112884801-8506/work Using already running instance of QA Monitor Calling /opt/oracle/oltest/olt-schedule/qamonitor/utils/auto.sh /x /home/oracle/work/ft-aio-dio_Oct05_09-02-48-02/ft-aio-dio-dbt2_1-attach_Oct-05-02-48-02-112788721-8506 /home/oracle/work/ft-aio-dio_Oct05_09-02-48-02/ft-aio-dio-dbt2_1-attach_Oct-05-02-48-02-112788721-8506/work aio_dio ORACLE_VERSION=11.1.0.6:numtests=2:FILESYSTEMLOC=/olt-storage/local:FILESYSTEMDETAILS=local:POSTREBOOT=0; 1 tasks running: /home/oracle/work/ft-aio-dio_Oct05_09-02-48-02/ft-aio-dio-dbt2_1_Oct-05-02-48-02-112884801-8506/work +++ Task added 2 tasks running: /home/oracle/work/ft-aio-dio_Oct05_09-02-48-02/ft-aio-dio-dbt2_1_Oct-05-02-48-02-112884801-8506/work /home/oracle/work/ft-aio-dio_Oct05_09-02-48-02/ft-aio-dio-dbt2_1-attach_Oct-05-02-48-02-112788721-8506/work 2 tasks running: /home/oracle/work/ft-aio-dio_Oct05_09-02-48-02/ft-aio-dio-dbt2_1_Oct-05-02-48-02-112884801-8506/work /home/oracle/work/ft-aio-dio_Oct05_09-02-48-02/ft-aio-dio-dbt2_1-attach_Oct-05-02-48-02-112788721-8506/work /home/oracle/work/ft-aio-dio_Oct05_09-02-48-02/ft-aio-dio-dbt2_1_Oct-05-02-48-02-112884801-8506/work--- Task removed 1 tasks running: /home/oracle/work/ft-aio-dio_Oct05_09-02-48-02/ft-aio-dio-dbt2_1_Oct-05-02-48-02-112884801-8506/work Done runtest Warning: Test logs will not be archived. Variable ARCHIVE_TEST_LOG_HOST is not set Enterprise Linux Enterprise Linux Server release 5 (Carthage) Done excuting ft-aio-dio-dbt2_1-attach ; exit 0 Done excuting ft-aio-dio-dbt2_1-attach ; exit 0 1 tasks running: /home/oracle/work/ft-aio-dio_Oct05_09-02-48-02/ft-aio-dio-dbt2_1_Oct-05-02-48-02-112884801-8506/work --- Task removed 0 tasks running: Done runtest /opt/oracle/oltest/olt-schedule/qamonitor/stop-triage: stopped shtriage 0 tasks running: /opt/oracle/oltest/olt-schedule/qamonitor/stop-triage: Copying system logs Warning: Test logs will not be archived. Variable ARCHIVE_TEST_LOG_HOST is not set Enterprise Linux Enterprise Linux Server release 5 (Carthage) Done excuting ft-aio-dio-dbt2_1 ; exit 0 make[1]: Leaving directory `/home/oracle/work/ft-aio-dio_Oct05_09-02-48-02/olt-schedule_work' make[1]: Entering directory `/home/oracle/work/ft-aio-dio_Oct05_09-02-48-02/olt-schedule_work' Executing Test ft-aio-dio-dbt2_2 Executing Test ft-aio-dio-dbt2_2-attach Configuring the Oracle Home ... Configuring the Oracle Home ... Sucessfully created the QA Home - /home/oracle/work/ft-aio-dio_Oct05_09-02-48-02/ft-aio-dio-dbt2_2_Oct-05-03-00-39-450726839-16742 for test oltdbt2 Create the necessary directory structures... start-triage successful Calling /opt/oracle/oltest/olt-schedule/qamonitor/utils/auto.sh /x /home/oracle/work/ft-aio-dio_Oct05_09-02-48-02/ft-aio-dio-dbt2_2_Oct-05-03-00-39-450726839-16742 /home/oracle/work/ft-aio-dio_Oct05_09-02-48-02/ft-aio-dio-dbt2_2_Oct-05-03-00-39-450726839-16742/work single ORACLE_VERSION=11.1.0.6:DCG=9G:USERS=100:DURATION=300:ODIRECT=true:AIO=true:DBBLOCKSIZE=8192:STORAGETYPE=ocfs:FILESYSTEMLOC=/olt-storage/local/ocfs:numtests=2:MODE=io:POSTREBOOT=0; 0 tasks running: +++ Task added 1 tasks running: /home/oracle/work/ft-aio-dio_Oct05_09-02-48-02/ft-aio-dio-dbt2_2_Oct-05-03-00-39-450726839-16742/work 1 tasks running: /home/oracle/work/ft-aio-dio_Oct05_09-02-48-02/ft-aio-dio-dbt2_2_Oct-05-03-00-39-450726839-16742/work --- Task removed 0 tasks running: Done runtest /opt/oracle/oltest/olt-schedule/qamonitor/stop-triage: stopped shtriage 0 tasks running: /opt/oracle/oltest/olt-schedule/qamonitor/stop-triage: Copying system logs Warning: Test logs will not be archived. Variable ARCHIVE_TEST_LOG_HOST is not set Enterprise Linux Enterprise Linux Server release 5 (Carthage) Done excuting ft-aio-dio-dbt2_2 ; exit 0 start-triage successful Calling /opt/oracle/oltest/olt-schedule/qamonitor/utils/auto.sh /x /home/oracle/work/ft-aio-dio_Oct05_09-02-48-02/ft-aio-dio-dbt2_2-attach_Oct-05-03-00-39-463170761-3538 /home/oracle/work/ft-aio-dio_Oct05_09-02-48-02/ft-aio-dio-dbt2_2-attach_Oct-05-03-00-39-463170761-3538/work aio_dio ORACLE_VERSION=11.1.0.6:numtests=2:FILESYSTEMLOC=/olt-storage/local/ocfs:FILESYSTEMDETAILS=ocfs:POSTREBOOT=0; 0 tasks running: +++ Task added 1 tasks running: /home/oracle/work/ft-aio-dio_Oct05_09-02-48-02/ft-aio-dio-dbt2_2-attach_Oct-05-03-00-39-463170761-3538/work 1 tasks running: /home/oracle/work/ft-aio-dio_Oct05_09-02-48-02/ft-aio-dio-dbt2_2-attach_Oct-05-03-00-39-463170761-3538/work --- Task removed 0 tasks running: Done runtest /opt/oracle/oltest/olt-schedule/qamonitor/stop-triage: stopped shtriage 0 tasks running: /opt/oracle/oltest/olt-schedule/qamonitor/stop-triage: Copying system logs Warning: Test logs will not be archived. Variable ARCHIVE_TEST_LOG_HOST is not set Enterprise Linux Enterprise Linux Server release 5 (Carthage) Done excuting ft-aio-dio-dbt2_2-attach ; exit 0 Done excuting ft-aio-dio-dbt2_2-attach ; exit 0 make[1]: Leaving directory `/home/oracle/work/ft-aio-dio_Oct05_09-02-48-02/olt-schedule_work' make[1]: Entering directory `/home/oracle/work/ft-aio-dio_Oct05_09-02-48-02/olt-schedule_work' Executing Test ft-aio-dio-dbt2_3-attach Executing Test ft-aio-dio-dbt2_3 Configuring the Oracle Home ... Create the necessary directory structures... Configuring the Oracle Home ... Sucessfully created the QA Home - /home/oracle/work/ft-aio-dio_Oct05_09-02-48-02/ft-aio-dio-dbt2_3_Oct-05-03-01-52-749405141-25637 for test oltdbt2 start-triage successful Calling /opt/oracle/oltest/olt-schedule/qamonitor/utils/auto.sh /x /home/oracle/work/ft-aio-dio_Oct05_09-02-48-02/ft-aio-dio-dbt2_3-attach_Oct-05-03-01-52-740809143-6981 /home/oracle/work/ft-aio-dio_Oct05_09-02-48-02/ft-aio-dio-dbt2_3-attach_Oct-05-03-01-52-740809143-6981/work aio_dio ORACLE_VERSION=11.1.0.6:numtests=2:FILESYSTEMLOC=/olt-storage/local/nas:FILESYSTEMDETAILS=nas:POSTREBOOT=0; 0 tasks running: +++ Task added 1 tasks running: /home/oracle/work/ft-aio-dio_Oct05_09-02-48-02/ft-aio-dio-dbt2_3-attach_Oct-05-03-01-52-740809143-6981/work 1 tasks running: /home/oracle/work/ft-aio-dio_Oct05_09-02-48-02/ft-aio-dio-dbt2_3-attach_Oct-05-03-01-52-740809143-6981/work --- Task removed 0 tasks running: Done runtest /opt/oracle/oltest/olt-schedule/qamonitor/stop-triage: stopped shtriage 0 tasks running: /opt/oracle/oltest/olt-schedule/qamonitor/stop-triage: Copying system logs Warning: Test logs will not be archived. Variable ARCHIVE_TEST_LOG_HOST is not set Enterprise Linux Enterprise Linux Server release 5 (Carthage) Done excuting ft-aio-dio-dbt2_3-attach ; exit 0 Done excuting ft-aio-dio-dbt2_3-attach ; exit 0 start-triage successful Calling /opt/oracle/oltest/olt-schedule/qamonitor/utils/auto.sh /x /home/oracle/work/ft-aio-dio_Oct05_09-02-48-02/ft-aio-dio-dbt2_3_Oct-05-03-01-52-749405141-25637 /home/oracle/work/ft-aio-dio_Oct05_09-02-48-02/ft-aio-dio-dbt2_3_Oct-05-03-01-52-749405141-25637/work single ORACLE_VERSION=11.1.0.6:DCG=9G:USERS=100:DURATION=300:ODIRECT=true:AIO=true:DBBLOCKSIZE=8192:STORAGETYPE=nas:FILESYSTEMLOC=/olt-storage/local/nas:numtests=2:MODE=io:POSTREBOOT=0; 0 tasks running: +++ Task added 1 tasks running: /home/oracle/work/ft-aio-dio_Oct05_09-02-48-02/ft-aio-dio-dbt2_3_Oct-05-03-01-52-749405141-25637/work 1 tasks running: /home/oracle/work/ft-aio-dio_Oct05_09-02-48-02/ft-aio-dio-dbt2_3_Oct-05-03-01-52-749405141-25637/work --- Task removed 0 tasks running: Done runtest /opt/oracle/oltest/olt-schedule/qamonitor/stop-triage: stopped shtriage 0 tasks running: /opt/oracle/oltest/olt-schedule/qamonitor/stop-triage: Copying system logs Warning: Test logs will not be archived. Variable ARCHIVE_TEST_LOG_HOST is not set Enterprise Linux Enterprise Linux Server release 5 (Carthage) Done excuting ft-aio-dio-dbt2_3 ; exit 0 make[1]: Leaving directory `/home/oracle/work/ft-aio-dio_Oct05_09-02-48-02/olt-schedule_work' INFO: (validate-power.cmd_ca-ostest293.us.oracle.com) - Removing the network schedule lock file now ========== oltest 0 INFO : fini ### RUNNING '080-hp_read_power_kill' ### hp_read_power: no process killed ### RUNNING '090-power-avg.sh' ### power average: 159 with 122 samples ### RUNNING '100-save_sysinfo' ### Warning: Run directory '/cruc/runs/6629/sysinfo/ca-ostest293.1' already exists. Incrementing... ========== /proc/buddyinfo: ========== ========== /proc/cgroups: ========== ========== /proc/cmdline: ========== ========== /proc/cpuinfo: ========== ========== /proc/crypto: ========== ========== /proc/devices: ========== ========== /proc/diskstats: ========== ========== /proc/dma: ========== ========== /proc/execdomains: ========== ========== /proc/fb: ========== ========== /proc/filesystems: ========== ========== /proc/hpqci: ========== ========== /proc/interrupts: ========== ========== /proc/iomem: ========== ========== /proc/ioports: ========== ========== /proc/keys: ========== ========== /proc/key-users: ========== ========== /proc/latency_stats: ========== ========== /proc/loadavg: ========== ========== /proc/locks: ========== ========== /proc/mdstat: ========== ========== /proc/meminfo: ========== ========== /proc/misc: ========== ========== /proc/modules: ========== ========== /proc/mounts: ========== ========== /proc/mtrr: ========== ========== /proc/pagetypeinfo: ========== ========== /proc/partitions: ========== ========== /proc/sched_debug: ========== ========== /proc/schedstat: ========== ========== /proc/slabinfo: ========== ========== /proc/softirqs: ========== ========== /proc/stat: ========== ========== /proc/swaps: ========== ========== /proc/timer_list: ========== ========== /proc/timer_stats: ========== ========== /proc/uptime: ========== ========== /proc/version: ========== ========== /proc/vmallocinfo: ========== ========== /proc/vmstat: ========== ========== /proc/zoneinfo: ========== ========== /proc/config.gz: ========== proc files dumped