<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html;charset=UTF-8" http-equiv="Content-Type">
</head>
<body bgcolor="#ffffcc" text="#000066">
<tt>Tristan,<br>
<br>
A diff of the multiple_run.sh is attached.<br>
<br>
Here is some errors I found:<br>
<br>
multiple_run.sh: line 189:
/scratch/mmatsuna/ocfs2test/bin/run_write_append_truncate.py: No such
file or directory <br>
<br>
multiple_run.sh: line 229:
/scratch/mmatsuna/ocfs2test/bin/run_multi_mmap.py: No such file or
directory<br>
<br>
+ /scratch/mmatsuna/ocfs2test/bin/run_create_racer.py -i 40000 -l
/scratch/mmatsuna/ocfs2test/log/create_racer_log/create_racer_test_2008-10-03-14-03.log
-n ca-test14,ca-test13 -p /ocfs2scratch<br>
Oct 03 14:05:50 ca-test14 : run_create_racer: main - current directory
/var/autofs/ca-server1/scratch/mmatsuna/runtest_el5_ia64/bin<br>
Oct 03 14:05:50 ca-test14 : run_create_racer: main - cmd =
/scratch/mmatsuna/ocfs2test/bin/create_racer<br>
Oct 03 14:05:50 ca-test14 : o2tf.StartOpenMPI: Trying to execute
/usr/lib/openmpi/1.2.5-gcc/bin/mpirun with a
simple command remotely among (ca-test14,ca-test13)<br>
Oct 03 14:05:51 ca-test14 : o2tf.mpi_run: MPIRUN =
/usr/lib/openmpi/1.2.5-gcc/bin/mpirun<br>
Oct 03 14:05:51 ca-test14 : o2tf.mpi_run: nproc = 2<br>
Oct 03 14:05:51 ca-test14 : o2tf.mpi_run: nodelen = 2<br>
Traceback (most recent call last):<br>
File "/scratch/mmatsuna/ocfs2test/bin/run_create_racer.py", line 156,
in ?<br>
'NOWAIT')<br>
File
"/var/autofs/ca-server1/scratch/mmatsuna/runtest_el5_ia64/bin/o2tf.py",
line 333, in openmpi_run<br>
printlog('o2tf.mpi_run: shopt = %d' % shopt,<br>
TypeError: int argument required<br>
<br>
<br>
+ /scratch/mmatsuna/ocfs2test/bin/open_delete.py -f
/ocfs2scratch/open_delete_test_file -i 10000 -l
/scratch/mmatsuna/ocfs2test/log/open_delete_log/open_delete_test_2008-10-03-14-03.log
-n ca-test14,ca-test13<br>
Oct 03 14:07:18 ca-test14 : o2tf.StartOpenMPI: Trying to execute
/usr/lib/openmpi/1.2.5-gcc/bin/mpirun with a
simple command remotely among (ca-test14,ca-test13)<br>
Oct 03 14:07:18 ca-test14 : command =
/scratch/mmatsuna/ocfs2test/bin/open_delete<br>
Oct 03 14:07:18 ca-test14 : o2tf.mpi_run: MPIRUN =
/usr/lib/openmpi/1.2.5-gcc/bin/mpirun<br>
Oct 03 14:07:18 ca-test14 : o2tf.mpi_run: nproc = C<br>
Oct 03 14:07:18 ca-test14 : o2tf.mpi_run: nodelen = 2<br>
Traceback (most recent call last):<br>
File "/scratch/mmatsuna/ocfs2test/bin/open_delete.py", line 119, in ?<br>
'NOWAIT')<br>
File
"/var/autofs/ca-server1/scratch/mmatsuna/runtest_el5_ia64/bin/o2tf.py",
line 333, in openmpi_run<br>
printlog('o2tf.mpi_run: shopt = %d' % shopt,<br>
TypeError: int argument required<br>
<br>
<br>
I guess you have to make </tt><tt>"printlog('o2tf.mpi_run: shopt = %d'
% shopt," to be "</tt><tt>printlog('o2tf.mpi_run: shopt = %s' % shopt,".<br>
<br>
One thing that you will see in the diff file, is that I made the label
ocfs2test instead of oracle_home. That's not a good label and may cause
confusion. May let one believe that it has Oracle binaries in it.<br>
</tt>
<pre class="moz-signature" cols="72">Regards,
Marcos Eduardo Matsunaga
Oracle USA
Linux Engineering
“The statements and opinions expressed here are my own and do not
necessarily represent those of Oracle Corporation.”
</pre>
<br>
<br>
Tristan Ye wrote:
<blockquote
cite="mid:1222250067-21254-1-git-send-email-tristan.ye@oracle.com"
type="cite">
<pre wrap="">Like single_run.sh,we also need an automatic launcher for existing multi-nodes testcase
to organize and manipulate these cases,statistic their running result and logs in a graceful way.
Currently,it includes following testcases,
xattr_test
inline_test
write_append_truncate_test
multi_mmap_test
lvb_torture_test
create_racer_test
flock_tests
open_delete_test
Need to run this wrapper script like this:
multiple_run.sh -n node1.us.oracle.com,node2.us.oracle.com -d /dev/sdd1 /storage
All its logs and behaviors kept a unification as single_run.sh did. More testcases were expected
to be integrated as we're going to develop more multi-nodes cases for ocfs2-test.
Signed-off-by: Tristan Ye <a class="moz-txt-link-rfc2396E" href="mailto:tristan.ye@oracle.com"><tristan.ye@oracle.com></a>
---
programs/python_common/multiple_run.sh | 403 ++++++++++++++++++++++++++++++++
1 files changed, 403 insertions(+), 0 deletions(-)
create mode 100755 programs/python_common/multiple_run.sh
diff --git a/programs/python_common/multiple_run.sh b/programs/python_common/multiple_run.sh
new file mode 100755
index 0000000..15e3998
--- /dev/null
+++ b/programs/python_common/multiple_run.sh
@@ -0,0 +1,403 @@
+#!/bin/bash
+# vi: set ts=8 sw=8 autoindent noexpandtab :
+################################################################################
+#
+# File : multiple_run.sh
+#
+# Description: The wrapper script help to organize all multi-nodes testcase
+# to perform a thorough test among multiple nodes.
+#
+#
+# Author: Tristan Ye, <a class="moz-txt-link-abbreviated" href="mailto:tristan.ye@oracle.com">tristan.ye@oracle.com</a>
+#
+# History: 22 Sep 2008
+#
+
+################################################################################
+# Global Variables
+################################################################################
+PATH=$PATH:/sbin # Add /sbin to the path for ocfs2 tools
+export PATH=$PATH:.
+
+USERNAME=`/usr/bin/whoami`
+DATE=`/bin/date +%F-%H-%M`
+DF=`which df`
+GREP=`which grep`
+AWK=`which awk`
+ECHO="`which echo` -e"
+
+SUDO="`which sudo` -u root"
+DEBUGFS_BIN="`which sudo` -u root `which debugfs.ocfs2`"
+TUNEFS_BIN="`which sudo` -u root `which tunefs.ocfs2`"
+MKFS_BIN="`which sudo` -u root `which mkfs.ocfs2`"
+
+REMOTE_MOUNT_BIN="`which sudo` -u root `which remote_mount.py`"
+REMOTE_UMOUNT_BIN="`which sudo` -u root `which remote_umount.py`"
+
+NODE_LIST=
+DEVICE_NAME=
+MOUNT_POINT=
+
+################################################################################
+# Utility Functions
+################################################################################
+f_usage()
+{
+ echo "usage: `basename ${0}` <-n nodes> <-d device> <mountpoint path>"
+ echo " -n nodelist,should be comma separated."
+ echo " -d device name used for ocfs2 volume."
+ echo " <mountpoint path> path of mountpoint where test will be performed."
+ echo
+ echo "Eaxamples:"
+ echo " `basename ${0}` -n node1.us.oracle.com,node2.us.oracle.com -d /dev/sdd1 /storage"
+ exit 1;
+
+}
+
+f_getoptions()
+{
+ if [ $# -eq 0 ]; then
+ f_usage;
+ exit 1
+ fi
+
+ while getopts "n:d:h:" options; do
+ case $options in
+ n ) NODE_LIST="$OPTARG";;
+ d ) DEVICE_NAME="$OPTARG";;
+ h ) f_usage
+ exit 1;;
+ * ) f_usage
+ exit 1;;
+ esac
+ done
+ shift $(($OPTIND -1))
+ MOUNT_POINT=${1}
+
+}
+
+f_setup()
+{
+ f_getoptions $*
+
+ if [ -z "${MOUNT_POINT}" ];then
+ f_usage
+ fi
+
+ . ./config.sh
+
+ LOGFILE=${O2TDIR}/log/single_run_${DATE}.log
+
+ if [ ! -d ${MOUNT_POINT} -o ! -w ${MOUNT_POINT} ]; then
+ ${ECHO} "Mount point ${MOUNT_POINT} does not exist or is not writable" \
+ |tee -a ${LOGFILE};
+ exit 1
+ fi
+
+ ${DF} -h|${GREP} -q ${DEVICE_NAME}
+ if [ "$?" == "0" ];then
+ ${ECHO} "Partition has been mounted,should umount first to perform test" \
+ |tee -a ${LOGFILE};
+ exit 1
+ fi
+
+}
+
+LogRC()
+{
+if [ ${1} -ne 0 ]; then
+ ${ECHO} "Failed." >> ${LOGFILE};
+else
+ ${ECHO} "Passed." >> ${LOGFILE};
+fi;
+END=$(date +%s);
+DIFF=$(( ${END} - ${START} ));
+${ECHO} "Runtime ${DIFF} seconds.\n" >> ${LOGFILE};
+}
+
+LogMsg()
+{
+${ECHO} `date` >> ${LOGFILE};
+${ECHO} "${1}\c" >> ${LOGFILE};
+i=${#1};
+while (( i < 60 ))
+do
+ ${ECHO} ".\c" >> ${LOGFILE};
+ (( ++i ));
+done;
+}
+
+IsDeviceMounted()
+{
+ local OCFS2_DEVICE=${1}
+ ${DF} -h | ${GREP} -q ${OCFS2_DEVICE}
+
+ if [ "$?" == "0" ]; then
+ return 0
+ else
+ return 1
+ fi
+}
+
+run_xattr_test()
+{
+ LogMsg "xattr-test"
+ ${BINDIR}/xattr-multi-run.sh -r 8 -f ${NODE_LIST} -a rsh -o ${O2TDIR}/log/xattr-tests-log -d ${DEVICE_NAME} ${MOUNT_POINT}
+ LogRC $?
+}
+
+run_inline_test()
+{
+ LogMsg "inline-test"
+ ${BINDIR}/multi-inline-run.sh -r 4 -f ${NODE_LIST} -a rsh -o ${O2TDIR}/log/inline-tests-log -d ${DEVICE_NAME} ${MOUNT_POINT}
+ LogRC $?
+
+}
+
+run_write_append_truncate_test()
+{
+ LogMsg "write-append-truncate-test"
+
+ local logdir=${O2TDIR}/log/write_append_truncate_log
+ local logfile=${logdir}/write_append_truncate_${DATE}.log
+
+ local testfile=${MOUNT_POINT}/write_append_truncate_test_file
+
+ ${SUDO} mkdir -p ${logdir}
+ #force to umount volume from all nodes
+ ${ECHO} "Try to umount volume from all nodes before test."|tee -a ${logfile}
+ ${REMOTE_UMOUNT_BIN} -m ${MOUNT_POINT} -n ${NODE_LIST}>>${logfile} 2>&1
+
+
+ CLUSTERSIZE=32k
+ BLOCKSIZE=4k
+ SLOTS=4
+ LABEL=oracle_home
+
+ ${ECHO} "Format volume to launch new test"|tee -a ${logfile}
+ echo y|${MKFS_BIN} -C ${CLUSTERSIZE} -b ${BLOCKSIZE} -N ${SLOTS} -L ${LABEL} ${DEVICE_NAME} || {
+ ${ECHO} "Can not format ${DEVICE_NAME}"
+ return 1
+ }
+
+ ${ECHO} "Mount volume to all nodes"|tee -a ${logfile}
+ ${REMOTE_MOUNT_BIN} -l ${LABEL} -m ${MOUNT_POINT} -n ${NODE_LIST}>>${logfile} 2>&1
+
+ ${BINDIR}/run_write_append_truncate.py -i 20000 -l ${logfile} -n ${NODE_LIST} -f ${testfile}
+
+ LogRC $?
+
+ ${ECHO} "Umount volume from all nodes after test."|tee -a ${logfile}
+ ${REMOTE_UMOUNT_BIN} -m ${MOUNT_POINT} -n ${NODE_LIST}>>${logfile} 2>&1
+}
+
+run_multi_mmap_test()
+{
+ LogMsg "multi-mmap-test"
+
+ local logdir=${O2TDIR}/log/multi_mmap_log
+ local logfile=${logdir}/multi_mmap_test_${DATE}.log
+
+ local testfile=${MOUNT_POINT}/multi_mmap_test_file
+
+ ${SUDO} mkdir -p ${logdir}
+ #force to umount volume from all nodes
+ ${ECHO} "Try to umount volume from all nodes before test."|tee -a ${logfile}
+ ${REMOTE_UMOUNT_BIN} -m ${MOUNT_POINT} -n ${NODE_LIST}>>${logfile} 2>&1
+
+
+ CLUSTERSIZE=32k
+ BLOCKSIZE=4k
+ SLOTS=4
+ LABEL=oracle_home
+
+ ${ECHO} "Format volume to launch new test"|tee -a ${logfile}
+ echo y|${MKFS_BIN} -C ${CLUSTERSIZE} -b ${BLOCKSIZE} -N ${SLOTS} -L ${LABEL} ${DEVICE_NAME} || {
+ ${ECHO} "Can not format ${DEVICE_NAME}"
+ return 1
+ }
+
+ ${ECHO} "Mount volume to all nodes"|tee -a ${logfile}
+ ${REMOTE_MOUNT_BIN} -l ${LABEL} -m ${MOUNT_POINT} -n ${NODE_LIST}>>${logfile} 2>&1
+
+ ${BINDIR}/run_multi_mmap.py -i 20000 -n ${NODE_LIST} -c -b 6000 --hole -f ${testfile} | tee -a ${logfile}
+
+ LogRC $?
+
+ ${ECHO} "Umount volume from all nodes after test."|tee -a ${logfile}
+ ${REMOTE_UMOUNT_BIN} -m ${MOUNT_POINT} -n ${NODE_LIST}>>${logfile} 2>&1
+}
+
+run_lvb_torture_test()
+{
+ LogMsg "lvb_torture_test"
+ :
+ LogRC $?
+
+}
+
+run_create_racer_test()
+{
+ LogMsg "create-racer-test"
+
+ local logdir=${O2TDIR}/log/create_racer_log
+ local logfile=${logdir}/create_racer_test_${DATE}.log
+
+ local testpath=${MOUNT_POINT}
+
+ ${SUDO} mkdir -p ${logdir}
+ #force to umount volume from all nodes
+ ${ECHO} "Try to umount volume from all nodes before test."|tee -a ${logfile}
+ ${REMOTE_UMOUNT_BIN} -m ${MOUNT_POINT} -n ${NODE_LIST}>>${logfile} 2>&1
+
+
+ CLUSTERSIZE=32k
+ BLOCKSIZE=4k
+ SLOTS=4
+ LABEL=oracle_home
+
+ ${ECHO} "Format volume to launch new test"|tee -a ${logfile}
+ echo y|${MKFS_BIN} -C ${CLUSTERSIZE} -b ${BLOCKSIZE} -N ${SLOTS} -L ${LABEL} ${DEVICE_NAME} || {
+ ${ECHO} "Can not format ${DEVICE_NAME}"
+ return 1
+ }
+
+ ${ECHO} "Mount volume to all nodes"|tee -a ${logfile}
+ ${REMOTE_MOUNT_BIN} -l ${LABEL} -m ${MOUNT_POINT} -n ${NODE_LIST}>>${logfile} 2>&1
+
+ ${BINDIR}/run_create_racer.py -i 40000 -l ${logfile} -n ${NODE_LIST} -p ${testpath}
+
+ LogRC $?
+
+ ${ECHO} "Umount volume from all nodes after test."|tee -a ${logfile}
+ ${REMOTE_UMOUNT_BIN} -m ${MOUNT_POINT} -n ${NODE_LIST}>>${logfile} 2>&1
+}
+
+run_flock_tests()
+{
+ LogMsg "flock-tests"
+
+ local logdir=${O2TDIR}/log/flock_log
+ local flock_logfile=${logdir}/flock_test_${DATE}.log
+ local fcntl_logfile=${logdir}/fcntl_test_${DATE}.log
+
+ local testfile1=${MOUNT_POINT}/flock_test_file1
+ local testfile2=${MOUNT_POINT}/flock_test_file2
+
+ ${SUDO} touch ${testfile1}
+ ${SUDO} touch ${testfile2}
+
+ ${SUDO} mkdir -p ${logdir}
+ #force to umount volume from all nodes
+ ${ECHO} "Try to umount volume from all nodes before test."|tee -a ${flock_logfile}
+ ${REMOTE_UMOUNT_BIN} -m ${MOUNT_POINT} -n ${NODE_LIST}>>${flock_logfile} 2>&1
+
+
+ CLUSTERSIZE=32k
+ BLOCKSIZE=4k
+ SLOTS=4
+ LABEL=oracle_home
+
+ ${ECHO} "Format volume to launch new test"|tee -a ${flock_logfile}
+ echo y|${MKFS_BIN} -C ${CLUSTERSIZE} -b ${BLOCKSIZE} -N ${SLOTS} -L ${LABEL} ${DEVICE_NAME} || {
+ ${ECHO} "Can not format ${DEVICE_NAME}"
+ return 1
+ }
+
+ ${ECHO} "Mount volume to all nodes"|tee -a ${flock_logfile}
+ ${REMOTE_MOUNT_BIN} -l ${LABEL} -m ${MOUNT_POINT} -n ${NODE_LIST}>>${flock_logfile} 2>&1
+
+ ${BINDIR}/run_flock_unit_test.py -l ${fcntl_logfile} -n ${NODE_LIST} -t fcntl -e ${testfile1} -f ${testfile2} \
+ && \
+ ${BINDIR}/run_flock_unit_test.py -l ${flock_logfile} -n ${NODE_LIST} -t flock -e ${testfile1} -f ${testfile2}
+
+ LogRC $?
+
+ ${ECHO} "Umount volume from all nodes after test."|tee -a ${logfile}
+ ${REMOTE_UMOUNT_BIN} -m ${MOUNT_POINT} -n ${NODE_LIST}>>${logfile} 2>&1
+
+}
+
+run_open_delete_test()
+{
+ LogMsg "open-delete-test"
+
+ local logdir=${O2TDIR}/log/open_delete_log
+ local logfile=${logdir}/open_delete_test_${DATE}.log
+
+ local testfile=${MOUNT_POINT}/open_delete_test_file
+
+ ${SUDO} mkdir -p ${logdir}
+ #force to umount volume from all nodes
+ ${ECHO} "Try to umount volume from all nodes before test."|tee -a ${logfile}
+ ${REMOTE_UMOUNT_BIN} -m ${MOUNT_POINT} -n ${NODE_LIST}>>${logfile} 2>&1
+
+
+ CLUSTERSIZE=32k
+ BLOCKSIZE=4k
+ SLOTS=4
+ LABEL=oracle_home
+
+ ${ECHO} "Format volume to launch new test"|tee -a ${logfile}
+ echo y|${MKFS_BIN} -C ${CLUSTERSIZE} -b ${BLOCKSIZE} -N ${SLOTS} -L ${LABEL} ${DEVICE_NAME} || {
+ ${ECHO} "Can not format ${DEVICE_NAME}"
+ return 1
+ }
+
+ ${ECHO} "Mount volume to all nodes"|tee -a ${logfile}
+ ${REMOTE_MOUNT_BIN} -l ${LABEL} -m ${MOUNT_POINT} -n ${NODE_LIST}>>${logfile} 2>&1
+
+ ${BINDIR}/open_delete.py -f ${testfile} -i 10000 -l ${logfile} -n ${NODE_LIST}
+
+ LogRC $?
+
+ ${ECHO} "Umount volume from all nodes after test."|tee -a ${logfile}
+ ${REMOTE_UMOUNT_BIN} -m ${MOUNT_POINT} -n ${NODE_LIST}>>${logfile} 2>&1
+}
+
+f_cleanup()
+{
+ :
+}
+################################################################################
+# Main Entry
+################################################################################
+
+trap 'echo -ne "\n\n">>${LOGFILE};echo "Interrupted by Ctrl+C,Cleanuping... "|tee -a ${LOGFILE}; f_cleanup;exit 1' SIGINT
+trap ' : ' SIGTERM
+
+f_setup $*
+
+STARTRUN=$(date +%s)
+${ECHO} "`date` - Starting Multiple Nodes Regress test" > ${LOGFILE}
+
+START=$(date +%s)
+run_xattr_test
+
+START=$(date +%s)
+run_inline_test
+
+START=$(date +%s)
+run_write_append_truncate_test
+
+START=$(date +%s)
+run_multi_mmap_test
+
+START=$(date +%s)
+run_lvb_torture_test
+
+START=$(date +%s)
+run_create_racer_test
+
+START=$(date +%s)
+run_flock_tests
+
+START=$(date +%s)
+run_open_delete_test
+
+END=$(date +%s)
+DIFF=$(( ${END} - ${STARTRUN} ));
+${ECHO} "Total Runtime ${DIFF} seconds.\n" >> ${LOGFILE}
+${ECHO} "`date` - Ended Multiple Nodes Regress test" >> ${LOGFILE}
+
+
</pre>
</blockquote>
</body>
</html>