[Ocfs2-tools-devel] [PATCH 1/1] Ocfs2-test: Add multiple_run.sh for ocfs2-test.
tristan.ye
tristan.ye at oracle.com
Sun Oct 5 21:24:57 PDT 2008
Marcos,
Thanks for your test.
On Fri, 2008-10-03 at 18:37 -0400, Marcos E. Matsunaga wrote:
> Tristan,
>
> A diff of the multiple_run.sh is attached.
>
> Here is some errors I found:
>
> multiple_run.sh: line
> 189: /scratch/mmatsuna/ocfs2test/bin/run_write_append_truncate.py: No
> such file or directory
>
> multiple_run.sh: line
> 229: /scratch/mmatsuna/ocfs2test/bin/run_multi_mmap.py: No such file
> or directory
Errors above due to the compilation error of test_truncate.c, your must
strike at following error when building the ocfs2-test:
test_truncate.c: In function ‘open_test_inode’:
test_truncate.c:150: error: too many arguments to function
‘ocfs2_expand_dir’
test_truncate.c: In function ‘create_file’:
test_truncate.c:221: warning: cast from pointer to integer of different
size
That's because the test_truncate.c still use the old version of
ocfs2_expand_dir,you can mannually fix it or just skip for TEST USE
ONLY. otherwise the building process for ocfs2-test exit abnormally and
left many testcases uncompiled(includes write_append_truncate and
multi_node_mmap),so that's why these two binaries missing when doing the
test. since a 'make install' will not install these into your given
working home directory.
>
> + /scratch/mmatsuna/ocfs2test/bin/run_create_racer.py -i 40000
> -l /scratch/mmatsuna/ocfs2test/log/create_racer_log/create_racer_test_2008-10-03-14-03.log -n ca-test14,ca-test13 -p /ocfs2scratch
> Oct 03 14:05:50 ca-test14 : run_create_racer: main - current
> directory /var/autofs/ca-server1/scratch/mmatsuna/runtest_el5_ia64/bin
> Oct 03 14:05:50 ca-test14 : run_create_racer: main - cmd
> = /scratch/mmatsuna/ocfs2test/bin/create_racer
> Oct 03 14:05:50 ca-test14 : o2tf.StartOpenMPI: Trying to
> execute /usr/lib/openmpi/1.2.5-gcc/bin/mpirun with
> a simple command remotely among (ca-test14,ca-test13)
> Oct 03 14:05:51 ca-test14 : o2tf.mpi_run: MPIRUN
> = /usr/lib/openmpi/1.2.5-gcc/bin/mpirun
> Oct 03 14:05:51 ca-test14 : o2tf.mpi_run: nproc = 2
> Oct 03 14:05:51 ca-test14 : o2tf.mpi_run: nodelen = 2
> Traceback (most recent call last):
> File "/scratch/mmatsuna/ocfs2test/bin/run_create_racer.py", line
> 156, in ?
> 'NOWAIT')
> File
> "/var/autofs/ca-server1/scratch/mmatsuna/runtest_el5_ia64/bin/o2tf.py", line 333, in openmpi_run
> printlog('o2tf.mpi_run: shopt = %d' % shopt,
> TypeError: int argument required
>
>
> + /scratch/mmatsuna/ocfs2test/bin/open_delete.py
> -f /ocfs2scratch/open_delete_test_file -i 10000
> -l /scratch/mmatsuna/ocfs2test/log/open_delete_log/open_delete_test_2008-10-03-14-03.log -n ca-test14,ca-test13
> Oct 03 14:07:18 ca-test14 : o2tf.StartOpenMPI: Trying to
> execute /usr/lib/openmpi/1.2.5-gcc/bin/mpirun with
> a simple command remotely among (ca-test14,ca-test13)
> Oct 03 14:07:18 ca-test14 : command
> = /scratch/mmatsuna/ocfs2test/bin/open_delete
> Oct 03 14:07:18 ca-test14 : o2tf.mpi_run: MPIRUN
> = /usr/lib/openmpi/1.2.5-gcc/bin/mpirun
> Oct 03 14:07:18 ca-test14 : o2tf.mpi_run: nproc = C
> Oct 03 14:07:18 ca-test14 : o2tf.mpi_run: nodelen = 2
> Traceback (most recent call last):
> File "/scratch/mmatsuna/ocfs2test/bin/open_delete.py", line 119,
> in ?
> 'NOWAIT')
> File
> "/var/autofs/ca-server1/scratch/mmatsuna/runtest_el5_ia64/bin/o2tf.py", line 333, in openmpi_run
> printlog('o2tf.mpi_run: shopt = %d' % shopt,
> TypeError: int argument required
>
>
> I guess you have to make "printlog('o2tf.mpi_run: shopt = %d' %
> shopt," to be "printlog('o2tf.mpi_run: shopt = %s' % shopt,".
Thanks for digging out this minor bug.
>
> One thing that you will see in the diff file, is that I made the label
> ocfs2test instead of oracle_home. That's not a good label and may
> cause confusion. May let one believe that it has Oracle binaries in
> it.
Good point, I'll apply this sonn.
Regards,
Tristan.
> Regards,
>
> Marcos Eduardo Matsunaga
>
> Oracle USA
> Linux Engineering
>
> “The statements and opinions expressed here are my own and do not
> necessarily represent those of Oracle Corporation.”
>
>
> Tristan Ye wrote:
> > Like single_run.sh,we also need an automatic launcher for existing multi-nodes testcase
> > to organize and manipulate these cases,statistic their running result and logs in a graceful way.
> >
> > Currently,it includes following testcases,
> >
> > xattr_test
> > inline_test
> > write_append_truncate_test
> > multi_mmap_test
> > lvb_torture_test
> > create_racer_test
> > flock_tests
> > open_delete_test
> >
> > Need to run this wrapper script like this:
> >
> > multiple_run.sh -n node1.us.oracle.com,node2.us.oracle.com -d /dev/sdd1 /storage
> >
> > All its logs and behaviors kept a unification as single_run.sh did. More testcases were expected
> > to be integrated as we're going to develop more multi-nodes cases for ocfs2-test.
> >
> > Signed-off-by: Tristan Ye <tristan.ye at oracle.com>
> > ---
> > programs/python_common/multiple_run.sh | 403 ++++++++++++++++++++++++++++++++
> > 1 files changed, 403 insertions(+), 0 deletions(-)
> > create mode 100755 programs/python_common/multiple_run.sh
> >
> > diff --git a/programs/python_common/multiple_run.sh b/programs/python_common/multiple_run.sh
> > new file mode 100755
> > index 0000000..15e3998
> > --- /dev/null
> > +++ b/programs/python_common/multiple_run.sh
> > @@ -0,0 +1,403 @@
> > +#!/bin/bash
> > +# vi: set ts=8 sw=8 autoindent noexpandtab :
> > +################################################################################
> > +#
> > +# File : multiple_run.sh
> > +#
> > +# Description: The wrapper script help to organize all multi-nodes testcase
> > +# to perform a thorough test among multiple nodes.
> > +#
> > +#
> > +# Author: Tristan Ye, tristan.ye at oracle.com
> > +#
> > +# History: 22 Sep 2008
> > +#
> > +
> > +################################################################################
> > +# Global Variables
> > +################################################################################
> > +PATH=$PATH:/sbin # Add /sbin to the path for ocfs2 tools
> > +export PATH=$PATH:.
> > +
> > +USERNAME=`/usr/bin/whoami`
> > +DATE=`/bin/date +%F-%H-%M`
> > +DF=`which df`
> > +GREP=`which grep`
> > +AWK=`which awk`
> > +ECHO="`which echo` -e"
> > +
> > +SUDO="`which sudo` -u root"
> > +DEBUGFS_BIN="`which sudo` -u root `which debugfs.ocfs2`"
> > +TUNEFS_BIN="`which sudo` -u root `which tunefs.ocfs2`"
> > +MKFS_BIN="`which sudo` -u root `which mkfs.ocfs2`"
> > +
> > +REMOTE_MOUNT_BIN="`which sudo` -u root `which remote_mount.py`"
> > +REMOTE_UMOUNT_BIN="`which sudo` -u root `which remote_umount.py`"
> > +
> > +NODE_LIST=
> > +DEVICE_NAME=
> > +MOUNT_POINT=
> > +
> > +################################################################################
> > +# Utility Functions
> > +################################################################################
> > +f_usage()
> > +{
> > + echo "usage: `basename ${0}` <-n nodes> <-d device> <mountpoint path>"
> > + echo " -n nodelist,should be comma separated."
> > + echo " -d device name used for ocfs2 volume."
> > + echo " <mountpoint path> path of mountpoint where test will be performed."
> > + echo
> > + echo "Eaxamples:"
> > + echo " `basename ${0}` -n node1.us.oracle.com,node2.us.oracle.com -d /dev/sdd1 /storage"
> > + exit 1;
> > +
> > +}
> > +
> > +f_getoptions()
> > +{
> > + if [ $# -eq 0 ]; then
> > + f_usage;
> > + exit 1
> > + fi
> > +
> > + while getopts "n:d:h:" options; do
> > + case $options in
> > + n ) NODE_LIST="$OPTARG";;
> > + d ) DEVICE_NAME="$OPTARG";;
> > + h ) f_usage
> > + exit 1;;
> > + * ) f_usage
> > + exit 1;;
> > + esac
> > + done
> > + shift $(($OPTIND -1))
> > + MOUNT_POINT=${1}
> > +
> > +}
> > +
> > +f_setup()
> > +{
> > + f_getoptions $*
> > +
> > + if [ -z "${MOUNT_POINT}" ];then
> > + f_usage
> > + fi
> > +
> > + . ./config.sh
> > +
> > + LOGFILE=${O2TDIR}/log/single_run_${DATE}.log
> > +
> > + if [ ! -d ${MOUNT_POINT} -o ! -w ${MOUNT_POINT} ]; then
> > + ${ECHO} "Mount point ${MOUNT_POINT} does not exist or is not writable" \
> > + |tee -a ${LOGFILE};
> > + exit 1
> > + fi
> > +
> > + ${DF} -h|${GREP} -q ${DEVICE_NAME}
> > + if [ "$?" == "0" ];then
> > + ${ECHO} "Partition has been mounted,should umount first to perform test" \
> > + |tee -a ${LOGFILE};
> > + exit 1
> > + fi
> > +
> > +}
> > +
> > +LogRC()
> > +{
> > +if [ ${1} -ne 0 ]; then
> > + ${ECHO} "Failed." >> ${LOGFILE};
> > +else
> > + ${ECHO} "Passed." >> ${LOGFILE};
> > +fi;
> > +END=$(date +%s);
> > +DIFF=$(( ${END} - ${START} ));
> > +${ECHO} "Runtime ${DIFF} seconds.\n" >> ${LOGFILE};
> > +}
> > +
> > +LogMsg()
> > +{
> > +${ECHO} `date` >> ${LOGFILE};
> > +${ECHO} "${1}\c" >> ${LOGFILE};
> > +i=${#1};
> > +while (( i < 60 ))
> > +do
> > + ${ECHO} ".\c" >> ${LOGFILE};
> > + (( ++i ));
> > +done;
> > +}
> > +
> > +IsDeviceMounted()
> > +{
> > + local OCFS2_DEVICE=${1}
> > + ${DF} -h | ${GREP} -q ${OCFS2_DEVICE}
> > +
> > + if [ "$?" == "0" ]; then
> > + return 0
> > + else
> > + return 1
> > + fi
> > +}
> > +
> > +run_xattr_test()
> > +{
> > + LogMsg "xattr-test"
> > + ${BINDIR}/xattr-multi-run.sh -r 8 -f ${NODE_LIST} -a rsh -o ${O2TDIR}/log/xattr-tests-log -d ${DEVICE_NAME} ${MOUNT_POINT}
> > + LogRC $?
> > +}
> > +
> > +run_inline_test()
> > +{
> > + LogMsg "inline-test"
> > + ${BINDIR}/multi-inline-run.sh -r 4 -f ${NODE_LIST} -a rsh -o ${O2TDIR}/log/inline-tests-log -d ${DEVICE_NAME} ${MOUNT_POINT}
> > + LogRC $?
> > +
> > +}
> > +
> > +run_write_append_truncate_test()
> > +{
> > + LogMsg "write-append-truncate-test"
> > +
> > + local logdir=${O2TDIR}/log/write_append_truncate_log
> > + local logfile=${logdir}/write_append_truncate_${DATE}.log
> > +
> > + local testfile=${MOUNT_POINT}/write_append_truncate_test_file
> > +
> > + ${SUDO} mkdir -p ${logdir}
> > + #force to umount volume from all nodes
> > + ${ECHO} "Try to umount volume from all nodes before test."|tee -a ${logfile}
> > + ${REMOTE_UMOUNT_BIN} -m ${MOUNT_POINT} -n ${NODE_LIST}>>${logfile} 2>&1
> > +
> > +
> > + CLUSTERSIZE=32k
> > + BLOCKSIZE=4k
> > + SLOTS=4
> > + LABEL=oracle_home
> > +
> > + ${ECHO} "Format volume to launch new test"|tee -a ${logfile}
> > + echo y|${MKFS_BIN} -C ${CLUSTERSIZE} -b ${BLOCKSIZE} -N ${SLOTS} -L ${LABEL} ${DEVICE_NAME} || {
> > + ${ECHO} "Can not format ${DEVICE_NAME}"
> > + return 1
> > + }
> > +
> > + ${ECHO} "Mount volume to all nodes"|tee -a ${logfile}
> > + ${REMOTE_MOUNT_BIN} -l ${LABEL} -m ${MOUNT_POINT} -n ${NODE_LIST}>>${logfile} 2>&1
> > +
> > + ${BINDIR}/run_write_append_truncate.py -i 20000 -l ${logfile} -n ${NODE_LIST} -f ${testfile}
> > +
> > + LogRC $?
> > +
> > + ${ECHO} "Umount volume from all nodes after test."|tee -a ${logfile}
> > + ${REMOTE_UMOUNT_BIN} -m ${MOUNT_POINT} -n ${NODE_LIST}>>${logfile} 2>&1
> > +}
> > +
> > +run_multi_mmap_test()
> > +{
> > + LogMsg "multi-mmap-test"
> > +
> > + local logdir=${O2TDIR}/log/multi_mmap_log
> > + local logfile=${logdir}/multi_mmap_test_${DATE}.log
> > +
> > + local testfile=${MOUNT_POINT}/multi_mmap_test_file
> > +
> > + ${SUDO} mkdir -p ${logdir}
> > + #force to umount volume from all nodes
> > + ${ECHO} "Try to umount volume from all nodes before test."|tee -a ${logfile}
> > + ${REMOTE_UMOUNT_BIN} -m ${MOUNT_POINT} -n ${NODE_LIST}>>${logfile} 2>&1
> > +
> > +
> > + CLUSTERSIZE=32k
> > + BLOCKSIZE=4k
> > + SLOTS=4
> > + LABEL=oracle_home
> > +
> > + ${ECHO} "Format volume to launch new test"|tee -a ${logfile}
> > + echo y|${MKFS_BIN} -C ${CLUSTERSIZE} -b ${BLOCKSIZE} -N ${SLOTS} -L ${LABEL} ${DEVICE_NAME} || {
> > + ${ECHO} "Can not format ${DEVICE_NAME}"
> > + return 1
> > + }
> > +
> > + ${ECHO} "Mount volume to all nodes"|tee -a ${logfile}
> > + ${REMOTE_MOUNT_BIN} -l ${LABEL} -m ${MOUNT_POINT} -n ${NODE_LIST}>>${logfile} 2>&1
> > +
> > + ${BINDIR}/run_multi_mmap.py -i 20000 -n ${NODE_LIST} -c -b 6000 --hole -f ${testfile} | tee -a ${logfile}
> > +
> > + LogRC $?
> > +
> > + ${ECHO} "Umount volume from all nodes after test."|tee -a ${logfile}
> > + ${REMOTE_UMOUNT_BIN} -m ${MOUNT_POINT} -n ${NODE_LIST}>>${logfile} 2>&1
> > +}
> > +
> > +run_lvb_torture_test()
> > +{
> > + LogMsg "lvb_torture_test"
> > + :
> > + LogRC $?
> > +
> > +}
> > +
> > +run_create_racer_test()
> > +{
> > + LogMsg "create-racer-test"
> > +
> > + local logdir=${O2TDIR}/log/create_racer_log
> > + local logfile=${logdir}/create_racer_test_${DATE}.log
> > +
> > + local testpath=${MOUNT_POINT}
> > +
> > + ${SUDO} mkdir -p ${logdir}
> > + #force to umount volume from all nodes
> > + ${ECHO} "Try to umount volume from all nodes before test."|tee -a ${logfile}
> > + ${REMOTE_UMOUNT_BIN} -m ${MOUNT_POINT} -n ${NODE_LIST}>>${logfile} 2>&1
> > +
> > +
> > + CLUSTERSIZE=32k
> > + BLOCKSIZE=4k
> > + SLOTS=4
> > + LABEL=oracle_home
> > +
> > + ${ECHO} "Format volume to launch new test"|tee -a ${logfile}
> > + echo y|${MKFS_BIN} -C ${CLUSTERSIZE} -b ${BLOCKSIZE} -N ${SLOTS} -L ${LABEL} ${DEVICE_NAME} || {
> > + ${ECHO} "Can not format ${DEVICE_NAME}"
> > + return 1
> > + }
> > +
> > + ${ECHO} "Mount volume to all nodes"|tee -a ${logfile}
> > + ${REMOTE_MOUNT_BIN} -l ${LABEL} -m ${MOUNT_POINT} -n ${NODE_LIST}>>${logfile} 2>&1
> > +
> > + ${BINDIR}/run_create_racer.py -i 40000 -l ${logfile} -n ${NODE_LIST} -p ${testpath}
> > +
> > + LogRC $?
> > +
> > + ${ECHO} "Umount volume from all nodes after test."|tee -a ${logfile}
> > + ${REMOTE_UMOUNT_BIN} -m ${MOUNT_POINT} -n ${NODE_LIST}>>${logfile} 2>&1
> > +}
> > +
> > +run_flock_tests()
> > +{
> > + LogMsg "flock-tests"
> > +
> > + local logdir=${O2TDIR}/log/flock_log
> > + local flock_logfile=${logdir}/flock_test_${DATE}.log
> > + local fcntl_logfile=${logdir}/fcntl_test_${DATE}.log
> > +
> > + local testfile1=${MOUNT_POINT}/flock_test_file1
> > + local testfile2=${MOUNT_POINT}/flock_test_file2
> > +
> > + ${SUDO} touch ${testfile1}
> > + ${SUDO} touch ${testfile2}
> > +
> > + ${SUDO} mkdir -p ${logdir}
> > + #force to umount volume from all nodes
> > + ${ECHO} "Try to umount volume from all nodes before test."|tee -a ${flock_logfile}
> > + ${REMOTE_UMOUNT_BIN} -m ${MOUNT_POINT} -n ${NODE_LIST}>>${flock_logfile} 2>&1
> > +
> > +
> > + CLUSTERSIZE=32k
> > + BLOCKSIZE=4k
> > + SLOTS=4
> > + LABEL=oracle_home
> > +
> > + ${ECHO} "Format volume to launch new test"|tee -a ${flock_logfile}
> > + echo y|${MKFS_BIN} -C ${CLUSTERSIZE} -b ${BLOCKSIZE} -N ${SLOTS} -L ${LABEL} ${DEVICE_NAME} || {
> > + ${ECHO} "Can not format ${DEVICE_NAME}"
> > + return 1
> > + }
> > +
> > + ${ECHO} "Mount volume to all nodes"|tee -a ${flock_logfile}
> > + ${REMOTE_MOUNT_BIN} -l ${LABEL} -m ${MOUNT_POINT} -n ${NODE_LIST}>>${flock_logfile} 2>&1
> > +
> > + ${BINDIR}/run_flock_unit_test.py -l ${fcntl_logfile} -n ${NODE_LIST} -t fcntl -e ${testfile1} -f ${testfile2} \
> > + && \
> > + ${BINDIR}/run_flock_unit_test.py -l ${flock_logfile} -n ${NODE_LIST} -t flock -e ${testfile1} -f ${testfile2}
> > +
> > + LogRC $?
> > +
> > + ${ECHO} "Umount volume from all nodes after test."|tee -a ${logfile}
> > + ${REMOTE_UMOUNT_BIN} -m ${MOUNT_POINT} -n ${NODE_LIST}>>${logfile} 2>&1
> > +
> > +}
> > +
> > +run_open_delete_test()
> > +{
> > + LogMsg "open-delete-test"
> > +
> > + local logdir=${O2TDIR}/log/open_delete_log
> > + local logfile=${logdir}/open_delete_test_${DATE}.log
> > +
> > + local testfile=${MOUNT_POINT}/open_delete_test_file
> > +
> > + ${SUDO} mkdir -p ${logdir}
> > + #force to umount volume from all nodes
> > + ${ECHO} "Try to umount volume from all nodes before test."|tee -a ${logfile}
> > + ${REMOTE_UMOUNT_BIN} -m ${MOUNT_POINT} -n ${NODE_LIST}>>${logfile} 2>&1
> > +
> > +
> > + CLUSTERSIZE=32k
> > + BLOCKSIZE=4k
> > + SLOTS=4
> > + LABEL=oracle_home
> > +
> > + ${ECHO} "Format volume to launch new test"|tee -a ${logfile}
> > + echo y|${MKFS_BIN} -C ${CLUSTERSIZE} -b ${BLOCKSIZE} -N ${SLOTS} -L ${LABEL} ${DEVICE_NAME} || {
> > + ${ECHO} "Can not format ${DEVICE_NAME}"
> > + return 1
> > + }
> > +
> > + ${ECHO} "Mount volume to all nodes"|tee -a ${logfile}
> > + ${REMOTE_MOUNT_BIN} -l ${LABEL} -m ${MOUNT_POINT} -n ${NODE_LIST}>>${logfile} 2>&1
> > +
> > + ${BINDIR}/open_delete.py -f ${testfile} -i 10000 -l ${logfile} -n ${NODE_LIST}
> > +
> > + LogRC $?
> > +
> > + ${ECHO} "Umount volume from all nodes after test."|tee -a ${logfile}
> > + ${REMOTE_UMOUNT_BIN} -m ${MOUNT_POINT} -n ${NODE_LIST}>>${logfile} 2>&1
> > +}
> > +
> > +f_cleanup()
> > +{
> > + :
> > +}
> > +################################################################################
> > +# Main Entry
> > +################################################################################
> > +
> > +trap 'echo -ne "\n\n">>${LOGFILE};echo "Interrupted by Ctrl+C,Cleanuping... "|tee -a ${LOGFILE}; f_cleanup;exit 1' SIGINT
> > +trap ' : ' SIGTERM
> > +
> > +f_setup $*
> > +
> > +STARTRUN=$(date +%s)
> > +${ECHO} "`date` - Starting Multiple Nodes Regress test" > ${LOGFILE}
> > +
> > +START=$(date +%s)
> > +run_xattr_test
> > +
> > +START=$(date +%s)
> > +run_inline_test
> > +
> > +START=$(date +%s)
> > +run_write_append_truncate_test
> > +
> > +START=$(date +%s)
> > +run_multi_mmap_test
> > +
> > +START=$(date +%s)
> > +run_lvb_torture_test
> > +
> > +START=$(date +%s)
> > +run_create_racer_test
> > +
> > +START=$(date +%s)
> > +run_flock_tests
> > +
> > +START=$(date +%s)
> > +run_open_delete_test
> > +
> > +END=$(date +%s)
> > +DIFF=$(( ${END} - ${STARTRUN} ));
> > +${ECHO} "Total Runtime ${DIFF} seconds.\n" >> ${LOGFILE}
> > +${ECHO} "`date` - Ended Multiple Nodes Regress test" >> ${LOGFILE}
> > +
> > +
> >
More information about the Ocfs2-tools-devel
mailing list