September 2015 - commits - mailman3.cels.anl.gov

[mpich] MPICH primary repository branch, master, updated. v3.2b4-222-gbf23698
by noreply＠mpich.org 16 Sep '15

16 Sep '15

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "MPICH primary repository". The branch, master has been updated via bf2369811689b096c7f82046d68c5ecd5d80e6dc (commit) via 2824d828421ad9b5d85e29f7cf14e339b882b27e (commit) from df38c36e7d88d3bc5d87fc2506802a47043fdd96 (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- http://git.mpich.org/mpich.git/commitdiff/bf2369811689b096c7f82046d68c5ecd5… commit bf2369811689b096c7f82046d68c5ecd5d80e6dc Author: Yanfei Guo <yguo(a)anl.gov> Date: Tue Sep 15 16:13:03 2015 -0500 maint/jenkins: fix a bug in the set-xfail.sh script The condition for sourcing apply-xfail.sh was wrong. This may results incorrectly setting xfails under certain cases. Signed-off-by: Ken Raffenetti <raffenet(a)mcs.anl.gov> diff --git a/maint/jenkins/set-xfail.sh b/maint/jenkins/set-xfail.sh index 918cfa0..7c4eb46 100755 --- a/maint/jenkins/set-xfail.sh +++ b/maint/jenkins/set-xfail.sh @@ -95,7 +95,7 @@ while read -r line; do fi done < "$XFAIL_CONF" -if [[ -f $XFAIL_CONF ]]; then +if [[ -f $SCRIPT ]]; then source $SCRIPT fi http://git.mpich.org/mpich.git/commitdiff/2824d828421ad9b5d85e29f7cf14e339b… commit 2824d828421ad9b5d85e29f7cf14e339b882b27e Author: Yanfei Guo <yguo(a)anl.gov> Date: Tue Sep 15 12:57:19 2015 -0500 maint/jenkins: add script to skip tests Skip test is set in maint/jenkins/skip_test.conf. It allows skipping all tests for a given combination of job and configurations including compiler, jenkins_configure, netmod, and SLURM queue. When Jenkins uses maint/jenkins/test-worker.sh to build project, it will execute maint/jenkins/skip_test.sh with the jobname, compiler, configure options, netmod, and queue right before initializing the build environment. maint/jenkins/skip_test.sh will read the skip_test.conf file and generate the summary.junit.xml file based on the conditions set in the skip_test.conf. Signed-off-by: Ken Raffenetti <raffenet(a)mcs.anl.gov> diff --git a/maint/jenkins/skip_test.conf b/maint/jenkins/skip_test.conf new file mode 100644 index 0000000..5bb79fa --- /dev/null +++ b/maint/jenkins/skip_test.conf @@ -0,0 +1,22 @@ +# Contitional skip test settings +# +# Syntax (similar to a cron file): +# [jobname] [compiler] [jenkins_configure] [netmod] [queue] +# Note that the [jobname] allows partial matches (see examples). Other +# conditions only allows exact matches. +# The actual allowed combinations are depending on the Jenkins job. For +# example, +# mxm * * tcp * +# will have no effect since none of the mxm jobs has tcp netmod in the +# configuration. +# +# Examples: +# mxm gnu debug * * +# This will skip all tests when the job is "mpich-master-mxm" or +# "mpich-review-mxm", the compiler is "gnu", and the jenkins_configure is +# "debug". +# +# master-ubuntu * * * ubuntu32 sed -i "..." +# This will skip all test when the job is "mpich-master-ubuntu" and the +# running queue is "ubuntu32". +# diff --git a/maint/jenkins/skip_test.sh b/maint/jenkins/skip_test.sh new file mode 100755 index 0000000..d832b96 --- /dev/null +++ b/maint/jenkins/skip_test.sh @@ -0,0 +1,112 @@ +#!/bin/zsh + +jobname="" +compiler="" +jenkins_configure="" +queue="" +netmod="" + +SKIP_TEST_CONF="maint/jenkins/skip_test.conf" +TEST_SUMMARY="test/mpi/summary.junit.xml" + +##################################################################### +## Initialization +##################################################################### + +while getopts ":f:j:c:o:q:m:s:" opt; do + case "$opt" in + j) + jobname=$OPTARG ;; + c) + compiler=$OPTARG ;; + o) + jenkins_configure=$OPTARG ;; + q) + queue=$OPTARG ;; + m) + netmod=$OPTARG ;; + f) + SKIP_TEST_CONF=$OPTARG ;; + s) + TEST_SUMMARY=$OPTARG ;; + \?) + echo "Invalid option: -$OPTARG" >&2 + exit 1 + esac +done + +##################################################################### +## Main ( +##################################################################### + +SkipTestCond() { + local job="$1" + local comp="$2" + local option="$3" + local nmod="$4" + local q="$5" + + local state=0 + + if [[ ! "$job" == "*" ]]; then + # clean up jobname and do substring match + if [[ ! "${jobname%%,*}" == *$job* ]]; then state=1; fi + fi + + if [[ ! "$comp" == "*" ]]; then + if [[ ! "$compiler" == "$comp" ]]; then state=1; fi + fi + + if [[ ! "$option" == "*" ]]; then + if [[ ! "$jenkins_configure" == "$option" ]]; then state=1; fi + fi + + if [[ ! "$nmod" == "*" ]]; then + if [[ ! "$netmod" == "$nmod" ]]; then state=1; fi + fi + + if [[ ! "$q" == "*" ]]; then + if [[ ! "$queue" == "$q" ]]; then state=1; fi + fi + + echo "$state" +} + +SkipTest() { + TEST_SUMMARY_DIR=$(dirname $TEST_SUMMARY) + if [[ -d "$TEST_SUMMARY_DIR" ]]; then + mkdir -p "$TEST_SUMMARY_DIR" + fi + cat > "$TEST_SUMMARY" << "EOF" +<testsuites> +<testsuite failures="0" errors="0" skipped="0" tests="1" date="$date" name="summary_junit_xml"> +<testcase name="none"/> +<system-out/> +<system-err/> +</testsuite> +</testsuites> +EOF +} + +if [[ -f "$TEST_SUMMARY" ]]; then + rm "$TEST_SUMMARY" +fi + +while read -r line; do + #clean leading whitespaces + line=$(echo "$line" | sed "s/^ *//g") + line=$(echo "$line" | sed "s/ *$//g") + echo $line + # skip comment line + if test -x "$line" -o "${line:1}" = "#" ; then + continue + fi + + arr=( $(echo $line) ) + if [[ "0" == $(SkipTestCond "${arr[1]}" "${arr[2]}" "${arr[3]}" "${arr[4]}" "${arr[5]}") ]]; then + SkipTest + exit 0 + fi +done < "$SKIP_TEST_CONF" + +exit 0 diff --git a/maint/jenkins/test-worker.sh b/maint/jenkins/test-worker.sh index 435011d..68a5cd2 100755 --- a/maint/jenkins/test-worker.sh +++ b/maint/jenkins/test-worker.sh @@ -45,14 +45,36 @@ done cd $WORKSPACE -TMP_WORKSPACE=$(mktemp -d /sandbox/jenkins.tmp.XXXXXXXX) -SRC=$WORKSPACE -TMP_SRC=$TMP_WORKSPACE - if test "$GIT_BRANCH" = "" ; then BUILD_MODE="nightly" fi +case "$BUILD_MODE" in + "nightly") + if [[ -x mpich-master/maint/jenkins/skip_test.sh ]]; then + ./mpich-master/maint/jenkins/skip_test.sh -j $JOB_NAME -c $compiler -o $jenkins_configure -q $queue -m $_netmod \ + -s mpich-master/test/mpi/summary.junit.xml + if [[ -f mpich-master/test/mpi/summary.junit.xml ]]; then + exit 0 + fi + fi + ;; + "per-commit") + if [[ -x maint/jenkins/skip_test.sh ]]; then + ./maint/jenkins/skip_test.sh -j $JOB_NAME -c $compiler -o $jenkins_configure -q $queue -m $_netmod \ + -s test/mpi/summary.junit.xml + if [[ -f test/mpi/summary.junit.xml ]]; then + exit 0 + fi + fi + ;; +esac + + +TMP_WORKSPACE=$(mktemp -d /sandbox/jenkins.tmp.XXXXXXXX) +SRC=$WORKSPACE +TMP_SRC=$TMP_WORKSPACE + # Preparing the source case "$BUILD_MODE" in "nightly") ----------------------------------------------------------------------- Summary of changes: maint/jenkins/set-xfail.sh | 2 +- maint/jenkins/skip_test.conf | 22 ++++++++++++ maint/jenkins/{set-xfail.sh => skip_test.sh} | 48 +++++++++++++++---------- maint/jenkins/test-worker.sh | 30 ++++++++++++++-- 4 files changed, 78 insertions(+), 24 deletions(-) create mode 100644 maint/jenkins/skip_test.conf copy maint/jenkins/{set-xfail.sh => skip_test.sh} (67%) hooks/post-receive -- MPICH primary repository

1 0

[mpich] MPICH primary repository branch, master, updated. v3.2b4-220-gdf38c36
by noreply＠mpich.org 15 Sep '15

15 Sep '15

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "MPICH primary repository". The branch, master has been updated via df38c36e7d88d3bc5d87fc2506802a47043fdd96 (commit) from 0750f10fcab0a04b33a86ff12ca95739c7376a27 (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- http://git.mpich.org/mpich.git/commitdiff/df38c36e7d88d3bc5d87fc2506802a470… commit df38c36e7d88d3bc5d87fc2506802a47043fdd96 Author: Yanfei Guo <yguo(a)anl.gov> Date: Tue Sep 15 21:00:42 2015 -0500 maint/jenkins: enable full test for nightly jobs Previous version incorrectly set MPI_TEST_DATATYPE_TEST_LEVEL=min for all jobs. This patch fix that by only setting this environment variable when the build mode is per-commit. Signed-off-by: Ken Raffenetti <raffenet(a)mcs.anl.gov> diff --git a/maint/jenkins/test-worker.sh b/maint/jenkins/test-worker.sh index bcb9edb..435011d 100755 --- a/maint/jenkins/test-worker.sh +++ b/maint/jenkins/test-worker.sh @@ -471,9 +471,12 @@ case "$netmod" in ;; esac -# run only the minimum level of datatype tests -MPITEST_DATATYPE_TEST_LEVEL=min -export MPITEST_DATATYPE_TEST_LEVEL +# run only the minimum level of datatype tests when it is per-commit job +if [[ "$BUILD_MODE" = "per-commit" ]]; then + MPITEST_DATATYPE_TEST_LEVEL=min + export MPITEST_DATATYPE_TEST_LEVEL +fi + make testing # Cleanup ----------------------------------------------------------------------- Summary of changes: maint/jenkins/test-worker.sh | 9 ++++++--- 1 files changed, 6 insertions(+), 3 deletions(-) hooks/post-receive -- MPICH primary repository

1 0

[mpich] MPICH primary repository branch, master, updated. v3.2b4-219-g0750f10
by noreply＠mpich.org 15 Sep '15

15 Sep '15

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "MPICH primary repository". The branch, master has been updated via 0750f10fcab0a04b33a86ff12ca95739c7376a27 (commit) from 19908df8262c49544befc288093470a7beda731d (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- http://git.mpich.org/mpich.git/commitdiff/0750f10fcab0a04b33a86ff12ca95739c… commit 0750f10fcab0a04b33a86ff12ca95739c7376a27 Author: Junchao Zhang <jczhang(a)mcs.anl.gov> Date: Tue Sep 15 12:52:19 2015 -0500 binding/f08: install all f08 module header files Installing only mpi_f08.mod is enough for GNU and Cray compilers, but the Intel Fortran compiler requires all f08 modules to be installed. Signed-off-by: Ken Raffenetti <raffenet(a)mcs.anl.gov> diff --git a/src/binding/fortran/use_mpi_f08/buildiface b/src/binding/fortran/use_mpi_f08/buildiface index 0555c83..c780106 100755 --- a/src/binding/fortran/use_mpi_f08/buildiface +++ b/src/binding/fortran/use_mpi_f08/buildiface @@ -139,7 +139,18 @@ mpi_fc_sources += \\ src/binding/fortran/use_mpi_f08/mpi_c_interface_nobuf.f90 \\ src/binding/fortran/use_mpi_f08/mpi_c_interface_types.f90 -mpi_fc_modules += src/binding/fortran/use_mpi_f08/\$(MPI_F08_NAME).\$(MOD) +mpi_fc_modules += \\ + src/binding/fortran/use_mpi_f08/\$(PMPI_F08_NAME).\$(MOD) \\ + src/binding/fortran/use_mpi_f08/\$(MPI_F08_NAME).\$(MOD) \\ + src/binding/fortran/use_mpi_f08/\$(MPI_F08_CALLBACKS_NAME).\$(MOD) \\ + src/binding/fortran/use_mpi_f08/\$(MPI_F08_COMPILE_CONSTANTS_NAME).\$(MOD) \\ + src/binding/fortran/use_mpi_f08/\$(MPI_F08_LINK_CONSTANTS_NAME).\$(MOD) \\ + src/binding/fortran/use_mpi_f08/\$(MPI_F08_TYPES_NAME).\$(MOD) \\ + src/binding/fortran/use_mpi_f08/\$(MPI_C_INTERFACE_NAME).\$(MOD) \\ + src/binding/fortran/use_mpi_f08/\$(MPI_C_INTERFACE_CDESC_NAME).\$(MOD) \\ + src/binding/fortran/use_mpi_f08/\$(MPI_C_INTERFACE_GLUE_NAME).\$(MOD) \\ + src/binding/fortran/use_mpi_f08/\$(MPI_C_INTERFACE_NOBUF_NAME).\$(MOD) \\ + src/binding/fortran/use_mpi_f08/\$(MPI_C_INTERFACE_TYPES_NAME).\$(MOD) EOT ----------------------------------------------------------------------- Summary of changes: src/binding/fortran/use_mpi_f08/buildiface | 13 ++++++++++++- 1 files changed, 12 insertions(+), 1 deletions(-) hooks/post-receive -- MPICH primary repository

1 0

[mpich] MPICH primary repository branch, master, updated. v3.2b4-218-g19908df
by noreply＠mpich.org 15 Sep '15

15 Sep '15

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "MPICH primary repository". The branch, master has been updated via 19908df8262c49544befc288093470a7beda731d (commit) from 3675cb81ccedc59bb50066623451ea24de48a506 (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- http://git.mpich.org/mpich.git/commitdiff/19908df8262c49544befc288093470a7b… commit 19908df8262c49544befc288093470a7beda731d Author: Ken Raffenetti <raffenet(a)mcs.anl.gov> Date: Thu Sep 10 12:49:40 2015 -0500 binding/f08: fix vpath builds The f08 binding Makefile did not account for sources located outside the current working directory. Fixes #2302 Signed-off-by: Junchao Zhang <jczhang(a)mcs.anl.gov> diff --git a/src/binding/fortran/use_mpi_f08/buildiface b/src/binding/fortran/use_mpi_f08/buildiface index d869633..0555c83 100755 --- a/src/binding/fortran/use_mpi_f08/buildiface +++ b/src/binding/fortran/use_mpi_f08/buildiface @@ -174,7 +174,7 @@ foreach my $x (keys %f08_mods) { print $makefile_fh <<EOT; \t\@rm -f src/binding/fortran/use_mpi_f08/$x.tmp \t\@touch src/binding/fortran/use_mpi_f08/$x.tmp -\t\$(mod_verbose)\$(F08_COMPILE_MODS) -c src/binding/fortran/use_mpi_f08/$x.f90 -o src/binding/fortran/use_mpi_f08/$x.lo +\t\$(mod_verbose)\$(F08_COMPILE_MODS) -c `test -f 'src/binding/fortran/use_mpi_f08/$x.f90' || echo '\$(srcdir)/'`src/binding/fortran/use_mpi_f08/$x.f90 -o src/binding/fortran/use_mpi_f08/$x.lo \t\@mv src/binding/fortran/use_mpi_f08/$x.tmp src/binding/fortran/use_mpi_f08/$x.stamp src/binding/fortran/use_mpi_f08/$DOLLARX.\$(MOD) src/binding/fortran/use_mpi_f08/$x.lo : src/binding/fortran/use_mpi_f08/$x.stamp ----------------------------------------------------------------------- Summary of changes: src/binding/fortran/use_mpi_f08/buildiface | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) hooks/post-receive -- MPICH primary repository

1 0

[mpich] MPICH primary repository branch, master, updated. v3.2b4-217-g3675cb8
by noreply＠mpich.org 14 Sep '15

14 Sep '15

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "MPICH primary repository". The branch, master has been updated via 3675cb81ccedc59bb50066623451ea24de48a506 (commit) via 1083e3050705774163e1c4301f75b605fca2eaec (commit) via 16905e60a726d421d522c502f2d065d0ba88ee24 (commit) from eaf8835f0711a08f425b4534d908a59390de3225 (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- http://git.mpich.org/mpich.git/commitdiff/3675cb81ccedc59bb50066623451ea24d… commit 3675cb81ccedc59bb50066623451ea24de48a506 Author: William Gropp <wgropp(a)illinois.edu> Date: Wed Sep 2 14:19:15 2015 -0400 Fix misspelled comment in getcoverage Signed-off-by: Ken Raffenetti <raffenet(a)mcs.anl.gov> diff --git a/maint/getcoverage.in b/maint/getcoverage.in index 43e0b96..c9c69f2 100644 --- a/maint/getcoverage.in +++ b/maint/getcoverage.in @@ -784,7 +784,7 @@ sub ExpandDir { next; } elsif (-d "$dir/$filename") { - # Skip pmpi directories used for mergeing gcov output + # Skip pmpi directories used for merging gcov output if ($filename =~ /\-pmpi$/) { next; } # Skip mpi directories used for handling name-mangled files if ($filename =~ /\-mpi$/) { next; } http://git.mpich.org/mpich.git/commitdiff/1083e3050705774163e1c4301f75b605f… commit 1083e3050705774163e1c4301f75b605fca2eaec Author: William Gropp <wgropp(a)illinois.edu> Date: Wed Sep 2 14:09:56 2015 -0400 Remove trailing blanks and typo in Makefile Signed-off-by: Ken Raffenetti <raffenet(a)mcs.anl.gov> diff --git a/Makefile.am b/Makefile.am index 2b6403d..811a4df 100644 --- a/Makefile.am +++ b/Makefile.am @@ -300,11 +300,11 @@ gcov_sources = $(lib_lib@MPILIBNAME@_la_SOURCES) # bcast.c --> {bcast.o,lib_libpmpi_la-bcast.o} # It still generates a bcast.c.gcov output for each object file. Long file # names and preserved paths do not help with this problem. -# What we can do in this case is to move the second files to a separate -# directory, rename name, and run the GCOV on the new files. +# What we can do in this case is to move the second files to a separate +# directory, rename name, and run gcov on the new files. # For the web-based coverage reports, the gcov output for the two files -# needs to be merged; see maint/gcovmerge . -# To make it easier to re-run the coverage generation, we copy, rather than +# needs to be merged; see maint/gcovmerge . +# To make it easier to re-run the coverage generation, we copy, rather than # move, when there are two object files for each source file. # A change to the build procedure broke this target. This is a partial though # nearly complete fix to the broken build process. The generated coverage http://git.mpich.org/mpich.git/commitdiff/16905e60a726d421d522c502f2d065d0b… commit 16905e60a726d421d522c502f2d065d0ba88ee24 Author: William Gropp <wgropp(a)illinois.edu> Date: Wed Sep 9 11:23:06 2015 -0500 Fixes for coverage target. Handles changes in the object file naming convention in both Makefile.am and getcoverage script. Refs #2252 Signed-off-by: Ken Raffenetti <raffenet(a)mcs.anl.gov> diff --git a/Makefile.am b/Makefile.am index 4ce3cd5..2b6403d 100644 --- a/Makefile.am +++ b/Makefile.am @@ -306,24 +306,78 @@ gcov_sources = $(lib_lib@MPILIBNAME@_la_SOURCES) # needs to be merged; see maint/gcovmerge . # To make it easier to re-run the coverage generation, we copy, rather than # move, when there are two object files for each source file. +# A change to the build procedure broke this target. This is a partial though +# nearly complete fix to the broken build process. The generated coverage +# files after the fix appear mostly correct, though there appear to also be +# errors in the gcov output itself (the format of this output changes +# frequently). However, the coverage analysis is still valuable and has +# identified missing tests. +# +# The problem: +# The source file is foo.c +# gcov expects foo.o, foo.gcno, and foo.gcda files +# The MPICH build process, unfortunately, does not produce a foo.o file; +# instead, it produces a .libs/lib_lib@MPILIBNAME@_la-foo.o file (and +# similarly .libs/lib_lib@MPILIBNAME@_la_foo.gcno etc. +# This change broke this coverage target. If further changes to the +# naming conventions for object files are made, make sure that the +# coverage target is also modified to correspond to those changes. +# coverage: @for file in $(gcov_sources) ; do \ dir=`dirname $$file` ; \ bname=`basename $$file` ; \ aux=`echo $$bname | sed -e 's,\.*$$,,'` ; \ - echo "( $(GCOV) -b -f -o $$file $$file && mv $${bname}.gcov $$dir )" ; \ - ( $(GCOV) -b -f -o $$file $$file && mv $${bname}.gcov $$dir ) ; \ sname=`echo $$bname | sed -e 's/\.[^\.]*$$//'` ;\ + if [ -s "$$dir/$$sname.o" ] ; then \ + echo "( $(GCOV) -b -f -o $$file $$file && mv $$bname.gcov $$dir )" ; \ + ( $(GCOV) -b -f -o $$file $$file && mv $$bname.gcov $$dir ) ; \ + elif [ -s "$$dir/lib_lib@MPILIBNAME@_la-$$sname.o" ] ; then \ + if [ ! -d $$dir-mpi ] ; then mkdir $$dir-mpi ; fi ; \ + echo "Copying name-mangled files to $$dir-mpi" ; \ + for pfile in $$dir/.libs/lib_lib@MPILIBNAME@_la-$$sname.* ; do \ + nfile=`echo $$pfile | sed -e 's/lib_lib@MPILIBNAME@_la-//'` ;\ + nfile=`basename $$nfile` ; \ + echo "cp -p $$pfile $$dir-mpi/$$nfile" ; \ + cp -p $$pfile $$dir-mpi/$$nfile ; \ + done ; \ + for pfile in $$dir/lib_lib@MPILIBNAME@_la-$$sname.o ; do \ + nfile=`echo $$pfile | sed -e 's/lib_lib@MPILIBNAME@_la-//'` ;\ + nfile=`basename $$nfile` ; \ + echo "( cd $$dir-mpi && $(GCOV) -b -f $$nfile )" ; \ + ( cd $$dir-mpi && $(GCOV) -b -f $$nfile ) ; \ + if [ ! -s $$dir-mpi/$$bname.gcov ] ; then \ + echo "No $$bname.gcov file created" ;\ + else \ + mv $$dir-mpi/$$bname.gcov $$dir ; \ + fi ; \ + done ; \ + else \ + echo "Missing object file for source $$file" ; \ + fi ; \ if [ -s "$$dir/lib_lib@PMPILIBNAME@_la-$$sname.o" ] ; then \ if [ ! -d $$dir-pmpi ] ; then mkdir $$dir-pmpi ; fi ; \ - for pfile in $$dir/lib_lib@PMPILIBNAME@_la-$$sname.* ; do \ + echo "Copying name-mangled files to $$dir-pmpi" ; \ + for pfile in $$dir/.libs/lib_lib@PMPILIBNAME@_la-$$sname.* ; do \ + nfile=`echo $$pfile | sed -e 's/lib_lib@PMPILIBNAME@_la-//'` ;\ + nfile=`basename $$nfile` ; \ + echo "cp -p $$pfile $$dir-pmpi/$$nfile" ; \ + cp -p $$pfile $$dir-pmpi/$$nfile ; \ + done ; \ + for pfile in $$dir/lib_lib@PMPILIBNAME@_la-$$sname.o ; do \ nfile=`echo $$pfile | sed -e 's/lib_lib@PMPILIBNAME@_la-//'` ;\ nfile=`basename $$nfile` ; \ - cp $$pfile $$dir-pmpi/$$nfile ; \ echo "( cd $$dir-pmpi && $(GCOV) -b -f $$nfile )" ; \ ( cd $$dir-pmpi && $(GCOV) -b -f $$nfile ) ; \ - $(srcdir)/maint/gcovmerge $$dir-pmpi/$$bname.gcov \ + if [ -s $$dir/$$bname.gcov ] ; then \ + echo "gcovmerge $$dir-pmpi/$$bname.gcov $$dir/$$bname.gcov > $$dir/$$bname.gcov.merge" ; \ + $(srcdir)/maint/gcovmerge $$dir-pmpi/$$bname.gcov \ $$dir/$$bname.gcov > $$dir/$$bname.gcov.merge ;\ + else \ + echo "Did not find file $$dir/$$bname.gcov" ; \ + echo "mv $$dir-pmpi/$$bname.gcov $$dir/$$bname.gcov";\ + mv $$dir-pmpi/$$bname.gcov $$dir/$$bname.gcov ; \ + fi ;\ done ; \ fi ; \ rm -f *.gcov ; \ diff --git a/maint/getcoverage.in b/maint/getcoverage.in index 9eb7239..43e0b96 100644 --- a/maint/getcoverage.in +++ b/maint/getcoverage.in @@ -170,6 +170,10 @@ for my $filename (@TopFilenames) { # Check for a "pmpi" directory - one that was used to merge # PMPI and MPI outputs. If found, we skip to the next directory if ($filename =~ /\-pmpi$/) { next; } + # Added to handle a change in the naming convention for object files + # in MPICH, which now needs a -mpi directory as well as the -pmpi + # directory needed to handle the pmpi object files. + if ($filename =~ /\-mpi$/) { next; } # Expand the directory into a list of files. If there are # subdirectories, expand them too (e.g., get a list of # *every* file within the directory tree. @@ -782,6 +786,8 @@ sub ExpandDir { elsif (-d "$dir/$filename") { # Skip pmpi directories used for mergeing gcov output if ($filename =~ /\-pmpi$/) { next; } + # Skip mpi directories used for handling name-mangled files + if ($filename =~ /\-mpi$/) { next; } $otherdirs[$#otherdirs+1] = "$dir/$filename"; } elsif ($filename =~ /(.*\.gcov)$/) { ----------------------------------------------------------------------- Summary of changes: Makefile.am | 72 +++++++++++++++++++++++++++++++++++++++++++------ maint/getcoverage.in | 8 +++++- 2 files changed, 70 insertions(+), 10 deletions(-) hooks/post-receive -- MPICH primary repository

1 0

[mpich] MPICH primary repository branch, master, updated. v3.2b4-214-geaf8835
by noreply＠mpich.org 14 Sep '15

14 Sep '15

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "MPICH primary repository". The branch, master has been updated via eaf8835f0711a08f425b4534d908a59390de3225 (commit) from fe6dd619711825de9513d72fc9903ab4ef01905e (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- http://git.mpich.org/mpich.git/commitdiff/eaf8835f0711a08f425b4534d908a5939… commit eaf8835f0711a08f425b4534d908a59390de3225 Author: Ken Raffenetti <raffenet(a)mcs.anl.gov> Date: Mon Sep 14 09:01:56 2015 -0500 maint/jenkins: xfail test failing on Portals 4 bug The Portals 4 reference implementation has a bug that makes recv cancel operations unreliable in threaded cases. We modified to the threaded alltoall test to consistently reproduce the failure, now we xfail it when running with Portals 4 to not dirty the Jenkins results. Signed-off-by: Yanfei Guo <yguo(a)anl.gov> diff --git a/maint/jenkins/xfail.conf b/maint/jenkins/xfail.conf index 62836e5..119f8ae 100644 --- a/maint/jenkins/xfail.conf +++ b/maint/jenkins/xfail.conf @@ -23,4 +23,4 @@ # For each build, all applied XFAILS will be summaried in # ${SOURCE}/apply-xfail.sh # -# master-portals * * portals4 * sed -i "s+$^alltoall .*$+\1 xfail=ticket0+g" test/mpi/threads/pt2pt/testlist +portals4 * * * * sed -i "s+$^alltoall .*$+\1 xfail=ticket0+g" test/mpi/threads/pt2pt/testlist ----------------------------------------------------------------------- Summary of changes: maint/jenkins/xfail.conf | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) hooks/post-receive -- MPICH primary repository

1 0

[mpich] MPICH primary repository branch, master, updated. v3.2b4-213-gfe6dd61
by noreply＠mpich.org 10 Sep '15

10 Sep '15

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "MPICH primary repository". The branch, master has been updated via fe6dd619711825de9513d72fc9903ab4ef01905e (commit) from 1e6c4d8e0f3b38ba3d9a9b6eff7602c6960f67cb (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- http://git.mpich.org/mpich.git/commitdiff/fe6dd619711825de9513d72fc9903ab4e… commit fe6dd619711825de9513d72fc9903ab4ef01905e Author: Yanfei Guo <yguo(a)anl.gov> Date: Sat Sep 5 20:36:31 2015 -0500 Add scripts for setting XFAIL XFAIL is set in maint/jenkins/xfail.conf. It allows an XFAIL to be applied based on various conditions including jobname, compiler, jenkins_configure, netmod, and SLURM queue. When Jenkins uses maint/jenkins/test-worker.sh to build project, it will execute maint/jenkins/set-xfail.sh with the jobname, compiler, configure options, netmod, and queue right before executing ./configure”. maint/jenkins/set-xfail.sh will read the xfail.conf file and apply the XFAIL based on the conditions set in the xfail.conf. All applied XFAILs will be summarized in apply-xfail.sh. xfail-apply.sh is added to gitignore to prevent from accidentally adding the xfail summary to the source. Signed-off-by: Ken Raffenetti <raffenet(a)mcs.anl.gov> diff --git a/.gitignore b/.gitignore index bb70b76..9f0e903 100644 --- a/.gitignore +++ b/.gitignore @@ -105,6 +105,7 @@ summary.xml summary.junit.xml summary.tap tags +apply-xfail.sh # the hooks dir shouldn't be accidentally committed /maint/hooks diff --git a/maint/jenkins/set-xfail.sh b/maint/jenkins/set-xfail.sh new file mode 100755 index 0000000..918cfa0 --- /dev/null +++ b/maint/jenkins/set-xfail.sh @@ -0,0 +1,102 @@ +#!/bin/zsh + +jobname="" +compiler="" +jenkins_configure="" +queue="" +netmod="" + +XFAIL_CONF="maint/jenkins/xfail.conf" + +##################################################################### +## Initialization +##################################################################### + +while getopts ":f:j:c:o:q:m:" opt; do + case "$opt" in + j) + jobname=$OPTARG ;; + c) + compiler=$OPTARG ;; + o) + jenkins_configure=$OPTARG ;; + q) + queue=$OPTARG ;; + m) + netmod=$OPTARG ;; + f) + XFAIL_CONF=$OPTARG ;; + \?) + echo "Invalid option: -$OPTARG" >&2 + exit 1 + esac +done + +##################################################################### +## Main ( +##################################################################### + +if test ! -f "$XFAIL_CONF" ; then + echo "Cannot find $XFAIL_CONF. No XFAIL will be applied" + exit 0 +fi + +XFAILCond() { + local job="$1" + local comp="$2" + local option="$3" + local nmod="$4" + local q="$5" + + local state=0 + + if [[ ! "$job" == "*" ]]; then + # clean up jobname and do substring match + if [[ ! "${jobname%%,*}" == *$job* ]]; then state=1; fi + fi + + if [[ ! "$comp" == "*" ]]; then + if [[ ! "$compiler" == "$comp" ]]; then state=1; fi + fi + + if [[ ! "$option" == "*" ]]; then + if [[ ! "$jenkins_configure" == "$option" ]]; then state=1; fi + fi + + if [[ ! "$nmod" == "*" ]]; then + if [[ ! "$netmod" == "$nmod" ]]; then state=1; fi + fi + + if [[ ! "$q" == "*" ]]; then + if [[ ! "$queue" == "$q" ]]; then state=1; fi + fi + + echo "$state" +} + +SCRIPT="apply-xfail.sh" +if [[ -f $SCRIPT ]]; then + rm $SCRIPT +fi + +while read -r line; do + #clean leading whitespaces + line=$(echo "$line" | sed "s/^ *//g") + line=$(echo "$line" | sed "s/ *$//g") + echo $line + # skip comment line + if test -x "$line" -o "${line:1}" = "#" ; then + continue + fi + + arr=( $(echo $line) ) + if [[ "0" == $(XFAILCond "${arr[1]}" "${arr[2]}" "${arr[3]}" "${arr[4]}" "${arr[5]}") ]]; then + echo "${arr[@]:5}" >> $SCRIPT + fi +done < "$XFAIL_CONF" + +if [[ -f $XFAIL_CONF ]]; then + source $SCRIPT +fi + +exit 0 diff --git a/maint/jenkins/test-worker.sh b/maint/jenkins/test-worker.sh index 7f94495..bcb9edb 100755 --- a/maint/jenkins/test-worker.sh +++ b/maint/jenkins/test-worker.sh @@ -79,6 +79,7 @@ CollectResults() { # TODO: copy saved test binaries (for failed cases) find . \ \( -name "filtered-make.txt" -o \ + -name "apply-xfail.sh" -o \ -name "autogen.log" -o \ -name "config.log" -o \ -name "c.txt" -o \ @@ -418,6 +419,10 @@ if test "$BUILD_MODE" = "per-commit" ; then ./autogen.sh 2>&1 | tee autogen.log fi +if [[ -x maint/jenkins/set-xfail.sh ]]; then + ./maint/jenkins/set-xfail.sh -j $JOB_NAME -c $compiler -o $jenkins_configure -q $queue -m $_netmod +fi + ./configure --prefix="$TMP_SRC/_inst" $(SetNetmod $netmod) $(SetConfigOpt $jenkins_configure) \ --disable-perftest \ 2>&1 | tee c.txt diff --git a/maint/jenkins/xfail.conf b/maint/jenkins/xfail.conf new file mode 100644 index 0000000..62836e5 --- /dev/null +++ b/maint/jenkins/xfail.conf @@ -0,0 +1,26 @@ +# Contitional XFAIL settings +# +# Syntax (similar to a cron file): +# [jobname] [compiler] [jenkins_configure] [netmod] [queue] [sed of XFAIL] +# Note that the [jobname] allows partial matches (see examples). Other +# conditions only allows exact matches. +# The actual allowed combinations are depending on the Jenkins job. For +# example, +# mxm * * tcp * sed -i "..." +# will have no effect since none of the mxm jobs has tcp netmod in the +# configuration. +# +# Examples: +# mxm gnu debug * * sed -i "..." +# This will apply the set the XFAIL when the job is "mpich-master-mxm" or +# "mpich-review-mxm", the compiler is "gnu", and the jenkins_configure is +# "debug". +# +# master-ubuntu * * * ubuntu32 sed -i "..." +# This will apply the set the XFAIL when the job is "mpich-master-ubuntu" and +# the running queue is "ubuntu32". +# +# For each build, all applied XFAILS will be summaried in +# ${SOURCE}/apply-xfail.sh +# +# master-portals * * portals4 * sed -i "s+$^alltoall .*$+\1 xfail=ticket0+g" test/mpi/threads/pt2pt/testlist ----------------------------------------------------------------------- Summary of changes: .gitignore | 1 + maint/jenkins/set-xfail.sh | 102 ++++++++++++++++++++++++++++++++++++++++++ maint/jenkins/test-worker.sh | 5 ++ maint/jenkins/xfail.conf | 26 +++++++++++ 4 files changed, 134 insertions(+), 0 deletions(-) create mode 100755 maint/jenkins/set-xfail.sh create mode 100644 maint/jenkins/xfail.conf hooks/post-receive -- MPICH primary repository

1 0

[mpich] MPICH primary repository branch, master, updated. v3.2b4-212-g1e6c4d8
by noreply＠mpich.org 09 Sep '15

09 Sep '15

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "MPICH primary repository". The branch, master has been updated via 1e6c4d8e0f3b38ba3d9a9b6eff7602c6960f67cb (commit) from f91baf0296ce0f0a9fd198e7f5f78321abb8179c (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- http://git.mpich.org/mpich.git/commitdiff/1e6c4d8e0f3b38ba3d9a9b6eff7602c69… commit 1e6c4d8e0f3b38ba3d9a9b6eff7602c6960f67cb Author: Ken Raffenetti <raffenet(a)mcs.anl.gov> Date: Wed Sep 9 14:31:29 2015 -0500 romio: remove stop statements from fortran tests Explicit stop statements are not required and can create unwanted noise during program termination. Remove these to prevent false positive test failures in the nightlies. No reviewer. diff --git a/src/mpi/romio/test/fcoll_test.f.in b/src/mpi/romio/test/fcoll_test.f.in index 9c2b172..d6bae74 100644 --- a/src/mpi/romio/test/fcoll_test.f.in +++ b/src/mpi/romio/test/fcoll_test.f.in @@ -172,5 +172,4 @@ call MPI_FINALIZE(ierr) - stop end diff --git a/src/mpi/romio/test/fmisc.f.in b/src/mpi/romio/test/fmisc.f.in index 5bfaa03..a75c4ee 100644 --- a/src/mpi/romio/test/fmisc.f.in +++ b/src/mpi/romio/test/fmisc.f.in @@ -201,5 +201,4 @@ call MPI_GROUP_FREE(group, ierr) call MPI_FINALIZE(ierr) - stop end diff --git a/src/mpi/romio/test/fperf.f.in b/src/mpi/romio/test/fperf.f.in index e4ff4d1..6277949 100644 --- a/src/mpi/romio/test/fperf.f.in +++ b/src/mpi/romio/test/fperf.f.in @@ -176,5 +176,4 @@ call MPI_FINALIZE(ierr) - stop end diff --git a/src/mpi/romio/test/pfcoll_test.f.in b/src/mpi/romio/test/pfcoll_test.f.in index 11433d2..d05db1d 100644 --- a/src/mpi/romio/test/pfcoll_test.f.in +++ b/src/mpi/romio/test/pfcoll_test.f.in @@ -173,5 +173,4 @@ call PMPI_FINALIZE(ierr) - stop end ----------------------------------------------------------------------- Summary of changes: src/mpi/romio/test/fcoll_test.f.in | 1 - src/mpi/romio/test/fmisc.f.in | 1 - src/mpi/romio/test/fperf.f.in | 1 - src/mpi/romio/test/pfcoll_test.f.in | 1 - 4 files changed, 0 insertions(+), 4 deletions(-) hooks/post-receive -- MPICH primary repository

1 0

[mpich] MPICH primary repository branch, master, updated. v3.2b4-211-gf91baf0
by noreply＠mpich.org 04 Sep '15

04 Sep '15

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "MPICH primary repository". The branch, master has been updated via f91baf0296ce0f0a9fd198e7f5f78321abb8179c (commit) from de6df61e8ceb2bbfc9bda82a66c6531c1ebd34ab (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- http://git.mpich.org/mpich.git/commitdiff/f91baf0296ce0f0a9fd198e7f5f78321a… commit f91baf0296ce0f0a9fd198e7f5f78321abb8179c Author: Yanfei Guo <yanf.guo(a)gmail.com> Date: Wed Sep 2 13:52:38 2015 -0500 Add build scripts for Jenkins test jobs The build scripts are maintained in one place to ensure the consistency. These scripts will be submitted as SLURM jobs and executed on SLURM nodes. Relationship between the build scripts and jobs: test-worker-abi-prolog.sh - mpich-master-abi-prolog test-worker-abi.sh - mpich-master-abi test-worker-tarball.sh - mpich-tarball test-worker-armci.sh - All armci jobs test-worker.sh - All other jobs Signed-off-by: Ken Raffenetti <raffenet(a)mcs.anl.gov> diff --git a/maint/jenkins/test-worker-abi-prolog.sh b/maint/jenkins/test-worker-abi-prolog.sh new file mode 100755 index 0000000..52dd32f --- /dev/null +++ b/maint/jenkins/test-worker-abi-prolog.sh @@ -0,0 +1,253 @@ +#!/bin/zsh -xe + +WORKSPACE="" +compiler="gnu" +jenkins_configure="default" +queue="ib64" +netmod="default" +ofi_prov="sockets" +N_MAKE_JOBS=8 + +##################################################################### +## Initialization +##################################################################### + +while getopts ":h:c:o:q:m:n:" opt; do + case "$opt" in + h) + WORKSPACE=$OPTARG ;; + c) + compiler=$OPTARG ;; + o) + jenkins_configure=$OPTARG ;; + q) + queue=$OPTARG ;; + m) + _netmod=${OPTARG%%,*} + if [[ "$_netmod" == "ofi" ]]; then + netmod=$_netmod + ofi_prov=${OPTARG/$_netmod,} + else + netmod=$_netmod + fi + ;; + n) + N_MAKE_JOBS=$OPTARG ;; + \?) + echo "Invalid option: -$OPTARG" >&2 + exit 1 + esac +done + +cd $WORKSPACE + +TMP_WORKSPACE=$(mktemp -d /sandbox/jenkins.tmp.XXXXXXXX) +SRC=$WORKSPACE +TMP_SRC=$TMP_WORKSPACE + +tar zxvf mpich-master.tar.gz +SRC=$WORKSPACE/mpich-master +TMP_SRC=$TMP_WORKSPACE/mpich-master +cp -a $WORKSPACE/mpich-master $TMP_WORKSPACE/ + + +##################################################################### +## Functions +##################################################################### + +PrepareEnv() { + case "$queue" in + "ubuntu32" | "ubuntu64" ) + source /software/common/adm/etc/softenv-aliases.sh + source /software/common/adm/etc/softenv-load.sh + soft add +intel + soft add +pgi + soft add +absoft + soft add +nagfor + soft add +solarisstudio-12.4 + soft add +ekopath + ;; + "freebsd64" | "freebsd32") + PATH=/usr/local/bin:$PATH + LD_LIBRARY_PATH=/usr/local/lib/gcc46:$LD_LIBRARY_PATH + export LD_LIBRARY_PATH + ;; + "solaris") + PATH=/usr/gcc/4.3/bin:/usr/gnu/bin:/usr/sbin:$PATH + CONFIG_SHELL=/usr/gnu/bin/sh + export CONFIG_SHELL + ;; + esac + PATH=$HOME/software/autotools/bin:$PATH + export PATH + echo "$PATH" +} + +SetCompiler() { + case "$compiler" in + "gnu") + CC=gcc + CXX=g++ + F77=gfortran + FC=gfortran + ;; + "clang") + CC=clang + CXX=clang++ + F77=gfortran + FC=gfortran + ;; + "intel") + CC=icc + CXX=icpc + F77=ifort + FC=ifort + ;; + "pgi") + CC=pgcc + CXX=pgcpp + F77=pgf77 + FC=pgfortran + ;; + "absoft") + CC=gcc + CXX=g++ + F77=af77 + FC=af90 + ;; + "nag") + CC=gcc + CXX=g++ + F77=nagfor + FC=nagfor + FFLAGS="-mismatch" + FCFLAGS="-mismatch" + export FFLAGS + export FCFLAGS + ;; + "solstudio") + CC=suncc + CXX=sunCC + F77=sunf77 + FC=sunf90 + ;; + "pathscale") + CC=pathcc + CXX=pathCC + F77=pathf95 + FC=pathf95 + ;; + *) + echo "Unknown compiler suite" + exit 1 + esac + + export CC + export CXX + export F77 + export FC + + which $CC + which $CXX + which $F77 + which $FC +} + +SetConfigOpt() { + config_opt="__TO_BE_FILLED__" + case "$jenkins_configure" in + "default") + config_opt= + ;; + "strict") + config_opt="--enable-strict" + ;; + "fast") + config_opt="--enable-fast=all" + ;; + "nofast") + config_opt="--disable-fast" + ;; + "noshared") + config_opt="--disable-shared" + ;; + "debug") + config_opt="--enable-g=all" + ;; + "noweak") + config_opt="--disable-weak-symbols" + ;; + "strictnoweak") + config_opt="--enable-strict --disable-weak-symbols" + ;; + "nofortran") + config_opt="--disable-f77 --disable-fc" + ;; + "nocxx") + config_opt="--disable-cxx" + ;; + "multithread") + config_opt="--enable-threads=multiple" + ;; + "debuginfo") + config_opt="--enable-debuginfo" + ;; + "noerrorchecking") + config_opt="--disable-error-checking" + ;; + "sock") # for solaris + sock + config_opt="--with-device=ch3:sock --disable-ft-tests --disable-comm-overlap-tests" + ;; + "mpd") + config_opt="--with-pm=mpd --with-namepublisher=file" + ;; + "gforker") + config_opt="--with-pm=gforker --with-namepublisher=file" + ;; + "shmem") + config_opt= + ;; + "async") + config_opt= + ;; + "random") + config_opt=$(RandArgs) + ;; + *) + echo "Bad configure option: $jenkins_configure" + exit 1 + esac + + if test "$jenkins_configure" != "shmem" ; then + config_opt="$config_opt --enable-nemesis-dbg-localoddeven" + fi + + if test "$queue" = "osx" -a "$FC" = "ifort"; then + config_opt="$config_opt lv_cv_ld_force_load=no" + fi + + export config_opt + echo "$config_opt" +} + +rm -rf $HOME/mpi/installs/mpich-master/$compiler/$jenkins_configure + +PrepareEnv + +SetCompiler "$compiler" + +pushd "$TMP_SRC" + +./configure --prefix=$HOME/mpi/installs/mpich-master/$compiler/$jenkins_configure $(SetConfigOpt $jenkins_configure) +make -j8 install + +popd + +if test "$compiler" = "gnu" -a "$jenkins_configure" = "default" ; then + cd /home/autotest/mpi/tests/mpich-tests + git clean -x -d -f + git pull + ./autogen.sh +fi + +exit 0 diff --git a/maint/jenkins/test-worker-abi.sh b/maint/jenkins/test-worker-abi.sh new file mode 100755 index 0000000..1722558 --- /dev/null +++ b/maint/jenkins/test-worker-abi.sh @@ -0,0 +1,363 @@ +#!/bin/zsh -x + +WORKSPACE="" +build_mpi="mpich-master" +compiler="gnu" +run_conf="default" +run_mpi="mpich-master" + +##################################################################### +## Initialization +##################################################################### + +while getopts ":h:c:o:b:r:" opt; do + case "$opt" in + h) + WORKSPACE=$OPTARG ;; + c) + compiler=$OPTARG ;; + o) + run_conf=$OPTARG ;; + b) + build_mpi=$OPTARG ;; + r) + run_mpi=$OPTARG ;; + \?) + echo "Invalid option: -$OPTARG" >&2 + exit 1 + esac +done + +cd $WORKSPACE +TMP_WORKSPACE=$(mktemp -d /sandbox/jenkins.tmp.XXXXXXXX) + +# $build_mpi -- MPI libraries used to build the tests +# $run_mpi -- MPI libraries used to run the tests +# $compiler -- gcc or icc. Compilers used to build $build_mpi _and_ $run_mpi. +# Intel MPI provides mpigcc, mpiicc etc. +# $run_conf -- debug or default. Options used to build $build_mpi _and_ $run_mpi + +##################################################################### +## Setup the paths for the compilers +##################################################################### +source /software/common/adm/etc/softenv-aliases.sh +source /software/common/adm/etc/softenv-load.sh +soft add +intel + +if test "$compiler" = "gnu" ; then + CC=gcc + CXX=g++ + F77=gfortran + FC=gfortran +elif test "$compiler" = "intel" ; then + CC=icc + CXX=icpc + F77=ifort + FC=ifort +else + echo "Unknown compiler suite" + exit 1 +fi + +export CC +export CXX +export F77 +export FC + +which $CC +which $CXX +which $F77 +which $FC + +## Setup tmp directory +mkdir ${TMP_WORKSPACE}/_tmp + +## Setup binaries in MPICH +if test "$build_mpi" != "impi-5.0"; then + cp -a ${HOME}/mpi/installs/${build_mpi}/${compiler}/${run_conf} ${TMP_WORKSPACE}/_tmp/${build_mpi}-inst + MPICH_BUILD_PATH=${TMP_WORKSPACE}/_tmp/${build_mpi}-inst + export MPICH_BUILD_PATH + cd ${MPICH_BUILD_PATH}/bin + for x in mpif77 mpif90 mpicc mpicxx ; do + sed -i "s+^libdir=.*+libdir=${MPICH_BUILD_PATH}/lib+g" $x + sed -i "s+^prefix=.*+prefix=${MPICH_BUILD_PATH}+g" $x + sed -i "s+^includedir=.*+includedir=${MPICH_BUILD_PATH}/include+g" $x + done +fi +if test "$run_mpi" != "impi-5.0"; then + if test "$build_mpi" != "$run_mpi"; then + cp -a ${HOME}/mpi/installs/${run_mpi}/${compiler}/${run_conf} ${TMP_WORKSPACE}/_tmp/${run_mpi}-inst + fi + MPICH_RUN_PATH=${TMP_WORKSPACE}/_tmp/${run_mpi}-inst + export MPICH_RUN_PATH +fi + +## Setup binaries in IMPI +cp -a /home/autotest/intel/impi/5.0.1.035 ${TMP_WORKSPACE}/_tmp/impi-inst +IMPI_PATH=${TMP_WORKSPACE}/_tmp/impi-inst +export IMPI_PATH +cd ${IMPI_PATH}/intel64/bin +for x in mpif77 mpif90 mpigcc mpigxx mpiicc mpiicpc mpiifort ; do + sed -i "s+^prefix=.*+prefix=${IMPI_PATH}+g" $x +done + + +## Setup compiler paths +if test "$build_mpi" = "mpich-master" -o "$build_mpi" = "mpich-3.1" ; then + MPICC=${MPICH_BUILD_PATH}/bin/mpicc + MPICXX=${MPICH_BUILD_PATH}/bin/mpicxx + MPIF77=${MPICH_BUILD_PATH}/bin/mpif77 + MPIFC=${MPICH_BUILD_PATH}/bin/mpif90 +elif test "$build_mpi" = "impi-5.0" ; then + if test "$compiler" = "gnu" ; then + MPICC=${IMPI_PATH}/intel64/bin/mpigcc + MPICXX=${IMPI_PATH}/intel64/bin/mpigxx + MPIF77=${IMPI_PATH}/intel64/bin/mpif77 + MPIFC=${IMPI_PATH}/intel64/bin/mpif90 + elif test "$compiler" = "intel" ; then + MPICC=${IMPI_PATH}/intel64/bin/mpiicc + MPICXX=${IMPI_PATH}/intel64/bin/mpiicpc + MPIF77=${IMPI_PATH}/intel64/bin/mpiifort + MPIFC=${IMPI_PATH}/intel64/bin/mpiifort + fi +fi +export MPICC +export MPICXX +export MPIF77 +export MPIFC + +if test "$run_mpi" = "mpich-master" -o "$run_mpi" = "mpich-3.1" ; then + MPIEXEC=${MPICH_RUN_PATH}/bin/mpiexec +elif test "$run_mpi" = "impi-5.0"; then + MPIEXEC=${IMPI_PATH}/intel64/bin/mpiexec.hydra +fi +export MPIEXEC + +cp -a /home/autotest/mpi/tests/mpich-tests ${TMP_WORKSPACE}/_tmp + +MPICH_TESTS_PATH=${TMP_WORKSPACE}/_tmp/mpich-tests/test/mpi +export MPICH_TESTS_PATH + +cd ${MPICH_TESTS_PATH} +./configure --disable-ft-tests --disable-dependency-tracking +make -j16 --keep-going + +## Some tests require special compilation +cd f77/ext +make allocmemf +cd ${MPICH_TESTS_PATH} + +cd f90/ext +make allocmemf90 +cd ${MPICH_TESTS_PATH} + +cd f90/profile +make profile1f90 +cd ${MPICH_TESTS_PATH} + +cd impls/mpich +make +cd ${MPICH_TESTS_PATH} + +# these tests will compile with old MPIs, but produce the wrong result at runtime +if test "$run_mpi" = "impi-5.0" -o "$run_mpi" = "mpich-3.1" ; then + sed -i "s+$^gather_big .*$+\1 xfail=ticket0+g" coll/testlist + sed -i "s+$^comm_idup_comm .*$+\1 xfail=ticket0+g" comm/testlist + sed -i "s+$^comm_idup_comm2 .*$+\1 xfail=ticket0+g" comm/testlist + sed -i "s+$^comm_idup_iallreduce .*$+\1 xfail=ticket0+g" comm/testlist + sed -i "s+$^comm_idup_isend .*$+\1 xfail=ticket0+g" comm/testlist + sed -i "s+$^comm_idup_mul .*$+\1 xfail=ticket0+g" comm/testlist + sed -i "s+$^comm_idup_nb .*$+\1 xfail=ticket0+g" comm/testlist + sed -i "s+$^comm_idup_overlap .*$+\1 xfail=ticket0+g" comm/testlist + sed -i "s+$^dynamic_errcode_predefined_errclass .*$+\1 xfail=ticket0+g" errhan/testlist + sed -i "s+$^errstring2 .*$+\1 xfail=ticket0+g" errhan/testlist + sed -i "s+$^noalias2 .*$+\1 xfail=ticket0+g" errors/coll/testlist + sed -i "s+$^noalias3 .*$+\1 xfail=ticket0+g" errors/coll/testlist + sed -i "s+$^too_many_icomms .*$+\1 xfail=ticket0+g" errors/comm/testlist + sed -i "s+$^too_many_icomms2 .*$+\1 xfail=ticket0+g" errors/comm/testlist + sed -i "s+$^file_errhdl .*$+\1 xfail=ticket0+g" errors/io/testlist + sed -i "s+$^cas_type_check .*$+\1 xfail=ticket0+g" errors/rma/testlist + sed -i "s+$^win_sync_lock_fence .*$+\1 xfail=ticket0+g" errors/rma/testlist + sed -i "s+$^unpub .*$+\1 xfail=ticket0+g" errors/spawn/testlist + sed -i "s+$^bottom .*$+\1 xfail=ticket0+g" f77/datatype/testlist + sed -i "s+$^bottom .*$+\1 xfail=ticket0+g" f90/datatype/testlist + sed -i "s+$^createf90types .*$+\1 xfail=ticket0+g" f90/f90types/testlist + sed -i "s+$^infoget .*$+\1 xfail=ticket0+g" info/testlist + sed -i "s+$^bigtype .*$+\1 xfail=ticket0+g" io/testlist + sed -i "s+$^hindexed_io .*$+\1 xfail=ticket0+g" io/testlist + sed -i "s+$^resized2 .*$+\1 xfail=ticket0+g" io/testlist + sed -i "s+$^acc-pairtype .*$+\1 xfail=ticket0+g" rma/testlist + sed -i "s+$^accfence1 .*$+\1 xfail=ticket0+g" rma/testlist + sed -i "s+$^accpscw1 .*$+\1 xfail=ticket0+g" rma/testlist + sed -i "s+$^atomic_get .*$+\1 xfail=ticket0+g" rma/testlist + sed -i "s+$^epochtest .*$+\1 xfail=ticket0+g" rma/testlist + sed -i "s+$^large-acc-flush_local .*$+\1 xfail=ticket0+g" rma/testlist + sed -i "s+$^lock_contention_dt .*$+\1 xfail=ticket0+g" rma/testlist + sed -i "s+$^lock_dt .*$+\1 xfail=ticket0+g" rma/testlist + sed -i "s+$^lock_dt_flush .*$+\1 xfail=ticket0+g" rma/testlist + sed -i "s+$^lock_dt_flushlocal .*$+\1 xfail=ticket0+g" rma/testlist + sed -i "s+$^lockall_dt .*$+\1 xfail=ticket0+g" rma/testlist + sed -i "s+$^lockall_dt_flush .*$+\1 xfail=ticket0+g" rma/testlist + sed -i "s+$^lockall_dt_flushall .*$+\1 xfail=ticket0+g" rma/testlist + sed -i "s+$^lockall_dt_flushlocal .*$+\1 xfail=ticket0+g" rma/testlist + sed -i "s+$^lockall_dt_flushlocalall .*$+\1 xfail=ticket0+g" rma/testlist + sed -i "s+$^putfence1 .*$+\1 xfail=ticket0+g" rma/testlist + sed -i "s+$^putpscw1 .*$+\1 xfail=ticket0+g" rma/testlist + sed -i "s+$^win_info .*$+\1 xfail=ticket0+g" rma/testlist + sed -i "s+$^win_shared_cas_flush_load .*$+\1 xfail=ticket0+g" rma/testlist + sed -i "s+$^win_shared_gacc_flush_load .*$+\1 xfail=ticket0+g" rma/testlist + sed -i "s+$^win_shared_zerobyte .*$+\1 xfail=ticket0+g" rma/testlist + + # regression tests added after mpich-3.1 + sed -i "s+$^manyrma2_shm .*$+\1 xfail=ticket0+g" rma/testlist + sed -i "s+$^nullpscw_shm .*$+\1 xfail=ticket0+g" rma/testlist + sed -i "s+$^pscw_ordering_shm .*$+\1 xfail=ticket0+g" rma/testlist + sed -i "s+$^test2_am_shm .*$+\1 xfail=ticket0+g" rma/testlist + sed -i "s+$^test3_am_shm .*$+\1 xfail=ticket0+g" rma/testlist + sed -i "s+$^test2_shm .*$+\1 xfail=ticket0+g" rma/testlist + sed -i "s+$^test3_shm .*$+\1 xfail=ticket0+g" rma/testlist + sed -i "s+$^transpose3_shm .*$+\1 xfail=ticket0+g" rma/testlist + sed -i "s+$^wintest_shm .*$+\1 xfail=ticket0+g" rma/testlist +fi + +# MPI-3.1 version +sed -i "s+$^version .*$+\1 xfail=ticket0+g" init/testlist +sed -i "s+$^baseenvf .*$+\1 xfail=ticket0+g" f77/init/testlist +sed -i "s+$^baseenv .*$+\1 xfail=ticket0+g" cxx/init/testlist +sed -i "s+$^baseenvf90 .*$+\1 xfail=ticket0+g" f90/init/testlist + +## Remove the MPI library that is not being used for running +if test "$build_mpi" = "mpich-master" -a "$run_mpi" = "mpich-master" ; then + rm -rf ${IMPI_PATH} + + LD_LIBRARY_PATH=${MPICH_RUN_PATH}/lib:$LD_LIBRARY_PATH + export LD_LIBRARY_PATH + +elif test "$build_mpi" = "mpich-master" -a "$run_mpi" = "mpich-3.1" ; then + rm -rf ${MPICH_BUILD_PATH} + rm -rf ${IMPI_PATH} + + cd ${MPICH_RUN_PATH}/lib + + ln -s libmpich.so.12.0.0 libmpi.so.0 + ln -s libmpichcxx.so.12.0.0 libmpicxx.so.0 + ln -s libmpichf90.so.12.0.0 libmpifort.so.0 + + LD_LIBRARY_PATH=${MPICH_RUN_PATH}/lib:$LD_LIBRARY_PATH + export LD_LIBRARY_PATH + + LD_PRELOAD=${MPICH_RUN_PATH}/lib/libmpich.so.12:${MPICH_RUN_PATH}/lib/libmpichf90.so.12 + export LD_PRELOAD + +elif test "$build_mpi" = "mpich-master" -a "$run_mpi" = "impi-5.0"; then + rm -rf ${MPICH_BUILD_PATH} + + cd ${IMPI_PATH}/intel64/lib + + if test "$run_conf" = "debug" ; then + ln -s debug/libmpi.so.12.0 libmpi.so.0 + else + ln -s release/libmpi.so.12.0 libmpi.so.0 + fi + ln -s libmpicxx.so.12.0 libmpicxx.so.0 + ln -s libmpifort.so.12.0 libmpifort.so.0 + + LD_LIBRARY_PATH=${IMPI_PATH}/intel64/lib:$LD_LIBRARY_PATH + export LD_LIBRARY_PATH + + LD_PRELOAD=${IMPI_PATH}/intel64/lib/libmpi.so.12:${IMPI_PATH}/intel64/lib/libmpifort.so.12 + export LD_PRELOAD + +elif test "$build_mpi" = "impi-5.0" -a "$run_mpi" = "mpich-master" ; then + rm -rf ${IMPI_PATH} + + cd ${MPICH_RUN_PATH}/lib + + ln -s libmpi.so.0.0.0 libmpi.so.12 + mkdir release + ln -s libmpi.so.0.0.0 release/libmpi.so.12 + mkdir debug + ln -s libmpi.so.0.0.0 debug/libmpi.so.12 + ln -s libmpicxx.so.0.0.0 libmpicxx.so.12 + ln -s libmpifort.so.0.0.0 libmpifort.so.12 + + LD_LIBRARY_PATH=${MPICH_RUN_PATH}/lib:$LD_LIBRARY_PATH + export LD_LIBRARY_PATH + +elif test "$build_mpi" = "impi-5.0" -a "$run_mpi" = "mpich-3.1" ; then + rm -rf ${IMPI_PATH} + + cd ${MPICH_RUN_PATH}/lib + + ln -s libmpich.so.12.0.0 libmpi.so.12 + ln -s libmpichcxx.so.12.0.0 libmpicxx.so.12 + ln -s libmpichf90.so.12.0.0 libmpifort.so.12 + + LD_LIBRARY_PATH=${MPICH_RUN_PATH}/lib:$LD_LIBRARY_PATH + export LD_LIBRARY_PATH + +elif test "$build_mpi" = "impi-5.0" -a "$run_mpi" = "impi-5.0" ; then + LD_LIBRARY_PATH=${IMPI_PATH}/intel64/lib:$LD_LIBRARY_PATH + export LD_LIBRARY_PATH + +elif test "$build_mpi" = "mpich-3.1" -a "$run_mpi" = "mpich-master" ; then + rm -rf ${IMPI_PATH} + rm -rf ${MPICH_BUILD_PATH} + + cd ${MPICH_RUN_PATH}/lib + + ln -s libmpi.so.0.0.0 libmpich.so.12 + ln -s libmpicxx.so.0.0.0 libmpichcxx.so.12 + ln -s libmpifort.so.0.0.0 libmpichf90.so.12 + ln -s libmpi.so.0.0.0 libmpl.so.1 + ln -s libmpi.so.0.0.0 libopa.so.1 + + LD_LIBRARY_PATH=${MPICH_RUN_PATH}/lib:$LD_LIBRARY_PATH + export LD_LIBRARY_PATH + + LD_PRELOAD=${MPICH_RUN_PATH}/lib/libmpich.so.12:${MPICH_RUN_PATH}/lib/libmpichf90.so.12 + export LD_PRELOAD + +elif test "$build_mpi" = "mpich-3.1" -a "$run_mpi" = "mpich-3.1" ; then + rm -rf ${IMPI_PATH} + + LD_LIBRARY_PATH=${MPICH_RUN_PATH}/lib:$LD_LIBRARY_PATH + export LD_LIBRARY_PATH + +elif test "$build_mpi" = "mpich-3.1" -a "$run_mpi" = "impi-5.0" ; then + rm -rf ${MPICH_BUILD_PATH} + + cd ${IMPI_PATH}/intel64/lib + + if test "$run_conf" = "debug" ; then + ln -s debug/libmpi.so.12.0 libmpich.so.12 + else + ln -s release/libmpi.so.12.0 libmpich.so.12 + fi + ln -s libmpicxx.so.12.0 libmpichcxx.so.12 + ln -s libmpifort.so.12.0 libmpichf90.so.12 + ln -s libmpich.so.12 libmpl.so.1 + ln -s libmpich.so.12 libopa.so.1 + + LD_LIBRARY_PATH=${IMPI_PATH}/intel64/lib:$LD_LIBRARY_PATH + export LD_LIBRARY_PATH + + LD_PRELOAD=${IMPI_PATH}/intel64/lib/libmpich.so.12:${IMPI_PATH}/intel64/lib/libmpichf90.so.12 + export LD_PRELOAD + +else + echo "Unknown combination" + exit 1 +fi + +## Run tests for MPI version 3.0 +cd ${MPICH_TESTS_PATH} +./runtests -mpiversion=3.0 -srcdir=. -tests=testlist -junitfile=summary.junit.xml -tapfile=summary.tap -xmlfile=summary.xml + +mv summary.junit.xml ${WORKSPACE} +mv summary.tap ${WORKSPACE} +mv summary.xml ${WORKSPACE} +cd ${WORKSPACE} +rm -rf ${TMP_WORKSPACE} +exit 0 + diff --git a/maint/jenkins/test-worker-armci.sh b/maint/jenkins/test-worker-armci.sh new file mode 100755 index 0000000..d11164c --- /dev/null +++ b/maint/jenkins/test-worker-armci.sh @@ -0,0 +1,86 @@ +#!/bin/zsh -xe + +WORKSPACE="" +GIT_BRANCH="" +BUILD_MODE="per-commit" + +##################################################################### +## Initialization +##################################################################### + +while getopts ":h:b:" opt; do + case "$opt" in + h) + WORKSPACE=$OPTARG ;; + b) + GIT_BRANCH=$OPTARG ;; + \?) + echo "Invalid option: -$OPTARG" >&2 + exit 1 + esac +done + +cd $WORKSPACE + +TMP_WORKSPACE=$(mktemp -d /sandbox/jenkins.tmp.XXXXXXXX) +SRC=$WORKSPACE +TMP_SRC=$TMP_WORKSPACE + +if test "$GIT_BRANCH" = "" ; then + BUILD_MODE="nightly" +fi + +# Preparing the source +case "$BUILD_MODE" in + "nightly") + tar zxvf mpich-master.tar.gz + SRC=$WORKSPACE/mpich-master + TMP_SRC=$TMP_WORKSPACE/mpich-master + cp -a $WORKSPACE/mpich-master $TMP_WORKSPACE/ + ;; + "per-commit") + cp -a $WORKSPACE/* $TMP_WORKSPACE/ + ;; + *) + echo "Invalid BUILD_MODE $BUILD_MODE. Set by mistake?" + exit 1 +esac + +PATH=$HOME/software/autotools/bin:$PATH +export PATH + +##################################################################### +## Main() { Setup Environment and Build +##################################################################### + +pushd "$TMP_SRC" + +if test "$BUILD_MODE" = "per-commit" ; then + ./autogen.sh 2>&1 | tee autogen.log +fi + +./configure --prefix="$TMP_SRC/_inst" CC=gcc CXX=g++ FC=gfortran 2>&1 | tee c.txt +make -j16 install 2>&1 | tee mi.txt + +##################################################################### +## Run tests +##################################################################### + +rm -rf ./armci-mpi +git clone -b mpi3rma git://git.mpich.org/armci-mpi + +export PATH=$TMP_SRC/_inst/bin:$PATH + +pushd $TMP_SRC/armci-mpi + +./autogen.sh +./configure +make -j8 +make check + +popd +popd + +rm -rf $TMP_WORKSPACE +exit 0 + diff --git a/maint/jenkins/test-worker-tarball.sh b/maint/jenkins/test-worker-tarball.sh new file mode 100755 index 0000000..ee443a6 --- /dev/null +++ b/maint/jenkins/test-worker-tarball.sh @@ -0,0 +1,17 @@ +#!/bin/zsh -xe + +export PATH=$HOME/software/autotools/bin:$HOME/software/sowing/bin:$PATH +export DOCTEXT_PATH=$HOME/software/sowing/share/doctext +export CC=gcc +export CXX=g++ +export FC=gfortran + +cd $WORKSPACE +git clean -x -d -f + +mkdir build +cd build + +../maint/release.pl --branch=master --version=master --git-repo=git://git.mpich.org/mpich.git + +exit 0 diff --git a/maint/jenkins/test-worker.sh b/maint/jenkins/test-worker.sh new file mode 100755 index 0000000..7f94495 --- /dev/null +++ b/maint/jenkins/test-worker.sh @@ -0,0 +1,497 @@ +#!/bin/zsh -xe + +WORKSPACE="" +compiler="gnu" +jenkins_configure="default" +queue="ib64" +netmod="default" +ofi_prov="sockets" +N_MAKE_JOBS=8 +GIT_BRANCH="" +BUILD_MODE="per-commit" + +##################################################################### +## Initialization +##################################################################### + +while getopts ":h:c:o:q:m:n:b:" opt; do + case "$opt" in + h) + WORKSPACE=$OPTARG ;; + c) + compiler=$OPTARG ;; + o) + jenkins_configure=$OPTARG ;; + q) + queue=$OPTARG ;; + m) + _netmod=${OPTARG%%,*} + if [[ "$_netmod" == "ofi" ]]; then + netmod=$_netmod + ofi_prov=${OPTARG/$_netmod,} + else + netmod=$_netmod + fi + ;; + n) + N_MAKE_JOBS=$OPTARG ;; + b) + GIT_BRANCH=$OPTARG ;; + \?) + echo "Invalid option: -$OPTARG" >&2 + exit 1 + esac +done + +cd $WORKSPACE + +TMP_WORKSPACE=$(mktemp -d /sandbox/jenkins.tmp.XXXXXXXX) +SRC=$WORKSPACE +TMP_SRC=$TMP_WORKSPACE + +if test "$GIT_BRANCH" = "" ; then + BUILD_MODE="nightly" +fi + +# Preparing the source +case "$BUILD_MODE" in + "nightly") + tar zxvf mpich-master.tar.gz + SRC=$WORKSPACE/mpich-master + TMP_SRC=$TMP_WORKSPACE/mpich-master + cp -a $WORKSPACE/mpich-master $TMP_WORKSPACE/ + ;; + "per-commit") + git clean -x -d -f + cp -a $WORKSPACE/* $TMP_WORKSPACE/ + ;; + *) + echo "Invalid BUILD_MODE $BUILD_MODE. Set by mistake?" + exit 1 +esac + + +##################################################################### +## Functions +##################################################################### + +CollectResults() { + # TODO: copy saved test binaries (for failed cases) + find . \ + $ -name "filtered-make.txt" -o \ + -name "autogen.log" -o \ + -name "config.log" -o \ + -name "c.txt" -o \ + -name "m.txt" -o \ + -name "mi.txt" -o \ + -name "summary.junit.xml" $ \ + -exec cp {} $SRC/{} \; +} + +##################################################################### +## Logic to generate random configure options +##################################################################### +RandArgs() { + # Chosen *without* replacement. If an option is chosen twice, + # then there will be fewer options + n_choice=$1 + array=(${(P)${2}}) + optname=$3 + negoptname=$4 + chosen=() + args="" + ret_args="" + array_len=$#array + idx=0 + + for i in `seq $array_len`; do + chosen[$i]=0 + done + + for i in `seq $n_choice`; do + let idx=$[RANDOM % $array_len]+1 + if [ $chosen[$idx] -eq 1 ]; then continue; fi + chosen[$idx]=1 + args=("${(s/;/)array[$idx]}") + name=$args[1] + if [ $#args -eq 1 ]; then + # Only the name is provided. Choose one of three + # choices: + # No option (skip this one) + # just --$optname-$name + # just --$negoptname-$name + let idx=$[RANDOM % 3]+1 + if [ $idx -eq 1 ]; then + ret_args="$ret_args --$optname-$name" + elif [ $idx -eq 2 ]; then + ret_args="$ret_args --$negoptname-$name" + fi + else + let idx=$[RANDOM % ($#args-1)]+2 + # Special cases + if [ "$args[$idx]" = "ch3:sock" ]; then + ret_args="$ret_args --disable-ft-tests --disable-comm-overlap-tests" + elif [ "$args[$idx]" = "gforker" ]; then + if [ $chosen[4] -eq 1 ]; then + continue + else + ret_args="$ret_args --with-namepublisher=file" + chosen[4]=1 + fi + elif [ "$name" = "namepublisher" -a "$args[$idx]" = "no" ]; then + if [ $chosen[3] -eq 1 ]; then + continue + fi + elif [ "$args[$idx]" = "ndebug" -a "$CC" = "suncc" -a "$label" = "ubuntu32" ]; then + # On ubuntu32, suncc has a bug whose workaround is to add -O flag (ticket #2105) + CFLAGS="-O1" + export CFLAGS + fi + ret_args="$ret_args --$optname-$name=$args[$idx]" + fi + done + echo $ret_args +} + +RandConfig() { + # WARNING: If moving anything in the two following arrays, check the indices in "Special cases" above + enable_array=( + 'error-checking;no;runtime;all' + 'error-messages;all;generic;class;none' + 'timer-type;linux86_cycle;clock_gettime;gettimeofday' + 'timing;none;all;runtime;log;log_detailed' + 'g;none;all;handle;dbg;log;meminit;handlealloc;instr;mem;mutex;mutexnesting' + 'fast;O0;O1;O2;O3;ndebug;all;yes;none' + 'fortran' + 'cxx' + 'romio' + 'check-compiler-flags' + 'strict;c99;posix' + 'debuginfo' + 'weak-symbols;no;yes' + 'threads;single;multiple;runtime' + 'thread-cs;global' + 'refcount;lock-free;none' + 'mutex-timing' + 'handle-allocation;tls;mutex' + 'multi-aliases' + 'predefined-refcount' + 'alloca' + 'yield;sched_yield;select' + 'runtimevalues' + ) + with_array=( + 'logging;none' + 'pmi;simple' + 'pm;gforker' + 'namepublisher;no;file' + 'device;ch3;ch3:sock' + ) + let n_enable=$#enable_array+1 + let n_with=$#with_array+1 + enable_args=$(RandArgs $n_enable "enable_array" "enable" "disable") + with_args=$(RandArgs $n_with "with_array" "with" "without") + echo "$enable_args $with_args" +} + +PrepareEnv() { + case "$queue" in + "ubuntu32" | "ubuntu64" ) + source /software/common/adm/etc/softenv-aliases.sh + source /software/common/adm/etc/softenv-load.sh + soft add +intel + soft add +pgi + soft add +absoft + soft add +nagfor + soft add +solarisstudio-12.4 + soft add +ekopath + ;; + "freebsd64" | "freebsd32") + PATH=/usr/local/bin:$PATH + LD_LIBRARY_PATH=/usr/local/lib/gcc46:$LD_LIBRARY_PATH + export LD_LIBRARY_PATH + ;; + "solaris") + PATH=/usr/gcc/4.3/bin:/usr/gnu/bin:/usr/sbin:$PATH + CONFIG_SHELL=/usr/gnu/bin/sh + export CONFIG_SHELL + ;; + esac + PATH=$HOME/software/autotools/bin:$PATH + export PATH + echo "$PATH" +} + +SetCompiler() { + case "$compiler" in + "gnu") + CC=gcc + CXX=g++ + F77=gfortran + FC=gfortran + ;; + "clang") + CC=clang + CXX=clang++ + F77=gfortran + FC=gfortran + ;; + "intel") + CC=icc + CXX=icpc + F77=ifort + FC=ifort + ;; + "pgi") + CC=pgcc + CXX=pgcpp + F77=pgf77 + FC=pgfortran + ;; + "absoft") + CC=gcc + CXX=g++ + F77=af77 + FC=af90 + ;; + "nag") + CC=gcc + CXX=g++ + F77=nagfor + FC=nagfor + FFLAGS="-mismatch" + FCFLAGS="-mismatch" + export FFLAGS + export FCFLAGS + ;; + "solstudio") + CC=suncc + CXX=sunCC + F77=sunf77 + FC=sunf90 + ;; + "sunstudio") + CC=cc + CXX=CC + F77=f77 + FC=f90 + ;; + "pathscale") + CC=pathcc + CXX=pathCC + F77=pathf95 + FC=pathf95 + ;; + *) + echo "Unknown compiler suite" + exit 1 + esac + + export CC + export CXX + export F77 + export FC + + which $CC + which $CXX + which $F77 + which $FC +} + +SetNetmod() { + netmod_opt="__NULL__" + case "$netmod" in + "default") # for solaris, may use with sock + netmod_opt= + ;; + "mxm") + netmod_opt="--with-device=ch3:nemesis:mxm --with-mxm=$HOME/software/mellanox/mxm --disable-spawn --disable-ft-tests" + ;; + "ofi") + netmod_opt="--with-device=ch3:nemesis:ofi --with-ofi=$HOME/software/libfabric/$ofi_prov --disable-spawn --disable-ft-tests LD_LIBRARY_PATH=$HOME/software/libfabric/lib" + ;; + "portals4") + netmod_opt="--with-device=ch3:nemesis:portals4 --with-portals4=$HOME/software/portals4 --disable-spawn --disable-ft-tests" + ;; + "sock") + netmod_opt="--with-device=ch3:sock --disable-ft-tests --disable-comm-overlap-tests" + ;; + "tcp") + netmod_opt= + ;; + *) + echo "Unknown netmod type" + exit 1 + esac + export netmod_opt + echo "$netmod_opt" +} + +SetConfigOpt() { + config_opt="__TO_BE_FILLED__" + case "$jenkins_configure" in + "default") + config_opt= + ;; + "strict") + config_opt="--enable-strict" + ;; + "fast") + config_opt="--enable-fast=all" + ;; + "nofast") + config_opt="--disable-fast" + ;; + "noshared") + config_opt="--disable-shared" + ;; + "debug") + config_opt="--enable-g=all" + ;; + "noweak") + config_opt="--disable-weak-symbols" + ;; + "strictnoweak") + config_opt="--enable-strict --disable-weak-symbols" + ;; + "nofortran") + config_opt="--disable-fortran" + ;; + "nocxx") + config_opt="--disable-cxx" + ;; + "multithread") + config_opt="--enable-threads=multiple" + ;; + "debuginfo") + config_opt="--enable-debuginfo" + ;; + "noerrorchecking") + config_opt="--disable-error-checking" + ;; + "sock") # for solaris + sock + config_opt="--with-device=ch3:sock --disable-ft-tests --disable-comm-overlap-tests" + ;; + "mpd") + config_opt="--with-pm=mpd --with-namepublisher=file" + ;; + "gforker") + config_opt="--with-pm=gforker --with-namepublisher=file" + ;; + "shmem") + config_opt= + ;; + "async") + config_opt= + ;; + "random") + config_opt=$(RandArgs) + ;; + *) + echo "Bad configure option: $jenkins_configure" + exit 1 + esac + + if test "$jenkins_configure" != "shmem" ; then + config_opt="$config_opt --enable-nemesis-dbg-localoddeven" + fi + + if test "$queue" = "osx" -a "$FC" = "ifort"; then + config_opt="$config_opt lv_cv_ld_force_load=no" + fi + + export config_opt + echo "$config_opt" +} + +##################################################################### +## Main() { Setup Environment and Build +##################################################################### +# determine if this is a nightly job or a per-commit job +PrepareEnv + +SetCompiler "$compiler" + +pushd "$TMP_SRC" + +if test "$BUILD_MODE" = "per-commit" ; then + ./autogen.sh 2>&1 | tee autogen.log +fi + +./configure --prefix="$TMP_SRC/_inst" $(SetNetmod $netmod) $(SetConfigOpt $jenkins_configure) \ + --disable-perftest \ + 2>&1 | tee c.txt +make -j$N_MAKE_JOBS 2>&1 | tee m.txt +if test "${pipestatus[-2]}" != "0"; then + CollectResults + exit 1 +fi +make -j$N_MAKE_JOBS install 2>&1 | tee mi.txt +if test "${pipestatus[-2]}" != "0"; then + CollectResults + exit 1 +fi +cat m.txt mi.txt | ./maint/clmake > filtered-make.txt 2>&1 + +##################################################################### +## Run tests +##################################################################### + +# Preparation +case "$jenkins_configure" in + "mpd") + $TMP_SRC/_inst/bin/mpd & + sleep 1 + ;; + "async") + MPIR_CVAR_ASYNC_PROGRESS=1 + export MPIR_CVAR_ASYNC_PROGRESS + ;; + "multithread") + MPIR_CVAR_DEFAULT_THREAD_LEVEL=MPI_THREAD_MULTIPLE + export MPIR_CVAR_DEFAULT_THREAD_LEVEL + ;; +esac + +case "$netmod" in + "mxm") + MXM_LOG_LEVEL=error + export MXM_LOG_LEVEL + MXM_SHM_KCOPY_MODE=off + export MXM_SHM_KCOPY_MODE + ;; + "ofi" | "portals4") + MXM_LOG_LEVEL=error + export MXM_LOG_LEVEL + ;; +esac + +# run only the minimum level of datatype tests +MPITEST_DATATYPE_TEST_LEVEL=min +export MPITEST_DATATYPE_TEST_LEVEL +make testing + +# Cleanup +case "$jenkins_configure" in + "mpd") + $TMP_SRC/_inst/bin/mpdallexit + ;; + "async") + unset MPIR_CVAR_ASYNC_PROGRESS + ;; +esac + +if killall -9 hydra_pmi_proxy; then + echo "leftover hydra_pmi_proxy processes killed" +fi + +##################################################################### +## Copy Test results and Cleanup +##################################################################### + +CollectResults + +popd +rm -rf $TMP_WORKSPACE +exit 0 + ----------------------------------------------------------------------- Summary of changes: maint/jenkins/test-worker-abi-prolog.sh | 253 ++++++++++++++++ maint/jenkins/test-worker-abi.sh | 363 ++++++++++++++++++++++ maint/jenkins/test-worker-armci.sh | 86 ++++++ maint/jenkins/test-worker-tarball.sh | 17 + maint/jenkins/test-worker.sh | 497 +++++++++++++++++++++++++++++++ 5 files changed, 1216 insertions(+), 0 deletions(-) create mode 100755 maint/jenkins/test-worker-abi-prolog.sh create mode 100755 maint/jenkins/test-worker-abi.sh create mode 100755 maint/jenkins/test-worker-armci.sh create mode 100755 maint/jenkins/test-worker-tarball.sh create mode 100755 maint/jenkins/test-worker.sh hooks/post-receive -- MPICH primary repository

1 0

[mpich] MPICH primary repository branch, master, updated. v3.2b4-210-gde6df61
by noreply＠mpich.org 04 Sep '15

04 Sep '15

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "MPICH primary repository". The branch, master has been updated via de6df61e8ceb2bbfc9bda82a66c6531c1ebd34ab (commit) from f12779988838ca5cb19788d08f2cd86d258e0cfd (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- http://git.mpich.org/mpich.git/commitdiff/de6df61e8ceb2bbfc9bda82a66c6531c1… commit de6df61e8ceb2bbfc9bda82a66c6531c1ebd34ab Author: Jeff Squyres <jsquyres(a)cisco.com> Date: Fri Sep 4 14:10:19 2015 -0500 netmod/ofi: specify tx/rx ordering Per https://github.com/ofiwg/libfabric/issues/1217#issuecomment-131264316, you really need to specify FI_ORDER_SAS for both transmit and receive if you want to guarantee ordering on FI_EP_RDM endpoints (and you do; the MPI-style tagging interface doesn't make much sense unless the underlying fabric provider guarnatees ordering). Signed-off-by: Ken Raffenetti <raffenet(a)mcs.anl.gov> diff --git a/src/mpid/ch3/channels/nemesis/netmod/ofi/ofi_init.c b/src/mpid/ch3/channels/nemesis/netmod/ofi/ofi_init.c index 5de47b7..7c445ac 100644 --- a/src/mpid/ch3/channels/nemesis/netmod/ofi/ofi_init.c +++ b/src/mpid/ch3/channels/nemesis/netmod/ofi/ofi_init.c @@ -81,6 +81,8 @@ int MPID_nem_ofi_init(MPIDI_PG_t * pg_p, int pg_rank, char **bc_val_p, int *val_ hints->mode = FI_CONTEXT; hints->ep_attr->type = FI_EP_RDM; /* Reliable datagram */ hints->caps = FI_TAGGED; /* Tag matching interface */ + hints->tx_attr->msg_order = FI_ORDER_SAS; + hints->rx_attr->msg_order = FI_ORDER_SAS; hints->ep_attr->mem_tag_format = MEM_TAG_FORMAT; MPIU_Assert(pg_p->size < ((1 << MPID_RANK_BITS) - 1)); ----------------------------------------------------------------------- Summary of changes: .../ch3/channels/nemesis/netmod/ofi/ofi_init.c | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) hooks/post-receive -- MPICH primary repository

1 0