December 2015 - devel - mailman3.cels.anl.gov

Re: [mpich-devel] Assertion failed in file src/mpid/ch3/channels/nemesis/src/ch3_progress.c at line 643: pkt->type >= 0 && pkt->type < MPIDI_CH3_PKT_END_ALL internal ABORT - process 1
by Jeff Hammond 30 Dec '15

30 Dec '15

Building from source solves the problem ( https://travis-ci.org/jeffhammond/armci-mpi/builds/99439479) so it seems that the deb file is the problem. I'll just avoid using it from now on. Best, Jeff On Tue, Dec 29, 2015 at 5:39 PM, Balaji, Pavan <balaji(a)anl.gov> wrote: > > Nothing suspicious in there. > > Can you try running the test with 3.2b3 manually to see if you still see > the error? > > -- Pavan > > > On Dec 29, 2015, at 2:52 PM, Jeff Hammond <jeff.science(a)gmail.com> > wrote: > > > > Build info: > > MPICH Version: 3.2b3 > > MPICH Release date: Thu Jun 4 07:59:11 CDT 2015 > > MPICH Device: ch3:nemesis > > MPICH configure: --prefix=/usr --sysconfdir=/etc --libdir=/usr/lib > --mandir=/usr/share/man --infodir=/usr/share/info > --docdir=/usr/share/doc/mpich --htmldir=/usr/share/doc/mpich > --dvidir=/usr/share/doc/mpich --pdfdir=/usr/share/doc/mpich > --psdir=/usr/share/doc/mpich > > MPICH CC: gcc -O2 > > MPICH CXX: g++ -O2 > > MPICH F77: gfortran -O2 > > MPICH FC: gfortran -O2 > > gcc -I/usr/include -L/usr/lib -Wl,-rpath -Wl,/usr/lib > -Wl,--enable-new-dtags -lmpi > > > > > > On Tue, Dec 29, 2015 at 12:44 PM, Jeff Hammond <jeff.science(a)gmail.com> > wrote: > > I do not, but I guess Marcelo Soares Souza does ( > http://lists.mpich.org/pipermail/discuss/2015-June/004045.html) > > > > Travis will log the output of mpichversion the next time it runs... > > > > Jeff > > > > On Tue, Dec 29, 2015 at 12:17 PM, Balaji, Pavan <balaji(a)anl.gov> wrote: > > > > Do you have more details on how mpich was built? > > > > -- Pavan > > > > > On Dec 29, 2015, at 2:05 PM, Jeff Hammond <jeff.science(a)gmail.com> > wrote: > > > > > > I am using mpich_3.2b3-1ubuntu_amd64.deb within Travis CI for > ARMCI-MPI testing. I never saw this problem in Jenkins, so I can't tell if > the problem is specific to the deb build, to Travis, or something else. > > > > > > Travis page has all the details so I will not repeat them here. > > > > > > Thanks, > > > > > > Jeff > > > > > > From https://travis-ci.org/jeffhammond/armci-mpi/jobs/99316268: > > > > > > Processing triggers for libc-bin ... > > > ldconfig deferred processing now taking place > > > +wget -q > http://www.cebacad.net/files/mpich/ubuntu/mpich-3.2b3/mpich_3.2b3-1ubuntu_a… > > > +sudo dpkg -i ./mpich_3.2b3-1ubuntu_amd64.deb > > > Selecting previously unselected package mpich. > > > (Reading database ... 124680 files and directories currently > installed.) > > > Unpacking mpich (from .../mpich_3.2b3-1ubuntu_amd64.deb) ... > > > Setting up mpich (3.2b3-1ubuntu) ... > > > Processing triggers for man-db ... > > > > > > > > > FAIL: benchmarks/contiguous-bench (exit: 1) > > > =========================================== > > > ARMCI-MPI initialized with 2 processes, MPI v3.1 > > > PROGRESS_THREAD = DISABLED > > > ALLOC_SHM used = TRUE > > > WINDOW type used = CREATE > > > STRIDED_METHOD = DIRECT > > > IOV_METHOD = DIRECT > > > IOV_CHECKS = FALSE > > > SHR_BUF_METHOD = COPY > > > NONCOLLECTIVE_GROUPS = FALSE > > > CACHE_RANK_TRANSLATION = TRUE > > > DEBUG_ALLOC = FALSE > > > Starting one-sided contiguous performance test with 2 processes > > > Trg. Rank Xfer Size Get (usec) Put (usec) Acc (usec) Get > (MiB/s) Put (MiB/s) Acc (MiB/s) > > > 1 8 0.869 0.707 0.935 > 8.783 10.795 8.158 > > > 1 16 0.926 0.768 0.914 > 16.482 19.859 16.698 > > > 1 32 0.945 0.856 0.974 > 32.284 35.671 31.336 > > > 1 64 0.938 0.781 0.956 > 65.097 78.199 63.852 > > > 1 128 0.946 0.833 1.055 > 128.992 146.623 115.730 > > > 1 256 0.969 0.819 1.065 > 251.986 297.954 229.235 > > > 1 512 0.953 0.867 1.024 > 512.188 562.918 477.005 > > > 1 1024 0.995 0.873 1.139 > 981.813 1119.078 857.161 > > > 1 2048 1.010 0.850 1.044 > 1932.970 2297.619 1870.578 > > > 1 4096 1.071 1.172 1.494 > 3647.221 3334.105 2614.495 > > > 1 8192 1.634 1.721 2.068 > 4780.515 4539.290 3777.801 > > > 1 16384 2.989 3.325 4.856 > 5227.361 4698.842 3217.416 > > > 1 32768 5.321 7.480 9.465 > 5872.833 4177.594 3301.464 > > > 1 65536 13.295 14.356 20.259 > 4701.146 4353.478 3085.039 > > > 1 131072 23.574 31.835 45.756 > 5302.534 3926.447 2731.862 > > > 1 262144 44.921 70.181 102.028 > 5565.275 3562.231 2450.313 > > > Assertion failed in file > src/mpid/ch3/channels/nemesis/src/ch3_progress.c at line 643: pkt->type >= > 0 && pkt->type < MPIDI_CH3_PKT_END_ALL > > > internal ABORT - process 1 > > > > > > > > > -- > > > Jeff Hammond > > > jeff.science(a)gmail.com > > > http://jeffhammond.github.io/ > > > _______________________________________________ > > > To manage subscription options or unsubscribe: > > > https://lists.mpich.org/mailman/listinfo/devel > > > > _______________________________________________ > > To manage subscription options or unsubscribe: > > https://lists.mpich.org/mailman/listinfo/devel > > > > > > > > -- > > Jeff Hammond > > jeff.science(a)gmail.com > > http://jeffhammond.github.io/ > > > > > > > > -- > > Jeff Hammond > > jeff.science(a)gmail.com > > http://jeffhammond.github.io/ > > _______________________________________________ > > To manage subscription options or unsubscribe: > > https://lists.mpich.org/mailman/listinfo/devel > > _______________________________________________ > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/devel > -- Jeff Hammond jeff.science(a)gmail.com http://jeffhammond.github.io/

1 0

Re: [mpich-devel] Assertion failed in file src/mpid/ch3/channels/nemesis/src/ch3_progress.c at line 643: pkt->type >= 0 && pkt->type < MPIDI_CH3_PKT_END_ALL internal ABORT - process 1
by Balaji, Pavan 29 Dec '15

29 Dec '15

Nothing suspicious in there. Can you try running the test with 3.2b3 manually to see if you still see the error? -- Pavan > On Dec 29, 2015, at 2:52 PM, Jeff Hammond <jeff.science(a)gmail.com> wrote: > > Build info: > MPICH Version: 3.2b3 > MPICH Release date: Thu Jun 4 07:59:11 CDT 2015 > MPICH Device: ch3:nemesis > MPICH configure: --prefix=/usr --sysconfdir=/etc --libdir=/usr/lib --mandir=/usr/share/man --infodir=/usr/share/info --docdir=/usr/share/doc/mpich --htmldir=/usr/share/doc/mpich --dvidir=/usr/share/doc/mpich --pdfdir=/usr/share/doc/mpich --psdir=/usr/share/doc/mpich > MPICH CC: gcc -O2 > MPICH CXX: g++ -O2 > MPICH F77: gfortran -O2 > MPICH FC: gfortran -O2 > gcc -I/usr/include -L/usr/lib -Wl,-rpath -Wl,/usr/lib -Wl,--enable-new-dtags -lmpi > > > On Tue, Dec 29, 2015 at 12:44 PM, Jeff Hammond <jeff.science(a)gmail.com> wrote: > I do not, but I guess Marcelo Soares Souza does (http://lists.mpich.org/pipermail/discuss/2015-June/004045.html) > > Travis will log the output of mpichversion the next time it runs... > > Jeff > > On Tue, Dec 29, 2015 at 12:17 PM, Balaji, Pavan <balaji(a)anl.gov> wrote: > > Do you have more details on how mpich was built? > > -- Pavan > > > On Dec 29, 2015, at 2:05 PM, Jeff Hammond <jeff.science(a)gmail.com> wrote: > > > > I am using mpich_3.2b3-1ubuntu_amd64.deb within Travis CI for ARMCI-MPI testing. I never saw this problem in Jenkins, so I can't tell if the problem is specific to the deb build, to Travis, or something else. > > > > Travis page has all the details so I will not repeat them here. > > > > Thanks, > > > > Jeff > > > > From https://travis-ci.org/jeffhammond/armci-mpi/jobs/99316268: > > > > Processing triggers for libc-bin ... > > ldconfig deferred processing now taking place > > +wget -q http://www.cebacad.net/files/mpich/ubuntu/mpich-3.2b3/mpich_3.2b3-1ubuntu_a… > > +sudo dpkg -i ./mpich_3.2b3-1ubuntu_amd64.deb > > Selecting previously unselected package mpich. > > (Reading database ... 124680 files and directories currently installed.) > > Unpacking mpich (from .../mpich_3.2b3-1ubuntu_amd64.deb) ... > > Setting up mpich (3.2b3-1ubuntu) ... > > Processing triggers for man-db ... > > > > > > FAIL: benchmarks/contiguous-bench (exit: 1) > > =========================================== > > ARMCI-MPI initialized with 2 processes, MPI v3.1 > > PROGRESS_THREAD = DISABLED > > ALLOC_SHM used = TRUE > > WINDOW type used = CREATE > > STRIDED_METHOD = DIRECT > > IOV_METHOD = DIRECT > > IOV_CHECKS = FALSE > > SHR_BUF_METHOD = COPY > > NONCOLLECTIVE_GROUPS = FALSE > > CACHE_RANK_TRANSLATION = TRUE > > DEBUG_ALLOC = FALSE > > Starting one-sided contiguous performance test with 2 processes > > Trg. Rank Xfer Size Get (usec) Put (usec) Acc (usec) Get (MiB/s) Put (MiB/s) Acc (MiB/s) > > 1 8 0.869 0.707 0.935 8.783 10.795 8.158 > > 1 16 0.926 0.768 0.914 16.482 19.859 16.698 > > 1 32 0.945 0.856 0.974 32.284 35.671 31.336 > > 1 64 0.938 0.781 0.956 65.097 78.199 63.852 > > 1 128 0.946 0.833 1.055 128.992 146.623 115.730 > > 1 256 0.969 0.819 1.065 251.986 297.954 229.235 > > 1 512 0.953 0.867 1.024 512.188 562.918 477.005 > > 1 1024 0.995 0.873 1.139 981.813 1119.078 857.161 > > 1 2048 1.010 0.850 1.044 1932.970 2297.619 1870.578 > > 1 4096 1.071 1.172 1.494 3647.221 3334.105 2614.495 > > 1 8192 1.634 1.721 2.068 4780.515 4539.290 3777.801 > > 1 16384 2.989 3.325 4.856 5227.361 4698.842 3217.416 > > 1 32768 5.321 7.480 9.465 5872.833 4177.594 3301.464 > > 1 65536 13.295 14.356 20.259 4701.146 4353.478 3085.039 > > 1 131072 23.574 31.835 45.756 5302.534 3926.447 2731.862 > > 1 262144 44.921 70.181 102.028 5565.275 3562.231 2450.313 > > Assertion failed in file src/mpid/ch3/channels/nemesis/src/ch3_progress.c at line 643: pkt->type >= 0 && pkt->type < MPIDI_CH3_PKT_END_ALL > > internal ABORT - process 1 > > > > > > -- > > Jeff Hammond > > jeff.science(a)gmail.com > > http://jeffhammond.github.io/ > > _______________________________________________ > > To manage subscription options or unsubscribe: > > https://lists.mpich.org/mailman/listinfo/devel > > _______________________________________________ > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/devel > > > > -- > Jeff Hammond > jeff.science(a)gmail.com > http://jeffhammond.github.io/ > > > > -- > Jeff Hammond > jeff.science(a)gmail.com > http://jeffhammond.github.io/ > _______________________________________________ > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/devel

1 0

Re: [mpich-devel] Assertion failed in file src/mpid/ch3/channels/nemesis/src/ch3_progress.c at line 643: pkt->type >= 0 && pkt->type < MPIDI_CH3_PKT_END_ALL internal ABORT - process 1
by Jeff Hammond 29 Dec '15

29 Dec '15

Build info: MPICH Version: 3.2b3 MPICH Release date: Thu Jun 4 07:59:11 CDT 2015 MPICH Device: ch3:nemesis MPICH configure: --prefix=/usr --sysconfdir=/etc --libdir=/usr/lib --mandir=/usr/share/man --infodir=/usr/share/info --docdir=/usr/share/doc/mpich --htmldir=/usr/share/doc/mpich --dvidir=/usr/share/doc/mpich --pdfdir=/usr/share/doc/mpich --psdir=/usr/share/doc/mpich MPICH CC: gcc -O2 MPICH CXX: g++ -O2 MPICH F77: gfortran -O2 MPICH FC: gfortran -O2 gcc -I/usr/include -L/usr/lib -Wl,-rpath -Wl,/usr/lib -Wl,--enable-new-dtags -lmpi On Tue, Dec 29, 2015 at 12:44 PM, Jeff Hammond <jeff.science(a)gmail.com> wrote: > I do not, but I guess Marcelo Soares Souza does ( > http://lists.mpich.org/pipermail/discuss/2015-June/004045.html) > > Travis will log the output of mpichversion the next time it runs... > > Jeff > > On Tue, Dec 29, 2015 at 12:17 PM, Balaji, Pavan <balaji(a)anl.gov> wrote: > >> >> Do you have more details on how mpich was built? >> >> -- Pavan >> >> > On Dec 29, 2015, at 2:05 PM, Jeff Hammond <jeff.science(a)gmail.com> >> wrote: >> > >> > I am using mpich_3.2b3-1ubuntu_amd64.deb within Travis CI for ARMCI-MPI >> testing. I never saw this problem in Jenkins, so I can't tell if the >> problem is specific to the deb build, to Travis, or something else. >> > >> > Travis page has all the details so I will not repeat them here. >> > >> > Thanks, >> > >> > Jeff >> > >> > From https://travis-ci.org/jeffhammond/armci-mpi/jobs/99316268: >> > >> > Processing triggers for libc-bin ... >> > ldconfig deferred processing now taking place >> > +wget -q >> http://www.cebacad.net/files/mpich/ubuntu/mpich-3.2b3/mpich_3.2b3-1ubuntu_a… >> > +sudo dpkg -i ./mpich_3.2b3-1ubuntu_amd64.deb >> > Selecting previously unselected package mpich. >> > (Reading database ... 124680 files and directories currently installed.) >> > Unpacking mpich (from .../mpich_3.2b3-1ubuntu_amd64.deb) ... >> > Setting up mpich (3.2b3-1ubuntu) ... >> > Processing triggers for man-db ... >> > >> > >> > FAIL: benchmarks/contiguous-bench (exit: 1) >> > =========================================== >> > ARMCI-MPI initialized with 2 processes, MPI v3.1 >> > PROGRESS_THREAD = DISABLED >> > ALLOC_SHM used = TRUE >> > WINDOW type used = CREATE >> > STRIDED_METHOD = DIRECT >> > IOV_METHOD = DIRECT >> > IOV_CHECKS = FALSE >> > SHR_BUF_METHOD = COPY >> > NONCOLLECTIVE_GROUPS = FALSE >> > CACHE_RANK_TRANSLATION = TRUE >> > DEBUG_ALLOC = FALSE >> > Starting one-sided contiguous performance test with 2 processes >> > Trg. Rank Xfer Size Get (usec) Put (usec) Acc (usec) Get >> (MiB/s) Put (MiB/s) Acc (MiB/s) >> > 1 8 0.869 0.707 0.935 >> 8.783 10.795 8.158 >> > 1 16 0.926 0.768 0.914 >> 16.482 19.859 16.698 >> > 1 32 0.945 0.856 0.974 >> 32.284 35.671 31.336 >> > 1 64 0.938 0.781 0.956 >> 65.097 78.199 63.852 >> > 1 128 0.946 0.833 1.055 >> 128.992 146.623 115.730 >> > 1 256 0.969 0.819 1.065 >> 251.986 297.954 229.235 >> > 1 512 0.953 0.867 1.024 >> 512.188 562.918 477.005 >> > 1 1024 0.995 0.873 1.139 >> 981.813 1119.078 857.161 >> > 1 2048 1.010 0.850 1.044 >> 1932.970 2297.619 1870.578 >> > 1 4096 1.071 1.172 1.494 >> 3647.221 3334.105 2614.495 >> > 1 8192 1.634 1.721 2.068 >> 4780.515 4539.290 3777.801 >> > 1 16384 2.989 3.325 4.856 >> 5227.361 4698.842 3217.416 >> > 1 32768 5.321 7.480 9.465 >> 5872.833 4177.594 3301.464 >> > 1 65536 13.295 14.356 20.259 >> 4701.146 4353.478 3085.039 >> > 1 131072 23.574 31.835 45.756 >> 5302.534 3926.447 2731.862 >> > 1 262144 44.921 70.181 102.028 >> 5565.275 3562.231 2450.313 >> > Assertion failed in file >> src/mpid/ch3/channels/nemesis/src/ch3_progress.c at line 643: pkt->type >= >> 0 && pkt->type < MPIDI_CH3_PKT_END_ALL >> > internal ABORT - process 1 >> > >> > >> > -- >> > Jeff Hammond >> > jeff.science(a)gmail.com >> > http://jeffhammond.github.io/ >> > _______________________________________________ >> > To manage subscription options or unsubscribe: >> > https://lists.mpich.org/mailman/listinfo/devel >> >> _______________________________________________ >> To manage subscription options or unsubscribe: >> https://lists.mpich.org/mailman/listinfo/devel >> > > > > -- > Jeff Hammond > jeff.science(a)gmail.com > http://jeffhammond.github.io/ > -- Jeff Hammond jeff.science(a)gmail.com http://jeffhammond.github.io/

1 0

Re: [mpich-devel] Assertion failed in file src/mpid/ch3/channels/nemesis/src/ch3_progress.c at line 643: pkt->type >= 0 && pkt->type < MPIDI_CH3_PKT_END_ALL internal ABORT - process 1
by Jeff Hammond 29 Dec '15

29 Dec '15

I do not, but I guess Marcelo Soares Souza does ( http://lists.mpich.org/pipermail/discuss/2015-June/004045.html) Travis will log the output of mpichversion the next time it runs... Jeff On Tue, Dec 29, 2015 at 12:17 PM, Balaji, Pavan <balaji(a)anl.gov> wrote: > > Do you have more details on how mpich was built? > > -- Pavan > > > On Dec 29, 2015, at 2:05 PM, Jeff Hammond <jeff.science(a)gmail.com> > wrote: > > > > I am using mpich_3.2b3-1ubuntu_amd64.deb within Travis CI for ARMCI-MPI > testing. I never saw this problem in Jenkins, so I can't tell if the > problem is specific to the deb build, to Travis, or something else. > > > > Travis page has all the details so I will not repeat them here. > > > > Thanks, > > > > Jeff > > > > From https://travis-ci.org/jeffhammond/armci-mpi/jobs/99316268: > > > > Processing triggers for libc-bin ... > > ldconfig deferred processing now taking place > > +wget -q > http://www.cebacad.net/files/mpich/ubuntu/mpich-3.2b3/mpich_3.2b3-1ubuntu_a… > > +sudo dpkg -i ./mpich_3.2b3-1ubuntu_amd64.deb > > Selecting previously unselected package mpich. > > (Reading database ... 124680 files and directories currently installed.) > > Unpacking mpich (from .../mpich_3.2b3-1ubuntu_amd64.deb) ... > > Setting up mpich (3.2b3-1ubuntu) ... > > Processing triggers for man-db ... > > > > > > FAIL: benchmarks/contiguous-bench (exit: 1) > > =========================================== > > ARMCI-MPI initialized with 2 processes, MPI v3.1 > > PROGRESS_THREAD = DISABLED > > ALLOC_SHM used = TRUE > > WINDOW type used = CREATE > > STRIDED_METHOD = DIRECT > > IOV_METHOD = DIRECT > > IOV_CHECKS = FALSE > > SHR_BUF_METHOD = COPY > > NONCOLLECTIVE_GROUPS = FALSE > > CACHE_RANK_TRANSLATION = TRUE > > DEBUG_ALLOC = FALSE > > Starting one-sided contiguous performance test with 2 processes > > Trg. Rank Xfer Size Get (usec) Put (usec) Acc (usec) Get > (MiB/s) Put (MiB/s) Acc (MiB/s) > > 1 8 0.869 0.707 0.935 > 8.783 10.795 8.158 > > 1 16 0.926 0.768 0.914 > 16.482 19.859 16.698 > > 1 32 0.945 0.856 0.974 > 32.284 35.671 31.336 > > 1 64 0.938 0.781 0.956 > 65.097 78.199 63.852 > > 1 128 0.946 0.833 1.055 > 128.992 146.623 115.730 > > 1 256 0.969 0.819 1.065 > 251.986 297.954 229.235 > > 1 512 0.953 0.867 1.024 > 512.188 562.918 477.005 > > 1 1024 0.995 0.873 1.139 > 981.813 1119.078 857.161 > > 1 2048 1.010 0.850 1.044 > 1932.970 2297.619 1870.578 > > 1 4096 1.071 1.172 1.494 > 3647.221 3334.105 2614.495 > > 1 8192 1.634 1.721 2.068 > 4780.515 4539.290 3777.801 > > 1 16384 2.989 3.325 4.856 > 5227.361 4698.842 3217.416 > > 1 32768 5.321 7.480 9.465 > 5872.833 4177.594 3301.464 > > 1 65536 13.295 14.356 20.259 > 4701.146 4353.478 3085.039 > > 1 131072 23.574 31.835 45.756 > 5302.534 3926.447 2731.862 > > 1 262144 44.921 70.181 102.028 > 5565.275 3562.231 2450.313 > > Assertion failed in file > src/mpid/ch3/channels/nemesis/src/ch3_progress.c at line 643: pkt->type >= > 0 && pkt->type < MPIDI_CH3_PKT_END_ALL > > internal ABORT - process 1 > > > > > > -- > > Jeff Hammond > > jeff.science(a)gmail.com > > http://jeffhammond.github.io/ > > _______________________________________________ > > To manage subscription options or unsubscribe: > > https://lists.mpich.org/mailman/listinfo/devel > > _______________________________________________ > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/devel > -- Jeff Hammond jeff.science(a)gmail.com http://jeffhammond.github.io/

1 0

Re: [mpich-devel] Assertion failed in file src/mpid/ch3/channels/nemesis/src/ch3_progress.c at line 643: pkt->type >= 0 && pkt->type < MPIDI_CH3_PKT_END_ALL internal ABORT - process 1
by Balaji, Pavan 29 Dec '15

29 Dec '15

Do you have more details on how mpich was built? -- Pavan > On Dec 29, 2015, at 2:05 PM, Jeff Hammond <jeff.science(a)gmail.com> wrote: > > I am using mpich_3.2b3-1ubuntu_amd64.deb within Travis CI for ARMCI-MPI testing. I never saw this problem in Jenkins, so I can't tell if the problem is specific to the deb build, to Travis, or something else. > > Travis page has all the details so I will not repeat them here. > > Thanks, > > Jeff > > From https://travis-ci.org/jeffhammond/armci-mpi/jobs/99316268: > > Processing triggers for libc-bin ... > ldconfig deferred processing now taking place > +wget -q http://www.cebacad.net/files/mpich/ubuntu/mpich-3.2b3/mpich_3.2b3-1ubuntu_a… > +sudo dpkg -i ./mpich_3.2b3-1ubuntu_amd64.deb > Selecting previously unselected package mpich. > (Reading database ... 124680 files and directories currently installed.) > Unpacking mpich (from .../mpich_3.2b3-1ubuntu_amd64.deb) ... > Setting up mpich (3.2b3-1ubuntu) ... > Processing triggers for man-db ... > > > FAIL: benchmarks/contiguous-bench (exit: 1) > =========================================== > ARMCI-MPI initialized with 2 processes, MPI v3.1 > PROGRESS_THREAD = DISABLED > ALLOC_SHM used = TRUE > WINDOW type used = CREATE > STRIDED_METHOD = DIRECT > IOV_METHOD = DIRECT > IOV_CHECKS = FALSE > SHR_BUF_METHOD = COPY > NONCOLLECTIVE_GROUPS = FALSE > CACHE_RANK_TRANSLATION = TRUE > DEBUG_ALLOC = FALSE > Starting one-sided contiguous performance test with 2 processes > Trg. Rank Xfer Size Get (usec) Put (usec) Acc (usec) Get (MiB/s) Put (MiB/s) Acc (MiB/s) > 1 8 0.869 0.707 0.935 8.783 10.795 8.158 > 1 16 0.926 0.768 0.914 16.482 19.859 16.698 > 1 32 0.945 0.856 0.974 32.284 35.671 31.336 > 1 64 0.938 0.781 0.956 65.097 78.199 63.852 > 1 128 0.946 0.833 1.055 128.992 146.623 115.730 > 1 256 0.969 0.819 1.065 251.986 297.954 229.235 > 1 512 0.953 0.867 1.024 512.188 562.918 477.005 > 1 1024 0.995 0.873 1.139 981.813 1119.078 857.161 > 1 2048 1.010 0.850 1.044 1932.970 2297.619 1870.578 > 1 4096 1.071 1.172 1.494 3647.221 3334.105 2614.495 > 1 8192 1.634 1.721 2.068 4780.515 4539.290 3777.801 > 1 16384 2.989 3.325 4.856 5227.361 4698.842 3217.416 > 1 32768 5.321 7.480 9.465 5872.833 4177.594 3301.464 > 1 65536 13.295 14.356 20.259 4701.146 4353.478 3085.039 > 1 131072 23.574 31.835 45.756 5302.534 3926.447 2731.862 > 1 262144 44.921 70.181 102.028 5565.275 3562.231 2450.313 > Assertion failed in file src/mpid/ch3/channels/nemesis/src/ch3_progress.c at line 643: pkt->type >= 0 && pkt->type < MPIDI_CH3_PKT_END_ALL > internal ABORT - process 1 > > > -- > Jeff Hammond > jeff.science(a)gmail.com > http://jeffhammond.github.io/ > _______________________________________________ > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/devel

1 0

Assertion failed in file src/mpid/ch3/channels/nemesis/src/ch3_progress.c at line 643: pkt->type >= 0 && pkt->type < MPIDI_CH3_PKT_END_ALL internal ABORT - process 1
by Jeff Hammond 29 Dec '15

29 Dec '15

I am using mpich_3.2b3-1ubuntu_amd64.deb within Travis CI for ARMCI-MPI testing. I never saw this problem in Jenkins, so I can't tell if the problem is specific to the deb build, to Travis, or something else. Travis page has all the details so I will not repeat them here. Thanks, Jeff >From https://travis-ci.org/jeffhammond/armci-mpi/jobs/99316268: Processing triggers for libc-bin ... ldconfig deferred processing now taking place +wget -q http://www.cebacad.net/files/mpich/ubuntu/mpich-3.2b3/mpich_3.2b3-1ubuntu_a… +sudo dpkg -i ./mpich_3.2b3-1ubuntu_amd64.deb Selecting previously unselected package mpich. (Reading database ... 124680 files and directories currently installed.) Unpacking mpich (from .../mpich_3.2b3-1ubuntu_amd64.deb) ... Setting up mpich (3.2b3-1ubuntu) ... Processing triggers for man-db ... FAIL: benchmarks/contiguous-bench (exit: 1) =========================================== ARMCI-MPI initialized with 2 processes, MPI v3.1 PROGRESS_THREAD = DISABLED ALLOC_SHM used = TRUE WINDOW type used = CREATE STRIDED_METHOD = DIRECT IOV_METHOD = DIRECT IOV_CHECKS = FALSE SHR_BUF_METHOD = COPY NONCOLLECTIVE_GROUPS = FALSE CACHE_RANK_TRANSLATION = TRUE DEBUG_ALLOC = FALSE Starting one-sided contiguous performance test with 2 processes Trg. Rank Xfer Size Get (usec) Put (usec) Acc (usec) Get (MiB/s) Put (MiB/s) Acc (MiB/s) 1 8 0.869 0.707 0.935 8.783 10.795 8.158 1 16 0.926 0.768 0.914 16.482 19.859 16.698 1 32 0.945 0.856 0.974 32.284 35.671 31.336 1 64 0.938 0.781 0.956 65.097 78.199 63.852 1 128 0.946 0.833 1.055 128.992 146.623 115.730 1 256 0.969 0.819 1.065 251.986 297.954 229.235 1 512 0.953 0.867 1.024 512.188 562.918 477.005 1 1024 0.995 0.873 1.139 981.813 1119.078 857.161 1 2048 1.010 0.850 1.044 1932.970 2297.619 1870.578 1 4096 1.071 1.172 1.494 3647.221 3334.105 2614.495 1 8192 1.634 1.721 2.068 4780.515 4539.290 3777.801 1 16384 2.989 3.325 4.856 5227.361 4698.842 3217.416 1 32768 5.321 7.480 9.465 5872.833 4177.594 3301.464 1 65536 13.295 14.356 20.259 4701.146 4353.478 3085.039 1 131072 23.574 31.835 45.756 5302.534 3926.447 2731.862 1 262144 44.921 70.181 102.028 5565.275 3562.231 2450.313 Assertion failed in file src/mpid/ch3/channels/nemesis/src/ch3_progress.c at line 643: pkt->type >= 0 && pkt->type < MPIDI_CH3_PKT_END_ALL internal ABORT - process 1 -- Jeff Hammond jeff.science(a)gmail.com http://jeffhammond.github.io/

1 0