February 2013 - discuss - mailman3.cels.anl.gov

Re: [mpich-discuss] MPICH2 -Problem
by hamideh jabalameli 24 Feb '13

24 Feb '13

I see file 'config.log' from the same directory where the 'configure' was started. I attached its to this email. In the content of this file, pthread libraries such as "pthread.h,pthread_yield" have make. but there are related error... Do you know that what do we do? On Sun, Feb 24, 2013 at 12:22 PM, Andrey V. Malyshev <amal(a)icm.krasn.ru>wrote: > Hello, hamideh! > You wrote to <discuss(a)mpich.org> on Sun, 24 Feb 2013 10:12:01 +0330: > > hj> I think configure doesn't done correctly! > > That's right. > > hj> configure command is : > hj> ../configure --with-device=ch3:sctp > hj> --with-sctp=/home/inet/src/applications/sctpapp/SCTPlikeAPI > hj> --prefix=/home/mpich2-hginit –disable-mpe > hj> What I am doing? :( > > See the file 'config.log' from the same directory where the 'configure' was > started, or where the 'configure' file is located. > Pthread library was not found, you should understand, why, and maybe then > it > becomes clear how to fix. > > hj> On Sun, Feb 24, 2013 at 10:07 AM, Andrey V. Malyshev > hj> <amal(a)icm.krasn.ru>wrote: > > ??>> Hello, hamideh! > ??>> You wrote to <discuss(a)mpich.org> on Sun, 24 Feb 2013 08:23:37 +0330: > ??>> > hj>>> checking for pthread.h... yes > ??>> checking for pthread_yield... no > ??>> checking for library containing pthread_key_create... no > ??>> configure: error: unable to find pthreads library > ??>> > ??>> See config.log, what error occurs during this check? > ??>> > ??>> -- > ??>> With best regards, Andrey V. Malyshev. > ??>> E-mail: amal(a)krasn.ru > ??>> _______________________________________________ > ??>> discuss mailing list discuss(a)mpich.org > ??>> To manage subscription options or unsubscribe: > ??>> https://lists.mpich.org/mailman/listinfo/discuss > ??>> > hj> -- > hj> *Best Regards* > hj> * > > hj> Hamideh Jabalameli* > > hj> M.Sc Candidate, > hj> School of Electrical and Computer Engineering, > hj> College of Engineering, > hj> Electronici Shahid Beheshti Tehran University > hj> h.jabal(a)Mail.sbu.ac.ir > hj> hamideh.jabalameli(a)gmail.com > > -- > With best regards, Andrey V. Malyshev. > E-mail: amal(a)krasn.ru > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > -- *Best Regards* * Hamideh Jabalameli* M.Sc Candidate, School of Electrical and Computer Engineering, College of Engineering, Electronici Shahid Beheshti Tehran University h.jabal(a)Mail.sbu.ac.ir hamideh.jabalameli(a)gmail.com

1 0

Re: [mpich-discuss] MPICH2 -Problem
by Andrey V. Malyshev 24 Feb '13

24 Feb '13

Hello, hamideh! You wrote to <discuss(a)mpich.org> on Sun, 24 Feb 2013 10:12:01 +0330: hj> I think configure doesn't done correctly! That's right. hj> configure command is : hj> ../configure --with-device=ch3:sctp hj> --with-sctp=/home/inet/src/applications/sctpapp/SCTPlikeAPI hj> --prefix=/home/mpich2-hginit ï¿½disable-mpe hj> What I am doing? :( See the file 'config.log' from the same directory where the 'configure' was started, or where the 'configure' file is located. Pthread library was not found, you should understand, why, and maybe then it becomes clear how to fix. hj> On Sun, Feb 24, 2013 at 10:07 AM, Andrey V. Malyshev hj> <amal(a)icm.krasn.ru>wrote: ??>> Hello, hamideh! ??>> You wrote to <discuss(a)mpich.org> on Sun, 24 Feb 2013 08:23:37 +0330: ??>> hj>>> checking for pthread.h... yes ??>> checking for pthread_yield... no ??>> checking for library containing pthread_key_create... no ??>> configure: error: unable to find pthreads library ??>> ??>> See config.log, what error occurs during this check? ??>> ??>> -- ??>> With best regards, Andrey V. Malyshev. ??>> E-mail: amal(a)krasn.ru ??>> _______________________________________________ ??>> discuss mailing list discuss(a)mpich.org ??>> To manage subscription options or unsubscribe: ??>> https://lists.mpich.org/mailman/listinfo/discuss ??>> hj> -- hj> *Best Regards* hj> * hj> Hamideh Jabalameli* hj> M.Sc Candidate, hj> School of Electrical and Computer Engineering, hj> College of Engineering, hj> Electronici Shahid Beheshti Tehran University hj> h.jabal(a)Mail.sbu.ac.ir hj> hamideh.jabalameli(a)gmail.com -- With best regards, Andrey V. Malyshev. E-mail: amal(a)krasn.ru

1 0

Re: [mpich-discuss] MPICH2 -Problem
by hamideh jabalameli 24 Feb '13

24 Feb '13

I think configure doesn't done correctly! configure command is : ../configure --with-device=ch3:sctp --with-sctp=/home/inet/src/applications/sctpapp/SCTPlikeAPI --prefix=/home/mpich2-hginit –disable-mpe What I am doing? :( On Sun, Feb 24, 2013 at 10:07 AM, Andrey V. Malyshev <amal(a)icm.krasn.ru>wrote: > Hello, hamideh! > You wrote to <discuss(a)mpich.org> on Sun, 24 Feb 2013 08:23:37 +0330: > > hj> checking for pthread.h... yes > checking for pthread_yield... no > checking for library containing pthread_key_create... no > configure: error: unable to find pthreads library > > > See config.log, what error occurs during this check? > > -- > With best regards, Andrey V. Malyshev. > E-mail: amal(a)krasn.ru > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > -- *Best Regards* * Hamideh Jabalameli* M.Sc Candidate, School of Electrical and Computer Engineering, College of Engineering, Electronici Shahid Beheshti Tehran University h.jabal(a)Mail.sbu.ac.ir hamideh.jabalameli(a)gmail.com

1 0

Re: [mpich-discuss] MPICH2 -Problem
by Andrey V. Malyshev 24 Feb '13

24 Feb '13

Hello, hamideh! You wrote to <discuss(a)mpich.org> on Sun, 24 Feb 2013 08:23:37 +0330: hj> checking for pthread.h... yes checking for pthread_yield... no checking for library containing pthread_key_create... no configure: error: unable to find pthreads library See config.log, what error occurs during this check? -- With best regards, Andrey V. Malyshev. E-mail: amal(a)krasn.ru

1 0

Re: [mpich-discuss] Affinity with MPICH_ASYNC_PROGRESS
by Jed Brown 23 Feb '13

23 Feb '13

On Sat, Feb 23, 2013 at 8:21 PM, Jeff Hammond <jhammond(a)alcf.anl.gov> wrote: > I personally think that MKL, meaning the BLAS, is an amazing > programming model. > Are you trolling me? > However, my point here was merely to say that > Intel has an integrated software stack that manages thread resources > intelligently relative to N independent implementations of MPI, > BLAS/LAPACK, and OpenMP/TBB, for example. I was under the impression > that Intel presents stupid oversubscription, which a good thing for > users but essentially impossible to implement portably. > It's relatively easy to be integrated when you're not interoperable. > I think the only way for async progress to work well is to have > fine-grain locking since of MPI, as is done in PAMID. Any > implementation that resorts to fat-locking is probably better off > without async progress unless the application is doing something > really silly (like never calling MPI on a rank that is the target of > MPI RMA). > There's more than 100k cycles between times that I enter the MPI stack. There are only two threads ever that contend for locks (my funneled thread and MPICH's async-progress thread). I'm not convinced that you need super fine-grained locks to make progress during that time period. FWIW, my experience has been that standard Nemesis does a _much_ better job of making asynchronous progress than Cray's implementation. If we could look at the code for Cray's implementation, we might be able to get a better idea of why.

1 0

MPICH2 -Problem
by hamideh jabalameli 23 Feb '13

23 Feb '13

Hello I have some problem for install MPICH2 in my project.I use Ubuntu 12.04. Please go to the following address: https://nplab.fh-muenster.de/groups/wiki/wiki/2b6b7/attachments/84e27/mpich… I do all of the stages correctly. But I have only some problem for custom version of MPICH2 which is necessary to connect real MPI programs to execute on the simulated network (Omnetpp): https://nplab.fh-muenster.de/groups/wiki/wiki/2b6b7/attachments/84e27/mpich… *Please help me for my problem immediately…* the related command: ../configure --with-device=ch3:sctp --with-sctp=/home/inet/src/applications/sctpapp/SCTPlikeAPI --prefix=/home/mpich2-hginit –disable-mpe Error: Configuring MPICH2 version 1.0.8rc1 with '--with-device=ch3:sctp' '--with-sctp=/home/inet/src/applications/sctpapp/SCTPlikeAPI' '--prefix=/home/mpich2-hginit' '--disable-mpe' Running on system: Linux myubuntu-VPCCW21FX 3.2.0-38-generic-pae #60-Ubuntu SMP Wed Feb 13 13:47:26 UTC 2013 i686 i686 i386 GNU/Linux Executing mpich2prereq in /home/my-ubuntu/mpich2-hginit/src/mpid/ch3 with sctp Executing mpich2prereq in /home/my-ubuntu/mpich2-hginit/src/mpid/ch3/channels/sctp sourcing /home/my-ubuntu/mpich2-hginit/src/pm/mpd/mpich2prereq sourcing /home/my-ubuntu/mpich2-hginit/src/pm/mpd/setup_pm checking for gcc... gcc checking for C compiler default output file name... a.out checking whether the C compiler works... yes checking whether we are cross compiling... no checking for suffix of executables... checking for suffix of object files... o checking whether we are using the GNU C compiler... yes checking whether gcc accepts -g... yes checking for gcc option to accept ISO C89... none needed checking whether C compiler accepts option -O2... yes checking whether routines compiled with -O2 can be linked with ones compiled without -O2... yes checking for type of weak symbol support... pragma weak checking whether __attribute__ ((weak)) allowed... yes checking for multiple weak symbol support... yes checking for g77... no checking for xlf... no checking for f77... no checking for frt... no checking for pgf77... no checking for cf77... no checking for fort77... no checking for fl32... no checking for af77... no checking for xlf90... no checking for f90... no checking for pgf90... no checking for pghpf... no checking for epcf90... no checking for gfortran... no checking for g95... no checking for xlf95... no checking for f95... no checking for fort... no checking for ifort... no checking for ifc... no checking for efc... no checking for pgf95... no checking for lf95... no checking for ftn... no checking whether we are using the GNU Fortran 77 compiler... no checking whether accepts -g... no checking for c++... c++ checking whether we are using the GNU C++ compiler... yes checking whether c++ accepts -g... yes checking whether the C++ compiler c++ can build an executable... yes checking whether C++ compiler works with string... yes checking whether the compiler supports exceptions... yes checking whether the compiler recognizes bool as a built-in type... yes checking whether the compiler implements namespaces... yes checking whether <iostream> available... yes checking whether the compiler implements the namespace std... yes checking whether <math> available... no checking for GNU g++ version... 4 . 6 checking whether C++ compiler accepts option -O2... yes checking whether routines compiled with -O2 can be linked with ones compiled without -O2... yes checking for perl... /usr/bin/perl checking for ar... ar checking for ranlib... ranlib checking for etags... etags checking for etags argument to specify language... --language=c checking for killall... killall checking for a BSD-compatible install... /usr/bin/install -c checking whether install works... yes checking whether install breaks libraries... no checking whether mkdir -p works... yes checking for make... make checking whether clock skew breaks make... no checking whether make supports include... yes checking whether make allows comments in actions... yes checking for virtual path format... VPATH checking whether make sets CFLAGS... no checking for bash... /bin/bash checking whether /bin/bash supports arrays... yes checking for doctext... false checking for location of doctext style files... unavailable checking for an ANSI C-conforming const... yes checking for volatile... yes checking for restrict... __restrict checking for inline... inline checking whether __attribute__ allowed... yes checking whether __attribute__((format)) allowed... yes checking for grep that handles long lines and -e... /bin/grep checking for egrep... /bin/grep -E checking for ANSI C header files... no checking for sys/types.h... yes checking for sys/stat.h... yes checking for stdlib.h... yes checking for string.h... yes checking for memory.h... yes checking for strings.h... yes checking for inttypes.h... yes checking for stdint.h... yes checking for unistd.h... yes checking whether byte ordering is bigendian... no checking whether C compiler allows unaligned doubles... yes checking whether gcc supports __func__... yes Using gcc to determine dependencies checking whether long double is supported... yes checking whether long long is supported... yes checking for max C struct integer alignment... four checking for max C struct floating point alignment... four checking for max C struct alignment of structs with doubles... four checking for max C struct floating point alignment with long doubles... four checking if alignment of structs with doubles is based on position... no checking if alignment of structs with long long ints is based on position... no checking if double alignment breaks rules, find actual alignment... no checking for alignment restrictions on pointers... int or better checking size of char... 1 checking size of unsigned char... 1 checking size of short... 2 checking size of unsigned short... 2 checking size of int... 4 checking size of unsigned int... 4 checking size of long... 4 checking size of unsigned long... 4 checking size of long long... 8 checking size of unsigned long long... 8 checking size of float... 4 checking size of double... 8 checking size of long double... 12 checking size of wchar_t... 4 checking size of void *... 4 checking for size of float int... 8 checking for size of double int... 12 checking for size of long int... 8 checking for size of short int... 6 checking for size of 2 int... 8 checking for size of long double int... 16 checking for sys/bitypes.h... yes checking for inttypes.h... (cached) yes checking for int8_t... yes checking for int16_t... yes checking for int32_t... yes checking for int64_t... yes checking for uint8_t... yes checking for uint16_t... yes checking for uint32_t... yes checking for uint64_t... yes checking size of bool... 1 checking how to run the C++ preprocessor... c++ -E checking for complex... yes checking size of Complex... 8 checking size of DoubleComplex... 16 checking size of LongDoubleComplex... 24 checking if char * pointers use byte addresses... yes checking for alignment restrictions on ... no checking for alignment restrictions on int32_t... no checking for size of MPI_BSEND_OVERHEAD... 59 checking for gcc __asm__ and pentium cmpxchgl instruction... yes checking for gcc __asm__ and AMD x86_64 cmpxchgq instruction... no checking for gcc __asm__ and IA64 xchg4 instruction... no checking for ANSI C header files... (cached) no checking for stdlib.h... (cached) yes checking for stdarg.h... yes checking for sys/types.h... (cached) yes checking for inttypes.h... (cached) yes checking for limits.h... yes checking for stddef.h... yes checking for errno.h... yes checking for sys/socket.h... yes checking for sys/time.h... yes checking for unistd.h... (cached) yes checking for endian.h... yes checking for assert.h... yes checking for sys/uio.h... yes checking for size_t... yes checking for setitimer... yes checking for alarm... yes checking for vsnprintf... yes checking for vsprintf... yes checking whether vsnprintf needs a declaration... no checking for strerror... yes checking for snprintf... yes checking whether snprintf needs a declaration... no checking for va_copy... yes checking for working alloca.h... yes checking for alloca... yes checking for strdup... yes checking whether strdup needs a declaration... no checking for clock_gettime... no checking for clock_getres... no checking for gethrtime... no checking for gettimeofday... yes sourcing /home/my-ubuntu/mpich2-hginit/src/mpid/ch3/setup_device RUNNING SETUP_DEVICE FOR THE CH3 DEVICE sourcing /home/my-ubuntu/mpich2-hginit/src/mpid/ch3/channels/sctp/setup_channel RUNNING SETUP_CHANNEL FOR THE SCTP CHANNEL checking for pthread.h... yes checking for pthread_yield... no checking for library containing pthread_key_create... no configure: error: unable to find pthreads library -- *Best Regards* * Hamideh Jabalameli* M.Sc Candidate, School of Electrical and Computer Engineering, College of Engineering, Electronici Shahid Beheshti Tehran University h.jabal(a)Mail.sbu.ac.ir hamideh.jabalameli(a)gmail.com

1 0

mpich 3.0.2 last release
by Angelo Paone 23 Feb '13

23 Feb '13

Hi there, I wonder if the last mpich release 3.0.2 has the following feature in it: pkgconfig(mpich2-ch3). any help appriciated Angelo Angelo Paone *********************************************** Research Professor Post-Doctoral Research Fellow Marie Curie Fellow PhD, MSc, BSc Pusan Natioanl University Department of Earth Science 30, Jangjeon-dong, Geumjeong-gu Busan, Republic of Korea Tel. +82-51-510-2987

1 0

Re: [mpich-discuss] Affinity with MPICH_ASYNC_PROGRESS
by Jeff Hammond 23 Feb '13

23 Feb '13

>> You should run Intel MPI, MKL, OpenMP and TBB and only compile your >> code with Intel compilers. That's your best chance to have everything >> work together. > > Those are terrible programming models, oversynchronizing and inflexible, and > they don't have a concept of memory locality anyway. ;-) I personally think that MKL, meaning the BLAS, is an amazing programming model. However, my point here was merely to say that Intel has an integrated software stack that manages thread resources intelligently relative to N independent implementations of MPI, BLAS/LAPACK, and OpenMP/TBB, for example. I was under the impression that Intel presents stupid oversubscription, which a good thing for users but essentially impossible to implement portably. >> I really don't see how Hydra is supposed to know what >> GOMP is doing and try to deal with it. > > AFAIK, GOMP doesn't set affinity at all. It doesn't really make sense for > TBB because the programming model doesn't have explicit memory locality. But > in this case, _I'm_ setting affinity for my pthreads or my OpenMP threads > because I know how I use them collectively. I guess I am thinking about simplistic affinity, like breadth-first placement and not letting the OS moves threads around arbitrarily. However, no affinity is better than bad affinity. For example, any application that uses Pthreads with MVAPICH has to set MV2_ENABLE_AFFINITY=0 or pay performance consequences because MVAPICH implements bad affinity by default. >> What testing I have done of MPICH_NEMESIS_ASYNC_PROGRESS=1 on Cray >> XC30 indicates that NWChem is better off without comm threads since it >> communicates and computes at fine enough granularity such that the >> lack of progress is less than the overhead of internal locking >> (because ASYNC_PROGRESS implies MPI_THREAD_MULTIPLE) and competition >> for execution resources (even though XC30 has Intel SNB with HT >> enabled). > > MPICH_NEMESIS_ASYNC_PROGRESS=1 is always slower when I've tried it on > Hopper, even when I have nonblocking communication running for a long time > while the application is computing. I'm hoping that works better in the > future, but for now, I still wait just as long when I get around to calling > MPI_Wait, except that everything in between ran slower due to > MPI_THREAD_MULTIPLE. I think the only way for async progress to work well is to have fine-grain locking since of MPI, as is done in PAMID. Any implementation that resorts to fat-locking is probably better off without async progress unless the application is doing something really silly (like never calling MPI on a rank that is the target of MPI RMA). Jeff -- Jeff Hammond Argonne Leadership Computing Facility University of Chicago Computation Institute jhammond(a)alcf.anl.gov / (630) 252-5381 http://www.linkedin.com/in/jeffhammond https://wiki.alcf.anl.gov/parts/index.php/User:Jhammond

1 0

Re: [mpich-discuss] Affinity with MPICH_ASYNC_PROGRESS
by Jed Brown 23 Feb '13

23 Feb '13

On Sat, Feb 23, 2013 at 7:48 PM, Jeff Hammond <jhammond(a)alcf.anl.gov> wrote: > There's absolutely no reason why it should. I am not trying to > maximize progress; I am trying to maximize computational performance. > Pinning the 7 comm threads to one core is going to be terrible for > them, but I am assuming that I don't need that much progress, whereas > I do need to the computation to run at top speed. DGEMM on 7 cores > and 1 core of MPI should be much better than DGEMM on 8 cores where > each core is time-shared with a comm thread. > Sure. > You should run Intel MPI, MKL, OpenMP and TBB and only compile your > code with Intel compilers. That's your best chance to have everything > work together. > Those are terrible programming models, oversynchronizing and inflexible, and they don't have a concept of memory locality anyway. ;-) > I really don't see how Hydra is supposed to know what > GOMP is doing and try to deal with it. > AFAIK, GOMP doesn't set affinity at all. It doesn't really make sense for TBB because the programming model doesn't have explicit memory locality. But in this case, _I'm_ setting affinity for my pthreads or my OpenMP threads because I know how I use them collectively. > > What testing I have done of MPICH_NEMESIS_ASYNC_PROGRESS=1 on Cray > XC30 indicates that NWChem is better off without comm threads since it > communicates and computes at fine enough granularity such that the > lack of progress is less than the overhead of internal locking > (because ASYNC_PROGRESS implies MPI_THREAD_MULTIPLE) and competition > for execution resources (even though XC30 has Intel SNB with HT > enabled). > MPICH_NEMESIS_ASYNC_PROGRESS=1 is always slower when I've tried it on Hopper, even when I have nonblocking communication running for a long time while the application is computing. I'm hoping that works better in the future, but for now, I still wait just as long when I get around to calling MPI_Wait, except that everything in between ran slower due to MPI_THREAD_MULTIPLE.

1 0

Re: [mpich-discuss] Affinity with MPICH_ASYNC_PROGRESS
by Jeff Hammond 23 Feb '13

23 Feb '13

>> >> For example, on an 8-core node, I was hoping to get async progress on >> >> 7 processes by pinning 7 comm threads to the 8th core. >> > >> > Did this work at all? >> >> What is your definition of work? > > Does it make better async progress than the default? There's absolutely no reason why it should. I am not trying to maximize progress; I am trying to maximize computational performance. Pinning the 7 comm threads to one core is going to be terrible for them, but I am assuming that I don't need that much progress, whereas I do need to the computation to run at top speed. DGEMM on 7 cores and 1 core of MPI should be much better than DGEMM on 8 cores where each core is time-shared with a comm thread. >> As far as I know, there is no API in MPICH for controlling thread >> affinity. The right way to improve this could would be to move it >> inside of Nemesis and then add support for hwloc for comm threads. I >> assume you can move the parent processes around such that your 7 >> comm-intensive procs are closer to the NIC though. You should look at >> hwloc though. > > I'm thinking about how to interact with other threads _because_ I'm writing > hwloc-based affinity management now. The behavior that I think people will > want is to use MPI to set process affinity and then tell me to use some > affinity policy to _restrict_ the process affinity to each thread. I prefer > using MPI to set process affinity because I think it's more messy for > software at a higher level than MPI to "figure out" what other processes are > running on the same node and agree among themselves how to divvy up > resources. If there was an important configuration that could not be > supported this way, however, then I suppose I could do it. You should run Intel MPI, MKL, OpenMP and TBB and only compile your code with Intel compilers. That's your best chance to have everything work together. I really don't see how Hydra is supposed to know what GOMP is doing and try to deal with it. Maybe I am reading too much into your statement, but it sounds a lot like "give me a low calorie stout that tastes fantastic." :-) >> My diff w.r.t. the SVN trunk (I did all of this before the SVN->Git >> conversion) is below. It is clearly a hack and I don't care. It only >> works on Linux or other systems that support CPU_SET. It does not >> work on my Mac, for example. >> >> I have not done very much experimenting with this code other than to >> verify that it works (as in "does not crash and gives the same result >> for cpi"). Eventually, I am going to see how it works with ARMCI-MPI. > > Cool, I'll be curious. What testing I have done of MPICH_NEMESIS_ASYNC_PROGRESS=1 on Cray XC30 indicates that NWChem is better off without comm threads since it communicates and computes at fine enough granularity such that the lack of progress is less than the overhead of internal locking (because ASYNC_PROGRESS implies MPI_THREAD_MULTIPLE) and competition for execution resources (even though XC30 has Intel SNB with HT enabled). Jeff -- Jeff Hammond Argonne Leadership Computing Facility University of Chicago Computation Institute jhammond(a)alcf.anl.gov / (630) 252-5381 http://www.linkedin.com/in/jeffhammond https://wiki.alcf.anl.gov/parts/index.php/User:Jhammond

1 0