July 2012 - mpi-forum - mailman3.cels.anl.gov

Proper use of LaTeX
by William Gropp 16 Jul '12

16 Jul '12

If you are editing the MPI 3 document, please read the instructions on the use of LaTeX macros - it is in instr.tex in the MPI-3.0 directory; make instr.pdf will produce a PDF. I've noted a number of errors in the document, including incorrect use of MPIdeleteBegin, use of the Verbatim where verbatim should be used, and use of font change commands. These errors will need to be fixed before the document is released. Bill William Gropp Director, Parallel Computing Institute Deputy Director for Research Institute for Advanced Computing Applications and Technologies Paul and Cynthia Saylor Professor of Computer Science University of Illinois Urbana-Champaign

1 0

Re: [Mpi-forum] MPI_Mprobe workaround
by N.M. Maclaren 13 Jul '12

13 Jul '12

On Jul 13 2012, Jeff Hammond wrote: > >There is no known workaround that addresses all of the problems with >MPI Probe and MPI Iprobe in multi-threaded MPI applications. > >Are there nuances regarding the use of MPI that I have missed? Do the >real-world use cases have too much to do in the "/* ? */" to make this >viable? Quite a lot do. Anything with lazy execution may handle any outstanding operations, and anything with eager execution may do the transfer in the Irecv. Not all I/O transports support asynchronicity. But, realistically, anyone who goes beyond MPI_THREAD_FUNNELED is writing non-portable code, because MPI specifies more than can be implemented using what POSIX actually defines. Inter alia, POSIX does not define any mechanism for synchronising signals between threads, and I have exposed inconsistencies. Just like passive one-sided communication. But I agree that it should be viable on at least some systems, some MPIs and some transports. Regards, Nick Maclaren.

1 0

Re: [Mpi-forum] MPI_Mprobe workaround
by Torsten Hoefler 13 Jul '12

13 Jul '12

Jeff, > Hmm, it seems I've stumbled upon a variant of Listing 1.1 in your paper :-) Yes. > I'm a big fan of buffer pools, so I am unfazed by the potential > performance impact of sbrk(). If you need to malloc() so much memory > that sbrk() is called, I suspect that the message transfer time is > going to take longer than a system call. Keep in mind that you may (hopefully!) have multiple interfaces. If the network bandwidth does not scale with the number of cores that one may have all kinds of different issues. The paper shows also that the benefit if Mprobe (using std malloc) can be measured, even on yesterday's systems (eight cores if I remember correctly). > But I confess to being somewhat ignorant of the performance of slow, > general purpose operating systems like Linux. You forgot "widely available" and "portable" in this list ;-). Torsten -- ### qreharg rug ebs fv crryF ------------- http://www.unixer.de/ ----- Torsten Hoefler | Performance Modeling and Simulation Lead Blue Waters Directorate | University of Illinois (UIUC) 1205 W Clark Street | Urbana, IL, 61801 NCSA Building | +01 (217) 244-7736

1 0

Re: [Mpi-forum] MPI_Mprobe workaround
by Jeff Hammond 13 Jul '12

13 Jul '12

Hmm, it seems I've stumbled upon a variant of Listing 1.1 in your paper :-) I'm a big fan of buffer pools, so I am unfazed by the potential performance impact of sbrk(). If you need to malloc() so much memory that sbrk() is called, I suspect that the message transfer time is going to take longer than a system call. But I confess to being somewhat ignorant of the performance of slow, general purpose operating systems like Linux. Jeff On Fri, Jul 13, 2012 at 2:19 PM, Torsten Hoefler <htor(a)illinois.edu> wrote: > Jeff, > >> I was reading through >> www.unixer.de/publications/img/mprobe-proposal-rev4.pdf and - just for >> fun - wondered if there is a workaround, not because MPI_Mprobe >> shouldn't be in the MPI-3 standard, but because some folks might want >> a workaround for backwards compatibility if they are forced to use >> MPI-2 somewhere. >> >> I apologize in advance if something equivalent to what I say below was >> discussed at the Forum and I was not present. >> >> Torsten and coworkers say the following. >> ============================ >> For example, the following code can not be executed concurrently by >> two threads in an MPI process, because a message could be found by the >> MPI Probe in both threads, while only one of the threads could >> successfully receive the message (the other will block): >> MPI_Status status; >> int value; >> MPI_Probe(MPI_ANY_SOURCE, /*tag=*/0, MPI_COMM_WORLD, &status); >> MPI_Recv(&value, 1, MPI_INT, status.MPI_SOURCE, /*tag=*/0, >> MPI_COMM_WORLD, MPI_STATUS_IGNORE); >> >> <snip> >> >> There is no known workaround that addresses all of the problems with >> MPI Probe and MPI Iprobe in multi-threaded MPI applications. >> ============================ >> >> Obviously, a fat mutex around this block solves the problem, but the >> time spent in the mutex will scale with the message size. I was >> curious the following workaround is reasonable when MPI-2 must be >> used. >> ============================ >> MPI_Status status; >> MPI_Request request; >> AppropriateMutex mutex; >> int value; >> >> ACQUIRE_MUTEX(&mutex); >> MPI_Probe(MPI_ANY_SOURCE, /*tag=*/0, MPI_COMM_WORLD, &status); >> /* ? */ >> MPI_Irecv(&value, 1, MPI_INT, status.MPI_SOURCE, /*tag=*/0, >> MPI_COMM_WORLD, &request); >> RELEASE_MUTEX(&mutex); >> MPI_Wait(&request, MPI_STATUS_IGNORE); >> ============================ > This is correct. But most use-cases for probe require a malloc and > malloc is not always fast (especially if it needs to execute sbrk). This > will still have the contention problem (assume 1000's of cores!). Mprobe > allows to use wait-free algorithms for the queue management. > >> Are there nuances regarding the use of MPI that I have missed? > Nope. > >> Do the real-world use cases have too much to do in the "/* ? */" to >> make this viable? > I think so (at least all use-cases I know). Do you have ones that only > require trivial "/* ? */"? > > This was brought up at a point when proposals were much harsher reviewed > than today and we had many discussions and even a full EuroMPI paper on > this http://www.unixer.de/publications/index.php?pub=103 . > > All the Best, > Torsten > > -- > ### qreharg rug ebs fv crryF ------------- http://www.unixer.de/ ----- > Torsten Hoefler | Performance Modeling and Simulation Lead > Blue Waters Directorate | University of Illinois (UIUC) > 1205 W Clark Street | Urbana, IL, 61801 > NCSA Building | +01 (217) 244-7736 > _______________________________________________ > mpi-forum mailing list > mpi-forum(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum -- Jeff Hammond Argonne Leadership Computing Facility University of Chicago Computation Institute jhammond(a)alcf.anl.gov / (630) 252-5381 http://www.linkedin.com/in/jeffhammond https://wiki.alcf.anl.gov/parts/index.php/User:Jhammond

1 0

Re: [Mpi-forum] MPI_Mprobe workaround
by Torsten Hoefler 13 Jul '12

13 Jul '12

Jeff, > I was reading through > www.unixer.de/publications/img/mprobe-proposal-rev4.pdf and - just for > fun - wondered if there is a workaround, not because MPI_Mprobe > shouldn't be in the MPI-3 standard, but because some folks might want > a workaround for backwards compatibility if they are forced to use > MPI-2 somewhere. > > I apologize in advance if something equivalent to what I say below was > discussed at the Forum and I was not present. > > Torsten and coworkers say the following. > ============================ > For example, the following code can not be executed concurrently by > two threads in an MPI process, because a message could be found by the > MPI Probe in both threads, while only one of the threads could > successfully receive the message (the other will block): > MPI_Status status; > int value; > MPI_Probe(MPI_ANY_SOURCE, /*tag=*/0, MPI_COMM_WORLD, &status); > MPI_Recv(&value, 1, MPI_INT, status.MPI_SOURCE, /*tag=*/0, > MPI_COMM_WORLD, MPI_STATUS_IGNORE); > > <snip> > > There is no known workaround that addresses all of the problems with > MPI Probe and MPI Iprobe in multi-threaded MPI applications. > ============================ > > Obviously, a fat mutex around this block solves the problem, but the > time spent in the mutex will scale with the message size. I was > curious the following workaround is reasonable when MPI-2 must be > used. > ============================ > MPI_Status status; > MPI_Request request; > AppropriateMutex mutex; > int value; > > ACQUIRE_MUTEX(&mutex); > MPI_Probe(MPI_ANY_SOURCE, /*tag=*/0, MPI_COMM_WORLD, &status); > /* ? */ > MPI_Irecv(&value, 1, MPI_INT, status.MPI_SOURCE, /*tag=*/0, > MPI_COMM_WORLD, &request); > RELEASE_MUTEX(&mutex); > MPI_Wait(&request, MPI_STATUS_IGNORE); > ============================ This is correct. But most use-cases for probe require a malloc and malloc is not always fast (especially if it needs to execute sbrk). This will still have the contention problem (assume 1000's of cores!). Mprobe allows to use wait-free algorithms for the queue management. > Are there nuances regarding the use of MPI that I have missed? Nope. > Do the real-world use cases have too much to do in the "/* ? */" to > make this viable? I think so (at least all use-cases I know). Do you have ones that only require trivial "/* ? */"? This was brought up at a point when proposals were much harsher reviewed than today and we had many discussions and even a full EuroMPI paper on this http://www.unixer.de/publications/index.php?pub=103 . All the Best, Torsten -- ### qreharg rug ebs fv crryF ------------- http://www.unixer.de/ ----- Torsten Hoefler | Performance Modeling and Simulation Lead Blue Waters Directorate | University of Illinois (UIUC) 1205 W Clark Street | Urbana, IL, 61801 NCSA Building | +01 (217) 244-7736

1 0

MPI_Mprobe workaround
by Jeff Hammond 13 Jul '12

13 Jul '12

I was reading through www.unixer.de/publications/img/mprobe-proposal-rev4.pdf and - just for fun - wondered if there is a workaround, not because MPI_Mprobe shouldn't be in the MPI-3 standard, but because some folks might want a workaround for backwards compatibility if they are forced to use MPI-2 somewhere. I apologize in advance if something equivalent to what I say below was discussed at the Forum and I was not present. Torsten and coworkers say the following. ============================ For example, the following code can not be executed concurrently by two threads in an MPI process, because a message could be found by the MPI Probe in both threads, while only one of the threads could successfully receive the message (the other will block): MPI_Status status; int value; MPI_Probe(MPI_ANY_SOURCE, /*tag=*/0, MPI_COMM_WORLD, &status); MPI_Recv(&value, 1, MPI_INT, status.MPI_SOURCE, /*tag=*/0, MPI_COMM_WORLD, MPI_STATUS_IGNORE); <snip> There is no known workaround that addresses all of the problems with MPI Probe and MPI Iprobe in multi-threaded MPI applications. ============================ Obviously, a fat mutex around this block solves the problem, but the time spent in the mutex will scale with the message size. I was curious the following workaround is reasonable when MPI-2 must be used. ============================ MPI_Status status; MPI_Request request; AppropriateMutex mutex; int value; ACQUIRE_MUTEX(&mutex); MPI_Probe(MPI_ANY_SOURCE, /*tag=*/0, MPI_COMM_WORLD, &status); /* ? */ MPI_Irecv(&value, 1, MPI_INT, status.MPI_SOURCE, /*tag=*/0, MPI_COMM_WORLD, &request); RELEASE_MUTEX(&mutex); MPI_Wait(&request, MPI_STATUS_IGNORE); ============================ Assuming MPI_Irecv returns quickly and there is nothing in between the two operations, then the time spent in the critical section defined by the mutex will be rather short. My solution may become inefficient if one has to allocate memory to receive the message while holding the mutex. However, instances where memory allocation is slow should be quite rare and there are workarounds when such scenarios exist. Are there nuances regarding the use of MPI that I have missed? Do the real-world use cases have too much to do in the "/* ? */" to make this viable? Thanks, Jeff -- Jeff Hammond Argonne Leadership Computing Facility University of Chicago Computation Institute jhammond(a)alcf.anl.gov / (630) 252-5381 http://www.linkedin.com/in/jeffhammond https://wiki.alcf.anl.gov/parts/index.php/User:Jhammond

1 0

Re: [Mpi-forum] Registration for the meeting next week
by Jeff Squyres 13 Jul '12

13 Jul '12

FWIW, I went in today w/ Chrome on Mac and it worked fine. Shrug. On Jul 13, 2012, at 11:43 AM, Jeff Hammond wrote: > I had the problem with Chrome on a Mac. I guess I'll try Safari in the future. > > Thanks for the info. > > Jeff > > On Fri, Jul 13, 2012 at 10:20 AM, Schulz, Martin <schulzm(a)llnl.gov> wrote: >> In case it helps someone: I had the same problems using Firefox on a Mac; using Safari it worked without problems. >> >> Martin >> >> >> On Jul 13, 2012, at 6:42 AM, Richard Graham wrote: >> >>> :-) >>> >>> It is annoying - I had the same issue ... >>> >>> Rich >>> >>> ----- Original Message ----- >>> From: Jeff Hammond [mailto:[email protected]] >>> Sent: Friday, July 13, 2012 06:39 AM >>> To: Main MPI Forum mailing list <mpi-forum(a)lists.mpi-forum.org> >>> Subject: Re: [Mpi-forum] Registration for the meeting next week >>> >>> I don't know if I'm incredibly stupid or there's something wrong with >>> the registration site, but it took me 7 times to get my registration >>> to go through. The first six times I tried to complete the process, >>> it just kicked me back to the payment info page and cleared all the >>> forms. >>> >>> If no one else had problems, perhaps my karma is in a bad state >>> because of my C++ tantrums. >>> >>> Jeff >>> >>> On Fri, Jul 13, 2012 at 8:04 AM, Richard Graham <richardg(a)mellanox.com> wrote: >>>> For those of you that have not yet registered, and plan to attend next week, >>>> please do so ASAP so that we have a good number for catering. I am aware of >>>> at least one person that plans to be at the meeting, and has not yet >>>> registered ï¿½ do not know how many more may be in the same situation. >>>> >>>> >>>> >>>> Thanks, >>>> >>>> Rich >>>> >>>> >>>> _______________________________________________ >>>> mpi-forum mailing list >>>> mpi-forum(a)lists.mpi-forum.org >>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum >>> >>> >>> >>> -- >>> Jeff Hammond >>> Argonne Leadership Computing Facility >>> University of Chicago Computation Institute >>> jhammond(a)alcf.anl.gov / (630) 252-5381 >>> http://www.linkedin.com/in/jeffhammond >>> https://wiki.alcf.anl.gov/parts/index.php/User:Jhammond >>> >>> _______________________________________________ >>> mpi-forum mailing list >>> mpi-forum(a)lists.mpi-forum.org >>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum >>> >>> _______________________________________________ >>> mpi-forum mailing list >>> mpi-forum(a)lists.mpi-forum.org >>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum >> >> ________________________________________________________________________ >> Martin Schulz, schulzm(a)llnl.gov, http://people.llnl.gov/schulzm >> CASC @ Lawrence Livermore National Laboratory, Livermore, USA >> >> >> >> >> _______________________________________________ >> mpi-forum mailing list >> mpi-forum(a)lists.mpi-forum.org >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum > > > > -- > Jeff Hammond > Argonne Leadership Computing Facility > University of Chicago Computation Institute > jhammond(a)alcf.anl.gov / (630) 252-5381 > http://www.linkedin.com/in/jeffhammond > https://wiki.alcf.anl.gov/parts/index.php/User:Jhammond > > _______________________________________________ > mpi-forum mailing list > mpi-forum(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum -- Jeff Squyres jsquyres(a)cisco.com For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/

1 0

Re: [Mpi-forum] Registration for the meeting next week
by Jeff Hammond 13 Jul '12

13 Jul '12

I had the problem with Chrome on a Mac. I guess I'll try Safari in the future. Thanks for the info. Jeff On Fri, Jul 13, 2012 at 10:20 AM, Schulz, Martin <schulzm(a)llnl.gov> wrote: > In case it helps someone: I had the same problems using Firefox on a Mac; using Safari it worked without problems. > > Martin > > > On Jul 13, 2012, at 6:42 AM, Richard Graham wrote: > >> :-) >> >> It is annoying - I had the same issue ... >> >> Rich >> >> ----- Original Message ----- >> From: Jeff Hammond [mailto:[email protected]] >> Sent: Friday, July 13, 2012 06:39 AM >> To: Main MPI Forum mailing list <mpi-forum(a)lists.mpi-forum.org> >> Subject: Re: [Mpi-forum] Registration for the meeting next week >> >> I don't know if I'm incredibly stupid or there's something wrong with >> the registration site, but it took me 7 times to get my registration >> to go through. The first six times I tried to complete the process, >> it just kicked me back to the payment info page and cleared all the >> forms. >> >> If no one else had problems, perhaps my karma is in a bad state >> because of my C++ tantrums. >> >> Jeff >> >> On Fri, Jul 13, 2012 at 8:04 AM, Richard Graham <richardg(a)mellanox.com> wrote: >>> For those of you that have not yet registered, and plan to attend next week, >>> please do so ASAP so that we have a good number for catering. I am aware of >>> at least one person that plans to be at the meeting, and has not yet >>> registered ï¿½ do not know how many more may be in the same situation. >>> >>> >>> >>> Thanks, >>> >>> Rich >>> >>> >>> _______________________________________________ >>> mpi-forum mailing list >>> mpi-forum(a)lists.mpi-forum.org >>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum >> >> >> >> -- >> Jeff Hammond >> Argonne Leadership Computing Facility >> University of Chicago Computation Institute >> jhammond(a)alcf.anl.gov / (630) 252-5381 >> http://www.linkedin.com/in/jeffhammond >> https://wiki.alcf.anl.gov/parts/index.php/User:Jhammond >> >> _______________________________________________ >> mpi-forum mailing list >> mpi-forum(a)lists.mpi-forum.org >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum >> >> _______________________________________________ >> mpi-forum mailing list >> mpi-forum(a)lists.mpi-forum.org >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum > > ________________________________________________________________________ > Martin Schulz, schulzm(a)llnl.gov, http://people.llnl.gov/schulzm > CASC @ Lawrence Livermore National Laboratory, Livermore, USA > > > > > _______________________________________________ > mpi-forum mailing list > mpi-forum(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum -- Jeff Hammond Argonne Leadership Computing Facility University of Chicago Computation Institute jhammond(a)alcf.anl.gov / (630) 252-5381 http://www.linkedin.com/in/jeffhammond https://wiki.alcf.anl.gov/parts/index.php/User:Jhammond

1 0

Re: [Mpi-forum] Registration for the meeting next week
by Schulz, Martin 13 Jul '12

13 Jul '12

In case it helps someone: I had the same problems using Firefox on a Mac; using Safari it worked without problems. Martin On Jul 13, 2012, at 6:42 AM, Richard Graham wrote: > :-) > > It is annoying - I had the same issue ... > > Rich > > ----- Original Message ----- > From: Jeff Hammond [mailto:[email protected]] > Sent: Friday, July 13, 2012 06:39 AM > To: Main MPI Forum mailing list <mpi-forum(a)lists.mpi-forum.org> > Subject: Re: [Mpi-forum] Registration for the meeting next week > > I don't know if I'm incredibly stupid or there's something wrong with > the registration site, but it took me 7 times to get my registration > to go through. The first six times I tried to complete the process, > it just kicked me back to the payment info page and cleared all the > forms. > > If no one else had problems, perhaps my karma is in a bad state > because of my C++ tantrums. > > Jeff > > On Fri, Jul 13, 2012 at 8:04 AM, Richard Graham <richardg(a)mellanox.com> wrote: >> For those of you that have not yet registered, and plan to attend next week, >> please do so ASAP so that we have a good number for catering. I am aware of >> at least one person that plans to be at the meeting, and has not yet >> registered ï¿½ do not know how many more may be in the same situation. >> >> >> >> Thanks, >> >> Rich >> >> >> _______________________________________________ >> mpi-forum mailing list >> mpi-forum(a)lists.mpi-forum.org >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum > > > > -- > Jeff Hammond > Argonne Leadership Computing Facility > University of Chicago Computation Institute > jhammond(a)alcf.anl.gov / (630) 252-5381 > http://www.linkedin.com/in/jeffhammond > https://wiki.alcf.anl.gov/parts/index.php/User:Jhammond > > _______________________________________________ > mpi-forum mailing list > mpi-forum(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum > > _______________________________________________ > mpi-forum mailing list > mpi-forum(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum ________________________________________________________________________ Martin Schulz, schulzm(a)llnl.gov, http://people.llnl.gov/schulzm CASC @ Lawrence Livermore National Laboratory, Livermore, USA

1 0

Re: [Mpi-forum] Registration for the meeting next week
by Richard Graham 13 Jul '12

13 Jul '12

:-) It is annoying - I had the same issue ... Rich ----- Original Message ----- From: Jeff Hammond [mailto:[email protected]] Sent: Friday, July 13, 2012 06:39 AM To: Main MPI Forum mailing list <mpi-forum(a)lists.mpi-forum.org> Subject: Re: [Mpi-forum] Registration for the meeting next week I don't know if I'm incredibly stupid or there's something wrong with the registration site, but it took me 7 times to get my registration to go through. The first six times I tried to complete the process, it just kicked me back to the payment info page and cleared all the forms. If no one else had problems, perhaps my karma is in a bad state because of my C++ tantrums. Jeff On Fri, Jul 13, 2012 at 8:04 AM, Richard Graham <richardg(a)mellanox.com> wrote: > For those of you that have not yet registered, and plan to attend next week, > please do so ASAP so that we have a good number for catering. I am aware of > at least one person that plans to be at the meeting, and has not yet > registered – do not know how many more may be in the same situation. > > > > Thanks, > > Rich > > > _______________________________________________ > mpi-forum mailing list > mpi-forum(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum -- Jeff Hammond Argonne Leadership Computing Facility University of Chicago Computation Institute jhammond(a)alcf.anl.gov / (630) 252-5381 http://www.linkedin.com/in/jeffhammond https://wiki.alcf.anl.gov/parts/index.php/User:Jhammond _______________________________________________ mpi-forum mailing list mpi-forum(a)lists.mpi-forum.org http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum

1 0