April 2008 - mpiwg-ft - mailman3.cels.anl.gov

Re: [Mpi3-ft] Telecon this week
by Edgar Gabriel 24 Apr '08

24 Apr '08

I will try to make it... Thanks Edgar Richard Graham wrote: > !!!!!! Please not the time change !!!!!! > > We will have our regular biweekly call this Friday, 4/25/2008. However, we > have moved the call an hour earlier. > > April 25, 2008 - 11-1 pm EDT, 4-6 pm GMT, US Toll Free number: 877-801-8130, > Toll number: 1-203-692-8690, Code: 1044056 > > We would like to go over the proposed changes to the mpi-dynamics, in > support of ft, and error handling. Edgar and Alexander, can you make this > call ? > > Thanks, > Rich > > _______________________________________________ > mpi3-ft mailing list > mpi3-ft(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-ft -- Edgar Gabriel Assistant Professor Parallel Software Technologies Lab http://pstl.cs.uh.edu Department of Computer Science University of Houston Philip G. Hoffman Hall, Room 524 Houston, TX-77204, USA Tel: +1 (713) 743-3857 Fax: +1 (713) 743-3335

1 0

Re: [Mpi3-ft] Piggybacking API
by Greg Bronevetsky 23 Apr '08

23 Apr '08

>That helps a little except for the fact that the spec is the spec and >the APIs will be documented >and available for all to see. I also believe that most general purpose >MPI libraries will be aiming >to support the full specification. So subsetting will not be a complete >hall pass to ignore the PB >calls nor would I recommend ignoring them. > >Note, I was just wondering if there was a different way to support this >feature than doubling the >amount of send apis. There are two types of complexity: API complexity and implementation complexity. Increasing the number of MPI calls relates to both but does not correlate exactly. The inclusion of collective communication increases both types of complexity. However, the inclusion of the PMPI functions doubles the number of function calls in the API but has minimal impact on both types of complexity. The piggybacking API is really aimed to increase implementation complexity a little by addressing a real need out there that is not served by current MPI implementations. Regardless of how we expose this new implementation complexity to users, we do need this new capability. Given that, the major goal becomes: how to expose this to users in a way that minimizes API complexity. To this end, I think that we need to break the API up into two components: user API and tool API. The user API makes up the bulk of the spec and represents the portions of the spec that are meant to be used by regular application developers. The tool API in MPI 2 is essentially the PMPI interface. However, I am hoping that it will grow to include the piggybacking API as well as the idea that every non-local blocking MPI function call should have a corresponding non-blocking variant. Seen in this light, the new additions have a minimal impact on the complexity of the API: they simply represent new points where tools can intercept and adjust the activities of user API functions. For example, PMPI+piggybacking can be used by tools to attach control information to application communication and PMPI+nonblocking can be used to allow any tool to perform its own synchronization without worrying that it will cause deadlock with the application's communication. In this sense it doesn't matter if we duplicate every call in the spec because this duplication is only visible to tools and can be implemented fairly easily because each of these APIs have only a single key feature that needs to be replicated across many MPI calls. In this sense the duplication associated with the tool API is closely related to aspect-oriented programming. In short, I believe that function call duplication associated with additions to the tool API is a natural part of how any tool API would operate. Because each new duplicated function represents a new tool attachment point for the same capability, rather than a new capability, this duplication represents minimal increases in API and implementation complexities. The primary complexity is in implementing the new required functionality and is thus, a tool complexity and not an API complexity. Greg Bronevetsky Post-Doctoral Researcher 1028 Building 451 Lawrence Livermore National Lab (925) 424-5756 bronevetsky1(a)llnl.gov

1 0

Re: [Mpi3-ft] Piggybacking API
by Terry Dontje 22 Apr '08

22 Apr '08

Supalov, Alexander wrote: > Possibly less so if this stuff is put into an optional subset, I hope. > In that case only those who care will, er, care. > That helps a little except for the fact that the spec is the spec and the APIs will be documented and available for all to see. I also believe that most general purpose MPI libraries will be aiming to support the full specification. So subsetting will not be a complete hall pass to ignore the PB calls nor would I recommend ignoring them. Note, I was just wondering if there was a different way to support this feature than doubling the amount of send apis. --td > -----Original Message----- > From: mpi3-ft-bounces(a)lists.mpi-forum.org > [mailto:[email protected]] On Behalf Of Terry Dontje > Sent: Tuesday, April 22, 2008 12:55 PM > To: MPI 3.0 Fault Tolerance and Dynamic Process Control working Group > Subject: Re: [Mpi3-ft] Piggybacking API > > Martin Schulz wrote: > >> At 06:36 AM 4/21/2008, Terry Dontje wrote: >> >> >>> So I reread the piggybacking document on wiki. I am not thrilled >>> > with > >>> the amount of new APIs this would be adding to the standard but can >>> > also > >>> see the point of the paper. I am curious how the new API is expected >>> > to > >>> be used? The proposal say's this API is needed for user-level fault >>> tolerance solutions. So do we expect a user to change all >>> > application > >>> calls to the MPI library to use the PB calls? I wonder if a more >>> general solution that doesn't require a direct change to the API >>> > would work. > >>> >>> >> Layers like the one for fault tolerance will have to intercept >> all MPI communication calls anyway (otherwise it will not be >> possible to capture the state of the MPI layer) and hence the >> most likely mechanism to implement them is the PMPI layer. At >> this point it will be very easy and fully transparent to also >> replace communication calls with the piggyback counterparts. >> The application would not see the difference (if we get the API >> right :) ). >> >> >> >> >>> I wonder if there might be a way one could register piggybacking with >>> > a > >>> communicator and somehow have the actual piggybacking occur as a >>> callback from an implementations messaging layer? >>> >>> >> Registering piggyback data beforehand (whether per communicator >> or globally) has the problem that in a fully multithreaded use >> of MPI piggyback data can no longer be associated with a single >> MPI call/message (unless we make the piggyback data thread local, >> but this would require a concept of threads in MPI and I don't >> think we should go there). >> >> >> > Fair enough, however with the numerous variations of send calls it is > already confusing as to which ones to use. IMO adding PB api's > for each of the send apis is just asking for more user confusion. > > --td > >> Martin >> >> >> >> >> >>> Just a thought, >>> >>> --td >>> _______________________________________________ >>> mpi3-ft mailing list >>> mpi3-ft(a)lists.mpi-forum.org >>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-ft >>> >>> >> > _______________________________________________________________________ > >> Martin Schulz, schulzm(a)llnl.gov, http://people.llnl.gov/schulz6 >> CASC @ Lawrence Livermore National Laboratory, Livermore, USA >> >> _______________________________________________ >> mpi3-ft mailing list >> mpi3-ft(a)lists.mpi-forum.org >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-ft >> >> > > _______________________________________________ > mpi3-ft mailing list > mpi3-ft(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-ft > --------------------------------------------------------------------- > Intel GmbH > Dornacher Strasse 1 > 85622 Feldkirchen/Muenchen Germany > Sitz der Gesellschaft: Feldkirchen bei Muenchen > Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer > Registergericht: Muenchen HRB 47456 Ust.-IdNr. > VAT Registration No.: DE129385895 > Citibank Frankfurt (BLZ 502 109 00) 600119052 > > This e-mail and any attachments may contain confidential material for > the sole use of the intended recipient(s). Any review or distribution > by others is strictly prohibited. If you are not the intended > recipient, please contact the sender and delete all copies. > > > _______________________________________________ > mpi3-ft mailing list > mpi3-ft(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-ft >

1 0

Re: [Mpi3-ft] Piggybacking API
by Supalov, Alexander 22 Apr '08

22 Apr '08

Possibly less so if this stuff is put into an optional subset, I hope. In that case only those who care will, er, care. -----Original Message----- From: mpi3-ft-bounces(a)lists.mpi-forum.org [mailto:[email protected]] On Behalf Of Terry Dontje Sent: Tuesday, April 22, 2008 12:55 PM To: MPI 3.0 Fault Tolerance and Dynamic Process Control working Group Subject: Re: [Mpi3-ft] Piggybacking API Martin Schulz wrote: > At 06:36 AM 4/21/2008, Terry Dontje wrote: > >> So I reread the piggybacking document on wiki. I am not thrilled with >> the amount of new APIs this would be adding to the standard but can also >> see the point of the paper. I am curious how the new API is expected to >> be used? The proposal say's this API is needed for user-level fault >> tolerance solutions. So do we expect a user to change all application >> calls to the MPI library to use the PB calls? I wonder if a more >> general solution that doesn't require a direct change to the API would work. >> > > Layers like the one for fault tolerance will have to intercept > all MPI communication calls anyway (otherwise it will not be > possible to capture the state of the MPI layer) and hence the > most likely mechanism to implement them is the PMPI layer. At > this point it will be very easy and fully transparent to also > replace communication calls with the piggyback counterparts. > The application would not see the difference (if we get the API > right :) ). > > > >> I wonder if there might be a way one could register piggybacking with a >> communicator and somehow have the actual piggybacking occur as a >> callback from an implementations messaging layer? >> > > Registering piggyback data beforehand (whether per communicator > or globally) has the problem that in a fully multithreaded use > of MPI piggyback data can no longer be associated with a single > MPI call/message (unless we make the piggyback data thread local, > but this would require a concept of threads in MPI and I don't > think we should go there). > > Fair enough, however with the numerous variations of send calls it is already confusing as to which ones to use. IMO adding PB api's for each of the send apis is just asking for more user confusion. --td > Martin > > > > >> Just a thought, >> >> --td >> _______________________________________________ >> mpi3-ft mailing list >> mpi3-ft(a)lists.mpi-forum.org >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-ft >> > > _______________________________________________________________________ > Martin Schulz, schulzm(a)llnl.gov, http://people.llnl.gov/schulz6 > CASC @ Lawrence Livermore National Laboratory, Livermore, USA > > _______________________________________________ > mpi3-ft mailing list > mpi3-ft(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-ft > _______________________________________________ mpi3-ft mailing list mpi3-ft(a)lists.mpi-forum.org http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-ft --------------------------------------------------------------------- Intel GmbH Dornacher Strasse 1 85622 Feldkirchen/Muenchen Germany Sitz der Gesellschaft: Feldkirchen bei Muenchen Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer Registergericht: Muenchen HRB 47456 Ust.-IdNr. VAT Registration No.: DE129385895 Citibank Frankfurt (BLZ 502 109 00) 600119052 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.

1 0

Re: [Mpi3-ft] Piggybacking API
by Terry Dontje 22 Apr '08

22 Apr '08

Martin Schulz wrote: > At 06:36 AM 4/21/2008, Terry Dontje wrote: > >> So I reread the piggybacking document on wiki. I am not thrilled with >> the amount of new APIs this would be adding to the standard but can also >> see the point of the paper. I am curious how the new API is expected to >> be used? The proposal say's this API is needed for user-level fault >> tolerance solutions. So do we expect a user to change all application >> calls to the MPI library to use the PB calls? I wonder if a more >> general solution that doesn't require a direct change to the API would work. >> > > Layers like the one for fault tolerance will have to intercept > all MPI communication calls anyway (otherwise it will not be > possible to capture the state of the MPI layer) and hence the > most likely mechanism to implement them is the PMPI layer. At > this point it will be very easy and fully transparent to also > replace communication calls with the piggyback counterparts. > The application would not see the difference (if we get the API > right :) ). > > > >> I wonder if there might be a way one could register piggybacking with a >> communicator and somehow have the actual piggybacking occur as a >> callback from an implementations messaging layer? >> > > Registering piggyback data beforehand (whether per communicator > or globally) has the problem that in a fully multithreaded use > of MPI piggyback data can no longer be associated with a single > MPI call/message (unless we make the piggyback data thread local, > but this would require a concept of threads in MPI and I don't > think we should go there). > > Fair enough, however with the numerous variations of send calls it is already confusing as to which ones to use. IMO adding PB api's for each of the send apis is just asking for more user confusion. --td > Martin > > > > >> Just a thought, >> >> --td >> _______________________________________________ >> mpi3-ft mailing list >> mpi3-ft(a)lists.mpi-forum.org >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-ft >> > > _______________________________________________________________________ > Martin Schulz, schulzm(a)llnl.gov, http://people.llnl.gov/schulz6 > CASC @ Lawrence Livermore National Laboratory, Livermore, USA > > _______________________________________________ > mpi3-ft mailing list > mpi3-ft(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-ft >

1 0

Re: [Mpi3-ft] Telecon this week
by Supalov, Alexander 21 Apr '08

21 Apr '08

Hi, Assuming CEST - EDT = 6 hours, I may have to miss the first part, depending on a standing Intel meeting I have to attend when it takes place. Best regards. Alexander -----Original Message----- From: mpi3-ft-bounces(a)lists.mpi-forum.org [mailto:[email protected]] On Behalf Of Richard Graham Sent: Monday, April 21, 2008 10:01 PM To: MPI 3.0 Fault Tolerance and Dynamic Process Control working Group Subject: [Mpi3-ft] Telecon this week !!!!!! Please not the time change !!!!!! We will have our regular biweekly call this Friday, 4/25/2008. However, we have moved the call an hour earlier. April 25, 2008 - 11-1 pm EDT, 4-6 pm GMT, US Toll Free number: 877-801-8130, Toll number: 1-203-692-8690, Code: 1044056 We would like to go over the proposed changes to the mpi-dynamics, in support of ft, and error handling. Edgar and Alexander, can you make this call ? Thanks, Rich _______________________________________________ mpi3-ft mailing list mpi3-ft(a)lists.mpi-forum.org http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-ft --------------------------------------------------------------------- Intel GmbH Dornacher Strasse 1 85622 Feldkirchen/Muenchen Germany Sitz der Gesellschaft: Feldkirchen bei Muenchen Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer Registergericht: Muenchen HRB 47456 Ust.-IdNr. VAT Registration No.: DE129385895 Citibank Frankfurt (BLZ 502 109 00) 600119052 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.

1 0

[Mpi3-ft] Telecon this week
by Richard Graham 21 Apr '08

21 Apr '08

!!!!!! Please not the time change !!!!!! We will have our regular biweekly call this Friday, 4/25/2008. However, we have moved the call an hour earlier. April 25, 2008 - 11-1 pm EDT, 4-6 pm GMT, US Toll Free number: 877-801-8130, Toll number: 1-203-692-8690, Code: 1044056 We would like to go over the proposed changes to the mpi-dynamics, in support of ft, and error handling. Edgar and Alexander, can you make this call ? Thanks, Rich

1 0

Re: [Mpi3-ft] Piggybacking API
by Martin Schulz 21 Apr '08

21 Apr '08

At 06:36 AM 4/21/2008, Terry Dontje wrote: >So I reread the piggybacking document on wiki. I am not thrilled with >the amount of new APIs this would be adding to the standard but can also >see the point of the paper. I am curious how the new API is expected to >be used? The proposal say's this API is needed for user-level fault >tolerance solutions. So do we expect a user to change all application >calls to the MPI library to use the PB calls? I wonder if a more >general solution that doesn't require a direct change to the API would work. Layers like the one for fault tolerance will have to intercept all MPI communication calls anyway (otherwise it will not be possible to capture the state of the MPI layer) and hence the most likely mechanism to implement them is the PMPI layer. At this point it will be very easy and fully transparent to also replace communication calls with the piggyback counterparts. The application would not see the difference (if we get the API right :) ). >I wonder if there might be a way one could register piggybacking with a >communicator and somehow have the actual piggybacking occur as a >callback from an implementations messaging layer? Registering piggyback data beforehand (whether per communicator or globally) has the problem that in a fully multithreaded use of MPI piggyback data can no longer be associated with a single MPI call/message (unless we make the piggyback data thread local, but this would require a concept of threads in MPI and I don't think we should go there). Martin >Just a thought, > >--td >_______________________________________________ >mpi3-ft mailing list >mpi3-ft(a)lists.mpi-forum.org >http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-ft _______________________________________________________________________ Martin Schulz, schulzm(a)llnl.gov, http://people.llnl.gov/schulz6 CASC @ Lawrence Livermore National Laboratory, Livermore, USA

1 0

Re: [Mpi3-ft] Piggybacking API
by Greg Bronevetsky 21 Apr '08

21 Apr '08

The following papers should clarify the motivation and the usage mechanism: http://greg.bronevetsky.com/papers/2003PPoPP.pdf http://sc07.supercomputing.org/schedule/pdf/pap224.pdf The first describes a checkpointing protocol that was implemented on top of MPI. The library intercepted all application MPI calls and added appropriate piggybacking and message logging logic to ensure that regardless of when each application process decided to take a checkpoint, it would still be possible to put together a consistent global state on restart. The user did not need to add any code. The second paper describes a library called PnMPI that uses the PMPI interface to make it possible to insert as many PMPI interception layers as one would like, with the possibility of having them interact and use each other's functionality. We expect that the piggybacking interface will primarily be used by tools written at this level, using tools like PnMPI or just plain PMPI. In particular, our focus here is on tool builders, not application programmers and we believe that APIs like this will be useful to all application developers simply because they're important to tool builders whose tools may be used by any developer. In addition to checkpointing, piggybacking is useful for tracing libraries, performance analysis tools and generic libraries that need to lazily propagate information and would prefer not to introduce any new communication for performance reasons. Greg Bronevetsky Post-Doctoral Researcher 1028 Building 451 Lawrence Livermore National Lab (925) 424-5756 bronevetsky1(a)llnl.gov At 06:36 AM 4/21/2008, Terry Dontje wrote: >So I reread the piggybacking document on wiki. I am not thrilled with >the amount of new APIs this would be adding to the standard but can also >see the point of the paper. I am curious how the new API is expected to >be used? The proposal say's this API is needed for user-level fault >tolerance solutions. So do we expect a user to change all application >calls to the MPI library to use the PB calls? I wonder if a more >general solution that doesn't require a direct change to the API would work. > >I wonder if there might be a way one could register piggybacking with a >communicator and somehow have the actual piggybacking occur as a >callback from an implementations messaging layer? > >Just a thought, > >--td >_______________________________________________ >mpi3-ft mailing list >mpi3-ft(a)lists.mpi-forum.org >http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-ft

1 0

[Mpi3-ft] Piggybacking API
by Terry Dontje 21 Apr '08

21 Apr '08

So I reread the piggybacking document on wiki. I am not thrilled with the amount of new APIs this would be adding to the standard but can also see the point of the paper. I am curious how the new API is expected to be used? The proposal say's this API is needed for user-level fault tolerance solutions. So do we expect a user to change all application calls to the MPI library to use the PB calls? I wonder if a more general solution that doesn't require a direct change to the API would work. I wonder if there might be a way one could register piggybacking with a communicator and somehow have the actual piggybacking occur as a callback from an implementations messaging layer? Just a thought, --td

1 0