January 2019 - mpiwg-ft - mailman3.cels.anl.gov

Re: [mpiwg-ft] Init / Finalize PR
by Aurelien Bouteiller 21 Jan '19

21 Jan '19

All, I have updated last week the text for PR#50 to accomodate for the notion of a ‘default handler’ that pre-exists and survives MPI_COMM_SELF. Comments on the updated text a most welcome. I believe there was some contention about the name of the -errh option, but I cannot come with something that makes sense and is not -very- long. Suggestions appreciated. Best, Aurelien > On Jan 18, 2019, at 10:04, Wesley Bland via mpiwg-ft <mpiwg-ft(a)lists.mpi-forum.org> wrote: > > meeting

1 0

Init / Finalize PR
by Wesley Bland 18 Jan '19

18 Jan '19

Hi all, We had a “reading” of PR #50 (https://github.com/mpi-forum/mpi-standard/pull/50) about error behavior during MPI_INIT and MPI_FINALIZE during the December meeting. There were a few things that came out of that which we need to discuss to decide next steps here. Some of this is probably simple and just needs to be added to the text: Add MPI_ERROR_CLASS and MPI_ERROR_STRING to the list of functions callable outside of INIT/FINALIZE The other is a more fundamental shift of the proposal to move specifying the error behavior from mpiexec to MPI_INFO_ENV (which obviously requires allowing info objects to be manipulatable outside of INIT/FINALIZE). It might still be a good idea to have command line flags to set these too. default_errhandler - An info key that defines the default error handler for all situations unless replaced by another. This would include situations currently handled by MPI_COMM_SELF such as errors that have no other place to be raised. This can only be MPI_ERRORS_ARE_FATAL or MPI_ERRORS_RETURN to simplify the requirement of a “universal error handler” predefined_errhandler - Attached to all predefined objects when they are created. This cannot be changed after MPI_INIT because the communicators would have already been created. If their error handlers need to be changed after that, change them directly. If this isn’t defined, it would fall back to default_errhandler. Do people agree with these changes? Should we move forward with updating the proposal to reflect them? Thanks, Wesley

1 0

Larger FT Proposal Next Steps
by Wesley Bland 18 Jan '19

18 Jan '19

Hi all, In lieu of a meeting, here’s a long email trying to move us forward: When we last talked at the December meeting, we went over the fundamentals of what the larger FT proposal needs to include: https://github.com/mpiwg-ft/ft-issues/wiki/2018-12-04#discussion-of-ft-inte…. We decided to work on a few things as a group: Error codes & Error handlers This includes both scoped error handlers (currently in MPI 3.1) and universal error handlers (which would alert you about an error anywhere in the set of connected processes). Function to get a group of failed processes This is different from MPI_COMM_FAILURE_ACK / MPI_COMM_FAILURE_GET_ACKEDbecause of objection to the order of acking and then getting the list of acked processes. Failure acknowledgement function that takes a group Allows the user to restart MPI_ANY_SOURCE communication MPI_COMM_CREATE_GROUP Communicator-based resilient broadcast that triggers error handling on other processes This is similar to the existing MPI_COMM_REVOKE Checkpoint MPI state Return to previous MPI state X Once we had these things in MPI, we could start looking at “Layer 1” (agree and revoke) and the “Layer 2” (shrink). So here’s the action items where I think we can make progress. I think we need someone to take the lead for each of these to keep moving them forward. Universal Error Handlers This will require adding a new type of error handler that doesn’t include any sort of communication object (communicator, window, file) as the MPI process getting the alert may not even be in the group of processes where it makes sense. Function to Retrieve Failed Processes of & Acknowledge Failures During the meeting, it was decided that the current proposal of MPI_COMM_FAILURE_ACK / MPI_COMM_FAILURE_GET_ACKED was not acceptable because of the confusion around acknowledging process failures that you haven’t yet seen. More likely, this would be of the form where a group of failed processes is provided by MPI and then the user acknowledges some subset of those processes via a second function in order to reenable MPI_ANY_SOURCE. If new failures arise that have not yet been acknowledged, MPI_ANY_SOURCE would again be disabled. MPI_COMM_CREATE_GROUP I think what we might have meant here is the new function being promoted by the Sessions working group to do MPI_COMM_CREATE_FROM_GROUP where a parent communicator is not involved. Otherwise the next topic gets very difficult. Communicator-based Resilient Broadcast that Triggers Error Handling on Other Processes This is very similar to the existing MPI_COMM_REVOKE. Something to remember if tempted to do much redesign here: attempting to allow a communicator to be “repaired” in place, rather than constructing a new one (transitioning from revoked to un-revoked) is very racy. It’s unclear what happens if the communicator is revoked twice and un-revoked by some in between. Also, because this function is now somewhat decoupled from the idea of “shrinking” a communicator, it needs to be clear how to create a new working communicator (perhaps with MPI_COMM_CREATE_FROM_GROUP as mentioned above). Checkpoint MPI State & Return to Previous State X This is a totally new topic that goes along with the reinit work that Ignacio has been doing so I think he’s better equipped to offer an initial proposal here. I’m willing to work on the failure reporting / acknowledgement piece and bring back a proposal in a future meeting. Can others choose other pieces to move forward? Thanks, Wesley

1 0

FTWG Topics and Monday Meeting
by Wesley Bland 18 Jan '19

18 Jan '19

Hi folks, Do people want to meet on Monday? At least for me, it will be a holiday so I’m not likely to be available during the time slot. I know at least at the moment, all of our other participants are also in the US. Should we cancel? Can we discuss progress over email? I propose that we do cancel and I’ll follow this up with another email to get the ball rolling on what we can do to make progress in the meantime. Thanks, Wesley

1 0

Re: [mpiwg-ft] [EXTERNAL] Re: Poll: Change of WG meeting timeslot
by Teranishi, Keita 07 Jan '19

07 Jan '19

All, I won’t be able to attend the meeting on 21th (travelling to Europe). Thanks, Keita From: mpiwg-ft <mpiwg-ft-bounces(a)lists.mpi-forum.org> on behalf of rob--- via mpiwg-ft <mpiwg-ft(a)lists.mpi-forum.org> Reply-To: MPI WG Fault Tolerance and Dynamic Process Control working Group <mpiwg-ft(a)lists.mpi-forum.org> Date: Monday, January 7, 2019 at 8:08 AM To: MPI WG Fault Tolerance and Dynamic Process Control working Group <mpiwg-ft(a)lists.mpi-forum.org> Cc: "rob(a)vanderwijngaart.com" <rob(a)vanderwijngaart.com> Subject: [EXTERNAL] Re: [mpiwg-ft] Poll: Change of WG meeting timeslot Sounds good. Happy Fault Tolerant New Year all! Rob On 2019-01-07 06:31, Wesley Bland via mpiwg-ft wrote: Unfortunately I'm in the same situation. Let's just try again on the 21st. On Mon, Jan 7, 2019 at 8:00 AM Aurelien Bouteiller via mpiwg-ft <mpiwg-ft(a)lists.mpi-forum.org<mailto:[email protected]>> wrote: I have a conflicting event that appeared for the 7th so I will not be able to attend. If someone wants to bring forward an agenda item, feel free to use the zoom meeting room. Otherwise I propose reconvening on the 21st. Best, Aurelien On Jan 2, 2019, at 13:52, Aurelien Bouteiller <bouteill(a)icl.utk.edu<mailto:[email protected]>> wrote: Dear all, The option that satisfied the most people was Monday noon EST, we'll repeat every two weeks starting from Monday the 7th. Hi there, Aurelien Bouteiller is inviting you to a scheduled Zoom meeting. Topic: MPI WG Errors and FT Time: This is a recurring meeting Meet anytime Join from PC, Mac, Linux, iOS or Android: https://tennessee.zoom.us/j/632356722?pwd=lI4_169CGcewIumekTziMw Password: mpiforum Or iPhone one-tap (US Toll): +16468769923,632356722# or +16699006833,632356722# Or Telephone: Dial: +1 646 876 9923 (US Toll) +1 669 900 6833 (US Toll) Meeting ID: 632 356 722 International numbers available: https://zoom.us/u/6uINe Or an H.323/SIP room system: H.323: 162.255.37.11 (US West) or 162.255.36.11 (US East) Meeting ID: 632 356 722 Password: 364216 SIP: 632356722(a)zoomcrc.com<mailto:[email protected]> Password: 364216 Best Regards, Aurelien -- Aurelien Bouteiller, Ph.D. Innovative Computing Laboratory; The University of Tennessee Claxton Suite 309; 1122 Volunteer Blvd.; 37996, Knoxville, TN, USA +1 865 974 9375 (p); +1 865 974 8296 (f); Claxton 309 (f2f) On Dec 13, 2018, at 14:20, Aurelien Bouteiller <bouteill(a)icl.utk.edu<mailto:[email protected]>> wrote: Some core contributors to the WG have expressed the need to revise the WG time slot FY19. Please find a Poll to express your preference on the time slot for the bi-weekly Zoom meeting. This will set the first time slot for January, from which we will proceed every two weeks. https://doodle.com/poll/88d22wqe4p74ugmw Best Regards, Aurelien -- Aurelien Bouteiller, Ph.D. Innovative Computing Laboratory; The University of Tennessee Claxton Suite 309; 1122 Volunteer Blvd.; 37996, Knoxville, TN, USA +1 865 974 9375 (p); +1 865 974 8296 (f); Claxton 309 (f2f) _______________________________________________ mpiwg-ft mailing list mpiwg-ft(a)lists.mpi-forum.org<mailto:[email protected]> https://lists.mpi-forum.org/mailman/listinfo/mpiwg-ft _______________________________________________ mpiwg-ft mailing list mpiwg-ft(a)lists.mpi-forum.org<mailto:[email protected]> https://lists.mpi-forum.org/mailman/listinfo/mpiwg-ft

1 0

Re: [mpiwg-ft] Poll: Change of WG meeting timeslot
by rob＠vanderwijngaart.com 07 Jan '19

07 Jan '19

Sounds good. Happy Fault Tolerant New Year all! Rob On 2019-01-07 06:31, Wesley Bland via mpiwg-ft wrote: > Unfortunately I'm in the same situation. Let's just try again on the 21st. > > On Mon, Jan 7, 2019 at 8:00 AM Aurelien Bouteiller via mpiwg-ft <mpiwg-ft(a)lists.mpi-forum.org> wrote: > I have a conflicting event that appeared for the 7th so I will not be able to attend. If someone wants to bring forward an agenda item, feel free to use the zoom meeting room. Otherwise I propose reconvening on the 21st. > > Best, > Aurelien > > On Jan 2, 2019, at 13:52, Aurelien Bouteiller <bouteill(a)icl.utk.edu> wrote: > > Dear all, > > The option that satisfied the most people was Monday noon EST, we'll repeat every two weeks starting from Monday the 7th. > > Hi there, > > Aurelien Bouteiller is inviting you to a scheduled Zoom meeting. > > Topic: MPI WG Errors and FT > Time: This is a recurring meeting Meet anytime > > Join from PC, Mac, Linux, iOS or Android: https://tennessee.zoom.us/j/632356722?pwd=lI4_169CGcewIumekTziMw > Password: mpiforum > > Or iPhone one-tap (US Toll): +16468769923,632356722# or +16699006833,632356722# > > Or Telephone: > Dial: > +1 646 876 9923 (US Toll) > +1 669 900 6833 (US Toll) > Meeting ID: 632 356 722 > International numbers available: https://zoom.us/u/6uINe > > Or an H.323/SIP room system: > H.323: 162.255.37.11 (US West) or 162.255.36.11 (US East) > Meeting ID: 632 356 722 > Password: 364216 > > SIP: 632356722(a)zoomcrc.com > Password: 364216 > > Best Regards, > Aurelien > > -- > Aurelien Bouteiller, Ph.D. > Innovative Computing Laboratory; The University of Tennessee > Claxton Suite 309; 1122 Volunteer Blvd.; 37996, Knoxville, TN, USA > +1 865 974 9375 (p); +1 865 974 8296 (f); Claxton 309 (f2f) > > On Dec 13, 2018, at 14:20, Aurelien Bouteiller <bouteill(a)icl.utk.edu> wrote: > > Some core contributors to the WG have expressed the need to revise the WG time slot FY19. > > Please find a Poll to express your preference on the time slot for the bi-weekly Zoom meeting. This will set the first time slot for January, from which we will proceed every two weeks. > > https://doodle.com/poll/88d22wqe4p74ugmw > > Best Regards, > Aurelien > > -- > Aurelien Bouteiller, Ph.D. > Innovative Computing Laboratory; The University of Tennessee > Claxton Suite 309; 1122 Volunteer Blvd.; 37996, Knoxville, TN, USA > +1 865 974 9375 (p); +1 865 974 8296 (f); Claxton 309 (f2f) _______________________________________________ mpiwg-ft mailing list mpiwg-ft(a)lists.mpi-forum.org https://lists.mpi-forum.org/mailman/listinfo/mpiwg-ft _______________________________________________ mpiwg-ft mailing list mpiwg-ft(a)lists.mpi-forum.org https://lists.mpi-forum.org/mailman/listinfo/mpiwg-ft

1 0

Re: [mpiwg-ft] Poll: Change of WG meeting timeslot
by Wesley Bland 07 Jan '19

07 Jan '19

Unfortunately I'm in the same situation. Let's just try again on the 21st. On Mon, Jan 7, 2019 at 8:00 AM Aurelien Bouteiller via mpiwg-ft < mpiwg-ft(a)lists.mpi-forum.org> wrote: > I have a conflicting event that appeared for the 7th so I will not be able > to attend. If someone wants to bring forward an agenda item, feel free to > use the zoom meeting room. Otherwise I propose reconvening on the 21st. > > Best, > Aurelien > > On Jan 2, 2019, at 13:52, Aurelien Bouteiller <bouteill(a)icl.utk.edu> > wrote: > > Dear all, > > The option that satisfied the most people was Monday noon EST, we’ll > repeat every two weeks starting from Monday the 7th. > > Hi there, > > Aurelien Bouteiller is inviting you to a scheduled Zoom meeting. > > Topic: MPI WG Errors and FT > Time: This is a recurring meeting Meet anytime > > Join from PC, Mac, Linux, iOS or Android: > https://tennessee.zoom.us/j/632356722?pwd=lI4_169CGcewIumekTziMw > Password: mpiforum > > Or iPhone one-tap (US Toll): +16468769923,632356722# or > +16699006833,632356722# > > Or Telephone: > Dial: > +1 646 876 9923 (US Toll) > +1 669 900 6833 (US Toll) > Meeting ID: 632 356 722 > International numbers available: https://zoom.us/u/6uINe > > Or an H.323/SIP room system: > H.323: 162.255.37.11 (US West) or 162.255.36.11 (US East) > Meeting ID: 632 356 722 > Password: 364216 > > SIP: 632356722(a)zoomcrc.com > Password: 364216 > > > > Best Regards, > Aurelien > > -- > Aurelien Bouteiller, Ph.D. > Innovative Computing Laboratory; The University of Tennessee > Claxton Suite 309; 1122 Volunteer Blvd.; 37996, Knoxville, TN, USA > +1 865 974 9375 (p); +1 865 974 8296 (f); Claxton 309 (f2f) > > > > > On Dec 13, 2018, at 14:20, Aurelien Bouteiller <bouteill(a)icl.utk.edu> > wrote: > > Some core contributors to the WG have expressed the need to revise the WG > time slot FY19. > > Please find a Poll to express your preference on the time slot for the > bi-weekly Zoom meeting. This will set the first time slot for January, from > which we will proceed every two weeks. > > https://doodle.com/poll/88d22wqe4p74ugmw > > Best Regards, > Aurelien > > -- > Aurelien Bouteiller, Ph.D. > Innovative Computing Laboratory; The University of Tennessee > Claxton Suite 309; 1122 Volunteer Blvd.; 37996, Knoxville, TN, USA > +1 865 974 9375 (p); +1 865 974 8296 (f); Claxton 309 (f2f) > > > > _______________________________________________ > mpiwg-ft mailing list > mpiwg-ft(a)lists.mpi-forum.org > https://lists.mpi-forum.org/mailman/listinfo/mpiwg-ft >

1 0

Re: [mpiwg-ft] Poll: Change of WG meeting timeslot
by Aurelien Bouteiller 07 Jan '19

07 Jan '19

I have a conflicting event that appeared for the 7th so I will not be able to attend. If someone wants to bring forward an agenda item, feel free to use the zoom meeting room. Otherwise I propose reconvening on the 21st. Best, Aurelien > On Jan 2, 2019, at 13:52, Aurelien Bouteiller <bouteill(a)icl.utk.edu> wrote: > > Dear all, > > The option that satisfied the most people was Monday noon EST, we’ll repeat every two weeks starting from Monday the 7th. > > Hi there, > > Aurelien Bouteiller is inviting you to a scheduled Zoom meeting. > > Topic: MPI WG Errors and FT > Time: This is a recurring meeting Meet anytime > > Join from PC, Mac, Linux, iOS or Android: https://tennessee.zoom.us/j/632356722?pwd=lI4_169CGcewIumekTziMw <https://tennessee.zoom.us/j/632356722?pwd=lI4_169CGcewIumekTziMw> > Password: mpiforum > > Or iPhone one-tap (US Toll): +16468769923,632356722# or +16699006833,632356722# > > Or Telephone: > Dial: > +1 646 876 9923 (US Toll) > +1 669 900 6833 (US Toll) > Meeting ID: 632 356 722 > International numbers available: https://zoom.us/u/6uINe <https://zoom.us/u/6uINe> > > Or an H.323/SIP room system: > H.323: 162.255.37.11 (US West) or 162.255.36.11 (US East) > Meeting ID: 632 356 722 > Password: 364216 > > SIP: 632356722(a)zoomcrc.com <mailto:[email protected]> > Password: 364216 > > > > Best Regards, > Aurelien > > -- > Aurelien Bouteiller, Ph.D. > Innovative Computing Laboratory; The University of Tennessee > Claxton Suite 309; 1122 Volunteer Blvd.; 37996, Knoxville, TN, USA > +1 865 974 9375 (p); +1 865 974 8296 (f); Claxton 309 (f2f) > > > > >> On Dec 13, 2018, at 14:20, Aurelien Bouteiller <bouteill(a)icl.utk.edu <mailto:[email protected]>> wrote: >> >> Some core contributors to the WG have expressed the need to revise the WG time slot FY19. >> >> Please find a Poll to express your preference on the time slot for the bi-weekly Zoom meeting. This will set the first time slot for January, from which we will proceed every two weeks. >> >> https://doodle.com/poll/88d22wqe4p74ugmw <https://doodle.com/poll/88d22wqe4p74ugmw> >> >> Best Regards, >> Aurelien >> >> -- >> Aurelien Bouteiller, Ph.D. >> Innovative Computing Laboratory; The University of Tennessee >> Claxton Suite 309; 1122 Volunteer Blvd.; 37996, Knoxville, TN, USA >> +1 865 974 9375 (p); +1 865 974 8296 (f); Claxton 309 (f2f) >> >

1 0

Re: [mpiwg-ft] Poll: Change of WG meeting timeslot
by Aurelien Bouteiller 02 Jan '19

02 Jan '19

Dear all, The option that satisfied the most people was Monday noon EST, we’ll repeat every two weeks starting from Monday the 7th. Hi there, Aurelien Bouteiller is inviting you to a scheduled Zoom meeting. Topic: MPI WG Errors and FT Time: This is a recurring meeting Meet anytime Join from PC, Mac, Linux, iOS or Android: https://tennessee.zoom.us/j/632356722?pwd=lI4_169CGcewIumekTziMw Password: mpiforum Or iPhone one-tap (US Toll): +16468769923,632356722# or +16699006833,632356722# Or Telephone: Dial: +1 646 876 9923 (US Toll) +1 669 900 6833 (US Toll) Meeting ID: 632 356 722 International numbers available: https://zoom.us/u/6uINe Or an H.323/SIP room system: H.323: 162.255.37.11 (US West) or 162.255.36.11 (US East) Meeting ID: 632 356 722 Password: 364216 SIP: 632356722(a)zoomcrc.com Password: 364216 Best Regards, Aurelien -- Aurelien Bouteiller, Ph.D. Innovative Computing Laboratory; The University of Tennessee Claxton Suite 309; 1122 Volunteer Blvd.; 37996, Knoxville, TN, USA +1 865 974 9375 (p); +1 865 974 8296 (f); Claxton 309 (f2f) > On Dec 13, 2018, at 14:20, Aurelien Bouteiller <bouteill(a)icl.utk.edu> wrote: > > Some core contributors to the WG have expressed the need to revise the WG time slot FY19. > > Please find a Poll to express your preference on the time slot for the bi-weekly Zoom meeting. This will set the first time slot for January, from which we will proceed every two weeks. > > https://doodle.com/poll/88d22wqe4p74ugmw > > Best Regards, > Aurelien > > -- > Aurelien Bouteiller, Ph.D. > Innovative Computing Laboratory; The University of Tennessee > Claxton Suite 309; 1122 Volunteer Blvd.; 37996, Knoxville, TN, USA > +1 865 974 9375 (p); +1 865 974 8296 (f); Claxton 309 (f2f) >

1 0