January 2021 - mpiwg-ft - mailman3.cels.anl.gov

Re: [mpiwg-ft] [EXTERNAL] Re: [Mpi-forum] FTWG Con Call Today
by Ignacio Laguna 25 Jan '21

25 Jan '21

Hi Keita, Good question. We have two operations mode, asynchronous and synchronous. In the asynchronous case, the application automatically jumps back to the reinit point. In the synchronous case, the user places MPI_test_failure calls to check for failures and jump back. The latter is to deal with OpenMP regions because we don't want to jump back in the middle of them. With GPU/Accelerator model, host code will get an error from a kernel launch and will trigger recovery after that. Th user is responsible for cleaning up state in the GPU as well as calling libraries to clean up their state as well. I hope that answers the question. Ignacio On 1/25/21 10:16 AM, Teranishi, Keita wrote: > All, > > From my experience with Fenix (our ULFM-based recovery model similar to ReInit), we found that this approach works with flat-MPI mode. > However, if the program is using an exotic runtime (such as HCLIB, on-node task parallel runtime from GaTech), setjump/longjump messes up the program. > This required us to either (1) refactor the runtime to be MPI-Aware, or give up setjump/longjump approach for the work presented in the last ExaMPI workshop. > > I am wondering how to handle a program with OpenMP/CUDA (and AMD and Intel accelerator runtime) in the setjump-longjump recovery model. It appears that it is safe to clean-up the runtime before calling longjump. With OpenMP, longjump should happen outside any OpenMP pragama (it is easy to implement. I think). With GPU/Accelerator model, all kernel executions should be completed or cancelled. My question is that who is responsible for this type of clean-up. Is it user's responsibility? > > Regards, > Keita > > On 1/25/21, 9:20 AM, "mpiwg-ft on behalf of Ignacio Laguna via mpiwg-ft" <mpiwg-ft-bounces(a)lists.mpi-forum.org on behalf of mpiwg-ft(a)lists.mpi-forum.org> wrote: > > Michael: > > It's very simple. The high-level idea is to encapsulate the main > function into a resilient_main function, which we can call again when a > failure occurs. In our implementation, we use set-jump/long-jump > semantics. As long the encapsulation is done properly, branches higher > up in the stack won't affect it. > > Ignacio > > On 1/25/21 9:13 AM, Steyer, Michael wrote: > > Thanks Ignacio, I'd be very interested in learning how that approach works. Especially the "goes always back to the resilient_function call 1" part, without adding another branch on the call stack? > > > > /Michael > > > > -----Original Message----- > > From: mpiwg-ft <mpiwg-ft-bounces(a)lists.mpi-forum.org> On Behalf Of Ignacio Laguna via mpiwg-ft > > Sent: Montag, 25. Januar 2021 17:33 > > To: MPI WG Fault Tolerance and Dynamic Process Control working Group <mpiwg-ft(a)lists.mpi-forum.org> > > Cc: Ignacio Laguna <lagunaperalt1(a)llnl.gov> > > Subject: Re: [mpiwg-ft] [Mpi-forum] FTWG Con Call Today > > > > The model is that the app goes always back to the resilient_function call 1 (we cannot call this function twice or more statically in the program). Perhaps we can discuss that again. > > > > Ignacio > > > > > > On 1/25/21 8:25 AM, Wesley Bland via mpiwg-ft wrote: > >> There was another question that came up in internal conversations > >> around here with Reinit: > >> > >> What's going to happen to the call stack. E.g. MPI_Init -> ... -> > >> resilient_function call 1 -> Failure -> ReInit - resilient_function > >> call > >> 2 -> End of Work -> back to resilient_function call 1? > >> > >> On Mon, Jan 25, 2021 at 10:01 AM Ignacio Laguna > >> <lagunaperalt1(a)llnl.gov <mailto:[email protected]>> wrote: > >> > >> That works for me (I couldn't attend today neither). > >> > >> We are almost done with the new Reinit spec but we have a few topics we > >> would like to discuss in the group: (1) using several error handlers > >> and > >> how this is specified in the standard, (2) the state of MPI between a > >> failure and its recovery (how does ULFM does it? Perhaps Reinit can > >> re-use the same text?). > >> > >> Thanks! > >> > >> Ignacio > >> > >> On 1/25/21 6:20 AM, Wesley Bland via mpi-forum wrote: > >> > Hi all, > >> > > >> > After talking to Tony, we're going to delay this discussion until > >> the > >> > next call on Feb 8. Today's call is cancelled. > >> > > >> > Thanks, > >> > Wes > >> > > >> > On Mon, Jan 25, 2021 at 8:15 AM work(a)wesbland.com > >> <mailto:[email protected]> > >> > <mailto:[email protected] <mailto:[email protected]>> > >> <work(a)wesbland.com <mailto:[email protected]> > >> > <mailto:[email protected] <mailto:[email protected]>>> wrote: > >> > > >> > The Fault Tolerance Working Group’s weekly con call is today at > >> > 12:00 PM Eastern. Today's agenda:____ > >> > > >> > __ __ > >> > > >> > * FA-MPI (Tony)____ > >> > > >> > * Other updates (All)____ > >> > > >> > __ __ > >> > > >> > If there's something else that people would like to discuss, > >> please > >> > just send an email to the WG so we can get it on the agenda.____ > >> > > >> > __ __ > >> > > >> > Thanks, ____ > >> > > >> > Wes ____ > >> > > >> > __ __ > >> > > >> > > >> ....................................................................................................................................... > >> > ____ > >> > > >> > Join from PC, Mac, Linux, iOS or Android: > >> > https://tennessee.zoom.us/j/632356722?pwd=lI4_169CGcewIumekTziMw____ > >> > > >> > Password: mpiforum____ > >> > > >> > __ __ > >> > > >> > Or iPhone one-tap (US Toll): +16468769923,632356722# or > >> > +16699006833,632356722# ____ > >> > > >> > __ __ > >> > > >> > Or Telephone:____ > >> > > >> > Dial:____ > >> > > >> > +1 646 876 9923 (US Toll)____ > >> > > >> > +1 669 900 6833 (US Toll)____ > >> > > >> > Meeting ID: 632 356 722____ > >> > > >> > International numbers available: https://zoom.us/u/6uINe____ > >> > > >> > __ __ > >> > > >> > Or an H.323/SIP room system:____ > >> > > >> > H.323: 162.255.37.11 (US West) or 162.255.36.11 (US > >> East) ____ > >> > > >> > Meeting ID: 632 356 722____ > >> > > >> > Password: 364216____ > >> > > >> > __ __ > >> > > >> > SIP: 632356722(a)zoomcrc.com > >> <mailto:[email protected]> <mailto:[email protected] > >> <mailto:[email protected]>>____ > >> > > >> > Password: 364216____ > >> > > >> > > >> .......................................................................................................................................____ > >> > > >> > > >> > _______________________________________________ > >> > mpi-forum mailing list > >> > mpi-forum(a)lists.mpi-forum.org <mailto:[email protected]> > >> > https://lists.mpi-forum.org/mailman/listinfo/mpi-forum > >> > > >> > >> > >> _______________________________________________ > >> mpiwg-ft mailing list > >> mpiwg-ft(a)lists.mpi-forum.org > >> https://lists.mpi-forum.org/mailman/listinfo/mpiwg-ft > >> > > _______________________________________________ > > mpiwg-ft mailing list > > mpiwg-ft(a)lists.mpi-forum.org > > https://lists.mpi-forum.org/mailman/listinfo/mpiwg-ft > > Intel Deutschland GmbH > > Registered Address: Am Campeon 10-12, 85579 Neubiberg, Germany > > Tel: +49 89 99 8853-0, www.intel.de > > Managing Directors: Christin Eisenschmid, Gary Kershaw > > Chairperson of the Supervisory Board: Nicole Lau > > Registered Office: Munich > > Commercial Register: Amtsgericht Muenchen HRB 186928 > > > _______________________________________________ > mpiwg-ft mailing list > mpiwg-ft(a)lists.mpi-forum.org > https://lists.mpi-forum.org/mailman/listinfo/mpiwg-ft > >

1 0

Re: [mpiwg-ft] [EXTERNAL] Re: [Mpi-forum] FTWG Con Call Today
by Teranishi, Keita 25 Jan '21

25 Jan '21

All, From my experience with Fenix (our ULFM-based recovery model similar to ReInit), we found that this approach works with flat-MPI mode. However, if the program is using an exotic runtime (such as HCLIB, on-node task parallel runtime from GaTech), setjump/longjump messes up the program. This required us to either (1) refactor the runtime to be MPI-Aware, or give up setjump/longjump approach for the work presented in the last ExaMPI workshop. I am wondering how to handle a program with OpenMP/CUDA (and AMD and Intel accelerator runtime) in the setjump-longjump recovery model. It appears that it is safe to clean-up the runtime before calling longjump. With OpenMP, longjump should happen outside any OpenMP pragama (it is easy to implement. I think). With GPU/Accelerator model, all kernel executions should be completed or cancelled. My question is that who is responsible for this type of clean-up. Is it user's responsibility? Regards, Keita On 1/25/21, 9:20 AM, "mpiwg-ft on behalf of Ignacio Laguna via mpiwg-ft" <mpiwg-ft-bounces(a)lists.mpi-forum.org on behalf of mpiwg-ft(a)lists.mpi-forum.org> wrote: Michael: It's very simple. The high-level idea is to encapsulate the main function into a resilient_main function, which we can call again when a failure occurs. In our implementation, we use set-jump/long-jump semantics. As long the encapsulation is done properly, branches higher up in the stack won't affect it. Ignacio On 1/25/21 9:13 AM, Steyer, Michael wrote: > Thanks Ignacio, I'd be very interested in learning how that approach works. Especially the "goes always back to the resilient_function call 1" part, without adding another branch on the call stack? > > /Michael > > -----Original Message----- > From: mpiwg-ft <mpiwg-ft-bounces(a)lists.mpi-forum.org> On Behalf Of Ignacio Laguna via mpiwg-ft > Sent: Montag, 25. Januar 2021 17:33 > To: MPI WG Fault Tolerance and Dynamic Process Control working Group <mpiwg-ft(a)lists.mpi-forum.org> > Cc: Ignacio Laguna <lagunaperalt1(a)llnl.gov> > Subject: Re: [mpiwg-ft] [Mpi-forum] FTWG Con Call Today > > The model is that the app goes always back to the resilient_function call 1 (we cannot call this function twice or more statically in the program). Perhaps we can discuss that again. > > Ignacio > > > On 1/25/21 8:25 AM, Wesley Bland via mpiwg-ft wrote: >> There was another question that came up in internal conversations >> around here with Reinit: >> >> What's going to happen to the call stack. E.g. MPI_Init -> ... -> >> resilient_function call 1 -> Failure -> ReInit - resilient_function >> call >> 2 -> End of Work -> back to resilient_function call 1? >> >> On Mon, Jan 25, 2021 at 10:01 AM Ignacio Laguna >> <lagunaperalt1(a)llnl.gov <mailto:[email protected]>> wrote: >> >> That works for me (I couldn't attend today neither). >> >> We are almost done with the new Reinit spec but we have a few topics we >> would like to discuss in the group: (1) using several error handlers >> and >> how this is specified in the standard, (2) the state of MPI between a >> failure and its recovery (how does ULFM does it? Perhaps Reinit can >> re-use the same text?). >> >> Thanks! >> >> Ignacio >> >> On 1/25/21 6:20 AM, Wesley Bland via mpi-forum wrote: >> > Hi all, >> > >> > After talking to Tony, we're going to delay this discussion until >> the >> > next call on Feb 8. Today's call is cancelled. >> > >> > Thanks, >> > Wes >> > >> > On Mon, Jan 25, 2021 at 8:15 AM work(a)wesbland.com >> <mailto:[email protected]> >> > <mailto:[email protected] <mailto:[email protected]>> >> <work(a)wesbland.com <mailto:[email protected]> >> > <mailto:[email protected] <mailto:[email protected]>>> wrote: >> > >> > The Fault Tolerance Working Group’s weekly con call is today at >> > 12:00 PM Eastern. Today's agenda:____ >> > >> > __ __ >> > >> > * FA-MPI (Tony)____ >> > >> > * Other updates (All)____ >> > >> > __ __ >> > >> > If there's something else that people would like to discuss, >> please >> > just send an email to the WG so we can get it on the agenda.____ >> > >> > __ __ >> > >> > Thanks, ____ >> > >> > Wes ____ >> > >> > __ __ >> > >> > >> ....................................................................................................................................... >> > ____ >> > >> > Join from PC, Mac, Linux, iOS or Android: >> > https://tennessee.zoom.us/j/632356722?pwd=lI4_169CGcewIumekTziMw____ >> > >> > Password: mpiforum____ >> > >> > __ __ >> > >> > Or iPhone one-tap (US Toll): +16468769923,632356722# or >> > +16699006833,632356722# ____ >> > >> > __ __ >> > >> > Or Telephone:____ >> > >> > Dial:____ >> > >> > +1 646 876 9923 (US Toll)____ >> > >> > +1 669 900 6833 (US Toll)____ >> > >> > Meeting ID: 632 356 722____ >> > >> > International numbers available: https://zoom.us/u/6uINe____ >> > >> > __ __ >> > >> > Or an H.323/SIP room system:____ >> > >> > H.323: 162.255.37.11 (US West) or 162.255.36.11 (US >> East) ____ >> > >> > Meeting ID: 632 356 722____ >> > >> > Password: 364216____ >> > >> > __ __ >> > >> > SIP: 632356722(a)zoomcrc.com >> <mailto:[email protected]> <mailto:[email protected] >> <mailto:[email protected]>>____ >> > >> > Password: 364216____ >> > >> > >> .......................................................................................................................................____ >> > >> > >> > _______________________________________________ >> > mpi-forum mailing list >> > mpi-forum(a)lists.mpi-forum.org <mailto:[email protected]> >> > https://lists.mpi-forum.org/mailman/listinfo/mpi-forum >> > >> >> >> _______________________________________________ >> mpiwg-ft mailing list >> mpiwg-ft(a)lists.mpi-forum.org >> https://lists.mpi-forum.org/mailman/listinfo/mpiwg-ft >> > _______________________________________________ > mpiwg-ft mailing list > mpiwg-ft(a)lists.mpi-forum.org > https://lists.mpi-forum.org/mailman/listinfo/mpiwg-ft > Intel Deutschland GmbH > Registered Address: Am Campeon 10-12, 85579 Neubiberg, Germany > Tel: +49 89 99 8853-0, www.intel.de > Managing Directors: Christin Eisenschmid, Gary Kershaw > Chairperson of the Supervisory Board: Nicole Lau > Registered Office: Munich > Commercial Register: Amtsgericht Muenchen HRB 186928 > _______________________________________________ mpiwg-ft mailing list mpiwg-ft(a)lists.mpi-forum.org https://lists.mpi-forum.org/mailman/listinfo/mpiwg-ft

1 0

Re: [mpiwg-ft] [Mpi-forum] FTWG Con Call Today
by Ignacio Laguna 25 Jan '21

25 Jan '21

Michael: It's very simple. The high-level idea is to encapsulate the main function into a resilient_main function, which we can call again when a failure occurs. In our implementation, we use set-jump/long-jump semantics. As long the encapsulation is done properly, branches higher up in the stack won't affect it. Ignacio On 1/25/21 9:13 AM, Steyer, Michael wrote: > Thanks Ignacio, I'd be very interested in learning how that approach works. Especially the "goes always back to the resilient_function call 1" part, without adding another branch on the call stack? > > /Michael > > -----Original Message----- > From: mpiwg-ft <mpiwg-ft-bounces(a)lists.mpi-forum.org> On Behalf Of Ignacio Laguna via mpiwg-ft > Sent: Montag, 25. Januar 2021 17:33 > To: MPI WG Fault Tolerance and Dynamic Process Control working Group <mpiwg-ft(a)lists.mpi-forum.org> > Cc: Ignacio Laguna <lagunaperalt1(a)llnl.gov> > Subject: Re: [mpiwg-ft] [Mpi-forum] FTWG Con Call Today > > The model is that the app goes always back to the resilient_function call 1 (we cannot call this function twice or more statically in the program). Perhaps we can discuss that again. > > Ignacio > > > On 1/25/21 8:25 AM, Wesley Bland via mpiwg-ft wrote: >> There was another question that came up in internal conversations >> around here with Reinit: >> >> What's going to happen to the call stack. E.g. MPI_Init -> ... -> >> resilient_function call 1 -> Failure -> ReInit - resilient_function >> call >> 2 -> End of Work -> back to resilient_function call 1? >> >> On Mon, Jan 25, 2021 at 10:01 AM Ignacio Laguna >> <lagunaperalt1(a)llnl.gov <mailto:[email protected]>> wrote: >> >> That works for me (I couldn't attend today neither). >> >> We are almost done with the new Reinit spec but we have a few topics we >> would like to discuss in the group: (1) using several error handlers >> and >> how this is specified in the standard, (2) the state of MPI between a >> failure and its recovery (how does ULFM does it? Perhaps Reinit can >> re-use the same text?). >> >> Thanks! >> >> Ignacio >> >> On 1/25/21 6:20 AM, Wesley Bland via mpi-forum wrote: >> > Hi all, >> > >> > After talking to Tony, we're going to delay this discussion until >> the >> > next call on Feb 8. Today's call is cancelled. >> > >> > Thanks, >> > Wes >> > >> > On Mon, Jan 25, 2021 at 8:15 AM work(a)wesbland.com >> <mailto:[email protected]> >> > <mailto:[email protected] <mailto:[email protected]>> >> <work(a)wesbland.com <mailto:[email protected]> >> > <mailto:[email protected] <mailto:[email protected]>>> wrote: >> > >> > The Fault Tolerance Working Group’s weekly con call is today at >> > 12:00 PM Eastern. Today's agenda:____ >> > >> > __ __ >> > >> > * FA-MPI (Tony)____ >> > >> > * Other updates (All)____ >> > >> > __ __ >> > >> > If there's something else that people would like to discuss, >> please >> > just send an email to the WG so we can get it on the agenda.____ >> > >> > __ __ >> > >> > Thanks, ____ >> > >> > Wes ____ >> > >> > __ __ >> > >> > >> ....................................................................................................................................... >> > ____ >> > >> > Join from PC, Mac, Linux, iOS or Android: >> > https://tennessee.zoom.us/j/632356722?pwd=lI4_169CGcewIumekTziMw____ >> > >> > Password: mpiforum____ >> > >> > __ __ >> > >> > Or iPhone one-tap (US Toll): +16468769923,632356722# or >> > +16699006833,632356722# ____ >> > >> > __ __ >> > >> > Or Telephone:____ >> > >> > Dial:____ >> > >> > +1 646 876 9923 (US Toll)____ >> > >> > +1 669 900 6833 (US Toll)____ >> > >> > Meeting ID: 632 356 722____ >> > >> > International numbers available: https://zoom.us/u/6uINe____ >> > >> > __ __ >> > >> > Or an H.323/SIP room system:____ >> > >> > H.323: 162.255.37.11 (US West) or 162.255.36.11 (US >> East) ____ >> > >> > Meeting ID: 632 356 722____ >> > >> > Password: 364216____ >> > >> > __ __ >> > >> > SIP: 632356722(a)zoomcrc.com >> <mailto:[email protected]> <mailto:[email protected] >> <mailto:[email protected]>>____ >> > >> > Password: 364216____ >> > >> > >> .......................................................................................................................................____ >> > >> > >> > _______________________________________________ >> > mpi-forum mailing list >> > mpi-forum(a)lists.mpi-forum.org <mailto:[email protected]> >> > https://lists.mpi-forum.org/mailman/listinfo/mpi-forum >> > >> >> >> _______________________________________________ >> mpiwg-ft mailing list >> mpiwg-ft(a)lists.mpi-forum.org >> https://lists.mpi-forum.org/mailman/listinfo/mpiwg-ft >> > _______________________________________________ > mpiwg-ft mailing list > mpiwg-ft(a)lists.mpi-forum.org > https://lists.mpi-forum.org/mailman/listinfo/mpiwg-ft > Intel Deutschland GmbH > Registered Address: Am Campeon 10-12, 85579 Neubiberg, Germany > Tel: +49 89 99 8853-0, www.intel.de > Managing Directors: Christin Eisenschmid, Gary Kershaw > Chairperson of the Supervisory Board: Nicole Lau > Registered Office: Munich > Commercial Register: Amtsgericht Muenchen HRB 186928 >

1 0

Re: [mpiwg-ft] [Mpi-forum] FTWG Con Call Today
by Steyer, Michael 25 Jan '21

25 Jan '21

Thanks Ignacio, I'd be very interested in learning how that approach works. Especially the "goes always back to the resilient_function call 1" part, without adding another branch on the call stack? /Michael -----Original Message----- From: mpiwg-ft <mpiwg-ft-bounces(a)lists.mpi-forum.org> On Behalf Of Ignacio Laguna via mpiwg-ft Sent: Montag, 25. Januar 2021 17:33 To: MPI WG Fault Tolerance and Dynamic Process Control working Group <mpiwg-ft(a)lists.mpi-forum.org> Cc: Ignacio Laguna <lagunaperalt1(a)llnl.gov> Subject: Re: [mpiwg-ft] [Mpi-forum] FTWG Con Call Today The model is that the app goes always back to the resilient_function call 1 (we cannot call this function twice or more statically in the program). Perhaps we can discuss that again. Ignacio On 1/25/21 8:25 AM, Wesley Bland via mpiwg-ft wrote: > There was another question that came up in internal conversations > around here with Reinit: > > What's going to happen to the call stack. E.g. MPI_Init -> ... -> > resilient_function call 1 -> Failure -> ReInit - resilient_function > call > 2 -> End of Work -> back to resilient_function call 1? > > On Mon, Jan 25, 2021 at 10:01 AM Ignacio Laguna > <lagunaperalt1(a)llnl.gov <mailto:[email protected]>> wrote: > > That works for me (I couldn't attend today neither). > > We are almost done with the new Reinit spec but we have a few topics we > would like to discuss in the group: (1) using several error handlers > and > how this is specified in the standard, (2) the state of MPI between a > failure and its recovery (how does ULFM does it? Perhaps Reinit can > re-use the same text?). > > Thanks! > > Ignacio > > On 1/25/21 6:20 AM, Wesley Bland via mpi-forum wrote: > > Hi all, > > > > After talking to Tony, we're going to delay this discussion until > the > > next call on Feb 8. Today's call is cancelled. > > > > Thanks, > > Wes > > > > On Mon, Jan 25, 2021 at 8:15 AM work(a)wesbland.com > <mailto:[email protected]> > > <mailto:[email protected] <mailto:[email protected]>> > <work(a)wesbland.com <mailto:[email protected]> > > <mailto:[email protected] <mailto:[email protected]>>> wrote: > > > > The Fault Tolerance Working Group’s weekly con call is today at > > 12:00 PM Eastern. Today's agenda:____ > > > > __ __ > > > > * FA-MPI (Tony)____ > > > > * Other updates (All)____ > > > > __ __ > > > > If there's something else that people would like to discuss, > please > > just send an email to the WG so we can get it on the agenda.____ > > > > __ __ > > > > Thanks, ____ > > > > Wes ____ > > > > __ __ > > > > > ....................................................................................................................................... > > ____ > > > > Join from PC, Mac, Linux, iOS or Android: > > https://tennessee.zoom.us/j/632356722?pwd=lI4_169CGcewIumekTziMw____ > > > > Password: mpiforum____ > > > > __ __ > > > > Or iPhone one-tap (US Toll): +16468769923,632356722# or > > +16699006833,632356722# ____ > > > > __ __ > > > > Or Telephone:____ > > > > Dial:____ > > > > +1 646 876 9923 (US Toll)____ > > > > +1 669 900 6833 (US Toll)____ > > > > Meeting ID: 632 356 722____ > > > > International numbers available: https://zoom.us/u/6uINe____ > > > > __ __ > > > > Or an H.323/SIP room system:____ > > > > H.323: 162.255.37.11 (US West) or 162.255.36.11 (US > East) ____ > > > > Meeting ID: 632 356 722____ > > > > Password: 364216____ > > > > __ __ > > > > SIP: 632356722(a)zoomcrc.com > <mailto:[email protected]> <mailto:[email protected] > <mailto:[email protected]>>____ > > > > Password: 364216____ > > > > > .......................................................................................................................................____ > > > > > > _______________________________________________ > > mpi-forum mailing list > > mpi-forum(a)lists.mpi-forum.org <mailto:[email protected]> > > https://lists.mpi-forum.org/mailman/listinfo/mpi-forum > > > > > _______________________________________________ > mpiwg-ft mailing list > mpiwg-ft(a)lists.mpi-forum.org > https://lists.mpi-forum.org/mailman/listinfo/mpiwg-ft > _______________________________________________ mpiwg-ft mailing list mpiwg-ft(a)lists.mpi-forum.org https://lists.mpi-forum.org/mailman/listinfo/mpiwg-ft Intel Deutschland GmbH Registered Address: Am Campeon 10-12, 85579 Neubiberg, Germany Tel: +49 89 99 8853-0, www.intel.de Managing Directors: Christin Eisenschmid, Gary Kershaw Chairperson of the Supervisory Board: Nicole Lau Registered Office: Munich Commercial Register: Amtsgericht Muenchen HRB 186928

1 0

Re: [mpiwg-ft] [Mpi-forum] FTWG Con Call Today
by Ignacio Laguna 25 Jan '21

25 Jan '21

The model is that the app goes always back to the resilient_function call 1 (we cannot call this function twice or more statically in the program). Perhaps we can discuss that again. Ignacio On 1/25/21 8:25 AM, Wesley Bland via mpiwg-ft wrote: > There was another question that came up in internal conversations around > here with Reinit: > > What's going to happen to the call stack. E.g. MPI_Init -> ... -> > resilient_function call 1 -> Failure -> ReInit - resilient_function call > 2 -> End of Work -> back to resilient_function call 1? > > On Mon, Jan 25, 2021 at 10:01 AM Ignacio Laguna <lagunaperalt1(a)llnl.gov > <mailto:[email protected]>> wrote: > > That works for me (I couldn't attend today neither). > > We are almost done with the new Reinit spec but we have a few topics we > would like to discuss in the group: (1) using several error handlers > and > how this is specified in the standard, (2) the state of MPI between a > failure and its recovery (how does ULFM does it? Perhaps Reinit can > re-use the same text?). > > Thanks! > > Ignacio > > On 1/25/21 6:20 AM, Wesley Bland via mpi-forum wrote: > > Hi all, > > > > After talking to Tony, we're going to delay this discussion until > the > > next call on Feb 8. Today's call is cancelled. > > > > Thanks, > > Wes > > > > On Mon, Jan 25, 2021 at 8:15 AM work(a)wesbland.com > <mailto:[email protected]> > > <mailto:[email protected] <mailto:[email protected]>> > <work(a)wesbland.com <mailto:[email protected]> > > <mailto:[email protected] <mailto:[email protected]>>> wrote: > > > > The Fault Tolerance Working Group’s weekly con call is today at > > 12:00 PM Eastern. Today's agenda:____ > > > > __ __ > > > > * FA-MPI (Tony)____ > > > > * Other updates (All)____ > > > > __ __ > > > > If there's something else that people would like to discuss, > please > > just send an email to the WG so we can get it on the agenda.____ > > > > __ __ > > > > Thanks, ____ > > > > Wes ____ > > > > __ __ > > > > > ....................................................................................................................................... > > ____ > > > > Join from PC, Mac, Linux, iOS or Android: > > https://tennessee.zoom.us/j/632356722?pwd=lI4_169CGcewIumekTziMw____ > > > > Password: mpiforum____ > > > > __ __ > > > > Or iPhone one-tap (US Toll): +16468769923,632356722# or > > +16699006833,632356722# ____ > > > > __ __ > > > > Or Telephone:____ > > > > Dial:____ > > > > +1 646 876 9923 (US Toll)____ > > > > +1 669 900 6833 (US Toll)____ > > > > Meeting ID: 632 356 722____ > > > > International numbers available: https://zoom.us/u/6uINe____ > > > > __ __ > > > > Or an H.323/SIP room system:____ > > > > H.323: 162.255.37.11 (US West) or 162.255.36.11 (US > East) ____ > > > > Meeting ID: 632 356 722____ > > > > Password: 364216____ > > > > __ __ > > > > SIP: 632356722(a)zoomcrc.com > <mailto:[email protected]> <mailto:[email protected] > <mailto:[email protected]>>____ > > > > Password: 364216____ > > > > > .......................................................................................................................................____ > > > > > > _______________________________________________ > > mpi-forum mailing list > > mpi-forum(a)lists.mpi-forum.org <mailto:[email protected]> > > https://lists.mpi-forum.org/mailman/listinfo/mpi-forum > > > > > _______________________________________________ > mpiwg-ft mailing list > mpiwg-ft(a)lists.mpi-forum.org > https://lists.mpi-forum.org/mailman/listinfo/mpiwg-ft >

1 0

Re: [mpiwg-ft] [Mpi-forum] FTWG Con Call Today
by Wesley Bland 25 Jan '21

25 Jan '21

There was another question that came up in internal conversations around here with Reinit: What's going to happen to the call stack. E.g. MPI_Init -> ... -> resilient_function call 1 -> Failure -> ReInit - resilient_function call 2 -> End of Work -> back to resilient_function call 1? On Mon, Jan 25, 2021 at 10:01 AM Ignacio Laguna <lagunaperalt1(a)llnl.gov> wrote: > That works for me (I couldn't attend today neither). > > We are almost done with the new Reinit spec but we have a few topics we > would like to discuss in the group: (1) using several error handlers and > how this is specified in the standard, (2) the state of MPI between a > failure and its recovery (how does ULFM does it? Perhaps Reinit can > re-use the same text?). > > Thanks! > > Ignacio > > On 1/25/21 6:20 AM, Wesley Bland via mpi-forum wrote: > > Hi all, > > > > After talking to Tony, we're going to delay this discussion until the > > next call on Feb 8. Today's call is cancelled. > > > > Thanks, > > Wes > > > > On Mon, Jan 25, 2021 at 8:15 AM work(a)wesbland.com > > <mailto:[email protected]> <work(a)wesbland.com > > <mailto:[email protected]>> wrote: > > > > The Fault Tolerance Working Group’s weekly con call is today at > > 12:00 PM Eastern. Today's agenda:____ > > > > __ __ > > > > * FA-MPI (Tony)____ > > > > * Other updates (All)____ > > > > __ __ > > > > If there's something else that people would like to discuss, please > > just send an email to the WG so we can get it on the agenda.____ > > > > __ __ > > > > Thanks, ____ > > > > Wes ____ > > > > __ __ > > > > > ....................................................................................................................................... > > ____ > > > > Join from PC, Mac, Linux, iOS or Android: > > https://tennessee.zoom.us/j/632356722?pwd=lI4_169CGcewIumekTziMw____ > > > > Password: mpiforum____ > > > > __ __ > > > > Or iPhone one-tap (US Toll): +16468769923,632356722# or > > +16699006833,632356722# ____ > > > > __ __ > > > > Or Telephone:____ > > > > Dial:____ > > > > +1 646 876 9923 (US Toll)____ > > > > +1 669 900 6833 (US Toll)____ > > > > Meeting ID: 632 356 722____ > > > > International numbers available: https://zoom.us/u/6uINe____ > > > > __ __ > > > > Or an H.323/SIP room system:____ > > > > H.323: 162.255.37.11 (US West) or 162.255.36.11 (US East) ____ > > > > Meeting ID: 632 356 722____ > > > > Password: 364216____ > > > > __ __ > > > > SIP: 632356722(a)zoomcrc.com <mailto:[email protected]>____ > > > > Password: 364216____ > > > > > .......................................................................................................................................____ > > > > > > _______________________________________________ > > mpi-forum mailing list > > mpi-forum(a)lists.mpi-forum.org > > https://lists.mpi-forum.org/mailman/listinfo/mpi-forum > > >

1 0

Cancelling Today's FTWG
by Wesley Bland 11 Jan '21

11 Jan '21

Hi all, Because of the deadline for MPI 4.0 tickets today, Tony won’t be able to present the FA-MPI work that was planned for today. So I’m proposing that we cancel this week’s meeting and have that talk in two weeks instead. Thanks, Wes

1 0