May 2016 - mpiwg-sessions - mailman3.cels.anl.gov

Re: [mpiwg-sessions] Cancel 30 May WG meeting / Forum WG time
by Jeff Squyres (jsquyres) 29 May '16

29 May '16

Don't forget: the Sessions WG webex is canceled tomorrow. See you Tuesday afternoon in Bellevue (June 7) for the Sessions WG meetings. > On May 26, 2016, at 6:53 AM, Jeff Squyres (jsquyres) <jsquyres(a)cisco.com> wrote: > > Monday, 30 May 2016 is a holiday for many of us here in the US (Memorial Day). I'm canceling the sessions working group meeting that day. > > Please look over the slides, anyway -- there's new content and questions (which can be discussed here on the mailing list): > > https://github.com/mpiwg-sessions/sessions-issues/wiki/2016-05-30-webex > > • Slides > • slides 14-15: required/provided for thread level > • slide 20: resolved -- only static sets (no dynamics) > • slide 29: only get set names for which you are a member > • slide 30: Updated the "open questions" > • slides 78-70: minor updates to MPI_EXEC > • slides 83-91: A bunch of new slides about "shrink the app" use case > > • Discussion points for today: > > • Has the FT WG discussed the universal naming mechanism idea? (slide 31) > • Paired with the idea of: names are private until I publish them > • Do we have a use case for getting group names to which you are not a member? > • Still unclear: do we want to create win/file from group? (slides 43/44) > > -- > Jeff Squyres > jsquyres(a)cisco.com > For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/ > > _______________________________________________ > mpiwg-sessions mailing list > mpiwg-sessions(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-sessions -- Jeff Squyres jsquyres(a)cisco.com For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/

1 0

Cancel 30 May WG meeting / Forum WG time
by Jeff Squyres (jsquyres) 26 May '16

26 May '16

Monday, 30 May 2016 is a holiday for many of us here in the US (Memorial Day). I'm canceling the sessions working group meeting that day. Please look over the slides, anyway -- there's new content and questions (which can be discussed here on the mailing list): https://github.com/mpiwg-sessions/sessions-issues/wiki/2016-05-30-webex • Slides • slides 14-15: required/provided for thread level • slide 20: resolved -- only static sets (no dynamics) • slide 29: only get set names for which you are a member • slide 30: Updated the "open questions" • slides 78-70: minor updates to MPI_EXEC • slides 83-91: A bunch of new slides about "shrink the app" use case • Discussion points for today: • Has the FT WG discussed the universal naming mechanism idea? (slide 31) • Paired with the idea of: names are private until I publish them • Do we have a use case for getting group names to which you are not a member? • Still unclear: do we want to create win/file from group? (slides 43/44) -- Jeff Squyres jsquyres(a)cisco.com For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/

1 0

Updates to slides for next Monday
by Jeff Squyres (jsquyres) 23 May '16

23 May '16

I talked with Dan today about a few items, and have updated the slides for next Monday based on our conversation: https://github.com/mpiwg-sessions/sessions-issues/wiki/2016-05-30-webex -- Jeff Squyres jsquyres(a)cisco.com For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/

1 0

Re: [mpiwg-sessions] Notes from today / slides for next time
by Teranishi, Keita 17 May '16

17 May '16

Jeff, Thanks for the clarification. Now the example makes sense to me. BTW, I know a couple of applied mathematicians in Argonne asked such a shrinking capability (when I talked ULFM and Fenix there). The picture in the slides matches their use case. :) Regards, Keita On 5/17/16, 7:57 AM, "mpiwg-sessions on behalf of Jeff Squyres (jsquyres)" <mpiwg-sessions-bounces(a)lists.mpi-forum.org on behalf of jsquyres(a)cisco.com> wrote: >Put simply: your use case is grow then shrink, right? > >The case I show in slides 83-91 is just shrink. But I think it generalizes to any case of shrink (i.e., regardless of whether you previously grew). I think that the main issues are: > >- previously-obtained sets may become stale >- the release manager may want to (implicitly or explicitly) release nodes that are no longer being used (this may or may not be in MPI's prevue) > > > >> On May 16, 2016, at 6:31 PM, Teranishi, Keita <knteran(a)sandia.gov> wrote: >> >> Jeff, >> >> Thanks for the update. Our potential use case in Sandia is less aggressive than the model in pages 83-91. Typically, the main application occupies the 70-90% of the total nodes for the entire execution time. The application data analytics routines are spawned and joined (shrunk) on the rest of the nodes. Data movement happens between the ranks running base application and the spawned ranks, but I believe that doing within MPI is faster than using external fast-IO services (such as DataSpaces). >> >> Maybe, the best use case I can map to your example is reduced order modeling, which can be seen as interactions between “big” and “small" simulation instances. Of course, it can be done with master-worker model, but spawn+shrink (join+leave) model would allow more flexible use of big machines. >> >> Thanks, >> Keita >> >> >> >> >> >> On 5/16/16, 3:00 PM, "mpiwg-sessions on behalf of Jeff Squyres (jsquyres)" <mpiwg-sessions-bounces(a)lists.mpi-forum.org on behalf of jsquyres(a)cisco.com> wrote: >> >>> Thanks for everyone's time today. Here's notes and slides from today: >>> >>> https://github.com/mpiwg-sessions/sessions-issues/wiki/2016-05-16-webex >>> >>> I consolidated the open issues, and added a bunch of information about the "grow" and "shrink" use cases that we want to support, and made slides for 2 weeks from now -- please have a look: >>> >>> https://github.com/mpiwg-sessions/sessions-issues/wiki/2016-05-30-webex >>> >>> Hilights: >>> >>> • slides 14-15: required/provided for thread level >>> • slide 20: resolved -- only static sets (no dynamics) >>> • slide 29: only get set names for which you are a member >>> • slide 30: Updated the "open questions" >>> • slides 78-70: minor updates to MPI_EXEC >>> • slides 83-91: A bunch of new slides about "shrink the app" use case >>> >>> -- >>> Jeff Squyres >>> jsquyres(a)cisco.com >>> For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/ >>> >>> _______________________________________________ >>> mpiwg-sessions mailing list >>> mpiwg-sessions(a)lists.mpi-forum.org >>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-sessions >> _______________________________________________ >> mpiwg-sessions mailing list >> mpiwg-sessions(a)lists.mpi-forum.org >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-sessions > > >-- >Jeff Squyres >jsquyres(a)cisco.com >For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/ > >_______________________________________________ >mpiwg-sessions mailing list >mpiwg-sessions(a)lists.mpi-forum.org >http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-sessions

1 0

Re: [mpiwg-sessions] [EXTERNAL] Notes from today / slides for next time
by Jeff Squyres (jsquyres) 17 May '16

17 May '16

Put simply: your use case is grow then shrink, right? The case I show in slides 83-91 is just shrink. But I think it generalizes to any case of shrink (i.e., regardless of whether you previously grew). I think that the main issues are: - previously-obtained sets may become stale - the release manager may want to (implicitly or explicitly) release nodes that are no longer being used (this may or may not be in MPI's prevue) > On May 16, 2016, at 6:31 PM, Teranishi, Keita <knteran(a)sandia.gov> wrote: > > Jeff, > > Thanks for the update. Our potential use case in Sandia is less aggressive than the model in pages 83-91. Typically, the main application occupies the 70-90% of the total nodes for the entire execution time. The application data analytics routines are spawned and joined (shrunk) on the rest of the nodes. Data movement happens between the ranks running base application and the spawned ranks, but I believe that doing within MPI is faster than using external fast-IO services (such as DataSpaces). > > Maybe, the best use case I can map to your example is reduced order modeling, which can be seen as interactions between “big” and “small" simulation instances. Of course, it can be done with master-worker model, but spawn+shrink (join+leave) model would allow more flexible use of big machines. > > Thanks, > Keita > > > > > > On 5/16/16, 3:00 PM, "mpiwg-sessions on behalf of Jeff Squyres (jsquyres)" <mpiwg-sessions-bounces(a)lists.mpi-forum.org on behalf of jsquyres(a)cisco.com> wrote: > >> Thanks for everyone's time today. Here's notes and slides from today: >> >> https://github.com/mpiwg-sessions/sessions-issues/wiki/2016-05-16-webex >> >> I consolidated the open issues, and added a bunch of information about the "grow" and "shrink" use cases that we want to support, and made slides for 2 weeks from now -- please have a look: >> >> https://github.com/mpiwg-sessions/sessions-issues/wiki/2016-05-30-webex >> >> Hilights: >> >> • slides 14-15: required/provided for thread level >> • slide 20: resolved -- only static sets (no dynamics) >> • slide 29: only get set names for which you are a member >> • slide 30: Updated the "open questions" >> • slides 78-70: minor updates to MPI_EXEC >> • slides 83-91: A bunch of new slides about "shrink the app" use case >> >> -- >> Jeff Squyres >> jsquyres(a)cisco.com >> For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/ >> >> _______________________________________________ >> mpiwg-sessions mailing list >> mpiwg-sessions(a)lists.mpi-forum.org >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-sessions > _______________________________________________ > mpiwg-sessions mailing list > mpiwg-sessions(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-sessions -- Jeff Squyres jsquyres(a)cisco.com For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/

1 0

Re: [mpiwg-sessions] [EXTERNAL] Notes from today / slides for next time
by Teranishi, Keita 16 May '16

16 May '16

Jeff, Thanks for the update. Our potential use case in Sandia is less aggressive than the model in pages 83-91. Typically, the main application occupies the 70-90% of the total nodes for the entire execution time. The application data analytics routines are spawned and joined (shrunk) on the rest of the nodes. Data movement happens between the ranks running base application and the spawned ranks, but I believe that doing within MPI is faster than using external fast-IO services (such as DataSpaces). Maybe, the best use case I can map to your example is reduced order modeling, which can be seen as interactions between “big” and “small" simulation instances. Of course, it can be done with master-worker model, but spawn+shrink (join+leave) model would allow more flexible use of big machines. Thanks, Keita On 5/16/16, 3:00 PM, "mpiwg-sessions on behalf of Jeff Squyres (jsquyres)" <mpiwg-sessions-bounces(a)lists.mpi-forum.org on behalf of jsquyres(a)cisco.com> wrote: >Thanks for everyone's time today. Here's notes and slides from today: > > https://github.com/mpiwg-sessions/sessions-issues/wiki/2016-05-16-webex > >I consolidated the open issues, and added a bunch of information about the "grow" and "shrink" use cases that we want to support, and made slides for 2 weeks from now -- please have a look: > > https://github.com/mpiwg-sessions/sessions-issues/wiki/2016-05-30-webex > >Hilights: > >• slides 14-15: required/provided for thread level >• slide 20: resolved -- only static sets (no dynamics) >• slide 29: only get set names for which you are a member >• slide 30: Updated the "open questions" >• slides 78-70: minor updates to MPI_EXEC >• slides 83-91: A bunch of new slides about "shrink the app" use case > >-- >Jeff Squyres >jsquyres(a)cisco.com >For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/ > >_______________________________________________ >mpiwg-sessions mailing list >mpiwg-sessions(a)lists.mpi-forum.org >http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-sessions

1 0

Notes from today / slides for next time
by Jeff Squyres (jsquyres) 16 May '16

16 May '16

Thanks for everyone's time today. Here's notes and slides from today: https://github.com/mpiwg-sessions/sessions-issues/wiki/2016-05-16-webex I consolidated the open issues, and added a bunch of information about the "grow" and "shrink" use cases that we want to support, and made slides for 2 weeks from now -- please have a look: https://github.com/mpiwg-sessions/sessions-issues/wiki/2016-05-30-webex Hilights: • slides 14-15: required/provided for thread level • slide 20: resolved -- only static sets (no dynamics) • slide 29: only get set names for which you are a member • slide 30: Updated the "open questions" • slides 78-70: minor updates to MPI_EXEC • slides 83-91: A bunch of new slides about "shrink the app" use case -- Jeff Squyres jsquyres(a)cisco.com For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/

1 0