June 2023 - mpiwg-sessions - mailman3.cels.anl.gov

Re: [mpiwg-sessions] Please meet today , i.e. undo the Cancel 6/19/23 meeting
by Rolf Rabenseifner 19 Jun '23

19 Jun '23

Dear Howard, Martin, and Session-Folks, PR 823 is now updated with the new pdf: https://github.com/mpi-forum/mpi-standard/files/11787848/mpi41-report_Issue… It now contains the additional options D, E, and F. These options should be discussed today. >From previous Email: the meeting today would be important because we should try to resolve the problems about datatype (and operation) handles - that are in principle not related to a session - and therefore can be already created before a specific session is initialized, where it is used, - and can be used in a session B after the session A during which the handle was created is already finalized, - and betweeen such sessions A and B can be a gab of no sessions or World Model is initialized. I just checked whether our sessions zoom meeting URL can be used at any time: Yes, it worked. Therefore, we can meet as we wanted to do, i.e., today 9 am CDT = 4 PM CEST. Therefore, @Howard, can we meet today? (I hope that all in US may find this email before the meeting starts today) @Martin, you may update internally the time slot of the existing weekly Zoom URL to start at 9 am CDT = 4 PM CEST I'll update the pdf of issue #710 / PR 823 today before the meeting starts. The PR 823 itself is already up-to-date. The pdf will include additional options for the discussion today. Details: - Sessions meeting protocol from last Monday: https://github.com/mpiwg-sessions/sessions-issues/wiki/2023-06-12-webex - #710 Errata: 'Pending communication' not defined in MPI_Comm_disconnect Rolf Issue https://github.com/mpi-forum/mpi-issues/issues/710 PR https://github.com/mpi-forum/mpi-standard/pull/823 PDF https://github.com/mpi-forum/mpi-standard/files/11787848/mpi41-report_Issue… Best regards Rolf ----- Original Message ----- > From: "Pritchard Jr., Howard via mpiwg-sessions" <mpiwg-sessions(a)lists.mpi-forum.org> > To: "MPI Sessions working group" <mpiwg-sessions(a)lists.mpi-forum.org> > Cc: "Pritchard" <howardp(a)lanl.gov> > Sent: Monday, June 19, 2023 2:30:58 AM > Subject: [mpiwg-sessions] Cancel 6/19/23 meeting > Hi Folks > > > > I never heard back from Martin Schulz about rescheduling our zoom meeting to the > new time so canceling for 6/19/23. > > I’ll try to reconnect with Martin about rescheduling. > > > > Howard > > > > > > — > > > > > > > Howard Pritchard > > Research Scientist > > HPC-ENV > > > > Los Alamos National Laboratory > > howardp(a)lanl.gov > > > > [ https://www.instagram.com/losalamosnatlab/ ] [ > https://twitter.com/LosAlamosNatLab ] [ > https://www.linkedin.com/company/los-alamos-national-laboratory/ ] [ > https://www.facebook.com/LosAlamosNationalLab/ ] > > > > > > > _______________________________________________ > mpiwg-sessions mailing list > mpiwg-sessions(a)lists.mpi-forum.org > https://lists.mpi-forum.org/mailman/listinfo/mpiwg-sessions -- Dr. Rolf Rabenseifner . . . . . . . . . .. . . rabenseifner(a)hlrs.de . High Performance Computing Center (HLRS) . . . ++49(0)711/685-65530 . University of Stuttgart . . . . . . www.hlrs.de/people/rabenseifner . Nobelstr. 19, 70569 Stuttgart, Germany

1 0

Please meet today , i.e. undo the Cancel 6/19/23 meeting
by Rolf Rabenseifner 19 Jun '23

19 Jun '23

Dear Howard, Martin, and Session-Folks, the meeting today would be important because we should try to resolve the problems about datatype (and operation) handles - that are in principle not related to a session - and therefore can be already created before a specific session is initialized, where it is used, - and can be used in a session B after the session A during which the handle was created is already finalized, - and betweeen such sessions A and B can be a gab of no sessions or World Model is initialized. I just checked whether our sessions zoom meeting URL can be used at any time: Yes, it worked. Therefore, we can meet as we wanted to do, i.e., today 9 am CDT = 4 PM CEST. Therefore, @Howard, can we meet today? (I hope that all in US may find this email before the meeting starts today) @Martin, you may update internally the time slot of the existing weekly Zoom URL to start at 9 am CDT = 4 PM CEST I'll update the pdf of issue #710 / PR 823 today before the meeting starts. The PR 823 itself is already up-to-date. The pdf will include additional options for the discussion today. Details: - Sessions meeting protocol from last Monday: https://github.com/mpiwg-sessions/sessions-issues/wiki/2023-06-12-webex - #710 Errata: 'Pending communication' not defined in MPI_Comm_disconnect Rolf Issue https://github.com/mpi-forum/mpi-issues/issues/710 PR https://github.com/mpi-forum/mpi-standard/pull/823 Best regards Rolf ----- Original Message ----- > From: "Pritchard Jr., Howard via mpiwg-sessions" <mpiwg-sessions(a)lists.mpi-forum.org> > To: "MPI Sessions working group" <mpiwg-sessions(a)lists.mpi-forum.org> > Cc: "Pritchard" <howardp(a)lanl.gov> > Sent: Monday, June 19, 2023 2:30:58 AM > Subject: [mpiwg-sessions] Cancel 6/19/23 meeting > Hi Folks > > > > I never heard back from Martin Schulz about rescheduling our zoom meeting to the > new time so canceling for 6/19/23. > > I’ll try to reconnect with Martin about rescheduling. > > > > Howard > > > > > > — > > > > > > > Howard Pritchard > > Research Scientist > > HPC-ENV > > > > Los Alamos National Laboratory > > howardp(a)lanl.gov > > > > [ https://www.instagram.com/losalamosnatlab/ ] [ > https://twitter.com/LosAlamosNatLab ] [ > https://www.linkedin.com/company/los-alamos-national-laboratory/ ] [ > https://www.facebook.com/LosAlamosNationalLab/ ] > > > > > > > _______________________________________________ > mpiwg-sessions mailing list > mpiwg-sessions(a)lists.mpi-forum.org > https://lists.mpi-forum.org/mailman/listinfo/mpiwg-sessions -- Dr. Rolf Rabenseifner . . . . . . . . . .. . . rabenseifner(a)hlrs.de . High Performance Computing Center (HLRS) . . . ++49(0)711/685-65530 . University of Stuttgart . . . . . . www.hlrs.de/people/rabenseifner . Nobelstr. 19, 70569 Stuttgart, Germany

1 0

Cancel 6/19/23 meeting
by Pritchard Jr., Howard 18 Jun '23

18 Jun '23

Hi Folks I never heard back from Martin Schulz about rescheduling our zoom meeting to the new time so canceling for 6/19/23. I’ll try to reconnect with Martin about rescheduling. Howard — [signature_61897647] Howard Pritchard Research Scientist HPC-ENV Los Alamos National Laboratory howardp(a)lanl.gov [signature_1948556898]<https://www.instagram.com/losalamosnatlab/>[signature_3161760955]<https://twitter.com/LosAlamosNatLab>[signature_3021907833]<https://www.linkedin.com/company/los-alamos-national-laboratory/>[signature_1307594743]<https://www.facebook.com/LosAlamosNationalLab/>

1 0

Re: [mpiwg-sessions] [EXTERNAL] Re: Sessions WG - meet today (at usual time)
by Pritchard Jr., Howard 12 Jun '23

12 Jun '23

Hi Rolf, It is at 7PM central European time today. Howard On 6/12/23, 10:57 AM, "mpiwg-sessions on behalf of Rolf Rabenseifner via mpiwg-sessions" <mpiwg-sessions-bounces(a)lists.mpi-forum.org <mailto:[email protected]> on behalf of mpiwg-sessions(a)lists.mpi-forum.org <mailto:[email protected]>> wrote: If the meeting is today, 7pm Amsterdam/Rom/Berlin time, then I'll be there. The pdf link is https://urldefense.com/v3/__https://github.com/mpi-forum/mpi-standard/files… <https://urldefense.com/v3/__https://github.com/mpi-forum/mpi-standard/files…> ----- Original Message ----- > From: "Rolf Rabenseifner via mpiwg-sessions" <mpiwg-sessions(a)lists.mpi-forum.org <mailto:[email protected]>> > To: "MPI Sessions working group" <mpiwg-sessions(a)lists.mpi-forum.org <mailto:[email protected]>> > Cc: "Rolf Rabenseifner" <rabenseifner(a)hlrs.de <mailto:[email protected]>> > Sent: Monday, June 12, 2023 6:49:12 PM > Subject: Re: [mpiwg-sessions] Sessions WG - meet today (at usual time) > I just saw, that you have also a meeting today. > I updated the PR 823 with a new commented pdf. > > ----- Original Message ----- >> From: "Pritchard Jr., Howard via mpiwg-sessions" >> <mpiwg-sessions(a)lists.mpi-forum.org <mailto:[email protected]>> >> To: "MPI Sessions working group" <mpiwg-sessions(a)lists.mpi-forum.org <mailto:[email protected]>> >> Cc: "Pritchard" <howardp(a)lanl.gov <mailto:[email protected]>> >> Sent: Monday, June 12, 2023 5:27:01 PM >> Subject: [mpiwg-sessions] Sessions WG - meet today (at usual time) > >> Hi All, >> >> >> >> Let’s try to meet today. >> >> >> >> Agenda items are >> >> * The pending discussion – what actions should we take >> * Results of doodle poll >> >> >> >> >> Howard >> >> >> >> >> >> — >> >> >> >> >> >> >> Howard Pritchard >> >> Research Scientist >> >> HPC-ENV >> >> >> >> Los Alamos National Laboratory >> >> howardp(a)lanl.gov <mailto:[email protected]> >> >> >> >> [ https://urldefense.com/v3/__https://www.instagram.com/losalamosnatlab/__;!!… <https://urldefense.com/v3/__https://www.instagram.com/losalamosnatlab/__;!!…> ] [ >> https://urldefense.com/v3/__https://twitter.com/LosAlamosNatLab__;!!Bt8fGhp… <https://urldefense.com/v3/__https://twitter.com/LosAlamosNatLab__;!!Bt8fGhp…> ] [ >> https://urldefense.com/v3/__https://www.linkedin.com/company/los-alamos-nat… <https://urldefense.com/v3/__https://www.linkedin.com/company/los-alamos-nat…> ] [ >> https://urldefense.com/v3/__https://www.facebook.com/LosAlamosNationalLab/_… <https://urldefense.com/v3/__https://www.facebook.com/LosAlamosNationalLab/_…> ] >> >> >> >> >> >> >> _______________________________________________ >> mpiwg-sessions mailing list >> mpiwg-sessions(a)lists.mpi-forum.org <mailto:[email protected]> >> https://urldefense.com/v3/__https://lists.mpi-forum.org/mailman/listinfo/mp… <https://urldefense.com/v3/__https://lists.mpi-forum.org/mailman/listinfo/mp…> > > -- > Dr. Rolf Rabenseifner . . . . . . . . . .. . . rabenseifner(a)hlrs.de <mailto:[email protected]> . > High Performance Computing Center (HLRS) . . . ++49(0)711/685-65530 . > University of Stuttgart . . . . . . https://urldefense.com/v3/__http://www.hlrs.de/people/rabenseifner__;!!Bt8f… <https://urldefense.com/v3/__http://www.hlrs.de/people/rabenseifner__;!!Bt8f…> . > Nobelstr. 19, 70569 Stuttgart, Germany > > _______________________________________________ > mpiwg-sessions mailing list > mpiwg-sessions(a)lists.mpi-forum.org <mailto:[email protected]> > https://urldefense.com/v3/__https://lists.mpi-forum.org/mailman/listinfo/mp… <https://urldefense.com/v3/__https://lists.mpi-forum.org/mailman/listinfo/mp…> -- Dr. Rolf Rabenseifner . . . . . . . . . .. . . rabenseifner(a)hlrs.de <mailto:[email protected]> . High Performance Computing Center (HLRS) . . . ++49(0)711/685-65530 . University of Stuttgart . . . . . . https://urldefense.com/v3/__http://www.hlrs.de/people/rabenseifner__;!!Bt8f… <https://urldefense.com/v3/__http://www.hlrs.de/people/rabenseifner__;!!Bt8f…> . Nobelstr. 19, 70569 Stuttgart, Germany _______________________________________________ mpiwg-sessions mailing list mpiwg-sessions(a)lists.mpi-forum.org <mailto:[email protected]> https://urldefense.com/v3/__https://lists.mpi-forum.org/mailman/listinfo/mp… <https://urldefense.com/v3/__https://lists.mpi-forum.org/mailman/listinfo/mp…>

1 0

Re: [mpiwg-sessions] Sessions WG - meet today (at usual time)
by Rolf Rabenseifner 12 Jun '23

12 Jun '23

If the meeting is today, 7pm Amsterdam/Rom/Berlin time, then I'll be there. The pdf link is https://github.com/mpi-forum/mpi-standard/files/11724540/mpi41-report_Issue… ----- Original Message ----- > From: "Rolf Rabenseifner via mpiwg-sessions" <mpiwg-sessions(a)lists.mpi-forum.org> > To: "MPI Sessions working group" <mpiwg-sessions(a)lists.mpi-forum.org> > Cc: "Rolf Rabenseifner" <rabenseifner(a)hlrs.de> > Sent: Monday, June 12, 2023 6:49:12 PM > Subject: Re: [mpiwg-sessions] Sessions WG - meet today (at usual time) > I just saw, that you have also a meeting today. > I updated the PR 823 with a new commented pdf. > > ----- Original Message ----- >> From: "Pritchard Jr., Howard via mpiwg-sessions" >> <mpiwg-sessions(a)lists.mpi-forum.org> >> To: "MPI Sessions working group" <mpiwg-sessions(a)lists.mpi-forum.org> >> Cc: "Pritchard" <howardp(a)lanl.gov> >> Sent: Monday, June 12, 2023 5:27:01 PM >> Subject: [mpiwg-sessions] Sessions WG - meet today (at usual time) > >> Hi All, >> >> >> >> Let’s try to meet today. >> >> >> >> Agenda items are >> >> * The pending discussion – what actions should we take >> * Results of doodle poll >> >> >> >> >> Howard >> >> >> >> >> >> — >> >> >> >> >> >> >> Howard Pritchard >> >> Research Scientist >> >> HPC-ENV >> >> >> >> Los Alamos National Laboratory >> >> howardp(a)lanl.gov >> >> >> >> [ https://www.instagram.com/losalamosnatlab/ ] [ >> https://twitter.com/LosAlamosNatLab ] [ >> https://www.linkedin.com/company/los-alamos-national-laboratory/ ] [ >> https://www.facebook.com/LosAlamosNationalLab/ ] >> >> >> >> >> >> >> _______________________________________________ >> mpiwg-sessions mailing list >> mpiwg-sessions(a)lists.mpi-forum.org >> https://lists.mpi-forum.org/mailman/listinfo/mpiwg-sessions > > -- > Dr. Rolf Rabenseifner . . . . . . . . . .. . . rabenseifner(a)hlrs.de . > High Performance Computing Center (HLRS) . . . ++49(0)711/685-65530 . > University of Stuttgart . . . . . . www.hlrs.de/people/rabenseifner . > Nobelstr. 19, 70569 Stuttgart, Germany > > _______________________________________________ > mpiwg-sessions mailing list > mpiwg-sessions(a)lists.mpi-forum.org > https://lists.mpi-forum.org/mailman/listinfo/mpiwg-sessions -- Dr. Rolf Rabenseifner . . . . . . . . . .. . . rabenseifner(a)hlrs.de . High Performance Computing Center (HLRS) . . . ++49(0)711/685-65530 . University of Stuttgart . . . . . . www.hlrs.de/people/rabenseifner . Nobelstr. 19, 70569 Stuttgart, Germany

1 0

Re: [mpiwg-sessions] Sessions WG - meet today (at usual time)
by Rolf Rabenseifner 12 Jun '23

12 Jun '23

I just saw, that you have also a meeting today. I updated the PR 823 with a new commented pdf. ----- Original Message ----- > From: "Pritchard Jr., Howard via mpiwg-sessions" <mpiwg-sessions(a)lists.mpi-forum.org> > To: "MPI Sessions working group" <mpiwg-sessions(a)lists.mpi-forum.org> > Cc: "Pritchard" <howardp(a)lanl.gov> > Sent: Monday, June 12, 2023 5:27:01 PM > Subject: [mpiwg-sessions] Sessions WG - meet today (at usual time) > Hi All, > > > > Let’s try to meet today. > > > > Agenda items are > > * The pending discussion – what actions should we take > * Results of doodle poll > > > > > Howard > > > > > > — > > > > > > > Howard Pritchard > > Research Scientist > > HPC-ENV > > > > Los Alamos National Laboratory > > howardp(a)lanl.gov > > > > [ https://www.instagram.com/losalamosnatlab/ ] [ > https://twitter.com/LosAlamosNatLab ] [ > https://www.linkedin.com/company/los-alamos-national-laboratory/ ] [ > https://www.facebook.com/LosAlamosNationalLab/ ] > > > > > > > _______________________________________________ > mpiwg-sessions mailing list > mpiwg-sessions(a)lists.mpi-forum.org > https://lists.mpi-forum.org/mailman/listinfo/mpiwg-sessions -- Dr. Rolf Rabenseifner . . . . . . . . . .. . . rabenseifner(a)hlrs.de . High Performance Computing Center (HLRS) . . . ++49(0)711/685-65530 . University of Stuttgart . . . . . . www.hlrs.de/people/rabenseifner . Nobelstr. 19, 70569 Stuttgart, Germany

1 0

Sessions WG - meet today (at usual time)
by Pritchard Jr., Howard 12 Jun '23

12 Jun '23

Hi All, Let’s try to meet today. Agenda items are * The pending discussion – what actions should we take * Results of doodle poll Howard — [signature_61897647] Howard Pritchard Research Scientist HPC-ENV Los Alamos National Laboratory howardp(a)lanl.gov [signature_628370459]<https://www.instagram.com/losalamosnatlab/>[signature_2122258513]<https://twitter.com/LosAlamosNatLab>[signature_1822735606]<https://www.linkedin.com/company/los-alamos-national-laboratory/>[signature_908372548]<https://www.facebook.com/LosAlamosNationalLab/>

1 0

Re: [mpiwg-sessions] Virtual MPI forum meeting June 7, 2023 / Vote announcement for the July 2023 meeting of the MPI Forum
by Holmes, Daniel John 08 Jun '23

08 Jun '23

Aurelien wrote: "there is some catchall text that makes not calling all procedures on WPM objects to complete them before calling MPI_FINALIZE erroneous code" See MPI-4.0 section 11.2.2 p495 lines 13-21 There is no mention of inactive persistent requests, the example given in that text is a nonblocking operation, which can only be inactive if MPI_REQUEST_GET_STATUS was used to complete it, but that procedure is not mentioned in that section of text. Thus, whether it is erroneous to have inactive persistent requests at the time of a call to MPI_FINALIZE has never been addressed by MPI. See MPI-4.0 section 11.10.4 p 547 lines 29-31 "complete and matched" is roughly equivalent to the text for MPI_FINALIZE but to avoid doubt the text for MPI_COMM_DISCONNECT also says "same as". There is again no mention of inactive persistent requests and treatment of inactive nonblocking requests relies on the link with MPI_FINALIZE. The proposal that we should "errata it out of existence" (nice phrase! I have a mental image of whack-a-mole for some reason.) has the counter-argument "but we've never before mandated that requests must be freed before calling these procedures: users will be upset" which hangs on a shoogly peg (IMHO). Best wishes, Dan. -----Original Message----- From: Aurelien Bouteiller <bouteill(a)icl.utk.edu> Sent: Thursday, June 8, 2023 4:53 PM To: MPI Sessions working group <mpiwg-sessions(a)lists.mpi-forum.org> Cc: Pritchard Jr., Howard <howardp(a)lanl.gov>; Holmes, Daniel John <daniel.john.holmes(a)intel.com> Subject: Re: [mpiwg-sessions] Virtual MPI forum meeting June 7, 2023 / Vote announcement for the July 2023 meeting of the MPI Forum > On Jun 8, 2023, at 10:15, Holmes, Daniel John via mpiwg-sessions <mpiwg-sessions(a)lists.mpi-forum.org> wrote: > > Hi Howard (& all), > > In the MPI Virtual meeting yesterday, Rolf presented a PR that tries to define the word "pending" in MPI. The discussion raised a whole bunch of issues and concerns. I took an action to raise some of them in the Sessions WG because they are introduced by (or made worse by) the sessions model. > > Quick summary of options in Rolf's proposal: > 1) Remove the word "pending" from the MPI Standard -- replace it with "active" where that makes sense, or something else when appropriate. > 2) Define "pending" to be a strict alias of "active" -- it can only be applied to MPI operations and it means "starting stage has been done but completion stage has not yet been done" > 2) Define "pending" to be anything task/activity on MPI 's to-do list -- encompassing all active operations but also all decoupled activities (i.e. progress). > > We discovered in the discussion of these options, in the context of MPI_FINALIZE and MPI_DISCONNECT (which must delay their return until "pending" things have been done) that an important restriction has been unintentionally removed by the introduction of sessions and we probably need to re-introduce parts of that restriction. > > When there was only the World Model, it was erroneous to call things like MPI_REQUEST_FREE after MPI_FINALIZE. Questions like "can I free a request after finalize?" -- derived from questions like "what happens to inactive persistent requests when I finalize MPI?" -- made no sense because you can look at the code with MPI_REQUEST_FREE after MPI_FINALIZE and immediate declare it is an erroneous program. > > Once sessions came into existence though, we can now call MPI_REQUEST_FREE after MPI_FINALIZE, as long as there is a session: > > ``` > MPI_SESSION_INIT(&sh) > MPI_INIT() > MPI_RECV_INIT(..., MPI_COMM_WORLD, &req) > MPI_FINALIZE() > MPI_REQUEST_FREE(&req) > MPI_SESSION_FINALIZE() > ``` > > This is clearly A Bad Idea (tm), but which MPI rule has been broken in the above pseudo-code? Without the two sessions lines of code, this is clearly erroneous, but with them it is not. Do we want this pattern to be erroneous or well-defined? > I think that example is still erroneous, because the request is posted on the WPM; there is some catchall text that makes not calling all procedures on WPM objects to complete them before calling MPI_FINALIZE erroneous code. If the current text doesn’t clearly says so (the completion criterion that everything should be inactive implies it for WAIT, etc, not as clearly for FREE on inactive persistent reqs), we should amend text so that it precludes using procedures on WPM handles after MPI_FINALIZE. However the same example with a request posted on the SPM, and calling free after SESSION_FINALIZE would be valid. As you say later in your essay, we have to choose if we want it to remain so, or if we want to errata this behavior out of existence. The SPM implementations do not support this, so there is no real backward incompatibility to be worried about when declaring this retro-actively that it was always erroneous. > There is no semantic reason why this cannot be deemed legal -- MPI_REQUEST_FREE is a local procedure, so the fact MPI_COMM_WORLD is no longer functional should be irrelevant -- it could just clean up locally allocated resources (the connections to other processes were cleaned up during MPI_FINALIZE). > > One technical reason to define this pattern to be erroneous is that the request probably has a ref-counted reference to the communicator and MPI_FINALIZE will hang waiting for that ref-count to reach zero, which it cannot do because the request will not be freed until after it returns. This is an implementation detail, but it is how at least one major MPI library is currently implemented. > > An implementation could instead choose to store a weak_ptr (https://en.cppreference.com/w/cpp/memory/weak_ptr) to the communicator in the request when the operation is inactive, attempt to get a shared_ptr from the weak_ptr (https://en.cppreference.com/w/cpp/memory/weak_ptr/lock) during the starting stage, and give up the shared_ptr during the completion stage. In such an implementation, an inactive request does not own the communicator associated with the operation, so finalize can destroy the communicator, and a freeing an inactive request never needs to dereference to get the communicator, so it doesn't care whether the communicator still exists or not. > > Much more important than technical feasibility is the question "why should we enable this pattern?" mumble mumble C++ destructors mumble garbage collected languages mumble. > > Taking a step back, there is a whole category of programs that used to be erroneous by definition (cannot make MPI calls after MPI_FINALIZE) that are no longer obviously erroneous. What do we do about them all? > > There are related issues/concerns here: if a rule is added to MPI that forces MPI_FINALIZE to free all requests associated with the World Model, should sessions follow suit and have MPI_SESSION_FINALIZE free requests from that session? There is text exhorting the user to enable and complete all MPI operations (in the World Mode) before calling MPI_FINALIZE, but there is no such exhortation for the sessions model; should there be one? Does finalizing a session destroy all the communicators from that session? Can a communicator handle be used after the session is finalized (e.g. in MPI_COMM_FREE)? > Not pushing in any particular direction here, just exposing some pros-cons: For the sake of uniformity, it would be desired that all of these FINALIZE/DISCONNECT procedures have a similar behavior, since they kind-of do the same thing in different contexts. This also has advantages in terms of implementation simplicity. However, one may consider the following example and want this to be valid code: ``` MPI_SESSION_INIT(&sh) MPI_COMM_CREATE_FROM_PSET(comm,...) MPI_SESSION_FINALIZE(&sh) MPI_SEND(comm); MPI_RECV(comm) MPI_COMM_DISCONNECT(comm) ``` Since the body of the application does not need to use session objects anymore after CREATE_FROM_PSET, users may be interested in closing the session immediately (which can obviously be done at that point, since no communications are pending when we close it, and they use disconnect to prevent straggling buffered message to linger in limbo when freeing the comm). We even had a discussion about having SESSION_FINALIZE_LOCAL at some point, presumably a good tool to support this. A disadvantage is obviously that we cannot assert that the library is valgrind-clean at SESSION_FINALIZE point, since a bunch of MPI objects have to remain allocated/refcounted for this to work. ~~~ One oblique consideration in that decision about uniformity is that we have the same problem for MPI_COMM_DISCONNECT, and that has existed for a sufficiently long time that it may not be as easy to just errata it out of existence in that case (I don’t think people are generally using DISCONNECT a lot to start with, and implementation review may reveal that problematic usage patterns never worked in practice, which would then open the way for the errata approach in the COMM_DISCONNECT case). Aurelien > Best wishes, > Dan. > > -----Original Message----- > From: Rolf Rabenseifner <rabenseifner(a)hlrs.de> > Sent: Wednesday, June 7, 2023 6:10 PM > To: Holmes, Daniel John <daniel.john.holmes(a)intel.com> > Subject: Fwd: Virtual MPI forum meeting June 7, 2023 / Vote > announcement for the July 2023 meeting of the MPI Forum > > The pdf is > https://github.com/mpi-forum/mpi-standard/files/11676448/mpi41-report_ > Issue710_PR823_update_for.2023-6-7-meeting.pdf > especially the questions on page 482 and 524. > > ----- Forwarded Message ----- > From: "Rolf Rabenseifner" <rabenseifner(a)hlrs.de> > To: "Main MPI Forum mailing list" <mpi-forum(a)lists.mpi-forum.org> > Sent: Wednesday, June 7, 2023 1:51:04 PM > Subject: Re: Virtual MPI forum meeting June 7, 2023 / Vote > announcement for the July 2023 meeting of the MPI Forum > > Dear MPI forum members, > > last update for #710/PR823: second option (alternative B) and open > questions for MPI_Session_finalize, see > https://github.com/mpi-forum/mpi-standard/files/11676448/mpi41-report_ > Issue710_PR823_update_for.2023-6-7-meeting.pdf > for the virtual meeting today. > > Kind regards > Rolf > > ________ email from yesterday: _________ > > for the virtual meeting this week (June 7, 2023) the following readings and discussions are scheduled. > Some technical decisions are needed and it may be best to do these decisions as institutional staw votes in an additional virtual meeting (e.g. next week), rather to do them ad hoc this week. > > The topics are #705/PR822, #710/PR823 with two options, #676/PR824 or PR825. > > Details: > __________________________ > #705 Errata: Fortran has only compile-time constants Rolf,Joseph > Issue https://github.com/mpi-forum/mpi-issues/issues/705 > (together with https://github.com/mpi-forum/mpi-issues/issues/657 Rolf, Jeff H.) > PR https://github.com/mpi-forum/mpi-standard/pull/822 > PDF > > https://github.com/mpi-forum/mpi-standard/files/11450065/mpi41-report_ > Issue705%2B657_PR822.pdf > > This proposal seems to be already stable. > > __________________________ > #710 Errata: 'Pending communication' not defined in MPI_Comm_disconnect Rolf > Issue https://github.com/mpi-forum/mpi-issues/issues/710 > PR https://github.com/mpi-forum/mpi-standard/pull/823 > updated: > PDF https://github.com/mpi-forum/mpi-standard/files/11676448/mpi41-report_Issue… > > > There is severe critism about the current proposal in PR 823: > > "In this PR here, as far as I can see, the implementations are mainly affected if they > incorrectly require that there must not be any inactive request handle using the given comm. > Whether an MPI lib really internally frees the inactive handles or not, > is mainly a question of having dangling handles or not." > "Is there broad agreement that this is incorrect implementation behavior? I'm skeptical." > > And therefore, there are two different solutions: > - An advice to users, telling the consequences if they do not free inactive > handles before calling MPI_Comm_disconnect or MPI_Session_finalize. > or > - Automatically freeing them. > > And furthermore: > > "Whatever we decide, I imagine this needs to extend to MPI_SESSION_FINALIZE as well." > > I'll try to update the proposal before the meeting to have text for both possible solutions. > > __________________________ > #676 Errata: 'Pending operation' not defined, pending proper definition Rolf,Joseph > Issue https://github.com/mpi-forum/mpi-issues/issues/676 > PR https://github.com/mpi-forum/mpi-standard/pull/824 > PDF https://github.com/mpi-forum/mpi-standard/files/11668075/mpi41-report_Issue… > > Substituting "pending operation" by "active operation" > > or PR https://github.com/mpi-forum/mpi-standard/pull/825 > PDF > https://github.com/mpi-forum/mpi-standard/files/11669354/mpi41-report_ > Issue676_PR825.pdf > > Defining "pending operation" as "active operation" (see page 12) > > Best regards > Rolf > > > ----- Original Message ----- >> From: "Rolf Rabenseifner" <rabenseifner(a)hlrs.de> >> To: "Main MPI Forum mailing list" <mpi-forum(a)lists.mpi-forum.org> >> Cc: "Christoph Niethammer" <niethammer(a)hlrs.de>, "Puri Bangalore" <pvbangalore(a)ua.edu>, "Joseph Schuchart" >> <schuchart(a)icl.utk.edu> >> Sent: Wednesday, May 24, 2023 1:36:42 PM >> Subject: Re: Update: Vote announcement for the July 2023 meeting of >> the MPI Forum > >> Martin, Wes, and all, >> >> I expect that we should reserve also June 14, 2023 for a possible >> continuation of open questions resulting from the discussions of >> >>> #705/PR822, #710/PR823, and #676/PR824 are now finalized at least >>> for the virtual forum meeting, June 7, 2023 >> >> and other pending items for MPI-4.1 from the June 7 virtual forum meeting. >> >> Best regards >> Rolf >> >> >> >> >> ----- Original Message ----- >>> From: "Rolf Rabenseifner" <rabenseifner(a)hlrs.de> >>> To: "Main MPI Forum mailing list" <mpi-forum(a)lists.mpi-forum.org> >>> Cc: "Christoph Niethammer" <niethammer(a)hlrs.de>, "Puri Bangalore" >>> <pvbangalore(a)ua.edu>, "Joseph Schuchart" >>> <schuchart(a)icl.utk.edu> >>> Sent: Thursday, May 11, 2023 8:05:07 PM >>> Subject: Update: Vote announcement for the July 2023 meeting of the >>> MPI Forum >> >>> Dear forum members, >>> >>> #705/PR822, #710/PR823, and #676/PR824 are now finalized at least >>> for the virtual forum meeting, June 7, 2023, details below: >>> >>> #705 Errata: Fortran has only compile-time constants Rolf,Joseph >>> Issue https://github.com/mpi-forum/mpi-issues/issues/705 >>> (together with https://github.com/mpi-forum/mpi-issues/issues/657 Rolf, Jeff >>> H.) >>> PR https://github.com/mpi-forum/mpi-standard/pull/822 >>> PDF >>> >>> https://github.com/mpi-forum/mpi-standard/files/11450065/mpi41-repor >>> t >>> _Issue705%2B657_PR822.pdf >>> >>> #710 Errata: 'Pending communication' not defined in MPI_Comm_disconnect Rolf >>> Issue https://github.com/mpi-forum/mpi-issues/issues/710 >>> PR https://github.com/mpi-forum/mpi-standard/pull/823 >>> PDF >>> >>> https://github.com/mpi-forum/mpi-standard/files/11454896/mpi41-repor >>> t >>> _Issue710_PR823.pdf >>> >>> #676 Errata: 'Pending operation' not defined, pending proper definition >>> Rolf,Joseph >>> Issue https://github.com/mpi-forum/mpi-issues/issues/676 >>> PR https://github.com/mpi-forum/mpi-standard/pull/824 >>> PDF >>> >>> https://github.com/mpi-forum/mpi-standard/files/11443976/mpi41-repor >>> t >>> _Issue676_PR824.pdf >>> >>> The agenda of the virtual meeting should be reading and discussions if needed. >>> I did the proposals based on the discussion and the major goals: >>> - backward compatible, >>> - no performance drawbacks on critical paths, >>> - full consistency of the solution with all related parts of the MPI standard. >>> >>> If I have overseen something, then my apologies. >>> >>> If somebody wants to provide a completely different solution to one >>> of these issues, please feel free to do it. >>> - Thanks to Joseph who did PR 822, which was such a counter proposal for #705. >>> - This method is often better than doing many change requests for the given PR. >>> and it helps to completely check whether it is really consistent >>> over all related parts of the MPI standard. >>> - And thanks for all the comments so far. They really helped me a >>> lot for preparing the three PRs. >>> >>> I'll be the next three weeks on vacation. >>> >>> Best regards >>> Rolf Rabenseifner >>> >>> >>> ----- Original Message ----- >>>> From: "Rolf Rabenseifner" <rabenseifner(a)hlrs.de> >>>> To: "Main MPI Forum mailing list" <mpi-forum(a)lists.mpi-forum.org> >>>> Cc: "Christoph Niethammer" <niethammer(a)hlrs.de>, "Puri Bangalore" >>>> <pvbangalore(a)ua.edu>, "Joseph Schuchart" >>>> <schuchart(a)icl.utk.edu> >>>> Sent: Tuesday, May 9, 2023 6:07:04 PM >>>> Subject: Vote announcement for the July 2023 meeting of the MPI >>>> Forum >>> >>>> Dear forum members, >>>> (correction: it is for the July 2023 meeting) >>>> >>>> I would like to make the following announcements for the next MPI >>>> Forum Meeting (July 10-13, 2023): >>>> >>>> - 2nd votes on >>>> #669 Add operation state 'enabled' and 'local calls' into Terms Rolf >>>> Issue https://github.com/mpi-forum/mpi-issues/issues/669 >>>> PR https://github.com/mpi-forum/mpi-standard/pull/788 >>>> >>>> #457 Improvements around the word "rank" in the Process Topologies chapter >>>> Christoph,Rolf >>>> Issue https://github.com/mpi-forum/mpi-issues/issues/457 >>>> PR https://github.com/mpi-forum/mpi-standard/pull/804 >>>> >>>> #485 Fix Incorrect Usage of Rank/Task/etc. in Language Bindings Chapter >>>> Rolf,Puri >>>> Issue https://github.com/mpi-forum/mpi-issues/issues/485 >>>> PR https://github.com/mpi-forum/mpi-standard/pull/803 >>>> PDF >>>> >>>> https://github.com/mpi-forum/mpi-standard/files/10831598/mpi41-repo >>>> r >>>> t_Issue485_PR803.pdf >>>> >>>> - errata reading and vote on >>>> >>>> #679 Errata: Noncollective (for procedure) and nonpersistent are not defined >>>> Rolf >>>> Issue https://github.com/mpi-forum/mpi-issues/issues/679 >>>> PR https://github.com/mpi-forum/mpi-standard/pull/820 >>>> PDF >>>> >>>> https://github.com/mpi-forum/mpi-standard/files/11432286/mpi41-repo >>>> r >>>> t_Issue679_PR820.pdf >>>> >>>> - pre-announcement for errata reading and vote on --> See also >>>> virtual meeting, June 7, 2023 !!! >>>> >>>> #705 Errata: Fortran has only compile-time constants Rolf,Joseph >>>> Issue https://github.com/mpi-forum/mpi-issues/issues/705 >>>> (together with https://github.com/mpi-forum/mpi-issues/issues/657 Rolf, Jeff >>>> H.) >>>> PR https://github.com/mpi-forum/mpi-standard/pull/822 >>>> >>>> #710 Errata: 'Pending communication' not defined in MPI_Comm_disconnect Rolf >>>> Issue https://github.com/mpi-forum/mpi-issues/issues/710 >>>> PR https://github.com/mpi-forum/mpi-standard/pull/823 >>>> >>>> #676 Errata: 'Pending operation' not defined, pending proper definition >>>> Rolf,Joseph >>>> Issue https://github.com/mpi-forum/mpi-issues/issues/676 >>>> PR https://github.com/mpi-forum/mpi-standard/pull/824 >>>> >>>> Best regards >>>> Rolf Rabenseifner > > > -- > Dr. Rolf Rabenseifner . . . . . . . . . .. . . rabenseifner(a)hlrs.de . > High Performance Computing Center (HLRS) . . . ++49(0)711/685-65530 . > University of Stuttgart . . . . . . www.hlrs.de/people/rabenseifner . > Nobelstr. 19, 70569 Stuttgart, Germany > -- > Dr. Rolf Rabenseifner . . . . . . . . . .. . . rabenseifner(a)hlrs.de . > High Performance Computing Center (HLRS) . . . ++49(0)711/685-65530 . > University of Stuttgart . . . . . . www.hlrs.de/people/rabenseifner . > Nobelstr. 19, 70569 Stuttgart, Germany > _______________________________________________ > mpiwg-sessions mailing list > mpiwg-sessions(a)lists.mpi-forum.org > https://lists.mpi-forum.org/mailman/listinfo/mpiwg-sessions

1 0

Re: [mpiwg-sessions] Virtual MPI forum meeting June 7, 2023 / Vote announcement for the July 2023 meeting of the MPI Forum
by Aurelien Bouteiller 08 Jun '23

08 Jun '23

> On Jun 8, 2023, at 10:15, Holmes, Daniel John via mpiwg-sessions <mpiwg-sessions(a)lists.mpi-forum.org> wrote: > > Hi Howard (& all), > > In the MPI Virtual meeting yesterday, Rolf presented a PR that tries to define the word "pending" in MPI. The discussion raised a whole bunch of issues and concerns. I took an action to raise some of them in the Sessions WG because they are introduced by (or made worse by) the sessions model. > > Quick summary of options in Rolf's proposal: > 1) Remove the word "pending" from the MPI Standard -- replace it with "active" where that makes sense, or something else when appropriate. > 2) Define "pending" to be a strict alias of "active" -- it can only be applied to MPI operations and it means "starting stage has been done but completion stage has not yet been done" > 2) Define "pending" to be anything task/activity on MPI 's to-do list -- encompassing all active operations but also all decoupled activities (i.e. progress). > > We discovered in the discussion of these options, in the context of MPI_FINALIZE and MPI_DISCONNECT (which must delay their return until "pending" things have been done) that an important restriction has been unintentionally removed by the introduction of sessions and we probably need to re-introduce parts of that restriction. > > When there was only the World Model, it was erroneous to call things like MPI_REQUEST_FREE after MPI_FINALIZE. Questions like "can I free a request after finalize?" -- derived from questions like "what happens to inactive persistent requests when I finalize MPI?" -- made no sense because you can look at the code with MPI_REQUEST_FREE after MPI_FINALIZE and immediate declare it is an erroneous program. > > Once sessions came into existence though, we can now call MPI_REQUEST_FREE after MPI_FINALIZE, as long as there is a session: > > ``` > MPI_SESSION_INIT(&sh) > MPI_INIT() > MPI_RECV_INIT(..., MPI_COMM_WORLD, &req) > MPI_FINALIZE() > MPI_REQUEST_FREE(&req) > MPI_SESSION_FINALIZE() > ``` > > This is clearly A Bad Idea (tm), but which MPI rule has been broken in the above pseudo-code? Without the two sessions lines of code, this is clearly erroneous, but with them it is not. Do we want this pattern to be erroneous or well-defined? > I think that example is still erroneous, because the request is posted on the WPM; there is some catchall text that makes not calling all procedures on WPM objects to complete them before calling MPI_FINALIZE erroneous code. If the current text doesn’t clearly says so (the completion criterion that everything should be inactive implies it for WAIT, etc, not as clearly for FREE on inactive persistent reqs), we should amend text so that it precludes using procedures on WPM handles after MPI_FINALIZE. However the same example with a request posted on the SPM, and calling free after SESSION_FINALIZE would be valid. As you say later in your essay, we have to choose if we want it to remain so, or if we want to errata this behavior out of existence. The SPM implementations do not support this, so there is no real backward incompatibility to be worried about when declaring this retro-actively that it was always erroneous. > There is no semantic reason why this cannot be deemed legal -- MPI_REQUEST_FREE is a local procedure, so the fact MPI_COMM_WORLD is no longer functional should be irrelevant -- it could just clean up locally allocated resources (the connections to other processes were cleaned up during MPI_FINALIZE). > > One technical reason to define this pattern to be erroneous is that the request probably has a ref-counted reference to the communicator and MPI_FINALIZE will hang waiting for that ref-count to reach zero, which it cannot do because the request will not be freed until after it returns. This is an implementation detail, but it is how at least one major MPI library is currently implemented. > > An implementation could instead choose to store a weak_ptr (https://en.cppreference.com/w/cpp/memory/weak_ptr) to the communicator in the request when the operation is inactive, attempt to get a shared_ptr from the weak_ptr (https://en.cppreference.com/w/cpp/memory/weak_ptr/lock) during the starting stage, and give up the shared_ptr during the completion stage. In such an implementation, an inactive request does not own the communicator associated with the operation, so finalize can destroy the communicator, and a freeing an inactive request never needs to dereference to get the communicator, so it doesn't care whether the communicator still exists or not. > > Much more important than technical feasibility is the question "why should we enable this pattern?" mumble mumble C++ destructors mumble garbage collected languages mumble. > > Taking a step back, there is a whole category of programs that used to be erroneous by definition (cannot make MPI calls after MPI_FINALIZE) that are no longer obviously erroneous. What do we do about them all? > > There are related issues/concerns here: if a rule is added to MPI that forces MPI_FINALIZE to free all requests associated with the World Model, should sessions follow suit and have MPI_SESSION_FINALIZE free requests from that session? There is text exhorting the user to enable and complete all MPI operations (in the World Mode) before calling MPI_FINALIZE, but there is no such exhortation for the sessions model; should there be one? Does finalizing a session destroy all the communicators from that session? Can a communicator handle be used after the session is finalized (e.g. in MPI_COMM_FREE)? > Not pushing in any particular direction here, just exposing some pros-cons: For the sake of uniformity, it would be desired that all of these FINALIZE/DISCONNECT procedures have a similar behavior, since they kind-of do the same thing in different contexts. This also has advantages in terms of implementation simplicity. However, one may consider the following example and want this to be valid code: ``` MPI_SESSION_INIT(&sh) MPI_COMM_CREATE_FROM_PSET(comm,...) MPI_SESSION_FINALIZE(&sh) MPI_SEND(comm); MPI_RECV(comm) MPI_COMM_DISCONNECT(comm) ``` Since the body of the application does not need to use session objects anymore after CREATE_FROM_PSET, users may be interested in closing the session immediately (which can obviously be done at that point, since no communications are pending when we close it, and they use disconnect to prevent straggling buffered message to linger in limbo when freeing the comm). We even had a discussion about having SESSION_FINALIZE_LOCAL at some point, presumably a good tool to support this. A disadvantage is obviously that we cannot assert that the library is valgrind-clean at SESSION_FINALIZE point, since a bunch of MPI objects have to remain allocated/refcounted for this to work. ~~~ One oblique consideration in that decision about uniformity is that we have the same problem for MPI_COMM_DISCONNECT, and that has existed for a sufficiently long time that it may not be as easy to just errata it out of existence in that case (I don’t think people are generally using DISCONNECT a lot to start with, and implementation review may reveal that problematic usage patterns never worked in practice, which would then open the way for the errata approach in the COMM_DISCONNECT case). Aurelien > Best wishes, > Dan. > > -----Original Message----- > From: Rolf Rabenseifner <rabenseifner(a)hlrs.de> > Sent: Wednesday, June 7, 2023 6:10 PM > To: Holmes, Daniel John <daniel.john.holmes(a)intel.com> > Subject: Fwd: Virtual MPI forum meeting June 7, 2023 / Vote announcement for the July 2023 meeting of the MPI Forum > > The pdf is > https://github.com/mpi-forum/mpi-standard/files/11676448/mpi41-report_Issue… > especially the questions on page 482 and 524. > > ----- Forwarded Message ----- > From: "Rolf Rabenseifner" <rabenseifner(a)hlrs.de> > To: "Main MPI Forum mailing list" <mpi-forum(a)lists.mpi-forum.org> > Sent: Wednesday, June 7, 2023 1:51:04 PM > Subject: Re: Virtual MPI forum meeting June 7, 2023 / Vote announcement for the July 2023 meeting of the MPI Forum > > Dear MPI forum members, > > last update for #710/PR823: second option (alternative B) and open questions for MPI_Session_finalize, see https://github.com/mpi-forum/mpi-standard/files/11676448/mpi41-report_Issue… > for the virtual meeting today. > > Kind regards > Rolf > > ________ email from yesterday: _________ > > for the virtual meeting this week (June 7, 2023) the following readings and discussions are scheduled. > Some technical decisions are needed and it may be best to do these decisions as institutional staw votes in an additional virtual meeting (e.g. next week), rather to do them ad hoc this week. > > The topics are #705/PR822, #710/PR823 with two options, #676/PR824 or PR825. > > Details: > __________________________ > #705 Errata: Fortran has only compile-time constants Rolf,Joseph > Issue https://github.com/mpi-forum/mpi-issues/issues/705 > (together with https://github.com/mpi-forum/mpi-issues/issues/657 Rolf, Jeff H.) > PR https://github.com/mpi-forum/mpi-standard/pull/822 > PDF > https://github.com/mpi-forum/mpi-standard/files/11450065/mpi41-report_Issue… > > This proposal seems to be already stable. > > __________________________ > #710 Errata: 'Pending communication' not defined in MPI_Comm_disconnect Rolf > Issue https://github.com/mpi-forum/mpi-issues/issues/710 > PR https://github.com/mpi-forum/mpi-standard/pull/823 > updated: > PDF https://github.com/mpi-forum/mpi-standard/files/11676448/mpi41-report_Issue… > > > There is severe critism about the current proposal in PR 823: > > "In this PR here, as far as I can see, the implementations are mainly affected if they > incorrectly require that there must not be any inactive request handle using the given comm. > Whether an MPI lib really internally frees the inactive handles or not, > is mainly a question of having dangling handles or not." > "Is there broad agreement that this is incorrect implementation behavior? I'm skeptical." > > And therefore, there are two different solutions: > - An advice to users, telling the consequences if they do not free inactive > handles before calling MPI_Comm_disconnect or MPI_Session_finalize. > or > - Automatically freeing them. > > And furthermore: > > "Whatever we decide, I imagine this needs to extend to MPI_SESSION_FINALIZE as well." > > I'll try to update the proposal before the meeting to have text for both possible solutions. > > __________________________ > #676 Errata: 'Pending operation' not defined, pending proper definition Rolf,Joseph > Issue https://github.com/mpi-forum/mpi-issues/issues/676 > PR https://github.com/mpi-forum/mpi-standard/pull/824 > PDF https://github.com/mpi-forum/mpi-standard/files/11668075/mpi41-report_Issue… > > Substituting "pending operation" by "active operation" > > or PR https://github.com/mpi-forum/mpi-standard/pull/825 > PDF https://github.com/mpi-forum/mpi-standard/files/11669354/mpi41-report_Issue… > > Defining "pending operation" as "active operation" (see page 12) > > Best regards > Rolf > > > ----- Original Message ----- >> From: "Rolf Rabenseifner" <rabenseifner(a)hlrs.de> >> To: "Main MPI Forum mailing list" <mpi-forum(a)lists.mpi-forum.org> >> Cc: "Christoph Niethammer" <niethammer(a)hlrs.de>, "Puri Bangalore" <pvbangalore(a)ua.edu>, "Joseph Schuchart" >> <schuchart(a)icl.utk.edu> >> Sent: Wednesday, May 24, 2023 1:36:42 PM >> Subject: Re: Update: Vote announcement for the July 2023 meeting of >> the MPI Forum > >> Martin, Wes, and all, >> >> I expect that we should reserve also June 14, 2023 for a possible >> continuation of open questions resulting from the discussions of >> >>> #705/PR822, #710/PR823, and #676/PR824 are now finalized at least for >>> the virtual forum meeting, June 7, 2023 >> >> and other pending items for MPI-4.1 from the June 7 virtual forum meeting. >> >> Best regards >> Rolf >> >> >> >> >> ----- Original Message ----- >>> From: "Rolf Rabenseifner" <rabenseifner(a)hlrs.de> >>> To: "Main MPI Forum mailing list" <mpi-forum(a)lists.mpi-forum.org> >>> Cc: "Christoph Niethammer" <niethammer(a)hlrs.de>, "Puri Bangalore" >>> <pvbangalore(a)ua.edu>, "Joseph Schuchart" >>> <schuchart(a)icl.utk.edu> >>> Sent: Thursday, May 11, 2023 8:05:07 PM >>> Subject: Update: Vote announcement for the July 2023 meeting of the >>> MPI Forum >> >>> Dear forum members, >>> >>> #705/PR822, #710/PR823, and #676/PR824 are now finalized at least for >>> the virtual forum meeting, June 7, 2023, details below: >>> >>> #705 Errata: Fortran has only compile-time constants Rolf,Joseph >>> Issue https://github.com/mpi-forum/mpi-issues/issues/705 >>> (together with https://github.com/mpi-forum/mpi-issues/issues/657 Rolf, Jeff >>> H.) >>> PR https://github.com/mpi-forum/mpi-standard/pull/822 >>> PDF >>> >>> https://github.com/mpi-forum/mpi-standard/files/11450065/mpi41-report >>> _Issue705%2B657_PR822.pdf >>> >>> #710 Errata: 'Pending communication' not defined in MPI_Comm_disconnect Rolf >>> Issue https://github.com/mpi-forum/mpi-issues/issues/710 >>> PR https://github.com/mpi-forum/mpi-standard/pull/823 >>> PDF >>> >>> https://github.com/mpi-forum/mpi-standard/files/11454896/mpi41-report >>> _Issue710_PR823.pdf >>> >>> #676 Errata: 'Pending operation' not defined, pending proper definition >>> Rolf,Joseph >>> Issue https://github.com/mpi-forum/mpi-issues/issues/676 >>> PR https://github.com/mpi-forum/mpi-standard/pull/824 >>> PDF >>> >>> https://github.com/mpi-forum/mpi-standard/files/11443976/mpi41-report >>> _Issue676_PR824.pdf >>> >>> The agenda of the virtual meeting should be reading and discussions if needed. >>> I did the proposals based on the discussion and the major goals: >>> - backward compatible, >>> - no performance drawbacks on critical paths, >>> - full consistency of the solution with all related parts of the MPI standard. >>> >>> If I have overseen something, then my apologies. >>> >>> If somebody wants to provide a completely different solution to one >>> of these issues, please feel free to do it. >>> - Thanks to Joseph who did PR 822, which was such a counter proposal for #705. >>> - This method is often better than doing many change requests for the given PR. >>> and it helps to completely check whether it is really consistent over >>> all related parts of the MPI standard. >>> - And thanks for all the comments so far. They really helped me a lot >>> for preparing the three PRs. >>> >>> I'll be the next three weeks on vacation. >>> >>> Best regards >>> Rolf Rabenseifner >>> >>> >>> ----- Original Message ----- >>>> From: "Rolf Rabenseifner" <rabenseifner(a)hlrs.de> >>>> To: "Main MPI Forum mailing list" <mpi-forum(a)lists.mpi-forum.org> >>>> Cc: "Christoph Niethammer" <niethammer(a)hlrs.de>, "Puri Bangalore" >>>> <pvbangalore(a)ua.edu>, "Joseph Schuchart" >>>> <schuchart(a)icl.utk.edu> >>>> Sent: Tuesday, May 9, 2023 6:07:04 PM >>>> Subject: Vote announcement for the July 2023 meeting of the MPI >>>> Forum >>> >>>> Dear forum members, >>>> (correction: it is for the July 2023 meeting) >>>> >>>> I would like to make the following announcements for the next MPI >>>> Forum Meeting (July 10-13, 2023): >>>> >>>> - 2nd votes on >>>> #669 Add operation state 'enabled' and 'local calls' into Terms Rolf >>>> Issue https://github.com/mpi-forum/mpi-issues/issues/669 >>>> PR https://github.com/mpi-forum/mpi-standard/pull/788 >>>> >>>> #457 Improvements around the word "rank" in the Process Topologies chapter >>>> Christoph,Rolf >>>> Issue https://github.com/mpi-forum/mpi-issues/issues/457 >>>> PR https://github.com/mpi-forum/mpi-standard/pull/804 >>>> >>>> #485 Fix Incorrect Usage of Rank/Task/etc. in Language Bindings Chapter >>>> Rolf,Puri >>>> Issue https://github.com/mpi-forum/mpi-issues/issues/485 >>>> PR https://github.com/mpi-forum/mpi-standard/pull/803 >>>> PDF >>>> >>>> https://github.com/mpi-forum/mpi-standard/files/10831598/mpi41-repor >>>> t_Issue485_PR803.pdf >>>> >>>> - errata reading and vote on >>>> >>>> #679 Errata: Noncollective (for procedure) and nonpersistent are not defined >>>> Rolf >>>> Issue https://github.com/mpi-forum/mpi-issues/issues/679 >>>> PR https://github.com/mpi-forum/mpi-standard/pull/820 >>>> PDF >>>> >>>> https://github.com/mpi-forum/mpi-standard/files/11432286/mpi41-repor >>>> t_Issue679_PR820.pdf >>>> >>>> - pre-announcement for errata reading and vote on --> See also >>>> virtual meeting, June 7, 2023 !!! >>>> >>>> #705 Errata: Fortran has only compile-time constants Rolf,Joseph >>>> Issue https://github.com/mpi-forum/mpi-issues/issues/705 >>>> (together with https://github.com/mpi-forum/mpi-issues/issues/657 Rolf, Jeff >>>> H.) >>>> PR https://github.com/mpi-forum/mpi-standard/pull/822 >>>> >>>> #710 Errata: 'Pending communication' not defined in MPI_Comm_disconnect Rolf >>>> Issue https://github.com/mpi-forum/mpi-issues/issues/710 >>>> PR https://github.com/mpi-forum/mpi-standard/pull/823 >>>> >>>> #676 Errata: 'Pending operation' not defined, pending proper definition >>>> Rolf,Joseph >>>> Issue https://github.com/mpi-forum/mpi-issues/issues/676 >>>> PR https://github.com/mpi-forum/mpi-standard/pull/824 >>>> >>>> Best regards >>>> Rolf Rabenseifner > > > -- > Dr. Rolf Rabenseifner . . . . . . . . . .. . . rabenseifner(a)hlrs.de . > High Performance Computing Center (HLRS) . . . ++49(0)711/685-65530 . > University of Stuttgart . . . . . . www.hlrs.de/people/rabenseifner . > Nobelstr. 19, 70569 Stuttgart, Germany > -- > Dr. Rolf Rabenseifner . . . . . . . . . .. . . rabenseifner(a)hlrs.de . > High Performance Computing Center (HLRS) . . . ++49(0)711/685-65530 . > University of Stuttgart . . . . . . www.hlrs.de/people/rabenseifner . > Nobelstr. 19, 70569 Stuttgart, Germany > _______________________________________________ > mpiwg-sessions mailing list > mpiwg-sessions(a)lists.mpi-forum.org > https://lists.mpi-forum.org/mailman/listinfo/mpiwg-sessions

1 0

FW: Virtual MPI forum meeting June 7, 2023 / Vote announcement for the July 2023 meeting of the MPI Forum
by Holmes, Daniel John 08 Jun '23

08 Jun '23

Hi Howard (& all), In the MPI Virtual meeting yesterday, Rolf presented a PR that tries to define the word "pending" in MPI. The discussion raised a whole bunch of issues and concerns. I took an action to raise some of them in the Sessions WG because they are introduced by (or made worse by) the sessions model. Quick summary of options in Rolf's proposal: 1) Remove the word "pending" from the MPI Standard -- replace it with "active" where that makes sense, or something else when appropriate. 2) Define "pending" to be a strict alias of "active" -- it can only be applied to MPI operations and it means "starting stage has been done but completion stage has not yet been done" 2) Define "pending" to be anything task/activity on MPI 's to-do list -- encompassing all active operations but also all decoupled activities (i.e. progress). We discovered in the discussion of these options, in the context of MPI_FINALIZE and MPI_DISCONNECT (which must delay their return until "pending" things have been done) that an important restriction has been unintentionally removed by the introduction of sessions and we probably need to re-introduce parts of that restriction. When there was only the World Model, it was erroneous to call things like MPI_REQUEST_FREE after MPI_FINALIZE. Questions like "can I free a request after finalize?" -- derived from questions like "what happens to inactive persistent requests when I finalize MPI?" -- made no sense because you can look at the code with MPI_REQUEST_FREE after MPI_FINALIZE and immediate declare it is an erroneous program. Once sessions came into existence though, we can now call MPI_REQUEST_FREE after MPI_FINALIZE, as long as there is a session: ``` MPI_SESSION_INIT(&sh) MPI_INIT() MPI_RECV_INIT(..., MPI_COMM_WORLD, &req) MPI_FINALIZE() MPI_REQUEST_FREE(&req) MPI_SESSION_FINALIZE() ``` This is clearly A Bad Idea (tm), but which MPI rule has been broken in the above pseudo-code? Without the two sessions lines of code, this is clearly erroneous, but with them it is not. Do we want this pattern to be erroneous or well-defined? There is no semantic reason why this cannot be deemed legal -- MPI_REQUEST_FREE is a local procedure, so the fact MPI_COMM_WORLD is no longer functional should be irrelevant -- it could just clean up locally allocated resources (the connections to other processes were cleaned up during MPI_FINALIZE). One technical reason to define this pattern to be erroneous is that the request probably has a ref-counted reference to the communicator and MPI_FINALIZE will hang waiting for that ref-count to reach zero, which it cannot do because the request will not be freed until after it returns. This is an implementation detail, but it is how at least one major MPI library is currently implemented. An implementation could instead choose to store a weak_ptr (https://en.cppreference.com/w/cpp/memory/weak_ptr) to the communicator in the request when the operation is inactive, attempt to get a shared_ptr from the weak_ptr (https://en.cppreference.com/w/cpp/memory/weak_ptr/lock) during the starting stage, and give up the shared_ptr during the completion stage. In such an implementation, an inactive request does not own the communicator associated with the operation, so finalize can destroy the communicator, and a freeing an inactive request never needs to dereference to get the communicator, so it doesn't care whether the communicator still exists or not. Much more important than technical feasibility is the question "why should we enable this pattern?" mumble mumble C++ destructors mumble garbage collected languages mumble. Taking a step back, there is a whole category of programs that used to be erroneous by definition (cannot make MPI calls after MPI_FINALIZE) that are no longer obviously erroneous. What do we do about them all? There are related issues/concerns here: if a rule is added to MPI that forces MPI_FINALIZE to free all requests associated with the World Model, should sessions follow suit and have MPI_SESSION_FINALIZE free requests from that session? There is text exhorting the user to enable and complete all MPI operations (in the World Mode) before calling MPI_FINALIZE, but there is no such exhortation for the sessions model; should there be one? Does finalizing a session destroy all the communicators from that session? Can a communicator handle be used after the session is finalized (e.g. in MPI_COMM_FREE)? Best wishes, Dan. -----Original Message----- From: Rolf Rabenseifner <rabenseifner(a)hlrs.de> Sent: Wednesday, June 7, 2023 6:10 PM To: Holmes, Daniel John <daniel.john.holmes(a)intel.com> Subject: Fwd: Virtual MPI forum meeting June 7, 2023 / Vote announcement for the July 2023 meeting of the MPI Forum The pdf is https://github.com/mpi-forum/mpi-standard/files/11676448/mpi41-report_Issue… especially the questions on page 482 and 524. ----- Forwarded Message ----- From: "Rolf Rabenseifner" <rabenseifner(a)hlrs.de> To: "Main MPI Forum mailing list" <mpi-forum(a)lists.mpi-forum.org> Sent: Wednesday, June 7, 2023 1:51:04 PM Subject: Re: Virtual MPI forum meeting June 7, 2023 / Vote announcement for the July 2023 meeting of the MPI Forum Dear MPI forum members, last update for #710/PR823: second option (alternative B) and open questions for MPI_Session_finalize, see https://github.com/mpi-forum/mpi-standard/files/11676448/mpi41-report_Issue… for the virtual meeting today. Kind regards Rolf ________ email from yesterday: _________ for the virtual meeting this week (June 7, 2023) the following readings and discussions are scheduled. Some technical decisions are needed and it may be best to do these decisions as institutional staw votes in an additional virtual meeting (e.g. next week), rather to do them ad hoc this week. The topics are #705/PR822, #710/PR823 with two options, #676/PR824 or PR825. Details: __________________________ #705 Errata: Fortran has only compile-time constants Rolf,Joseph Issue https://github.com/mpi-forum/mpi-issues/issues/705 (together with https://github.com/mpi-forum/mpi-issues/issues/657 Rolf, Jeff H.) PR https://github.com/mpi-forum/mpi-standard/pull/822 PDF https://github.com/mpi-forum/mpi-standard/files/11450065/mpi41-report_Issue… This proposal seems to be already stable. __________________________ #710 Errata: 'Pending communication' not defined in MPI_Comm_disconnect Rolf Issue https://github.com/mpi-forum/mpi-issues/issues/710 PR https://github.com/mpi-forum/mpi-standard/pull/823 updated: PDF https://github.com/mpi-forum/mpi-standard/files/11676448/mpi41-report_Issue… There is severe critism about the current proposal in PR 823: "In this PR here, as far as I can see, the implementations are mainly affected if they incorrectly require that there must not be any inactive request handle using the given comm. Whether an MPI lib really internally frees the inactive handles or not, is mainly a question of having dangling handles or not." "Is there broad agreement that this is incorrect implementation behavior? I'm skeptical." And therefore, there are two different solutions: - An advice to users, telling the consequences if they do not free inactive handles before calling MPI_Comm_disconnect or MPI_Session_finalize. or - Automatically freeing them. And furthermore: "Whatever we decide, I imagine this needs to extend to MPI_SESSION_FINALIZE as well." I'll try to update the proposal before the meeting to have text for both possible solutions. __________________________ #676 Errata: 'Pending operation' not defined, pending proper definition Rolf,Joseph Issue https://github.com/mpi-forum/mpi-issues/issues/676 PR https://github.com/mpi-forum/mpi-standard/pull/824 PDF https://github.com/mpi-forum/mpi-standard/files/11668075/mpi41-report_Issue… Substituting "pending operation" by "active operation" or PR https://github.com/mpi-forum/mpi-standard/pull/825 PDF https://github.com/mpi-forum/mpi-standard/files/11669354/mpi41-report_Issue… Defining "pending operation" as "active operation" (see page 12) Best regards Rolf ----- Original Message ----- > From: "Rolf Rabenseifner" <rabenseifner(a)hlrs.de> > To: "Main MPI Forum mailing list" <mpi-forum(a)lists.mpi-forum.org> > Cc: "Christoph Niethammer" <niethammer(a)hlrs.de>, "Puri Bangalore" <pvbangalore(a)ua.edu>, "Joseph Schuchart" > <schuchart(a)icl.utk.edu> > Sent: Wednesday, May 24, 2023 1:36:42 PM > Subject: Re: Update: Vote announcement for the July 2023 meeting of > the MPI Forum > Martin, Wes, and all, > > I expect that we should reserve also June 14, 2023 for a possible > continuation of open questions resulting from the discussions of > >> #705/PR822, #710/PR823, and #676/PR824 are now finalized at least for >> the virtual forum meeting, June 7, 2023 > > and other pending items for MPI-4.1 from the June 7 virtual forum meeting. > > Best regards > Rolf > > > > > ----- Original Message ----- >> From: "Rolf Rabenseifner" <rabenseifner(a)hlrs.de> >> To: "Main MPI Forum mailing list" <mpi-forum(a)lists.mpi-forum.org> >> Cc: "Christoph Niethammer" <niethammer(a)hlrs.de>, "Puri Bangalore" >> <pvbangalore(a)ua.edu>, "Joseph Schuchart" >> <schuchart(a)icl.utk.edu> >> Sent: Thursday, May 11, 2023 8:05:07 PM >> Subject: Update: Vote announcement for the July 2023 meeting of the >> MPI Forum > >> Dear forum members, >> >> #705/PR822, #710/PR823, and #676/PR824 are now finalized at least for >> the virtual forum meeting, June 7, 2023, details below: >> >> #705 Errata: Fortran has only compile-time constants Rolf,Joseph >> Issue https://github.com/mpi-forum/mpi-issues/issues/705 >> (together with https://github.com/mpi-forum/mpi-issues/issues/657 Rolf, Jeff >> H.) >> PR https://github.com/mpi-forum/mpi-standard/pull/822 >> PDF >> >> https://github.com/mpi-forum/mpi-standard/files/11450065/mpi41-report >> _Issue705%2B657_PR822.pdf >> >> #710 Errata: 'Pending communication' not defined in MPI_Comm_disconnect Rolf >> Issue https://github.com/mpi-forum/mpi-issues/issues/710 >> PR https://github.com/mpi-forum/mpi-standard/pull/823 >> PDF >> >> https://github.com/mpi-forum/mpi-standard/files/11454896/mpi41-report >> _Issue710_PR823.pdf >> >> #676 Errata: 'Pending operation' not defined, pending proper definition >> Rolf,Joseph >> Issue https://github.com/mpi-forum/mpi-issues/issues/676 >> PR https://github.com/mpi-forum/mpi-standard/pull/824 >> PDF >> >> https://github.com/mpi-forum/mpi-standard/files/11443976/mpi41-report >> _Issue676_PR824.pdf >> >> The agenda of the virtual meeting should be reading and discussions if needed. >> I did the proposals based on the discussion and the major goals: >> - backward compatible, >> - no performance drawbacks on critical paths, >> - full consistency of the solution with all related parts of the MPI standard. >> >> If I have overseen something, then my apologies. >> >> If somebody wants to provide a completely different solution to one >> of these issues, please feel free to do it. >> - Thanks to Joseph who did PR 822, which was such a counter proposal for #705. >> - This method is often better than doing many change requests for the given PR. >> and it helps to completely check whether it is really consistent over >> all related parts of the MPI standard. >> - And thanks for all the comments so far. They really helped me a lot >> for preparing the three PRs. >> >> I'll be the next three weeks on vacation. >> >> Best regards >> Rolf Rabenseifner >> >> >> ----- Original Message ----- >>> From: "Rolf Rabenseifner" <rabenseifner(a)hlrs.de> >>> To: "Main MPI Forum mailing list" <mpi-forum(a)lists.mpi-forum.org> >>> Cc: "Christoph Niethammer" <niethammer(a)hlrs.de>, "Puri Bangalore" >>> <pvbangalore(a)ua.edu>, "Joseph Schuchart" >>> <schuchart(a)icl.utk.edu> >>> Sent: Tuesday, May 9, 2023 6:07:04 PM >>> Subject: Vote announcement for the July 2023 meeting of the MPI >>> Forum >> >>> Dear forum members, >>> (correction: it is for the July 2023 meeting) >>> >>> I would like to make the following announcements for the next MPI >>> Forum Meeting (July 10-13, 2023): >>> >>> - 2nd votes on >>> #669 Add operation state 'enabled' and 'local calls' into Terms Rolf >>> Issue https://github.com/mpi-forum/mpi-issues/issues/669 >>> PR https://github.com/mpi-forum/mpi-standard/pull/788 >>> >>> #457 Improvements around the word "rank" in the Process Topologies chapter >>> Christoph,Rolf >>> Issue https://github.com/mpi-forum/mpi-issues/issues/457 >>> PR https://github.com/mpi-forum/mpi-standard/pull/804 >>> >>> #485 Fix Incorrect Usage of Rank/Task/etc. in Language Bindings Chapter >>> Rolf,Puri >>> Issue https://github.com/mpi-forum/mpi-issues/issues/485 >>> PR https://github.com/mpi-forum/mpi-standard/pull/803 >>> PDF >>> >>> https://github.com/mpi-forum/mpi-standard/files/10831598/mpi41-repor >>> t_Issue485_PR803.pdf >>> >>> - errata reading and vote on >>> >>> #679 Errata: Noncollective (for procedure) and nonpersistent are not defined >>> Rolf >>> Issue https://github.com/mpi-forum/mpi-issues/issues/679 >>> PR https://github.com/mpi-forum/mpi-standard/pull/820 >>> PDF >>> >>> https://github.com/mpi-forum/mpi-standard/files/11432286/mpi41-repor >>> t_Issue679_PR820.pdf >>> >>> - pre-announcement for errata reading and vote on --> See also >>> virtual meeting, June 7, 2023 !!! >>> >>> #705 Errata: Fortran has only compile-time constants Rolf,Joseph >>> Issue https://github.com/mpi-forum/mpi-issues/issues/705 >>> (together with https://github.com/mpi-forum/mpi-issues/issues/657 Rolf, Jeff >>> H.) >>> PR https://github.com/mpi-forum/mpi-standard/pull/822 >>> >>> #710 Errata: 'Pending communication' not defined in MPI_Comm_disconnect Rolf >>> Issue https://github.com/mpi-forum/mpi-issues/issues/710 >>> PR https://github.com/mpi-forum/mpi-standard/pull/823 >>> >>> #676 Errata: 'Pending operation' not defined, pending proper definition >>> Rolf,Joseph >>> Issue https://github.com/mpi-forum/mpi-issues/issues/676 >>> PR https://github.com/mpi-forum/mpi-standard/pull/824 >>> >>> Best regards >>> Rolf Rabenseifner -- Dr. Rolf Rabenseifner . . . . . . . . . .. . . rabenseifner(a)hlrs.de . High Performance Computing Center (HLRS) . . . ++49(0)711/685-65530 . University of Stuttgart . . . . . . www.hlrs.de/people/rabenseifner . Nobelstr. 19, 70569 Stuttgart, Germany -- Dr. Rolf Rabenseifner . . . . . . . . . .. . . rabenseifner(a)hlrs.de . High Performance Computing Center (HLRS) . . . ++49(0)711/685-65530 . University of Stuttgart . . . . . . www.hlrs.de/people/rabenseifner . Nobelstr. 19, 70569 Stuttgart, Germany

1 0