July 2020 - mpiwg-persistence - mailman3.cels.anl.gov

Re: [mpiwg-persistence] [mpiwg-semantic-terms] Summary of Semantics for Partitioned Communication Operations
by Bangalore, Purushotham 28 Jul '20

28 Jul '20

I have created a presentation here<https://docs.google.com/presentation/d/1CdPbSL_hKtHc5y-LN-IJ-IbG6fPlnWIhDaJ…> based on today's WG meeting, please feel free to update this. See you all tomorrow. Thanks, Puri ________________________________ From: mpiwg-persistence <mpiwg-persistence-bounces(a)lists.mpi-forum.org> on behalf of Bangalore, Purushotham via mpiwg-persistence <mpiwg-persistence(a)lists.mpi-forum.org> Sent: Friday, July 24, 2020 11:10 AM To: mpiwg-semantic-terms(a)lists.mpi-forum.org <mpiwg-semantic-terms(a)lists.mpi-forum.org>; mpiwg-persistence(a)lists.mpi-forum.org <mpiwg-persistence(a)lists.mpi-forum.org> Subject: Re: [mpiwg-persistence] [mpiwg-semantic-terms] Summary of Semantics for Partitioned Communication Operations Hi All, Here is the updated spreadsheet <https://drive.google.com/file/d/1VkkqAd56lgxEt3JEt-zmGR0aC1VvLTDh/view?usp=…> for the appendix and the updated text<https://docs.google.com/document/d/12M6glWfNDt3WpLsf8FNug2ma0cRG2QZ_mNKnvUl…> for the semantic terms section based on the discussion with Ryan related to partitioned communication functions this week. Please review. The basic idea is to introduce a new operation type called partitioned operation. This would require the introduction of a new stage called ready stage after the starting stage. Ryan has updated the text for the ready stage and we have added a new state transition diagram for the partitioned operations. Thanks, Puri ________________________________ From: mpiwg-semantic-terms <mpiwg-semantic-terms-bounces(a)lists.mpi-forum.org> on behalf of Bangalore, Purushotham via mpiwg-semantic-terms <mpiwg-semantic-terms(a)lists.mpi-forum.org> Sent: Tuesday, July 21, 2020 4:01 PM To: Rolf Rabenseifner <rabenseifner(a)hlrs.de> Cc: mpiwg-semantic-terms <mpiwg-semantic-terms(a)lists.mpi-forum.org>; mpiwg-persistence(a)lists.mpi-forum.org <mpiwg-persistence(a)lists.mpi-forum.org> Subject: Re: [mpiwg-semantic-terms] [mpiwg-persistence] Summary of Semantics for Partitioned Communication Operations Hi Rolf, I used the text as a placeholder. This text currently appears as an advice to implementors in the partitioned communication chapter. I will let the partitioned communication WG decide the appropriate text for the footnote 🙂 Thanks, Puri ________________________________ From: Rolf Rabenseifner <rabenseifner(a)hlrs.de> Sent: Tuesday, July 21, 2020 3:43 PM To: Bangalore, Purushotham <puri(a)uab.edu> Cc: mpiwg-persistence(a)lists.mpi-forum.org <mpiwg-persistence(a)lists.mpi-forum.org>; mpiwg-semantic-terms <mpiwg-semantic-terms(a)lists.mpi-forum.org> Subject: Re: [mpiwg-semantic-terms] [mpiwg-persistence] Summary of Semantics for Partitioned Communication Operations Dear puri and all, I expect that the proposed footnote "20) Unlike MPI_SEND_INIT and MPI_RECV_INIT, MPI_PSEND_INIT and MPI_PRECV_INIT may communicate." is incorrect. MPI_SEND_INIT and MPI_RECV_INIT, MPI_PSEND_INIT and MPI_PRECV_INIT are all local procedures and we know, that local is weak local and no line in the MPI Standard can prohibit that a (weak) local is performing some "internal" communication for optimization. As long as the user's (= application programmer's) viewpoint can see only a local routine, and the freeing is also a local routine, even in the case that there was never an MPI_START, then all is fine. Maybe you meant: 20) Although these are local routines, an MPI implementation may internally already start some communication of metadata, but must guarantee that the INIT and the MPI_REQUEST_FREE are both local procedures. But such a comment should be an advice to implementors in the definition of the MPI_P..._INIT routines, and not as a footnote here. My comment is completely independent of what is written in the MPI Standard elsewhere. MPI-3.1 p 73 line 41 on persistent pt-to-pt init: "These calls involve no communication." In principle, this sentence should be substituted by "These procedures are local and do not send or receive any buffer content." But this may be a cleanup for MPI 4.1 ;-) Such a sentence should be part of the new partitioned comm. chapter! Best regards Rolf ----- Original Message ----- > From: "mpiwg-semantic-terms" <mpiwg-semantic-terms(a)lists.mpi-forum.org> > To: mpiwg-persistence(a)lists.mpi-forum.org > Cc: "mpiwg-semantic-terms" <mpiwg-semantic-terms(a)lists.mpi-forum.org> > Sent: Tuesday, July 21, 2020 8:34:09 PM > Subject: Re: [mpiwg-semantic-terms] [mpiwg-persistence] Summary of Semantics for Partitioned Communication Operations > Here is the updated spreadsheet based on Dan's suggestions. I would really like > the persistence WG to finalize this ASAP. > > I know Ryan would like to review the changes to the chapter before it is merged > during tomorrow's meeting. If there is time left after that, I would like to > discuss this. > > Thanks, > Puri > > > From: mpiwg-persistence <mpiwg-persistence-bounces(a)lists.mpi-forum.org> on > behalf of Bangalore, Purushotham via mpiwg-persistence > <mpiwg-persistence(a)lists.mpi-forum.org> > Sent: Wednesday, July 15, 2020 12:54 PM > To: HOLMES Daniel <d.holmes(a)epcc.ed.ac.uk> > Cc: mpiwg-semantic-terms(a)lists.mpi-forum.org > <mpiwg-semantic-terms(a)lists.mpi-forum.org>; > mpiwg-persistence(a)lists.mpi-forum.org <mpiwg-persistence(a)lists.mpi-forum.org> > Subject: Re: [mpiwg-persistence] Summary of Semantics for Partitioned > Communication Operations > Here are the other footnotes (I should have pasted them near footnote 20): > > 7) Addresses are cached on the request handle. > > 9) One shall not free or deallocate the bu er before the operation is freed, > that is MPI_REQUEST_FREE returned. > > 14) Nonblocking procedure without an I pre fix. > > ===== > I will move PREADY,...,PARRIVED after START... good suggestion. > > I also agree with "ic" for the PREADY, PARRIVED procedures. > > > From: HOLMES Daniel <d.holmes(a)epcc.ed.ac.uk> > Sent: Wednesday, July 15, 2020 12:42 PM > To: Bangalore, Purushotham <puri(a)uab.edu> > Cc: mpiwg-persistence(a)lists.mpi-forum.org > <mpiwg-persistence(a)lists.mpi-forum.org>; > mpiwg-semantic-terms(a)lists.mpi-forum.org > <mpiwg-semantic-terms(a)lists.mpi-forum.org> > Subject: Re: [mpiwg-persistence] Summary of Semantics for Partitioned > Communication Operations > Hi Puri, > > Thanks for this - a great way to provoke some progress (no pun intended). > > I would suggest: move the rows dealing with MPI_PREADY and MPI_PARRIVED between > MPI_START and MPI_WAIT, i.e. in the approximate calling order. > > Also, I would add “ic” for the new MPI_PREADY, MPI_PARRIVED, etc procedures - > because the operation is definitely incomplete at the point in time they are > called, during the time interval of their entire execution, and at the point in > time they return - there is no possibility they could be completing or freeing > procedures. > > What are the other footnotes: 7, 9, and 14? (I could look them up but I’d have > to choose the same draft version as you because the numbering changed > recently.) > > Cheers, > Dan. > — > Dr Daniel Holmes PhD > Architect (HPC Research) > [ mailto:[email protected] | d.holmes(a)epcc.ed.ac.uk ] > Phone: +44 (0) 131 651 3465 > Mobile: +44 (0) 7940 524 088 > Address: Room 2.09, Bayes Centre, 47 Potterrow, Central Area, Edinburgh, EH8 9BT > — > The University of Edinburgh is a charitable body, registered in Scotland, with > registration number SC005336. > — > > > > > On 15 Jul 2020, at 18:24, Bangalore, Purushotham via mpiwg-persistence < [ > mailto:[email protected] | > mpiwg-persistence(a)lists.mpi-forum.org ] > wrote: > > <MPI-semantics-appendix.xlsx> > > > -- > mpiwg-semantic-terms mailing list > mpiwg-semantic-terms(a)lists.mpi-forum.org > https://lists.mpi-forum.org/mailman/listinfo/mpiwg-semantic-terms -- Dr. Rolf Rabenseifner . . . . . . . . . .. email rabenseifner(a)hlrs.de . High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 . University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 . Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner<http://www.hlrs.de/people/rabenseifner> . Nobelstr. 19, D-70550 Stuttgart, Germany . . . . (Office: Room 1.307) .

1 0

Re: [mpiwg-persistence] [mpiwg-semantic-terms] Summary of Semantics for Partitioned Communication Operations
by Bangalore, Purushotham 24 Jul '20

24 Jul '20

Hi All, Here is the updated spreadsheet <https://drive.google.com/file/d/1VkkqAd56lgxEt3JEt-zmGR0aC1VvLTDh/view?usp=…> for the appendix and the updated text<https://docs.google.com/document/d/12M6glWfNDt3WpLsf8FNug2ma0cRG2QZ_mNKnvUl…> for the semantic terms section based on the discussion with Ryan related to partitioned communication functions this week. Please review. The basic idea is to introduce a new operation type called partitioned operation. This would require the introduction of a new stage called ready stage after the starting stage. Ryan has updated the text for the ready stage and we have added a new state transition diagram for the partitioned operations. Thanks, Puri ________________________________ From: mpiwg-semantic-terms <mpiwg-semantic-terms-bounces(a)lists.mpi-forum.org> on behalf of Bangalore, Purushotham via mpiwg-semantic-terms <mpiwg-semantic-terms(a)lists.mpi-forum.org> Sent: Tuesday, July 21, 2020 4:01 PM To: Rolf Rabenseifner <rabenseifner(a)hlrs.de> Cc: mpiwg-semantic-terms <mpiwg-semantic-terms(a)lists.mpi-forum.org>; mpiwg-persistence(a)lists.mpi-forum.org <mpiwg-persistence(a)lists.mpi-forum.org> Subject: Re: [mpiwg-semantic-terms] [mpiwg-persistence] Summary of Semantics for Partitioned Communication Operations Hi Rolf, I used the text as a placeholder. This text currently appears as an advice to implementors in the partitioned communication chapter. I will let the partitioned communication WG decide the appropriate text for the footnote 🙂 Thanks, Puri ________________________________ From: Rolf Rabenseifner <rabenseifner(a)hlrs.de> Sent: Tuesday, July 21, 2020 3:43 PM To: Bangalore, Purushotham <puri(a)uab.edu> Cc: mpiwg-persistence(a)lists.mpi-forum.org <mpiwg-persistence(a)lists.mpi-forum.org>; mpiwg-semantic-terms <mpiwg-semantic-terms(a)lists.mpi-forum.org> Subject: Re: [mpiwg-semantic-terms] [mpiwg-persistence] Summary of Semantics for Partitioned Communication Operations Dear puri and all, I expect that the proposed footnote "20) Unlike MPI_SEND_INIT and MPI_RECV_INIT, MPI_PSEND_INIT and MPI_PRECV_INIT may communicate." is incorrect. MPI_SEND_INIT and MPI_RECV_INIT, MPI_PSEND_INIT and MPI_PRECV_INIT are all local procedures and we know, that local is weak local and no line in the MPI Standard can prohibit that a (weak) local is performing some "internal" communication for optimization. As long as the user's (= application programmer's) viewpoint can see only a local routine, and the freeing is also a local routine, even in the case that there was never an MPI_START, then all is fine. Maybe you meant: 20) Although these are local routines, an MPI implementation may internally already start some communication of metadata, but must guarantee that the INIT and the MPI_REQUEST_FREE are both local procedures. But such a comment should be an advice to implementors in the definition of the MPI_P..._INIT routines, and not as a footnote here. My comment is completely independent of what is written in the MPI Standard elsewhere. MPI-3.1 p 73 line 41 on persistent pt-to-pt init: "These calls involve no communication." In principle, this sentence should be substituted by "These procedures are local and do not send or receive any buffer content." But this may be a cleanup for MPI 4.1 ;-) Such a sentence should be part of the new partitioned comm. chapter! Best regards Rolf ----- Original Message ----- > From: "mpiwg-semantic-terms" <mpiwg-semantic-terms(a)lists.mpi-forum.org> > To: mpiwg-persistence(a)lists.mpi-forum.org > Cc: "mpiwg-semantic-terms" <mpiwg-semantic-terms(a)lists.mpi-forum.org> > Sent: Tuesday, July 21, 2020 8:34:09 PM > Subject: Re: [mpiwg-semantic-terms] [mpiwg-persistence] Summary of Semantics for Partitioned Communication Operations > Here is the updated spreadsheet based on Dan's suggestions. I would really like > the persistence WG to finalize this ASAP. > > I know Ryan would like to review the changes to the chapter before it is merged > during tomorrow's meeting. If there is time left after that, I would like to > discuss this. > > Thanks, > Puri > > > From: mpiwg-persistence <mpiwg-persistence-bounces(a)lists.mpi-forum.org> on > behalf of Bangalore, Purushotham via mpiwg-persistence > <mpiwg-persistence(a)lists.mpi-forum.org> > Sent: Wednesday, July 15, 2020 12:54 PM > To: HOLMES Daniel <d.holmes(a)epcc.ed.ac.uk> > Cc: mpiwg-semantic-terms(a)lists.mpi-forum.org > <mpiwg-semantic-terms(a)lists.mpi-forum.org>; > mpiwg-persistence(a)lists.mpi-forum.org <mpiwg-persistence(a)lists.mpi-forum.org> > Subject: Re: [mpiwg-persistence] Summary of Semantics for Partitioned > Communication Operations > Here are the other footnotes (I should have pasted them near footnote 20): > > 7) Addresses are cached on the request handle. > > 9) One shall not free or deallocate the bu er before the operation is freed, > that is MPI_REQUEST_FREE returned. > > 14) Nonblocking procedure without an I pre fix. > > ===== > I will move PREADY,...,PARRIVED after START... good suggestion. > > I also agree with "ic" for the PREADY, PARRIVED procedures. > > > From: HOLMES Daniel <d.holmes(a)epcc.ed.ac.uk> > Sent: Wednesday, July 15, 2020 12:42 PM > To: Bangalore, Purushotham <puri(a)uab.edu> > Cc: mpiwg-persistence(a)lists.mpi-forum.org > <mpiwg-persistence(a)lists.mpi-forum.org>; > mpiwg-semantic-terms(a)lists.mpi-forum.org > <mpiwg-semantic-terms(a)lists.mpi-forum.org> > Subject: Re: [mpiwg-persistence] Summary of Semantics for Partitioned > Communication Operations > Hi Puri, > > Thanks for this - a great way to provoke some progress (no pun intended). > > I would suggest: move the rows dealing with MPI_PREADY and MPI_PARRIVED between > MPI_START and MPI_WAIT, i.e. in the approximate calling order. > > Also, I would add “ic” for the new MPI_PREADY, MPI_PARRIVED, etc procedures - > because the operation is definitely incomplete at the point in time they are > called, during the time interval of their entire execution, and at the point in > time they return - there is no possibility they could be completing or freeing > procedures. > > What are the other footnotes: 7, 9, and 14? (I could look them up but I’d have > to choose the same draft version as you because the numbering changed > recently.) > > Cheers, > Dan. > — > Dr Daniel Holmes PhD > Architect (HPC Research) > [ mailto:[email protected] | d.holmes(a)epcc.ed.ac.uk ] > Phone: +44 (0) 131 651 3465 > Mobile: +44 (0) 7940 524 088 > Address: Room 2.09, Bayes Centre, 47 Potterrow, Central Area, Edinburgh, EH8 9BT > — > The University of Edinburgh is a charitable body, registered in Scotland, with > registration number SC005336. > — > > > > > On 15 Jul 2020, at 18:24, Bangalore, Purushotham via mpiwg-persistence < [ > mailto:[email protected] | > mpiwg-persistence(a)lists.mpi-forum.org ] > wrote: > > <MPI-semantics-appendix.xlsx> > > > -- > mpiwg-semantic-terms mailing list > mpiwg-semantic-terms(a)lists.mpi-forum.org > https://lists.mpi-forum.org/mailman/listinfo/mpiwg-semantic-terms -- Dr. Rolf Rabenseifner . . . . . . . . . .. email rabenseifner(a)hlrs.de . High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 . University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 . Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner<http://www.hlrs.de/people/rabenseifner> . Nobelstr. 19, D-70550 Stuttgart, Germany . . . . (Office: Room 1.307) .

1 0

Re: [mpiwg-persistence] MPI Persistent/Collective WGs Weekly Meeting
by Jim Dinan 22 Jul '20

22 Jul '20

Hi All, Apologies, I had a conflict today and wasn't able to join. I'm also unavailable next week, but should be able to join the following meeting (in two weeks). Cheers, ~Jim. On Wed, Jul 22, 2020 at 7:49 AM Skjellum, Anthony via mpiwg-persistence < mpiwg-persistence(a)lists.mpi-forum.org> wrote: > All, we will continue discussion of partitioned pt2pt with > GPU/FPGA/accelerator offload semantics, and need for synchronization of the > sender/receiver ... > > Anthony Skjellum, PhD > > Professor of Computer Science and Chair of Excellence > > Director, SimCenter > > University of Tennessee at Chattanooga (UTC) > > tony-skjellum(a)utc.edu [or skjellum(a)gmail.com] > > cell: 205-807-4968 > > > > ------------------------------ > *From:* Skjellum, Anthony > *Sent:* Wednesday, October 11, 2017 7:04 AM > *To:* mpiwg-persistence(a)lists.mpi-forum.org < > mpiwg-persistence(a)lists.mpi-forum.org>; mpiwg-coll(a)lists.mpi-forum.org < > mpiwg-coll(a)lists.mpi-forum.org>; Julien JAEGER <Julien.JAEGER(a)CEA.FR> > *Subject:* MPI Persistent/Collective WGs Weekly Meeting > *When:* Wednesday, July 22, 2020 10:00 AM-11:00 AM. > *Where:* > > > Now, we use only zoom meetings: > > > > Hi there, > > Anthony Skjellum is inviting you to a scheduled Zoom meeting. > > Topic: Anthony Skjellum's Personal Meeting Room > > Join from PC, Mac, Linux, iOS or Android:* > https://tennessee.zoom.us/j/9916910516 > <https://tennessee.zoom.us/j/9916910516>* > > Or iPhone one-tap (US Toll): +16468769923,9916910516# or > +16699006833,9916910516# > > Or Telephone: > Dial: > +1 646 876 9923 (US Toll) > +1 669 900 6833 (US Toll) > Meeting ID: 991 691 0516 > International numbers available: https://zoom.us/u/acVk33oBeV > > Or an H.323/SIP room system: > H.323: 162.255.37.11 (US West) or 162.255.36.11 (US East) > Meeting ID: 991 691 0516 > > SIP: 9916910516(a)zoomcrc.com > > > > > _______________________________________________ > mpiwg-persistence mailing list > mpiwg-persistence(a)lists.mpi-forum.org > https://lists.mpi-forum.org/mailman/listinfo/mpiwg-persistence >

1 0

Re: [mpiwg-persistence] [EXTERNAL] Re: MPI Persistent/Collective WGs Weekly Meeting
by Grant, Ryan Eric (-EXP) 22 Jul '20

22 Jul '20

Hi everyone, Here is the link to the latest changes that I mentioned on today's call: https://github.com/mpi-forum/mpi-standard/pull/117/commits/690caace73b1c56f… Comments are welcome. Thanks, Ryan ________________________________ From: mpiwg-persistence <mpiwg-persistence-bounces(a)lists.mpi-forum.org> on behalf of Skjellum, Anthony via mpiwg-persistence <mpiwg-persistence(a)lists.mpi-forum.org> Sent: Wednesday, July 22, 2020 5:49 AM To: mpiwg-persistence(a)lists.mpi-forum.org; mpiwg-coll(a)lists.mpi-forum.org; Julien JAEGER Subject: [EXTERNAL] Re: [mpiwg-persistence] MPI Persistent/Collective WGs Weekly Meeting All, we will continue discussion of partitioned pt2pt with GPU/FPGA/accelerator offload semantics, and need for synchronization of the sender/receiver ... Anthony Skjellum, PhD Professor of Computer Science and Chair of Excellence Director, SimCenter University of Tennessee at Chattanooga (UTC) tony-skjellum(a)utc.edu [or skjellum(a)gmail.com] cell: 205-807-4968 ________________________________ From: Skjellum, Anthony Sent: Wednesday, October 11, 2017 7:04 AM To: mpiwg-persistence(a)lists.mpi-forum.org <mpiwg-persistence(a)lists.mpi-forum.org>; mpiwg-coll(a)lists.mpi-forum.org <mpiwg-coll(a)lists.mpi-forum.org>; Julien JAEGER <Julien.JAEGER(a)CEA.FR> Subject: MPI Persistent/Collective WGs Weekly Meeting When: Wednesday, July 22, 2020 10:00 AM-11:00 AM. Where: Now, we use only zoom meetings: Hi there, Anthony Skjellum is inviting you to a scheduled Zoom meeting. Topic: Anthony Skjellum's Personal Meeting Room Join from PC, Mac, Linux, iOS or Android: https://tennessee.zoom.us/j/9916910516 Or iPhone one-tap (US Toll): +16468769923,9916910516# or +16699006833,9916910516# Or Telephone: Dial: +1 646 876 9923 (US Toll) +1 669 900 6833 (US Toll) Meeting ID: 991 691 0516 International numbers available: https://zoom.us/u/acVk33oBeV Or an H.323/SIP room system: H.323: 162.255.37.11 (US West) or 162.255.36.11 (US East) Meeting ID: 991 691 0516 SIP: 9916910516(a)zoomcrc.com

1 0

Re: [mpiwg-persistence] MPI Persistent/Collective WGs Weekly Meeting
by Skjellum, Anthony 22 Jul '20

22 Jul '20

All, we will continue discussion of partitioned pt2pt with GPU/FPGA/accelerator offload semantics, and need for synchronization of the sender/receiver ... Anthony Skjellum, PhD Professor of Computer Science and Chair of Excellence Director, SimCenter University of Tennessee at Chattanooga (UTC) tony-skjellum(a)utc.edu [or skjellum(a)gmail.com] cell: 205-807-4968 ________________________________ From: Skjellum, Anthony Sent: Wednesday, October 11, 2017 7:04 AM To: mpiwg-persistence(a)lists.mpi-forum.org <mpiwg-persistence(a)lists.mpi-forum.org>; mpiwg-coll(a)lists.mpi-forum.org <mpiwg-coll(a)lists.mpi-forum.org>; Julien JAEGER <Julien.JAEGER(a)CEA.FR> Subject: MPI Persistent/Collective WGs Weekly Meeting When: Wednesday, July 22, 2020 10:00 AM-11:00 AM. Where: Now, we use only zoom meetings: Hi there, Anthony Skjellum is inviting you to a scheduled Zoom meeting. Topic: Anthony Skjellum's Personal Meeting Room Join from PC, Mac, Linux, iOS or Android: https://tennessee.zoom.us/j/9916910516 Or iPhone one-tap (US Toll): +16468769923,9916910516# or +16699006833,9916910516# Or Telephone: Dial: +1 646 876 9923 (US Toll) +1 669 900 6833 (US Toll) Meeting ID: 991 691 0516 International numbers available: https://zoom.us/u/acVk33oBeV Or an H.323/SIP room system: H.323: 162.255.37.11 (US West) or 162.255.36.11 (US East) Meeting ID: 991 691 0516 SIP: 9916910516(a)zoomcrc.com

1 0

Re: [mpiwg-persistence] [mpiwg-semantic-terms] Summary of Semantics for Partitioned Communication Operations
by Bangalore, Purushotham 21 Jul '20

21 Jul '20

Hi Rolf, I used the text as a placeholder. This text currently appears as an advice to implementors in the partitioned communication chapter. I will let the partitioned communication WG decide the appropriate text for the footnote 🙂 Thanks, Puri ________________________________ From: Rolf Rabenseifner <rabenseifner(a)hlrs.de> Sent: Tuesday, July 21, 2020 3:43 PM To: Bangalore, Purushotham <puri(a)uab.edu> Cc: mpiwg-persistence(a)lists.mpi-forum.org <mpiwg-persistence(a)lists.mpi-forum.org>; mpiwg-semantic-terms <mpiwg-semantic-terms(a)lists.mpi-forum.org> Subject: Re: [mpiwg-semantic-terms] [mpiwg-persistence] Summary of Semantics for Partitioned Communication Operations Dear puri and all, I expect that the proposed footnote "20) Unlike MPI_SEND_INIT and MPI_RECV_INIT, MPI_PSEND_INIT and MPI_PRECV_INIT may communicate." is incorrect. MPI_SEND_INIT and MPI_RECV_INIT, MPI_PSEND_INIT and MPI_PRECV_INIT are all local procedures and we know, that local is weak local and no line in the MPI Standard can prohibit that a (weak) local is performing some "internal" communication for optimization. As long as the user's (= application programmer's) viewpoint can see only a local routine, and the freeing is also a local routine, even in the case that there was never an MPI_START, then all is fine. Maybe you meant: 20) Although these are local routines, an MPI implementation may internally already start some communication of metadata, but must guarantee that the INIT and the MPI_REQUEST_FREE are both local procedures. But such a comment should be an advice to implementors in the definition of the MPI_P..._INIT routines, and not as a footnote here. My comment is completely independent of what is written in the MPI Standard elsewhere. MPI-3.1 p 73 line 41 on persistent pt-to-pt init: "These calls involve no communication." In principle, this sentence should be substituted by "These procedures are local and do not send or receive any buffer content." But this may be a cleanup for MPI 4.1 ;-) Such a sentence should be part of the new partitioned comm. chapter! Best regards Rolf ----- Original Message ----- > From: "mpiwg-semantic-terms" <mpiwg-semantic-terms(a)lists.mpi-forum.org> > To: mpiwg-persistence(a)lists.mpi-forum.org > Cc: "mpiwg-semantic-terms" <mpiwg-semantic-terms(a)lists.mpi-forum.org> > Sent: Tuesday, July 21, 2020 8:34:09 PM > Subject: Re: [mpiwg-semantic-terms] [mpiwg-persistence] Summary of Semantics for Partitioned Communication Operations > Here is the updated spreadsheet based on Dan's suggestions. I would really like > the persistence WG to finalize this ASAP. > > I know Ryan would like to review the changes to the chapter before it is merged > during tomorrow's meeting. If there is time left after that, I would like to > discuss this. > > Thanks, > Puri > > > From: mpiwg-persistence <mpiwg-persistence-bounces(a)lists.mpi-forum.org> on > behalf of Bangalore, Purushotham via mpiwg-persistence > <mpiwg-persistence(a)lists.mpi-forum.org> > Sent: Wednesday, July 15, 2020 12:54 PM > To: HOLMES Daniel <d.holmes(a)epcc.ed.ac.uk> > Cc: mpiwg-semantic-terms(a)lists.mpi-forum.org > <mpiwg-semantic-terms(a)lists.mpi-forum.org>; > mpiwg-persistence(a)lists.mpi-forum.org <mpiwg-persistence(a)lists.mpi-forum.org> > Subject: Re: [mpiwg-persistence] Summary of Semantics for Partitioned > Communication Operations > Here are the other footnotes (I should have pasted them near footnote 20): > > 7) Addresses are cached on the request handle. > > 9) One shall not free or deallocate the bu er before the operation is freed, > that is MPI_REQUEST_FREE returned. > > 14) Nonblocking procedure without an I pre fix. > > ===== > I will move PREADY,...,PARRIVED after START... good suggestion. > > I also agree with "ic" for the PREADY, PARRIVED procedures. > > > From: HOLMES Daniel <d.holmes(a)epcc.ed.ac.uk> > Sent: Wednesday, July 15, 2020 12:42 PM > To: Bangalore, Purushotham <puri(a)uab.edu> > Cc: mpiwg-persistence(a)lists.mpi-forum.org > <mpiwg-persistence(a)lists.mpi-forum.org>; > mpiwg-semantic-terms(a)lists.mpi-forum.org > <mpiwg-semantic-terms(a)lists.mpi-forum.org> > Subject: Re: [mpiwg-persistence] Summary of Semantics for Partitioned > Communication Operations > Hi Puri, > > Thanks for this - a great way to provoke some progress (no pun intended). > > I would suggest: move the rows dealing with MPI_PREADY and MPI_PARRIVED between > MPI_START and MPI_WAIT, i.e. in the approximate calling order. > > Also, I would add “ic” for the new MPI_PREADY, MPI_PARRIVED, etc procedures - > because the operation is definitely incomplete at the point in time they are > called, during the time interval of their entire execution, and at the point in > time they return - there is no possibility they could be completing or freeing > procedures. > > What are the other footnotes: 7, 9, and 14? (I could look them up but I’d have > to choose the same draft version as you because the numbering changed > recently.) > > Cheers, > Dan. > — > Dr Daniel Holmes PhD > Architect (HPC Research) > [ mailto:[email protected] | d.holmes(a)epcc.ed.ac.uk ] > Phone: +44 (0) 131 651 3465 > Mobile: +44 (0) 7940 524 088 > Address: Room 2.09, Bayes Centre, 47 Potterrow, Central Area, Edinburgh, EH8 9BT > — > The University of Edinburgh is a charitable body, registered in Scotland, with > registration number SC005336. > — > > > > > On 15 Jul 2020, at 18:24, Bangalore, Purushotham via mpiwg-persistence < [ > mailto:[email protected] | > mpiwg-persistence(a)lists.mpi-forum.org ] > wrote: > > <MPI-semantics-appendix.xlsx> > > > -- > mpiwg-semantic-terms mailing list > mpiwg-semantic-terms(a)lists.mpi-forum.org > https://lists.mpi-forum.org/mailman/listinfo/mpiwg-semantic-terms -- Dr. Rolf Rabenseifner . . . . . . . . . .. email rabenseifner(a)hlrs.de . High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 . University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 . Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner<http://www.hlrs.de/people/rabenseifner> . Nobelstr. 19, D-70550 Stuttgart, Germany . . . . (Office: Room 1.307) .

1 0

Re: [mpiwg-persistence] [mpiwg-semantic-terms] Summary of Semantics for Partitioned Communication Operations
by Rolf Rabenseifner 21 Jul '20

21 Jul '20

Dear puri and all, I expect that the proposed footnote "20) Unlike MPI_SEND_INIT and MPI_RECV_INIT, MPI_PSEND_INIT and MPI_PRECV_INIT may communicate." is incorrect. MPI_SEND_INIT and MPI_RECV_INIT, MPI_PSEND_INIT and MPI_PRECV_INIT are all local procedures and we know, that local is weak local and no line in the MPI Standard can prohibit that a (weak) local is performing some "internal" communication for optimization. As long as the user's (= application programmer's) viewpoint can see only a local routine, and the freeing is also a local routine, even in the case that there was never an MPI_START, then all is fine. Maybe you meant: 20) Although these are local routines, an MPI implementation may internally already start some communication of metadata, but must guarantee that the INIT and the MPI_REQUEST_FREE are both local procedures. But such a comment should be an advice to implementors in the definition of the MPI_P..._INIT routines, and not as a footnote here. My comment is completely independent of what is written in the MPI Standard elsewhere. MPI-3.1 p 73 line 41 on persistent pt-to-pt init: "These calls involve no communication." In principle, this sentence should be substituted by "These procedures are local and do not send or receive any buffer content." But this may be a cleanup for MPI 4.1 ;-) Such a sentence should be part of the new partitioned comm. chapter! Best regards Rolf ----- Original Message ----- > From: "mpiwg-semantic-terms" <mpiwg-semantic-terms(a)lists.mpi-forum.org> > To: mpiwg-persistence(a)lists.mpi-forum.org > Cc: "mpiwg-semantic-terms" <mpiwg-semantic-terms(a)lists.mpi-forum.org> > Sent: Tuesday, July 21, 2020 8:34:09 PM > Subject: Re: [mpiwg-semantic-terms] [mpiwg-persistence] Summary of Semantics for Partitioned Communication Operations > Here is the updated spreadsheet based on Dan's suggestions. I would really like > the persistence WG to finalize this ASAP. > > I know Ryan would like to review the changes to the chapter before it is merged > during tomorrow's meeting. If there is time left after that, I would like to > discuss this. > > Thanks, > Puri > > > From: mpiwg-persistence <mpiwg-persistence-bounces(a)lists.mpi-forum.org> on > behalf of Bangalore, Purushotham via mpiwg-persistence > <mpiwg-persistence(a)lists.mpi-forum.org> > Sent: Wednesday, July 15, 2020 12:54 PM > To: HOLMES Daniel <d.holmes(a)epcc.ed.ac.uk> > Cc: mpiwg-semantic-terms(a)lists.mpi-forum.org > <mpiwg-semantic-terms(a)lists.mpi-forum.org>; > mpiwg-persistence(a)lists.mpi-forum.org <mpiwg-persistence(a)lists.mpi-forum.org> > Subject: Re: [mpiwg-persistence] Summary of Semantics for Partitioned > Communication Operations > Here are the other footnotes (I should have pasted them near footnote 20): > > 7) Addresses are cached on the request handle. > > 9) One shall not free or deallocate the bu er before the operation is freed, > that is MPI_REQUEST_FREE returned. > > 14) Nonblocking procedure without an I pre fix. > > ===== > I will move PREADY,...,PARRIVED after START... good suggestion. > > I also agree with "ic" for the PREADY, PARRIVED procedures. > > > From: HOLMES Daniel <d.holmes(a)epcc.ed.ac.uk> > Sent: Wednesday, July 15, 2020 12:42 PM > To: Bangalore, Purushotham <puri(a)uab.edu> > Cc: mpiwg-persistence(a)lists.mpi-forum.org > <mpiwg-persistence(a)lists.mpi-forum.org>; > mpiwg-semantic-terms(a)lists.mpi-forum.org > <mpiwg-semantic-terms(a)lists.mpi-forum.org> > Subject: Re: [mpiwg-persistence] Summary of Semantics for Partitioned > Communication Operations > Hi Puri, > > Thanks for this - a great way to provoke some progress (no pun intended). > > I would suggest: move the rows dealing with MPI_PREADY and MPI_PARRIVED between > MPI_START and MPI_WAIT, i.e. in the approximate calling order. > > Also, I would add “ic” for the new MPI_PREADY, MPI_PARRIVED, etc procedures - > because the operation is definitely incomplete at the point in time they are > called, during the time interval of their entire execution, and at the point in > time they return - there is no possibility they could be completing or freeing > procedures. > > What are the other footnotes: 7, 9, and 14? (I could look them up but I’d have > to choose the same draft version as you because the numbering changed > recently.) > > Cheers, > Dan. > — > Dr Daniel Holmes PhD > Architect (HPC Research) > [ mailto:[email protected] | d.holmes(a)epcc.ed.ac.uk ] > Phone: +44 (0) 131 651 3465 > Mobile: +44 (0) 7940 524 088 > Address: Room 2.09, Bayes Centre, 47 Potterrow, Central Area, Edinburgh, EH8 9BT > — > The University of Edinburgh is a charitable body, registered in Scotland, with > registration number SC005336. > — > > > > > On 15 Jul 2020, at 18:24, Bangalore, Purushotham via mpiwg-persistence < [ > mailto:[email protected] | > mpiwg-persistence(a)lists.mpi-forum.org ] > wrote: > > <MPI-semantics-appendix.xlsx> > > > -- > mpiwg-semantic-terms mailing list > mpiwg-semantic-terms(a)lists.mpi-forum.org > https://lists.mpi-forum.org/mailman/listinfo/mpiwg-semantic-terms -- Dr. Rolf Rabenseifner . . . . . . . . . .. email rabenseifner(a)hlrs.de . High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 . University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 . Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner . Nobelstr. 19, D-70550 Stuttgart, Germany . . . . (Office: Room 1.307) .

1 0

Re: [mpiwg-persistence] Summary of Semantics for Partitioned Communication Operations
by Bangalore, Purushotham 21 Jul '20

21 Jul '20

Here is the updated spreadsheet based on Dan's suggestions. I would really like the persistence WG to finalize this ASAP. I know Ryan would like to review the changes to the chapter before it is merged during tomorrow's meeting. If there is time left after that, I would like to discuss this. Thanks, Puri ________________________________ From: mpiwg-persistence <mpiwg-persistence-bounces(a)lists.mpi-forum.org> on behalf of Bangalore, Purushotham via mpiwg-persistence <mpiwg-persistence(a)lists.mpi-forum.org> Sent: Wednesday, July 15, 2020 12:54 PM To: HOLMES Daniel <d.holmes(a)epcc.ed.ac.uk> Cc: mpiwg-semantic-terms(a)lists.mpi-forum.org <mpiwg-semantic-terms(a)lists.mpi-forum.org>; mpiwg-persistence(a)lists.mpi-forum.org <mpiwg-persistence(a)lists.mpi-forum.org> Subject: Re: [mpiwg-persistence] Summary of Semantics for Partitioned Communication Operations Here are the other footnotes (I should have pasted them near footnote 20): 7) Addresses are cached on the request handle. 9) One shall not free or deallocate the bu er before the operation is freed, that is MPI_REQUEST_FREE returned. 14) Nonblocking procedure without an I pre fix. ===== I will move PREADY,...,PARRIVED after START... good suggestion. I also agree with "ic" for the PREADY, PARRIVED procedures. ________________________________ From: HOLMES Daniel <d.holmes(a)epcc.ed.ac.uk> Sent: Wednesday, July 15, 2020 12:42 PM To: Bangalore, Purushotham <puri(a)uab.edu> Cc: mpiwg-persistence(a)lists.mpi-forum.org <mpiwg-persistence(a)lists.mpi-forum.org>; mpiwg-semantic-terms(a)lists.mpi-forum.org <mpiwg-semantic-terms(a)lists.mpi-forum.org> Subject: Re: [mpiwg-persistence] Summary of Semantics for Partitioned Communication Operations Hi Puri, Thanks for this - a great way to provoke some progress (no pun intended). I would suggest: move the rows dealing with MPI_PREADY and MPI_PARRIVED between MPI_START and MPI_WAIT, i.e. in the approximate calling order. Also, I would add “ic” for the new MPI_PREADY, MPI_PARRIVED, etc procedures - because the operation is definitely incomplete at the point in time they are called, during the time interval of their entire execution, and at the point in time they return - there is no possibility they could be completing or freeing procedures. What are the other footnotes: 7, 9, and 14? (I could look them up but I’d have to choose the same draft version as you because the numbering changed recently.) Cheers, Dan. — Dr Daniel Holmes PhD Architect (HPC Research) d.holmes(a)epcc.ed.ac.uk<mailto:[email protected]> Phone: +44 (0) 131 651 3465 Mobile: +44 (0) 7940 524 088 Address: Room 2.09, Bayes Centre, 47 Potterrow, Central Area, Edinburgh, EH8 9BT — The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. — On 15 Jul 2020, at 18:24, Bangalore, Purushotham via mpiwg-persistence <mpiwg-persistence(a)lists.mpi-forum.org<mailto:[email protected]>> wrote: <MPI-semantics-appendix.xlsx>

1 0

Re: [mpiwg-persistence] [EXT]: Partitioned communication - Accelerator support
by Skjellum, Anthony 15 Jul '20

15 Jul '20

thank you. Anthony Skjellum, PhD Professor of Computer Science and Chair of Excellence Director, SimCenter University of Tennessee at Chattanooga (UTC) tony-skjellum(a)utc.edu [or skjellum(a)gmail.com] cell: 205-807-4968 ________________________________ From: mpiwg-persistence <mpiwg-persistence-bounces(a)lists.mpi-forum.org> on behalf of Bangalore, Purushotham via mpiwg-persistence <mpiwg-persistence(a)lists.mpi-forum.org> Sent: Wednesday, July 15, 2020 3:37 PM To: mpiwg-persistence(a)lists.mpi-forum.org <mpiwg-persistence(a)lists.mpi-forum.org>; mpiwg-coll(a)lists.mpi-forum.org <mpiwg-coll(a)lists.mpi-forum.org> Subject: [EXT]: [mpiwg-persistence] Partitioned communication - Accelerator support External Email I am starting a new thread with the appropriate subject header to separate out the discussion on this topic. I have also cc'ed the collective communications group and removed the semantic terms group for this discussion. Thanks, Puri ________________________________ From: Skjellum, Anthony <Tony-Skjellum(a)utc.edu> Sent: Wednesday, July 15, 2020 12:53 PM To: Bangalore, Purushotham <puri(a)uab.edu>; HOLMES Daniel <d.holmes(a)epcc.ed.ac.uk> Cc: mpiwg-semantic-terms(a)lists.mpi-forum.org <mpiwg-semantic-terms(a)lists.mpi-forum.org>; mpiwg-persistence(a)lists.mpi-forum.org <mpiwg-persistence(a)lists.mpi-forum.org> Subject: Re: [mpiwg-persistence] Summary of Semantics for Partitioned Communication Operations We had a short meeting with Ryan and students---he advocates for a blocking PSYNC on both sides now... after start... before kernel launch on send side... Anthony Skjellum, PhD Professor of Computer Science and Chair of Excellence Director, SimCenter University of Tennessee at Chattanooga (UTC) tony-skjellum(a)utc.edu [or skjellum(a)gmail.com] cell: 205-807-4968 ________________________________ From: mpiwg-persistence <mpiwg-persistence-bounces(a)lists.mpi-forum.org> on behalf of HOLMES Daniel via mpiwg-persistence <mpiwg-persistence(a)lists.mpi-forum.org> Sent: Wednesday, July 15, 2020 1:42 PM To: Purushotham Bangalore <puri(a)uab.edu> Cc: mpiwg-semantic-terms(a)lists.mpi-forum.org <mpiwg-semantic-terms(a)lists.mpi-forum.org>; mpiwg-persistence(a)lists.mpi-forum.org <mpiwg-persistence(a)lists.mpi-forum.org> Subject: Re: [mpiwg-persistence] Summary of Semantics for Partitioned Communication Operations Hi Puri, Thanks for this - a great way to provoke some progress (no pun intended). I would suggest: move the rows dealing with MPI_PREADY and MPI_PARRIVED between MPI_START and MPI_WAIT, i.e. in the approximate calling order. Also, I would add “ic” for the new MPI_PREADY, MPI_PARRIVED, etc procedures - because the operation is definitely incomplete at the point in time they are called, during the time interval of their entire execution, and at the point in time they return - there is no possibility they could be completing or freeing procedures. What are the other footnotes: 7, 9, and 14? (I could look them up but I’d have to choose the same draft version as you because the numbering changed recently.) Cheers, Dan. ― Dr Daniel Holmes PhD Architect (HPC Research) d.holmes(a)epcc.ed.ac.uk<mailto:[email protected]> Phone: +44 (0) 131 651 3465 Mobile: +44 (0) 7940 524 088 Address: Room 2.09, Bayes Centre, 47 Potterrow, Central Area, Edinburgh, EH8 9BT ― The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. ― On 15 Jul 2020, at 18:24, Bangalore, Purushotham via mpiwg-persistence <mpiwg-persistence(a)lists.mpi-forum.org<mailto:[email protected]>> wrote: <MPI-semantics-appendix.xlsx> This message is not from a UTC.EDU address. Caution should be used in clicking links and downloading attachments from unknown senders or unexpected email.

1 0

Partitioned communication - Accelerator support
by Bangalore, Purushotham 15 Jul '20

15 Jul '20

I am starting a new thread with the appropriate subject header to separate out the discussion on this topic. I have also cc'ed the collective communications group and removed the semantic terms group for this discussion. Thanks, Puri ________________________________ From: Skjellum, Anthony <Tony-Skjellum(a)utc.edu> Sent: Wednesday, July 15, 2020 12:53 PM To: Bangalore, Purushotham <puri(a)uab.edu>; HOLMES Daniel <d.holmes(a)epcc.ed.ac.uk> Cc: mpiwg-semantic-terms(a)lists.mpi-forum.org <mpiwg-semantic-terms(a)lists.mpi-forum.org>; mpiwg-persistence(a)lists.mpi-forum.org <mpiwg-persistence(a)lists.mpi-forum.org> Subject: Re: [mpiwg-persistence] Summary of Semantics for Partitioned Communication Operations We had a short meeting with Ryan and students---he advocates for a blocking PSYNC on both sides now... after start... before kernel launch on send side... Anthony Skjellum, PhD Professor of Computer Science and Chair of Excellence Director, SimCenter University of Tennessee at Chattanooga (UTC) tony-skjellum(a)utc.edu [or skjellum(a)gmail.com] cell: 205-807-4968 ________________________________ From: mpiwg-persistence <mpiwg-persistence-bounces(a)lists.mpi-forum.org> on behalf of HOLMES Daniel via mpiwg-persistence <mpiwg-persistence(a)lists.mpi-forum.org> Sent: Wednesday, July 15, 2020 1:42 PM To: Purushotham Bangalore <puri(a)uab.edu> Cc: mpiwg-semantic-terms(a)lists.mpi-forum.org <mpiwg-semantic-terms(a)lists.mpi-forum.org>; mpiwg-persistence(a)lists.mpi-forum.org <mpiwg-persistence(a)lists.mpi-forum.org> Subject: Re: [mpiwg-persistence] Summary of Semantics for Partitioned Communication Operations Hi Puri, Thanks for this - a great way to provoke some progress (no pun intended). I would suggest: move the rows dealing with MPI_PREADY and MPI_PARRIVED between MPI_START and MPI_WAIT, i.e. in the approximate calling order. Also, I would add “ic” for the new MPI_PREADY, MPI_PARRIVED, etc procedures - because the operation is definitely incomplete at the point in time they are called, during the time interval of their entire execution, and at the point in time they return - there is no possibility they could be completing or freeing procedures. What are the other footnotes: 7, 9, and 14? (I could look them up but I’d have to choose the same draft version as you because the numbering changed recently.) Cheers, Dan. — Dr Daniel Holmes PhD Architect (HPC Research) d.holmes(a)epcc.ed.ac.uk<mailto:[email protected]> Phone: +44 (0) 131 651 3465 Mobile: +44 (0) 7940 524 088 Address: Room 2.09, Bayes Centre, 47 Potterrow, Central Area, Edinburgh, EH8 9BT — The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. — On 15 Jul 2020, at 18:24, Bangalore, Purushotham via mpiwg-persistence <mpiwg-persistence(a)lists.mpi-forum.org<mailto:[email protected]>> wrote: <MPI-semantics-appendix.xlsx>

1 0