October 2022 - mpiwg-persistence - mailman3.cels.anl.gov

Re: [mpiwg-persistence] MPI Persistent/HACC/Partitioned WGs Weekly Meeting---ZOOM NOT TEAMS
by Skjellum, Anthony 18 Oct '22

18 Oct '22

HI, We do not have any Partitioned/Persistent/Collective material for 10/19, see you at the virtual meeting. There may be HACC meeting, I will let Jim remind you. -Tony Anthony Skjellum, PhD Professor of Computer Science and Chair of Excellence Director, SimCenter University of Tennessee at Chattanooga (UTC) tony-skjellum(a)utc.edu [or skjellum(a)gmail.com] cell: 205-807-4968 ________________________________ From: Skjellum, Anthony Sent: Wednesday, May 12, 2021 10:53 AM To: mpiwg-coll(a)lists.mpi-forum.org <mpiwg-coll(a)lists.mpi-forum.org>; mpiwg-persistence(a)lists.mpi-forum.org <mpiwg-persistence(a)lists.mpi-forum.org>; mpiwg-hybridpm(a)lists.mpi-forum.urg <mpiwg-hybridpm(a)lists.mpi-forum.urg> Cc: m.a.ruefenacht(a)gmail.com <m.a.ruefenacht(a)gmail.com>; Guillaume Mercier <guillaume.mercier(a)u-bordeaux.fr>; Amanzholov, Muzakhir <msamanzhol42(a)tntech.edu>; Christoph Niethammer <niethammer(a)hlrs.de>; Ghafoor, Sheikh <SGhafoor(a)tntech.edu>; Schulz, Martin <martin.w.j.schulz(a)tum.de>; Julien.JAEGER(a)CEA.FR <Julien.JAEGER(a)CEA.FR>; daniel.collis.holmes(a)gmail.com <daniel.collis.holmes(a)gmail.com>; Garzaran, Maria <maria.garzaran(a)intel.com>; Raffenetti, Kenneth J. <raffenet(a)mcs.anl.gov>; Shipley, Riley Prescott <riley-shipley(a)utc.edu>; Dosanjh, Matthew <mdosanj(a)sandia.gov>; Hugo Taboada <hugo.taboada(a)cea.fr>; jeff.science(a)gmail.com <jeff.science(a)gmail.com>; Suggs, Evan <evan-suggs(a)utc.edu>; Smith, Brian <smithbe(a)ornl.gov>; Klemm, Michael <Michael.Klemm(a)amd.com>; Patrick Bridges <patrickb(a)unm.edu>; Purushotham Bangalore <pvbangalore(a)ua.edu>; Ryan Grant <regrant233(a)gmail.com>; Wesley Bland <work(a)wesbland.com>; Bronis R. de Supinski <bronis(a)llnl.gov>; Elis, Bengisu <bengisu.elis(a)tum.de>; de Supinski, Bronis R. <desupinski1(a)llnl.gov>; Raffenetti, Ken <raffenet(a)anl.gov>; Schonbein, William Whitney <wwschon(a)sandia.gov>; McMahon, Kevin; Subramoni, Hari <subramoni.1(a)osu.edu>; Hines, Thomas; Thakur, Rajeev <thakur(a)anl.gov> Subject: MPI Persistent/HACC/Partitioned WGs Weekly Meeting---ZOOM NOT TEAMS When: Wednesday, October 19, 2022 10:00 AM-11:00 AM. Where: NOTE: Zoom, not TEAMS, see below. As of May 12, 2021, WG materials being archived here https://drive.google.com/drive/folders/1dxWKDUxvzol5PQfMET2bwiiNs7rZfimX?us… Weekly: https://tennessee.zoom.us/meeting/tJIsfuqtrDsiE9S0ICfXsuy-47_0ZqWvG71z/ics?… Join from PC, Mac, Linux, iOS or Android: https://tennessee.zoom.us/j/96193520890?pwd=T3k1NkFRQ3BJaUFkeXFQcmtYTlRHQT09 Password: 399650 ---- Hi there, Anthony Skjellum is inviting you to a scheduled Zoom meeting. Topic: Partition/HACC/etc Working Groups Time: Apr 6, 2022 10:00 AM Eastern Time (US and Canada) Every week on Wed, until Jan 25, 2023, 43 occurrence(s) Apr 6, 2022 10:00 AM Apr 13, 2022 10:00 AM Apr 20, 2022 10:00 AM Apr 27, 2022 10:00 AM May 4, 2022 10:00 AM May 11, 2022 10:00 AM May 18, 2022 10:00 AM May 25, 2022 10:00 AM Jun 1, 2022 10:00 AM Jun 8, 2022 10:00 AM Jun 15, 2022 10:00 AM Jun 22, 2022 10:00 AM Jun 29, 2022 10:00 AM Jul 6, 2022 10:00 AM Jul 13, 2022 10:00 AM Jul 20, 2022 10:00 AM Jul 27, 2022 10:00 AM Aug 3, 2022 10:00 AM Aug 10, 2022 10:00 AM Aug 17, 2022 10:00 AM Aug 24, 2022 10:00 AM Aug 31, 2022 10:00 AM Sep 7, 2022 10:00 AM Sep 14, 2022 10:00 AM Sep 21, 2022 10:00 AM Sep 28, 2022 10:00 AM Oct 5, 2022 10:00 AM Oct 12, 2022 10:00 AM Oct 19, 2022 10:00 AM Oct 26, 2022 10:00 AM Nov 2, 2022 10:00 AM Nov 9, 2022 10:00 AM Nov 16, 2022 10:00 AM Nov 23, 2022 10:00 AM Nov 30, 2022 10:00 AM Dec 7, 2022 10:00 AM Dec 14, 2022 10:00 AM Dec 21, 2022 10:00 AM Dec 28, 2022 10:00 AM Jan 4, 2023 10:00 AM Jan 11, 2023 10:00 AM Jan 18, 2023 10:00 AM Jan 25, 2023 10:00 AM Please download and import the following iCalendar (.ics) files to your calendar system. Weekly: https://tennessee.zoom.us/meeting/tJIsfuqtrDsiE9S0ICfXsuy-47_0ZqWvG71z/ics?… Join from PC, Mac, Linux, iOS or Android: https://tennessee.zoom.us/j/96193520890?pwd=T3k1NkFRQ3BJaUFkeXFQcmtYTlRHQT09 Password: 399650 Or iPhone one-tap (US Toll): +16468769923,96193520890# or +13017158592,96193520890# Or Telephone: Dial: +1 646 876 9923 (US Toll) +1 301 715 8592 (US Toll) +1 312 626 6799 (US Toll) +1 669 900 6833 (US Toll) +1 253 215 8782 (US Toll) +1 346 248 7799 (US Toll) Meeting ID: 961 9352 0890 International numbers available: https://tennessee.zoom.us/u/adIMYx6Jko Or an H.323/SIP room system: H.323: 162.255.37.11 (US West) or 162.255.36.11 (US East) Meeting ID: 961 9352 0890 Password: 399650 SIP: 96193520890(a)zoomcrc.com Password: 399650 <https://teams.microsoft.com/meetingOptions/?organizerId=e544ff3d-a271-4377-…> ________________________________________________________________________________

1 0

Re: [mpiwg-persistence] Completion call before all partitions marked ready
by Joachim Jenke 12 Oct '22

12 Oct '22

I think it is worth clarifying that it is valid behavior, since the sentence in MPI4,p104,l15 seems to state that the completion call follows all the pready calls: > Once an MPI_PSEND_INIT call has been made, the user may start the > operation with a call to a starting procedure and complete the > operation with a number of MPI_PREADY calls equal to the requested > number of send partitions followed by a call to a completing > procedure. The responsibility to avoid deadlocks is clear. From the DL analysis point of view, such ordering requirement would make the analysis actually easier :) Without such ordering requirement, a runtime analysis tool like MUST cannot clearly identify a spin-loop on test as deadlock/livelock. Searching for above text, I found an inconsistency in the restrictions on the partition argument. MPI_Pready has (MPI4,p109,l11): > Specifying a partition number that is equal to or larger than the > number of partitions is erroneous. First, I was surprised that there is no restriction on negative values, but this restriction is buried in the argument list comments. Having an unsigned int argument would make it more clear. MPI_Parrived does not have such upper-bound restriction. - Joachim Am 12.10.22 um 17:09 schrieb Ryan Grant: > This is correct, you’re allowed to call test/wait whenever you want as > long as you abide by the rules that all pready calls must be made in > order for completion to happen. So don’t MPI_Wait on an operation to > complete when you haven’t called all of the pready calls (that the > process/thread is responsible for) from that process/thread yet (similar > to your responsibilities to make sure Send/Recv don’t deadlock, but > obviously a different semantic for completion). > > -Ryan > > *From: *James Dinan <james.dinan(a)gmail.com> > *Date: *Wednesday, October 12, 2022 at 6:43 AM > *To: *Joachim Jenke <protze(a)itc.rwth-aachen.de>, Ryan Grant > <ryan.grant(a)queensu.ca> > *Cc: *"mpiwg-persistence(a)lists.mpi-forum.org" > <mpiwg-persistence(a)lists.mpi-forum.org> > *Subject: *Re: [mpiwg-persistence] Completion call before all partitions > marked ready > > This came up recently and I believe it is allowed to wait/test a send > request before all partitions are marked as ready. It's not explicitly > stated, rather there is no restriction disallowing this usage. It may be > good to clarify this semantic (e.g. in Section 4.2.2) for MPI 4.1 since > this is a topic that the WG debated at length. > > @Ryan Grant <mailto:[email protected]> Please fact check. > > ~Jim. > > On Mon, Oct 10, 2022 at 7:22 AM Joachim Jenke via mpiwg-persistence > <mpiwg-persistence(a)lists.mpi-forum.org > <mailto:[email protected]>> wrote: > > Hello wg-persistence, > > Looking at the MPI 4.0 document, it is not clear to us, whether it is > allowed to call a completion call for a partitioned communication > request before all partitions are marked ready. A simple > single-threaded > example would be: > > ```C > MPI_Psend_init(message, partitions, COUNT, MPI_DOUBLE, dest, tag, > MPI_COMM_WORLD, MPI_INFO_NULL, &request); > MPI_Start(&request); > for(i = 0; i < partitions-1; ++i) > { > MPI_Pready(i, request); > } > MPI_Test(&request, &flag, MPI_STATUS_IGNORE); // flag will always be 0 > MPI_Pready(partitions-1, request); > MPI_Wait(&request, MPI_STATUS_IGNORE); > MPI_Request_free(&request); > ``` > > The question becomes more relevant in a multi-threaded context. One > thread could finish the work early and call MPI_Wait to detect when all > partitions were sent. > > From my understanding, the only requirement is that all partitions > must > be marked ready with explicit ready calls before the operation can > complete. Replacing the test in above example with a wait call would > result in deadlock. > > Best > Joachim > > -- > Dr. rer. nat. Joachim Jenke > > IT Center > Group: High Performance Computing > Division: Computational Science and Engineering > RWTH Aachen University > Seffenter Weg 23 > D 52074 Aachen (Germany) > Tel: +49 241 80- 24765 > Fax: +49 241 80-624765 > protze(a)itc.rwth-aachen.de <mailto:[email protected]> > www.itc.rwth-aachen.de > <https://can01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.itc.rw…> > _______________________________________________ > mpiwg-persistence mailing list > mpiwg-persistence(a)lists.mpi-forum.org > <mailto:[email protected]> > https://lists.mpi-forum.org/mailman/listinfo/mpiwg-persistence > <https://can01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.mpi…> > -- Dr. rer. nat. Joachim Jenke IT Center Group: High Performance Computing Division: Computational Science and Engineering RWTH Aachen University Seffenter Weg 23 D 52074 Aachen (Germany) Tel: +49 241 80- 24765 Fax: +49 241 80-624765 protze(a)itc.rwth-aachen.de www.itc.rwth-aachen.de

1 0

Re: [mpiwg-persistence] Completion call before all partitions marked ready
by Ryan Grant 12 Oct '22

12 Oct '22

This is correct, you’re allowed to call test/wait whenever you want as long as you abide by the rules that all pready calls must be made in order for completion to happen. So don’t MPI_Wait on an operation to complete when you haven’t called all of the pready calls (that the process/thread is responsible for) from that process/thread yet (similar to your responsibilities to make sure Send/Recv don’t deadlock, but obviously a different semantic for completion). -Ryan From: James Dinan <james.dinan(a)gmail.com> Date: Wednesday, October 12, 2022 at 6:43 AM To: Joachim Jenke <protze(a)itc.rwth-aachen.de>, Ryan Grant <ryan.grant(a)queensu.ca> Cc: "mpiwg-persistence(a)lists.mpi-forum.org" <mpiwg-persistence(a)lists.mpi-forum.org> Subject: Re: [mpiwg-persistence] Completion call before all partitions marked ready This came up recently and I believe it is allowed to wait/test a send request before all partitions are marked as ready. It's not explicitly stated, rather there is no restriction disallowing this usage. It may be good to clarify this semantic (e.g. in Section 4.2.2) for MPI 4.1 since this is a topic that the WG debated at length. @Ryan Grant<mailto:[email protected]> Please fact check. ~Jim. On Mon, Oct 10, 2022 at 7:22 AM Joachim Jenke via mpiwg-persistence <mpiwg-persistence(a)lists.mpi-forum.org<mailto:[email protected]>> wrote: Hello wg-persistence, Looking at the MPI 4.0 document, it is not clear to us, whether it is allowed to call a completion call for a partitioned communication request before all partitions are marked ready. A simple single-threaded example would be: ```C MPI_Psend_init(message, partitions, COUNT, MPI_DOUBLE, dest, tag, MPI_COMM_WORLD, MPI_INFO_NULL, &request); MPI_Start(&request); for(i = 0; i < partitions-1; ++i) { MPI_Pready(i, request); } MPI_Test(&request, &flag, MPI_STATUS_IGNORE); // flag will always be 0 MPI_Pready(partitions-1, request); MPI_Wait(&request, MPI_STATUS_IGNORE); MPI_Request_free(&request); ``` The question becomes more relevant in a multi-threaded context. One thread could finish the work early and call MPI_Wait to detect when all partitions were sent. From my understanding, the only requirement is that all partitions must be marked ready with explicit ready calls before the operation can complete. Replacing the test in above example with a wait call would result in deadlock. Best Joachim -- Dr. rer. nat. Joachim Jenke IT Center Group: High Performance Computing Division: Computational Science and Engineering RWTH Aachen University Seffenter Weg 23 D 52074 Aachen (Germany) Tel: +49 241 80- 24765 Fax: +49 241 80-624765 protze(a)itc.rwth-aachen.de<mailto:[email protected]> www.itc.rwth-aachen.de<https://can01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.itc.rw…> _______________________________________________ mpiwg-persistence mailing list mpiwg-persistence(a)lists.mpi-forum.org<mailto:[email protected]> https://lists.mpi-forum.org/mailman/listinfo/mpiwg-persistence<https://can01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.mpi…>

1 0

Re: [mpiwg-persistence] Completion call before all partitions marked ready
by Jim Dinan 12 Oct '22

12 Oct '22

This came up recently and I believe it is allowed to wait/test a send request before all partitions are marked as ready. It's not explicitly stated, rather there is no restriction disallowing this usage. It may be good to clarify this semantic (e.g. in Section 4.2.2) for MPI 4.1 since this is a topic that the WG debated at length. @Ryan Grant <ryan.grant(a)queensu.ca> Please fact check. ~Jim. On Mon, Oct 10, 2022 at 7:22 AM Joachim Jenke via mpiwg-persistence < mpiwg-persistence(a)lists.mpi-forum.org> wrote: > Hello wg-persistence, > > Looking at the MPI 4.0 document, it is not clear to us, whether it is > allowed to call a completion call for a partitioned communication > request before all partitions are marked ready. A simple single-threaded > example would be: > > ```C > MPI_Psend_init(message, partitions, COUNT, MPI_DOUBLE, dest, tag, > MPI_COMM_WORLD, MPI_INFO_NULL, &request); > MPI_Start(&request); > for(i = 0; i < partitions-1; ++i) > { > MPI_Pready(i, request); > } > MPI_Test(&request, &flag, MPI_STATUS_IGNORE); // flag will always be 0 > MPI_Pready(partitions-1, request); > MPI_Wait(&request, MPI_STATUS_IGNORE); > MPI_Request_free(&request); > ``` > > The question becomes more relevant in a multi-threaded context. One > thread could finish the work early and call MPI_Wait to detect when all > partitions were sent. > > From my understanding, the only requirement is that all partitions must > be marked ready with explicit ready calls before the operation can > complete. Replacing the test in above example with a wait call would > result in deadlock. > > Best > Joachim > > -- > Dr. rer. nat. Joachim Jenke > > IT Center > Group: High Performance Computing > Division: Computational Science and Engineering > RWTH Aachen University > Seffenter Weg 23 > D 52074 Aachen (Germany) > Tel: +49 241 80- 24765 > Fax: +49 241 80-624765 > protze(a)itc.rwth-aachen.de > www.itc.rwth-aachen.de > _______________________________________________ > mpiwg-persistence mailing list > mpiwg-persistence(a)lists.mpi-forum.org > https://lists.mpi-forum.org/mailman/listinfo/mpiwg-persistence >

1 0

Completion call before all partitions marked ready
by Joachim Jenke 10 Oct '22

10 Oct '22

Hello wg-persistence, Looking at the MPI 4.0 document, it is not clear to us, whether it is allowed to call a completion call for a partitioned communication request before all partitions are marked ready. A simple single-threaded example would be: ```C MPI_Psend_init(message, partitions, COUNT, MPI_DOUBLE, dest, tag, MPI_COMM_WORLD, MPI_INFO_NULL, &request); MPI_Start(&request); for(i = 0; i < partitions-1; ++i) { MPI_Pready(i, request); } MPI_Test(&request, &flag, MPI_STATUS_IGNORE); // flag will always be 0 MPI_Pready(partitions-1, request); MPI_Wait(&request, MPI_STATUS_IGNORE); MPI_Request_free(&request); ``` The question becomes more relevant in a multi-threaded context. One thread could finish the work early and call MPI_Wait to detect when all partitions were sent. From my understanding, the only requirement is that all partitions must be marked ready with explicit ready calls before the operation can complete. Replacing the test in above example with a wait call would result in deadlock. Best Joachim -- Dr. rer. nat. Joachim Jenke IT Center Group: High Performance Computing Division: Computational Science and Engineering RWTH Aachen University Seffenter Weg 23 D 52074 Aachen (Germany) Tel: +49 241 80- 24765 Fax: +49 241 80-624765 protze(a)itc.rwth-aachen.de www.itc.rwth-aachen.de

1 0