July 2014 - mpiwg-rma - mailman3.cels.anl.gov

Re: [mpiwg-rma] doodle poll for RMA working group telecon
by Rajeev Thakur 28 Jul '14

28 Jul '14

A reminder of this telecon on Wednesday. Please read ticket 437 and the other tickets mentioned in it. https://svn.mpi-forum.org/trac/mpi-forum-web/ticket/437 Rajeev On Jul 19, 2014, at 5:11 PM, Rajeev Thakur <thakur(a)mcs.anl.gov> wrote: > OK, let's have the telecon on Wed, July 30, 9.00am Central Time (7am Pacific, 4pm German time). > > We can use this telecon number: 1-567-314-8345, passcode: 262144 > > The agenda is ticket 437 and other tickets mentioned therein. > https://svn.mpi-forum.org/trac/mpi-forum-web/ticket/437 > > Rajeev > > > On Jul 18, 2014, at 2:26 PM, William Gropp <wgropp(a)illinois.edu> wrote: > >> Starting with the first available time, 9 CDT on July 30, is fine with me. >> Bill >> >> William Gropp >> Director, Parallel Computing Institute >> Thomas M. Siebel Chair in Computer Science >> University of Illinois Urbana-Champaign >> >> >> >> >> >> On Jul 18, 2014, at 10:45 AM, Rolf Rabenseifner wrote: >> >>> Dear all, >>> >>> Rajeev, thank you for starting the doodle. >>> I expect that you are still on vacation. >>> Therefore a short summary before the weekend starts (in Europe): >>> >>> As I can see at http://doodle.com/hx3p4cbwfcyszam6 >>> there are only three slots available where all can come together: >>> >>> Wednesday July 30, >>> - 09:00-10:00 USA Central time >>> - 16:00-17:00 Berlin timezone >>> >>> Wednesday July 30, >>> - 12:00-13:00 USA Central time >>> - 19:00-20:00 Berlin timezone >>> (this slot is also available, because my evening-date was removed) >>> >>> and Thursday July 31, >>> - 12:00-13:00 USA Central time >>> - 19:00-20:00 Berlin timezone >>> >>>> We need to discuss a few tickets over telecon. Let us start with a >>>> telecon for ticket 437 that Rolf has created: >>>> https://svn.mpi-forum.org/trac/mpi-forum-web/ticket/437. It is to >>>> clarify whether remote accesses to shared memory are to be >>>> considered similar to RMA operations or local loads/stores. A few >>>> related tickets are linked there. >>> >>> I would recommend that we start with the first slot and >>> we will see, whether we need to continue with the other slots. >>> Otherwise those other slots could be used for other RMA tickets. >>> >>> Okay? >>> >>> Participants on the doodle were: >>> - Rajeev Thakur >>> - Pavan Balaji >>> - Jeff Hammond >>> - Rolf Rabenseifner >>> - Bill Gropp >>> - Jim Dinan >>> - Hubert Ritzdorf >>> >>> I hope that other members of the RMA WG can also participate, >>> especially if you will go to the Japanese MPI forum meeting. >>> >>> Best regards >>> Rolf >>> >>> >>> ----- Original Message ----- >>>> From: "Rajeev Thakur" <thakur(a)mcs.anl.gov> >>>> To: "MPI WG Remote Memory Access working group" <mpiwg-rma(a)lists.mpi-forum.org> >>>> Sent: Monday, July 14, 2014 8:39:12 PM >>>> Subject: [mpiwg-rma] doodle poll for RMA working group telecon >>>> >>>> We need to discuss a few tickets over telecon. Let us start with a >>>> telecon for ticket 437 that Rolf has created: >>>> https://svn.mpi-forum.org/trac/mpi-forum-web/ticket/437. It is to >>>> clarify whether remote accesses to shared memory are to be >>>> considered similar to RMA operations or local loads/stores. A few >>>> related tickets are linked there. >>>> >>>> Please indicate your availability in the weeks of July 21 and 28 in >>>> the doodle poll at http://doodle.com/hx3p4cbwfcyszam6. I have listed >>>> only a few slots on each day to accommodate European and Pacific >>>> time zones. >>>> >>>> Thanks, >>>> Rajeev >>>> >>>> _______________________________________________ >>>> mpiwg-rma mailing list >>>> mpiwg-rma(a)lists.mpi-forum.org >>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma >>>> >>> >>> -- >>> Dr. Rolf Rabenseifner . . . . . . . . . .. email rabenseifner(a)hlrs.de >>> High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 >>> University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 >>> Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner >>> Nobelstr. 19, D-70550 Stuttgart, Germany . . . . (Office: Room 1.307) >>> _______________________________________________ >>> mpiwg-rma mailing list >>> mpiwg-rma(a)lists.mpi-forum.org >>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma >> >> _______________________________________________ >> mpiwg-rma mailing list >> mpiwg-rma(a)lists.mpi-forum.org >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma > > _______________________________________________ > mpiwg-rma mailing list > mpiwg-rma(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma

1 0

Re: [mpiwg-rma] doodle poll for RMA working group telecon
by Rajeev Thakur 19 Jul '14

19 Jul '14

OK, let's have the telecon on Wed, July 30, 9.00am Central Time (7am Pacific, 4pm German time). We can use this telecon number: 1-567-314-8345, passcode: 262144 The agenda is ticket 437 and other tickets mentioned therein. https://svn.mpi-forum.org/trac/mpi-forum-web/ticket/437 Rajeev On Jul 18, 2014, at 2:26 PM, William Gropp <wgropp(a)illinois.edu> wrote: > Starting with the first available time, 9 CDT on July 30, is fine with me. > Bill > > William Gropp > Director, Parallel Computing Institute > Thomas M. Siebel Chair in Computer Science > University of Illinois Urbana-Champaign > > > > > > On Jul 18, 2014, at 10:45 AM, Rolf Rabenseifner wrote: > >> Dear all, >> >> Rajeev, thank you for starting the doodle. >> I expect that you are still on vacation. >> Therefore a short summary before the weekend starts (in Europe): >> >> As I can see at http://doodle.com/hx3p4cbwfcyszam6 >> there are only three slots available where all can come together: >> >> Wednesday July 30, >> - 09:00-10:00 USA Central time >> - 16:00-17:00 Berlin timezone >> >> Wednesday July 30, >> - 12:00-13:00 USA Central time >> - 19:00-20:00 Berlin timezone >> (this slot is also available, because my evening-date was removed) >> >> and Thursday July 31, >> - 12:00-13:00 USA Central time >> - 19:00-20:00 Berlin timezone >> >>> We need to discuss a few tickets over telecon. Let us start with a >>> telecon for ticket 437 that Rolf has created: >>> https://svn.mpi-forum.org/trac/mpi-forum-web/ticket/437. It is to >>> clarify whether remote accesses to shared memory are to be >>> considered similar to RMA operations or local loads/stores. A few >>> related tickets are linked there. >> >> I would recommend that we start with the first slot and >> we will see, whether we need to continue with the other slots. >> Otherwise those other slots could be used for other RMA tickets. >> >> Okay? >> >> Participants on the doodle were: >> - Rajeev Thakur >> - Pavan Balaji >> - Jeff Hammond >> - Rolf Rabenseifner >> - Bill Gropp >> - Jim Dinan >> - Hubert Ritzdorf >> >> I hope that other members of the RMA WG can also participate, >> especially if you will go to the Japanese MPI forum meeting. >> >> Best regards >> Rolf >> >> >> ----- Original Message ----- >>> From: "Rajeev Thakur" <thakur(a)mcs.anl.gov> >>> To: "MPI WG Remote Memory Access working group" <mpiwg-rma(a)lists.mpi-forum.org> >>> Sent: Monday, July 14, 2014 8:39:12 PM >>> Subject: [mpiwg-rma] doodle poll for RMA working group telecon >>> >>> We need to discuss a few tickets over telecon. Let us start with a >>> telecon for ticket 437 that Rolf has created: >>> https://svn.mpi-forum.org/trac/mpi-forum-web/ticket/437. It is to >>> clarify whether remote accesses to shared memory are to be >>> considered similar to RMA operations or local loads/stores. A few >>> related tickets are linked there. >>> >>> Please indicate your availability in the weeks of July 21 and 28 in >>> the doodle poll at http://doodle.com/hx3p4cbwfcyszam6. I have listed >>> only a few slots on each day to accommodate European and Pacific >>> time zones. >>> >>> Thanks, >>> Rajeev >>> >>> _______________________________________________ >>> mpiwg-rma mailing list >>> mpiwg-rma(a)lists.mpi-forum.org >>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma >>> >> >> -- >> Dr. Rolf Rabenseifner . . . . . . . . . .. email rabenseifner(a)hlrs.de >> High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 >> University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 >> Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner >> Nobelstr. 19, D-70550 Stuttgart, Germany . . . . (Office: Room 1.307) >> _______________________________________________ >> mpiwg-rma mailing list >> mpiwg-rma(a)lists.mpi-forum.org >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma > > _______________________________________________ > mpiwg-rma mailing list > mpiwg-rma(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma

1 0

Re: [mpiwg-rma] doodle poll for RMA working group telecon
by Jeff Hammond 18 Jul '14

18 Jul '14

Although I'm amazed that I agreed to a 7 am telecon - probably bad offset calculation when I was filling this out in DC - that's a perfect time from the standpoint of not having any last-minute conflicts due to Intel west coast activities :-) Jeff Sent from my iPhone > On Jul 18, 2014, at 12:26 PM, William Gropp <wgropp(a)illinois.edu> wrote: > > Starting with the first available time, 9 CDT on July 30, is fine with me. > Bill > > William Gropp > Director, Parallel Computing Institute > Thomas M. Siebel Chair in Computer Science > University of Illinois Urbana-Champaign > > > > > >> On Jul 18, 2014, at 10:45 AM, Rolf Rabenseifner wrote: >> >> Dear all, >> >> Rajeev, thank you for starting the doodle. >> I expect that you are still on vacation. >> Therefore a short summary before the weekend starts (in Europe): >> >> As I can see at http://doodle.com/hx3p4cbwfcyszam6 >> there are only three slots available where all can come together: >> >> Wednesday July 30, >> - 09:00-10:00 USA Central time >> - 16:00-17:00 Berlin timezone >> >> Wednesday July 30, >> - 12:00-13:00 USA Central time >> - 19:00-20:00 Berlin timezone >> (this slot is also available, because my evening-date was removed) >> >> and Thursday July 31, >> - 12:00-13:00 USA Central time >> - 19:00-20:00 Berlin timezone >> >>> We need to discuss a few tickets over telecon. Let us start with a >>> telecon for ticket 437 that Rolf has created: >>> https://svn.mpi-forum.org/trac/mpi-forum-web/ticket/437. It is to >>> clarify whether remote accesses to shared memory are to be >>> considered similar to RMA operations or local loads/stores. A few >>> related tickets are linked there. >> >> I would recommend that we start with the first slot and >> we will see, whether we need to continue with the other slots. >> Otherwise those other slots could be used for other RMA tickets. >> >> Okay? >> >> Participants on the doodle were: >> - Rajeev Thakur >> - Pavan Balaji >> - Jeff Hammond >> - Rolf Rabenseifner >> - Bill Gropp >> - Jim Dinan >> - Hubert Ritzdorf >> >> I hope that other members of the RMA WG can also participate, >> especially if you will go to the Japanese MPI forum meeting. >> >> Best regards >> Rolf >> >> >> ----- Original Message ----- >>> From: "Rajeev Thakur" <thakur(a)mcs.anl.gov> >>> To: "MPI WG Remote Memory Access working group" <mpiwg-rma(a)lists.mpi-forum.org> >>> Sent: Monday, July 14, 2014 8:39:12 PM >>> Subject: [mpiwg-rma] doodle poll for RMA working group telecon >>> >>> We need to discuss a few tickets over telecon. Let us start with a >>> telecon for ticket 437 that Rolf has created: >>> https://svn.mpi-forum.org/trac/mpi-forum-web/ticket/437. It is to >>> clarify whether remote accesses to shared memory are to be >>> considered similar to RMA operations or local loads/stores. A few >>> related tickets are linked there. >>> >>> Please indicate your availability in the weeks of July 21 and 28 in >>> the doodle poll at http://doodle.com/hx3p4cbwfcyszam6. I have listed >>> only a few slots on each day to accommodate European and Pacific >>> time zones. >>> >>> Thanks, >>> Rajeev >>> >>> _______________________________________________ >>> mpiwg-rma mailing list >>> mpiwg-rma(a)lists.mpi-forum.org >>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma >> >> -- >> Dr. Rolf Rabenseifner . . . . . . . . . .. email rabenseifner(a)hlrs.de >> High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 >> University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 >> Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner >> Nobelstr. 19, D-70550 Stuttgart, Germany . . . . (Office: Room 1.307) >> _______________________________________________ >> mpiwg-rma mailing list >> mpiwg-rma(a)lists.mpi-forum.org >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma > > _______________________________________________ > mpiwg-rma mailing list > mpiwg-rma(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma

1 0

Re: [mpiwg-rma] doodle poll for RMA working group telecon
by William Gropp 18 Jul '14

18 Jul '14

Starting with the first available time, 9 CDT on July 30, is fine with me. Bill William Gropp Director, Parallel Computing Institute Thomas M. Siebel Chair in Computer Science University of Illinois Urbana-Champaign On Jul 18, 2014, at 10:45 AM, Rolf Rabenseifner wrote: > Dear all, > > Rajeev, thank you for starting the doodle. > I expect that you are still on vacation. > Therefore a short summary before the weekend starts (in Europe): > > As I can see at http://doodle.com/hx3p4cbwfcyszam6 > there are only three slots available where all can come together: > > Wednesday July 30, > - 09:00-10:00 USA Central time > - 16:00-17:00 Berlin timezone > > Wednesday July 30, > - 12:00-13:00 USA Central time > - 19:00-20:00 Berlin timezone > (this slot is also available, because my evening-date was removed) > > and Thursday July 31, > - 12:00-13:00 USA Central time > - 19:00-20:00 Berlin timezone > >> We need to discuss a few tickets over telecon. Let us start with a >> telecon for ticket 437 that Rolf has created: >> https://svn.mpi-forum.org/trac/mpi-forum-web/ticket/437. It is to >> clarify whether remote accesses to shared memory are to be >> considered similar to RMA operations or local loads/stores. A few >> related tickets are linked there. > > I would recommend that we start with the first slot and > we will see, whether we need to continue with the other slots. > Otherwise those other slots could be used for other RMA tickets. > > Okay? > > Participants on the doodle were: > - Rajeev Thakur > - Pavan Balaji > - Jeff Hammond > - Rolf Rabenseifner > - Bill Gropp > - Jim Dinan > - Hubert Ritzdorf > > I hope that other members of the RMA WG can also participate, > especially if you will go to the Japanese MPI forum meeting. > > Best regards > Rolf > > > ----- Original Message ----- >> From: "Rajeev Thakur" <thakur(a)mcs.anl.gov> >> To: "MPI WG Remote Memory Access working group" <mpiwg-rma(a)lists.mpi-forum.org> >> Sent: Monday, July 14, 2014 8:39:12 PM >> Subject: [mpiwg-rma] doodle poll for RMA working group telecon >> >> We need to discuss a few tickets over telecon. Let us start with a >> telecon for ticket 437 that Rolf has created: >> https://svn.mpi-forum.org/trac/mpi-forum-web/ticket/437. It is to >> clarify whether remote accesses to shared memory are to be >> considered similar to RMA operations or local loads/stores. A few >> related tickets are linked there. >> >> Please indicate your availability in the weeks of July 21 and 28 in >> the doodle poll at http://doodle.com/hx3p4cbwfcyszam6. I have listed >> only a few slots on each day to accommodate European and Pacific >> time zones. >> >> Thanks, >> Rajeev >> >> _______________________________________________ >> mpiwg-rma mailing list >> mpiwg-rma(a)lists.mpi-forum.org >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma >> > > -- > Dr. Rolf Rabenseifner . . . . . . . . . .. email rabenseifner(a)hlrs.de > High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 > University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 > Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner > Nobelstr. 19, D-70550 Stuttgart, Germany . . . . (Office: Room 1.307) > _______________________________________________ > mpiwg-rma mailing list > mpiwg-rma(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma

1 0

Re: [mpiwg-rma] doodle poll for RMA working group telecon
by Rolf Rabenseifner 18 Jul '14

18 Jul '14

Dear all, Rajeev, thank you for starting the doodle. I expect that you are still on vacation. Therefore a short summary before the weekend starts (in Europe): As I can see at http://doodle.com/hx3p4cbwfcyszam6 there are only three slots available where all can come together: Wednesday July 30, - 09:00-10:00 USA Central time - 16:00-17:00 Berlin timezone Wednesday July 30, - 12:00-13:00 USA Central time - 19:00-20:00 Berlin timezone (this slot is also available, because my evening-date was removed) and Thursday July 31, - 12:00-13:00 USA Central time - 19:00-20:00 Berlin timezone > We need to discuss a few tickets over telecon. Let us start with a > telecon for ticket 437 that Rolf has created: > https://svn.mpi-forum.org/trac/mpi-forum-web/ticket/437. It is to > clarify whether remote accesses to shared memory are to be > considered similar to RMA operations or local loads/stores. A few > related tickets are linked there. I would recommend that we start with the first slot and we will see, whether we need to continue with the other slots. Otherwise those other slots could be used for other RMA tickets. Okay? Participants on the doodle were: - Rajeev Thakur - Pavan Balaji - Jeff Hammond - Rolf Rabenseifner - Bill Gropp - Jim Dinan - Hubert Ritzdorf I hope that other members of the RMA WG can also participate, especially if you will go to the Japanese MPI forum meeting. Best regards Rolf ----- Original Message ----- > From: "Rajeev Thakur" <thakur(a)mcs.anl.gov> > To: "MPI WG Remote Memory Access working group" <mpiwg-rma(a)lists.mpi-forum.org> > Sent: Monday, July 14, 2014 8:39:12 PM > Subject: [mpiwg-rma] doodle poll for RMA working group telecon > > We need to discuss a few tickets over telecon. Let us start with a > telecon for ticket 437 that Rolf has created: > https://svn.mpi-forum.org/trac/mpi-forum-web/ticket/437. It is to > clarify whether remote accesses to shared memory are to be > considered similar to RMA operations or local loads/stores. A few > related tickets are linked there. > > Please indicate your availability in the weeks of July 21 and 28 in > the doodle poll at http://doodle.com/hx3p4cbwfcyszam6. I have listed > only a few slots on each day to accommodate European and Pacific > time zones. > > Thanks, > Rajeev > > _______________________________________________ > mpiwg-rma mailing list > mpiwg-rma(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma > -- Dr. Rolf Rabenseifner . . . . . . . . . .. email rabenseifner(a)hlrs.de High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner Nobelstr. 19, D-70550 Stuttgart, Germany . . . . (Office: Room 1.307)

1 0

Re: [mpiwg-rma] Memory barriers in passive-target RMA
by Thakur, Rajeev 15 Jul '14

15 Jul '14

The next telecon is for ticket 437 and other tickets mentioned therein, which should include this topic. https://svn.mpi-forum.org/trac/mpi-forum-web/ticket/437 Rajeev On Jul 15, 2014, at 8:26 AM, Balaji, Pavan <balaji(a)anl.gov> wrote: > > I think this item is already on the agenda for discussion for the next telecon? If not, can we add it? > > Thanks, > > — Pavan > > On Jul 15, 2014, at 8:19 AM, Balaji, Pavan <balaji(a)anl.gov> wrote: > >> >> We keep going back to the same issue every time. We had this discussion a number of times, so many that we really need to clarify it in the standard. >> >> The UNIFIED window should not claim that the public and private windows are identical. It should say that they are separate, but will eventually become the same without additional RMA calls. It should also say that if you want to make them consistent immediately, you should use MPI_WIN_SYNC. >> >> This, of course, doesn’t help with shared memory, since two processes might be working on their own private copies of the same window, and both need to be made consistent. But we could add some wording for that as well. >> >> I strongly suggest we clean up the standard in this respect. >> >> — Pavan >> >> On Jul 15, 2014, at 7:51 AM, Thakur, Rajeev <thakur(a)mcs.anl.gov> wrote: >> >>> The unified memory model is supposed to be covered by the word “eventually” in the standard. >>> >>> pg 436, ln 37-41: "In the RMA unified model, public and private copies are identical and updates via put or accumulate calls are eventually observed by load operations without additional RMA calls. A store access to a window is *eventually* visible to remote get or accumulate calls without additional RMA calls. These stronger semantics of the RMA unified model allow the user to omit some synchronization calls and potentially improve performance.” >>> >>> pg 455, ln 16-21: "An update by a put or accumulate call to a public window copy becomes visible in the private copy in process memory at latest when an ensuing call to MPI_WIN_WAIT, MPI_WIN_FENCE, MPI_WIN_LOCK, MPI_WIN_LOCK_ALL, or MPI_WIN_SYNC is executed on that window by the window owner. In the RMA unified memory model, an update by a put or accumulate call to a public window copy *eventually* becomes visible in the private copy in process memory without additional RMA calls." >>> >>> Rajeev >>> >>> >>> On Jul 15, 2014, at 6:31 AM, Balaji, Pavan <balaji(a)anl.gov> wrote: >>> >>>> >>>> I should have clarified in my original email — I’m talking about the UNIFIED memory model. >>>> >>>> — Pavan >>>> >>>> On Jul 15, 2014, at 12:14 AM, Thakur, Rajeev <thakur(a)mcs.anl.gov> wrote: >>>> >>>>> The equivalent text in the standard is on pg 454, ln 25-29. >>>>> >>>>> "If a put or accumulate access was synchronized with a lock, then the update of the public window copy is complete as soon as the updating process executed MPI_WIN_UNLOCK or MPI_WIN_UNLOCK_ALL. In the RMA separate memory model, the update of a private copy in the process memory may be delayed until the target process executes a synchronization call on that window (6).” >>>>> >>>>> Rajeev >>>>> >>>>> On Jul 14, 2014, at 1:45 PM, Balaji, Pavan <balaji(a)anl.gov> wrote: >>>>> >>>>>> >>>>>> Ah, you meant the caption, not the example itself. Some equivalent text should go into the standard, since the example is non-binding. >>>>>> >>>>>> — Pavan >>>>>> >>>>>> On Jul 14, 2014, at 1:41 PM, Thakur, Rajeev <thakur(a)mcs.anl.gov> wrote: >>>>>> >>>>>>> On lines 40-42 it explains why the lock-unlock on process B is needed: >>>>>>> >>>>>>> "Although the MPI_WIN_UNLOCK on process A and the MPI_BARRIER ensure that the public copy on process B reflects the updated value of X, the call to MPI_WIN_LOCK by process B is necessary to synchronize the private copy with the public copy.” >>>>>>> >>>>>>> Rajeev >>>>>>> >>>>>>> On Jul 14, 2014, at 1:36 PM, Balaji, Pavan <balaji(a)anl.gov> wrote: >>>>>>> >>>>>>>> >>>>>>>> I don’t understand your point. The example uses lock/unlock for local access, but that’s not a confirmation that it’s required. It just means that you are allowed to use lock/unlock. >>>>>>>> >>>>>>>> — Pavan >>>>>>>> >>>>>>>> P.S.: Adding a unicode character to try to trick mac to use UTF-8. Let’s see if this will work. What the heck, apple? ✉️ >>>>>>>> >>>>>>>> On Jul 14, 2014, at 1:30 PM, Thakur, Rajeev <thakur(a)mcs.anl.gov> wrote: >>>>>>>> >>>>>>>>> See example 11.8 on pg 458. >>>>>>>>> >>>>>>>>> Rajeev >>>>>>>>> >>>>>>>>> On Jul 14, 2014, at 1:15 PM, Balaji, Pavan <balaji(a)anl.gov> wrote: >>>>>>>>> >>>>>>>>>> >>>>>>>>>> The MPI standard seems to allow access to the local window without requiring a lock (at least, I couldn’t find any text requiring a lock). Does this mean that the following example is correct? >>>>>>>>>> >>>>>>>>>> P0: >>>>>>>>>> Win_lock(P1) >>>>>>>>>> Put(X, 1) >>>>>>>>>> Win_unlock(P1) >>>>>>>>>> MPI_Send(P1) >>>>>>>>>> >>>>>>>>>> P1: >>>>>>>>>> MPI_Recv(P0) >>>>>>>>>> assert(X == 1) >>>>>>>>>> >>>>>>>>>> If the above is correct, shouldn’t there be a memory read barrier on P1 somewhere? Since P1 is not making any RMA calls, I’d assume that’ll need to somehow come from the lock and unlock operations. That is, the MPI implementation will need to do an active message in Win_lock and Win_unlock forcing a memory barrier at the target. Assuming that’s correct, I'll have to send out a lock packet even if the user gave the MPI_MODE_NOCHECK hint, for memory consistency reasons. That sounds awful, so I’m really hoping that I missed something in the standard which will say I don’t need to do all this. >>>>>>>>>> >>>>>>>>>> Note that all this active message problem will go away if I need P1 to do a lock/unlock to itself in order to access X. >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> >>>>>>>>>> — Pavan >>>>>>>>>> >>>>>>>>>> _______________________________________________ >>>>>>>>>> mpiwg-rma mailing list >>>>>>>>>> mpiwg-rma(a)lists.mpi-forum.org >>>>>>>>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma >>>>>>>>> >>>>>>>>> _______________________________________________ >>>>>>>>> mpiwg-rma mailing list >>>>>>>>> mpiwg-rma(a)lists.mpi-forum.org >>>>>>>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> mpiwg-rma mailing list >>>>>>>> mpiwg-rma(a)lists.mpi-forum.org >>>>>>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma >>>>>>> >>>>>>> _______________________________________________ >>>>>>> mpiwg-rma mailing list >>>>>>> mpiwg-rma(a)lists.mpi-forum.org >>>>>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma >>>>>> >>>>>> -- >>>>>> Pavan Balaji >>>>>> http://www.mcs.anl.gov/~balaji >>>>>> >>>>>> Unicode character to trick mac into using UTF-8 (✉️). >>>>>> _______________________________________________ >>>>>> mpiwg-rma mailing list >>>>>> mpiwg-rma(a)lists.mpi-forum.org >>>>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma >>>>> >>>>> _______________________________________________ >>>>> mpiwg-rma mailing list >>>>> mpiwg-rma(a)lists.mpi-forum.org >>>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma >>>> >>>> -- >>>> Pavan Balaji ✉️ >>>> http://www.mcs.anl.gov/~balaji >>>> >>>> _______________________________________________ >>>> mpiwg-rma mailing list >>>> mpiwg-rma(a)lists.mpi-forum.org >>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma >>> >>> _______________________________________________ >>> mpiwg-rma mailing list >>> mpiwg-rma(a)lists.mpi-forum.org >>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma >> >> -- >> Pavan Balaji ✉️ >> http://www.mcs.anl.gov/~balaji >> >> _______________________________________________ >> mpiwg-rma mailing list >> mpiwg-rma(a)lists.mpi-forum.org >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma > > -- > Pavan Balaji ✉️ > http://www.mcs.anl.gov/~balaji > > _______________________________________________ > mpiwg-rma mailing list > mpiwg-rma(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma

1 0

Re: [mpiwg-rma] Memory barriers in passive-target RMA
by Balaji, Pavan 15 Jul '14

15 Jul '14

I think this item is already on the agenda for discussion for the next telecon? If not, can we add it? Thanks, — Pavan On Jul 15, 2014, at 8:19 AM, Balaji, Pavan <balaji(a)anl.gov> wrote: > > We keep going back to the same issue every time. We had this discussion a number of times, so many that we really need to clarify it in the standard. > > The UNIFIED window should not claim that the public and private windows are identical. It should say that they are separate, but will eventually become the same without additional RMA calls. It should also say that if you want to make them consistent immediately, you should use MPI_WIN_SYNC. > > This, of course, doesn’t help with shared memory, since two processes might be working on their own private copies of the same window, and both need to be made consistent. But we could add some wording for that as well. > > I strongly suggest we clean up the standard in this respect. > > — Pavan > > On Jul 15, 2014, at 7:51 AM, Thakur, Rajeev <thakur(a)mcs.anl.gov> wrote: > >> The unified memory model is supposed to be covered by the word “eventually” in the standard. >> >> pg 436, ln 37-41: "In the RMA unified model, public and private copies are identical and updates via put or accumulate calls are eventually observed by load operations without additional RMA calls. A store access to a window is *eventually* visible to remote get or accumulate calls without additional RMA calls. These stronger semantics of the RMA unified model allow the user to omit some synchronization calls and potentially improve performance.” >> >> pg 455, ln 16-21: "An update by a put or accumulate call to a public window copy becomes visible in the private copy in process memory at latest when an ensuing call to MPI_WIN_WAIT, MPI_WIN_FENCE, MPI_WIN_LOCK, MPI_WIN_LOCK_ALL, or MPI_WIN_SYNC is executed on that window by the window owner. In the RMA unified memory model, an update by a put or accumulate call to a public window copy *eventually* becomes visible in the private copy in process memory without additional RMA calls." >> >> Rajeev >> >> >> On Jul 15, 2014, at 6:31 AM, Balaji, Pavan <balaji(a)anl.gov> wrote: >> >>> >>> I should have clarified in my original email — I’m talking about the UNIFIED memory model. >>> >>> — Pavan >>> >>> On Jul 15, 2014, at 12:14 AM, Thakur, Rajeev <thakur(a)mcs.anl.gov> wrote: >>> >>>> The equivalent text in the standard is on pg 454, ln 25-29. >>>> >>>> "If a put or accumulate access was synchronized with a lock, then the update of the public window copy is complete as soon as the updating process executed MPI_WIN_UNLOCK or MPI_WIN_UNLOCK_ALL. In the RMA separate memory model, the update of a private copy in the process memory may be delayed until the target process executes a synchronization call on that window (6).” >>>> >>>> Rajeev >>>> >>>> On Jul 14, 2014, at 1:45 PM, Balaji, Pavan <balaji(a)anl.gov> wrote: >>>> >>>>> >>>>> Ah, you meant the caption, not the example itself. Some equivalent text should go into the standard, since the example is non-binding. >>>>> >>>>> — Pavan >>>>> >>>>> On Jul 14, 2014, at 1:41 PM, Thakur, Rajeev <thakur(a)mcs.anl.gov> wrote: >>>>> >>>>>> On lines 40-42 it explains why the lock-unlock on process B is needed: >>>>>> >>>>>> "Although the MPI_WIN_UNLOCK on process A and the MPI_BARRIER ensure that the public copy on process B reflects the updated value of X, the call to MPI_WIN_LOCK by process B is necessary to synchronize the private copy with the public copy.” >>>>>> >>>>>> Rajeev >>>>>> >>>>>> On Jul 14, 2014, at 1:36 PM, Balaji, Pavan <balaji(a)anl.gov> wrote: >>>>>> >>>>>>> >>>>>>> I don’t understand your point. The example uses lock/unlock for local access, but that’s not a confirmation that it’s required. It just means that you are allowed to use lock/unlock. >>>>>>> >>>>>>> — Pavan >>>>>>> >>>>>>> P.S.: Adding a unicode character to try to trick mac to use UTF-8. Let’s see if this will work. What the heck, apple? ✉️ >>>>>>> >>>>>>> On Jul 14, 2014, at 1:30 PM, Thakur, Rajeev <thakur(a)mcs.anl.gov> wrote: >>>>>>> >>>>>>>> See example 11.8 on pg 458. >>>>>>>> >>>>>>>> Rajeev >>>>>>>> >>>>>>>> On Jul 14, 2014, at 1:15 PM, Balaji, Pavan <balaji(a)anl.gov> wrote: >>>>>>>> >>>>>>>>> >>>>>>>>> The MPI standard seems to allow access to the local window without requiring a lock (at least, I couldn’t find any text requiring a lock). Does this mean that the following example is correct? >>>>>>>>> >>>>>>>>> P0: >>>>>>>>> Win_lock(P1) >>>>>>>>> Put(X, 1) >>>>>>>>> Win_unlock(P1) >>>>>>>>> MPI_Send(P1) >>>>>>>>> >>>>>>>>> P1: >>>>>>>>> MPI_Recv(P0) >>>>>>>>> assert(X == 1) >>>>>>>>> >>>>>>>>> If the above is correct, shouldn’t there be a memory read barrier on P1 somewhere? Since P1 is not making any RMA calls, I’d assume that’ll need to somehow come from the lock and unlock operations. That is, the MPI implementation will need to do an active message in Win_lock and Win_unlock forcing a memory barrier at the target. Assuming that’s correct, I'll have to send out a lock packet even if the user gave the MPI_MODE_NOCHECK hint, for memory consistency reasons. That sounds awful, so I’m really hoping that I missed something in the standard which will say I don’t need to do all this. >>>>>>>>> >>>>>>>>> Note that all this active message problem will go away if I need P1 to do a lock/unlock to itself in order to access X. >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> >>>>>>>>> — Pavan >>>>>>>>> >>>>>>>>> _______________________________________________ >>>>>>>>> mpiwg-rma mailing list >>>>>>>>> mpiwg-rma(a)lists.mpi-forum.org >>>>>>>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> mpiwg-rma mailing list >>>>>>>> mpiwg-rma(a)lists.mpi-forum.org >>>>>>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma >>>>>>> >>>>>>> _______________________________________________ >>>>>>> mpiwg-rma mailing list >>>>>>> mpiwg-rma(a)lists.mpi-forum.org >>>>>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma >>>>>> >>>>>> _______________________________________________ >>>>>> mpiwg-rma mailing list >>>>>> mpiwg-rma(a)lists.mpi-forum.org >>>>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma >>>>> >>>>> -- >>>>> Pavan Balaji >>>>> http://www.mcs.anl.gov/~balaji >>>>> >>>>> Unicode character to trick mac into using UTF-8 (✉️). >>>>> _______________________________________________ >>>>> mpiwg-rma mailing list >>>>> mpiwg-rma(a)lists.mpi-forum.org >>>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma >>>> >>>> _______________________________________________ >>>> mpiwg-rma mailing list >>>> mpiwg-rma(a)lists.mpi-forum.org >>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma >>> >>> -- >>> Pavan Balaji ✉️ >>> http://www.mcs.anl.gov/~balaji >>> >>> _______________________________________________ >>> mpiwg-rma mailing list >>> mpiwg-rma(a)lists.mpi-forum.org >>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma >> >> _______________________________________________ >> mpiwg-rma mailing list >> mpiwg-rma(a)lists.mpi-forum.org >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma > > -- > Pavan Balaji ✉️ > http://www.mcs.anl.gov/~balaji > > _______________________________________________ > mpiwg-rma mailing list > mpiwg-rma(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma -- Pavan Balaji ✉️ http://www.mcs.anl.gov/~balaji

1 0

Re: [mpiwg-rma] Memory barriers in passive-target RMA
by Balaji, Pavan 15 Jul '14

15 Jul '14

We keep going back to the same issue every time. We had this discussion a number of times, so many that we really need to clarify it in the standard. The UNIFIED window should not claim that the public and private windows are identical. It should say that they are separate, but will eventually become the same without additional RMA calls. It should also say that if you want to make them consistent immediately, you should use MPI_WIN_SYNC. This, of course, doesn’t help with shared memory, since two processes might be working on their own private copies of the same window, and both need to be made consistent. But we could add some wording for that as well. I strongly suggest we clean up the standard in this respect. — Pavan On Jul 15, 2014, at 7:51 AM, Thakur, Rajeev <thakur(a)mcs.anl.gov> wrote: > The unified memory model is supposed to be covered by the word “eventually” in the standard. > > pg 436, ln 37-41: "In the RMA unified model, public and private copies are identical and updates via put or accumulate calls are eventually observed by load operations without additional RMA calls. A store access to a window is *eventually* visible to remote get or accumulate calls without additional RMA calls. These stronger semantics of the RMA unified model allow the user to omit some synchronization calls and potentially improve performance.” > > pg 455, ln 16-21: "An update by a put or accumulate call to a public window copy becomes visible in the private copy in process memory at latest when an ensuing call to MPI_WIN_WAIT, MPI_WIN_FENCE, MPI_WIN_LOCK, MPI_WIN_LOCK_ALL, or MPI_WIN_SYNC is executed on that window by the window owner. In the RMA unified memory model, an update by a put or accumulate call to a public window copy *eventually* becomes visible in the private copy in process memory without additional RMA calls." > > Rajeev > > > On Jul 15, 2014, at 6:31 AM, Balaji, Pavan <balaji(a)anl.gov> wrote: > >> >> I should have clarified in my original email — I’m talking about the UNIFIED memory model. >> >> — Pavan >> >> On Jul 15, 2014, at 12:14 AM, Thakur, Rajeev <thakur(a)mcs.anl.gov> wrote: >> >>> The equivalent text in the standard is on pg 454, ln 25-29. >>> >>> "If a put or accumulate access was synchronized with a lock, then the update of the public window copy is complete as soon as the updating process executed MPI_WIN_UNLOCK or MPI_WIN_UNLOCK_ALL. In the RMA separate memory model, the update of a private copy in the process memory may be delayed until the target process executes a synchronization call on that window (6).” >>> >>> Rajeev >>> >>> On Jul 14, 2014, at 1:45 PM, Balaji, Pavan <balaji(a)anl.gov> wrote: >>> >>>> >>>> Ah, you meant the caption, not the example itself. Some equivalent text should go into the standard, since the example is non-binding. >>>> >>>> — Pavan >>>> >>>> On Jul 14, 2014, at 1:41 PM, Thakur, Rajeev <thakur(a)mcs.anl.gov> wrote: >>>> >>>>> On lines 40-42 it explains why the lock-unlock on process B is needed: >>>>> >>>>> "Although the MPI_WIN_UNLOCK on process A and the MPI_BARRIER ensure that the public copy on process B reflects the updated value of X, the call to MPI_WIN_LOCK by process B is necessary to synchronize the private copy with the public copy.” >>>>> >>>>> Rajeev >>>>> >>>>> On Jul 14, 2014, at 1:36 PM, Balaji, Pavan <balaji(a)anl.gov> wrote: >>>>> >>>>>> >>>>>> I don’t understand your point. The example uses lock/unlock for local access, but that’s not a confirmation that it’s required. It just means that you are allowed to use lock/unlock. >>>>>> >>>>>> — Pavan >>>>>> >>>>>> P.S.: Adding a unicode character to try to trick mac to use UTF-8. Let’s see if this will work. What the heck, apple? ✉️ >>>>>> >>>>>> On Jul 14, 2014, at 1:30 PM, Thakur, Rajeev <thakur(a)mcs.anl.gov> wrote: >>>>>> >>>>>>> See example 11.8 on pg 458. >>>>>>> >>>>>>> Rajeev >>>>>>> >>>>>>> On Jul 14, 2014, at 1:15 PM, Balaji, Pavan <balaji(a)anl.gov> wrote: >>>>>>> >>>>>>>> >>>>>>>> The MPI standard seems to allow access to the local window without requiring a lock (at least, I couldn’t find any text requiring a lock). Does this mean that the following example is correct? >>>>>>>> >>>>>>>> P0: >>>>>>>> Win_lock(P1) >>>>>>>> Put(X, 1) >>>>>>>> Win_unlock(P1) >>>>>>>> MPI_Send(P1) >>>>>>>> >>>>>>>> P1: >>>>>>>> MPI_Recv(P0) >>>>>>>> assert(X == 1) >>>>>>>> >>>>>>>> If the above is correct, shouldn’t there be a memory read barrier on P1 somewhere? Since P1 is not making any RMA calls, I’d assume that’ll need to somehow come from the lock and unlock operations. That is, the MPI implementation will need to do an active message in Win_lock and Win_unlock forcing a memory barrier at the target. Assuming that’s correct, I'll have to send out a lock packet even if the user gave the MPI_MODE_NOCHECK hint, for memory consistency reasons. That sounds awful, so I’m really hoping that I missed something in the standard which will say I don’t need to do all this. >>>>>>>> >>>>>>>> Note that all this active message problem will go away if I need P1 to do a lock/unlock to itself in order to access X. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> >>>>>>>> — Pavan >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> mpiwg-rma mailing list >>>>>>>> mpiwg-rma(a)lists.mpi-forum.org >>>>>>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma >>>>>>> >>>>>>> _______________________________________________ >>>>>>> mpiwg-rma mailing list >>>>>>> mpiwg-rma(a)lists.mpi-forum.org >>>>>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma >>>>>> >>>>>> _______________________________________________ >>>>>> mpiwg-rma mailing list >>>>>> mpiwg-rma(a)lists.mpi-forum.org >>>>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma >>>>> >>>>> _______________________________________________ >>>>> mpiwg-rma mailing list >>>>> mpiwg-rma(a)lists.mpi-forum.org >>>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma >>>> >>>> -- >>>> Pavan Balaji >>>> http://www.mcs.anl.gov/~balaji >>>> >>>> Unicode character to trick mac into using UTF-8 (✉️). >>>> _______________________________________________ >>>> mpiwg-rma mailing list >>>> mpiwg-rma(a)lists.mpi-forum.org >>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma >>> >>> _______________________________________________ >>> mpiwg-rma mailing list >>> mpiwg-rma(a)lists.mpi-forum.org >>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma >> >> -- >> Pavan Balaji ✉️ >> http://www.mcs.anl.gov/~balaji >> >> _______________________________________________ >> mpiwg-rma mailing list >> mpiwg-rma(a)lists.mpi-forum.org >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma > > _______________________________________________ > mpiwg-rma mailing list > mpiwg-rma(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma -- Pavan Balaji ✉️ http://www.mcs.anl.gov/~balaji

1 0

Re: [mpiwg-rma] Memory barriers in passive-target RMA
by Rolf Rabenseifner 15 Jul '14

15 Jul '14

We have seen in the long discussion on the shared memory example ticket #413 (passed) that if two processes use load and store on the same location, additional memory fences are needed, i.e., p436:37-38 "In the RMA unified model, public and private copies are identical" is not really true, because with this sentence, the memory fence (i.e., MPI_WIN_SYNC) in #413 would not be needed. The new ticket #436 corrects this sentence on p436, ln 37-39, i.e., it substitutes the whole sentence by "In the RMA unified model, public and private copies are eventually synchronized without additional RMA calls at a byte-level granularity." Best regards Rolf ----- Original Message ----- > From: "Rajeev Thakur" <thakur(a)mcs.anl.gov> > To: "MPI WG Remote Memory Access working group" <mpiwg-rma(a)lists.mpi-forum.org> > Sent: Tuesday, July 15, 2014 2:51:21 PM > Subject: Re: [mpiwg-rma] Memory barriers in passive-target RMA > > The unified memory model is supposed to be covered by the word > “eventually” in the standard. > > pg 436, ln 37-41: "In the RMA unified model, public and private > copies are identical and updates via put or accumulate calls are > eventually observed by load operations without additional RMA calls. > A store access to a window is *eventually* visible to remote get or > accumulate calls without additional RMA calls. These stronger > semantics of the RMA unified model allow the user to omit some > synchronization calls and potentially improve performance.” > > pg 455, ln 16-21: "An update by a put or accumulate call to a public > window copy becomes visible in the private copy in process memory at > latest when an ensuing call to MPI_WIN_WAIT, MPI_WIN_FENCE, > MPI_WIN_LOCK, MPI_WIN_LOCK_ALL, or MPI_WIN_SYNC is executed on that > window by the window owner. In the RMA unified memory model, an > update by a put or accumulate call to a public window copy > *eventually* becomes visible in the private copy in process memory > without additional RMA calls." > > Rajeev > > > On Jul 15, 2014, at 6:31 AM, Balaji, Pavan <balaji(a)anl.gov> wrote: > > > > > I should have clarified in my original email — I’m talking about > > the UNIFIED memory model. > > > > — Pavan > > > > On Jul 15, 2014, at 12:14 AM, Thakur, Rajeev <thakur(a)mcs.anl.gov> > > wrote: > > > >> The equivalent text in the standard is on pg 454, ln 25-29. > >> > >> "If a put or accumulate access was synchronized with a lock, then > >> the update of the public window copy is complete as soon as the > >> updating process executed MPI_WIN_UNLOCK or MPI_WIN_UNLOCK_ALL. > >> In the RMA separate memory model, the update of a private copy in > >> the process memory may be delayed until the target process > >> executes a synchronization call on that window (6).” > >> > >> Rajeev > >> > >> On Jul 14, 2014, at 1:45 PM, Balaji, Pavan <balaji(a)anl.gov> wrote: > >> > >>> > >>> Ah, you meant the caption, not the example itself. Some > >>> equivalent text should go into the standard, since the example > >>> is non-binding. > >>> > >>> — Pavan > >>> > >>> On Jul 14, 2014, at 1:41 PM, Thakur, Rajeev <thakur(a)mcs.anl.gov> > >>> wrote: > >>> > >>>> On lines 40-42 it explains why the lock-unlock on process B is > >>>> needed: > >>>> > >>>> "Although the MPI_WIN_UNLOCK on process A and the MPI_BARRIER > >>>> ensure that the public copy on process B reflects the updated > >>>> value of X, the call to MPI_WIN_LOCK by process B is necessary > >>>> to synchronize the private copy with the public copy.” > >>>> > >>>> Rajeev > >>>> > >>>> On Jul 14, 2014, at 1:36 PM, Balaji, Pavan <balaji(a)anl.gov> > >>>> wrote: > >>>> > >>>>> > >>>>> I don’t understand your point. The example uses lock/unlock > >>>>> for local access, but that’s not a confirmation that it’s > >>>>> required. It just means that you are allowed to use > >>>>> lock/unlock. > >>>>> > >>>>> — Pavan > >>>>> > >>>>> P.S.: Adding a unicode character to try to trick mac to use > >>>>> UTF-8. Let’s see if this will work. What the heck, apple? > >>>>> ✉️ > >>>>> > >>>>> On Jul 14, 2014, at 1:30 PM, Thakur, Rajeev > >>>>> <thakur(a)mcs.anl.gov> wrote: > >>>>> > >>>>>> See example 11.8 on pg 458. > >>>>>> > >>>>>> Rajeev > >>>>>> > >>>>>> On Jul 14, 2014, at 1:15 PM, Balaji, Pavan <balaji(a)anl.gov> > >>>>>> wrote: > >>>>>> > >>>>>>> > >>>>>>> The MPI standard seems to allow access to the local window > >>>>>>> without requiring a lock (at least, I couldn’t find any text > >>>>>>> requiring a lock). Does this mean that the following > >>>>>>> example is correct? > >>>>>>> > >>>>>>> P0: > >>>>>>> Win_lock(P1) > >>>>>>> Put(X, 1) > >>>>>>> Win_unlock(P1) > >>>>>>> MPI_Send(P1) > >>>>>>> > >>>>>>> P1: > >>>>>>> MPI_Recv(P0) > >>>>>>> assert(X == 1) > >>>>>>> > >>>>>>> If the above is correct, shouldn’t there be a memory read > >>>>>>> barrier on P1 somewhere? Since P1 is not making any RMA > >>>>>>> calls, I’d assume that’ll need to somehow come from the lock > >>>>>>> and unlock operations. That is, the MPI implementation will > >>>>>>> need to do an active message in Win_lock and Win_unlock > >>>>>>> forcing a memory barrier at the target. Assuming that’s > >>>>>>> correct, I'll have to send out a lock packet even if the > >>>>>>> user gave the MPI_MODE_NOCHECK hint, for memory consistency > >>>>>>> reasons. That sounds awful, so I’m really hoping that I > >>>>>>> missed something in the standard which will say I don’t need > >>>>>>> to do all this. > >>>>>>> > >>>>>>> Note that all this active message problem will go away if I > >>>>>>> need P1 to do a lock/unlock to itself in order to access X. > >>>>>>> > >>>>>>> Thanks, > >>>>>>> > >>>>>>> — Pavan > >>>>>>> > >>>>>>> _______________________________________________ > >>>>>>> mpiwg-rma mailing list > >>>>>>> mpiwg-rma(a)lists.mpi-forum.org > >>>>>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma > >>>>>> > >>>>>> _______________________________________________ > >>>>>> mpiwg-rma mailing list > >>>>>> mpiwg-rma(a)lists.mpi-forum.org > >>>>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma > >>>>> > >>>>> _______________________________________________ > >>>>> mpiwg-rma mailing list > >>>>> mpiwg-rma(a)lists.mpi-forum.org > >>>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma > >>>> > >>>> _______________________________________________ > >>>> mpiwg-rma mailing list > >>>> mpiwg-rma(a)lists.mpi-forum.org > >>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma > >>> > >>> -- > >>> Pavan Balaji > >>> http://www.mcs.anl.gov/~balaji > >>> > >>> Unicode character to trick mac into using UTF-8 (✉️). > >>> _______________________________________________ > >>> mpiwg-rma mailing list > >>> mpiwg-rma(a)lists.mpi-forum.org > >>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma > >> > >> _______________________________________________ > >> mpiwg-rma mailing list > >> mpiwg-rma(a)lists.mpi-forum.org > >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma > > > > -- > > Pavan Balaji ✉️ > > http://www.mcs.anl.gov/~balaji > > > > _______________________________________________ > > mpiwg-rma mailing list > > mpiwg-rma(a)lists.mpi-forum.org > > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma > > _______________________________________________ > mpiwg-rma mailing list > mpiwg-rma(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma -- Dr. Rolf Rabenseifner . . . . . . . . . .. email rabenseifner(a)hlrs.de High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner Nobelstr. 19, D-70550 Stuttgart, Germany . . . . (Office: Room 1.307)

1 0

Re: [mpiwg-rma] Memory barriers in passive-target RMA
by Thakur, Rajeev 15 Jul '14

15 Jul '14

The unified memory model is supposed to be covered by the word “eventually” in the standard. pg 436, ln 37-41: "In the RMA unified model, public and private copies are identical and updates via put or accumulate calls are eventually observed by load operations without additional RMA calls. A store access to a window is *eventually* visible to remote get or accumulate calls without additional RMA calls. These stronger semantics of the RMA unified model allow the user to omit some synchronization calls and potentially improve performance.” pg 455, ln 16-21: "An update by a put or accumulate call to a public window copy becomes visible in the private copy in process memory at latest when an ensuing call to MPI_WIN_WAIT, MPI_WIN_FENCE, MPI_WIN_LOCK, MPI_WIN_LOCK_ALL, or MPI_WIN_SYNC is executed on that window by the window owner. In the RMA unified memory model, an update by a put or accumulate call to a public window copy *eventually* becomes visible in the private copy in process memory without additional RMA calls." Rajeev On Jul 15, 2014, at 6:31 AM, Balaji, Pavan <balaji(a)anl.gov> wrote: > > I should have clarified in my original email — I’m talking about the UNIFIED memory model. > > — Pavan > > On Jul 15, 2014, at 12:14 AM, Thakur, Rajeev <thakur(a)mcs.anl.gov> wrote: > >> The equivalent text in the standard is on pg 454, ln 25-29. >> >> "If a put or accumulate access was synchronized with a lock, then the update of the public window copy is complete as soon as the updating process executed MPI_WIN_UNLOCK or MPI_WIN_UNLOCK_ALL. In the RMA separate memory model, the update of a private copy in the process memory may be delayed until the target process executes a synchronization call on that window (6).” >> >> Rajeev >> >> On Jul 14, 2014, at 1:45 PM, Balaji, Pavan <balaji(a)anl.gov> wrote: >> >>> >>> Ah, you meant the caption, not the example itself. Some equivalent text should go into the standard, since the example is non-binding. >>> >>> — Pavan >>> >>> On Jul 14, 2014, at 1:41 PM, Thakur, Rajeev <thakur(a)mcs.anl.gov> wrote: >>> >>>> On lines 40-42 it explains why the lock-unlock on process B is needed: >>>> >>>> "Although the MPI_WIN_UNLOCK on process A and the MPI_BARRIER ensure that the public copy on process B reflects the updated value of X, the call to MPI_WIN_LOCK by process B is necessary to synchronize the private copy with the public copy.” >>>> >>>> Rajeev >>>> >>>> On Jul 14, 2014, at 1:36 PM, Balaji, Pavan <balaji(a)anl.gov> wrote: >>>> >>>>> >>>>> I don’t understand your point. The example uses lock/unlock for local access, but that’s not a confirmation that it’s required. It just means that you are allowed to use lock/unlock. >>>>> >>>>> — Pavan >>>>> >>>>> P.S.: Adding a unicode character to try to trick mac to use UTF-8. Let’s see if this will work. What the heck, apple? ✉️ >>>>> >>>>> On Jul 14, 2014, at 1:30 PM, Thakur, Rajeev <thakur(a)mcs.anl.gov> wrote: >>>>> >>>>>> See example 11.8 on pg 458. >>>>>> >>>>>> Rajeev >>>>>> >>>>>> On Jul 14, 2014, at 1:15 PM, Balaji, Pavan <balaji(a)anl.gov> wrote: >>>>>> >>>>>>> >>>>>>> The MPI standard seems to allow access to the local window without requiring a lock (at least, I couldn’t find any text requiring a lock). Does this mean that the following example is correct? >>>>>>> >>>>>>> P0: >>>>>>> Win_lock(P1) >>>>>>> Put(X, 1) >>>>>>> Win_unlock(P1) >>>>>>> MPI_Send(P1) >>>>>>> >>>>>>> P1: >>>>>>> MPI_Recv(P0) >>>>>>> assert(X == 1) >>>>>>> >>>>>>> If the above is correct, shouldn’t there be a memory read barrier on P1 somewhere? Since P1 is not making any RMA calls, I’d assume that’ll need to somehow come from the lock and unlock operations. That is, the MPI implementation will need to do an active message in Win_lock and Win_unlock forcing a memory barrier at the target. Assuming that’s correct, I'll have to send out a lock packet even if the user gave the MPI_MODE_NOCHECK hint, for memory consistency reasons. That sounds awful, so I’m really hoping that I missed something in the standard which will say I don’t need to do all this. >>>>>>> >>>>>>> Note that all this active message problem will go away if I need P1 to do a lock/unlock to itself in order to access X. >>>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> — Pavan >>>>>>> >>>>>>> _______________________________________________ >>>>>>> mpiwg-rma mailing list >>>>>>> mpiwg-rma(a)lists.mpi-forum.org >>>>>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma >>>>>> >>>>>> _______________________________________________ >>>>>> mpiwg-rma mailing list >>>>>> mpiwg-rma(a)lists.mpi-forum.org >>>>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma >>>>> >>>>> _______________________________________________ >>>>> mpiwg-rma mailing list >>>>> mpiwg-rma(a)lists.mpi-forum.org >>>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma >>>> >>>> _______________________________________________ >>>> mpiwg-rma mailing list >>>> mpiwg-rma(a)lists.mpi-forum.org >>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma >>> >>> -- >>> Pavan Balaji >>> http://www.mcs.anl.gov/~balaji >>> >>> Unicode character to trick mac into using UTF-8 (✉️). >>> _______________________________________________ >>> mpiwg-rma mailing list >>> mpiwg-rma(a)lists.mpi-forum.org >>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma >> >> _______________________________________________ >> mpiwg-rma mailing list >> mpiwg-rma(a)lists.mpi-forum.org >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma > > -- > Pavan Balaji ✉️ > http://www.mcs.anl.gov/~balaji > > _______________________________________________ > mpiwg-rma mailing list > mpiwg-rma(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma

1 0