May 2010 - mpiwg-rma - mailman3.cels.anl.gov

Re: [Mpi3-rma] mpi3-rma post from [email protected] requires approval
by Underwood, Keith D 30 May '10

30 May '10

I'll contribute to that when I am on a more stable internet connection ;-) Keith > -----Original Message----- > From: mpi3-rma-bounces(a)lists.mpi-forum.org [mailto:mpi3-rma- > bounces(a)lists.mpi-forum.org] On Behalf Of William Gropp > Sent: Sunday, May 30, 2010 11:04 AM > To: MPI 3.0 Remote Memory Access working group > Cc: 'bradc(a)cray.com' > Subject: Re: [Mpi3-rma] mpi3-rma post from bradc(a)cray.com requires > approval > > I'd like to get all of the various claims and constraints documented > in one place. I've created a page on the wiki into which we should > place these constraints, along with the reference. That page is > > https://svn.mpi-forum.org/trac/mpi-forum-web/wiki/mpi3-rma-goals-and- > constraint > > The rule for this page is that the statement should be concise and > precise and they should not claim that one approach is better than > another. Feel free to add additional top-level items. I'm fully > aware that the intersection of these requirements may produce the null > set; in fact, I think that is part of the problem. Putting a summary > on the web page, rather than in these email exchanges, will help us > move forward. > > Bill > > On May 30, 2010, at 8:05 AM, Underwood, Keith D wrote: > > > Well, when Jeff and Bill successfully argued against a definition > > for atomicity of access for conflicting accesses beyond "undefined", > > we lost direct support for the UPC memory model. After that, I no > > longer cared about ordering and Jeff was still arguing vehemently > > against ordering, so my understanding was that we had dropped > > ordering. I am not sure why we would support ordering without an > > access granularity definition. > > > > Keith > > > > > > ----- Original Message ----- > > From: mpi3-rma-bounces(a)lists.mpi-forum.org <mpi3-rma- > bounces(a)lists.mpi-forum.org > > > > > To: MPI 3.0 Remote Memory Access working group <mpi3-rma(a)lists.mpi- > forum.org > > > > > Cc: Brad Chamberlain <bradc(a)cray.com> > > Sent: Sat May 29 22:38:03 2010 > > Subject: Re: [Mpi3-rma] mpi3-rma post from bradc(a)cray.com requires > > approval > > > > > > On 05/29/2010 06:24 AM, Underwood, Keith D wrote: > >> 1) Ordering: > >> a) Ordered from a given source to a given address on a given > >> target (unordered otherwise), or > >> b) completely unordered > > > > My understanding was that we were providing both 1a and 1b. Or > rather, > > MPI-2 already gives 1b, and we are proposing 1a together with it (not > > instead of it). > > > > -- Pavan > > > > -- > > Pavan Balaji > > http://www.mcs.anl.gov/~balaji > > _______________________________________________ > > mpi3-rma mailing list > > mpi3-rma(a)lists.mpi-forum.org > > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma > > > > _______________________________________________ > > mpi3-rma mailing list > > mpi3-rma(a)lists.mpi-forum.org > > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma > > William Gropp > Deputy Director for Research > Institute for Advanced Computing Applications and Technologies > Paul and Cynthia Saylor Professor of Computer Science > University of Illinois Urbana-Champaign > > > > > _______________________________________________ > mpi3-rma mailing list > mpi3-rma(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma

1 0

Re: [Mpi3-rma] mpi3-rma post from [email protected] requires approval
by Underwood, Keith D 30 May '10

30 May '10

You are right, I should have chosen my words more carefully. I believe we tossed ordering on MPI_Put when we tossed atomicity. I don't think we ever got to the discussion of what ordering would be for the atomics. And, this is where life gets weird. I'm not positive that you need ordering for atomics, except that they are the MPI_Puts you would actually use when you needed ordering. So... I will go back to... all UPC needs is ordering from one source to one target address. I see three options: 1) Just define accumulates/get_accumulates to have the ordering that UPC needs 2) Default to the UPC ordering and allow an assert to relax that restriction. This would be consistent with how the current one-sided operations handle locks 3) Add some other call to handle ordering, but we wouldn't want some call that you had to call with EVERY put or anything... After all, all we need is relatively minimal ordering. Keith > -----Original Message----- > From: mpi3-rma-bounces(a)lists.mpi-forum.org [mailto:mpi3-rma- > bounces(a)lists.mpi-forum.org] On Behalf Of balaji(a)mcs.anl.gov > Sent: Sunday, May 30, 2010 10:56 AM > To: MPI 3.0 Remote Memory Access working group > Cc: bradc(a)cray.com > Subject: Re: [Mpi3-rma] mpi3-rma post from bradc(a)cray.com requires > approval > > Keith, > > The argument was against making MPI_Put more restrictive by requiring > atomicity, since MPI_Accumulate with REPLACE already provides it. So, I > think ordering is still important in this case. > > With respect to ordering, again, I think the argument was against > *only* ordered communication. I don't think there was opposition to > providing both ordered and unordered. > > -- Pavan > > -- > Pavan Balaji > http://www.mcs.anl.gov/~balaji > > ----- "Keith D Underwood" <keith.d.underwood(a)intel.com> wrote: > > > Well, when Jeff and Bill successfully argued against a definition for > > atomicity of access for conflicting accesses beyond "undefined", we > > lost direct support for the UPC memory model. After that, I no > longer > > cared about ordering and Jeff was still arguing vehemently against > > ordering, so my understanding was that we had dropped ordering. I am > > not sure why we would support ordering without an access granularity > > definition. > > > > Keith > > > > > > ----- Original Message ----- > > From: mpi3-rma-bounces(a)lists.mpi-forum.org > > <mpi3-rma-bounces(a)lists.mpi-forum.org> > > To: MPI 3.0 Remote Memory Access working group > > <mpi3-rma(a)lists.mpi-forum.org> > > Cc: Brad Chamberlain <bradc(a)cray.com> > > Sent: Sat May 29 22:38:03 2010 > > Subject: Re: [Mpi3-rma] mpi3-rma post from bradc(a)cray.com requires > > approval > > > > > > On 05/29/2010 06:24 AM, Underwood, Keith D wrote: > > > 1) Ordering: > > > a) Ordered from a given source to a given address on a given > target > > (unordered otherwise), or > > > b) completely unordered > > > > My understanding was that we were providing both 1a and 1b. Or > rather, > > > > MPI-2 already gives 1b, and we are proposing 1a together with it (not > > > > instead of it). > > > > -- Pavan > > > > -- > > Pavan Balaji > > http://www.mcs.anl.gov/~balaji > > _______________________________________________ > > mpi3-rma mailing list > > mpi3-rma(a)lists.mpi-forum.org > > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma > > > > _______________________________________________ > > mpi3-rma mailing list > > mpi3-rma(a)lists.mpi-forum.org > > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma > _______________________________________________ > mpi3-rma mailing list > mpi3-rma(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma

1 0

Re: [Mpi3-rma] mpi3-rma post from [email protected] requires approval
by William Gropp 30 May '10

30 May '10

Yes - the requirement that *all* one-sided communication be ordered was what we objected to - not that there be no way to specify ordered communication. This is one of the reasons I started the wiki page about which I just sent mail. Bill On May 30, 2010, at 9:55 AM, balaji(a)mcs.anl.gov wrote: > Keith, > > The argument was against making MPI_Put more restrictive by > requiring atomicity, since MPI_Accumulate with REPLACE already > provides it. So, I think ordering is still important in this case. > > With respect to ordering, again, I think the argument was against > *only* ordered communication. I don't think there was opposition to > providing both ordered and unordered. > > -- Pavan > > -- > Pavan Balaji > http://www.mcs.anl.gov/~balaji > > ----- "Keith D Underwood" <keith.d.underwood(a)intel.com> wrote: > >> Well, when Jeff and Bill successfully argued against a definition for >> atomicity of access for conflicting accesses beyond "undefined", we >> lost direct support for the UPC memory model. After that, I no >> longer >> cared about ordering and Jeff was still arguing vehemently against >> ordering, so my understanding was that we had dropped ordering. I am >> not sure why we would support ordering without an access granularity >> definition. >> >> Keith >> >> >> ----- Original Message ----- >> From: mpi3-rma-bounces(a)lists.mpi-forum.org >> <mpi3-rma-bounces(a)lists.mpi-forum.org> >> To: MPI 3.0 Remote Memory Access working group >> <mpi3-rma(a)lists.mpi-forum.org> >> Cc: Brad Chamberlain <bradc(a)cray.com> >> Sent: Sat May 29 22:38:03 2010 >> Subject: Re: [Mpi3-rma] mpi3-rma post from bradc(a)cray.com requires >> approval >> >> >> On 05/29/2010 06:24 AM, Underwood, Keith D wrote: >>> 1) Ordering: >>> a) Ordered from a given source to a given address on a given target >> (unordered otherwise), or >>> b) completely unordered >> >> My understanding was that we were providing both 1a and 1b. Or >> rather, >> >> MPI-2 already gives 1b, and we are proposing 1a together with it (not >> >> instead of it). >> >> -- Pavan >> >> -- >> Pavan Balaji >> http://www.mcs.anl.gov/~balaji >> _______________________________________________ >> mpi3-rma mailing list >> mpi3-rma(a)lists.mpi-forum.org >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma >> >> _______________________________________________ >> mpi3-rma mailing list >> mpi3-rma(a)lists.mpi-forum.org >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma > _______________________________________________ > mpi3-rma mailing list > mpi3-rma(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma William Gropp Deputy Director for Research Institute for Advanced Computing Applications and Technologies Paul and Cynthia Saylor Professor of Computer Science University of Illinois Urbana-Champaign

1 0

Re: [Mpi3-rma] mpi3-rma post from [email protected] requires approval
by William Gropp 30 May '10

30 May '10

I'd like to get all of the various claims and constraints documented in one place. I've created a page on the wiki into which we should place these constraints, along with the reference. That page is https://svn.mpi-forum.org/trac/mpi-forum-web/wiki/mpi3-rma-goals-and-constr… The rule for this page is that the statement should be concise and precise and they should not claim that one approach is better than another. Feel free to add additional top-level items. I'm fully aware that the intersection of these requirements may produce the null set; in fact, I think that is part of the problem. Putting a summary on the web page, rather than in these email exchanges, will help us move forward. Bill On May 30, 2010, at 8:05 AM, Underwood, Keith D wrote: > Well, when Jeff and Bill successfully argued against a definition > for atomicity of access for conflicting accesses beyond "undefined", > we lost direct support for the UPC memory model. After that, I no > longer cared about ordering and Jeff was still arguing vehemently > against ordering, so my understanding was that we had dropped > ordering. I am not sure why we would support ordering without an > access granularity definition. > > Keith > > > ----- Original Message ----- > From: mpi3-rma-bounces(a)lists.mpi-forum.org <mpi3-rma-bounces(a)lists.mpi-forum.org > > > To: MPI 3.0 Remote Memory Access working group <mpi3-rma(a)lists.mpi-forum.org > > > Cc: Brad Chamberlain <bradc(a)cray.com> > Sent: Sat May 29 22:38:03 2010 > Subject: Re: [Mpi3-rma] mpi3-rma post from bradc(a)cray.com requires > approval > > > On 05/29/2010 06:24 AM, Underwood, Keith D wrote: >> 1) Ordering: >> a) Ordered from a given source to a given address on a given >> target (unordered otherwise), or >> b) completely unordered > > My understanding was that we were providing both 1a and 1b. Or rather, > MPI-2 already gives 1b, and we are proposing 1a together with it (not > instead of it). > > -- Pavan > > -- > Pavan Balaji > http://www.mcs.anl.gov/~balaji > _______________________________________________ > mpi3-rma mailing list > mpi3-rma(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma > > _______________________________________________ > mpi3-rma mailing list > mpi3-rma(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma William Gropp Deputy Director for Research Institute for Advanced Computing Applications and Technologies Paul and Cynthia Saylor Professor of Computer Science University of Illinois Urbana-Champaign

1 0

Re: [Mpi3-rma] mpi3-rma post from [email protected] requires approval
by balaji＠mcs.anl.gov 30 May '10

30 May '10

Keith, The argument was against making MPI_Put more restrictive by requiring atomicity, since MPI_Accumulate with REPLACE already provides it. So, I think ordering is still important in this case. With respect to ordering, again, I think the argument was against *only* ordered communication. I don't think there was opposition to providing both ordered and unordered. -- Pavan -- Pavan Balaji http://www.mcs.anl.gov/~balaji ----- "Keith D Underwood" <keith.d.underwood(a)intel.com> wrote: > Well, when Jeff and Bill successfully argued against a definition for > atomicity of access for conflicting accesses beyond "undefined", we > lost direct support for the UPC memory model. After that, I no longer > cared about ordering and Jeff was still arguing vehemently against > ordering, so my understanding was that we had dropped ordering. I am > not sure why we would support ordering without an access granularity > definition. > > Keith > > > ----- Original Message ----- > From: mpi3-rma-bounces(a)lists.mpi-forum.org > <mpi3-rma-bounces(a)lists.mpi-forum.org> > To: MPI 3.0 Remote Memory Access working group > <mpi3-rma(a)lists.mpi-forum.org> > Cc: Brad Chamberlain <bradc(a)cray.com> > Sent: Sat May 29 22:38:03 2010 > Subject: Re: [Mpi3-rma] mpi3-rma post from bradc(a)cray.com requires > approval > > > On 05/29/2010 06:24 AM, Underwood, Keith D wrote: > > 1) Ordering: > > a) Ordered from a given source to a given address on a given target > (unordered otherwise), or > > b) completely unordered > > My understanding was that we were providing both 1a and 1b. Or rather, > > MPI-2 already gives 1b, and we are proposing 1a together with it (not > > instead of it). > > -- Pavan > > -- > Pavan Balaji > http://www.mcs.anl.gov/~balaji > _______________________________________________ > mpi3-rma mailing list > mpi3-rma(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma > > _______________________________________________ > mpi3-rma mailing list > mpi3-rma(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma

1 0

Re: [Mpi3-rma] mpi3-rma post from [email protected] requires approval
by Underwood, Keith D 30 May '10

30 May '10

Well, when Jeff and Bill successfully argued against a definition for atomicity of access for conflicting accesses beyond "undefined", we lost direct support for the UPC memory model. After that, I no longer cared about ordering and Jeff was still arguing vehemently against ordering, so my understanding was that we had dropped ordering. I am not sure why we would support ordering without an access granularity definition. Keith ----- Original Message ----- From: mpi3-rma-bounces(a)lists.mpi-forum.org <mpi3-rma-bounces(a)lists.mpi-forum.org> To: MPI 3.0 Remote Memory Access working group <mpi3-rma(a)lists.mpi-forum.org> Cc: Brad Chamberlain <bradc(a)cray.com> Sent: Sat May 29 22:38:03 2010 Subject: Re: [Mpi3-rma] mpi3-rma post from bradc(a)cray.com requires approval On 05/29/2010 06:24 AM, Underwood, Keith D wrote: > 1) Ordering: > a) Ordered from a given source to a given address on a given target (unordered otherwise), or > b) completely unordered My understanding was that we were providing both 1a and 1b. Or rather, MPI-2 already gives 1b, and we are proposing 1a together with it (not instead of it). -- Pavan -- Pavan Balaji http://www.mcs.anl.gov/~balaji _______________________________________________ mpi3-rma mailing list mpi3-rma(a)lists.mpi-forum.org http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma

1 0

Re: [Mpi3-rma] mpi3-rma post from [email protected] requires approval
by Pavan Balaji 30 May '10

30 May '10

On 05/29/2010 06:24 AM, Underwood, Keith D wrote: > 1) Ordering: > a) Ordered from a given source to a given address on a given target (unordered otherwise), or > b) completely unordered My understanding was that we were providing both 1a and 1b. Or rather, MPI-2 already gives 1b, and we are proposing 1a together with it (not instead of it). -- Pavan -- Pavan Balaji http://www.mcs.anl.gov/~balaji

1 0

Re: [Mpi3-rma] mpi3-rma post from [email protected] requires approval
by Underwood, Keith D 29 May '10

29 May '10

Brad, If the question you were asked as merely "more strict or more relaxed", then it needs more context from the conversation. The specific semantics in question were: 1) Ordering: a) Ordered from a given source to a given address on a given target (unordered otherwise), or b) completely unordered 2) Granularity of access/atomicity: a) Two accesses from two sources to the same address appear in some (undefined) order, or b) two accesses from two sources to the same address yield undefined behavior I argued strongly for 1a and 2a, since they correspond to the weakest UPC memory model, but Jeff (and others) argued that those were not necessary and imposed unacceptable performance penalties. Our current trend is toward 1b and 2b. When Jeff reported back to us that you were fine with the latter two, I must admit that I was rather surprised given that I assumed that Chapel's weakest memory model was on par with UPCs. Keith ------------------------------------------- Hi MPI-3 RMA team -- I ran into Jeff Hammond at a workshop a few weeks back and we had a brief chat about whether, as a potential client of MPI-3 RMA, I would prefer its semantics to err more on the strict or relaxed side. He requested that I consider sending a brief note to this group with my thoughts, so this is that note. I hope that this opinion will be considered useful and not out-of-turn given how little time I've had to invest in following the work of the MPI-3 team. I should start with the disclaimer that I'm not an expert on memory consistency models -- I probably know more than the average programmer, but have typically been insulated from worrying about it in a great amount of detail, either by relying on other software layers or languages to take care of it for me or by having the fortune to work with codes and idioms that don't fall afoul of the differences. My gut response to the question is that I'd prefer things to be on the more relaxed side. I think one of the key benefits of single-sided communication is its separation of data transfer from synchronization. I'd worry that by trying to enforce too much strictness in the RMA interface, it would work break down this separation and result in performance overheads that couldn't be recouped. On the other hand, if MPI-3 exported a model that was more relaxed than a particular programmer/programming model wanted, my assumption is that they could increase the strictness by doing more manual synchronization/memory fences/etc. themselves. That is, a relaxed model would not seem to exclude strictness while a strict model may impact performance negatively without any recourse. If that's a correct interpretation, the relaxed approach seems like the one to take to me. I'm reluctant to speak for others, but wanted to note (if he hasn't already done so) that Dave Grove from IBM's X10 team was with us and seemed to agree with this point-of-view (though perhaps we were both simply falling prey to Jeff's subliminal hypnosis? :). All that said, owing to my lack of depth in this area, I would say that if the GASNet team and/or the UPC/Titanium teams who built on top of GASNet felt that this was clearly the wrong approach, I would tend to cast my vote with them since I think they've studied this issue in far more detail than most parallel language groups, ours included. (I do think that Kathy Yelick voiced a compatible opinion in another context at this same workshop, which gave me some reassurance that relaxed was the way to go, but again, these were fairly high-level conversations. More generally, I would encourage you to get input from the GASNet team as you consider this issue and others related to 1-sided communication if you haven't). If you think it would be useful for me to hear the other side of the debate and/or consider some specific case examples in more detail, I'd be happy to do so as time permits. Have a good weekend, -Brad

1 0

[Mpi3-rma] Fw: mpi3-rma post from [email protected] requires approval
by Graham, Richard L. 28 May '10

28 May '10

----- Original Message ----- From: osl-mailman-bounces(a)osl.iu.edu <osl-mailman-bounces(a)osl.iu.edu> To: mpi3-rma-owner(a)lists.mpi-forum.org <mpi3-rma-owner(a)lists.mpi-forum.org> Sent: Fri May 28 19:44:21 2010 Subject: mpi3-rma post from bradc(a)cray.com requires approval As list administrator, your authorization is requested for the following mailing list posting: List: mpi3-rma(a)lists.mpi-forum.org From: bradc(a)cray.com Subject: Strict vs. relaxed RMA in MPI Reason: Post by non-member to a members-only list At your convenience, visit: http://lists.mpi-forum.org/mailman/admindb.cgi/mpi3-rma to approve or deny the request.

1 0

[Mpi3-rma] Strict vs. relaxed RMA in MPI
by Brad Chamberlain 28 May '10

28 May '10

Hi MPI-3 RMA team -- I ran into Jeff Hammond at a workshop a few weeks back and we had a brief chat about whether, as a potential client of MPI-3 RMA, I would prefer its semantics to err more on the strict or relaxed side. He requested that I consider sending a brief note to this group with my thoughts, so this is that note. I hope that this opinion will be considered useful and not out-of-turn given how little time I've had to invest in following the work of the MPI-3 team. I should start with the disclaimer that I'm not an expert on memory consistency models -- I probably know more than the average programmer, but have typically been insulated from worrying about it in a great amount of detail, either by relying on other software layers or languages to take care of it for me or by having the fortune to work with codes and idioms that don't fall afoul of the differences. My gut response to the question is that I'd prefer things to be on the more relaxed side. I think one of the key benefits of single-sided communication is its separation of data transfer from synchronization. I'd worry that by trying to enforce too much strictness in the RMA interface, it would work break down this separation and result in performance overheads that couldn't be recouped. On the other hand, if MPI-3 exported a model that was more relaxed than a particular programmer/programming model wanted, my assumption is that they could increase the strictness by doing more manual synchronization/memory fences/etc. themselves. That is, a relaxed model would not seem to exclude strictness while a strict model may impact performance negatively without any recourse. If that's a correct interpretation, the relaxed approach seems like the one to take to me. I'm reluctant to speak for others, but wanted to note (if he hasn't already done so) that Dave Grove from IBM's X10 team was with us and seemed to agree with this point-of-view (though perhaps we were both simply falling prey to Jeff's subliminal hypnosis? :). All that said, owing to my lack of depth in this area, I would say that if the GASNet team and/or the UPC/Titanium teams who built on top of GASNet felt that this was clearly the wrong approach, I would tend to cast my vote with them since I think they've studied this issue in far more detail than most parallel language groups, ours included. (I do think that Kathy Yelick voiced a compatible opinion in another context at this same workshop, which gave me some reassurance that relaxed was the way to go, but again, these were fairly high-level conversations. More generally, I would encourage you to get input from the GASNet team as you consider this issue and others related to 1-sided communication if you haven't). If you think it would be useful for me to hear the other side of the debate and/or consider some specific case examples in more detail, I'd be happy to do so as time permits. Have a good weekend, -Brad

1 0