October 2008 - mpiwg-rma - mailman3.cels.anl.gov

[Mpi3-rma] signaling / remote notification of one-sided call
by Vinod tipparaju 30 Oct '08

30 Oct '08

Howard brought up this interesting topic. Some one-sided model implementation have looked at remote notification of RMA operation primarily as a simple, low overhead, mechanism to inform the remote side of a one-sided operation. ARMCI has notify and wait calls per-process. GASNet has split-phase barrier with notify and wait collective calls. However, I am not very convinced that this is as important for MPI RMA. This can be achieved with a send-recv call after a one-sided call. Lets start a small discussion about this. signaling/notification of a remote call eg: Sender - RMA_Xfer_notify or RMA_Xfer followed by Notify(context) Receiver: Wait(context) Merits: performance? ease of programming? Demerits: duplicate functionality? many alternatives to do the same thing? not a noticeable performance gain? Thanks, Vinod.

1 0

Re: [Mpi3-rma] draft of a proposal for RMA interfaces
by Vinod tipparaju 30 Oct '08

30 Oct '08

Hello Howard, Thanks for the corrections. Req_set_attr and get_attr are _alternatives_ proposed if a new request type were to be used for RMA and RMW operations (only for discussion, in addition to existing MPI request) instead of passing the attributes as a parameter to the MPI call. This new request type is passed to MPI call as a parameter. In this case, the attributes can be set on this new RMA/W Request and this can be used in the RMA/RMW MPI call instead of passing the attributes as a parameter. As you know, the example interfaces are included in the document only to initiate discussions. I will start two separate discussions from your questions 1)signaling / remote notification of one-sided call 2)proposing interfaces/semantics that can potentially be implemented with point-to-point communication protocols Thanks, vinod. ---------------------------------------- > Date: Thu, 16 Oct 2008 14:34:07 -0600 > From: howardp(a)cray.com > To: mpi3-rma(a)lists.mpi-forum.org > Subject: Re: [Mpi3-rma] draft of a proposal for RMA interfaces > > Hello Folks, > > Try again with the correction: > > original - > > the GET transfers from the caller memory(origin) to the target memory; > PUT transfer data from the target memory to the caller memory > > what was probably meant - > > the PUT transfers from the caller memory(origin) to the target memory; > GET transfer data from the target memory to the caller memory > > Sorry about that, > > Howard > > Howard Pritchard wrote: > >> Hello Rich, >> >> I meant to respond earlier to Vinod's proposal, sorry for the delay. >> >> First a minor correction in section 5.1. I think the prhase: >> >> /the GET transfers from the caller memory(origin) to the target memory; >> PUT transfer data from the target memory to the caller memory/ >> >> near the beginning of section. >> >> I think overall the proposal looks interesting. I am somewhat confused >> by the MPI_Req_set_attr and MPI_Req_get_attr and how they are suppose >> to be used. >> >> The main item that is missing seems to be a method for signaling >> completion >> of an rma operation at a remote process. Is the assumption that if >> this was required, >> the remote process would spin wait on a volatile? This would seem to >> introduce >> other problems - like how to 'emulate' this functionality using >> point-to-point >> protocols. I believe that any new rma proposal needs to have semantics >> such that >> one could base a correct, if low performance, implementation using >> point-to-point >> communication protocols. >> >> Howard >> >> Richard Graham wrote: >> >>> Just to get discussion going again. Talking with several folks I have >>> heard several concerns expressed about the proposal. I think it would >>> be good if these (and others) could be raised on the list, so we can >>> start discussion. We can continue this next week in Chicago, but >>> Vinod will not be able to make this meeting, so an e-mail discussion >>> will help. >>> >>> Here are the issues I have hear of so far: >>> - May not work well on current h/w that is not cache coherent, as it >>> requires a remote thread in this case. I believe this is for the SX >>> series of machines, but Jesper please correct me if I am wrong here. >>> What would be an alternative approach that could provide expected >>> performance on platforms that may require work on the remote end for >>> RMA for correctness, and work well on platforms that do require very >>> specific remote cache management (or other actions) for correctness ? >>> - Concern about future high-end platforms, under that assumption that >>> these will not be cache coherent (and will actually have caches ï¿½ if >>> they donï¿½t this is not a concern), and therefore this proposal is >>> aimed at a short-lived technical capability. >>> - What is missing ? >>> >>> Rich >>> >>> >>> On 9/6/08 6:06 PM, "Vinod tipparaju" wrote: >>> >>> >>> >>> Rajeev, >>> >>> Excellent questions. >>> >>> Technically, yes. However, ARMCI does more thinking about Global >>> Arrays. Particularly because of stride data transfers (multiple >>> stride levels). However, I don't see why we cannot have an >>> alternative port. The only way to know is to prototype the >>> interfaces, implement Global Arrays on top of them and see how the >>> interfaces fare. From superficial thinking, I think they will do >>> just fine ;-). We however cannot say this about all potential >>> implementations. >>> >>> I am not sure if Dan Bonachea is a member of this list. I will >>> write him a message if he isn't. >>> >>> Thanks, >>> Vinod. >>> >>> ---------------------------------------- >>> >>>> From: thakur(a)mcs.anl.gov >>>> To: mpi3-rma(a)lists.mpi-forum.org >>>> Date: Sat, 6 Sep 2008 16:24:35 -0500 >>>> Subject: Re: [Mpi3-rma] draft of a proposal for RMA interfaces >>>> >>>> Vinod, >>>> Thanks for the proposal and the nice presentation. A question I had >>>> was does it meet the needs of the Global Arrays library? That is, >>> >>> if this >>> >>>> were available in MPI today, would you implement Global Arrays >>> >>> with MPI and >>> >>>> nothing else (not ARMCI). Similarly, it would be good to run it >>> >>> by Dan >>> >>>> Bonachea and see what he has to say regarding implementing UPC. >>>> >>>> Regards, >>>> Rajeev >>>> >>>> >>>>> -----Original Message----- >>>>> From: mpi3-rma-bounces(a)lists.mpi-forum.org >>>>> [mailto:[email protected]] On Behalf Of >>>>> Vinod tipparaju >>>>> Sent: Thursday, September 04, 2008 1:50 AM >>>>> To: mpi3-rma(a)lists.mpi-forum.org >>>>> Subject: [Mpi3-rma] draft of a proposal for RMA interfaces >>>>> >>>>> >>>>> Folks, >>>>> >>>>> Attached is a draft of a new proposal for MPI3-RMA interfaces >>>>> for your perusal. >>>>> >>>>> Thanks, >>>>> Vinod Tipparaju, ORNL. >>>>> >>>> >>>> _______________________________________________ >>>> mpi3-rma mailing list >>>> mpi3-rma(a)lists.mpi-forum.org >>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma >>> >>> >>> _______________________________________________ >>> mpi3-rma mailing list >>> mpi3-rma(a)lists.mpi-forum.org >>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma >>> >>> ------------------------------------------------------------------------ >>> >>> _______________________________________________ >>> mpi3-rma mailing list >>> mpi3-rma(a)lists.mpi-forum.org >>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma >>> >>> >> >> > > > -- > > Howard Pritchard > Cray Inc. > > > _______________________________________________ > mpi3-rma mailing list > mpi3-rma(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma

1 0

Re: [Mpi3-rma] draft of a proposal for RMA interfaces
by Rajeev Thakur 19 Oct '08

19 Oct '08

I think it would be good to add an FAQ section at the end, containing answers to questions that will be asked of any RMA proposal, such as non-cache-coherent, does it meet the needs of PGAS/Global Arrays, support for heterogeneous, how does the target know of completion, how does it interplay with existing RMA spec, etc. It will make sure that the proposal addresses those issues, that we ourselves are clear of the answers, and that they are not repeatedly raised at each meeting. Rajeev > Richard Graham wrote: > >> Just to get discussion going again. Talking with several folks I have >> heard several concerns expressed about the proposal. I think it would >> be good if these (and others) could be raised on the list, so we can >> start discussion. We can continue this next week in Chicago, but >> Vinod will not be able to make this meeting, so an e-mail discussion >> will help. >> >> Here are the issues I have hear of so far: >> - May not work well on current h/w that is not cache coherent, as it >> requires a remote thread in this case. I believe this is for the SX >> series of machines, but Jesper please correct me if I am wrong here. >> What would be an alternative approach that could provide expected >> performance on platforms that may require work on the remote end for >> RMA for correctness, and work well on platforms that do require very >> specific remote cache management (or other actions) for correctness ? >> - Concern about future high-end platforms, under that assumption that >> these will not be cache coherent (and will actually have caches - if >> they don't this is not a concern), and therefore this proposal is >> aimed at a short-lived technical capability. >> - What is missing ?

1 0

Re: [Mpi3-rma] draft of a proposal for RMA interfaces
by Howard Pritchard 16 Oct '08

16 Oct '08

Hello Folks, Try again with the correction: original - the GET transfers from the caller memory(origin) to the target memory; PUT transfer data from the target memory to the caller memory what was probably meant - the PUT transfers from the caller memory(origin) to the target memory; GET transfer data from the target memory to the caller memory Sorry about that, Howard Howard Pritchard wrote: > Hello Rich, > > I meant to respond earlier to Vinod's proposal, sorry for the delay. > > First a minor correction in section 5.1. I think the prhase: > > /the GET transfers from the caller memory(origin) to the target memory; > PUT transfer data from the target memory to the caller memory/ > > near the beginning of section. > > I think overall the proposal looks interesting. I am somewhat confused > by the MPI_Req_set_attr and MPI_Req_get_attr and how they are suppose > to be used. > > The main item that is missing seems to be a method for signaling > completion > of an rma operation at a remote process. Is the assumption that if > this was required, > the remote process would spin wait on a volatile? This would seem to > introduce > other problems - like how to 'emulate' this functionality using > point-to-point > protocols. I believe that any new rma proposal needs to have semantics > such that > one could base a correct, if low performance, implementation using > point-to-point > communication protocols. > > Howard > > Richard Graham wrote: > >> Just to get discussion going again. Talking with several folks I have >> heard several concerns expressed about the proposal. I think it would >> be good if these (and others) could be raised on the list, so we can >> start discussion. We can continue this next week in Chicago, but >> Vinod will not be able to make this meeting, so an e-mail discussion >> will help. >> >> Here are the issues I have hear of so far: >> - May not work well on current h/w that is not cache coherent, as it >> requires a remote thread in this case. I believe this is for the SX >> series of machines, but Jesper please correct me if I am wrong here. >> What would be an alternative approach that could provide expected >> performance on platforms that may require work on the remote end for >> RMA for correctness, and work well on platforms that do require very >> specific remote cache management (or other actions) for correctness ? >> - Concern about future high-end platforms, under that assumption that >> these will not be cache coherent (and will actually have caches ï¿½ if >> they donï¿½t this is not a concern), and therefore this proposal is >> aimed at a short-lived technical capability. >> - What is missing ? >> >> Rich >> >> >> On 9/6/08 6:06 PM, "Vinod tipparaju" <tipparajuv(a)hotmail.com> wrote: >> >> >> >> Rajeev, >> >> Excellent questions. >> >> Technically, yes. However, ARMCI does more thinking about Global >> Arrays. Particularly because of stride data transfers (multiple >> stride levels). However, I don't see why we cannot have an >> alternative port. The only way to know is to prototype the >> interfaces, implement Global Arrays on top of them and see how the >> interfaces fare. From superficial thinking, I think they will do >> just fine ;-). We however cannot say this about all potential >> implementations. >> >> I am not sure if Dan Bonachea is a member of this list. I will >> write him a message if he isn't. >> >> Thanks, >> Vinod. >> >> ---------------------------------------- >> >>> From: thakur(a)mcs.anl.gov >>> To: mpi3-rma(a)lists.mpi-forum.org >>> Date: Sat, 6 Sep 2008 16:24:35 -0500 >>> Subject: Re: [Mpi3-rma] draft of a proposal for RMA interfaces >>> >>> Vinod, >>> Thanks for the proposal and the nice presentation. A question I had >>> was does it meet the needs of the Global Arrays library? That is, >> >> if this >> >>> were available in MPI today, would you implement Global Arrays >> >> with MPI and >> >>> nothing else (not ARMCI). Similarly, it would be good to run it >> >> by Dan >> >>> Bonachea and see what he has to say regarding implementing UPC. >>> >>> Regards, >>> Rajeev >>> >>> >>>> -----Original Message----- >>>> From: mpi3-rma-bounces(a)lists.mpi-forum.org >>>> [mailto:[email protected]] On Behalf Of >>>> Vinod tipparaju >>>> Sent: Thursday, September 04, 2008 1:50 AM >>>> To: mpi3-rma(a)lists.mpi-forum.org >>>> Subject: [Mpi3-rma] draft of a proposal for RMA interfaces >>>> >>>> >>>> Folks, >>>> >>>> Attached is a draft of a new proposal for MPI3-RMA interfaces >>>> for your perusal. >>>> >>>> Thanks, >>>> Vinod Tipparaju, ORNL. >>>> >>> >>> _______________________________________________ >>> mpi3-rma mailing list >>> mpi3-rma(a)lists.mpi-forum.org >>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma >> >> >> _______________________________________________ >> mpi3-rma mailing list >> mpi3-rma(a)lists.mpi-forum.org >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma >> >> ------------------------------------------------------------------------ >> >> _______________________________________________ >> mpi3-rma mailing list >> mpi3-rma(a)lists.mpi-forum.org >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma >> >> > > -- Howard Pritchard Cray Inc.

1 0

Re: [Mpi3-rma] draft of a proposal for RMA interfaces
by Howard Pritchard 16 Oct '08

16 Oct '08

Hello Rich, I meant to respond earlier to Vinod's proposal, sorry for the delay. First a minor correction in section 5.1. I think the prhase: /the GET transfers from the caller memory(origin) to the target memory; PUT transfer data from the target memory to the caller memory/ near the beginning of section. I think overall the proposal looks interesting. I am somewhat confused by the MPI_Req_set_attr and MPI_Req_get_attr and how they are suppose to be used. The main item that is missing seems to be a method for signaling completion of an rma operation at a remote process. Is the assumption that if this was required, the remote process would spin wait on a volatile? This would seem to introduce other problems - like how to 'emulate' this functionality using point-to-point protocols. I believe that any new rma proposal needs to have semantics such that one could base a correct, if low performance, implementation using point-to-point communication protocols. Howard Richard Graham wrote: > Just to get discussion going again. Talking with several folks I have > heard several concerns expressed about the proposal. I think it would > be good if these (and others) could be raised on the list, so we can > start discussion. We can continue this next week in Chicago, but Vinod > will not be able to make this meeting, so an e-mail discussion will help. > > Here are the issues I have hear of so far: > - May not work well on current h/w that is not cache coherent, as it > requires a remote thread in this case. I believe this is for the SX > series of machines, but Jesper please correct me if I am wrong here. > What would be an alternative approach that could provide expected > performance on platforms that may require work on the remote end for > RMA for correctness, and work well on platforms that do require very > specific remote cache management (or other actions) for correctness ? > - Concern about future high-end platforms, under that assumption that > these will not be cache coherent (and will actually have caches ï¿½ if > they donï¿½t this is not a concern), and therefore this proposal is > aimed at a short-lived technical capability. > - What is missing ? > > Rich > > > On 9/6/08 6:06 PM, "Vinod tipparaju" <tipparajuv(a)hotmail.com> wrote: > > > > Rajeev, > > Excellent questions. > > Technically, yes. However, ARMCI does more thinking about Global > Arrays. Particularly because of stride data transfers (multiple > stride levels). However, I don't see why we cannot have an > alternative port. The only way to know is to prototype the > interfaces, implement Global Arrays on top of them and see how the > interfaces fare. From superficial thinking, I think they will do > just fine ;-). We however cannot say this about all potential > implementations. > > I am not sure if Dan Bonachea is a member of this list. I will > write him a message if he isn't. > > Thanks, > Vinod. > > ---------------------------------------- >> From: thakur(a)mcs.anl.gov >> To: mpi3-rma(a)lists.mpi-forum.org >> Date: Sat, 6 Sep 2008 16:24:35 -0500 >> Subject: Re: [Mpi3-rma] draft of a proposal for RMA interfaces >> >> Vinod, >> Thanks for the proposal and the nice presentation. A question I had >> was does it meet the needs of the Global Arrays library? That is, > if this >> were available in MPI today, would you implement Global Arrays > with MPI and >> nothing else (not ARMCI). Similarly, it would be good to run it > by Dan >> Bonachea and see what he has to say regarding implementing UPC. >> >> Regards, >> Rajeev >> >> >>> -----Original Message----- >>> From: mpi3-rma-bounces(a)lists.mpi-forum.org >>> [mailto:[email protected]] On Behalf Of >>> Vinod tipparaju >>> Sent: Thursday, September 04, 2008 1:50 AM >>> To: mpi3-rma(a)lists.mpi-forum.org >>> Subject: [Mpi3-rma] draft of a proposal for RMA interfaces >>> >>> >>> Folks, >>> >>> Attached is a draft of a new proposal for MPI3-RMA interfaces >>> for your perusal. >>> >>> Thanks, >>> Vinod Tipparaju, ORNL. >>> >> >> _______________________________________________ >> mpi3-rma mailing list >> mpi3-rma(a)lists.mpi-forum.org >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma > > _______________________________________________ > mpi3-rma mailing list > mpi3-rma(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma > >------------------------------------------------------------------------ > >_______________________________________________ >mpi3-rma mailing list >mpi3-rma(a)lists.mpi-forum.org >http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma > > -- Howard Pritchard Cray Inc.

1 0

[Mpi3-rma] A mild extension/restriction: Registration of datatypes/ops for one-sided communication
by Jesper Larsson Traeff 16 Oct '08

16 Oct '08

dear all, I've added a small proposal on registration of datatypes/ops for one-sided communication for discussion. It is ticket #35. This proposal concerns the use of derived datatypes and is orthogonal to the other proposals so far jesper

1 0

Re: [Mpi3-rma] draft of a proposal for RMA interfaces
by Richard Graham 16 Oct '08

16 Oct '08

Just to get discussion going again. Talking with several folks I have heard several concerns expressed about the proposal. I think it would be good if these (and others) could be raised on the list, so we can start discussion. We can continue this next week in Chicago, but Vinod will not be able to make this meeting, so an e-mail discussion will help. Here are the issues I have hear of so far: - May not work well on current h/w that is not cache coherent, as it requires a remote thread in this case. I believe this is for the SX series of machines, but Jesper please correct me if I am wrong here. What would be an alternative approach that could provide expected performance on platforms that may require work on the remote end for RMA for correctness, and work well on platforms that do require very specific remote cache management (or other actions) for correctness ? - Concern about future high-end platforms, under that assumption that these will not be cache coherent (and will actually have caches if they don¹t this is not a concern), and therefore this proposal is aimed at a short-lived technical capability. - What is missing ? Rich On 9/6/08 6:06 PM, "Vinod tipparaju" <tipparajuv(a)hotmail.com> wrote: > > > Rajeev, > > Excellent questions. > > Technically, yes. However, ARMCI does more thinking about Global Arrays. > Particularly because of stride data transfers (multiple stride levels). > However, I don't see why we cannot have an alternative port. The only way to > know is to prototype the interfaces, implement Global Arrays on top of them > and see how the interfaces fare. From superficial thinking, I think they will > do just fine ;-). We however cannot say this about all potential > implementations. > > I am not sure if Dan Bonachea is a member of this list. I will write him a > message if he isn't. > > Thanks, > Vinod. > > ---------------------------------------- >> > From: thakur(a)mcs.anl.gov >> > To: mpi3-rma(a)lists.mpi-forum.org >> > Date: Sat, 6 Sep 2008 16:24:35 -0500 >> > Subject: Re: [Mpi3-rma] draft of a proposal for RMA interfaces >> > >> > Vinod, >> > Thanks for the proposal and the nice presentation. A question I had >> > was does it meet the needs of the Global Arrays library? That is, if this >> > were available in MPI today, would you implement Global Arrays with MPI and >> > nothing else (not ARMCI). Similarly, it would be good to run it by Dan >> > Bonachea and see what he has to say regarding implementing UPC. >> > >> > Regards, >> > Rajeev >> > >> > >>> >> -----Original Message----- >>> >> From: mpi3-rma-bounces(a)lists.mpi-forum.org >>> >> [mailto:[email protected]] On Behalf Of >>> >> Vinod tipparaju >>> >> Sent: Thursday, September 04, 2008 1:50 AM >>> >> To: mpi3-rma(a)lists.mpi-forum.org >>> >> Subject: [Mpi3-rma] draft of a proposal for RMA interfaces >>> >> >>> >> >>> >> Folks, >>> >> >>> >> Attached is a draft of a new proposal for MPI3-RMA interfaces >>> >> for your perusal. >>> >> >>> >> Thanks, >>> >> Vinod Tipparaju, ORNL. >>> >> >> > >> > _______________________________________________ >> > mpi3-rma mailing list >> > mpi3-rma(a)lists.mpi-forum.org >> > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma > > _______________________________________________ > mpi3-rma mailing list > mpi3-rma(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma >

1 0