October 2013 - mpiwg-rma - mailman3.cels.anl.gov

Re: [mpiwg-rma] pirate RMA revisited
by Jeff Hammond 31 Oct '13

31 Oct '13

I decided it was too much work to write my suggested changes in Trac markup since I'd have to redo in LaTeX eventually anyways so I just made them in LaTeX. https://svn.mpi-forum.org/trac/mpi-forum-web/ticket/398 The ticket says almost nothing. I will add to it later. For now, the attachments should be sufficient. Jeff -- Jeff Hammond jeff.science(a)gmail.com

1 0

Re: [mpiwg-rma] pirate RMA revisited
by Jeff Hammond 31 Oct '13

31 Oct '13

Read my emoticon. It said "the previous sentence is not meant to be taken seriously" :-) :-) :-) :-) :-) :-) :-) :-) :-) Jeff On Thu, Oct 31, 2013 at 2:50 PM, Pavan Balaji <balaji(a)mcs.anl.gov> wrote: > > Read my email. It didn’t say "I’ll fix it". It said, “don’t screw it up further”. > > On Oct 31, 2013, at 1:53 PM, Jeff Hammond <jeff.science(a)gmail.com> wrote: > >> Okay, I have no problem with adding all the pirate RMA functions - >> Rrput, Rraccumulate and Rrrget_accumulate - but I wanted to throw this >> out for discussion. I will create a ticket for this now for tracking >> purposes. >> >> I look forward to your ticket proposing {Scatter,Gather,Allgather}w >> (and the nonblocking and neighborhood variants) to give the MPI >> standard the symmetry it deserves :-) >> >> Jeff >> >> On Thu, Oct 31, 2013 at 1:46 PM, Pavan Balaji <balaji(a)mcs.anl.gov> wrote: >>> >>> I’d be opposed to making the standard more assymetric than what it already is. If you are adding a concept, it should be orthogonal to other functionality. Number of functions is not the criteria; number of concepts is. >>> >>> —- Pavan >>> >>> On Oct 31, 2013, at 1:40 PM, Jeff Hammond <jeff.science(a)gmail.com> wrote: >>> >>>> We've discussed this at MCS coffee hour but I wanted to post to the WG >>>> regarding the interface issue. >>>> >>>> I understand that the addition of Rrput, Rraccumulate, and >>>> Rrrget_accumulate (Rget is already fully pirated-out) may be >>>> unpalatable to some people. >>>> >>>> I propose that we add _only_ Rrrget_accumulate and tell users to >>>> implement Rrput and Rraccumulate in terms of it using MPI_NO_OP and - >>>> in the first case - MPI_REPLACE. The reasoning is by analogy with >>>> MPI_Alltoallw, which is sufficient to implement MPI_Scatterw, etc., >>>> albeit inefficiently. >>>> >>>> If we add only Rrrget_accumulate, the loss of efficiency will be >>>> comparatively quite small compared to W-collectives (and certainly not >>>> order-N buffer copies) and should only be due to a few unnecessary >>>> arguments and a couple of branches, which I argue is not sufficient to >>>> motivate the addition of two more functions to the standard. I don't >>>> believe that users want pirate RMA to optimize for latency... >>>> >>>> Is this reasonable? I'm hoping to present this at the Chicago Forum >>>> meeting if the WG believes we are sufficiently converged in our >>>> thinking. >>>> >>>> Thanks, >>>> >>>> Jeff >>>> >>>> On Sat, Jun 15, 2013 at 9:47 AM, Jim Dinan <james.dinan(a)gmail.com> wrote: >>>>> I think the pirate RMA (separate per operation local/remote completion) idea >>>>> is good. I'm not sure I'm sold on the interface; I'd rather not add so many >>>>> new functions, if there's some other way we could do it. I think we should >>>>> explore the different design options so that we can verify for the Forum >>>>> that we've chosen the best one. >>>> >>>> -- >>>> Jeff Hammond >>>> jeff.science(a)gmail.com >>>> _______________________________________________ >>>> mpiwg-rma mailing list >>>> mpiwg-rma(a)lists.mpi-forum.org >>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma >>> >>> -- >>> Pavan Balaji >>> http://www.mcs.anl.gov/~balaji >>> >>> _______________________________________________ >>> mpiwg-rma mailing list >>> mpiwg-rma(a)lists.mpi-forum.org >>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma >> >> >> >> -- >> Jeff Hammond >> jeff.science(a)gmail.com >> _______________________________________________ >> mpiwg-rma mailing list >> mpiwg-rma(a)lists.mpi-forum.org >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma > > -- > Pavan Balaji > http://www.mcs.anl.gov/~balaji > > _______________________________________________ > mpiwg-rma mailing list > mpiwg-rma(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma -- Jeff Hammond jeff.science(a)gmail.com

1 0

Re: [mpiwg-rma] pirate RMA revisited
by Pavan Balaji 31 Oct '13

31 Oct '13

Read my email. It didn’t say "I’ll fix it". It said, “don’t screw it up further”. On Oct 31, 2013, at 1:53 PM, Jeff Hammond <jeff.science(a)gmail.com> wrote: > Okay, I have no problem with adding all the pirate RMA functions - > Rrput, Rraccumulate and Rrrget_accumulate - but I wanted to throw this > out for discussion. I will create a ticket for this now for tracking > purposes. > > I look forward to your ticket proposing {Scatter,Gather,Allgather}w > (and the nonblocking and neighborhood variants) to give the MPI > standard the symmetry it deserves :-) > > Jeff > > On Thu, Oct 31, 2013 at 1:46 PM, Pavan Balaji <balaji(a)mcs.anl.gov> wrote: >> >> I’d be opposed to making the standard more assymetric than what it already is. If you are adding a concept, it should be orthogonal to other functionality. Number of functions is not the criteria; number of concepts is. >> >> —- Pavan >> >> On Oct 31, 2013, at 1:40 PM, Jeff Hammond <jeff.science(a)gmail.com> wrote: >> >>> We've discussed this at MCS coffee hour but I wanted to post to the WG >>> regarding the interface issue. >>> >>> I understand that the addition of Rrput, Rraccumulate, and >>> Rrrget_accumulate (Rget is already fully pirated-out) may be >>> unpalatable to some people. >>> >>> I propose that we add _only_ Rrrget_accumulate and tell users to >>> implement Rrput and Rraccumulate in terms of it using MPI_NO_OP and - >>> in the first case - MPI_REPLACE. The reasoning is by analogy with >>> MPI_Alltoallw, which is sufficient to implement MPI_Scatterw, etc., >>> albeit inefficiently. >>> >>> If we add only Rrrget_accumulate, the loss of efficiency will be >>> comparatively quite small compared to W-collectives (and certainly not >>> order-N buffer copies) and should only be due to a few unnecessary >>> arguments and a couple of branches, which I argue is not sufficient to >>> motivate the addition of two more functions to the standard. I don't >>> believe that users want pirate RMA to optimize for latency... >>> >>> Is this reasonable? I'm hoping to present this at the Chicago Forum >>> meeting if the WG believes we are sufficiently converged in our >>> thinking. >>> >>> Thanks, >>> >>> Jeff >>> >>> On Sat, Jun 15, 2013 at 9:47 AM, Jim Dinan <james.dinan(a)gmail.com> wrote: >>>> I think the pirate RMA (separate per operation local/remote completion) idea >>>> is good. I'm not sure I'm sold on the interface; I'd rather not add so many >>>> new functions, if there's some other way we could do it. I think we should >>>> explore the different design options so that we can verify for the Forum >>>> that we've chosen the best one. >>> >>> -- >>> Jeff Hammond >>> jeff.science(a)gmail.com >>> _______________________________________________ >>> mpiwg-rma mailing list >>> mpiwg-rma(a)lists.mpi-forum.org >>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma >> >> -- >> Pavan Balaji >> http://www.mcs.anl.gov/~balaji >> >> _______________________________________________ >> mpiwg-rma mailing list >> mpiwg-rma(a)lists.mpi-forum.org >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma > > > > -- > Jeff Hammond > jeff.science(a)gmail.com > _______________________________________________ > mpiwg-rma mailing list > mpiwg-rma(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma -- Pavan Balaji http://www.mcs.anl.gov/~balaji

1 0

Re: [mpiwg-rma] pirate RMA revisited
by Jeff Hammond 31 Oct '13

31 Oct '13

Okay, I have no problem with adding all the pirate RMA functions - Rrput, Rraccumulate and Rrrget_accumulate - but I wanted to throw this out for discussion. I will create a ticket for this now for tracking purposes. I look forward to your ticket proposing {Scatter,Gather,Allgather}w (and the nonblocking and neighborhood variants) to give the MPI standard the symmetry it deserves :-) Jeff On Thu, Oct 31, 2013 at 1:46 PM, Pavan Balaji <balaji(a)mcs.anl.gov> wrote: > > I’d be opposed to making the standard more assymetric than what it already is. If you are adding a concept, it should be orthogonal to other functionality. Number of functions is not the criteria; number of concepts is. > > —- Pavan > > On Oct 31, 2013, at 1:40 PM, Jeff Hammond <jeff.science(a)gmail.com> wrote: > >> We've discussed this at MCS coffee hour but I wanted to post to the WG >> regarding the interface issue. >> >> I understand that the addition of Rrput, Rraccumulate, and >> Rrrget_accumulate (Rget is already fully pirated-out) may be >> unpalatable to some people. >> >> I propose that we add _only_ Rrrget_accumulate and tell users to >> implement Rrput and Rraccumulate in terms of it using MPI_NO_OP and - >> in the first case - MPI_REPLACE. The reasoning is by analogy with >> MPI_Alltoallw, which is sufficient to implement MPI_Scatterw, etc., >> albeit inefficiently. >> >> If we add only Rrrget_accumulate, the loss of efficiency will be >> comparatively quite small compared to W-collectives (and certainly not >> order-N buffer copies) and should only be due to a few unnecessary >> arguments and a couple of branches, which I argue is not sufficient to >> motivate the addition of two more functions to the standard. I don't >> believe that users want pirate RMA to optimize for latency... >> >> Is this reasonable? I'm hoping to present this at the Chicago Forum >> meeting if the WG believes we are sufficiently converged in our >> thinking. >> >> Thanks, >> >> Jeff >> >> On Sat, Jun 15, 2013 at 9:47 AM, Jim Dinan <james.dinan(a)gmail.com> wrote: >>> I think the pirate RMA (separate per operation local/remote completion) idea >>> is good. I'm not sure I'm sold on the interface; I'd rather not add so many >>> new functions, if there's some other way we could do it. I think we should >>> explore the different design options so that we can verify for the Forum >>> that we've chosen the best one. >> >> -- >> Jeff Hammond >> jeff.science(a)gmail.com >> _______________________________________________ >> mpiwg-rma mailing list >> mpiwg-rma(a)lists.mpi-forum.org >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma > > -- > Pavan Balaji > http://www.mcs.anl.gov/~balaji > > _______________________________________________ > mpiwg-rma mailing list > mpiwg-rma(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma -- Jeff Hammond jeff.science(a)gmail.com

1 0

Re: [mpiwg-rma] pirate RMA revisited
by Pavan Balaji 31 Oct '13

31 Oct '13

I’d be opposed to making the standard more assymetric than what it already is. If you are adding a concept, it should be orthogonal to other functionality. Number of functions is not the criteria; number of concepts is. —- Pavan On Oct 31, 2013, at 1:40 PM, Jeff Hammond <jeff.science(a)gmail.com> wrote: > We've discussed this at MCS coffee hour but I wanted to post to the WG > regarding the interface issue. > > I understand that the addition of Rrput, Rraccumulate, and > Rrrget_accumulate (Rget is already fully pirated-out) may be > unpalatable to some people. > > I propose that we add _only_ Rrrget_accumulate and tell users to > implement Rrput and Rraccumulate in terms of it using MPI_NO_OP and - > in the first case - MPI_REPLACE. The reasoning is by analogy with > MPI_Alltoallw, which is sufficient to implement MPI_Scatterw, etc., > albeit inefficiently. > > If we add only Rrrget_accumulate, the loss of efficiency will be > comparatively quite small compared to W-collectives (and certainly not > order-N buffer copies) and should only be due to a few unnecessary > arguments and a couple of branches, which I argue is not sufficient to > motivate the addition of two more functions to the standard. I don't > believe that users want pirate RMA to optimize for latency... > > Is this reasonable? I'm hoping to present this at the Chicago Forum > meeting if the WG believes we are sufficiently converged in our > thinking. > > Thanks, > > Jeff > > On Sat, Jun 15, 2013 at 9:47 AM, Jim Dinan <james.dinan(a)gmail.com> wrote: >> I think the pirate RMA (separate per operation local/remote completion) idea >> is good. I'm not sure I'm sold on the interface; I'd rather not add so many >> new functions, if there's some other way we could do it. I think we should >> explore the different design options so that we can verify for the Forum >> that we've chosen the best one. > > -- > Jeff Hammond > jeff.science(a)gmail.com > _______________________________________________ > mpiwg-rma mailing list > mpiwg-rma(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma -- Pavan Balaji http://www.mcs.anl.gov/~balaji

1 0

pirate RMA revisited
by Jeff Hammond 31 Oct '13

31 Oct '13

We've discussed this at MCS coffee hour but I wanted to post to the WG regarding the interface issue. I understand that the addition of Rrput, Rraccumulate, and Rrrget_accumulate (Rget is already fully pirated-out) may be unpalatable to some people. I propose that we add _only_ Rrrget_accumulate and tell users to implement Rrput and Rraccumulate in terms of it using MPI_NO_OP and - in the first case - MPI_REPLACE. The reasoning is by analogy with MPI_Alltoallw, which is sufficient to implement MPI_Scatterw, etc., albeit inefficiently. If we add only Rrrget_accumulate, the loss of efficiency will be comparatively quite small compared to W-collectives (and certainly not order-N buffer copies) and should only be due to a few unnecessary arguments and a couple of branches, which I argue is not sufficient to motivate the addition of two more functions to the standard. I don't believe that users want pirate RMA to optimize for latency... Is this reasonable? I'm hoping to present this at the Chicago Forum meeting if the WG believes we are sufficiently converged in our thinking. Thanks, Jeff On Sat, Jun 15, 2013 at 9:47 AM, Jim Dinan <james.dinan(a)gmail.com> wrote: > I think the pirate RMA (separate per operation local/remote completion) idea > is good. I'm not sure I'm sold on the interface; I'd rather not add so many > new functions, if there's some other way we could do it. I think we should > explore the different design options so that we can verify for the Forum > that we've chosen the best one. -- Jeff Hammond jeff.science(a)gmail.com

1 0

Re: [mpiwg-rma] [EXTERNAL] Re: same_op_no_op and SHMEM
by Jim Dinan 24 Oct '13

24 Oct '13

Amongst our operations are such diverse elements as ... :) ~Jim. http://en.wikipedia.org/wiki/The_Spanish_Inquisition_(Monty_Python) On Thu, Oct 24, 2013 at 2:59 PM, Barrett, Brian W <bwbarre(a)sandia.gov>wrote: > I would have no objection to adding yet another info key. I think if we > keep at this for another year, we can make sure we have the longest > pre-defined info key in the spec. > > I admit to having very little medium term memory; which is the > type-homogeneity suggestion? > > Brian > > > On 10/24/13 12:53 PM, "Jeff Hammond" <jeff.science(a)gmail.com> wrote: > > >Honestly, I think REPLACE+NO_OP is a useful option, I just think > >REPLACE+NO_OP+<SUM or ...> is _also_ a useful option. Why don't we > >just turn our frowns upside down and add this to the standard? > > > >Does anyone object to info = same_op_no_op_replace? > > > >I would appreciate any feedback on my type-homogeneity suggestion as > >well. Do people agree that is worth adding? I imagine that there is > >hardware that can do e.g. sum+long but not sum+double, especially if > >it has to be atomic. > > > >Jeff > > > >On Thu, Oct 24, 2013 at 1:46 PM, Barrett, Brian W <bwbarre(a)sandia.gov> > >wrote: > >> Or "I will only do gets and compare-and-swap", which is something a > >>couple > >> of graph codes I've looked at do. But I agree, we probably should have > >> made it REPLACE, NO_OP, and 1 other op or not added it at all. Sigh :). > >> > >> Brian > >> > >> On 10/24/13 11:39 AM, "Jeff Hammond" <jeff.science(a)gmail.com> wrote: > >> > >>>I read same_op_no_op as "I will use only MPI_REPLACE and MPI_NO_OP" > >>>i.e. give me nothing more than atomic Put/Get, I do not want to > >>>actually accumulate anything. > >>> > >>>Jeff > >>> > >>>On Thu, Oct 24, 2013 at 12:34 PM, Underwood, Keith D > >>><keith.d.underwood(a)intel.com> wrote: > >>>> Yes, that's the motivation, but I'm not sure if anybody does atomics > >>>>without puts.... It seems to me like we should have included > >>>>MPI_REPLACE in that list > >>>> > >>>>> -----Original Message----- > >>>>> From: mpiwg-rma [mailto:[email protected]] On > >>>>> Behalf Of Pavan Balaji > >>>>> Sent: Thursday, October 24, 2013 1:33 PM > >>>>> To: MPI WG Remote Memory Access working group > >>>>> Subject: Re: [mpiwg-rma] same_op_no_op and SHMEM > >>>>> > >>>>> > >>>>> The motivation was that it's hard to maintain atomicity when > >>>>>different > >>>>> operations are done. For example, if the hardware only supports some > >>>>> atomic operations, but not all, this might result in some operations > >>>>> happening in hardware and some in software making atomicity hard. In > >>>>>such > >>>>> cases, the MPI implementation might need to fall back to > >>>>>software-only > >>>>> implementations. > >>>>> > >>>>> -- Pavan > >>>>> > >>>>> On Oct 24, 2013, at 12:23 PM, Jeff Hammond <jeff.science(a)gmail.com> > >>>>> wrote: > >>>>> > >>>>> > I recall that Brian and/or Keith wanted same_op_no_op because of > >>>>> > SHMEM. However, SHMEM requires the use of MPI_NO_OP (for atomic > >>>>> Get > >>>>> > via Get_accumulate), MPI_REPLACE (for atomic Put via Accumulate) > >>>>>and > >>>>> > MPI_SUM (for add, fadd, inc and finc). So what is the benefit of > >>>>> > same_op_no_op to SHMEM? Perhaps I remember completely wrong and > >>>>> the > >>>>> > motivation was something that does not use the latter atomics. Or > >>>>> > perhaps it is common for SHMEM codes to not use these and thus the > >>>>> > assumption is MPI_SUM can be ignored. > >>>>> > > >>>>> > Jeff > >>>>> > > >>>>> > -- > >>>>> > Jeff Hammond > >>>>> > jeff.science(a)gmail.com > >>>>> > _______________________________________________ > >>>>> > mpiwg-rma mailing list > >>>>> > mpiwg-rma(a)lists.mpi-forum.org > >>>>> > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma > >>>>> > >>>>> -- > >>>>> Pavan Balaji > >>>>> http://www.mcs.anl.gov/~balaji > >>>>> > >>>>> _______________________________________________ > >>>>> mpiwg-rma mailing list > >>>>> mpiwg-rma(a)lists.mpi-forum.org > >>>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma > >>>> _______________________________________________ > >>>> mpiwg-rma mailing list > >>>> mpiwg-rma(a)lists.mpi-forum.org > >>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma > >>> > >>> > >>> > >>>-- > >>>Jeff Hammond > >>>jeff.science(a)gmail.com > >>>_______________________________________________ > >>>mpiwg-rma mailing list > >>>mpiwg-rma(a)lists.mpi-forum.org > >>>http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma > >>> > >> > >> > >> -- > >> Brian W. Barrett > >> Scalable System Software Group > >> Sandia National Laboratories > >> > >> > >> > >> > >> _______________________________________________ > >> mpiwg-rma mailing list > >> mpiwg-rma(a)lists.mpi-forum.org > >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma > > > > > > > >-- > >Jeff Hammond > >jeff.science(a)gmail.com > >_______________________________________________ > >mpiwg-rma mailing list > >mpiwg-rma(a)lists.mpi-forum.org > >http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma > > > > > -- > Brian W. Barrett > Scalable System Software Group > Sandia National Laboratories > > > > > _______________________________________________ > mpiwg-rma mailing list > mpiwg-rma(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma >

1 0

Re: [mpiwg-rma] [EXTERNAL] Re: same_op_no_op and SHMEM
by Barrett, Brian W 24 Oct '13

24 Oct '13

On 10/24/13 1:24 PM, "Jeff Hammond" <jeff.science(a)gmail.com> wrote: >> I would have no objection to adding yet another info key. I think if we >> keep at this for another year, we can make sure we have the longest >> pre-defined info key in the spec. > >Or we can focus on the values. Instead of "same_op_no_op_replace = >true" we can have "op_list = no_op,replace,sum" and let the >implementation instantiate same_op_no_op_replace on its own. The advantage here would be that if the bad thing is FP divide, I can list no_op,sum,replace,max,min and still have goodness. The disadvantage would be all the test to list the options. So maybe long true/false keys have some advantage. >> I admit to having very little medium term memory; which is the >> type-homogeneity suggestion? > >Medium-term meaning in the last 2 hours? :-) Fine, no memory at all. >It addresses the lack of type-specification in MPI window creating. >If I can say at window allocation time that I only want to use >MPI_INT, then maybe the implementation can avoid a software >implementation that might be required if I was to use MPI_FLOAT, for >example. In my experience, NICs are more likely to support >fixed-point arithmetic in hardware than floating-point. As an info key? I think that would be helpful for exactly the reason you said. If I know the user isn't going to use long double complex data types, that makes life a bit easier :). Brian -- Brian W. Barrett Scalable System Software Group Sandia National Laboratories

1 0

Re: [mpiwg-rma] [EXTERNAL] Re: same_op_no_op and SHMEM
by Jeff Hammond 24 Oct '13

24 Oct '13

> I would have no objection to adding yet another info key. I think if we > keep at this for another year, we can make sure we have the longest > pre-defined info key in the spec. Or we can focus on the values. Instead of "same_op_no_op_replace = true" we can have "op_list = no_op,replace,sum" and let the implementation instantiate same_op_no_op_replace on its own. However, the German method of long-word generation (Deutschverfahrenlangewortschöpfung for those keeping score at home*) has some appeal to me :-) > I admit to having very little medium term memory; which is the > type-homogeneity suggestion? Medium-term meaning in the last 2 hours? :-) It addresses the lack of type-specification in MPI window creating. If I can say at window allocation time that I only want to use MPI_INT, then maybe the implementation can avoid a software implementation that might be required if I was to use MPI_FLOAT, for example. In my experience, NICs are more likely to support fixed-point arithmetic in hardware than floating-point. Jeff * I made this up. And $20 says Jim will call me on it. > On 10/24/13 12:53 PM, "Jeff Hammond" <jeff.science(a)gmail.com> wrote: > >>Honestly, I think REPLACE+NO_OP is a useful option, I just think >>REPLACE+NO_OP+<SUM or ...> is _also_ a useful option. Why don't we >>just turn our frowns upside down and add this to the standard? >> >>Does anyone object to info = same_op_no_op_replace? >> >>I would appreciate any feedback on my type-homogeneity suggestion as >>well. Do people agree that is worth adding? I imagine that there is >>hardware that can do e.g. sum+long but not sum+double, especially if >>it has to be atomic. >> >>Jeff >> >>On Thu, Oct 24, 2013 at 1:46 PM, Barrett, Brian W <bwbarre(a)sandia.gov> >>wrote: >>> Or "I will only do gets and compare-and-swap", which is something a >>>couple >>> of graph codes I've looked at do. But I agree, we probably should have >>> made it REPLACE, NO_OP, and 1 other op or not added it at all. Sigh :). >>> >>> Brian >>> >>> On 10/24/13 11:39 AM, "Jeff Hammond" <jeff.science(a)gmail.com> wrote: >>> >>>>I read same_op_no_op as "I will use only MPI_REPLACE and MPI_NO_OP" >>>>i.e. give me nothing more than atomic Put/Get, I do not want to >>>>actually accumulate anything. >>>> >>>>Jeff >>>> >>>>On Thu, Oct 24, 2013 at 12:34 PM, Underwood, Keith D >>>><keith.d.underwood(a)intel.com> wrote: >>>>> Yes, that's the motivation, but I'm not sure if anybody does atomics >>>>>without puts.... It seems to me like we should have included >>>>>MPI_REPLACE in that list >>>>> >>>>>> -----Original Message----- >>>>>> From: mpiwg-rma [mailto:[email protected]] On >>>>>> Behalf Of Pavan Balaji >>>>>> Sent: Thursday, October 24, 2013 1:33 PM >>>>>> To: MPI WG Remote Memory Access working group >>>>>> Subject: Re: [mpiwg-rma] same_op_no_op and SHMEM >>>>>> >>>>>> >>>>>> The motivation was that it's hard to maintain atomicity when >>>>>>different >>>>>> operations are done. For example, if the hardware only supports some >>>>>> atomic operations, but not all, this might result in some operations >>>>>> happening in hardware and some in software making atomicity hard. In >>>>>>such >>>>>> cases, the MPI implementation might need to fall back to >>>>>>software-only >>>>>> implementations. >>>>>> >>>>>> -- Pavan >>>>>> >>>>>> On Oct 24, 2013, at 12:23 PM, Jeff Hammond <jeff.science(a)gmail.com> >>>>>> wrote: >>>>>> >>>>>> > I recall that Brian and/or Keith wanted same_op_no_op because of >>>>>> > SHMEM. However, SHMEM requires the use of MPI_NO_OP (for atomic >>>>>> Get >>>>>> > via Get_accumulate), MPI_REPLACE (for atomic Put via Accumulate) >>>>>>and >>>>>> > MPI_SUM (for add, fadd, inc and finc). So what is the benefit of >>>>>> > same_op_no_op to SHMEM? Perhaps I remember completely wrong and >>>>>> the >>>>>> > motivation was something that does not use the latter atomics. Or >>>>>> > perhaps it is common for SHMEM codes to not use these and thus the >>>>>> > assumption is MPI_SUM can be ignored. >>>>>> > >>>>>> > Jeff >>>>>> > >>>>>> > -- >>>>>> > Jeff Hammond >>>>>> > jeff.science(a)gmail.com >>>>>> > _______________________________________________ >>>>>> > mpiwg-rma mailing list >>>>>> > mpiwg-rma(a)lists.mpi-forum.org >>>>>> > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma >>>>>> >>>>>> -- >>>>>> Pavan Balaji >>>>>> http://www.mcs.anl.gov/~balaji >>>>>> >>>>>> _______________________________________________ >>>>>> mpiwg-rma mailing list >>>>>> mpiwg-rma(a)lists.mpi-forum.org >>>>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma >>>>> _______________________________________________ >>>>> mpiwg-rma mailing list >>>>> mpiwg-rma(a)lists.mpi-forum.org >>>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma >>>> >>>> >>>> >>>>-- >>>>Jeff Hammond >>>>jeff.science(a)gmail.com >>>>_______________________________________________ >>>>mpiwg-rma mailing list >>>>mpiwg-rma(a)lists.mpi-forum.org >>>>http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma >>>> >>> >>> >>> -- >>> Brian W. Barrett >>> Scalable System Software Group >>> Sandia National Laboratories >>> >>> >>> >>> >>> _______________________________________________ >>> mpiwg-rma mailing list >>> mpiwg-rma(a)lists.mpi-forum.org >>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma >> >> >> >>-- >>Jeff Hammond >>jeff.science(a)gmail.com >>_______________________________________________ >>mpiwg-rma mailing list >>mpiwg-rma(a)lists.mpi-forum.org >>http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma >> > > > -- > Brian W. Barrett > Scalable System Software Group > Sandia National Laboratories > > > > > _______________________________________________ > mpiwg-rma mailing list > mpiwg-rma(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma -- Jeff Hammond jeff.science(a)gmail.com

1 0

Re: [mpiwg-rma] [EXTERNAL] Re: same_op_no_op and SHMEM
by Barrett, Brian W 24 Oct '13

24 Oct '13

I would have no objection to adding yet another info key. I think if we keep at this for another year, we can make sure we have the longest pre-defined info key in the spec. I admit to having very little medium term memory; which is the type-homogeneity suggestion? Brian On 10/24/13 12:53 PM, "Jeff Hammond" <jeff.science(a)gmail.com> wrote: >Honestly, I think REPLACE+NO_OP is a useful option, I just think >REPLACE+NO_OP+<SUM or ...> is _also_ a useful option. Why don't we >just turn our frowns upside down and add this to the standard? > >Does anyone object to info = same_op_no_op_replace? > >I would appreciate any feedback on my type-homogeneity suggestion as >well. Do people agree that is worth adding? I imagine that there is >hardware that can do e.g. sum+long but not sum+double, especially if >it has to be atomic. > >Jeff > >On Thu, Oct 24, 2013 at 1:46 PM, Barrett, Brian W <bwbarre(a)sandia.gov> >wrote: >> Or "I will only do gets and compare-and-swap", which is something a >>couple >> of graph codes I've looked at do. But I agree, we probably should have >> made it REPLACE, NO_OP, and 1 other op or not added it at all. Sigh :). >> >> Brian >> >> On 10/24/13 11:39 AM, "Jeff Hammond" <jeff.science(a)gmail.com> wrote: >> >>>I read same_op_no_op as "I will use only MPI_REPLACE and MPI_NO_OP" >>>i.e. give me nothing more than atomic Put/Get, I do not want to >>>actually accumulate anything. >>> >>>Jeff >>> >>>On Thu, Oct 24, 2013 at 12:34 PM, Underwood, Keith D >>><keith.d.underwood(a)intel.com> wrote: >>>> Yes, that's the motivation, but I'm not sure if anybody does atomics >>>>without puts.... It seems to me like we should have included >>>>MPI_REPLACE in that list >>>> >>>>> -----Original Message----- >>>>> From: mpiwg-rma [mailto:[email protected]] On >>>>> Behalf Of Pavan Balaji >>>>> Sent: Thursday, October 24, 2013 1:33 PM >>>>> To: MPI WG Remote Memory Access working group >>>>> Subject: Re: [mpiwg-rma] same_op_no_op and SHMEM >>>>> >>>>> >>>>> The motivation was that it's hard to maintain atomicity when >>>>>different >>>>> operations are done. For example, if the hardware only supports some >>>>> atomic operations, but not all, this might result in some operations >>>>> happening in hardware and some in software making atomicity hard. In >>>>>such >>>>> cases, the MPI implementation might need to fall back to >>>>>software-only >>>>> implementations. >>>>> >>>>> -- Pavan >>>>> >>>>> On Oct 24, 2013, at 12:23 PM, Jeff Hammond <jeff.science(a)gmail.com> >>>>> wrote: >>>>> >>>>> > I recall that Brian and/or Keith wanted same_op_no_op because of >>>>> > SHMEM. However, SHMEM requires the use of MPI_NO_OP (for atomic >>>>> Get >>>>> > via Get_accumulate), MPI_REPLACE (for atomic Put via Accumulate) >>>>>and >>>>> > MPI_SUM (for add, fadd, inc and finc). So what is the benefit of >>>>> > same_op_no_op to SHMEM? Perhaps I remember completely wrong and >>>>> the >>>>> > motivation was something that does not use the latter atomics. Or >>>>> > perhaps it is common for SHMEM codes to not use these and thus the >>>>> > assumption is MPI_SUM can be ignored. >>>>> > >>>>> > Jeff >>>>> > >>>>> > -- >>>>> > Jeff Hammond >>>>> > jeff.science(a)gmail.com >>>>> > _______________________________________________ >>>>> > mpiwg-rma mailing list >>>>> > mpiwg-rma(a)lists.mpi-forum.org >>>>> > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma >>>>> >>>>> -- >>>>> Pavan Balaji >>>>> http://www.mcs.anl.gov/~balaji >>>>> >>>>> _______________________________________________ >>>>> mpiwg-rma mailing list >>>>> mpiwg-rma(a)lists.mpi-forum.org >>>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma >>>> _______________________________________________ >>>> mpiwg-rma mailing list >>>> mpiwg-rma(a)lists.mpi-forum.org >>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma >>> >>> >>> >>>-- >>>Jeff Hammond >>>jeff.science(a)gmail.com >>>_______________________________________________ >>>mpiwg-rma mailing list >>>mpiwg-rma(a)lists.mpi-forum.org >>>http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma >>> >> >> >> -- >> Brian W. Barrett >> Scalable System Software Group >> Sandia National Laboratories >> >> >> >> >> _______________________________________________ >> mpiwg-rma mailing list >> mpiwg-rma(a)lists.mpi-forum.org >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma > > > >-- >Jeff Hammond >jeff.science(a)gmail.com >_______________________________________________ >mpiwg-rma mailing list >mpiwg-rma(a)lists.mpi-forum.org >http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma > -- Brian W. Barrett Scalable System Software Group Sandia National Laboratories

1 0