July 2012 - mpi-forum - mailman3.cels.anl.gov

Re: [Mpi-forum] [Mpi3-rma] [EXTERNAL] Re: Disp_unit in allocate_shared should be returned by win_shared_query?
by Torsten Hoefler 22 Jul '12

22 Jul '12

On Sun, Jul 22, 2012 at 08:30:26PM +0200, Rolf Rabenseifner wrote: > Torsten and Bill, > > Torsten: > > I would agree to Bill. We should have an info argument allowing the user > > to state that all displ_units are the same like we have for count :-). > > A topic for MPI-3.1! > > Yes, it is not needed for MPI-3.0, because with two Allreduces with > MIN and MAX of disp_unit, all processes know in O(log(p)) whether > it is the same value in all processes. Sure, this is trivial but I still like an O(1) (zero communication) info argument better :-). > Bill, yes, here a draft: > > Advice to users. If an application should scale to a large > number of processes, then it is recommended for scalability reasons > that all processes provide the same disp_unit argument > on all processes. End of advice to users. I think this does not belong into the standard. If we start those things then we need to add the same for the size and dozens of other "collective" arguments (think compressibility of graphs etc.). I also think that we're digressing :-). We still didn't decide if the WG likes the displ in the query function or not. Best, Torsten -- ### qreharg rug ebs fv crryF ------------- http://www.unixer.de/ ----- Torsten Hoefler | Performance Modeling and Simulation Lead Blue Waters Directorate | University of Illinois (UIUC) 1205 W Clark Street | Urbana, IL, 61801 NCSA Building | +01 (217) 244-7736

1 0

Re: [Mpi-forum] [Mpi3-rma] [EXTERNAL] Re: Disp_unit in allocate_shared should be returned by win_shared_query?
by Rolf Rabenseifner 22 Jul '12

22 Jul '12

Torsten and Bill, Torsten: > I would agree to Bill. We should have an info argument allowing the user > to state that all displ_units are the same like we have for count :-). > A topic for MPI-3.1! Yes, it is not needed for MPI-3.0, because with two Allreduces with MIN and MAX of disp_unit, all processes know in O(log(p)) whether it is the same value in all processes. Bill, yes, here a draft: Advice to users. If an application should scale to a large number of processes, then it is recommended for scalability reasons that all processes provide the same disp_unit argument on all processes. End of advice to users. >>> Advice to implementors. If the user provides the same >>> disp_unit argument on all processes of the communicator, or a compressible list of values, it is recommended for >>> scalability reasons to store this data as compact as possible. >>> End of advice to users. (I substituted necessary by recommended) Is this text now better? Best regards Rolf ----- Original Message ----- > From: "Torsten Hoefler" <htor(a)illinois.edu> > To: "Main MPI Forum mailing list" <mpi-forum(a)lists.mpi-forum.org> > Cc: "MPI 3.0 Remote Memory Access working group" <mpi3-rma(a)lists.mpi-forum.org> > Sent: Sunday, July 22, 2012 8:06:55 PM > Subject: Re: [Mpi-forum] [Mpi3-rma] [EXTERNAL] Re: Disp_unit in allocate_shared should be returned by > win_shared_query? > I would agree to Bill. We should have an info argument allowing the > user > to state that all displ_units are the same like we have for count :-). > A > topic for MPI-3.1! > > Thanks, > Torsten > > On Sun, Jul 22, 2012 at 01:03:39PM -0500, William Gropp wrote: > > I think this is a little strong. This isn't necessary for the vast > > majority of MPI users; just for those at extreme scale. A > > modification that restricted this to extreme scale might be > > acceptable. > > > > Bill > > > > William Gropp > > Director, Parallel Computing Institute > > Deputy Director for Research > > Institute for Advanced Computing Applications and Technologies > > Paul and Cynthia Saylor Professor of Computer Science > > University of Illinois Urbana-Champaign > > > > > > > > On Jul 22, 2012, at 12:28 PM, Rolf Rabenseifner wrote: > > > > > On Wed, Jul 18, 2012 at 08:11:23PM +0000, Barrett, Brian W wrote: > > >> I'm concerned with the change (and don't think it should happen > > >> at this point, as I want to be able to tell other people their > > >> changes are too big). It's probably not another O(p) structure, > > >> since a process will have to know the other rank's disp_unit to > > >> do the memcpy during the communication operations, so we crossed > > >> the O(p) bridge when we added disp_unit to allocate_shared. > > > > > > I would recommend to add two advices: > > > > > > Advice to users. For scalability reasons it is recommended > > > that all processes provide the same disp_unit argument > > > on all processes. End of advice to users. > > > > > > Advice to implementors. If the user provides the same > > > disp_unit argument on all processes of the communicator, > > > or a compressible list of values, it is necessary for > > > scalability reasons to store this data as compact as possible. > > > End of advice to users. > > > > > > Of course for all thre MPI_WIN_CREATE, MPI_WIN_AALOCATE, and > > > MPI_WIN_ALLOCATE_SHARED. > > > > > > Is this a good idea? > > > If yes, can someone find a better wording for this proposal? > > > > > > Best regards > > > Rolf > > > > > > ----- Original Message ----- > > >> From: "William Gropp" <wgropp(a)illinois.edu> > > >> To: "MPI 3.0 Remote Memory Access working group" > > >> <mpi3-rma(a)lists.mpi-forum.org> > > >> Sent: Sunday, July 22, 2012 7:12:27 PM > > >> Subject: Re: [Mpi3-rma] [EXTERNAL] Re: Disp_unit in > > >> allocate_shared should be returned by win_shared_query? > > >> What *exactly* are the proposals? I know in general, but some of > > >> the > > >> statements make me wonder if I understand the proposal. > > >> > > >> Bill > > >> > > >> William Gropp > > >> Director, Parallel Computing Institute > > >> Deputy Director for Research > > >> Institute for Advanced Computing Applications and Technologies > > >> Paul and Cynthia Saylor Professor of Computer Science > > >> University of Illinois Urbana-Champaign > > >> > > >> > > >> > > >> On Jul 22, 2012, at 10:55 AM, Torsten Hoefler wrote: > > >> > > >>> Dear All, > > >>> > > >>> I think we should discuss this more. Right now, we still have > > >>> more > > >>> votes > > >>> in favor of this than against. The change is still committed and > > >>> part of > > >>> the "official release". The deadline is 7/26. > > >>> > > >>> Please advise what to do! > > >>> > > >>> Thanks, > > >>> Torsten > > >>> > > >>> On Wed, Jul 18, 2012 at 08:11:23PM +0000, Barrett, Brian W > > >>> wrote: > > >>>> I'm concerned with the change (and don't think it should happen > > >>>> at > > >>>> this point, as I want to be able to tell other people their > > >>>> changes > > >>>> are too big). It's probably not another O(p) structure, since a > > >>>> process will have to know the other rank's disp_unit to do the > > >>>> memcpy during the communication operations, so we crossed the > > >>>> O(p) > > >>>> bridge when we added disp_unit to allocate_shared. > > >>>> > > >>>> > > >>>> Brian > > >>>> > > >>>> -- > > >>>> Brian W. Barrett > > >>>> Scalable System Software Group > > >>>> Sandia National Laboratories > > >>>> ________________________________________ > > >>>> From: mpi3-rma-bounces(a)lists.mpi-forum.org > > >>>> [mpi3-rma-bounces(a)lists.mpi-forum.org] on behalf of Jim Dinan > > >>>> [dinan(a)mcs.anl.gov] > > >>>> Sent: Wednesday, July 18, 2012 12:46 PM > > >>>> To: mpi3-rma(a)lists.mpi-forum.org > > >>>> Subject: [EXTERNAL] Re: [Mpi3-rma] Disp_unit in allocate_shared > > >>>> should be returned by win_shared_query? > > >>>> > > >>>> It's not inherently damaging, but it requires us to > > >>>> maintain/provide > > >>>> access to another O(p) structure and suggests to users that > > >>>> another > > >>>> process' disp_unit is useful information, which is incorrect. > > >>>> IIRC, > > >>>> we > > >>>> did discuss this in the working group and intentionally omitted > > >>>> it. > > >>>> > > >>>> ~Jim. > > >>>> > > >>>> On 7/18/12 1:38 PM, Torsten Hoefler wrote: > > >>>>> On Wed, Jul 18, 2012 at 01:18:53PM -0500, James Dinan wrote: > > >>>>>> On 7/18/12 1:06 PM, Torsten Hoefler wrote: > > >>>>>>> On Wed, Jul 18, 2012 at 01:08:43PM -0500, Pavan Balaji > > >>>>>>> wrote: > > >>>>>>>> > > >>>>>>>> On 07/18/2012 01:02 PM, Torsten Hoefler wrote: > > >>>>>>>>> Jim, > > >>>>>>>>>> Is one process' disp_unit meaningful at another process? > > >>>>>>>>>> I > > >>>>>>>>>> thought that > > >>>>>>>>>> this is not the case. MPI provides no other routines to > > >>>>>>>>>> query > > >>>>>>>>>> another > > >>>>>>>>>> process' disp_unit. > > >>>>>>>>> It is something that the user is passing in. I don't know > > >>>>>>>>> of a > > >>>>>>>>> particular use-case but we can query all other arguments > > >>>>>>>>> that > > >>>>>>>>> each > > >>>>>>>>> process passed. One more integer seemed safe to me. > > >>>>>>>>> > > >>>>>>>>>> Also, what meaning does a disp_unit have in the context > > >>>>>>>>>> of > > >>>>>>>>>> load/store > > >>>>>>>>>> operations that will result from win_shared_query? > > >>>>>>>>> Nothing. > > >>>>>>>>> > > >>>>>>>>>> Maybe I'm missing something, but this doesn't seem > > >>>>>>>>>> necessary. > > >>>>>>>>> Well, it's better to have it than to introduce it with a > > >>>>>>>>> different name > > >>>>>>>>> later. That was my main motivation. One could also use it > > >>>>>>>>> in a > > >>>>>>>>> library > > >>>>>>>>> context to query. But that's weak, I know. > > >>>>>>>> > > >>>>>>>> This doesn't seem critical, and can always be added in > > >>>>>>>> MPI-3.1 > > >>>>>>>> if we > > >>>>>>>> find it necessary. No reason to rush it. > > >>>>>>> > > >>>>>>> It cannot be added without a new function name. That was the > > >>>>>>> concern > > >>>>>>> here. Adding it seems less risk than not adding it. In > > >>>>>>> addition, > > >>>>>>> it is > > >>>>>>> already added ;-). > > >>>>>> > > >>>>>> I am concerned that this is a harmful change. The disp_unit > > >>>>>> from > > >>>>>> one > > >>>>>> process has no meaningful use at another process. If the user > > >>>>>> wants to > > >>>>>> know it for some reason, they can use a separate > > >>>>>> communication > > >>>>>> operation. > > >>>>> I believe the potential damage is very limited. But we can > > >>>>> have a > > >>>>> vote. > > >>>>> > > >>>>> So far it's still yes: 5, no: 1 > > >>>>> > > >>>>> I'm just mediating, not making decisions. > > >>>>> > > >>>>> Torsten > > >>>>> > > >>>> > > >>>> _______________________________________________ > > >>>> mpi3-rma mailing list > > >>>> mpi3-rma(a)lists.mpi-forum.org > > >>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma > > >>>> > > >>>> > > >>>> > > >>>> _______________________________________________ > > >>>> mpi3-rma mailing list > > >>>> mpi3-rma(a)lists.mpi-forum.org > > >>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma > > >>>> > > >>> > > >>> -- > > >>> ### qreharg rug ebs fv crryF ------------- http://www.unixer.de/ > > >>> ----- > > >>> Torsten Hoefler | Performance Modeling and Simulation Lead > > >>> Blue Waters Directorate | University of Illinois (UIUC) > > >>> 1205 W Clark Street | Urbana, IL, 61801 > > >>> NCSA Building | +01 (217) 244-7736 > > >>> _______________________________________________ > > >>> mpi3-rma mailing list > > >>> mpi3-rma(a)lists.mpi-forum.org > > >>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma > > >> > > >> > > >> _______________________________________________ > > >> mpi3-rma mailing list > > >> mpi3-rma(a)lists.mpi-forum.org > > >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma > > > > > > -- > > > Dr. Rolf Rabenseifner . . . . . . . . . .. email > > > rabenseifner(a)hlrs.de > > > High Performance Computing Center (HLRS) . phone > > > ++49(0)711/685-65530 > > > University of Stuttgart . . . . . . . . .. fax ++49(0)711 / > > > 685-65832 > > > Head of Dpmt Parallel Computing . . . > > > www.hlrs.de/people/rabenseifner > > > Nobelstr. 19, D-70550 Stuttgart, Germany . (Office: Allmandring > > > 30) > > > _______________________________________________ > > > mpi-forum mailing list > > > mpi-forum(a)lists.mpi-forum.org > > > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum > > > > > > _______________________________________________ > > mpi-forum mailing list > > mpi-forum(a)lists.mpi-forum.org > > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum > > > > -- > ### qreharg rug ebs fv crryF ------------- http://www.unixer.de/ ----- > Torsten Hoefler | Performance Modeling and Simulation Lead > Blue Waters Directorate | University of Illinois (UIUC) > 1205 W Clark Street | Urbana, IL, 61801 > NCSA Building | +01 (217) 244-7736 > _______________________________________________ > mpi-forum mailing list > mpi-forum(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum -- Dr. Rolf Rabenseifner . . . . . . . . . .. email rabenseifner(a)hlrs.de High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner Nobelstr. 19, D-70550 Stuttgart, Germany . (Office: Allmandring 30)

1 0

Re: [Mpi-forum] [Mpi3-rma] [EXTERNAL] Re: Disp_unit in allocate_shared should be returned by win_shared_query?
by William Gropp 22 Jul '12

22 Jul '12

Good point - but what this tells me is that we should leave this out - its something that isn't necessary in specifying the standard, and enumerating the cases where it might apply is non-trivial. Bill William Gropp Director, Parallel Computing Institute Deputy Director for Research Institute for Advanced Computing Applications and Technologies Paul and Cynthia Saylor Professor of Computer Science University of Illinois Urbana-Champaign On Jul 22, 2012, at 1:12 PM, Jeff Hammond wrote: > I think "s/at extreme scale/when memory is limited/". An adapteva > chip has 32KB SRAM per core and is pushing 100 cores and farther, so > having >128*4 bytes used in the window object is no different than > having >1 of 64 GB used for the window if one has 2^29 ranks. > > Jeff > > On Sun, Jul 22, 2012 at 1:03 PM, William Gropp <wgropp(a)illinois.edu> wrote: >> I think this is a little strong. This isn't necessary for the vast majority of MPI users; just for those at extreme scale. A modification that restricted this to extreme scale might be acceptable. >> >> Bill >> >> William Gropp >> Director, Parallel Computing Institute >> Deputy Director for Research >> Institute for Advanced Computing Applications and Technologies >> Paul and Cynthia Saylor Professor of Computer Science >> University of Illinois Urbana-Champaign >> >> >> >> On Jul 22, 2012, at 12:28 PM, Rolf Rabenseifner wrote: >> >>> On Wed, Jul 18, 2012 at 08:11:23PM +0000, Barrett, Brian W wrote: >>>> I'm concerned with the change (and don't think it should happen >>>> at this point, as I want to be able to tell other people their >>>> changes are too big). It's probably not another O(p) structure, >>>> since a process will have to know the other rank's disp_unit to >>>> do the memcpy during the communication operations, so we crossed >>>> the O(p) bridge when we added disp_unit to allocate_shared. >>> >>> I would recommend to add two advices: >>> >>> Advice to users. For scalability reasons it is recommended >>> that all processes provide the same disp_unit argument >>> on all processes. End of advice to users. >>> >>> Advice to implementors. If the user provides the same >>> disp_unit argument on all processes of the communicator, >>> or a compressible list of values, it is necessary for >>> scalability reasons to store this data as compact as possible. >>> End of advice to users. >>> >>> Of course for all thre MPI_WIN_CREATE, MPI_WIN_AALOCATE, and >>> MPI_WIN_ALLOCATE_SHARED. >>> >>> Is this a good idea? >>> If yes, can someone find a better wording for this proposal? >>> >>> Best regards >>> Rolf >>> >>> ----- Original Message ----- >>>> From: "William Gropp" <wgropp(a)illinois.edu> >>>> To: "MPI 3.0 Remote Memory Access working group" <mpi3-rma(a)lists.mpi-forum.org> >>>> Sent: Sunday, July 22, 2012 7:12:27 PM >>>> Subject: Re: [Mpi3-rma] [EXTERNAL] Re: Disp_unit in allocate_shared should be returned by win_shared_query? >>>> What *exactly* are the proposals? I know in general, but some of the >>>> statements make me wonder if I understand the proposal. >>>> >>>> Bill >>>> >>>> William Gropp >>>> Director, Parallel Computing Institute >>>> Deputy Director for Research >>>> Institute for Advanced Computing Applications and Technologies >>>> Paul and Cynthia Saylor Professor of Computer Science >>>> University of Illinois Urbana-Champaign >>>> >>>> >>>> >>>> On Jul 22, 2012, at 10:55 AM, Torsten Hoefler wrote: >>>> >>>>> Dear All, >>>>> >>>>> I think we should discuss this more. Right now, we still have more >>>>> votes >>>>> in favor of this than against. The change is still committed and >>>>> part of >>>>> the "official release". The deadline is 7/26. >>>>> >>>>> Please advise what to do! >>>>> >>>>> Thanks, >>>>> Torsten >>>>> >>>>> On Wed, Jul 18, 2012 at 08:11:23PM +0000, Barrett, Brian W wrote: >>>>>> I'm concerned with the change (and don't think it should happen at >>>>>> this point, as I want to be able to tell other people their changes >>>>>> are too big). It's probably not another O(p) structure, since a >>>>>> process will have to know the other rank's disp_unit to do the >>>>>> memcpy during the communication operations, so we crossed the O(p) >>>>>> bridge when we added disp_unit to allocate_shared. >>>>>> >>>>>> >>>>>> Brian >>>>>> >>>>>> -- >>>>>> Brian W. Barrett >>>>>> Scalable System Software Group >>>>>> Sandia National Laboratories >>>>>> ________________________________________ >>>>>> From: mpi3-rma-bounces(a)lists.mpi-forum.org >>>>>> [mpi3-rma-bounces(a)lists.mpi-forum.org] on behalf of Jim Dinan >>>>>> [dinan(a)mcs.anl.gov] >>>>>> Sent: Wednesday, July 18, 2012 12:46 PM >>>>>> To: mpi3-rma(a)lists.mpi-forum.org >>>>>> Subject: [EXTERNAL] Re: [Mpi3-rma] Disp_unit in allocate_shared >>>>>> should be returned by win_shared_query? >>>>>> >>>>>> It's not inherently damaging, but it requires us to >>>>>> maintain/provide >>>>>> access to another O(p) structure and suggests to users that another >>>>>> process' disp_unit is useful information, which is incorrect. IIRC, >>>>>> we >>>>>> did discuss this in the working group and intentionally omitted it. >>>>>> >>>>>> ~Jim. >>>>>> >>>>>> On 7/18/12 1:38 PM, Torsten Hoefler wrote: >>>>>>> On Wed, Jul 18, 2012 at 01:18:53PM -0500, James Dinan wrote: >>>>>>>> On 7/18/12 1:06 PM, Torsten Hoefler wrote: >>>>>>>>> On Wed, Jul 18, 2012 at 01:08:43PM -0500, Pavan Balaji wrote: >>>>>>>>>> >>>>>>>>>> On 07/18/2012 01:02 PM, Torsten Hoefler wrote: >>>>>>>>>>> Jim, >>>>>>>>>>>> Is one process' disp_unit meaningful at another process? I >>>>>>>>>>>> thought that >>>>>>>>>>>> this is not the case. MPI provides no other routines to query >>>>>>>>>>>> another >>>>>>>>>>>> process' disp_unit. >>>>>>>>>>> It is something that the user is passing in. I don't know of a >>>>>>>>>>> particular use-case but we can query all other arguments that >>>>>>>>>>> each >>>>>>>>>>> process passed. One more integer seemed safe to me. >>>>>>>>>>> >>>>>>>>>>>> Also, what meaning does a disp_unit have in the context of >>>>>>>>>>>> load/store >>>>>>>>>>>> operations that will result from win_shared_query? >>>>>>>>>>> Nothing. >>>>>>>>>>> >>>>>>>>>>>> Maybe I'm missing something, but this doesn't seem necessary. >>>>>>>>>>> Well, it's better to have it than to introduce it with a >>>>>>>>>>> different name >>>>>>>>>>> later. That was my main motivation. One could also use it in a >>>>>>>>>>> library >>>>>>>>>>> context to query. But that's weak, I know. >>>>>>>>>> >>>>>>>>>> This doesn't seem critical, and can always be added in MPI-3.1 >>>>>>>>>> if we >>>>>>>>>> find it necessary. No reason to rush it. >>>>>>>>> >>>>>>>>> It cannot be added without a new function name. That was the >>>>>>>>> concern >>>>>>>>> here. Adding it seems less risk than not adding it. In addition, >>>>>>>>> it is >>>>>>>>> already added ;-). >>>>>>>> >>>>>>>> I am concerned that this is a harmful change. The disp_unit from >>>>>>>> one >>>>>>>> process has no meaningful use at another process. If the user >>>>>>>> wants to >>>>>>>> know it for some reason, they can use a separate communication >>>>>>>> operation. >>>>>>> I believe the potential damage is very limited. But we can have a >>>>>>> vote. >>>>>>> >>>>>>> So far it's still yes: 5, no: 1 >>>>>>> >>>>>>> I'm just mediating, not making decisions. >>>>>>> >>>>>>> Torsten >>>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> mpi3-rma mailing list >>>>>> mpi3-rma(a)lists.mpi-forum.org >>>>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma >>>>>> >>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> mpi3-rma mailing list >>>>>> mpi3-rma(a)lists.mpi-forum.org >>>>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma >>>>>> >>>>> >>>>> -- >>>>> ### qreharg rug ebs fv crryF ------------- http://www.unixer.de/ >>>>> ----- >>>>> Torsten Hoefler | Performance Modeling and Simulation Lead >>>>> Blue Waters Directorate | University of Illinois (UIUC) >>>>> 1205 W Clark Street | Urbana, IL, 61801 >>>>> NCSA Building | +01 (217) 244-7736 >>>>> _______________________________________________ >>>>> mpi3-rma mailing list >>>>> mpi3-rma(a)lists.mpi-forum.org >>>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma >>>> >>>> >>>> _______________________________________________ >>>> mpi3-rma mailing list >>>> mpi3-rma(a)lists.mpi-forum.org >>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma >>> >>> -- >>> Dr. Rolf Rabenseifner . . . . . . . . . .. email rabenseifner(a)hlrs.de >>> High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 >>> University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 >>> Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner >>> Nobelstr. 19, D-70550 Stuttgart, Germany . (Office: Allmandring 30) >>> _______________________________________________ >>> mpi-forum mailing list >>> mpi-forum(a)lists.mpi-forum.org >>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum >> >> >> _______________________________________________ >> mpi3-rma mailing list >> mpi3-rma(a)lists.mpi-forum.org >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma > > > > -- > Jeff Hammond > Argonne Leadership Computing Facility > University of Chicago Computation Institute > jhammond(a)alcf.anl.gov / (630) 252-5381 > http://www.linkedin.com/in/jeffhammond > https://wiki.alcf.anl.gov/parts/index.php/User:Jhammond > _______________________________________________ > mpi-forum mailing list > mpi-forum(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum

1 0

Re: [Mpi-forum] [Mpi3-rma] [EXTERNAL] Re: Disp_unit in allocate_shared should be returned by win_shared_query?
by Jeff Hammond 22 Jul '12

22 Jul '12

I think "s/at extreme scale/when memory is limited/". An adapteva chip has 32KB SRAM per core and is pushing 100 cores and farther, so having >128*4 bytes used in the window object is no different than having >1 of 64 GB used for the window if one has 2^29 ranks. Jeff On Sun, Jul 22, 2012 at 1:03 PM, William Gropp <wgropp(a)illinois.edu> wrote: > I think this is a little strong. This isn't necessary for the vast majority of MPI users; just for those at extreme scale. A modification that restricted this to extreme scale might be acceptable. > > Bill > > William Gropp > Director, Parallel Computing Institute > Deputy Director for Research > Institute for Advanced Computing Applications and Technologies > Paul and Cynthia Saylor Professor of Computer Science > University of Illinois Urbana-Champaign > > > > On Jul 22, 2012, at 12:28 PM, Rolf Rabenseifner wrote: > >> On Wed, Jul 18, 2012 at 08:11:23PM +0000, Barrett, Brian W wrote: >>> I'm concerned with the change (and don't think it should happen >>> at this point, as I want to be able to tell other people their >>> changes are too big). It's probably not another O(p) structure, >>> since a process will have to know the other rank's disp_unit to >>> do the memcpy during the communication operations, so we crossed >>> the O(p) bridge when we added disp_unit to allocate_shared. >> >> I would recommend to add two advices: >> >> Advice to users. For scalability reasons it is recommended >> that all processes provide the same disp_unit argument >> on all processes. End of advice to users. >> >> Advice to implementors. If the user provides the same >> disp_unit argument on all processes of the communicator, >> or a compressible list of values, it is necessary for >> scalability reasons to store this data as compact as possible. >> End of advice to users. >> >> Of course for all thre MPI_WIN_CREATE, MPI_WIN_AALOCATE, and >> MPI_WIN_ALLOCATE_SHARED. >> >> Is this a good idea? >> If yes, can someone find a better wording for this proposal? >> >> Best regards >> Rolf >> >> ----- Original Message ----- >>> From: "William Gropp" <wgropp(a)illinois.edu> >>> To: "MPI 3.0 Remote Memory Access working group" <mpi3-rma(a)lists.mpi-forum.org> >>> Sent: Sunday, July 22, 2012 7:12:27 PM >>> Subject: Re: [Mpi3-rma] [EXTERNAL] Re: Disp_unit in allocate_shared should be returned by win_shared_query? >>> What *exactly* are the proposals? I know in general, but some of the >>> statements make me wonder if I understand the proposal. >>> >>> Bill >>> >>> William Gropp >>> Director, Parallel Computing Institute >>> Deputy Director for Research >>> Institute for Advanced Computing Applications and Technologies >>> Paul and Cynthia Saylor Professor of Computer Science >>> University of Illinois Urbana-Champaign >>> >>> >>> >>> On Jul 22, 2012, at 10:55 AM, Torsten Hoefler wrote: >>> >>>> Dear All, >>>> >>>> I think we should discuss this more. Right now, we still have more >>>> votes >>>> in favor of this than against. The change is still committed and >>>> part of >>>> the "official release". The deadline is 7/26. >>>> >>>> Please advise what to do! >>>> >>>> Thanks, >>>> Torsten >>>> >>>> On Wed, Jul 18, 2012 at 08:11:23PM +0000, Barrett, Brian W wrote: >>>>> I'm concerned with the change (and don't think it should happen at >>>>> this point, as I want to be able to tell other people their changes >>>>> are too big). It's probably not another O(p) structure, since a >>>>> process will have to know the other rank's disp_unit to do the >>>>> memcpy during the communication operations, so we crossed the O(p) >>>>> bridge when we added disp_unit to allocate_shared. >>>>> >>>>> >>>>> Brian >>>>> >>>>> -- >>>>> Brian W. Barrett >>>>> Scalable System Software Group >>>>> Sandia National Laboratories >>>>> ________________________________________ >>>>> From: mpi3-rma-bounces(a)lists.mpi-forum.org >>>>> [mpi3-rma-bounces(a)lists.mpi-forum.org] on behalf of Jim Dinan >>>>> [dinan(a)mcs.anl.gov] >>>>> Sent: Wednesday, July 18, 2012 12:46 PM >>>>> To: mpi3-rma(a)lists.mpi-forum.org >>>>> Subject: [EXTERNAL] Re: [Mpi3-rma] Disp_unit in allocate_shared >>>>> should be returned by win_shared_query? >>>>> >>>>> It's not inherently damaging, but it requires us to >>>>> maintain/provide >>>>> access to another O(p) structure and suggests to users that another >>>>> process' disp_unit is useful information, which is incorrect. IIRC, >>>>> we >>>>> did discuss this in the working group and intentionally omitted it. >>>>> >>>>> ~Jim. >>>>> >>>>> On 7/18/12 1:38 PM, Torsten Hoefler wrote: >>>>>> On Wed, Jul 18, 2012 at 01:18:53PM -0500, James Dinan wrote: >>>>>>> On 7/18/12 1:06 PM, Torsten Hoefler wrote: >>>>>>>> On Wed, Jul 18, 2012 at 01:08:43PM -0500, Pavan Balaji wrote: >>>>>>>>> >>>>>>>>> On 07/18/2012 01:02 PM, Torsten Hoefler wrote: >>>>>>>>>> Jim, >>>>>>>>>>> Is one process' disp_unit meaningful at another process? I >>>>>>>>>>> thought that >>>>>>>>>>> this is not the case. MPI provides no other routines to query >>>>>>>>>>> another >>>>>>>>>>> process' disp_unit. >>>>>>>>>> It is something that the user is passing in. I don't know of a >>>>>>>>>> particular use-case but we can query all other arguments that >>>>>>>>>> each >>>>>>>>>> process passed. One more integer seemed safe to me. >>>>>>>>>> >>>>>>>>>>> Also, what meaning does a disp_unit have in the context of >>>>>>>>>>> load/store >>>>>>>>>>> operations that will result from win_shared_query? >>>>>>>>>> Nothing. >>>>>>>>>> >>>>>>>>>>> Maybe I'm missing something, but this doesn't seem necessary. >>>>>>>>>> Well, it's better to have it than to introduce it with a >>>>>>>>>> different name >>>>>>>>>> later. That was my main motivation. One could also use it in a >>>>>>>>>> library >>>>>>>>>> context to query. But that's weak, I know. >>>>>>>>> >>>>>>>>> This doesn't seem critical, and can always be added in MPI-3.1 >>>>>>>>> if we >>>>>>>>> find it necessary. No reason to rush it. >>>>>>>> >>>>>>>> It cannot be added without a new function name. That was the >>>>>>>> concern >>>>>>>> here. Adding it seems less risk than not adding it. In addition, >>>>>>>> it is >>>>>>>> already added ;-). >>>>>>> >>>>>>> I am concerned that this is a harmful change. The disp_unit from >>>>>>> one >>>>>>> process has no meaningful use at another process. If the user >>>>>>> wants to >>>>>>> know it for some reason, they can use a separate communication >>>>>>> operation. >>>>>> I believe the potential damage is very limited. But we can have a >>>>>> vote. >>>>>> >>>>>> So far it's still yes: 5, no: 1 >>>>>> >>>>>> I'm just mediating, not making decisions. >>>>>> >>>>>> Torsten >>>>>> >>>>> >>>>> _______________________________________________ >>>>> mpi3-rma mailing list >>>>> mpi3-rma(a)lists.mpi-forum.org >>>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma >>>>> >>>>> >>>>> >>>>> _______________________________________________ >>>>> mpi3-rma mailing list >>>>> mpi3-rma(a)lists.mpi-forum.org >>>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma >>>>> >>>> >>>> -- >>>> ### qreharg rug ebs fv crryF ------------- http://www.unixer.de/ >>>> ----- >>>> Torsten Hoefler | Performance Modeling and Simulation Lead >>>> Blue Waters Directorate | University of Illinois (UIUC) >>>> 1205 W Clark Street | Urbana, IL, 61801 >>>> NCSA Building | +01 (217) 244-7736 >>>> _______________________________________________ >>>> mpi3-rma mailing list >>>> mpi3-rma(a)lists.mpi-forum.org >>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma >>> >>> >>> _______________________________________________ >>> mpi3-rma mailing list >>> mpi3-rma(a)lists.mpi-forum.org >>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma >> >> -- >> Dr. Rolf Rabenseifner . . . . . . . . . .. email rabenseifner(a)hlrs.de >> High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 >> University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 >> Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner >> Nobelstr. 19, D-70550 Stuttgart, Germany . (Office: Allmandring 30) >> _______________________________________________ >> mpi-forum mailing list >> mpi-forum(a)lists.mpi-forum.org >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum > > > _______________________________________________ > mpi3-rma mailing list > mpi3-rma(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma -- Jeff Hammond Argonne Leadership Computing Facility University of Chicago Computation Institute jhammond(a)alcf.anl.gov / (630) 252-5381 http://www.linkedin.com/in/jeffhammond https://wiki.alcf.anl.gov/parts/index.php/User:Jhammond

1 0

Re: [Mpi-forum] [Mpi3-rma] [EXTERNAL] Re: Disp_unit in allocate_shared should be returned by win_shared_query?
by Torsten Hoefler 22 Jul '12

22 Jul '12

I would agree to Bill. We should have an info argument allowing the user to state that all displ_units are the same like we have for count :-). A topic for MPI-3.1! Thanks, Torsten On Sun, Jul 22, 2012 at 01:03:39PM -0500, William Gropp wrote: > I think this is a little strong. This isn't necessary for the vast majority of MPI users; just for those at extreme scale. A modification that restricted this to extreme scale might be acceptable. > > Bill > > William Gropp > Director, Parallel Computing Institute > Deputy Director for Research > Institute for Advanced Computing Applications and Technologies > Paul and Cynthia Saylor Professor of Computer Science > University of Illinois Urbana-Champaign > > > > On Jul 22, 2012, at 12:28 PM, Rolf Rabenseifner wrote: > > > On Wed, Jul 18, 2012 at 08:11:23PM +0000, Barrett, Brian W wrote: > >> I'm concerned with the change (and don't think it should happen > >> at this point, as I want to be able to tell other people their > >> changes are too big). It's probably not another O(p) structure, > >> since a process will have to know the other rank's disp_unit to > >> do the memcpy during the communication operations, so we crossed > >> the O(p) bridge when we added disp_unit to allocate_shared. > > > > I would recommend to add two advices: > > > > Advice to users. For scalability reasons it is recommended > > that all processes provide the same disp_unit argument > > on all processes. End of advice to users. > > > > Advice to implementors. If the user provides the same > > disp_unit argument on all processes of the communicator, > > or a compressible list of values, it is necessary for > > scalability reasons to store this data as compact as possible. > > End of advice to users. > > > > Of course for all thre MPI_WIN_CREATE, MPI_WIN_AALOCATE, and > > MPI_WIN_ALLOCATE_SHARED. > > > > Is this a good idea? > > If yes, can someone find a better wording for this proposal? > > > > Best regards > > Rolf > > > > ----- Original Message ----- > >> From: "William Gropp" <wgropp(a)illinois.edu> > >> To: "MPI 3.0 Remote Memory Access working group" <mpi3-rma(a)lists.mpi-forum.org> > >> Sent: Sunday, July 22, 2012 7:12:27 PM > >> Subject: Re: [Mpi3-rma] [EXTERNAL] Re: Disp_unit in allocate_shared should be returned by win_shared_query? > >> What *exactly* are the proposals? I know in general, but some of the > >> statements make me wonder if I understand the proposal. > >> > >> Bill > >> > >> William Gropp > >> Director, Parallel Computing Institute > >> Deputy Director for Research > >> Institute for Advanced Computing Applications and Technologies > >> Paul and Cynthia Saylor Professor of Computer Science > >> University of Illinois Urbana-Champaign > >> > >> > >> > >> On Jul 22, 2012, at 10:55 AM, Torsten Hoefler wrote: > >> > >>> Dear All, > >>> > >>> I think we should discuss this more. Right now, we still have more > >>> votes > >>> in favor of this than against. The change is still committed and > >>> part of > >>> the "official release". The deadline is 7/26. > >>> > >>> Please advise what to do! > >>> > >>> Thanks, > >>> Torsten > >>> > >>> On Wed, Jul 18, 2012 at 08:11:23PM +0000, Barrett, Brian W wrote: > >>>> I'm concerned with the change (and don't think it should happen at > >>>> this point, as I want to be able to tell other people their changes > >>>> are too big). It's probably not another O(p) structure, since a > >>>> process will have to know the other rank's disp_unit to do the > >>>> memcpy during the communication operations, so we crossed the O(p) > >>>> bridge when we added disp_unit to allocate_shared. > >>>> > >>>> > >>>> Brian > >>>> > >>>> -- > >>>> Brian W. Barrett > >>>> Scalable System Software Group > >>>> Sandia National Laboratories > >>>> ________________________________________ > >>>> From: mpi3-rma-bounces(a)lists.mpi-forum.org > >>>> [mpi3-rma-bounces(a)lists.mpi-forum.org] on behalf of Jim Dinan > >>>> [dinan(a)mcs.anl.gov] > >>>> Sent: Wednesday, July 18, 2012 12:46 PM > >>>> To: mpi3-rma(a)lists.mpi-forum.org > >>>> Subject: [EXTERNAL] Re: [Mpi3-rma] Disp_unit in allocate_shared > >>>> should be returned by win_shared_query? > >>>> > >>>> It's not inherently damaging, but it requires us to > >>>> maintain/provide > >>>> access to another O(p) structure and suggests to users that another > >>>> process' disp_unit is useful information, which is incorrect. IIRC, > >>>> we > >>>> did discuss this in the working group and intentionally omitted it. > >>>> > >>>> ~Jim. > >>>> > >>>> On 7/18/12 1:38 PM, Torsten Hoefler wrote: > >>>>> On Wed, Jul 18, 2012 at 01:18:53PM -0500, James Dinan wrote: > >>>>>> On 7/18/12 1:06 PM, Torsten Hoefler wrote: > >>>>>>> On Wed, Jul 18, 2012 at 01:08:43PM -0500, Pavan Balaji wrote: > >>>>>>>> > >>>>>>>> On 07/18/2012 01:02 PM, Torsten Hoefler wrote: > >>>>>>>>> Jim, > >>>>>>>>>> Is one process' disp_unit meaningful at another process? I > >>>>>>>>>> thought that > >>>>>>>>>> this is not the case. MPI provides no other routines to query > >>>>>>>>>> another > >>>>>>>>>> process' disp_unit. > >>>>>>>>> It is something that the user is passing in. I don't know of a > >>>>>>>>> particular use-case but we can query all other arguments that > >>>>>>>>> each > >>>>>>>>> process passed. One more integer seemed safe to me. > >>>>>>>>> > >>>>>>>>>> Also, what meaning does a disp_unit have in the context of > >>>>>>>>>> load/store > >>>>>>>>>> operations that will result from win_shared_query? > >>>>>>>>> Nothing. > >>>>>>>>> > >>>>>>>>>> Maybe I'm missing something, but this doesn't seem necessary. > >>>>>>>>> Well, it's better to have it than to introduce it with a > >>>>>>>>> different name > >>>>>>>>> later. That was my main motivation. One could also use it in a > >>>>>>>>> library > >>>>>>>>> context to query. But that's weak, I know. > >>>>>>>> > >>>>>>>> This doesn't seem critical, and can always be added in MPI-3.1 > >>>>>>>> if we > >>>>>>>> find it necessary. No reason to rush it. > >>>>>>> > >>>>>>> It cannot be added without a new function name. That was the > >>>>>>> concern > >>>>>>> here. Adding it seems less risk than not adding it. In addition, > >>>>>>> it is > >>>>>>> already added ;-). > >>>>>> > >>>>>> I am concerned that this is a harmful change. The disp_unit from > >>>>>> one > >>>>>> process has no meaningful use at another process. If the user > >>>>>> wants to > >>>>>> know it for some reason, they can use a separate communication > >>>>>> operation. > >>>>> I believe the potential damage is very limited. But we can have a > >>>>> vote. > >>>>> > >>>>> So far it's still yes: 5, no: 1 > >>>>> > >>>>> I'm just mediating, not making decisions. > >>>>> > >>>>> Torsten > >>>>> > >>>> > >>>> _______________________________________________ > >>>> mpi3-rma mailing list > >>>> mpi3-rma(a)lists.mpi-forum.org > >>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma > >>>> > >>>> > >>>> > >>>> _______________________________________________ > >>>> mpi3-rma mailing list > >>>> mpi3-rma(a)lists.mpi-forum.org > >>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma > >>>> > >>> > >>> -- > >>> ### qreharg rug ebs fv crryF ------------- http://www.unixer.de/ > >>> ----- > >>> Torsten Hoefler | Performance Modeling and Simulation Lead > >>> Blue Waters Directorate | University of Illinois (UIUC) > >>> 1205 W Clark Street | Urbana, IL, 61801 > >>> NCSA Building | +01 (217) 244-7736 > >>> _______________________________________________ > >>> mpi3-rma mailing list > >>> mpi3-rma(a)lists.mpi-forum.org > >>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma > >> > >> > >> _______________________________________________ > >> mpi3-rma mailing list > >> mpi3-rma(a)lists.mpi-forum.org > >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma > > > > -- > > Dr. Rolf Rabenseifner . . . . . . . . . .. email rabenseifner(a)hlrs.de > > High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 > > University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 > > Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner > > Nobelstr. 19, D-70550 Stuttgart, Germany . (Office: Allmandring 30) > > _______________________________________________ > > mpi-forum mailing list > > mpi-forum(a)lists.mpi-forum.org > > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum > > > _______________________________________________ > mpi-forum mailing list > mpi-forum(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum > -- ### qreharg rug ebs fv crryF ------------- http://www.unixer.de/ ----- Torsten Hoefler | Performance Modeling and Simulation Lead Blue Waters Directorate | University of Illinois (UIUC) 1205 W Clark Street | Urbana, IL, 61801 NCSA Building | +01 (217) 244-7736

1 0

Re: [Mpi-forum] [Mpi3-rma] [EXTERNAL] Re: Disp_unit in allocate_shared should be returned by win_shared_query?
by William Gropp 22 Jul '12

22 Jul '12

I think this is a little strong. This isn't necessary for the vast majority of MPI users; just for those at extreme scale. A modification that restricted this to extreme scale might be acceptable. Bill William Gropp Director, Parallel Computing Institute Deputy Director for Research Institute for Advanced Computing Applications and Technologies Paul and Cynthia Saylor Professor of Computer Science University of Illinois Urbana-Champaign On Jul 22, 2012, at 12:28 PM, Rolf Rabenseifner wrote: > On Wed, Jul 18, 2012 at 08:11:23PM +0000, Barrett, Brian W wrote: >> I'm concerned with the change (and don't think it should happen >> at this point, as I want to be able to tell other people their >> changes are too big). It's probably not another O(p) structure, >> since a process will have to know the other rank's disp_unit to >> do the memcpy during the communication operations, so we crossed >> the O(p) bridge when we added disp_unit to allocate_shared. > > I would recommend to add two advices: > > Advice to users. For scalability reasons it is recommended > that all processes provide the same disp_unit argument > on all processes. End of advice to users. > > Advice to implementors. If the user provides the same > disp_unit argument on all processes of the communicator, > or a compressible list of values, it is necessary for > scalability reasons to store this data as compact as possible. > End of advice to users. > > Of course for all thre MPI_WIN_CREATE, MPI_WIN_AALOCATE, and > MPI_WIN_ALLOCATE_SHARED. > > Is this a good idea? > If yes, can someone find a better wording for this proposal? > > Best regards > Rolf > > ----- Original Message ----- >> From: "William Gropp" <wgropp(a)illinois.edu> >> To: "MPI 3.0 Remote Memory Access working group" <mpi3-rma(a)lists.mpi-forum.org> >> Sent: Sunday, July 22, 2012 7:12:27 PM >> Subject: Re: [Mpi3-rma] [EXTERNAL] Re: Disp_unit in allocate_shared should be returned by win_shared_query? >> What *exactly* are the proposals? I know in general, but some of the >> statements make me wonder if I understand the proposal. >> >> Bill >> >> William Gropp >> Director, Parallel Computing Institute >> Deputy Director for Research >> Institute for Advanced Computing Applications and Technologies >> Paul and Cynthia Saylor Professor of Computer Science >> University of Illinois Urbana-Champaign >> >> >> >> On Jul 22, 2012, at 10:55 AM, Torsten Hoefler wrote: >> >>> Dear All, >>> >>> I think we should discuss this more. Right now, we still have more >>> votes >>> in favor of this than against. The change is still committed and >>> part of >>> the "official release". The deadline is 7/26. >>> >>> Please advise what to do! >>> >>> Thanks, >>> Torsten >>> >>> On Wed, Jul 18, 2012 at 08:11:23PM +0000, Barrett, Brian W wrote: >>>> I'm concerned with the change (and don't think it should happen at >>>> this point, as I want to be able to tell other people their changes >>>> are too big). It's probably not another O(p) structure, since a >>>> process will have to know the other rank's disp_unit to do the >>>> memcpy during the communication operations, so we crossed the O(p) >>>> bridge when we added disp_unit to allocate_shared. >>>> >>>> >>>> Brian >>>> >>>> -- >>>> Brian W. Barrett >>>> Scalable System Software Group >>>> Sandia National Laboratories >>>> ________________________________________ >>>> From: mpi3-rma-bounces(a)lists.mpi-forum.org >>>> [mpi3-rma-bounces(a)lists.mpi-forum.org] on behalf of Jim Dinan >>>> [dinan(a)mcs.anl.gov] >>>> Sent: Wednesday, July 18, 2012 12:46 PM >>>> To: mpi3-rma(a)lists.mpi-forum.org >>>> Subject: [EXTERNAL] Re: [Mpi3-rma] Disp_unit in allocate_shared >>>> should be returned by win_shared_query? >>>> >>>> It's not inherently damaging, but it requires us to >>>> maintain/provide >>>> access to another O(p) structure and suggests to users that another >>>> process' disp_unit is useful information, which is incorrect. IIRC, >>>> we >>>> did discuss this in the working group and intentionally omitted it. >>>> >>>> ~Jim. >>>> >>>> On 7/18/12 1:38 PM, Torsten Hoefler wrote: >>>>> On Wed, Jul 18, 2012 at 01:18:53PM -0500, James Dinan wrote: >>>>>> On 7/18/12 1:06 PM, Torsten Hoefler wrote: >>>>>>> On Wed, Jul 18, 2012 at 01:08:43PM -0500, Pavan Balaji wrote: >>>>>>>> >>>>>>>> On 07/18/2012 01:02 PM, Torsten Hoefler wrote: >>>>>>>>> Jim, >>>>>>>>>> Is one process' disp_unit meaningful at another process? I >>>>>>>>>> thought that >>>>>>>>>> this is not the case. MPI provides no other routines to query >>>>>>>>>> another >>>>>>>>>> process' disp_unit. >>>>>>>>> It is something that the user is passing in. I don't know of a >>>>>>>>> particular use-case but we can query all other arguments that >>>>>>>>> each >>>>>>>>> process passed. One more integer seemed safe to me. >>>>>>>>> >>>>>>>>>> Also, what meaning does a disp_unit have in the context of >>>>>>>>>> load/store >>>>>>>>>> operations that will result from win_shared_query? >>>>>>>>> Nothing. >>>>>>>>> >>>>>>>>>> Maybe I'm missing something, but this doesn't seem necessary. >>>>>>>>> Well, it's better to have it than to introduce it with a >>>>>>>>> different name >>>>>>>>> later. That was my main motivation. One could also use it in a >>>>>>>>> library >>>>>>>>> context to query. But that's weak, I know. >>>>>>>> >>>>>>>> This doesn't seem critical, and can always be added in MPI-3.1 >>>>>>>> if we >>>>>>>> find it necessary. No reason to rush it. >>>>>>> >>>>>>> It cannot be added without a new function name. That was the >>>>>>> concern >>>>>>> here. Adding it seems less risk than not adding it. In addition, >>>>>>> it is >>>>>>> already added ;-). >>>>>> >>>>>> I am concerned that this is a harmful change. The disp_unit from >>>>>> one >>>>>> process has no meaningful use at another process. If the user >>>>>> wants to >>>>>> know it for some reason, they can use a separate communication >>>>>> operation. >>>>> I believe the potential damage is very limited. But we can have a >>>>> vote. >>>>> >>>>> So far it's still yes: 5, no: 1 >>>>> >>>>> I'm just mediating, not making decisions. >>>>> >>>>> Torsten >>>>> >>>> >>>> _______________________________________________ >>>> mpi3-rma mailing list >>>> mpi3-rma(a)lists.mpi-forum.org >>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma >>>> >>>> >>>> >>>> _______________________________________________ >>>> mpi3-rma mailing list >>>> mpi3-rma(a)lists.mpi-forum.org >>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma >>>> >>> >>> -- >>> ### qreharg rug ebs fv crryF ------------- http://www.unixer.de/ >>> ----- >>> Torsten Hoefler | Performance Modeling and Simulation Lead >>> Blue Waters Directorate | University of Illinois (UIUC) >>> 1205 W Clark Street | Urbana, IL, 61801 >>> NCSA Building | +01 (217) 244-7736 >>> _______________________________________________ >>> mpi3-rma mailing list >>> mpi3-rma(a)lists.mpi-forum.org >>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma >> >> >> _______________________________________________ >> mpi3-rma mailing list >> mpi3-rma(a)lists.mpi-forum.org >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma > > -- > Dr. Rolf Rabenseifner . . . . . . . . . .. email rabenseifner(a)hlrs.de > High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 > University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 > Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner > Nobelstr. 19, D-70550 Stuttgart, Germany . (Office: Allmandring 30) > _______________________________________________ > mpi-forum mailing list > mpi-forum(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum

1 0

Re: [Mpi-forum] Indices
by Torsten Hoefler 22 Jul '12

22 Jul '12

George, Thanks! The change is committed in r1502. Torsten On Thu, Jul 19, 2012 at 10:25:08PM +0200, George Bosilca wrote: > Torsten, > > When the usage of the MPI_LB and MPI_UB has been replaced by the resize function the arrays used by the MPI_Type_struct (displacement and blacken) have been left behind. The patch below clean one of the examples from the collective chapter. > > george. > > Index: chap-coll/coll.tex > =================================================================== > --- chap-coll/coll.tex (revision 1482) > +++ chap-coll/coll.tex (working copy) > @@ -1212,14 +1211,15 @@ > %%HEADER > %%LANG: C > %%FRAGMENT > +%%SUBST:...: > %%SKIPELIPSIS > %%ENDHEADER > \begin{Verbatim}[commandchars=\\\$\^] > MPI_Comm comm; > - int gsize,sendarray[100][150],*sptr; > - int root, *rbuf, stride, myrank, disp[2], blocklen[2]; > - MPI_Datatype stype,type[2]; > - int *displs,i,*rcounts; > + int gsize, sendarray[100][150], *sptr; > + int root, *rbuf, stride, myrank; > + MPI_Datatype stype; > + int *displs, i, *rcounts; > > ... > > > On Jul 19, 2012, at 20:21 , William Gropp wrote: > > > Yes, these need to be fixed. > > > > Mixed case in the function index means either the C binding name (or Fortran08 name) was used where the language-independent name should be used, or \func or \mpifunc were used for a language-binding-specific name (which is incorrect macro usage). Chapter authors need to fix this. There as also some non-functions in the function index; those must be fixed. > > > > Example index entries need to be checked; I added a bunch to context as many examples were missing the index entries (made with \exindex). > > > > I think it is ok for removed and deprecated names to show up in the index. > > > > Bill > > > > > > William Gropp > > Director, Parallel Computing Institute > > Deputy Director for Research > > Institute for Advanced Computing Applications and Technologies > > Paul and Cynthia Saylor Professor of Computer Science > > University of Illinois Urbana-Champaign > > > > > > > > On Jul 19, 2012, at 12:17 PM, Mohamad Chaarawi wrote: > > > >> Hi All, > >> > >> After reviewing the indices, I found a few issues: > >> > >> - MPI_Type_struct and MPI_TYPE_EXTENT show up in the example indices. This should not happen as they are removed. > >> - I'm not sure if removed and deprecated routines should show up in the Function Index, but right now, they do. > >> - similarly in the MPI Callback Function Prototype Index > >> - similarly in Constant and Predefined Handle Index > >> - in the function index, many routines show up twice (caps and no caps) and some show up only with no caps.. All should be caps (I think). > >> > >> Do the chapter authors need to fix those in their chapters? > >> > >> Thanks, > >> Mohamad > >> _______________________________________________ > >> mpi-forum mailing list > >> mpi-forum(a)lists.mpi-forum.org > >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum > > > > > > _______________________________________________ > > mpi-forum mailing list > > mpi-forum(a)lists.mpi-forum.org > > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum > > > _______________________________________________ > mpi-forum mailing list > mpi-forum(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum > -- ### qreharg rug ebs fv crryF ------------- http://www.unixer.de/ ----- Torsten Hoefler | Performance Modeling and Simulation Lead Blue Waters Directorate | University of Illinois (UIUC) 1205 W Clark Street | Urbana, IL, 61801 NCSA Building | +01 (217) 244-7736

1 0

Re: [Mpi-forum] Indices
by Torsten Hoefler 22 Jul '12

22 Jul '12

George, Thanks! I assume Rolf's commit fixed all of those! Thanks, Torsten On Thu, Jul 19, 2012 at 10:22:05PM +0200, George Bosilca wrote: > Here is the patch for the collective chapter removing the MPI_Type_struct from the example index. > > george. > > Index: chap-coll/coll.tex > =================================================================== > --- chap-coll/coll.tex (revision 1482) > +++ chap-coll/coll.tex (working copy) > @@ -1163,7 +1163,6 @@ > \begin{example} {\rm > \label{coll-exF} > \exindex{MPI\_Gatherv}% > -\exindex{MPI\_Type\_struct}% > \exindex{MPI\_Type\_create\_struct}% > \exindex{MPI\_Type\_commit}% > > @@ -1329,7 +1329,6 @@ > \label{coll-exH} > \exindex{MPI\_Gather}% > \exindex{MPI\_Gatherv}% > -\exindex{MPI\_Type\_struct}% > \exindex{MPI\_Type\_create\_struct}% > \exindex{MPI\_Type\_commit}% > > @@ -3999,7 +3998,6 @@ > \label{coll-exW} > \exindex{MPI\_Op\_create}% > \exindex{MPI\_Scan}% > -\exindex{MPI\_Type\_struct}% > \exindex{MPI\_Type\_create\_struct}% > \exindex{MPI\_Type\_commit}% > > > > > On Jul 19, 2012, at 20:21 , William Gropp wrote: > > > Yes, these need to be fixed. > > > > Mixed case in the function index means either the C binding name (or Fortran08 name) was used where the language-independent name should be used, or \func or \mpifunc were used for a language-binding-specific name (which is incorrect macro usage). Chapter authors need to fix this. There as also some non-functions in the function index; those must be fixed. > > > > Example index entries need to be checked; I added a bunch to context as many examples were missing the index entries (made with \exindex). > > > > I think it is ok for removed and deprecated names to show up in the index. > > > > Bill > > > > > > William Gropp > > Director, Parallel Computing Institute > > Deputy Director for Research > > Institute for Advanced Computing Applications and Technologies > > Paul and Cynthia Saylor Professor of Computer Science > > University of Illinois Urbana-Champaign > > > > > > > > On Jul 19, 2012, at 12:17 PM, Mohamad Chaarawi wrote: > > > >> Hi All, > >> > >> After reviewing the indices, I found a few issues: > >> > >> - MPI_Type_struct and MPI_TYPE_EXTENT show up in the example indices. This should not happen as they are removed. > >> - I'm not sure if removed and deprecated routines should show up in the Function Index, but right now, they do. > >> - similarly in the MPI Callback Function Prototype Index > >> - similarly in Constant and Predefined Handle Index > >> - in the function index, many routines show up twice (caps and no caps) and some show up only with no caps.. All should be caps (I think). > >> > >> Do the chapter authors need to fix those in their chapters? > >> > >> Thanks, > >> Mohamad > >> _______________________________________________ > >> mpi-forum mailing list > >> mpi-forum(a)lists.mpi-forum.org > >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum > > > > > > _______________________________________________ > > mpi-forum mailing list > > mpi-forum(a)lists.mpi-forum.org > > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum > > > _______________________________________________ > mpi-forum mailing list > mpi-forum(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum > -- ### qreharg rug ebs fv crryF ------------- http://www.unixer.de/ ----- Torsten Hoefler | Performance Modeling and Simulation Lead Blue Waters Directorate | University of Illinois (UIUC) 1205 W Clark Street | Urbana, IL, 61801 NCSA Building | +01 (217) 244-7736

1 0

Re: [Mpi-forum] [Mpi3-rma] [EXTERNAL] Re: Disp_unit in allocate_shared should be returned by win_shared_query?
by Rolf Rabenseifner 22 Jul '12

22 Jul '12

On Wed, Jul 18, 2012 at 08:11:23PM +0000, Barrett, Brian W wrote: > I'm concerned with the change (and don't think it should happen > at this point, as I want to be able to tell other people their > changes are too big). It's probably not another O(p) structure, > since a process will have to know the other rank's disp_unit to > do the memcpy during the communication operations, so we crossed > the O(p) bridge when we added disp_unit to allocate_shared. I would recommend to add two advices: Advice to users. For scalability reasons it is recommended that all processes provide the same disp_unit argument on all processes. End of advice to users. Advice to implementors. If the user provides the same disp_unit argument on all processes of the communicator, or a compressible list of values, it is necessary for scalability reasons to store this data as compact as possible. End of advice to users. Of course for all thre MPI_WIN_CREATE, MPI_WIN_AALOCATE, and MPI_WIN_ALLOCATE_SHARED. Is this a good idea? If yes, can someone find a better wording for this proposal? Best regards Rolf ----- Original Message ----- > From: "William Gropp" <wgropp(a)illinois.edu> > To: "MPI 3.0 Remote Memory Access working group" <mpi3-rma(a)lists.mpi-forum.org> > Sent: Sunday, July 22, 2012 7:12:27 PM > Subject: Re: [Mpi3-rma] [EXTERNAL] Re: Disp_unit in allocate_shared should be returned by win_shared_query? > What *exactly* are the proposals? I know in general, but some of the > statements make me wonder if I understand the proposal. > > Bill > > William Gropp > Director, Parallel Computing Institute > Deputy Director for Research > Institute for Advanced Computing Applications and Technologies > Paul and Cynthia Saylor Professor of Computer Science > University of Illinois Urbana-Champaign > > > > On Jul 22, 2012, at 10:55 AM, Torsten Hoefler wrote: > > > Dear All, > > > > I think we should discuss this more. Right now, we still have more > > votes > > in favor of this than against. The change is still committed and > > part of > > the "official release". The deadline is 7/26. > > > > Please advise what to do! > > > > Thanks, > > Torsten > > > > On Wed, Jul 18, 2012 at 08:11:23PM +0000, Barrett, Brian W wrote: > >> I'm concerned with the change (and don't think it should happen at > >> this point, as I want to be able to tell other people their changes > >> are too big). It's probably not another O(p) structure, since a > >> process will have to know the other rank's disp_unit to do the > >> memcpy during the communication operations, so we crossed the O(p) > >> bridge when we added disp_unit to allocate_shared. > >> > >> > >> Brian > >> > >> -- > >> Brian W. Barrett > >> Scalable System Software Group > >> Sandia National Laboratories > >> ________________________________________ > >> From: mpi3-rma-bounces(a)lists.mpi-forum.org > >> [mpi3-rma-bounces(a)lists.mpi-forum.org] on behalf of Jim Dinan > >> [dinan(a)mcs.anl.gov] > >> Sent: Wednesday, July 18, 2012 12:46 PM > >> To: mpi3-rma(a)lists.mpi-forum.org > >> Subject: [EXTERNAL] Re: [Mpi3-rma] Disp_unit in allocate_shared > >> should be returned by win_shared_query? > >> > >> It's not inherently damaging, but it requires us to > >> maintain/provide > >> access to another O(p) structure and suggests to users that another > >> process' disp_unit is useful information, which is incorrect. IIRC, > >> we > >> did discuss this in the working group and intentionally omitted it. > >> > >> ~Jim. > >> > >> On 7/18/12 1:38 PM, Torsten Hoefler wrote: > >>> On Wed, Jul 18, 2012 at 01:18:53PM -0500, James Dinan wrote: > >>>> On 7/18/12 1:06 PM, Torsten Hoefler wrote: > >>>>> On Wed, Jul 18, 2012 at 01:08:43PM -0500, Pavan Balaji wrote: > >>>>>> > >>>>>> On 07/18/2012 01:02 PM, Torsten Hoefler wrote: > >>>>>>> Jim, > >>>>>>>> Is one process' disp_unit meaningful at another process? I > >>>>>>>> thought that > >>>>>>>> this is not the case. MPI provides no other routines to query > >>>>>>>> another > >>>>>>>> process' disp_unit. > >>>>>>> It is something that the user is passing in. I don't know of a > >>>>>>> particular use-case but we can query all other arguments that > >>>>>>> each > >>>>>>> process passed. One more integer seemed safe to me. > >>>>>>> > >>>>>>>> Also, what meaning does a disp_unit have in the context of > >>>>>>>> load/store > >>>>>>>> operations that will result from win_shared_query? > >>>>>>> Nothing. > >>>>>>> > >>>>>>>> Maybe I'm missing something, but this doesn't seem necessary. > >>>>>>> Well, it's better to have it than to introduce it with a > >>>>>>> different name > >>>>>>> later. That was my main motivation. One could also use it in a > >>>>>>> library > >>>>>>> context to query. But that's weak, I know. > >>>>>> > >>>>>> This doesn't seem critical, and can always be added in MPI-3.1 > >>>>>> if we > >>>>>> find it necessary. No reason to rush it. > >>>>> > >>>>> It cannot be added without a new function name. That was the > >>>>> concern > >>>>> here. Adding it seems less risk than not adding it. In addition, > >>>>> it is > >>>>> already added ;-). > >>>> > >>>> I am concerned that this is a harmful change. The disp_unit from > >>>> one > >>>> process has no meaningful use at another process. If the user > >>>> wants to > >>>> know it for some reason, they can use a separate communication > >>>> operation. > >>> I believe the potential damage is very limited. But we can have a > >>> vote. > >>> > >>> So far it's still yes: 5, no: 1 > >>> > >>> I'm just mediating, not making decisions. > >>> > >>> Torsten > >>> > >> > >> _______________________________________________ > >> mpi3-rma mailing list > >> mpi3-rma(a)lists.mpi-forum.org > >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma > >> > >> > >> > >> _______________________________________________ > >> mpi3-rma mailing list > >> mpi3-rma(a)lists.mpi-forum.org > >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma > >> > > > > -- > > ### qreharg rug ebs fv crryF ------------- http://www.unixer.de/ > > ----- > > Torsten Hoefler | Performance Modeling and Simulation Lead > > Blue Waters Directorate | University of Illinois (UIUC) > > 1205 W Clark Street | Urbana, IL, 61801 > > NCSA Building | +01 (217) 244-7736 > > _______________________________________________ > > mpi3-rma mailing list > > mpi3-rma(a)lists.mpi-forum.org > > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma > > > _______________________________________________ > mpi3-rma mailing list > mpi3-rma(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma -- Dr. Rolf Rabenseifner . . . . . . . . . .. email rabenseifner(a)hlrs.de High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner Nobelstr. 19, D-70550 Stuttgart, Germany . (Office: Allmandring 30)

1 0

Re: [Mpi-forum] [Mpi3-rma] [EXTERNAL] Re: Disp_unit in allocate_shared should be returned by win_shared_query?
by Rolf Rabenseifner 22 Jul '12

22 Jul '12

I'll come back to the technical discussion in a few minutes with a proposal based on Brian's technical argument: On Wed, Jul 18, 2012 at 08:11:23PM +0000, Barrett, Brian W wrote: > I'm concerned with the change (and don't think it should happen at this point, as I want to be able to tell other people their changes are too big). It's probably not another O(p) structure, since a process will have to know the other rank's disp_unit to do the memcpy during the communication operations, so we crossed the O(p) bridge when we added disp_unit to allocate_shared. Rolf ----- Original Message ----- > From: "Torsten Hoefler" <htor(a)illinois.edu> > To: "Bronis R. de Supinski" <bronis(a)llnl.gov> > Cc: "Main MPI Forum List" <mpi-forum(a)lists.mpi-forum.org>, "MPI 3.0 Remote Memory Access working group" > <mpi3-rma(a)lists.mpi-forum.org> > Sent: Sunday, July 22, 2012 7:07:45 PM > Subject: Re: [Mpi3-rma] [EXTERNAL] Re: Disp_unit in allocate_shared should be returned by win_shared_query? > On Sun, Jul 22, 2012 at 10:10:36AM -0700, Bronis R. de Supinski wrote: > > > > Internal consensus cannot include changes to the approved branch. > Again, the changes were approved by the Forum and they are still > committed. > > Torsten > > -- > ### qreharg rug ebs fv crryF ------------- http://www.unixer.de/ ----- > Torsten Hoefler | Performance Modeling and Simulation Lead > Blue Waters Directorate | University of Illinois (UIUC) > 1205 W Clark Street | Urbana, IL, 61801 > NCSA Building | +01 (217) 244-7736 > _______________________________________________ > mpi3-rma mailing list > mpi3-rma(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma -- Dr. Rolf Rabenseifner . . . . . . . . . .. email rabenseifner(a)hlrs.de High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner Nobelstr. 19, D-70550 Stuttgart, Germany . (Office: Allmandring 30)

1 0