July 2012 - mpiwg-rma - mailman3.cels.anl.gov

Re: [Mpi3-rma] [EXTERNAL] Re: Disp_unit in allocate_shared should be returned by win_shared_query?
by Barrett, Brian W 27 Jul '12

27 Jul '12

Sorry to chime in late; I've been on vacation the last week. I'm comfortable with option #2 and agree entirely with Bill that it's too late to add an advice to users about allocation. Brian On Jul 26, 2012, at 7:54 PM, William Gropp wrote: > The advice to users is not essential, and at this point, we should *only* be making essential changes. This is a standards document, not a user manual. I'm all for having man pages and tutorials say things like this, but we don't need it in the standard. > > Bill > > William Gropp > Director, Parallel Computing Institute > Deputy Director for Research > Institute for Advanced Computing Applications and Technologies > Paul and Cynthia Saylor Professor of Computer Science > University of Illinois Urbana-Champaign > > > > On Jul 26, 2012, at 4:22 PM, Rolf Rabenseifner wrote: > >>> I guess we can add this advice nevertheless. I think this would also >>> cover Rolf's comment. >> >> Not really. The text is orthogonal to my request. >> >>>> Advice to users. If disp_unit values differ across processes in a >>>> shared memory window, the programmer may need to use the target's >>>> disp_unit value in address calculations and use different load/store >>>> operations depending on the target's disp_unit value. (End of advice >>>> to >>>> users.) >> >> This is an advice especially for MPI_WIN_ALLOCATE_SHARED. >> It does not fit to MPI_WIN_ALLOCATE and the use of MPI_PUT/GET >> because in PUT/GET, the disp_unit works perfect. >> >> My question was a general one at MPI_WIN_CREATE >> to tell the users that the major intend is that all disp_unit values >> (related to the processes of one window handle) >> represent the extent of one common datatype. >> And, if one is doing different, this may cause a >> non-scaling memory overhead. >> >> Rolf >> >> ----- Original Message ----- >>> From: "Torsten Hoefler" <htor(a)illinois.edu> >>> To: "MPI 3.0 Remote Memory Access working group" <mpi3-rma(a)lists.mpi-forum.org> >>> Sent: Thursday, July 26, 2012 11:05:56 PM >>> Subject: Re: [Mpi3-rma] [EXTERNAL] Re: Disp_unit in allocate_shared should be returned by win_shared_query? >>> On Thu, Jul 26, 2012 at 11:18:05AM -0500, James Dinan wrote: >>>> I do have a concern that different disp_units can occur unexpectedly >>>> because of heterogeneous hardware or software -- e.g. MPMD launch >>>> with >>>> different executables or language incompatibility where both C and >>>> Fortran code are involved in creating the window. >>>> >>>> Rather than ruling these cases out (option #1), how about including >>>> the >>>> following advice to users: >>>> >>>> Advice to users. If disp_unit values differ across processes in a >>>> shared memory window, the programmer may need to use the target's >>>> disp_unit value in address calculations and use different load/store >>>> operations depending on the target's disp_unit value. (End of advice >>>> to >>>> users.) >>>> >>>> If we included this, I think I would be ok with option #2. >>> I don't object, however, we're supposed to freeze the document by >>> today. >>> I guess we can add this advice nevertheless. I think this would also >>> cover Rolf's comment. >>> >>>> An orthogonal observation -- it would have given us a lot more >>>> flexibility to define MPI_Win_shared_query as: >>>> >>>> int MPI_Win_shared_get_attr(MPI_Win win, int target_rank, int >>>> win_keyval, void *attribute_val, int *flag) >>> Of course. I still like the MPI_Func(...) idea :-). >>> >>> But I agree, this would also fit the current design with attributes >>> better. Hmm, is probably too late for such a syntactic sugar change. >>> Any >>> opinions? >>> >>> All the Best, >>> Torsten >>> >>> -- >>> ### qreharg rug ebs fv crryF ------------- http://www.unixer.de/ ----- >>> Torsten Hoefler | Performance Modeling and Simulation Lead >>> Blue Waters Directorate | University of Illinois (UIUC) >>> 1205 W Clark Street | Urbana, IL, 61801 >>> NCSA Building | +01 (217) 244-7736 >>> _______________________________________________ >>> mpi3-rma mailing list >>> mpi3-rma(a)lists.mpi-forum.org >>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma >> >> -- >> Dr. Rolf Rabenseifner . . . . . . . . . .. email rabenseifner(a)hlrs.de >> High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 >> University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 >> Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner >> Nobelstr. 19, D-70550 Stuttgart, Germany . (Office: Allmandring 30) >> _______________________________________________ >> mpi3-rma mailing list >> mpi3-rma(a)lists.mpi-forum.org >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma > > > _______________________________________________ > mpi3-rma mailing list > mpi3-rma(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma >

1 0

Re: [Mpi3-rma] [EXTERNAL] Re: Disp_unit in allocate_shared should be returned by win_shared_query?
by William Gropp 26 Jul '12

26 Jul '12

The advice to users is not essential, and at this point, we should *only* be making essential changes. This is a standards document, not a user manual. I'm all for having man pages and tutorials say things like this, but we don't need it in the standard. Bill William Gropp Director, Parallel Computing Institute Deputy Director for Research Institute for Advanced Computing Applications and Technologies Paul and Cynthia Saylor Professor of Computer Science University of Illinois Urbana-Champaign On Jul 26, 2012, at 4:22 PM, Rolf Rabenseifner wrote: >> I guess we can add this advice nevertheless. I think this would also >> cover Rolf's comment. > > Not really. The text is orthogonal to my request. > >>> Advice to users. If disp_unit values differ across processes in a >>> shared memory window, the programmer may need to use the target's >>> disp_unit value in address calculations and use different load/store >>> operations depending on the target's disp_unit value. (End of advice >>> to >>> users.) > > This is an advice especially for MPI_WIN_ALLOCATE_SHARED. > It does not fit to MPI_WIN_ALLOCATE and the use of MPI_PUT/GET > because in PUT/GET, the disp_unit works perfect. > > My question was a general one at MPI_WIN_CREATE > to tell the users that the major intend is that all disp_unit values > (related to the processes of one window handle) > represent the extent of one common datatype. > And, if one is doing different, this may cause a > non-scaling memory overhead. > > Rolf > > ----- Original Message ----- >> From: "Torsten Hoefler" <htor(a)illinois.edu> >> To: "MPI 3.0 Remote Memory Access working group" <mpi3-rma(a)lists.mpi-forum.org> >> Sent: Thursday, July 26, 2012 11:05:56 PM >> Subject: Re: [Mpi3-rma] [EXTERNAL] Re: Disp_unit in allocate_shared should be returned by win_shared_query? >> On Thu, Jul 26, 2012 at 11:18:05AM -0500, James Dinan wrote: >>> I do have a concern that different disp_units can occur unexpectedly >>> because of heterogeneous hardware or software -- e.g. MPMD launch >>> with >>> different executables or language incompatibility where both C and >>> Fortran code are involved in creating the window. >>> >>> Rather than ruling these cases out (option #1), how about including >>> the >>> following advice to users: >>> >>> Advice to users. If disp_unit values differ across processes in a >>> shared memory window, the programmer may need to use the target's >>> disp_unit value in address calculations and use different load/store >>> operations depending on the target's disp_unit value. (End of advice >>> to >>> users.) >>> >>> If we included this, I think I would be ok with option #2. >> I don't object, however, we're supposed to freeze the document by >> today. >> I guess we can add this advice nevertheless. I think this would also >> cover Rolf's comment. >> >>> An orthogonal observation -- it would have given us a lot more >>> flexibility to define MPI_Win_shared_query as: >>> >>> int MPI_Win_shared_get_attr(MPI_Win win, int target_rank, int >>> win_keyval, void *attribute_val, int *flag) >> Of course. I still like the MPI_Func(...) idea :-). >> >> But I agree, this would also fit the current design with attributes >> better. Hmm, is probably too late for such a syntactic sugar change. >> Any >> opinions? >> >> All the Best, >> Torsten >> >> -- >> ### qreharg rug ebs fv crryF ------------- http://www.unixer.de/ ----- >> Torsten Hoefler | Performance Modeling and Simulation Lead >> Blue Waters Directorate | University of Illinois (UIUC) >> 1205 W Clark Street | Urbana, IL, 61801 >> NCSA Building | +01 (217) 244-7736 >> _______________________________________________ >> mpi3-rma mailing list >> mpi3-rma(a)lists.mpi-forum.org >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma > > -- > Dr. Rolf Rabenseifner . . . . . . . . . .. email rabenseifner(a)hlrs.de > High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 > University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 > Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner > Nobelstr. 19, D-70550 Stuttgart, Germany . (Office: Allmandring 30) > _______________________________________________ > mpi3-rma mailing list > mpi3-rma(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma

1 0

Re: [Mpi3-rma] [EXTERNAL] Re: Disp_unit in allocate_shared should be returned by win_shared_query?
by Rolf Rabenseifner 26 Jul '12

26 Jul '12

> I guess we can add this advice nevertheless. I think this would also > cover Rolf's comment. Not really. The text is orthogonal to my request. > > Advice to users. If disp_unit values differ across processes in a > > shared memory window, the programmer may need to use the target's > > disp_unit value in address calculations and use different load/store > > operations depending on the target's disp_unit value. (End of advice > > to > > users.) This is an advice especially for MPI_WIN_ALLOCATE_SHARED. It does not fit to MPI_WIN_ALLOCATE and the use of MPI_PUT/GET because in PUT/GET, the disp_unit works perfect. My question was a general one at MPI_WIN_CREATE to tell the users that the major intend is that all disp_unit values (related to the processes of one window handle) represent the extent of one common datatype. And, if one is doing different, this may cause a non-scaling memory overhead. Rolf ----- Original Message ----- > From: "Torsten Hoefler" <htor(a)illinois.edu> > To: "MPI 3.0 Remote Memory Access working group" <mpi3-rma(a)lists.mpi-forum.org> > Sent: Thursday, July 26, 2012 11:05:56 PM > Subject: Re: [Mpi3-rma] [EXTERNAL] Re: Disp_unit in allocate_shared should be returned by win_shared_query? > On Thu, Jul 26, 2012 at 11:18:05AM -0500, James Dinan wrote: > > I do have a concern that different disp_units can occur unexpectedly > > because of heterogeneous hardware or software -- e.g. MPMD launch > > with > > different executables or language incompatibility where both C and > > Fortran code are involved in creating the window. > > > > Rather than ruling these cases out (option #1), how about including > > the > > following advice to users: > > > > Advice to users. If disp_unit values differ across processes in a > > shared memory window, the programmer may need to use the target's > > disp_unit value in address calculations and use different load/store > > operations depending on the target's disp_unit value. (End of advice > > to > > users.) > > > > If we included this, I think I would be ok with option #2. > I don't object, however, we're supposed to freeze the document by > today. > I guess we can add this advice nevertheless. I think this would also > cover Rolf's comment. > > > An orthogonal observation -- it would have given us a lot more > > flexibility to define MPI_Win_shared_query as: > > > > int MPI_Win_shared_get_attr(MPI_Win win, int target_rank, int > > win_keyval, void *attribute_val, int *flag) > Of course. I still like the MPI_Func(...) idea :-). > > But I agree, this would also fit the current design with attributes > better. Hmm, is probably too late for such a syntactic sugar change. > Any > opinions? > > All the Best, > Torsten > > -- > ### qreharg rug ebs fv crryF ------------- http://www.unixer.de/ ----- > Torsten Hoefler | Performance Modeling and Simulation Lead > Blue Waters Directorate | University of Illinois (UIUC) > 1205 W Clark Street | Urbana, IL, 61801 > NCSA Building | +01 (217) 244-7736 > _______________________________________________ > mpi3-rma mailing list > mpi3-rma(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma -- Dr. Rolf Rabenseifner . . . . . . . . . .. email rabenseifner(a)hlrs.de High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner Nobelstr. 19, D-70550 Stuttgart, Germany . (Office: Allmandring 30)

1 0

Re: [Mpi3-rma] [EXTERNAL] Re: Disp_unit in allocate_shared should be returned by win_shared_query?
by Torsten Hoefler 26 Jul '12

26 Jul '12

On Thu, Jul 26, 2012 at 11:18:05AM -0500, James Dinan wrote: > I do have a concern that different disp_units can occur unexpectedly > because of heterogeneous hardware or software -- e.g. MPMD launch with > different executables or language incompatibility where both C and > Fortran code are involved in creating the window. > > Rather than ruling these cases out (option #1), how about including the > following advice to users: > > Advice to users. If disp_unit values differ across processes in a > shared memory window, the programmer may need to use the target's > disp_unit value in address calculations and use different load/store > operations depending on the target's disp_unit value. (End of advice to > users.) > > If we included this, I think I would be ok with option #2. I don't object, however, we're supposed to freeze the document by today. I guess we can add this advice nevertheless. I think this would also cover Rolf's comment. > An orthogonal observation -- it would have given us a lot more > flexibility to define MPI_Win_shared_query as: > > int MPI_Win_shared_get_attr(MPI_Win win, int target_rank, int > win_keyval, void *attribute_val, int *flag) Of course. I still like the MPI_Func(...) idea :-). But I agree, this would also fit the current design with attributes better. Hmm, is probably too late for such a syntactic sugar change. Any opinions? All the Best, Torsten -- ### qreharg rug ebs fv crryF ------------- http://www.unixer.de/ ----- Torsten Hoefler | Performance Modeling and Simulation Lead Blue Waters Directorate | University of Illinois (UIUC) 1205 W Clark Street | Urbana, IL, 61801 NCSA Building | +01 (217) 244-7736

1 0

Re: [Mpi3-rma] [EXTERNAL] Re: Disp_unit in allocate_shared should be returned by win_shared_query?
by Hubert Ritzdorf 26 Jul '12

26 Jul '12

I vote for #2. Hubert ________________________________________ From: mpi3-rma-bounces(a)lists.mpi-forum.org [mpi3-rma-bounces(a)lists.mpi-forum.org] on behalf of Jim Dinan [dinan(a)mcs.anl.gov] Sent: Wednesday, July 25, 2012 8:45 PM To: mpi3-rma(a)lists.mpi-forum.org Subject: Re: [Mpi3-rma] [EXTERNAL] Re: Disp_unit in allocate_shared should be returned by win_shared_query? Hi All, We should try to reach a decision on this issue before the end of the week. I think it would be helpful to hear some other opinions. So far, we have three options to choose from: 1) Require that the displacement unit be the same at all processes for MPI_Win_allocate_shared(). 2) Allow different displacement units at each process in shared memory windows and allow processes to determine the displacement unit at remote processes via MPI_Win_shared_query(). This is in addition to the MPI-2 self-query already possible with MPI_Win_get_attr(MPI_WIN_DISP_UNIT). 3) Allow different displacement units at each process in shared memory windows and provide only the MPI-2 self-query already possible with MPI_Win_get_attr(MPI_WIN_DISP_UNIT). Manual communication would be needed to exchange disp_unit information. A (very) rough summary of the discussion on this issue: * Given that non-uniform disp_unit values are currently allowed, processes may need to use disp_unit in address calculations. * Requiring programmers to use the disp_unit at each process in address calculations diminishes the usefulness of the shared memory programming model. It's also not clear if there is a use case to support the non-uniform disp_unit model. Thanks, ~Jim. _______________________________________________ mpi3-rma mailing list mpi3-rma(a)lists.mpi-forum.org http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma Click https://www.mailcontrol.com/sr/wycxuHidNhjTndxI!oX7UoEGNNKKZK+evdchg5q1g6rv… to report this email as spam.

1 0

Re: [Mpi3-rma] MPI-3.1 RMA planning
by Pavan Balaji 26 Jul '12

26 Jul '12

On 07/26/2012 11:31 AM, Torsten Hoefler wrote: > On Thu, Jul 26, 2012 at 11:18:54AM -0500, Pavan Balaji wrote: >> >> On 07/26/2012 08:52 AM, Pavan Balaji wrote: >>>> I would like to add some info arguments to the list of things to >>>> consider. I don't have the full list at this point, but we have the >>>> "same_size" argument for create and allocate. However, we have no >>>> "same_displ_unit", which goes by the same rationale. We could also add >>>> some info arguments to dynamic windows to mitigate some of the >>>> implementation issues (allow optimized implementations on RMA systems). >>> >>> Sounds good. >> >> What does "same_size" mean for heterogeneous systems? > The size is in bytes, it's clearly defined in the draft standard. That was a rhetorical question to illustrate my point below :-). >> If I give 100 * sizeof(int) on both sides, that will not be the same >> size on a heterogeneous system. This might be worth a user advise. > Well, this is a user error. Correct. But it deserves a user or implementor advice, since it's pretty hard for the implementor to catch these details. >> 1. The standard currently states, "A put or accumulate must not access a >> target window once a load/store update or a put ... ". The term >> "load/store update" doesn't make sense. This has caused a major >> confusion even to the RMA working group, for example on the slide that >> we put together for compatibility between load, store, put, get, >> accumulate, etc. It should really say "load/store accesses" -- that is, >> simultaneous loads and PUTs are not allowed in the SEPARATE model. > Ah, I thought we cleaned all of those things up - one more straggler, > probably a ticket 0 (will not do it now since we're supposed to be > finalized). We can clean it up in 3.1. >> 2. The disp_unit is a weird semantic which is really meant to >> demonstrate what datatype I will be using. We are jumping through hoops >> to get the same_size and same_disp_unit measures which make little sense >> on heterogeneous systems. The correct way to do this would have been to >> not take a disp_unit parameter at all, and instead take a MPI_Datatype >> parameter. In this case, two different processes can give MPI_INT but >> have different type sizes. That adds better safety checks in MPI. > Maybe, but for some reasons, RMA windows are completely specified with > bytes. I am not sure what the reason for this was. > >> Unfortunately, all our window creation routines are screwed up in this >> manner. We should consider adding MPI_Win_create_type, >> MPI_Win_allocate_type, MPI_Win_allocate_shared_type in MPI-3.1 and >> deprecating the older routines. It would have been much better to do >> this in 3.0, but it's not a small change. > Ugs, this makes me shiver. I agree in principle, but it's ugly. This is just a starting point for discussion. I was just trying to illustrate the problem and give a starting recommendation for discussion. The underlying issue is that using bytes directly is a bad model in MPI, whether it is for send/recv or RMA. -- Pavan -- Pavan Balaji http://www.mcs.anl.gov/~balaji

1 0

Re: [Mpi3-rma] MPI-3.1 RMA planning
by Torsten Hoefler 26 Jul '12

26 Jul '12

On Thu, Jul 26, 2012 at 11:18:54AM -0500, Pavan Balaji wrote: > > On 07/26/2012 08:52 AM, Pavan Balaji wrote: >>> I would like to add some info arguments to the list of things to >>> consider. I don't have the full list at this point, but we have the >>> "same_size" argument for create and allocate. However, we have no >>> "same_displ_unit", which goes by the same rationale. We could also add >>> some info arguments to dynamic windows to mitigate some of the >>> implementation issues (allow optimized implementations on RMA systems). >> >> Sounds good. > > What does "same_size" mean for heterogeneous systems? The size is in bytes, it's clearly defined in the draft standard. > If I give 100 * sizeof(int) on both sides, that will not be the same > size on a heterogeneous system. This might be worth a user advise. Well, this is a user error. > A few other things that we noticed: > > 1. The standard currently states, "A put or accumulate must not access a > target window once a load/store update or a put ... ". The term > "load/store update" doesn't make sense. This has caused a major > confusion even to the RMA working group, for example on the slide that > we put together for compatibility between load, store, put, get, > accumulate, etc. It should really say "load/store accesses" -- that is, > simultaneous loads and PUTs are not allowed in the SEPARATE model. Ah, I thought we cleaned all of those things up - one more straggler, probably a ticket 0 (will not do it now since we're supposed to be finalized). > 2. The disp_unit is a weird semantic which is really meant to > demonstrate what datatype I will be using. We are jumping through hoops > to get the same_size and same_disp_unit measures which make little sense > on heterogeneous systems. The correct way to do this would have been to > not take a disp_unit parameter at all, and instead take a MPI_Datatype > parameter. In this case, two different processes can give MPI_INT but > have different type sizes. That adds better safety checks in MPI. Maybe, but for some reasons, RMA windows are completely specified with bytes. I am not sure what the reason for this was. > Unfortunately, all our window creation routines are screwed up in this > manner. We should consider adding MPI_Win_create_type, > MPI_Win_allocate_type, MPI_Win_allocate_shared_type in MPI-3.1 and > deprecating the older routines. It would have been much better to do > this in 3.0, but it's not a small change. Ugs, this makes me shiver. I agree in principle, but it's ugly. Torsten -- ### qreharg rug ebs fv crryF ------------- http://www.unixer.de/ ----- Torsten Hoefler | Performance Modeling and Simulation Lead Blue Waters Directorate | University of Illinois (UIUC) 1205 W Clark Street | Urbana, IL, 61801 NCSA Building | +01 (217) 244-7736

1 0

Re: [Mpi3-rma] MPI-3.1 RMA planning
by Pavan Balaji 26 Jul '12

26 Jul '12

On 07/26/2012 08:52 AM, Pavan Balaji wrote: >> I would like to add some info arguments to the list of things to >> consider. I don't have the full list at this point, but we have the >> "same_size" argument for create and allocate. However, we have no >> "same_displ_unit", which goes by the same rationale. We could also add >> some info arguments to dynamic windows to mitigate some of the >> implementation issues (allow optimized implementations on RMA systems). > > Sounds good. What does "same_size" mean for heterogeneous systems? If I give 100 * sizeof(int) on both sides, that will not be the same size on a heterogeneous system. This might be worth a user advise. A few other things that we noticed: 1. The standard currently states, "A put or accumulate must not access a target window once a load/store update or a put ... ". The term "load/store update" doesn't make sense. This has caused a major confusion even to the RMA working group, for example on the slide that we put together for compatibility between load, store, put, get, accumulate, etc. It should really say "load/store accesses" -- that is, simultaneous loads and PUTs are not allowed in the SEPARATE model. 2. The disp_unit is a weird semantic which is really meant to demonstrate what datatype I will be using. We are jumping through hoops to get the same_size and same_disp_unit measures which make little sense on heterogeneous systems. The correct way to do this would have been to not take a disp_unit parameter at all, and instead take a MPI_Datatype parameter. In this case, two different processes can give MPI_INT but have different type sizes. That adds better safety checks in MPI. Unfortunately, all our window creation routines are screwed up in this manner. We should consider adding MPI_Win_create_type, MPI_Win_allocate_type, MPI_Win_allocate_shared_type in MPI-3.1 and deprecating the older routines. It would have been much better to do this in 3.0, but it's not a small change. -- Pavan -- Pavan Balaji http://www.mcs.anl.gov/~balaji

1 0

Re: [Mpi3-rma] [EXTERNAL] Re: Disp_unit in allocate_shared should be returned by win_shared_query?
by Jim Dinan 26 Jul '12

26 Jul '12

I do have a concern that different disp_units can occur unexpectedly because of heterogeneous hardware or software -- e.g. MPMD launch with different executables or language incompatibility where both C and Fortran code are involved in creating the window. Rather than ruling these cases out (option #1), how about including the following advice to users: Advice to users. If disp_unit values differ across processes in a shared memory window, the programmer may need to use the target's disp_unit value in address calculations and use different load/store operations depending on the target's disp_unit value. (End of advice to users.) If we included this, I think I would be ok with option #2. An orthogonal observation -- it would have given us a lot more flexibility to define MPI_Win_shared_query as: int MPI_Win_shared_get_attr(MPI_Win win, int target_rank, int win_keyval, void *attribute_val, int *flag) ~Jim. On 7/26/12 9:29 AM, William Gropp wrote: > Thanks for the support :) > > I too vote for #2. I note that applications do not need to use the disp_unit in accessing data under this model, but will need it if they are acting to replace PUT etc (as they might if the code is designed to work with and without shared memory). > > Bill > > William Gropp > Director, Parallel Computing Institute > Deputy Director for Research > Institute for Advanced Computing Applications and Technologies > Paul and Cynthia Saylor Professor of Computer Science > University of Illinois Urbana-Champaign > > > > On Jul 25, 2012, at 2:16 PM, Torsten Hoefler wrote: > >> Hi Jim, >> >> Bill's argument convinced me, I think we should have the displ_unit in >> the query function because this would support future systems and comes >> at essentially no cost. If the unit is the same on all processes, it can >> be stored. This can be determined in O(log(P)) or O(1) time with an info >> argument. Compression is trivial. >> >>> We should try to reach a decision on this issue before the end of the >>> week. I think it would be helpful to hear some other opinions. So far, >>> we have three options to choose from: >>> >>> 1) Require that the displacement unit be the same at all processes for >>> MPI_Win_allocate_shared(). >> This would never allow us to run across heterogeneous systems. I don't >> know any heterogeneous system that supports the unified model but GPUs >> are coming :-). I agree to Jim that coherency would be tricky, so this >> hardware is unlikely. We could also always lift this restriction in >> future MPI versions, so I'm not opposed. >> >>> 2) Allow different displacement units at each process in shared memory >>> windows and allow processes to determine the displacement unit at remote >>> processes via MPI_Win_shared_query(). This is in addition to the MPI-2 >>> self-query already possible with MPI_Win_get_attr(MPI_WIN_DISP_UNIT). >> Yes, this is what we have at this point. >> >>> 3) Allow different displacement units at each process in shared memory >>> windows and provide only the MPI-2 self-query already possible with >>> MPI_Win_get_attr(MPI_WIN_DISP_UNIT). Manual communication would be >>> needed to exchange disp_unit information. >> This would require a wording change. >> >>> A (very) rough summary of the discussion on this issue: >>> >>> * Given that non-uniform disp_unit values are currently allowed, >>> processes may need to use disp_unit in address calculations. >> They may check if all processes' disp_unit is the same, but that's >> another complication in the interface. >> >>> * Requiring programmers to use the disp_unit at each process in address >>> calculations diminishes the usefulness of the shared memory programming >>> model. It's also not clear if there is a use case to support the >>> non-uniform disp_unit model. >> Agreed. >> >> I vote for #2. >> >> Best, >> Torsten >> >> -- >> ### qreharg rug ebs fv crryF ------------- http://www.unixer.de/ ----- >> Torsten Hoefler | Performance Modeling and Simulation Lead >> Blue Waters Directorate | University of Illinois (UIUC) >> 1205 W Clark Street | Urbana, IL, 61801 >> NCSA Building | +01 (217) 244-7736 >> _______________________________________________ >> mpi3-rma mailing list >> mpi3-rma(a)lists.mpi-forum.org >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma > > > _______________________________________________ > mpi3-rma mailing list > mpi3-rma(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma >

1 0

Re: [Mpi3-rma] [EXTERNAL] Re: Disp_unit in allocate_shared should be returned by win_shared_query?
by Rajeev Thakur 26 Jul '12

26 Jul '12

If there is no objection from Forum members to the addition of the parameter at a late stage (July mtg), we can leave it in there. If there is objection, we can revert back to the original version since the information is available via MPI_Win_get_attr. Rajeev On Jul 25, 2012, at 1:45 PM, Jim Dinan wrote: > Hi All, > > We should try to reach a decision on this issue before the end of the week. I think it would be helpful to hear some other opinions. So far, we have three options to choose from: > > 1) Require that the displacement unit be the same at all processes for MPI_Win_allocate_shared(). > > 2) Allow different displacement units at each process in shared memory windows and allow processes to determine the displacement unit at remote processes via MPI_Win_shared_query(). This is in addition to the MPI-2 self-query already possible with MPI_Win_get_attr(MPI_WIN_DISP_UNIT). > > 3) Allow different displacement units at each process in shared memory windows and provide only the MPI-2 self-query already possible with MPI_Win_get_attr(MPI_WIN_DISP_UNIT). Manual communication would be needed to exchange disp_unit information. > > > A (very) rough summary of the discussion on this issue: > > * Given that non-uniform disp_unit values are currently allowed, processes may need to use disp_unit in address calculations. > > * Requiring programmers to use the disp_unit at each process in address calculations diminishes the usefulness of the shared memory programming model. It's also not clear if there is a use case to support the non-uniform disp_unit model. > > Thanks, > ~Jim. > _______________________________________________ > mpi3-rma mailing list > mpi3-rma(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma

1 0