September 2012 - mpiwg-rma - mailman3.cels.anl.gov

[Mpi3-rma] MPI RMA revisions from public comment
by William Gropp 06 Sep '12

06 Sep '12

I have read through Jeremiah Willcock's careful reading of the RMA chapter and applied many of his fixes. I've attached a version of the chapter with changes in red (including ALL changes, not just the public comment changes). Deletions are not marked. I'll send a reply to Willcock to the comments list, but I wanted to get this out for checking now. Most of the non-trivial notes I've left for MPI-next. Bill William Gropp Director, Parallel Computing Institute Deputy Director for Research Institute for Advanced Computing Applications and Technologies Paul and Cynthia Saylor Professor of Computer Science University of Illinois Urbana-Champaign

1 0

[Mpi3-rma] MPI_NO_OP and origin buffer
by Jim Dinan 06 Sep '12

06 Sep '12

Hi All, It looks like we forgot to specify that for MPI_NO_OP in the new accumulate ops, the origin buffer is ignored (i.e. it's acceptable to pass NULL). We can add this text on page 441, line 30: When MPI_NO_OP is specified, the origin buffer is ignored. We already have such a use case in the linked list example. ~Jim.

1 0

Re: [Mpi3-rma] Fwd: [Mpi-comments] Freeing memory allocated in MPI_WIN_ALLOCATE_SHARED
by Hubert Ritzdorf 05 Sep '12

05 Sep '12

This is fine for me. Hubert ________________________________________ From: mpi3-rma-bounces(a)lists.mpi-forum.org [mpi3-rma-bounces(a)lists.mpi-forum.org] on behalf of William Gropp [wgropp(a)illinois.edu] Sent: Wednesday, September 05, 2012 5:47 PM To: MPI 3.0 Remote Memory Access working group Subject: Re: [Mpi3-rma] Fwd: [Mpi-comments] Freeing memory allocated in MPI_WIN_ALLOCATE_SHARED Torsten has pointed out that this approach has the fragile list problem, and I agree. A better option might be: Include this information on the affected window create routines (MPI_WIN_ALLOCATE and MPI_WIN_ALLOCATE_SHARED). Add to MPI_WIN_FREE that any memory allocated by a window creating routine is freed, without listing the affected routines. Comments? Bill William Gropp Director, Parallel Computing Institute Deputy Director for Research Institute for Advanced Computing Applications and Technologies Paul and Cynthia Saylor Professor of Computer Science University of Illinois Urbana-Champaign On Sep 5, 2012, at 4:11 PM, Jim Dinan wrote: > We could also shorten it to: > > If the window was created with \mpifunc{MPI\_WIN\_ALLOCATE} or \mpifunc{MPI\_WIN\_ALLOCATE\_SHARED}, \mpifunc{MPI\_WIN\_FREE} will free the window memory that was allocated during window creation. > > ~Jim. > > On 9/5/12 8:38 AM, Rajeev Thakur wrote: >> Another option is to just add an "or \mpifunc{MPI\_WIN\_ALLOCATE\_SHARED}" to the first sentence, as below, but either version is fine with me. >> >> If the window was created with \mpifunc{MPI\_WIN\_ALLOCATE} or \mpifunc{MPI\_WIN\_ALLOCATE\_SHARED}, >> \mpifunc{MPI\_WIN\_FREE} will free the window memory that was allocated in \mpifunc{MPI\_WIN\_ALLOCATE} or \mpifunc{MPI\_WIN\_ALLOCATE\_SHARED}. >> >> >> On Sep 5, 2012, at 2:50 AM, William Gropp wrote: >> >>> Here's the current text (p 415 line 6): >>> >>> If the >>> window was created with \mpifunc{MPI\_WIN\_ALLOCATE}, >>> \mpifunc{MPI\_WIN\_FREE} will free the window memory that was allocated >>> in \mpifunc{MPI\_WIN\_ALLOCATE}.} >>> >>> Here is my proposed correction >>> >>> (above text, followed by) >>> >>> If the window was created with \mpifunc{MPI\_WIN\_ALLOCATE\_SHARED}, >>> \mpifunc{MPI\_WIN\_FREE} will free the window memory that was allocated >>> in \mpifunc{MPI\_WIN\_ALLOCATE\_SHARED}. >>> >>> This is a sufficient change, since the document is a standard and we don't need to state the same behavior everywhere (note that MPI_WIN_ALLOCATE doesn't mention MPI_WIN_FREE in the description of MPI_WIN_ALLOCATE). >>> >>> Thoughts? Is this a "simple oversight" error that we can add? >>> >>> Bill >>> >>> >>> William Gropp >>> Director, Parallel Computing Institute >>> Deputy Director for Research >>> Institute for Advanced Computing Applications and Technologies >>> Paul and Cynthia Saylor Professor of Computer Science >>> University of Illinois Urbana-Champaign >>> >>> >>> >>> On Sep 5, 2012, at 12:39 AM, Jeff Hammond wrote: >>> >>>> Okay, I thought it might do that but despite having a paper copy of >>>> the MPI-3 RMA chapter three feet from me at the time, I was too lazy >>>> to look it up. >>>> >>>> Thanks for the clarification. >>>> >>>> Jeff >>>> >>>> On Tue, Sep 4, 2012 at 11:47 PM, Jim Dinan <dinan(a)mcs.anl.gov> wrote: >>>>> MPI_Win_free also frees memory allocated by MPI_Win_allocate. >>>>> >>>>> ~Jim. >>>>> >>>>> >>>>> On 9/4/12 4:22 PM, Jeff Hammond wrote: >>>>>> >>>>>> Shouldn't that be a new call (MPI_Win_free_shared) to keep the usage >>>>>> of MPI_Win_free uniform? MPI_Win_free doesn't deallocate the buffers >>>>>> in any other situation, does it? >>>>>> >>>>>> Jeff >>>>>> >>>>>> On Tue, Sep 4, 2012 at 4:07 PM, Jim Dinan <dinan(a)mcs.anl.gov> wrote: >>>>>>> >>>>>>> Hi All, >>>>>>> >>>>>>> It looks like we forgot to include the text specifying that >>>>>>> MPI_Win_allocate_shared memory is freed along with the window. >>>>>>> >>>>>>> ~Jim. >>>>>>> >>>>>>> >>>>>>> -------- Original Message -------- >>>>>>> Subject: [Mpi-comments] Freeing memory allocated in >>>>>>> MPI_WIN_ALLOCATE_SHARED >>>>>>> Date: Tue, 4 Sep 2012 12:49:10 +0000 >>>>>>> From: Hubert Ritzdorf <Hubert.Ritzdorf(a)emea.nec.com> >>>>>>> Reply-To: Comments on the MPI Standard >>>>>>> <mpi-comments(a)lists.mpi-forum.org> >>>>>>> To: mpi-comments(a)lists.mpi-forum.org >>>>>>> <mpi-comments(a)lists.mpi-forum.org> >>>>>>> >>>>>>> >>>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> is it correct that MPI_Win_free should free the memory allocated in >>>>>>> MPI_Win_allocate_shared ? >>>>>>> >>>>>>> MPI_Win_free (Page 416, Line 5-6) specifies this only for >>>>>>> memory allocated by MPI_Win_allocate. >>>>>>> I think that an additional reference to MPI_Win_allocate_shared is >>>>>>> missing also on Page 420, Line 31, Page 448 Line 25, Page 609 Line 31, >>>>>>> Page 613 Line 1, Page 614 Line 9, Page 614 Line 31. >>>>>>> >>>>>>> Hubert >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> _______________________________________________ >>>>>>> mpi3-rma mailing list >>>>>>> mpi3-rma(a)lists.mpi-forum.org >>>>>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma >>>>>> >>>>>> >>>>>> >>>>>> >>>>> _______________________________________________ >>>>> mpi3-rma mailing list >>>>> mpi3-rma(a)lists.mpi-forum.org >>>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma >>>> >>>> >>>> >>>> -- >>>> Jeff Hammond >>>> Argonne Leadership Computing Facility >>>> University of Chicago Computation Institute >>>> jhammond(a)alcf.anl.gov / (630) 252-5381 >>>> http://www.linkedin.com/in/jeffhammond >>>> https://wiki.alcf.anl.gov/parts/index.php/User:Jhammond >>>> _______________________________________________ >>>> mpi3-rma mailing list >>>> mpi3-rma(a)lists.mpi-forum.org >>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma >>> >>> >>> _______________________________________________ >>> mpi3-rma mailing list >>> mpi3-rma(a)lists.mpi-forum.org >>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma >> >> >> _______________________________________________ >> mpi3-rma mailing list >> mpi3-rma(a)lists.mpi-forum.org >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma >> > _______________________________________________ > mpi3-rma mailing list > mpi3-rma(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma _______________________________________________ mpi3-rma mailing list mpi3-rma(a)lists.mpi-forum.org http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma Click https://www.mailcontrol.com/sr/l43gx!pqRPLTndxI!oX7Uj!xNJBUaHW2GNON2WcL7Gy5… to report this email as spam.

1 0

[Mpi3-rma] MPI_WIN_CREATE_DYNAMIC and target_disp
by Hubert Ritzdorf 05 Sep '12

05 Sep '12

Hi, dependent on the actual discussion, I propose a change in the current version of MPI 3.0 draft. I send it to the rma list instead of mpi-comments list in order to get one of the discussed changes or this change into the MPI document: The argument target_disp in the RMA function MPI_{GET, RGET, PUT, RPUT, ACCUMULATE, ...} is qualified as non-negative integer. I think, that this cannot be guaranteed for the Fortran version of new function MPI_WIN_CREATE_DYNAMIC. The displacement is actually relative to MPI_BOTTOM which is a variable in a common block or module. The location of MPI_BOTTOM may be dependent on compiler (and possibly compiler options) and the sequence of common blocks/modules in memory. (Example: The memory of a window might be located in a common block, which is located in front of the common block which contains MPI_BOTTOM). Therefore, non-negative displacements cannot be guaranteed for windows created by MPI_WIN_CREATE_DYNAMIC in Fortran . The simple change would be to remove "non-negative". But I assume that a rationale would be required. Hubert

1 0

Re: [Mpi3-rma] Fwd: [Mpi-comments] Freeing memory allocated in MPI_WIN_ALLOCATE_SHARED
by William Gropp 05 Sep '12

05 Sep '12

Torsten has pointed out that this approach has the fragile list problem, and I agree. A better option might be: Include this information on the affected window create routines (MPI_WIN_ALLOCATE and MPI_WIN_ALLOCATE_SHARED). Add to MPI_WIN_FREE that any memory allocated by a window creating routine is freed, without listing the affected routines. Comments? Bill William Gropp Director, Parallel Computing Institute Deputy Director for Research Institute for Advanced Computing Applications and Technologies Paul and Cynthia Saylor Professor of Computer Science University of Illinois Urbana-Champaign On Sep 5, 2012, at 4:11 PM, Jim Dinan wrote: > We could also shorten it to: > > If the window was created with \mpifunc{MPI\_WIN\_ALLOCATE} or \mpifunc{MPI\_WIN\_ALLOCATE\_SHARED}, \mpifunc{MPI\_WIN\_FREE} will free the window memory that was allocated during window creation. > > ~Jim. > > On 9/5/12 8:38 AM, Rajeev Thakur wrote: >> Another option is to just add an "or \mpifunc{MPI\_WIN\_ALLOCATE\_SHARED}" to the first sentence, as below, but either version is fine with me. >> >> If the window was created with \mpifunc{MPI\_WIN\_ALLOCATE} or \mpifunc{MPI\_WIN\_ALLOCATE\_SHARED}, >> \mpifunc{MPI\_WIN\_FREE} will free the window memory that was allocated in \mpifunc{MPI\_WIN\_ALLOCATE} or \mpifunc{MPI\_WIN\_ALLOCATE\_SHARED}. >> >> >> On Sep 5, 2012, at 2:50 AM, William Gropp wrote: >> >>> Here's the current text (p 415 line 6): >>> >>> If the >>> window was created with \mpifunc{MPI\_WIN\_ALLOCATE}, >>> \mpifunc{MPI\_WIN\_FREE} will free the window memory that was allocated >>> in \mpifunc{MPI\_WIN\_ALLOCATE}.} >>> >>> Here is my proposed correction >>> >>> (above text, followed by) >>> >>> If the window was created with \mpifunc{MPI\_WIN\_ALLOCATE\_SHARED}, >>> \mpifunc{MPI\_WIN\_FREE} will free the window memory that was allocated >>> in \mpifunc{MPI\_WIN\_ALLOCATE\_SHARED}. >>> >>> This is a sufficient change, since the document is a standard and we don't need to state the same behavior everywhere (note that MPI_WIN_ALLOCATE doesn't mention MPI_WIN_FREE in the description of MPI_WIN_ALLOCATE). >>> >>> Thoughts? Is this a "simple oversight" error that we can add? >>> >>> Bill >>> >>> >>> William Gropp >>> Director, Parallel Computing Institute >>> Deputy Director for Research >>> Institute for Advanced Computing Applications and Technologies >>> Paul and Cynthia Saylor Professor of Computer Science >>> University of Illinois Urbana-Champaign >>> >>> >>> >>> On Sep 5, 2012, at 12:39 AM, Jeff Hammond wrote: >>> >>>> Okay, I thought it might do that but despite having a paper copy of >>>> the MPI-3 RMA chapter three feet from me at the time, I was too lazy >>>> to look it up. >>>> >>>> Thanks for the clarification. >>>> >>>> Jeff >>>> >>>> On Tue, Sep 4, 2012 at 11:47 PM, Jim Dinan <dinan(a)mcs.anl.gov> wrote: >>>>> MPI_Win_free also frees memory allocated by MPI_Win_allocate. >>>>> >>>>> ~Jim. >>>>> >>>>> >>>>> On 9/4/12 4:22 PM, Jeff Hammond wrote: >>>>>> >>>>>> Shouldn't that be a new call (MPI_Win_free_shared) to keep the usage >>>>>> of MPI_Win_free uniform? MPI_Win_free doesn't deallocate the buffers >>>>>> in any other situation, does it? >>>>>> >>>>>> Jeff >>>>>> >>>>>> On Tue, Sep 4, 2012 at 4:07 PM, Jim Dinan <dinan(a)mcs.anl.gov> wrote: >>>>>>> >>>>>>> Hi All, >>>>>>> >>>>>>> It looks like we forgot to include the text specifying that >>>>>>> MPI_Win_allocate_shared memory is freed along with the window. >>>>>>> >>>>>>> ~Jim. >>>>>>> >>>>>>> >>>>>>> -------- Original Message -------- >>>>>>> Subject: [Mpi-comments] Freeing memory allocated in >>>>>>> MPI_WIN_ALLOCATE_SHARED >>>>>>> Date: Tue, 4 Sep 2012 12:49:10 +0000 >>>>>>> From: Hubert Ritzdorf <Hubert.Ritzdorf(a)emea.nec.com> >>>>>>> Reply-To: Comments on the MPI Standard >>>>>>> <mpi-comments(a)lists.mpi-forum.org> >>>>>>> To: mpi-comments(a)lists.mpi-forum.org >>>>>>> <mpi-comments(a)lists.mpi-forum.org> >>>>>>> >>>>>>> >>>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> is it correct that MPI_Win_free should free the memory allocated in >>>>>>> MPI_Win_allocate_shared ? >>>>>>> >>>>>>> MPI_Win_free (Page 416, Line 5-6) specifies this only for >>>>>>> memory allocated by MPI_Win_allocate. >>>>>>> I think that an additional reference to MPI_Win_allocate_shared is >>>>>>> missing also on Page 420, Line 31, Page 448 Line 25, Page 609 Line 31, >>>>>>> Page 613 Line 1, Page 614 Line 9, Page 614 Line 31. >>>>>>> >>>>>>> Hubert >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> _______________________________________________ >>>>>>> mpi3-rma mailing list >>>>>>> mpi3-rma(a)lists.mpi-forum.org >>>>>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma >>>>>> >>>>>> >>>>>> >>>>>> >>>>> _______________________________________________ >>>>> mpi3-rma mailing list >>>>> mpi3-rma(a)lists.mpi-forum.org >>>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma >>>> >>>> >>>> >>>> -- >>>> Jeff Hammond >>>> Argonne Leadership Computing Facility >>>> University of Chicago Computation Institute >>>> jhammond(a)alcf.anl.gov / (630) 252-5381 >>>> http://www.linkedin.com/in/jeffhammond >>>> https://wiki.alcf.anl.gov/parts/index.php/User:Jhammond >>>> _______________________________________________ >>>> mpi3-rma mailing list >>>> mpi3-rma(a)lists.mpi-forum.org >>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma >>> >>> >>> _______________________________________________ >>> mpi3-rma mailing list >>> mpi3-rma(a)lists.mpi-forum.org >>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma >> >> >> _______________________________________________ >> mpi3-rma mailing list >> mpi3-rma(a)lists.mpi-forum.org >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma >> > _______________________________________________ > mpi3-rma mailing list > mpi3-rma(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma

1 0

Re: [Mpi3-rma] Target displacement sign issue
by Hubert Ritzdorf 05 Sep '12

05 Sep '12

Right, I was thinking in displacements for windows. In C , it could be a macro similar to ptr = (char *) ( ((unsigned long long) MPI_MAX_INT) + displacement_dynamic + 1) In Fortran, there is actually no MPI possibility to recompute the address from a displacement relative to MPI_BOTTOM. Hubert ________________________________________ From: mpi3-rma-bounces(a)lists.mpi-forum.org [mpi3-rma-bounces(a)lists.mpi-forum.org] on behalf of Jim Dinan [dinan(a)mcs.anl.gov] Sent: Wednesday, September 05, 2012 4:13 PM To: mpi3-rma(a)lists.mpi-forum.org Subject: Re: [Mpi3-rma] Target displacement sign issue Hi Hubert, If I understand this correctly, then one would no longer be able to cast an MPI_Aint back to a pointer. We would have to add another routine, MPI_Get_pointer_dynamic() to do this conversion. ~Jim. On 9/4/12 6:25 PM, Hubert Ritzdorf wrote: > > Hi, > > Possibly another solution: > > (*) Let MPI_AINT_MAX the maximal value of MPI_Aint variable > (*) Let the window_base of a window created by MPI_WIN_CREATE_DYNAMIC > be MPI_AINT_MAX + 1 > (*) Define a new function MPI_Get_address_dynamic returning the > address of a location relative to MPI_AINT_MAX + 1 > (*) Remove the non-negative qualification in the description > of target_disp in MPI_{Put/Get/Accumulate/...). > > Hubert > >> -----Original Message----- >> From: mpi3-rma-bounces(a)lists.mpi-forum.org [mailto:mpi3-rma- >> bounces(a)lists.mpi-forum.org] On Behalf Of Hubert Ritzdorf >> Sent: Tuesday, September 04, 2012 7:23 PM >> To: MPI 3.0 Remote Memory Access working group >> Subject: Re: [Mpi3-rma] Target displacement sign issue >> >> Hi, >> >> MPI_Get_address returns an MPI_Aint which is signed.(integer). >> >> C: >> MPI_Aint's may be used in Reduce Operations with Min/Max >> or the application can internally use min/max computations. >> Can anybody guarantee that applications don't perform >> min/max operations on these displacements (which results in different >> results for unsigned and signed variables) or use derived values of these >> displacements in other signed/unsigned significant cases. >> >> Fortran: >> MPI_BOTTOM is a variable in a common block/module. Thus, >> MPI_Get_address might really return a signed integer (depending on the >> compiler). >> Fortran doesn't have unsigned integers and cannot handle them. >> Thus, the non-negative integer specification for target_disp in >> MPI_{Put/Get/Accumulate/...) >> might be incorrect for Fortran and MPI_Win_Create_dynamic(). >> >> Hubert >> ________________________________________ >> From: mpi3-rma-bounces(a)lists.mpi-forum.org [mpi3-rma-bounces(a)lists.mpi- >> forum.org] on behalf of Jim Dinan [dinan(a)mcs.anl.gov] >> Sent: Tuesday, September 04, 2012 5:10 PM >> To: mpi3-rma(a)lists.mpi-forum.org >> Subject: Re: [Mpi3-rma] Target displacement sign issue >> >> Hi All, >> >> This is an important issue that we need to resolve before releasing the >> 3.0 spec. Please take a look and send feedback. >> >> Dave has voiced a concern that the implicit type conversion in my >> proposed text below may not be portable. If anyone has more experience >> with this aspect (signed->unsigned conversion) of C/Fortran/etc type >> systems, please chime in. >> >> ~Jim. >> >> On 8/28/12 2:29 PM, Jim Dinan wrote: >>> Hi All, >>> >>> Here is the suggested change to address this issue. In the paragraph on >>> page 425, line 1, add the bracketed sentence: >>> >>> In the case of a window created with MPI_WIN_CREATE_DYNAMIC, the >>> target_disp for all RMA functions is the address at the target; i.e., >>> the effective window_base is MPI_BOTTOM and the disp_unit is one. [Thus, >>> for dynamic windows, the target_disp argument to all RMA functions in >>> interpreted as unsigned.] Users should use MPI_GET_ADDRESS at the target >>> process to determine the address of a target memory location and >>> communicate this address to the origin process. >>> >>> ~Jim. >>> >>> On 8/28/12 9:46 AM, Jim Dinan wrote: >>>> Hi All, >>>> >>>> Currently, target displacements in put/get/acc/etc are defined to be >>>> non-negative integers of type, MPI_Aint. MPI_Aint is defined to be a >>>> signed integer. However, displacements associated with dynamic windows >>>> are addresses (unsigned) and can range into the space of negative >>>> integers. >>>> >>>> It seems like the least impact change would be to add a special case >>>> that the displacement argument will be interpreted as unsigned for >>>> dynamic windows. Changing the interface to take an 'unsigned MPI_Aint' >>>> would be the most clear (and preserve the existing and intended new >>>> semantics), but would break backward compatibility. >>>> >>>> Interested to hear your thoughts. >>>> >>>> ~Jim. >> _______________________________________________ >> mpi3-rma mailing list >> mpi3-rma(a)lists.mpi-forum.org >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma >> >> >> Click >> https://www.mailcontrol.com/sr/O6SiJI5AP4bTndxI!oX7Up2F0f+S6NySj!qlvEoKFyZ >> NTFyVkk9WjRzJbmnfHdyw83VGVWKbSfC8ewgPpp7kgg== to report this email as >> spam. >> >> _______________________________________________ >> mpi3-rma mailing list >> mpi3-rma(a)lists.mpi-forum.org >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma > > _______________________________________________ > mpi3-rma mailing list > mpi3-rma(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma > _______________________________________________ mpi3-rma mailing list mpi3-rma(a)lists.mpi-forum.org http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma

1 0

Re: [Mpi3-rma] Target displacement sign issue
by Jeff Hammond 05 Sep '12

05 Sep '12

It is a not a 3.0 change but it would be better to make these routines macro-able like mpi_wtime so that a user could call MPI_Get_address inside of a loop and not get hosed. I don't see a way to avoid this behavior in some cases. Jeff On Wed, Sep 5, 2012 at 4:18 PM, Jim Dinan <dinan(a)mcs.anl.gov> wrote: > This is true, but we've relied on a correspondence between Aints and > pointers in a variety use cases for dynamic windows, e.g. the linked list > example. > > ~Jim. > > > On 9/5/12 9:15 AM, Pavan Balaji wrote: >> >> >> FYI, casting back MPI_Aint to a pointer was never safe (even in >> MPI-2.2), since MPI_Aint can be larger than void *. >> >> -- Pavan >> >> On 09/05/2012 09:13 AM, Jim Dinan wrote: >>> >>> Hi Hubert, >>> >>> If I understand this correctly, then one would no longer be able to cast >>> an MPI_Aint back to a pointer. We would have to add another routine, >>> MPI_Get_pointer_dynamic() to do this conversion. >>> >>> ~Jim. >>> >>> On 9/4/12 6:25 PM, Hubert Ritzdorf wrote: >>>> >>>> >>>> Hi, >>>> >>>> Possibly another solution: >>>> >>>> (*) Let MPI_AINT_MAX the maximal value of MPI_Aint variable >>>> (*) Let the window_base of a window created by MPI_WIN_CREATE_DYNAMIC >>>> be MPI_AINT_MAX + 1 >>>> (*) Define a new function MPI_Get_address_dynamic returning the >>>> address of a location relative to MPI_AINT_MAX + 1 >>>> (*) Remove the non-negative qualification in the description >>>> of target_disp in MPI_{Put/Get/Accumulate/...). >>>> >>>> Hubert >>>> >>>>> -----Original Message----- >>>>> From: mpi3-rma-bounces(a)lists.mpi-forum.org [mailto:mpi3-rma- >>>>> bounces(a)lists.mpi-forum.org] On Behalf Of Hubert Ritzdorf >>>>> Sent: Tuesday, September 04, 2012 7:23 PM >>>>> To: MPI 3.0 Remote Memory Access working group >>>>> Subject: Re: [Mpi3-rma] Target displacement sign issue >>>>> >>>>> Hi, >>>>> >>>>> MPI_Get_address returns an MPI_Aint which is signed.(integer). >>>>> >>>>> C: >>>>> MPI_Aint's may be used in Reduce Operations with Min/Max >>>>> or the application can internally use min/max computations. >>>>> Can anybody guarantee that applications don't perform >>>>> min/max operations on these displacements (which results in different >>>>> results for unsigned and signed variables) or use derived values of >>>>> these >>>>> displacements in other signed/unsigned significant cases. >>>>> >>>>> Fortran: >>>>> MPI_BOTTOM is a variable in a common block/module. Thus, >>>>> MPI_Get_address might really return a signed integer (depending on the >>>>> compiler). >>>>> Fortran doesn't have unsigned integers and cannot handle them. >>>>> Thus, the non-negative integer specification for target_disp in >>>>> MPI_{Put/Get/Accumulate/...) >>>>> might be incorrect for Fortran and MPI_Win_Create_dynamic(). >>>>> >>>>> Hubert >>>>> ________________________________________ >>>>> From: mpi3-rma-bounces(a)lists.mpi-forum.org [mpi3-rma-bounces(a)lists.mpi- >>>>> forum.org] on behalf of Jim Dinan [dinan(a)mcs.anl.gov] >>>>> Sent: Tuesday, September 04, 2012 5:10 PM >>>>> To: mpi3-rma(a)lists.mpi-forum.org >>>>> Subject: Re: [Mpi3-rma] Target displacement sign issue >>>>> >>>>> Hi All, >>>>> >>>>> This is an important issue that we need to resolve before releasing the >>>>> 3.0 spec. Please take a look and send feedback. >>>>> >>>>> Dave has voiced a concern that the implicit type conversion in my >>>>> proposed text below may not be portable. If anyone has more experience >>>>> with this aspect (signed->unsigned conversion) of C/Fortran/etc type >>>>> systems, please chime in. >>>>> >>>>> ~Jim. >>>>> >>>>> On 8/28/12 2:29 PM, Jim Dinan wrote: >>>>>> >>>>>> Hi All, >>>>>> >>>>>> Here is the suggested change to address this issue. In the >>>>>> paragraph on >>>>>> page 425, line 1, add the bracketed sentence: >>>>>> >>>>>> In the case of a window created with MPI_WIN_CREATE_DYNAMIC, the >>>>>> target_disp for all RMA functions is the address at the target; i.e., >>>>>> the effective window_base is MPI_BOTTOM and the disp_unit is one. >>>>>> [Thus, >>>>>> for dynamic windows, the target_disp argument to all RMA functions in >>>>>> interpreted as unsigned.] Users should use MPI_GET_ADDRESS at the >>>>>> target >>>>>> process to determine the address of a target memory location and >>>>>> communicate this address to the origin process. >>>>>> >>>>>> ~Jim. >>>>>> >>>>>> On 8/28/12 9:46 AM, Jim Dinan wrote: >>>>>>> >>>>>>> Hi All, >>>>>>> >>>>>>> Currently, target displacements in put/get/acc/etc are defined to be >>>>>>> non-negative integers of type, MPI_Aint. MPI_Aint is defined to be a >>>>>>> signed integer. However, displacements associated with dynamic >>>>>>> windows >>>>>>> are addresses (unsigned) and can range into the space of negative >>>>>>> integers. >>>>>>> >>>>>>> It seems like the least impact change would be to add a special case >>>>>>> that the displacement argument will be interpreted as unsigned for >>>>>>> dynamic windows. Changing the interface to take an 'unsigned >>>>>>> MPI_Aint' >>>>>>> would be the most clear (and preserve the existing and intended new >>>>>>> semantics), but would break backward compatibility. >>>>>>> >>>>>>> Interested to hear your thoughts. >>>>>>> >>>>>>> ~Jim. >>>>> >>>>> _______________________________________________ >>>>> mpi3-rma mailing list >>>>> mpi3-rma(a)lists.mpi-forum.org >>>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma >>>>> >>>>> >>>>> Click >>>>> >>>>> https://www.mailcontrol.com/sr/O6SiJI5AP4bTndxI!oX7Up2F0f+S6NySj!qlvEoKFyZ >>>>> >>>>> NTFyVkk9WjRzJbmnfHdyw83VGVWKbSfC8ewgPpp7kgg== to report this email as >>>>> spam. >>>>> >>>>> _______________________________________________ >>>>> mpi3-rma mailing list >>>>> mpi3-rma(a)lists.mpi-forum.org >>>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma >>>> >>>> >>>> _______________________________________________ >>>> mpi3-rma mailing list >>>> mpi3-rma(a)lists.mpi-forum.org >>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma >>>> >>> _______________________________________________ >>> mpi3-rma mailing list >>> mpi3-rma(a)lists.mpi-forum.org >>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma >>> >> > _______________________________________________ > mpi3-rma mailing list > mpi3-rma(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma -- Jeff Hammond Argonne Leadership Computing Facility University of Chicago Computation Institute jhammond(a)alcf.anl.gov / (630) 252-5381 http://www.linkedin.com/in/jeffhammond https://wiki.alcf.anl.gov/parts/index.php/User:Jhammond

1 0

Re: [Mpi3-rma] Target displacement sign issue
by Jim Dinan 05 Sep '12

05 Sep '12

This is true, but we've relied on a correspondence between Aints and pointers in a variety use cases for dynamic windows, e.g. the linked list example. ~Jim. On 9/5/12 9:15 AM, Pavan Balaji wrote: > > FYI, casting back MPI_Aint to a pointer was never safe (even in > MPI-2.2), since MPI_Aint can be larger than void *. > > -- Pavan > > On 09/05/2012 09:13 AM, Jim Dinan wrote: >> Hi Hubert, >> >> If I understand this correctly, then one would no longer be able to cast >> an MPI_Aint back to a pointer. We would have to add another routine, >> MPI_Get_pointer_dynamic() to do this conversion. >> >> ~Jim. >> >> On 9/4/12 6:25 PM, Hubert Ritzdorf wrote: >>> >>> Hi, >>> >>> Possibly another solution: >>> >>> (*) Let MPI_AINT_MAX the maximal value of MPI_Aint variable >>> (*) Let the window_base of a window created by MPI_WIN_CREATE_DYNAMIC >>> be MPI_AINT_MAX + 1 >>> (*) Define a new function MPI_Get_address_dynamic returning the >>> address of a location relative to MPI_AINT_MAX + 1 >>> (*) Remove the non-negative qualification in the description >>> of target_disp in MPI_{Put/Get/Accumulate/...). >>> >>> Hubert >>> >>>> -----Original Message----- >>>> From: mpi3-rma-bounces(a)lists.mpi-forum.org [mailto:mpi3-rma- >>>> bounces(a)lists.mpi-forum.org] On Behalf Of Hubert Ritzdorf >>>> Sent: Tuesday, September 04, 2012 7:23 PM >>>> To: MPI 3.0 Remote Memory Access working group >>>> Subject: Re: [Mpi3-rma] Target displacement sign issue >>>> >>>> Hi, >>>> >>>> MPI_Get_address returns an MPI_Aint which is signed.(integer). >>>> >>>> C: >>>> MPI_Aint's may be used in Reduce Operations with Min/Max >>>> or the application can internally use min/max computations. >>>> Can anybody guarantee that applications don't perform >>>> min/max operations on these displacements (which results in different >>>> results for unsigned and signed variables) or use derived values of >>>> these >>>> displacements in other signed/unsigned significant cases. >>>> >>>> Fortran: >>>> MPI_BOTTOM is a variable in a common block/module. Thus, >>>> MPI_Get_address might really return a signed integer (depending on the >>>> compiler). >>>> Fortran doesn't have unsigned integers and cannot handle them. >>>> Thus, the non-negative integer specification for target_disp in >>>> MPI_{Put/Get/Accumulate/...) >>>> might be incorrect for Fortran and MPI_Win_Create_dynamic(). >>>> >>>> Hubert >>>> ________________________________________ >>>> From: mpi3-rma-bounces(a)lists.mpi-forum.org [mpi3-rma-bounces(a)lists.mpi- >>>> forum.org] on behalf of Jim Dinan [dinan(a)mcs.anl.gov] >>>> Sent: Tuesday, September 04, 2012 5:10 PM >>>> To: mpi3-rma(a)lists.mpi-forum.org >>>> Subject: Re: [Mpi3-rma] Target displacement sign issue >>>> >>>> Hi All, >>>> >>>> This is an important issue that we need to resolve before releasing the >>>> 3.0 spec. Please take a look and send feedback. >>>> >>>> Dave has voiced a concern that the implicit type conversion in my >>>> proposed text below may not be portable. If anyone has more experience >>>> with this aspect (signed->unsigned conversion) of C/Fortran/etc type >>>> systems, please chime in. >>>> >>>> ~Jim. >>>> >>>> On 8/28/12 2:29 PM, Jim Dinan wrote: >>>>> Hi All, >>>>> >>>>> Here is the suggested change to address this issue. In the >>>>> paragraph on >>>>> page 425, line 1, add the bracketed sentence: >>>>> >>>>> In the case of a window created with MPI_WIN_CREATE_DYNAMIC, the >>>>> target_disp for all RMA functions is the address at the target; i.e., >>>>> the effective window_base is MPI_BOTTOM and the disp_unit is one. >>>>> [Thus, >>>>> for dynamic windows, the target_disp argument to all RMA functions in >>>>> interpreted as unsigned.] Users should use MPI_GET_ADDRESS at the >>>>> target >>>>> process to determine the address of a target memory location and >>>>> communicate this address to the origin process. >>>>> >>>>> ~Jim. >>>>> >>>>> On 8/28/12 9:46 AM, Jim Dinan wrote: >>>>>> Hi All, >>>>>> >>>>>> Currently, target displacements in put/get/acc/etc are defined to be >>>>>> non-negative integers of type, MPI_Aint. MPI_Aint is defined to be a >>>>>> signed integer. However, displacements associated with dynamic >>>>>> windows >>>>>> are addresses (unsigned) and can range into the space of negative >>>>>> integers. >>>>>> >>>>>> It seems like the least impact change would be to add a special case >>>>>> that the displacement argument will be interpreted as unsigned for >>>>>> dynamic windows. Changing the interface to take an 'unsigned >>>>>> MPI_Aint' >>>>>> would be the most clear (and preserve the existing and intended new >>>>>> semantics), but would break backward compatibility. >>>>>> >>>>>> Interested to hear your thoughts. >>>>>> >>>>>> ~Jim. >>>> _______________________________________________ >>>> mpi3-rma mailing list >>>> mpi3-rma(a)lists.mpi-forum.org >>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma >>>> >>>> >>>> Click >>>> https://www.mailcontrol.com/sr/O6SiJI5AP4bTndxI!oX7Up2F0f+S6NySj!qlvEoKFyZ >>>> >>>> NTFyVkk9WjRzJbmnfHdyw83VGVWKbSfC8ewgPpp7kgg== to report this email as >>>> spam. >>>> >>>> _______________________________________________ >>>> mpi3-rma mailing list >>>> mpi3-rma(a)lists.mpi-forum.org >>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma >>> >>> _______________________________________________ >>> mpi3-rma mailing list >>> mpi3-rma(a)lists.mpi-forum.org >>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma >>> >> _______________________________________________ >> mpi3-rma mailing list >> mpi3-rma(a)lists.mpi-forum.org >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma >> >

1 0

Re: [Mpi3-rma] Target displacement sign issue
by Pavan Balaji 05 Sep '12

05 Sep '12

FYI, casting back MPI_Aint to a pointer was never safe (even in MPI-2.2), since MPI_Aint can be larger than void *. -- Pavan On 09/05/2012 09:13 AM, Jim Dinan wrote: > Hi Hubert, > > If I understand this correctly, then one would no longer be able to cast > an MPI_Aint back to a pointer. We would have to add another routine, > MPI_Get_pointer_dynamic() to do this conversion. > > ~Jim. > > On 9/4/12 6:25 PM, Hubert Ritzdorf wrote: >> >> Hi, >> >> Possibly another solution: >> >> (*) Let MPI_AINT_MAX the maximal value of MPI_Aint variable >> (*) Let the window_base of a window created by MPI_WIN_CREATE_DYNAMIC >> be MPI_AINT_MAX + 1 >> (*) Define a new function MPI_Get_address_dynamic returning the >> address of a location relative to MPI_AINT_MAX + 1 >> (*) Remove the non-negative qualification in the description >> of target_disp in MPI_{Put/Get/Accumulate/...). >> >> Hubert >> >>> -----Original Message----- >>> From: mpi3-rma-bounces(a)lists.mpi-forum.org [mailto:mpi3-rma- >>> bounces(a)lists.mpi-forum.org] On Behalf Of Hubert Ritzdorf >>> Sent: Tuesday, September 04, 2012 7:23 PM >>> To: MPI 3.0 Remote Memory Access working group >>> Subject: Re: [Mpi3-rma] Target displacement sign issue >>> >>> Hi, >>> >>> MPI_Get_address returns an MPI_Aint which is signed.(integer). >>> >>> C: >>> MPI_Aint's may be used in Reduce Operations with Min/Max >>> or the application can internally use min/max computations. >>> Can anybody guarantee that applications don't perform >>> min/max operations on these displacements (which results in different >>> results for unsigned and signed variables) or use derived values of these >>> displacements in other signed/unsigned significant cases. >>> >>> Fortran: >>> MPI_BOTTOM is a variable in a common block/module. Thus, >>> MPI_Get_address might really return a signed integer (depending on the >>> compiler). >>> Fortran doesn't have unsigned integers and cannot handle them. >>> Thus, the non-negative integer specification for target_disp in >>> MPI_{Put/Get/Accumulate/...) >>> might be incorrect for Fortran and MPI_Win_Create_dynamic(). >>> >>> Hubert >>> ________________________________________ >>> From: mpi3-rma-bounces(a)lists.mpi-forum.org [mpi3-rma-bounces(a)lists.mpi- >>> forum.org] on behalf of Jim Dinan [dinan(a)mcs.anl.gov] >>> Sent: Tuesday, September 04, 2012 5:10 PM >>> To: mpi3-rma(a)lists.mpi-forum.org >>> Subject: Re: [Mpi3-rma] Target displacement sign issue >>> >>> Hi All, >>> >>> This is an important issue that we need to resolve before releasing the >>> 3.0 spec. Please take a look and send feedback. >>> >>> Dave has voiced a concern that the implicit type conversion in my >>> proposed text below may not be portable. If anyone has more experience >>> with this aspect (signed->unsigned conversion) of C/Fortran/etc type >>> systems, please chime in. >>> >>> ~Jim. >>> >>> On 8/28/12 2:29 PM, Jim Dinan wrote: >>>> Hi All, >>>> >>>> Here is the suggested change to address this issue. In the paragraph on >>>> page 425, line 1, add the bracketed sentence: >>>> >>>> In the case of a window created with MPI_WIN_CREATE_DYNAMIC, the >>>> target_disp for all RMA functions is the address at the target; i.e., >>>> the effective window_base is MPI_BOTTOM and the disp_unit is one. [Thus, >>>> for dynamic windows, the target_disp argument to all RMA functions in >>>> interpreted as unsigned.] Users should use MPI_GET_ADDRESS at the target >>>> process to determine the address of a target memory location and >>>> communicate this address to the origin process. >>>> >>>> ~Jim. >>>> >>>> On 8/28/12 9:46 AM, Jim Dinan wrote: >>>>> Hi All, >>>>> >>>>> Currently, target displacements in put/get/acc/etc are defined to be >>>>> non-negative integers of type, MPI_Aint. MPI_Aint is defined to be a >>>>> signed integer. However, displacements associated with dynamic windows >>>>> are addresses (unsigned) and can range into the space of negative >>>>> integers. >>>>> >>>>> It seems like the least impact change would be to add a special case >>>>> that the displacement argument will be interpreted as unsigned for >>>>> dynamic windows. Changing the interface to take an 'unsigned MPI_Aint' >>>>> would be the most clear (and preserve the existing and intended new >>>>> semantics), but would break backward compatibility. >>>>> >>>>> Interested to hear your thoughts. >>>>> >>>>> ~Jim. >>> _______________________________________________ >>> mpi3-rma mailing list >>> mpi3-rma(a)lists.mpi-forum.org >>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma >>> >>> >>> Click >>> https://www.mailcontrol.com/sr/O6SiJI5AP4bTndxI!oX7Up2F0f+S6NySj!qlvEoKFyZ >>> NTFyVkk9WjRzJbmnfHdyw83VGVWKbSfC8ewgPpp7kgg== to report this email as >>> spam. >>> >>> _______________________________________________ >>> mpi3-rma mailing list >>> mpi3-rma(a)lists.mpi-forum.org >>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma >> >> _______________________________________________ >> mpi3-rma mailing list >> mpi3-rma(a)lists.mpi-forum.org >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma >> > _______________________________________________ > mpi3-rma mailing list > mpi3-rma(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma > -- Pavan Balaji http://www.mcs.anl.gov/~balaji

1 0

Re: [Mpi3-rma] Target displacement sign issue
by Jim Dinan 05 Sep '12

05 Sep '12

Hi Hubert, If I understand this correctly, then one would no longer be able to cast an MPI_Aint back to a pointer. We would have to add another routine, MPI_Get_pointer_dynamic() to do this conversion. ~Jim. On 9/4/12 6:25 PM, Hubert Ritzdorf wrote: > > Hi, > > Possibly another solution: > > (*) Let MPI_AINT_MAX the maximal value of MPI_Aint variable > (*) Let the window_base of a window created by MPI_WIN_CREATE_DYNAMIC > be MPI_AINT_MAX + 1 > (*) Define a new function MPI_Get_address_dynamic returning the > address of a location relative to MPI_AINT_MAX + 1 > (*) Remove the non-negative qualification in the description > of target_disp in MPI_{Put/Get/Accumulate/...). > > Hubert > >> -----Original Message----- >> From: mpi3-rma-bounces(a)lists.mpi-forum.org [mailto:mpi3-rma- >> bounces(a)lists.mpi-forum.org] On Behalf Of Hubert Ritzdorf >> Sent: Tuesday, September 04, 2012 7:23 PM >> To: MPI 3.0 Remote Memory Access working group >> Subject: Re: [Mpi3-rma] Target displacement sign issue >> >> Hi, >> >> MPI_Get_address returns an MPI_Aint which is signed.(integer). >> >> C: >> MPI_Aint's may be used in Reduce Operations with Min/Max >> or the application can internally use min/max computations. >> Can anybody guarantee that applications don't perform >> min/max operations on these displacements (which results in different >> results for unsigned and signed variables) or use derived values of these >> displacements in other signed/unsigned significant cases. >> >> Fortran: >> MPI_BOTTOM is a variable in a common block/module. Thus, >> MPI_Get_address might really return a signed integer (depending on the >> compiler). >> Fortran doesn't have unsigned integers and cannot handle them. >> Thus, the non-negative integer specification for target_disp in >> MPI_{Put/Get/Accumulate/...) >> might be incorrect for Fortran and MPI_Win_Create_dynamic(). >> >> Hubert >> ________________________________________ >> From: mpi3-rma-bounces(a)lists.mpi-forum.org [mpi3-rma-bounces(a)lists.mpi- >> forum.org] on behalf of Jim Dinan [dinan(a)mcs.anl.gov] >> Sent: Tuesday, September 04, 2012 5:10 PM >> To: mpi3-rma(a)lists.mpi-forum.org >> Subject: Re: [Mpi3-rma] Target displacement sign issue >> >> Hi All, >> >> This is an important issue that we need to resolve before releasing the >> 3.0 spec. Please take a look and send feedback. >> >> Dave has voiced a concern that the implicit type conversion in my >> proposed text below may not be portable. If anyone has more experience >> with this aspect (signed->unsigned conversion) of C/Fortran/etc type >> systems, please chime in. >> >> ~Jim. >> >> On 8/28/12 2:29 PM, Jim Dinan wrote: >>> Hi All, >>> >>> Here is the suggested change to address this issue. In the paragraph on >>> page 425, line 1, add the bracketed sentence: >>> >>> In the case of a window created with MPI_WIN_CREATE_DYNAMIC, the >>> target_disp for all RMA functions is the address at the target; i.e., >>> the effective window_base is MPI_BOTTOM and the disp_unit is one. [Thus, >>> for dynamic windows, the target_disp argument to all RMA functions in >>> interpreted as unsigned.] Users should use MPI_GET_ADDRESS at the target >>> process to determine the address of a target memory location and >>> communicate this address to the origin process. >>> >>> ~Jim. >>> >>> On 8/28/12 9:46 AM, Jim Dinan wrote: >>>> Hi All, >>>> >>>> Currently, target displacements in put/get/acc/etc are defined to be >>>> non-negative integers of type, MPI_Aint. MPI_Aint is defined to be a >>>> signed integer. However, displacements associated with dynamic windows >>>> are addresses (unsigned) and can range into the space of negative >>>> integers. >>>> >>>> It seems like the least impact change would be to add a special case >>>> that the displacement argument will be interpreted as unsigned for >>>> dynamic windows. Changing the interface to take an 'unsigned MPI_Aint' >>>> would be the most clear (and preserve the existing and intended new >>>> semantics), but would break backward compatibility. >>>> >>>> Interested to hear your thoughts. >>>> >>>> ~Jim. >> _______________________________________________ >> mpi3-rma mailing list >> mpi3-rma(a)lists.mpi-forum.org >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma >> >> >> Click >> https://www.mailcontrol.com/sr/O6SiJI5AP4bTndxI!oX7Up2F0f+S6NySj!qlvEoKFyZ >> NTFyVkk9WjRzJbmnfHdyw83VGVWKbSfC8ewgPpp7kgg== to report this email as >> spam. >> >> _______________________________________________ >> mpi3-rma mailing list >> mpi3-rma(a)lists.mpi-forum.org >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma > > _______________________________________________ > mpi3-rma mailing list > mpi3-rma(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma >

1 0