December 2012 - mpiwg-rma - mailman3.cels.anl.gov

Re: [Mpi3-rma] RMA WG discussion
by Jim Dinan 10 Dec '12

10 Dec '12

Ah, ok -- yes, you are correct. Sorry for the misunderstanding on my end. The output buffer must be greater or equal to the size of the input buffer in RMA calls (same semantic as send/recv); thus in accumulate calls, the basic datatype count in the result buffer can be larger than in the input buffer. Coming back to Hubert's original question, I don't think this is adequately covered by the spec. The target buffer parameters should dictate the data that will be placed into the result buffer. And this datatype is allowed to be larger than the origin datatype. I created a ticket for this issue with suggested text to correct it: https://svn.mpi-forum.org/trac/mpi-forum-web/ticket/355 Thanks, ~Jim. On 12/7/12 1:40 PM, Jeff Hammond wrote: > Are we talking about the same thing? I am asserting that it is not > required that the size of the input and output buffers - regardless of > how they are represented with datatypes - do not have to be the same > size, but only need to satisfy the requirement that no buffer overrun > occurs on either end. > > Are you disagreeing with this or are you saying something else? > > Jeff > > On Fri, Dec 7, 2012 at 1:29 PM, Jim Dinan <dinan(a)mcs.anl.gov> wrote: >> This restriction on derived datatypes does not apply to get or put. It only >> applies to accumulate operations, and is stated in the 3.0 spec in the first >> paragraph of page 426 for accumulate, and again in the second paragraph on >> page 428 for get_accumulate: >> >> "Each datatype argument must be a prede fined datatype or a derived >> datatype, where all basic components are of the same predefined datatype. >> Both datatype arguments must be constructed from the same predefined >> datatype." >> >> ~Jim. >> >> >> On 12/7/12 1:13 PM, Jeff Hammond wrote: >>>>>> >>>>>> Which count variable defines the number of basic elements to be >>>>>> returned? >>>>>> (result_count, result_type) or (target_count, target_type) ? >>>>> >>>>> >>>>> result_addr is the buffer that is written at the origin so I believe >>>>> the answer is (result_count, result_type). >>>> >>>> >>>> The number of basic elements needs to be the same across all <count, >>>> dtype> >>>> pairs. The count and datatype for the result buffer determine where >>>> these >>>> elements will be in the origin process' memory. >>> >>> >>> This is not what the MPI standard says for MPI_GET and MPI_ACCUMULATE, >>> as I previously noted, or MPI_PUT, as noted here. If we're going to >>> change the semantics in this respect, it needs to be considered >>> carefully, as it will break backwards compatibility with codes that >>> assume the same behavior as MPI_RECV, as is stipulated in 11.3.1 >>> (MPI_PUT): >>> >>> "The communication must satisfy the same constraints as for a similar >>> message-passing communication. The target_datatype may not specify >>> overlapping entries in the target buffer. The message sent must fit, >>> without truncation, in the target buffer. Furthermore, the target >>> buffer must fit in the target window or in attached memory in a >>> dynamic window." >>> >>> "Fitting without truncation" is weaker than "needs to be the same". >>> >>> Best, >>> >>> Jeff >>> >> _______________________________________________ >> mpi3-rma mailing list >> mpi3-rma(a)lists.mpi-forum.org >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma > > >

1 0

Re: [Mpi3-rma] RMA WG discussion
by Jeff Hammond 07 Dec '12

07 Dec '12

Are we talking about the same thing? I am asserting that it is not required that the size of the input and output buffers - regardless of how they are represented with datatypes - do not have to be the same size, but only need to satisfy the requirement that no buffer overrun occurs on either end. Are you disagreeing with this or are you saying something else? Jeff On Fri, Dec 7, 2012 at 1:29 PM, Jim Dinan <dinan(a)mcs.anl.gov> wrote: > This restriction on derived datatypes does not apply to get or put. It only > applies to accumulate operations, and is stated in the 3.0 spec in the first > paragraph of page 426 for accumulate, and again in the second paragraph on > page 428 for get_accumulate: > > "Each datatype argument must be a prede fined datatype or a derived > datatype, where all basic components are of the same predefined datatype. > Both datatype arguments must be constructed from the same predefined > datatype." > > ~Jim. > > > On 12/7/12 1:13 PM, Jeff Hammond wrote: >>>>> >>>>> Which count variable defines the number of basic elements to be >>>>> returned? >>>>> (result_count, result_type) or (target_count, target_type) ? >>>> >>>> >>>> result_addr is the buffer that is written at the origin so I believe >>>> the answer is (result_count, result_type). >>> >>> >>> The number of basic elements needs to be the same across all <count, >>> dtype> >>> pairs. The count and datatype for the result buffer determine where >>> these >>> elements will be in the origin process' memory. >> >> >> This is not what the MPI standard says for MPI_GET and MPI_ACCUMULATE, >> as I previously noted, or MPI_PUT, as noted here. If we're going to >> change the semantics in this respect, it needs to be considered >> carefully, as it will break backwards compatibility with codes that >> assume the same behavior as MPI_RECV, as is stipulated in 11.3.1 >> (MPI_PUT): >> >> "The communication must satisfy the same constraints as for a similar >> message-passing communication. The target_datatype may not specify >> overlapping entries in the target buffer. The message sent must fit, >> without truncation, in the target buffer. Furthermore, the target >> buffer must fit in the target window or in attached memory in a >> dynamic window." >> >> "Fitting without truncation" is weaker than "needs to be the same". >> >> Best, >> >> Jeff >> > _______________________________________________ > mpi3-rma mailing list > mpi3-rma(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma -- Jeff Hammond Argonne Leadership Computing Facility University of Chicago Computation Institute jhammond(a)alcf.anl.gov / (630) 252-5381 http://www.linkedin.com/in/jeffhammond https://wiki.alcf.anl.gov/parts/index.php/User:Jhammond

1 0

Re: [Mpi3-rma] RMA WG discussion
by Jim Dinan 07 Dec '12

07 Dec '12

This restriction on derived datatypes does not apply to get or put. It only applies to accumulate operations, and is stated in the 3.0 spec in the first paragraph of page 426 for accumulate, and again in the second paragraph on page 428 for get_accumulate: "Each datatype argument must be a predefined datatype or a derived datatype, where all basic components are of the same predefined datatype. Both datatype arguments must be constructed from the same predefined datatype." ~Jim. On 12/7/12 1:13 PM, Jeff Hammond wrote: >>>> Which count variable defines the number of basic elements to be returned? >>>> (result_count, result_type) or (target_count, target_type) ? >>> >>> result_addr is the buffer that is written at the origin so I believe >>> the answer is (result_count, result_type). >> >> The number of basic elements needs to be the same across all <count, dtype> >> pairs. The count and datatype for the result buffer determine where these >> elements will be in the origin process' memory. > > This is not what the MPI standard says for MPI_GET and MPI_ACCUMULATE, > as I previously noted, or MPI_PUT, as noted here. If we're going to > change the semantics in this respect, it needs to be considered > carefully, as it will break backwards compatibility with codes that > assume the same behavior as MPI_RECV, as is stipulated in 11.3.1 > (MPI_PUT): > > "The communication must satisfy the same constraints as for a similar > message-passing communication. The target_datatype may not specify > overlapping entries in the target buffer. The message sent must fit, > without truncation, in the target buffer. Furthermore, the target > buffer must fit in the target window or in attached memory in a > dynamic window." > > "Fitting without truncation" is weaker than "needs to be the same". > > Best, > > Jeff >

1 0

Re: [Mpi3-rma] RMA WG discussion
by Jeff Hammond 07 Dec '12

07 Dec '12

>>> Which count variable defines the number of basic elements to be returned? >>> (result_count, result_type) or (target_count, target_type) ? >> >> result_addr is the buffer that is written at the origin so I believe >> the answer is (result_count, result_type). > > The number of basic elements needs to be the same across all <count, dtype> > pairs. The count and datatype for the result buffer determine where these > elements will be in the origin process' memory. This is not what the MPI standard says for MPI_GET and MPI_ACCUMULATE, as I previously noted, or MPI_PUT, as noted here. If we're going to change the semantics in this respect, it needs to be considered carefully, as it will break backwards compatibility with codes that assume the same behavior as MPI_RECV, as is stipulated in 11.3.1 (MPI_PUT): "The communication must satisfy the same constraints as for a similar message-passing communication. The target_datatype may not specify overlapping entries in the target buffer. The message sent must fit, without truncation, in the target buffer. Furthermore, the target buffer must fit in the target window or in attached memory in a dynamic window." "Fitting without truncation" is weaker than "needs to be the same". Best, Jeff -- Jeff Hammond Argonne Leadership Computing Facility University of Chicago Computation Institute jhammond(a)alcf.anl.gov / (630) 252-5381 http://www.linkedin.com/in/jeffhammond https://wiki.alcf.anl.gov/parts/index.php/User:Jhammond

1 0

Re: [Mpi3-rma] RMA WG discussion
by Jim Dinan 07 Dec '12

07 Dec '12

On 12/7/12 9:56 AM, Jeff Hammond wrote: >> I have a further question to MPI_NO_OP and MPI_{R}Get_accumulate. >> I assume that origin_count is not significant for MPI_NO_OP >> (such as origin_datatype) > > Like origin_addr, origin_count and origin_datatype should be ignored > when MPI_NO_OP is used. I'm not sure that the text needs to be > changed for this though. Pavan said that there is some text that > asserts that buffers (i.e. pointers) be valid (i.e. not NULL) unless > otherwise stipulated. Is there text to the same effect for count and > datatype? If one uses count=0 in MPI_SEND, does the datatype have to > be valid? If not, then we can ignore that argument at least. Since > we don't have precedent for MPI_NO_OP, it seems that the notion that > count can be ignored is implicitly obvious but nonetheless should be > stated (or we could require count=0 for MPI_NO_OP...). This is covered by ticket #350: https://svn.mpi-forum.org/trac/mpi-forum-web/ticket/350 Please take a look and attach any feedback to the ticket. The current proposed text, that would be added to cover all accumulate ops where NO_OP is allowed, states: "When MPI_NO_OP is specified as the operation, the origin buffer arguments are ignored." This includes the pointer, count, and datatype. >> Which count variable defines the number of basic elements to be returned ? >> (result_count, result_type) or (target_count, target_type) ? > > result_addr is the buffer that is written at the origin so I believe > the answer is (result_count, result_type). The number of basic elements needs to be the same across all <count, dtype> pairs. The count and datatype for the result buffer determine where these elements will be in the origin process' memory. >> Should an error code be returned if the other number is too small ? It is erroneous if the number of basic elements does not agree across origin, result, and target buffers. Although, I would expect most MPI implementations to not report this error, since there is a significant cost to detect it. ~Jim.

1 0

Re: [Mpi3-rma] RMA WG discussion
by Jeff Hammond 07 Dec '12

07 Dec '12

> I have a further question to MPI_NO_OP and MPI_{R}Get_accumulate. > I assume that origin_count is not significant for MPI_NO_OP > (such as origin_datatype) Like origin_addr, origin_count and origin_datatype should be ignored when MPI_NO_OP is used. I'm not sure that the text needs to be changed for this though. Pavan said that there is some text that asserts that buffers (i.e. pointers) be valid (i.e. not NULL) unless otherwise stipulated. Is there text to the same effect for count and datatype? If one uses count=0 in MPI_SEND, does the datatype have to be valid? If not, then we can ignore that argument at least. Since we don't have precedent for MPI_NO_OP, it seems that the notion that count can be ignored is implicitly obvious but nonetheless should be stated (or we could require count=0 for MPI_NO_OP...). > Which count variable defines the number of basic elements to be returned ? > (result_count, result_type) or (target_count, target_type) ? result_addr is the buffer that is written at the origin so I believe the answer is (result_count, result_type). > Should an error code be returned if the other number is too small ? MPI-3 11.3.2 (MPI_GET_ ends with "The target buffer must be contained within the target window or within attached memory in a dynamic window, and the copied data must fit, without truncation, in the origin buffer." MPI-3 11.3.4 (MPI_ACCUMULATE) says something similar. I would contend that the same behavior applies to MPI_(R)GET_ACCUMULATE, but it might be appropriate to add text to this effect, just to be clear. We need to say slightly more than for MPI_GET though, because there are three (buffer, count, datatype) that need to be compatible. If we combine the restrictions for GET and ACCUMULATE, result_size<=target_size and origin_size<=target_size - size is derived from (count, datatype) - but I don't know that there has to be any restriction relating result_size and origin_size that isn't included in the former. Best, Jeff -- Jeff Hammond Argonne Leadership Computing Facility University of Chicago Computation Institute jhammond(a)alcf.anl.gov / (630) 252-5381 http://www.linkedin.com/in/jeffhammond https://wiki.alcf.anl.gov/parts/index.php/User:Jhammond

1 0

Re: [Mpi3-rma] RMA WG discussion
by Hubert Ritzdorf 07 Dec '12

07 Dec '12

Hi, I have a further question to MPI_NO_OP and MPI_{R}Get_accumulate. I assume that origin_count is not significant for MPI_NO_OP (such as origin_datatype) Which count variable defines the number of basic elements to be returned ? (result_count, result_type) or (target_count, target_type) ? Should an error code be returned if the other number is too small ? Hubert ________________________________________ From: mpi3-rma-bounces(a)lists.mpi-forum.org [mpi3-rma-bounces(a)lists.mpi-forum.org] on behalf of Pavan Balaji [balaji(a)mcs.anl.gov] Sent: Wednesday, December 05, 2012 11:14 PM To: mpi3-rma(a)lists.mpi-forum.org Subject: [Mpi3-rma] RMA WG discussion We talked about different items that need to be addressed for (1) MPI-3 Errata (should only be bug fixes, not all small changes), and (2) MPI-3.1. We still need to discuss the below items in a telecon; this is just a basic separation we did based on the WG discussion. MPI-3 Errata ============ 1. Ticket #348 -- one of the examples has a bug in it, which should be fixed. 2. Ticket #350 -- we should state that for MPI_ACCUMULATE with MPI_NO_OP, the MPI implementation is not allowed to throw an error when the origin buffer is NULL. MPI-3.1 ======= 1. Ticket #344 -- Shared memory attribute query. 2. Ticket #346 -- Info keys are not consistent across window creation routines. 3. Ticket #347 -- Info keys are not listed in the appendix. 4. Ticket #349 -- Addition of MPI_Disp_to_aint and MPI_Aint_to_disp. 5. Addition of the MPI_MODE_NOLOAD assertion. RMA slides have info on this. 6. Info argument addition for req_mem_model. RMA slides have info on this. -- Pavan -- Pavan Balaji http://www.mcs.anl.gov/~balaji _______________________________________________ mpi3-rma mailing list mpi3-rma(a)lists.mpi-forum.org http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma Click https://www.mailcontrol.com/sr/gHazrfKyo63GX2PQPOmvUkBugPZ!3WFCbdvywdMxQAVv… to report this email as spam.

1 0

Re: [Mpi3-rma] RMA WG discussion
by Jeff Hammond 05 Dec '12

05 Dec '12

An additional work item is that I am going to discuss offline with Hubert and hopefully Tim Mattson about shared-memory windows on non-coherent systems to see if this is viable. If it is, I will provide all the details to the WG for further discussion. I wanted this to be stated for all to see in case others are interested in talking about non-coherent systems in detail. Jeff On Wed, Dec 5, 2012 at 2:14 PM, Pavan Balaji <balaji(a)mcs.anl.gov> wrote: > > We talked about different items that need to be addressed for (1) MPI-3 > Errata (should only be bug fixes, not all small changes), and (2) MPI-3.1. > > We still need to discuss the below items in a telecon; this is just a > basic separation we did based on the WG discussion. > > MPI-3 Errata > ============ > 1. Ticket #348 -- one of the examples has a bug in it, which should be > fixed. > > 2. Ticket #350 -- we should state that for MPI_ACCUMULATE with > MPI_NO_OP, the MPI implementation is not allowed to throw an error when > the origin buffer is NULL. > > MPI-3.1 > ======= > 1. Ticket #344 -- Shared memory attribute query. > > 2. Ticket #346 -- Info keys are not consistent across window creation > routines. > > 3. Ticket #347 -- Info keys are not listed in the appendix. > > 4. Ticket #349 -- Addition of MPI_Disp_to_aint and MPI_Aint_to_disp. > > 5. Addition of the MPI_MODE_NOLOAD assertion. RMA slides have info on this. > > 6. Info argument addition for req_mem_model. RMA slides have info on this. > > -- Pavan > > -- > Pavan Balaji > http://www.mcs.anl.gov/~balaji > _______________________________________________ > mpi3-rma mailing list > mpi3-rma(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma -- Jeff Hammond Argonne Leadership Computing Facility University of Chicago Computation Institute jhammond(a)alcf.anl.gov / (630) 252-5381 http://www.linkedin.com/in/jeffhammond https://wiki.alcf.anl.gov/parts/index.php/User:Jhammond

1 0

Re: [Mpi3-rma] [EXTERNAL] RMA WG discussion
by Barrett, Brian W 05 Dec '12

05 Dec '12

Also, this morning we had a fairly good meeting with the FT group. The outcome of the discussion was that it would seem like a good path forward would be for the RMA working group to define what we can and can not provide in terms of failure reporting and state stability after a fail-stop failure of a process. I offered to write up a starting document, which I hope to have circulated inside the RMA WG by January, so that we can make progress at the March meeting. Brian On 12/5/12 2:14 PM, "Pavan Balaji" <balaji(a)mcs.anl.gov> wrote: > >We talked about different items that need to be addressed for (1) MPI-3 >Errata (should only be bug fixes, not all small changes), and (2) MPI-3.1. > >We still need to discuss the below items in a telecon; this is just a >basic separation we did based on the WG discussion. > >MPI-3 Errata >============ >1. Ticket #348 -- one of the examples has a bug in it, which should be >fixed. > >2. Ticket #350 -- we should state that for MPI_ACCUMULATE with >MPI_NO_OP, the MPI implementation is not allowed to throw an error when >the origin buffer is NULL. > >MPI-3.1 >======= >1. Ticket #344 -- Shared memory attribute query. > >2. Ticket #346 -- Info keys are not consistent across window creation >routines. > >3. Ticket #347 -- Info keys are not listed in the appendix. > >4. Ticket #349 -- Addition of MPI_Disp_to_aint and MPI_Aint_to_disp. > >5. Addition of the MPI_MODE_NOLOAD assertion. RMA slides have info on >this. > >6. Info argument addition for req_mem_model. RMA slides have info on >this. > > -- Pavan > >-- >Pavan Balaji >http://www.mcs.anl.gov/~balaji >_______________________________________________ >mpi3-rma mailing list >mpi3-rma(a)lists.mpi-forum.org >http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma > > -- Brian W. Barrett Scalable System Software Group Sandia National Laboratories

1 0

[Mpi3-rma] RMA WG discussion
by Pavan Balaji 05 Dec '12

05 Dec '12

We talked about different items that need to be addressed for (1) MPI-3 Errata (should only be bug fixes, not all small changes), and (2) MPI-3.1. We still need to discuss the below items in a telecon; this is just a basic separation we did based on the WG discussion. MPI-3 Errata ============ 1. Ticket #348 -- one of the examples has a bug in it, which should be fixed. 2. Ticket #350 -- we should state that for MPI_ACCUMULATE with MPI_NO_OP, the MPI implementation is not allowed to throw an error when the origin buffer is NULL. MPI-3.1 ======= 1. Ticket #344 -- Shared memory attribute query. 2. Ticket #346 -- Info keys are not consistent across window creation routines. 3. Ticket #347 -- Info keys are not listed in the appendix. 4. Ticket #349 -- Addition of MPI_Disp_to_aint and MPI_Aint_to_disp. 5. Addition of the MPI_MODE_NOLOAD assertion. RMA slides have info on this. 6. Info argument addition for req_mem_model. RMA slides have info on this. -- Pavan -- Pavan Balaji http://www.mcs.anl.gov/~balaji

1 0