November 2012 - mpi-forum - mailman3.cels.anl.gov

Re: [Mpi-forum] [EXTERNAL] Wording in MPI standard
by Brightwell, Ronald 27 Nov '12

27 Nov '12

On Nov 26, 2012, at 2:37 PM, Scott Pakin wrote: > Ron, > > On 11/26/2012 12:18 PM, Brightwell, Ronald wrote: >> The issue is not whether the local non-blocking operation will complete, but whether starting a local non-blocking operation is enough to satisfy the matching non-local operation. It is essentially saying that once an operation has been started and matched, completion is local. That is, completion of a send operation that has been started and satisfied by a matching receive operation (that has also been started) is not dependent on whether that receive operation has been completed. The send side doesn't have to wait for the receive side to call MPI_WAIT in order for the send to complete. > > Yes, I just want the standard to clarify that what you wrote above is > guaranteed MPI behavior, not simply desirable behavior. I think it is guaranteed behavior. > > Let me provide a little more detail on what I'm dealing with. I > believe, but have not yet managed to prove, that a large LANL > application is doing something like the following: > > Rank 0 Rank 1 > ------ ------ > ISSEND 1 IRECV 0 > WAIT ALLREDUCE > ALLREDUCE WAIT > > If MPI guarantees that "the send side doesn't have to wait for the > receive side to call MPI_WAIT in order for the send to complete" > (i.e., the non-local matching you refer to), then the preceding > pseudocode represents a correct use of MPI. If, however, it would > merely be "nice" for the send not to have to wait for the receive-side > MPI_WAIT, then the preceding pseudocode represents an incorrect use of > MPI that happens to work with all tested MPI implementations but may > deadlock under some future implementation. I think this should always work. > >> I'm not sure there's any semantic difference between "... the receive should complete..." versus "... the receive must complete...", but the latter does seem stronger. > > Some standards bodies explicitly distinguish "must" as meaning, "or > else you don't have a compliant implementation" and "should" as > meaning, "or else you have a technically compliant but probably poorly > performing implementation." The key is whether users can rely on the > stated behavior for correct application execution, as in my pseudocode > example above. I don't think the Standard is usually that subtle about quality of implementation issues, so it seems to me like it should be "must". -Ron

1 0

Dec meeting registraiton
by Richard Graham 26 Nov '12

26 Nov '12

A request to those stragglers who have not yet registered and plan to attend (I am in this crowd, and am aware of others) - please register ASAP. Jeff is arranging for network logins for attendees, so would be nice for not to have to do this at the last moment. Thanks, Rich

1 0

Re: [Mpi-forum] [EXTERNAL] Wording in MPI standard
by Scott Pakin 26 Nov '12

26 Nov '12

Ron, On 11/26/2012 12:18 PM, Brightwell, Ronald wrote: > The issue is not whether the local non-blocking operation will complete, but whether starting a local non-blocking operation is enough to satisfy the matching non-local operation. It is essentially saying that once an operation has been started and matched, completion is local. That is, completion of a send operation that has been started and satisfied by a matching receive operation (that has also been started) is not dependent on whether that receive operation has been completed. The send side doesn't have to wait for the receive side to call MPI_WAIT in order for the send to complete. Yes, I just want the standard to clarify that what you wrote above is guaranteed MPI behavior, not simply desirable behavior. Let me provide a little more detail on what I'm dealing with. I believe, but have not yet managed to prove, that a large LANL application is doing something like the following: Rank 0 Rank 1 ------ ------ ISSEND 1 IRECV 0 WAIT ALLREDUCE ALLREDUCE WAIT If MPI guarantees that "the send side doesn't have to wait for the receive side to call MPI_WAIT in order for the send to complete" (i.e., the non-local matching you refer to), then the preceding pseudocode represents a correct use of MPI. If, however, it would merely be "nice" for the send not to have to wait for the receive-side MPI_WAIT, then the preceding pseudocode represents an incorrect use of MPI that happens to work with all tested MPI implementations but may deadlock under some future implementation. > I'm not sure there's any semantic difference between "... the receive should complete..." versus "... the receive must complete...", but the latter does seem stronger. Some standards bodies explicitly distinguish "must" as meaning, "or else you don't have a compliant implementation" and "should" as meaning, "or else you have a technically compliant but probably poorly performing implementation." The key is whether users can rely on the stated behavior for correct application execution, as in my pseudocode example above. -- Scott

1 0

Re: [Mpi-forum] Wording in MPI standard
by Bronis R. de Supinski 26 Nov '12

26 Nov '12

Scott: My apologies. I misread your message. I thought you were referring to the last sentence in the paragraph, which says "and even if no call is executed to complete the receive". I agree with your suggestion that "should" should be changed to "must" in "the receive should complete even if no call is executed by the sender to complete the send" ("shall" would also work). Bronis On Mon, 26 Nov 2012, Scott Pakin wrote: > Bronis, > > On 11/26/2012 11:44 AM, Bronis R. de Supinski wrote: >> The problem with your suggestion is that completion >> has a well defined meaning in MPI that requires the >> completion call. If the receive completed without >> any call then a subsequent attempt to complete it >> would be an error. I believe that is reason the term >> "satisfied" was used. > > The current text says that "the receive *should* complete even if no > call is executed by the sender to complete the send." If a subsequent > attempt to complete such a send would be an error, then the standard > would have to say, "the receive *must not* complete even if no call is > executed by the sender to complete the send," which is in disagreement > with what a typical (all?) MPI implementation does. > > I'm merely taking issue with the use of "should," which seems to let > an MPI implementation decide whether to complete the receive but not > allow applications to rely on this happening. "Must" or "is required > to" would let applications reason better about progress. > > -- Scott >

1 0

Re: [Mpi-forum] [EXTERNAL] Wording in MPI standard
by Brightwell, Ronald 26 Nov '12

26 Nov '12

The issue is not whether the local non-blocking operation will complete, but whether starting a local non-blocking operation is enough to satisfy the matching non-local operation. It is essentially saying that once an operation has been started and matched, completion is local. That is, completion of a send operation that has been started and satisfied by a matching receive operation (that has also been started) is not dependent on whether that receive operation has been completed. The send side doesn't have to wait for the receive side to call MPI_WAIT in order for the send to complete. I'm not sure there's any semantic difference between "... the receive should complete..." versus "... the receive must complete...", but the latter does seem stronger. -Ron On Nov 26, 2012, at 11:44 AM, Bronis R. de Supinski wrote: > > Scott: > > The problem with your suggestion is that completion > has a well defined meaning in MPI that requires the > completion call. If the receive completed without > any call then a subsequent attempt to complete it > would be an error. I believe that is reason the term > "satisfied" was used. > > Bronis > > > > On Mon, 26 Nov 2012, Pavan Balaji wrote: > >> Scott, >> >> I'm not sure. I've cc'ed the list on this email. >> >> -- Pavan >> >> On 11/26/2012 10:42 AM US Central Time, Scott Pakin wrote: >>> Pavan, >>> >>> On 11/19/2012 11:59 PM, Pavan Balaji wrote: >>>>> Thanks. I didn't see the email come through, though. >>> Do you know if email from non-subscribers gets silently dropped? >>> >>> I attached the message I tried posting. Want to try resending it >>> yourself? If not, I suppose I can subscribe, resend, then probably >>> unsubscribe. >>> >>> Thanks, >>> -- Scott >>> >>> >>> Attached Message >>> >>> Subject: >>> Unclear wording in MPI spec >>> From: >>> Scott Pakin <pakin(a)lanl.gov> >>> Date: >>> 11/19/2012 11:14 AM >>> >>> To: >>> <mpi-forum(a)lists.mpi-forum.org> >>> >>> >>> From a discussion I had with Pavan Balaji at SC'12, I believe a minor >>> wording change is needed in the MPI 3.0 standard, page 56, lines >>> 30-35: >>> >>> Progress: A call to MPI_WAIT that completes a receive will >>> eventually terminate and return if a matching send has been >>> started, unless the send is satisfied by another receive. In >>> particular, if the matching send is nonblocking, then the receive >>> should complete even if no call is executed by the sender to >>> complete the send. Similarly, a call to MPI_WAIT that completes a >>> send will eventually return if a matching receive has been >>> started, unless the receive is satisfied by another send, and even >>> if no call is executed to complete the receive. >>> >>> Perhaps that should say, "...the receive *is required* to complete >>> even if no call is executed...". >>> >>> -- Scott >> >> -- >> Pavan Balaji >> http://www.mcs.anl.gov/~balaji >> _______________________________________________ >> mpi-forum mailing list >> mpi-forum(a)lists.mpi-forum.org >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum >> > _______________________________________________ > mpi-forum mailing list > mpi-forum(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum >

1 0

Re: [Mpi-forum] Wording in MPI standard
by Scott Pakin 26 Nov '12

26 Nov '12

Bronis, On 11/26/2012 11:44 AM, Bronis R. de Supinski wrote: > The problem with your suggestion is that completion > has a well defined meaning in MPI that requires the > completion call. If the receive completed without > any call then a subsequent attempt to complete it > would be an error. I believe that is reason the term > "satisfied" was used. The current text says that "the receive *should* complete even if no call is executed by the sender to complete the send." If a subsequent attempt to complete such a send would be an error, then the standard would have to say, "the receive *must not* complete even if no call is executed by the sender to complete the send," which is in disagreement with what a typical (all?) MPI implementation does. I'm merely taking issue with the use of "should," which seems to let an MPI implementation decide whether to complete the receive but not allow applications to rely on this happening. "Must" or "is required to" would let applications reason better about progress. -- Scott

1 0

Re: [Mpi-forum] Wording in MPI standard
by Bronis R. de Supinski 26 Nov '12

26 Nov '12

Scott: The problem with your suggestion is that completion has a well defined meaning in MPI that requires the completion call. If the receive completed without any call then a subsequent attempt to complete it would be an error. I believe that is reason the term "satisfied" was used. Bronis On Mon, 26 Nov 2012, Pavan Balaji wrote: > Scott, > > I'm not sure. I've cc'ed the list on this email. > > -- Pavan > > On 11/26/2012 10:42 AM US Central Time, Scott Pakin wrote: >> Pavan, >> >> On 11/19/2012 11:59 PM, Pavan Balaji wrote: >>>> Thanks. I didn't see the email come through, though. >> Do you know if email from non-subscribers gets silently dropped? >> >> I attached the message I tried posting. Want to try resending it >> yourself? If not, I suppose I can subscribe, resend, then probably >> unsubscribe. >> >> Thanks, >> -- Scott >> >> >> Attached Message >> >> Subject: >> Unclear wording in MPI spec >> From: >> Scott Pakin <pakin(a)lanl.gov> >> Date: >> 11/19/2012 11:14 AM >> >> To: >> <mpi-forum(a)lists.mpi-forum.org> >> >> >> From a discussion I had with Pavan Balaji at SC'12, I believe a minor >> wording change is needed in the MPI 3.0 standard, page 56, lines >> 30-35: >> >> Progress: A call to MPI_WAIT that completes a receive will >> eventually terminate and return if a matching send has been >> started, unless the send is satisfied by another receive. In >> particular, if the matching send is nonblocking, then the receive >> should complete even if no call is executed by the sender to >> complete the send. Similarly, a call to MPI_WAIT that completes a >> send will eventually return if a matching receive has been >> started, unless the receive is satisfied by another send, and even >> if no call is executed to complete the receive. >> >> Perhaps that should say, "...the receive *is required* to complete >> even if no call is executed...". >> >> -- Scott > > -- > Pavan Balaji > http://www.mcs.anl.gov/~balaji > _______________________________________________ > mpi-forum mailing list > mpi-forum(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum >

1 0

Re: [Mpi-forum] Wording in MPI standard
by Jeff Hammond 26 Nov '12

26 Nov '12

Page ii of the MPI-3 standard says that messages from unsubscribed addresses will be ignored. I guess this technically applied just to mpi-comments@ but I assume the policy is general to @mpi-forum.org. Jeff Sent from my iPhone On Nov 26, 2012, at 12:09 PM, Pavan Balaji <balaji(a)mcs.anl.gov> wrote: > Scott, > > I'm not sure. I've cc'ed the list on this email. > > -- Pavan > > On 11/26/2012 10:42 AM US Central Time, Scott Pakin wrote: >> Pavan, >> >> On 11/19/2012 11:59 PM, Pavan Balaji wrote: >>>> Thanks. I didn't see the email come through, though. >> Do you know if email from non-subscribers gets silently dropped? >> >> I attached the message I tried posting. Want to try resending it >> yourself? If not, I suppose I can subscribe, resend, then probably >> unsubscribe. >> >> Thanks, >> -- Scott >> >> >> Attached Message >> >> Subject: >> Unclear wording in MPI spec >> From: >> Scott Pakin <pakin(a)lanl.gov> >> Date: >> 11/19/2012 11:14 AM >> >> To: >> <mpi-forum(a)lists.mpi-forum.org> >> >> >> From a discussion I had with Pavan Balaji at SC'12, I believe a minor >> wording change is needed in the MPI 3.0 standard, page 56, lines >> 30-35: >> >> Progress: A call to MPI_WAIT that completes a receive will >> eventually terminate and return if a matching send has been >> started, unless the send is satisfied by another receive. In >> particular, if the matching send is nonblocking, then the receive >> should complete even if no call is executed by the sender to >> complete the send. Similarly, a call to MPI_WAIT that completes a >> send will eventually return if a matching receive has been >> started, unless the receive is satisfied by another send, and even >> if no call is executed to complete the receive. >> >> Perhaps that should say, "...the receive *is required* to complete >> even if no call is executed...". >> >> -- Scott > > -- > Pavan Balaji > http://www.mcs.anl.gov/~balaji > _______________________________________________ > mpi-forum mailing list > mpi-forum(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum

1 0

Mailing list behavior
by Jeff Squyres 26 Nov '12

26 Nov '12

Per your question on mailing list behavior: the list will either silently drop or explicitly send you a reject message. I'm not sure which, offhand. (I actually just tried it and it seemed to silently drop it) Bottom line: you must be subscribed to post. On Nov 26, 2012, at 12:09 PM, Pavan Balaji wrote: > Scott, > > I'm not sure. I've cc'ed the list on this email. > > -- Pavan > > On 11/26/2012 10:42 AM US Central Time, Scott Pakin wrote: >> Pavan, >> >> On 11/19/2012 11:59 PM, Pavan Balaji wrote: >>>> Thanks. I didn't see the email come through, though. >> Do you know if email from non-subscribers gets silently dropped? >> >> I attached the message I tried posting. Want to try resending it >> yourself? If not, I suppose I can subscribe, resend, then probably >> unsubscribe. >> >> Thanks, >> -- Scott -- Jeff Squyres jsquyres(a)cisco.com For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/

1 0

Re: [Mpi-forum] Wording in MPI standard
by Pavan Balaji 26 Nov '12

26 Nov '12

Scott, I'm not sure. I've cc'ed the list on this email. -- Pavan On 11/26/2012 10:42 AM US Central Time, Scott Pakin wrote: > Pavan, > > On 11/19/2012 11:59 PM, Pavan Balaji wrote: >> > Thanks. I didn't see the email come through, though. > Do you know if email from non-subscribers gets silently dropped? > > I attached the message I tried posting. Want to try resending it > yourself? If not, I suppose I can subscribe, resend, then probably > unsubscribe. > > Thanks, > -- Scott > > > Attached Message > > Subject: > Unclear wording in MPI spec > From: > Scott Pakin <pakin(a)lanl.gov> > Date: > 11/19/2012 11:14 AM > > To: > <mpi-forum(a)lists.mpi-forum.org> > > > From a discussion I had with Pavan Balaji at SC'12, I believe a minor > wording change is needed in the MPI 3.0 standard, page 56, lines > 30-35: > > Progress: A call to MPI_WAIT that completes a receive will > eventually terminate and return if a matching send has been > started, unless the send is satisfied by another receive. In > particular, if the matching send is nonblocking, then the receive > should complete even if no call is executed by the sender to > complete the send. Similarly, a call to MPI_WAIT that completes a > send will eventually return if a matching receive has been > started, unless the receive is satisfied by another send, and even > if no call is executed to complete the receive. > > Perhaps that should say, "...the receive *is required* to complete > even if no call is executed...". > > -- Scott -- Pavan Balaji http://www.mcs.anl.gov/~balaji

1 0