June 2015 - mpiwg-p2p - mailman3.cels.anl.gov

Next meeting: 8th June 2015 11am Central US
by Daniel Holmes 08 Jun '15

08 Jun '15

Hi all, The next point-to-point teleconference will be today on Monday 8th June 2015 at 11am Central US via webex. Connection details are on the point-to-point wiki page: https://svn.mpi-forum.org/trac/mpi-forum-web/wiki/PtpWikiPage Agenda: 1) Ticket #476: "Can INFO keys change the semantic behaviour of MPI?" issue - the communicator assertions ticket was presented at the June Forum meeting and we got some useful feedback that we should discuss. 2) Tickets #478, #479, #480: "everything you ever wanted to know about MPI send cancel, plus a whole lot more" [new!] 3) New communicator INFO keys - it looks like these will be allowed, so we should define a few and get a proposal ready for the next face-to-face meeting 4) Progress on Arecv and Fsend or ReceiveReduce? 5) Other stuff? Cheers, Dan. -------- Forwarded Message -------- Subject: MPI Pt2pt WG tickets Date: Sun, 7 Jun 2015 21:04:50 +0000 From: Balaji, Pavan <balaji(a)anl.gov> To: Daniel Holmes <dholmes(a)epcc.ed.ac.uk> Hi Dan, I'd like to bring the following tickets forward to be vetted by the pt2pt WG before taking it to the Forum. Can you schedule some time for these in the WG telecon? When is the next telecon? https://svn.mpi-forum.org/trac/mpi-forum-web/ticket/478 https://svn.mpi-forum.org/trac/mpi-forum-web/ticket/479 https://svn.mpi-forum.org/trac/mpi-forum-web/ticket/480 Thanks, -- Pavan -- Pavan Balaji http://www.mcs.anl.gov/~balaji -- Dan Holmes Applications Consultant in HPC Research EPCC, The University of Edinburgh James Clerk Maxwell Building The Kings Buildings Peter Guthrie Tait Road Edinburgh EH9 3FD T: +44(0)131 651 3465 E: dholmes(a)epcc.ed.ac.uk *Please consider the environment before printing this email.* The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336.

1 0

Re: [mpiwg-p2p] cancel in standard
by Sur, Sayantan 03 Jun '15

03 Jun '15

Agreed. On 6/3/15, 8:18 PM, "Balaji, Pavan" <balaji(a)anl.gov> wrote: > >Yes, I spoke loosely, but what I said doesn't change with that >distinction. "Matched" is a precursor to "received". > >"If the user never posts a receive, the send cancel should always >succeed." > >However, if you interpret "received at the destination" to include >unexpected queues, then one should not be able to cancel the request. > >My interpretation is the former one. But for either interpretation, the >standard must be clarified. > > -- Pavan > > >On 6/3/15, 8:13 PM, "Sur, Sayantan" <sayantan.sur(a)intel.com> wrote: > >>The standard distinguishes between “received” and “matched” very >>carefully. For example, MPI_Ssend() can return when the message is >>matched >>but not necessarily fully received. >> >>"A send that uses the synchronous mode can be started whether or not a >>matching >>receive was posted. However, the send will complete successfully only if >>a >>matching receive is >>posted, and the receive operation has started to receive the message sent >>by the synchronous >>send." >> >>Therefore, if the standard says “received”, I do not want to interpret >>that as “matched”. >> >>On 6/3/15, 8:03 PM, "Balaji, Pavan" <balaji(a)anl.gov> wrote: >> >>> >>>If you interpret received at the destination as "matched", then what I >>>mentioned below holds -- if the user never posts a receive, the send >>>cancel should always succeed. >>> >>> -- Pavan >>> >>> >>> >>> >>>On 6/3/15, 7:59 PM, "Sur, Sayantan" <sayantan.sur(a)intel.com> wrote: >>> >>>>+MPI P2P WG. >>>> >>>>We should probably discuss it in the P2P WG as well. >>>> >>>>I read the snippet of text and thought the “send” mentioned should >>>>correspond to the MPI send operation (the one the user issued), and not >>>>any internal buffering. “send completes normally” should mean the same >>>>thing as sends completing as if cancel wasn’t called. Similarly, >>>>“received >>>>at the destination” should mean the normal way messages are received at >>>>the destination i.e. by posting an MPI receive. Of course, these are my >>>>interpretations :) >>>> >>>>On 6/3/15, 7:19 PM, "Balaji, Pavan" <balaji(a)anl.gov> wrote: >>>> >>>>> >>>>>[Cc'ed the Forum] >>>>> >>>>>This wording has been debated many times at the Forum. Specifically, >>>>>the >>>>>"completes normally" and "received at the destination" pieces are >>>>>ambiguous. For instance, if the data is copied into a bounce buffer, >>>>>the >>>>>send would complete normally, but that does not mean that there is a >>>>>receive posted for that data. In this case, it would be the user's >>>>>responsibility to post a corresponding receive. If the MPI >>>>>implementation does not have a bounce buffer (e.g., for large >>>>>messages), >>>>>"received at the destination" could mean into an unexpected buffer, >>>>>not >>>>>necessarily a user buffer. >>>>> >>>>>The consensus from MPI-1 veterans (Marc, Bill, etc.) during the last >>>>>discussion was that if the application does not post a receive, then >>>>>it >>>>>is guaranteed that the send should be cancelable. At least that was >>>>>the >>>>>intent. The wording, of course, is too vague for such a definition. >>>>> >>>>>You could try to weaken that interpretation and say that if the MPI >>>>>implementation is not able to cancel a send operation, then the user >>>>>is >>>>>responsible for posting a matching receive to receive that data. >>>>>Irrespective of which interpretation we go with, I think it is >>>>>important >>>>>to clarify it in the standard. Perhaps an MPI-3.1 errata or MPI-3.2 >>>>>change instead of waiting for MPI-4? Do you want to create a ticket? >>>>> >>>>> -- Pavan >>>>> >>>>> >>>>> >>>>> >>>>>On 6/3/15, 7:05 PM, "Sur, Sayantan" <sayantan.sur(a)intel.com> wrote: >>>>> >>>>>>What lines in the standard support the interpretation that “a message >>>>>>that >>>>>>will not be successfully matched, must be cancelable”? >>>>>> >>>>>>This is what I found, and it doesn’t support the stricter >>>>>>interpretation. >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>>"If a >>>>>>send is marked for cancellation, then it must be the case that either >>>>>>the >>>>>>send completes >>>>>>normally, in which case the message sent was received at the >>>>>>destination >>>>>>process, or that >>>>>>the send is successfully cancelled, in which case no part of the >>>>>>message >>>>>>was received at the >>>>>>destination. Then, any matching receive has to be satisfied by >>>>>>another >>>>>>send." >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>> >>

1 0

Re: [mpiwg-p2p] cancel in standard
by Balaji, Pavan 03 Jun '15

03 Jun '15

Yes, I spoke loosely, but what I said doesn't change with that distinction. "Matched" is a precursor to "received". "If the user never posts a receive, the send cancel should always succeed." However, if you interpret "received at the destination" to include unexpected queues, then one should not be able to cancel the request. My interpretation is the former one. But for either interpretation, the standard must be clarified. -- Pavan On 6/3/15, 8:13 PM, "Sur, Sayantan" <sayantan.sur(a)intel.com> wrote: >The standard distinguishes between “received” and “matched” very >carefully. For example, MPI_Ssend() can return when the message is matched >but not necessarily fully received. > >"A send that uses the synchronous mode can be started whether or not a >matching >receive was posted. However, the send will complete successfully only if a >matching receive is >posted, and the receive operation has started to receive the message sent >by the synchronous >send." > >Therefore, if the standard says “received”, I do not want to interpret >that as “matched”. > >On 6/3/15, 8:03 PM, "Balaji, Pavan" <balaji(a)anl.gov> wrote: > >> >>If you interpret received at the destination as "matched", then what I >>mentioned below holds -- if the user never posts a receive, the send >>cancel should always succeed. >> >> -- Pavan >> >> >> >> >>On 6/3/15, 7:59 PM, "Sur, Sayantan" <sayantan.sur(a)intel.com> wrote: >> >>>+MPI P2P WG. >>> >>>We should probably discuss it in the P2P WG as well. >>> >>>I read the snippet of text and thought the “send” mentioned should >>>correspond to the MPI send operation (the one the user issued), and not >>>any internal buffering. “send completes normally” should mean the same >>>thing as sends completing as if cancel wasn’t called. Similarly, >>>“received >>>at the destination” should mean the normal way messages are received at >>>the destination i.e. by posting an MPI receive. Of course, these are my >>>interpretations :) >>> >>>On 6/3/15, 7:19 PM, "Balaji, Pavan" <balaji(a)anl.gov> wrote: >>> >>>> >>>>[Cc'ed the Forum] >>>> >>>>This wording has been debated many times at the Forum. Specifically, >>>>the >>>>"completes normally" and "received at the destination" pieces are >>>>ambiguous. For instance, if the data is copied into a bounce buffer, >>>>the >>>>send would complete normally, but that does not mean that there is a >>>>receive posted for that data. In this case, it would be the user's >>>>responsibility to post a corresponding receive. If the MPI >>>>implementation does not have a bounce buffer (e.g., for large messages), >>>>"received at the destination" could mean into an unexpected buffer, not >>>>necessarily a user buffer. >>>> >>>>The consensus from MPI-1 veterans (Marc, Bill, etc.) during the last >>>>discussion was that if the application does not post a receive, then it >>>>is guaranteed that the send should be cancelable. At least that was the >>>>intent. The wording, of course, is too vague for such a definition. >>>> >>>>You could try to weaken that interpretation and say that if the MPI >>>>implementation is not able to cancel a send operation, then the user is >>>>responsible for posting a matching receive to receive that data. >>>>Irrespective of which interpretation we go with, I think it is important >>>>to clarify it in the standard. Perhaps an MPI-3.1 errata or MPI-3.2 >>>>change instead of waiting for MPI-4? Do you want to create a ticket? >>>> >>>> -- Pavan >>>> >>>> >>>> >>>> >>>>On 6/3/15, 7:05 PM, "Sur, Sayantan" <sayantan.sur(a)intel.com> wrote: >>>> >>>>>What lines in the standard support the interpretation that “a message >>>>>that >>>>>will not be successfully matched, must be cancelable”? >>>>> >>>>>This is what I found, and it doesn’t support the stricter >>>>>interpretation. >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>>"If a >>>>>send is marked for cancellation, then it must be the case that either >>>>>the >>>>>send completes >>>>>normally, in which case the message sent was received at the >>>>>destination >>>>>process, or that >>>>>the send is successfully cancelled, in which case no part of the >>>>>message >>>>>was received at the >>>>>destination. Then, any matching receive has to be satisfied by another >>>>>send." >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>> >

1 0

Re: [mpiwg-p2p] cancel in standard
by Sur, Sayantan 03 Jun '15

03 Jun '15

The standard distinguishes between “received” and “matched” very carefully. For example, MPI_Ssend() can return when the message is matched but not necessarily fully received. "A send that uses the synchronous mode can be started whether or not a matching receive was posted. However, the send will complete successfully only if a matching receive is posted, and the receive operation has started to receive the message sent by the synchronous send." Therefore, if the standard says “received”, I do not want to interpret that as “matched”. On 6/3/15, 8:03 PM, "Balaji, Pavan" <balaji(a)anl.gov> wrote: > >If you interpret received at the destination as "matched", then what I >mentioned below holds -- if the user never posts a receive, the send >cancel should always succeed. > > -- Pavan > > > > >On 6/3/15, 7:59 PM, "Sur, Sayantan" <sayantan.sur(a)intel.com> wrote: > >>+MPI P2P WG. >> >>We should probably discuss it in the P2P WG as well. >> >>I read the snippet of text and thought the “send” mentioned should >>correspond to the MPI send operation (the one the user issued), and not >>any internal buffering. “send completes normally” should mean the same >>thing as sends completing as if cancel wasn’t called. Similarly, >>“received >>at the destination” should mean the normal way messages are received at >>the destination i.e. by posting an MPI receive. Of course, these are my >>interpretations :) >> >>On 6/3/15, 7:19 PM, "Balaji, Pavan" <balaji(a)anl.gov> wrote: >> >>> >>>[Cc'ed the Forum] >>> >>>This wording has been debated many times at the Forum. Specifically, >>>the >>>"completes normally" and "received at the destination" pieces are >>>ambiguous. For instance, if the data is copied into a bounce buffer, >>>the >>>send would complete normally, but that does not mean that there is a >>>receive posted for that data. In this case, it would be the user's >>>responsibility to post a corresponding receive. If the MPI >>>implementation does not have a bounce buffer (e.g., for large messages), >>>"received at the destination" could mean into an unexpected buffer, not >>>necessarily a user buffer. >>> >>>The consensus from MPI-1 veterans (Marc, Bill, etc.) during the last >>>discussion was that if the application does not post a receive, then it >>>is guaranteed that the send should be cancelable. At least that was the >>>intent. The wording, of course, is too vague for such a definition. >>> >>>You could try to weaken that interpretation and say that if the MPI >>>implementation is not able to cancel a send operation, then the user is >>>responsible for posting a matching receive to receive that data. >>>Irrespective of which interpretation we go with, I think it is important >>>to clarify it in the standard. Perhaps an MPI-3.1 errata or MPI-3.2 >>>change instead of waiting for MPI-4? Do you want to create a ticket? >>> >>> -- Pavan >>> >>> >>> >>> >>>On 6/3/15, 7:05 PM, "Sur, Sayantan" <sayantan.sur(a)intel.com> wrote: >>> >>>>What lines in the standard support the interpretation that “a message >>>>that >>>>will not be successfully matched, must be cancelable”? >>>> >>>>This is what I found, and it doesn’t support the stricter >>>>interpretation. >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>>"If a >>>>send is marked for cancellation, then it must be the case that either >>>>the >>>>send completes >>>>normally, in which case the message sent was received at the >>>>destination >>>>process, or that >>>>the send is successfully cancelled, in which case no part of the >>>>message >>>>was received at the >>>>destination. Then, any matching receive has to be satisfied by another >>>>send." >>>> >>>> >>>> >>>> >>>> >>>> >>

1 0

Re: [mpiwg-p2p] cancel in standard
by Balaji, Pavan 03 Jun '15

03 Jun '15

If you interpret received at the destination as "matched", then what I mentioned below holds -- if the user never posts a receive, the send cancel should always succeed. -- Pavan On 6/3/15, 7:59 PM, "Sur, Sayantan" <sayantan.sur(a)intel.com> wrote: >+MPI P2P WG. > >We should probably discuss it in the P2P WG as well. > >I read the snippet of text and thought the “send” mentioned should >correspond to the MPI send operation (the one the user issued), and not >any internal buffering. “send completes normally” should mean the same >thing as sends completing as if cancel wasn’t called. Similarly, “received >at the destination” should mean the normal way messages are received at >the destination i.e. by posting an MPI receive. Of course, these are my >interpretations :) > >On 6/3/15, 7:19 PM, "Balaji, Pavan" <balaji(a)anl.gov> wrote: > >> >>[Cc'ed the Forum] >> >>This wording has been debated many times at the Forum. Specifically, the >>"completes normally" and "received at the destination" pieces are >>ambiguous. For instance, if the data is copied into a bounce buffer, the >>send would complete normally, but that does not mean that there is a >>receive posted for that data. In this case, it would be the user's >>responsibility to post a corresponding receive. If the MPI >>implementation does not have a bounce buffer (e.g., for large messages), >>"received at the destination" could mean into an unexpected buffer, not >>necessarily a user buffer. >> >>The consensus from MPI-1 veterans (Marc, Bill, etc.) during the last >>discussion was that if the application does not post a receive, then it >>is guaranteed that the send should be cancelable. At least that was the >>intent. The wording, of course, is too vague for such a definition. >> >>You could try to weaken that interpretation and say that if the MPI >>implementation is not able to cancel a send operation, then the user is >>responsible for posting a matching receive to receive that data. >>Irrespective of which interpretation we go with, I think it is important >>to clarify it in the standard. Perhaps an MPI-3.1 errata or MPI-3.2 >>change instead of waiting for MPI-4? Do you want to create a ticket? >> >> -- Pavan >> >> >> >> >>On 6/3/15, 7:05 PM, "Sur, Sayantan" <sayantan.sur(a)intel.com> wrote: >> >>>What lines in the standard support the interpretation that “a message >>>that >>>will not be successfully matched, must be cancelable”? >>> >>>This is what I found, and it doesn’t support the stricter interpretation. >>> >>> >>> >>> >>> >>> >>> >>> >>> >>>"If a >>>send is marked for cancellation, then it must be the case that either the >>>send completes >>>normally, in which case the message sent was received at the destination >>>process, or that >>>the send is successfully cancelled, in which case no part of the message >>>was received at the >>>destination. Then, any matching receive has to be satisfied by another >>>send." >>> >>> >>> >>> >>> >>> >

1 0

Re: [mpiwg-p2p] cancel in standard
by Sur, Sayantan 03 Jun '15

03 Jun '15

+MPI P2P WG. We should probably discuss it in the P2P WG as well. I read the snippet of text and thought the “send” mentioned should correspond to the MPI send operation (the one the user issued), and not any internal buffering. “send completes normally” should mean the same thing as sends completing as if cancel wasn’t called. Similarly, “received at the destination” should mean the normal way messages are received at the destination i.e. by posting an MPI receive. Of course, these are my interpretations :) On 6/3/15, 7:19 PM, "Balaji, Pavan" <balaji(a)anl.gov> wrote: > >[Cc'ed the Forum] > >This wording has been debated many times at the Forum. Specifically, the >"completes normally" and "received at the destination" pieces are >ambiguous. For instance, if the data is copied into a bounce buffer, the >send would complete normally, but that does not mean that there is a >receive posted for that data. In this case, it would be the user's >responsibility to post a corresponding receive. If the MPI >implementation does not have a bounce buffer (e.g., for large messages), >"received at the destination" could mean into an unexpected buffer, not >necessarily a user buffer. > >The consensus from MPI-1 veterans (Marc, Bill, etc.) during the last >discussion was that if the application does not post a receive, then it >is guaranteed that the send should be cancelable. At least that was the >intent. The wording, of course, is too vague for such a definition. > >You could try to weaken that interpretation and say that if the MPI >implementation is not able to cancel a send operation, then the user is >responsible for posting a matching receive to receive that data. >Irrespective of which interpretation we go with, I think it is important >to clarify it in the standard. Perhaps an MPI-3.1 errata or MPI-3.2 >change instead of waiting for MPI-4? Do you want to create a ticket? > > -- Pavan > > > > >On 6/3/15, 7:05 PM, "Sur, Sayantan" <sayantan.sur(a)intel.com> wrote: > >>What lines in the standard support the interpretation that “a message >>that >>will not be successfully matched, must be cancelable”? >> >>This is what I found, and it doesn’t support the stricter interpretation. >> >> >> >> >> >> >> >> >> >>"If a >>send is marked for cancellation, then it must be the case that either the >>send completes >>normally, in which case the message sent was received at the destination >>process, or that >>the send is successfully cancelled, in which case no part of the message >>was received at the >>destination. Then, any matching receive has to be satisfied by another >>send." >> >> >> >> >> >>

1 0