November 2015 - mpi-forum - mailman3.cels.anl.gov

Re: [Mpi-forum] Cancelling a send matched by a matching probe
by William Gropp 04 Nov '15

04 Nov '15

Yes, the current text is buggy and also trying to address a very subtle point. The intent of the text you quote about MPI_Wait is to ensure that either a message is cancelled successfully, or it completes, rather than being in some deferred not-yet-cancelled-and-not-yet-completed state. The phrase “irrespective of the activities of the other processes” is the most confusing. It was intended to mean that, once cancel is called on the request, the request will either be successfully cancelled or it will successfully complete without relying on further MPI calls in the user program at the target rank. It was not intended to mean that this might not require processing at the target rank (and in fact some of the cancel text points this out), just that it can’t wait for some subsequent MPI call made by the user to decide whether the cancel was successful. I know, the text isn’t very clear about that, but that was the intent. The text is also not correct once you include any way to “observe” a message other than posting a receive. Here’s the thinking on this. Assume that there are only sends and receives. In the case where the cancel failed, the MPI_Wait should succeed because the send is matched. Thus, the receive will complete, and the MPI_Wait will return without the process on the target node needing to execute any *other* MPI calls (this assumes something about progress, of course). Thus, in the MPI sense, the MPI_Wait is local because it doesn’t require any (more) MPI calls at the target. This breaks once you can observe a message with probe or mprobe (or MPI_T). Then you have an observation without a message match, and then MPI_Wait might not complete until there is a matching receive issued by the user at the target process. Fixing this would require a careful definition of the states and the consequences; in particular, it would probably require that MPI_Wait complete only in the case where the send had been matched by a receive or not observed at all; if observed without a receive, we’d need another state and would require the user to issue the necessary receive commands on it. Getting this right is not easy, which is why removing cancel of send might be the best route. Bill William Gropp Director, Parallel Computing Institute Thomas M. Siebel Chair in Computer Science Chief Scientist, NCSA University of Illinois Urbana-Champaign On Nov 3, 2015, at 3:41 PM, Marek Tomáštík <tomastik.marek(a)gmail.com> wrote: > Thank you for the detailed explanation. The described intent makes sense, but I am not sure how this statement follows from the standard: > > 2015-11-02 15:41 GMT+01:00 William Gropp <wgropp(a)illinois.edu>: > However, note that if the cancel fails, then the communication is not marked for cancellation, and an MPI_Wait could then wait until the message is received. > > The standard says that "[a] call to MPI_CANCEL marks for cancellation a pending, nonblocking communication operation". It does not say that the cancellation has to suceed, but it does say that MPI_Cancel marks the operation for cancellation -- unconditionally, as far as I can see (unless an error occurs, of course -- but that wasn't what you meant by the cancel failing, or was it?). From this and the statement I cited previously ("[i]f a communication is marked for cancellation, then a MPI_WAIT call for that communication is guaranteed to return, irrespective of the activities of other processes (i.e., MPI_WAIT behaves as a local function)") it follows by modus ponens that MPI_Wait for a request on which MPI_Cancel has been called must behave as a local function. Am I interpreting this incorrectly? > > Once again, thank you for your time. > > Marek Tomáštík > _______________________________________________ > mpi-forum mailing list > mpi-forum(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum

1 0

Re: [Mpi-forum] Cancelling a send matched by a matching probe
by Marek Tomáštík 03 Nov '15

03 Nov '15

Thank you for the detailed explanation. The described intent makes sense, but I am not sure how this statement follows from the standard: 2015-11-02 15:41 GMT+01:00 William Gropp <wgropp(a)illinois.edu>: > However, note that if the cancel fails, then the communication is not > marked for cancellation, and an MPI_Wait could then wait until the message > is received. > The standard says that "[a] call to MPI_CANCEL marks for cancellation a pending, nonblocking communication operation". It does not say that the cancellation has to suceed, but it does say that MPI_Cancel marks the operation for cancellation -- unconditionally, as far as I can see (unless an error occurs, of course -- but that wasn't what you meant by the cancel failing, or was it?). From this and the statement I cited previously ("[i]f a communication is marked for cancellation, then a MPI_WAIT call for that communication is guaranteed to return, irrespective of the activities of other processes (i.e., MPI_WAIT behaves as a local function)") it follows by modus ponens that MPI_Wait for a request on which MPI_Cancel has been called must behave as a local function. Am I interpreting this incorrectly? Once again, thank you for your time. Marek Tomáštík

1 0

MPI Forum Git/GitHub introduction
by Jeff Squyres (jsquyres) 02 Nov '15

02 Nov '15

We recorded the webex today for those who were not able to join us: MPI Forum GitHub wiki (NEWBIES START HERE): https://github.com/mpi-forum/mpi-issues/wiki Webex recording: https://cisco.webex.com/ciscosales/lsr.php?RCID=e41fbcf8c4d94f8b86544f053c0… Slides we presented on the webex today: https://docs.google.com/presentation/d/1QREx9kFPITvAeg5SP08QR3E7zJdF5X48g98… -- Jeff Squyres jsquyres(a)cisco.com For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/

1 0

Re: [Mpi-forum] Cancelling a send matched by a matching probe
by William Gropp 02 Nov '15

02 Nov '15

That’s one of the reasons for the various tickets to address the send-cancel description. However, note that if the cancel fails, then the communication is not marked for cancellation, and an MPI_Wait could then wait until the message is received. The intent was this: P0 sends a message. Message arrives at P1 and and is processed by the MPI runtime, but the user code hasn’t matched it (e.g., message is placed in the unexpected message queue). P0 attempts to cancel the message. This causes a cancel request to be sent to P1. If the message is in the unexpected queue, and hasn’t been matched by any user MPI call (recv, mprobe, or iprobe), then the message is removed from the unexpected message queue, and a ack is sent from P1 to P0 that the message was successfully canceled. Otherwise, P1 sends a nack to P0, saying that it was unable to cancel the message. Note that this requires the MPI runtime to track which unexpected messages have been “observed”. See below. However, the MPI standard doesn’t have an unexpected message queue concept; that is a (nearly universal) implementation detail. If we retain cancel of sends (for which there are very few but unfortunately not zero uses), we’ll need to update the text to clearly define the behavior without referring to implementation approaches (except as an advice to implementors). It is further complicated by the MPI_T interface - what if a performance variable is queried about the number of messages in the unexpected queue? Does that count as an observation of the message? Bill William Gropp Director, Parallel Computing Institute Thomas M. Siebel Chair in Computer Science Chief Scientist, NCSA University of Illinois Urbana-Champaign On Nov 1, 2015, at 11:59 AM, Marek Tomáštík <tomastik.marek(a)gmail.com> wrote: > 2015-10-31 17:33 GMT+01:00 William Gropp <wgropp(a)illinois.edu>: > >Under the interpretation of using whether the target process has observed, > >rather than received the message, a consistent interpretation is that the > >MPI_Cancel fails at the source, as it would if the message had been received. > > But does that satisfy the requirement that "[i]f a communication is marked for cancellation, then a MPI_WAIT call for that communication is guaranteed to return, irrespective of the activities of other processes (i.e., MPI_WAIT behaves as a local function); similarly if MPI_TEST is repeatedly called in a busy wait loop for a cancelled communication, then MPI_TEST will eventually be successful"? As far as I can see, MPI_Wait for the matched send cannot be local, since it can complete only once the receiver actually receives the message. > > I understand that cancelling of sends is in the process of being phased out of the standard, but I would like to find out how/if it's possible to implement this feature correctly given that it is, currently, a part of the standard. > > Marek Tomáštík > _______________________________________________ > mpi-forum mailing list > mpi-forum(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum

1 0

Reminder: GitHub WebEx - 11:00 AM Eastern TODAY
by Bland, Wesley 02 Nov '15

02 Nov '15

As a reminder for those of you who didn’t make it to the bottom of the email from a couple of weeks ago. Jeff Squyres and I will be hosting a Webex TODAY at 11:00 AM Eastern to go over the basics of our GitHub workflow and do a short demo. The connection information is below: Webex URL: https://cisco.webex.com/ciscosales/j.phpMTID=m2b0cc8e68181a2288cec8658c3bb7… <https://cisco.webex.com/ciscosales/j.php?MTID=m2b0cc8e68181a2288cec8658c3bb…> Webex pw: nAmKMSEH --Wesley

1 0

Re: [Mpi-forum] Cancelling a send matched by a matching probe
by Marek Tomáštík 01 Nov '15

01 Nov '15

2015-10-31 17:33 GMT+01:00 William Gropp <wgropp(a)illinois.edu>: >Under the interpretation of using whether the target process has observed, >rather than received the message, a consistent interpretation is that the >MPI_Cancel fails at the source, as it would if the message had been received. But does that satisfy the requirement that "[i]f a communication is marked for cancellation, then a MPI_WAIT call for that communication is guaranteed to return, irrespective of the activities of other processes (i.e., MPI_WAIT behaves as a local function); similarly if MPI_TEST is repeatedly called in a busy wait loop for a cancelled communication, then MPI_TEST will eventually be successful"? As far as I can see, MPI_Wait for the matched send cannot be local, since it can complete only once the receiver actually receives the message. I understand that cancelling of sends is in the process of being phased out of the standard, but I would like to find out how/if it's possible to implement this feature correctly given that it is, currently, a part of the standard. Marek Tomáštík

1 0