Ron, On 11/26/2012 12:18 PM, Brightwell, Ronald wrote:

The issue is not whether the local non-blocking operation will complete, but whether starting a local non-blocking operation is enough to satisfy the matching non-local operation. It is essentially saying that once an operation has been started and matched, completion is local. That is, completion of a send operation that has been started and satisfied by a matching receive operation (that has also been started) is not dependent on whether that receive operation has been completed. The send side doesn't have to wait for the receive side to call MPI_WAIT in order for the send to complete.

Yes, I just want the standard to clarify that what you wrote above is guaranteed MPI behavior, not simply desirable behavior. Let me provide a little more detail on what I'm dealing with. I believe, but have not yet managed to prove, that a large LANL application is doing something like the following: Rank 0 Rank 1 ------ ------ ISSEND 1 IRECV 0 WAIT ALLREDUCE ALLREDUCE WAIT If MPI guarantees that "the send side doesn't have to wait for the receive side to call MPI_WAIT in order for the send to complete" (i.e., the non-local matching you refer to), then the preceding pseudocode represents a correct use of MPI. If, however, it would merely be "nice" for the send not to have to wait for the receive-side MPI_WAIT, then the preceding pseudocode represents an incorrect use of MPI that happens to work with all tested MPI implementations but may deadlock under some future implementation.

I'm not sure there's any semantic difference between "... the receive should complete..." versus "... the receive must complete...", but the latter does seem stronger.

Some standards bodies explicitly distinguish "must" as meaning, "or else you don't have a compliant implementation" and "should" as meaning, "or else you have a technically compliant but probably poorly performing implementation." The key is whether users can rely on the stated behavior for correct application execution, as in my pseudocode example above. -- Scott