On a distributed environment it is difficult to enforce any particular behavior, due to all possible race-condition of the message arrival. Thus, we left this up to the implementation to define the policy in this case. This is specificed on page 538 in the "Advice to implementors". However, if the Rank (0) use MPI_Comm_failure_ack to acknowledge the past failures (and nobody MPI_Revoke the communicator), the request pertaining to the MPI_Irecv will be matching the MPI_Send from Rank (2) (Page 538 line 38). Thus the named MPI_Recv from 2 will never complete based on the provided example. George. On Mar 15, 2013, at 22:19 , David Solt <[email protected]> wrote:
Oh good.
I have another interesting issue:
Rank (0) Rank (1) Rank (2)
MPI_Irecv(.., ANYSOURCE, MPI_ANY_TAG, comm=C) !!!!FAIL!!!! MPI_Wait(req, status) -- > return MPI_ERR_PENDING
MPI_Send(..., 0, 0, comm=C)
MPI_Recv(..., 2, MPI_ANY_TAG, comm=C);
Should the MPI_Recv return or should it block because the pending Irecv could match the MPI_Send from rank 2 and therefore "queues" on it?
I can see arguments for both cases.
Thanks, Dave
From: Aurélien Bouteiller <[email protected]> To: "MPI 3.0 Fault Tolerance and Dynamic Process Control working Group" <[email protected]>, Date: 03/15/2013 01:07 PM Subject: Re: [Mpi3-ft] MPI_Recv + MPI_Comm_failure_ack Sent by: [email protected]
The intent was to return ERR_PROC_FAILED (nothing is pending) and the Ack should also stop this return of PROC_FAILED for blocking ANY_SOURCE.
Good catch Dave!
Aurelien
Le 15 mars 2013 à 14:01, Wesley Bland <[email protected]> a écrit :
You're right. A blocking call shouldn't return MPI_ERR_PENDING when there is no request to be pending. I did think we'd covered this some other way. It's definitely the intent for both versions of receive to be able to ignore acknowledged failures.
On Fri, Mar 15, 2013 at 1:53 PM, David Solt <[email protected]> wrote: I'm pretty sure the intent was that MPI_Recv should NOT return MPI_ERR_PENDING as there is no request on which the error can be pending, but I don't know if much thought was given to allowing MPI_Recv to ignore acknowledge ranks. Dave
From: Wesley Bland <[email protected]> To: "MPI 3.0 Fault Tolerance and Dynamic Process Control working Group" <[email protected]>, Date: 03/15/2013 12:45 PM Subject: Re: [Mpi3-ft] MPI_Recv + MPI_Comm_failure_ack Sent by: [email protected]
I think you are correct in your evaluation, though I also think that wasn't our intent. I think the intent (unless I'm forgetting a discussion) was to allow MPI_ERR_PENDING to be returned by MPI_RECV and let MPI_COMM_FAILURE_ACK cover both cases. Can anyone else confirm that this was the goal.
If that's the case, it's something we'll need to fix in the text.
Thanks, Wesley
On Fri, Mar 15, 2013 at 12:32 PM, David Solt <[email protected]> wrote: Based on the proposal:
MPI_Comm_failure_ack(blah, blah)
This local operation gives the users a way to acknowledge all locally noticed failures on comm. After the call, unmatched MPI_ANY_SOURCE receptions that would have raised an error code MPI_ERR_PENDING due to process failure (see Section 17.2.2) proceed without further reporting of errors due to those acknowledged failures.
I think this clearly indicates that MPI_Recv is uninfluenced by calls to MPI_Comm_failure_ack. Therefore, there is no way to call MPI_Recv(MPI_ANY_SOURCE) and ignore failures reported by MPI_Comm_failure_ack.
I believe the following code will NOT work (i.e. after the first failure, the MPI_Recv will continuously fail):
MPI_Comm_size(intercomm, &size); while (failures < size) { err = MPI_Recv(blah, blah, MPI_ANY_SOURCE, intercomm, &status); if (err == MPI_PROC_FAILED) { MPI_Comm_failure_ack(intercomm); MPI_Comm_failure_get_acked(intercomm, &group); MPI_Group_size(group, &failures); } else { /* process received data */ } }
and has to be written as:
MPI_Comm_size(intercomm, &size); while (failures < size) {
if (request == MPI_REQUEST_NULL) { err = MPI_Irecv(blah, blah, MPI_ANY_SOURCE, intercomm, &request); } err = MPI_Wait(&request, &status);
if (err == MPI_ERR_PENDING) { MPI_Comm_failure_ack(intercomm); MPI_Comm_failure_get_acked(intercomm, &group); MPI_Group_size(group, &failures); } else { /* process received data */ } }
Am I correct in my thinking? If so, was there a reason why MPI_Recv could not also "obey" MPI_Comm_failure_ack calls?
Thanks, Dave _______________________________________________ mpi3-ft mailing list [email protected] http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-ft _______________________________________________ mpi3-ft mailing list [email protected] http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-ft
_______________________________________________ mpi3-ft mailing list [email protected] http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-ft
_______________________________________________ mpi3-ft mailing list [email protected] http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-ft
-- * Dr. Aurélien Bouteiller * Researcher at Innovative Computing Laboratory * University of Tennessee * 1122 Volunteer Boulevard, suite 309b * Knoxville, TN 37996 * 865 974 9375
_______________________________________________ mpi3-ft mailing list [email protected] http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-ft
_______________________________________________ mpi3-ft mailing list [email protected] http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-ft