March 2013 - mpiwg-ft - mailman3.cels.anl.gov

[Mpi3-ft] FTWG phone meeting
by Aurélien Bouteiller 19 Mar '13

19 Mar '13

Dear WG members, This is a reminder that according to our planning, we are having our regular phone meeting. Feel free to contact me if you would like to add to the agenda. Agenda: - Kickoff RMA work (new interface) - Kickoff "other" collective inspection round (non communicator/window/file functions that are collective) Date: Mar. 20, 2013 Time: Noon EDT/New York Dial-in information: 218-339-4600 Code: 623998# Next Meeting: * Apr. 3, 2013 -- * Dr. Aurï¿½lien Bouteiller * Researcher at Innovative Computing Laboratory * University of Tennessee * 1122 Volunteer Boulevard, suite 309b * Knoxville, TN 37996 * 865 974 9375

1 0

Re: [Mpi3-ft] MPI_Recv + MPI_Comm_failure_ack
by David Solt 18 Mar '13

18 Mar '13

My diagram is a sequence of events, but one can write a code sequence that will enforce the ordering at least in an environment where the only errors are forced errors. I agree that page 538 line 38 covers my question. MPI_ERR_PENDING is simply a return status for the wait/test call and in no way changes the request that triggered it. Thanks. Dave From: George Bosilca <bosilca(a)icl.utk.edu> To: "MPI 3.0 Fault Tolerance and Dynamic Process Control working Group" <mpi3-ft(a)lists.mpi-forum.org>, Cc: Geoffrey Paulsen/Dallas/IBM@IBMUS Date: 03/17/2013 08:51 AM Subject: Re: [Mpi3-ft] MPI_Recv + MPI_Comm_failure_ack Sent by: mpi3-ft-bounces(a)lists.mpi-forum.org On a distributed environment it is difficult to enforce any particular behavior, due to all possible race-condition of the message arrival. Thus, we left this up to the implementation to define the policy in this case. This is specificed on page 538 in the "Advice to implementors". However, if the Rank (0) use MPI_Comm_failure_ack to acknowledge the past failures (and nobody MPI_Revoke the communicator), the request pertaining to the MPI_Irecv will be matching the MPI_Send from Rank (2) (Page 538 line 38). Thus the named MPI_Recv from 2 will never complete based on the provided example. George. On Mar 15, 2013, at 22:19 , David Solt <dsolt(a)us.ibm.com> wrote: Oh good. I have another interesting issue: Rank (0) Rank (1) Rank (2) MPI_Irecv(.., ANYSOURCE, MPI_ANY_TAG, comm=C) !!!!FAIL!!!! MPI_Wait(req, status) -- > return MPI_ERR_PENDING MPI_Send(..., 0, 0, comm=C) MPI_Recv(..., 2, MPI_ANY_TAG, comm=C); Should the MPI_Recv return or should it block because the pending Irecv could match the MPI_Send from rank 2 and therefore "queues" on it? I can see arguments for both cases. Thanks, Dave From: Aurélien Bouteiller <bouteill(a)icl.utk.edu> To: "MPI 3.0 Fault Tolerance and Dynamic Process Control working Group" <mpi3-ft(a)lists.mpi-forum.org>, Date: 03/15/2013 01:07 PM Subject: Re: [Mpi3-ft] MPI_Recv + MPI_Comm_failure_ack Sent by: mpi3-ft-bounces(a)lists.mpi-forum.org The intent was to return ERR_PROC_FAILED (nothing is pending) and the Ack should also stop this return of PROC_FAILED for blocking ANY_SOURCE. Good catch Dave! Aurelien Le 15 mars 2013 à 14:01, Wesley Bland <wbland(a)icl.utk.edu> a écrit : > You're right. A blocking call shouldn't return MPI_ERR_PENDING when there is no request to be pending. I did think we'd covered this some other way. It's definitely the intent for both versions of receive to be able to ignore acknowledged failures. > > > On Fri, Mar 15, 2013 at 1:53 PM, David Solt <dsolt(a)us.ibm.com> wrote: > I'm pretty sure the intent was that MPI_Recv should NOT return MPI_ERR_PENDING as there is no request on which the error can be pending, but I don't know if much thought was given to allowing MPI_Recv to ignore acknowledge ranks. > Dave > > > > From: Wesley Bland <wbland(a)icl.utk.edu> > To: "MPI 3.0 Fault Tolerance and Dynamic Process Control working Group" <mpi3-ft(a)lists.mpi-forum.org>, > Date: 03/15/2013 12:45 PM > Subject: Re: [Mpi3-ft] MPI_Recv + MPI_Comm_failure_ack > Sent by: mpi3-ft-bounces(a)lists.mpi-forum.org > > > > I think you are correct in your evaluation, though I also think that wasn't our intent. I think the intent (unless I'm forgetting a discussion) was to allow MPI_ERR_PENDING to be returned by MPI_RECV and let MPI_COMM_FAILURE_ACK cover both cases. Can anyone else confirm that this was the goal. > > If that's the case, it's something we'll need to fix in the text. > > Thanks, > Wesley > > > On Fri, Mar 15, 2013 at 12:32 PM, David Solt <dsolt(a)us.ibm.com> wrote: > Based on the proposal: > > MPI_Comm_failure_ack(blah, blah) > > This local operation gives the users a way to acknowledge all locally noticed failures on > comm. After the call, unmatched MPI_ANY_SOURCE receptions that would have raised an > error code MPI_ERR_PENDING due to process failure (see Section 17.2.2) proceed without > further reporting of errors due to those acknowledged failures. > > I think this clearly indicates that MPI_Recv is uninfluenced by calls to MPI_Comm_failure_ack. Therefore, there is no way to call MPI_Recv(MPI_ANY_SOURCE) and ignore failures reported by MPI_Comm_failure_ack. > > I believe the following code will NOT work (i.e. after the first failure, the MPI_Recv will continuously fail): > > > MPI_Comm_size(intercomm, &size); > while (failures < size) { > err = MPI_Recv(blah, blah, MPI_ANY_SOURCE, intercomm, &status); > if (err == MPI_PROC_FAILED) { > MPI_Comm_failure_ack(intercomm); > MPI_Comm_failure_get_acked(intercomm, &group); > MPI_Group_size(group, &failures); > } else { > /* process received data */ > } > } > > and has to be written as: > > MPI_Comm_size(intercomm, &size); > while (failures < size) { > > if (request == MPI_REQUEST_NULL) { > err = MPI_Irecv(blah, blah, MPI_ANY_SOURCE, intercomm, &request); > } > err = MPI_Wait(&request, &status); > > if (err == MPI_ERR_PENDING) { > MPI_Comm_failure_ack(intercomm); > MPI_Comm_failure_get_acked(intercomm, &group); > MPI_Group_size(group, &failures); > } else { > /* process received data */ > } > } > > Am I correct in my thinking? > If so, was there a reason why MPI_Recv could not also "obey" MPI_Comm_failure_ack calls? > > Thanks, > Dave > _______________________________________________ > mpi3-ft mailing list > mpi3-ft(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-ft > _______________________________________________ > mpi3-ft mailing list > mpi3-ft(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-ft > > _______________________________________________ > mpi3-ft mailing list > mpi3-ft(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-ft > > _______________________________________________ > mpi3-ft mailing list > mpi3-ft(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-ft -- * Dr. Aurélien Bouteiller * Researcher at Innovative Computing Laboratory * University of Tennessee * 1122 Volunteer Boulevard, suite 309b * Knoxville, TN 37996 * 865 974 9375 _______________________________________________ mpi3-ft mailing list mpi3-ft(a)lists.mpi-forum.org http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-ft _______________________________________________ mpi3-ft mailing list mpi3-ft(a)lists.mpi-forum.org http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-ft _______________________________________________ mpi3-ft mailing list mpi3-ft(a)lists.mpi-forum.org http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-ft

1 0

Re: [Mpi3-ft] MPI_Recv + MPI_Comm_failure_ack
by George Bosilca 17 Mar '13

17 Mar '13

On a distributed environment it is difficult to enforce any particular behavior, due to all possible race-condition of the message arrival. Thus, we left this up to the implementation to define the policy in this case. This is specificed on page 538 in the "Advice to implementors". However, if the Rank (0) use MPI_Comm_failure_ack to acknowledge the past failures (and nobody MPI_Revoke the communicator), the request pertaining to the MPI_Irecv will be matching the MPI_Send from Rank (2) (Page 538 line 38). Thus the named MPI_Recv from 2 will never complete based on the provided example. George. On Mar 15, 2013, at 22:19 , David Solt <dsolt(a)us.ibm.com> wrote: > Oh good. > > I have another interesting issue: > > Rank (0) Rank (1) Rank (2) > > MPI_Irecv(.., ANYSOURCE, MPI_ANY_TAG, comm=C) > !!!!FAIL!!!! > MPI_Wait(req, status) -- > return MPI_ERR_PENDING > > MPI_Send(..., 0, 0, comm=C) > > MPI_Recv(..., 2, MPI_ANY_TAG, comm=C); > > > Should the MPI_Recv return or should it block because the pending Irecv could match the MPI_Send from rank 2 and therefore "queues" on it? > > I can see arguments for both cases. > > Thanks, > Dave > > > > > > > From: Aurélien Bouteiller <bouteill(a)icl.utk.edu> > To: "MPI 3.0 Fault Tolerance and Dynamic Process Control working Group" <mpi3-ft(a)lists.mpi-forum.org>, > Date: 03/15/2013 01:07 PM > Subject: Re: [Mpi3-ft] MPI_Recv + MPI_Comm_failure_ack > Sent by: mpi3-ft-bounces(a)lists.mpi-forum.org > > > > The intent was to return ERR_PROC_FAILED (nothing is pending) and the Ack should also stop this return of PROC_FAILED for blocking ANY_SOURCE. > > Good catch Dave! > > Aurelien > > Le 15 mars 2013 à 14:01, Wesley Bland <wbland(a)icl.utk.edu> a écrit : > > > You're right. A blocking call shouldn't return MPI_ERR_PENDING when there is no request to be pending. I did think we'd covered this some other way. It's definitely the intent for both versions of receive to be able to ignore acknowledged failures. > > > > > > On Fri, Mar 15, 2013 at 1:53 PM, David Solt <dsolt(a)us.ibm.com> wrote: > > I'm pretty sure the intent was that MPI_Recv should NOT return MPI_ERR_PENDING as there is no request on which the error can be pending, but I don't know if much thought was given to allowing MPI_Recv to ignore acknowledge ranks. > > Dave > > > > > > > > From: Wesley Bland <wbland(a)icl.utk.edu> > > To: "MPI 3.0 Fault Tolerance and Dynamic Process Control working Group" <mpi3-ft(a)lists.mpi-forum.org>, > > Date: 03/15/2013 12:45 PM > > Subject: Re: [Mpi3-ft] MPI_Recv + MPI_Comm_failure_ack > > Sent by: mpi3-ft-bounces(a)lists.mpi-forum.org > > > > > > > > I think you are correct in your evaluation, though I also think that wasn't our intent. I think the intent (unless I'm forgetting a discussion) was to allow MPI_ERR_PENDING to be returned by MPI_RECV and let MPI_COMM_FAILURE_ACK cover both cases. Can anyone else confirm that this was the goal. > > > > If that's the case, it's something we'll need to fix in the text. > > > > Thanks, > > Wesley > > > > > > On Fri, Mar 15, 2013 at 12:32 PM, David Solt <dsolt(a)us.ibm.com> wrote: > > Based on the proposal: > > > > MPI_Comm_failure_ack(blah, blah) > > > > This local operation gives the users a way to acknowledge all locally noticed failures on > > comm. After the call, unmatched MPI_ANY_SOURCE receptions that would have raised an > > error code MPI_ERR_PENDING due to process failure (see Section 17.2.2) proceed without > > further reporting of errors due to those acknowledged failures. > > > > I think this clearly indicates that MPI_Recv is uninfluenced by calls to MPI_Comm_failure_ack. Therefore, there is no way to call MPI_Recv(MPI_ANY_SOURCE) and ignore failures reported by MPI_Comm_failure_ack. > > > > I believe the following code will NOT work (i.e. after the first failure, the MPI_Recv will continuously fail): > > > > > > MPI_Comm_size(intercomm, &size); > > while (failures < size) { > > err = MPI_Recv(blah, blah, MPI_ANY_SOURCE, intercomm, &status); > > if (err == MPI_PROC_FAILED) { > > MPI_Comm_failure_ack(intercomm); > > MPI_Comm_failure_get_acked(intercomm, &group); > > MPI_Group_size(group, &failures); > > } else { > > /* process received data */ > > } > > } > > > > and has to be written as: > > > > MPI_Comm_size(intercomm, &size); > > while (failures < size) { > > > > if (request == MPI_REQUEST_NULL) { > > err = MPI_Irecv(blah, blah, MPI_ANY_SOURCE, intercomm, &request); > > } > > err = MPI_Wait(&request, &status); > > > > if (err == MPI_ERR_PENDING) { > > MPI_Comm_failure_ack(intercomm); > > MPI_Comm_failure_get_acked(intercomm, &group); > > MPI_Group_size(group, &failures); > > } else { > > /* process received data */ > > } > > } > > > > Am I correct in my thinking? > > If so, was there a reason why MPI_Recv could not also "obey" MPI_Comm_failure_ack calls? > > > > Thanks, > > Dave > > _______________________________________________ > > mpi3-ft mailing list > > mpi3-ft(a)lists.mpi-forum.org > > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-ft > > _______________________________________________ > > mpi3-ft mailing list > > mpi3-ft(a)lists.mpi-forum.org > > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-ft > > > > _______________________________________________ > > mpi3-ft mailing list > > mpi3-ft(a)lists.mpi-forum.org > > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-ft > > > > _______________________________________________ > > mpi3-ft mailing list > > mpi3-ft(a)lists.mpi-forum.org > > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-ft > > -- > * Dr. Aurélien Bouteiller > * Researcher at Innovative Computing Laboratory > * University of Tennessee > * 1122 Volunteer Boulevard, suite 309b > * Knoxville, TN 37996 > * 865 974 9375 > > > > > > > > > _______________________________________________ > mpi3-ft mailing list > mpi3-ft(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-ft > > > _______________________________________________ > mpi3-ft mailing list > mpi3-ft(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-ft

1 0

Re: [Mpi3-ft] MPI_Recv + MPI_Comm_failure_ack
by David Solt 15 Mar '13

15 Mar '13

Oh good. I have another interesting issue: Rank (0) Rank (1) Rank (2) MPI_Irecv(.., ANYSOURCE, MPI_ANY_TAG, comm=C) !!!!FAIL!!!! MPI_Wait(req, status) -- > return MPI_ERR_PENDING MPI_Send(..., 0, 0, comm=C) MPI_Recv(..., 2, MPI_ANY_TAG, comm=C); Should the MPI_Recv return or should it block because the pending Irecv could match the MPI_Send from rank 2 and therefore "queues" on it? I can see arguments for both cases. Thanks, Dave From: Aurélien Bouteiller <bouteill(a)icl.utk.edu> To: "MPI 3.0 Fault Tolerance and Dynamic Process Control working Group" <mpi3-ft(a)lists.mpi-forum.org>, Date: 03/15/2013 01:07 PM Subject: Re: [Mpi3-ft] MPI_Recv + MPI_Comm_failure_ack Sent by: mpi3-ft-bounces(a)lists.mpi-forum.org The intent was to return ERR_PROC_FAILED (nothing is pending) and the Ack should also stop this return of PROC_FAILED for blocking ANY_SOURCE. Good catch Dave! Aurelien Le 15 mars 2013 à 14:01, Wesley Bland <wbland(a)icl.utk.edu> a écrit : > You're right. A blocking call shouldn't return MPI_ERR_PENDING when there is no request to be pending. I did think we'd covered this some other way. It's definitely the intent for both versions of receive to be able to ignore acknowledged failures. > > > On Fri, Mar 15, 2013 at 1:53 PM, David Solt <dsolt(a)us.ibm.com> wrote: > I'm pretty sure the intent was that MPI_Recv should NOT return MPI_ERR_PENDING as there is no request on which the error can be pending, but I don't know if much thought was given to allowing MPI_Recv to ignore acknowledge ranks. > Dave > > > > From: Wesley Bland <wbland(a)icl.utk.edu> > To: "MPI 3.0 Fault Tolerance and Dynamic Process Control working Group" <mpi3-ft(a)lists.mpi-forum.org>, > Date: 03/15/2013 12:45 PM > Subject: Re: [Mpi3-ft] MPI_Recv + MPI_Comm_failure_ack > Sent by: mpi3-ft-bounces(a)lists.mpi-forum.org > > > > I think you are correct in your evaluation, though I also think that wasn't our intent. I think the intent (unless I'm forgetting a discussion) was to allow MPI_ERR_PENDING to be returned by MPI_RECV and let MPI_COMM_FAILURE_ACK cover both cases. Can anyone else confirm that this was the goal. > > If that's the case, it's something we'll need to fix in the text. > > Thanks, > Wesley > > > On Fri, Mar 15, 2013 at 12:32 PM, David Solt <dsolt(a)us.ibm.com> wrote: > Based on the proposal: > > MPI_Comm_failure_ack(blah, blah) > > This local operation gives the users a way to acknowledge all locally noticed failures on > comm. After the call, unmatched MPI_ANY_SOURCE receptions that would have raised an > error code MPI_ERR_PENDING due to process failure (see Section 17.2.2) proceed without > further reporting of errors due to those acknowledged failures. > > I think this clearly indicates that MPI_Recv is uninfluenced by calls to MPI_Comm_failure_ack. Therefore, there is no way to call MPI_Recv(MPI_ANY_SOURCE) and ignore failures reported by MPI_Comm_failure_ack. > > I believe the following code will NOT work (i.e. after the first failure, the MPI_Recv will continuously fail): > > > MPI_Comm_size(intercomm, &size); > while (failures < size) { > err = MPI_Recv(blah, blah, MPI_ANY_SOURCE, intercomm, &status); > if (err == MPI_PROC_FAILED) { > MPI_Comm_failure_ack(intercomm); > MPI_Comm_failure_get_acked(intercomm, &group); > MPI_Group_size(group, &failures); > } else { > /* process received data */ > } > } > > and has to be written as: > > MPI_Comm_size(intercomm, &size); > while (failures < size) { > > if (request == MPI_REQUEST_NULL) { > err = MPI_Irecv(blah, blah, MPI_ANY_SOURCE, intercomm, &request); > } > err = MPI_Wait(&request, &status); > > if (err == MPI_ERR_PENDING) { > MPI_Comm_failure_ack(intercomm); > MPI_Comm_failure_get_acked(intercomm, &group); > MPI_Group_size(group, &failures); > } else { > /* process received data */ > } > } > > Am I correct in my thinking? > If so, was there a reason why MPI_Recv could not also "obey" MPI_Comm_failure_ack calls? > > Thanks, > Dave > _______________________________________________ > mpi3-ft mailing list > mpi3-ft(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-ft > _______________________________________________ > mpi3-ft mailing list > mpi3-ft(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-ft > > _______________________________________________ > mpi3-ft mailing list > mpi3-ft(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-ft > > _______________________________________________ > mpi3-ft mailing list > mpi3-ft(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-ft -- * Dr. Aurélien Bouteiller * Researcher at Innovative Computing Laboratory * University of Tennessee * 1122 Volunteer Boulevard, suite 309b * Knoxville, TN 37996 * 865 974 9375 _______________________________________________ mpi3-ft mailing list mpi3-ft(a)lists.mpi-forum.org http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-ft

1 0

Re: [Mpi3-ft] MPI_Recv + MPI_Comm_failure_ack
by Aurélien Bouteiller 15 Mar '13

15 Mar '13

The intent was to return ERR_PROC_FAILED (nothing is pending) and the Ack should also stop this return of PROC_FAILED for blocking ANY_SOURCE. Good catch Dave! Aurelien Le 15 mars 2013 ï¿½ 14:01, Wesley Bland <wbland(a)icl.utk.edu> a ï¿½crit : > You're right. A blocking call shouldn't return MPI_ERR_PENDING when there is no request to be pending. I did think we'd covered this some other way. It's definitely the intent for both versions of receive to be able to ignore acknowledged failures. > > > On Fri, Mar 15, 2013 at 1:53 PM, David Solt <dsolt(a)us.ibm.com> wrote: > I'm pretty sure the intent was that MPI_Recv should NOT return MPI_ERR_PENDING as there is no request on which the error can be pending, but I don't know if much thought was given to allowing MPI_Recv to ignore acknowledge ranks. > Dave > > > > From: Wesley Bland <wbland(a)icl.utk.edu> > To: "MPI 3.0 Fault Tolerance and Dynamic Process Control working Group" <mpi3-ft(a)lists.mpi-forum.org>, > Date: 03/15/2013 12:45 PM > Subject: Re: [Mpi3-ft] MPI_Recv + MPI_Comm_failure_ack > Sent by: mpi3-ft-bounces(a)lists.mpi-forum.org > > > > I think you are correct in your evaluation, though I also think that wasn't our intent. I think the intent (unless I'm forgetting a discussion) was to allow MPI_ERR_PENDING to be returned by MPI_RECV and let MPI_COMM_FAILURE_ACK cover both cases. Can anyone else confirm that this was the goal. > > If that's the case, it's something we'll need to fix in the text. > > Thanks, > Wesley > > > On Fri, Mar 15, 2013 at 12:32 PM, David Solt <dsolt(a)us.ibm.com> wrote: > Based on the proposal: > > MPI_Comm_failure_ack(blah, blah) > > This local operation gives the users a way to acknowledge all locally noticed failures on > comm. After the call, unmatched MPI_ANY_SOURCE receptions that would have raised an > error code MPI_ERR_PENDING due to process failure (see Section 17.2.2) proceed without > further reporting of errors due to those acknowledged failures. > > I think this clearly indicates that MPI_Recv is uninfluenced by calls to MPI_Comm_failure_ack. Therefore, there is no way to call MPI_Recv(MPI_ANY_SOURCE) and ignore failures reported by MPI_Comm_failure_ack. > > I believe the following code will NOT work (i.e. after the first failure, the MPI_Recv will continuously fail): > > > MPI_Comm_size(intercomm, &size); > while (failures < size) { > err = MPI_Recv(blah, blah, MPI_ANY_SOURCE, intercomm, &status); > if (err == MPI_PROC_FAILED) { > MPI_Comm_failure_ack(intercomm); > MPI_Comm_failure_get_acked(intercomm, &group); > MPI_Group_size(group, &failures); > } else { > /* process received data */ > } > } > > and has to be written as: > > MPI_Comm_size(intercomm, &size); > while (failures < size) { > > if (request == MPI_REQUEST_NULL) { > err = MPI_Irecv(blah, blah, MPI_ANY_SOURCE, intercomm, &request); > } > err = MPI_Wait(&request, &status); > > if (err == MPI_ERR_PENDING) { > MPI_Comm_failure_ack(intercomm); > MPI_Comm_failure_get_acked(intercomm, &group); > MPI_Group_size(group, &failures); > } else { > /* process received data */ > } > } > > Am I correct in my thinking? > If so, was there a reason why MPI_Recv could not also "obey" MPI_Comm_failure_ack calls? > > Thanks, > Dave > _______________________________________________ > mpi3-ft mailing list > mpi3-ft(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-ft > _______________________________________________ > mpi3-ft mailing list > mpi3-ft(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-ft > > _______________________________________________ > mpi3-ft mailing list > mpi3-ft(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-ft > > _______________________________________________ > mpi3-ft mailing list > mpi3-ft(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-ft -- * Dr. Aurï¿½lien Bouteiller * Researcher at Innovative Computing Laboratory * University of Tennessee * 1122 Volunteer Boulevard, suite 309b * Knoxville, TN 37996 * 865 974 9375

1 0

Re: [Mpi3-ft] MPI_Recv + MPI_Comm_failure_ack
by Wesley Bland 15 Mar '13

15 Mar '13

You're right. A blocking call shouldn't return MPI_ERR_PENDING when there is no request to be pending. I did think we'd covered this some other way. It's definitely the intent for both versions of receive to be able to ignore acknowledged failures. On Fri, Mar 15, 2013 at 1:53 PM, David Solt <dsolt(a)us.ibm.com> wrote: > I'm pretty sure the intent was that MPI_Recv should NOT return > MPI_ERR_PENDING as there is no request on which the error can be pending, > but I don't know if much thought was given to allowing MPI_Recv to ignore > acknowledge ranks. > Dave > > > > From: Wesley Bland <wbland(a)icl.utk.edu> > To: "MPI 3.0 Fault Tolerance and Dynamic Process Control working > Group" <mpi3-ft(a)lists.mpi-forum.org>, > Date: 03/15/2013 12:45 PM > Subject: Re: [Mpi3-ft] MPI_Recv + MPI_Comm_failure_ack > Sent by: mpi3-ft-bounces(a)lists.mpi-forum.org > ------------------------------ > > > > I think you are correct in your evaluation, though I also think that > wasn't our intent. I think the intent (unless I'm forgetting a discussion) > was to allow MPI_ERR_PENDING to be returned by MPI_RECV and let > MPI_COMM_FAILURE_ACK cover both cases. Can anyone else confirm that this > was the goal. > > If that's the case, it's something we'll need to fix in the text. > > Thanks, > Wesley > > > On Fri, Mar 15, 2013 at 12:32 PM, David Solt <*dsolt(a)us.ibm.com*<dsolt(a)us.ibm.com>> > wrote: > Based on the proposal: > > MPI_Comm_failure_ack(blah, blah) > > This local operation gives the users a way to acknowledge all locally > noticed failures on > comm. After the call, unmatched MPI_ANY_SOURCE receptions that would have > raised an > error code MPI_ERR_PENDING due to process failure (see Section 17.2.2) > proceed without > further reporting of errors due to those acknowledged failures. > > I think this clearly indicates that MPI_Recv is uninfluenced by calls to > MPI_Comm_failure_ack. Therefore, there is no way to call > MPI_Recv(MPI_ANY_SOURCE) and ignore failures reported by > MPI_Comm_failure_ack. > > I believe the following code will NOT work (i.e. after the first failure, > the MPI_Recv will continuously fail): > > > MPI_Comm_size(intercomm, &size); > while (failures < size) { > err = MPI_Recv(blah, blah, MPI_ANY_SOURCE, intercomm, &status); > if (err == MPI_PROC_FAILED) { > MPI_Comm_failure_ack(intercomm); > MPI_Comm_failure_get_acked(intercomm, &group); > MPI_Group_size(group, &failures); > } else { > /* process received data */ > } > } > > and has to be written as: > > MPI_Comm_size(intercomm, &size); > while (failures < size) { > > if (request == MPI_REQUEST_NULL) { > err = MPI_Irecv(blah, blah, MPI_ANY_SOURCE, intercomm, > &request); > } > err = MPI_Wait(&request, &status); > > if (err == MPI_ERR_PENDING) { > MPI_Comm_failure_ack(intercomm); > MPI_Comm_failure_get_acked(intercomm, &group); > MPI_Group_size(group, &failures); > } else { > /* process received data */ > } > } > > Am I correct in my thinking? > If so, was there a reason why MPI_Recv could not also "obey" > MPI_Comm_failure_ack calls? > > Thanks, > Dave > _______________________________________________ > mpi3-ft mailing list* > **mpi3-ft(a)lists.mpi-forum.org* <mpi3-ft(a)lists.mpi-forum.org>* > **http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-ft*<http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-ft> > _______________________________________________ > mpi3-ft mailing list > mpi3-ft(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-ft > > _______________________________________________ > mpi3-ft mailing list > mpi3-ft(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-ft >

1 0

Re: [Mpi3-ft] MPI_Recv + MPI_Comm_failure_ack
by David Solt 15 Mar '13

15 Mar '13

I'm pretty sure the intent was that MPI_Recv should NOT return MPI_ERR_PENDING as there is no request on which the error can be pending, but I don't know if much thought was given to allowing MPI_Recv to ignore acknowledge ranks. Dave From: Wesley Bland <wbland(a)icl.utk.edu> To: "MPI 3.0 Fault Tolerance and Dynamic Process Control working Group" <mpi3-ft(a)lists.mpi-forum.org>, Date: 03/15/2013 12:45 PM Subject: Re: [Mpi3-ft] MPI_Recv + MPI_Comm_failure_ack Sent by: mpi3-ft-bounces(a)lists.mpi-forum.org I think you are correct in your evaluation, though I also think that wasn't our intent. I think the intent (unless I'm forgetting a discussion) was to allow MPI_ERR_PENDING to be returned by MPI_RECV and let MPI_COMM_FAILURE_ACK cover both cases. Can anyone else confirm that this was the goal. If that's the case, it's something we'll need to fix in the text. Thanks, Wesley On Fri, Mar 15, 2013 at 12:32 PM, David Solt <dsolt(a)us.ibm.com> wrote: Based on the proposal: MPI_Comm_failure_ack(blah, blah) This local operation gives the users a way to acknowledge all locally noticed failures on comm. After the call, unmatched MPI_ANY_SOURCE receptions that would have raised an error code MPI_ERR_PENDING due to process failure (see Section 17.2.2) proceed without further reporting of errors due to those acknowledged failures. I think this clearly indicates that MPI_Recv is uninfluenced by calls to MPI_Comm_failure_ack. Therefore, there is no way to call MPI_Recv(MPI_ANY_SOURCE) and ignore failures reported by MPI_Comm_failure_ack. I believe the following code will NOT work (i.e. after the first failure, the MPI_Recv will continuously fail): MPI_Comm_size(intercomm, &size); while (failures < size) { err = MPI_Recv(blah, blah, MPI_ANY_SOURCE, intercomm, &status); if (err == MPI_PROC_FAILED) { MPI_Comm_failure_ack(intercomm); MPI_Comm_failure_get_acked(intercomm, &group); MPI_Group_size(group, &failures); } else { /* process received data */ } } and has to be written as: MPI_Comm_size(intercomm, &size); while (failures < size) { if (request == MPI_REQUEST_NULL) { err = MPI_Irecv(blah, blah, MPI_ANY_SOURCE, intercomm, &request); } err = MPI_Wait(&request, &status); if (err == MPI_ERR_PENDING) { MPI_Comm_failure_ack(intercomm); MPI_Comm_failure_get_acked(intercomm, &group); MPI_Group_size(group, &failures); } else { /* process received data */ } } Am I correct in my thinking? If so, was there a reason why MPI_Recv could not also "obey" MPI_Comm_failure_ack calls? Thanks, Dave _______________________________________________ mpi3-ft mailing list mpi3-ft(a)lists.mpi-forum.org http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-ft _______________________________________________ mpi3-ft mailing list mpi3-ft(a)lists.mpi-forum.org http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-ft

1 0

Re: [Mpi3-ft] MPI_Recv + MPI_Comm_failure_ack
by Wesley Bland 15 Mar '13

15 Mar '13

I think you are correct in your evaluation, though I also think that wasn't our intent. I think the intent (unless I'm forgetting a discussion) was to allow MPI_ERR_PENDING to be returned by MPI_RECV and let MPI_COMM_FAILURE_ACK cover both cases. Can anyone else confirm that this was the goal. If that's the case, it's something we'll need to fix in the text. Thanks, Wesley On Fri, Mar 15, 2013 at 12:32 PM, David Solt <dsolt(a)us.ibm.com> wrote: > Based on the proposal: > > MPI_Comm_failure_ack(blah, blah) > > This local operation gives the users a way to acknowledge all locally > noticed failures on > comm. After the call, unmatched MPI_ANY_SOURCE receptions that would have > raised an > error code MPI_ERR_PENDING due to process failure (see Section 17.2.2) > proceed without > further reporting of errors due to those acknowledged failures. > > I think this clearly indicates that MPI_Recv is uninfluenced by calls to > MPI_Comm_failure_ack. Therefore, there is no way to call > MPI_Recv(MPI_ANY_SOURCE) and ignore failures reported by > MPI_Comm_failure_ack. > > I believe the following code will NOT work (i.e. after the first failure, > the MPI_Recv will continuously fail): > > > MPI_Comm_size(intercomm, &size); > while (failures < size) { > err = MPI_Recv(blah, blah, MPI_ANY_SOURCE, intercomm, &status); > if (err == MPI_PROC_FAILED) { > MPI_Comm_failure_ack(intercomm); > MPI_Comm_failure_get_acked(intercomm, &group); > MPI_Group_size(group, &failures); > } else { > /* process received data */ > } > } > > and has to be written as: > > MPI_Comm_size(intercomm, &size); > while (failures < size) { > > if (request == MPI_REQUEST_NULL) { > err = MPI_Irecv(blah, blah, MPI_ANY_SOURCE, intercomm, > &request); > } > err = MPI_Wait(&request, &status); > > if (err == MPI_ERR_PENDING) { > MPI_Comm_failure_ack(intercomm); > MPI_Comm_failure_get_acked(intercomm, &group); > MPI_Group_size(group, &failures); > } else { > /* process received data */ > } > } > > Am I correct in my thinking? > If so, was there a reason why MPI_Recv could not also "obey" > MPI_Comm_failure_ack calls? > > Thanks, > Dave > _______________________________________________ > mpi3-ft mailing list > mpi3-ft(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-ft >

1 0

[Mpi3-ft] MPI_Recv + MPI_Comm_failure_ack
by David Solt 15 Mar '13

15 Mar '13

Based on the proposal: MPI_Comm_failure_ack(blah, blah) This local operation gives the users a way to acknowledge all locally noticed failures on comm. After the call, unmatched MPI_ANY_SOURCE receptions that would have raised an error code MPI_ERR_PENDING due to process failure (see Section 17.2.2) proceed without further reporting of errors due to those acknowledged failures. I think this clearly indicates that MPI_Recv is uninfluenced by calls to MPI_Comm_failure_ack. Therefore, there is no way to call MPI_Recv(MPI_ANY_SOURCE) and ignore failures reported by MPI_Comm_failure_ack. I believe the following code will NOT work (i.e. after the first failure, the MPI_Recv will continuously fail): MPI_Comm_size(intercomm, &size); while (failures < size) { err = MPI_Recv(blah, blah, MPI_ANY_SOURCE, intercomm, &status); if (err == MPI_PROC_FAILED) { MPI_Comm_failure_ack(intercomm); MPI_Comm_failure_get_acked(intercomm, &group); MPI_Group_size(group, &failures); } else { /* process received data */ } } and has to be written as: MPI_Comm_size(intercomm, &size); while (failures < size) { if (request == MPI_REQUEST_NULL) { err = MPI_Irecv(blah, blah, MPI_ANY_SOURCE, intercomm, &request); } err = MPI_Wait(&request, &status); if (err == MPI_ERR_PENDING) { MPI_Comm_failure_ack(intercomm); MPI_Comm_failure_get_acked(intercomm, &group); MPI_Group_size(group, &failures); } else { /* process received data */ } } Am I correct in my thinking? If so, was there a reason why MPI_Recv could not also "obey" MPI_Comm_failure_ack calls? Thanks, Dave

1 0