January 2012 - mpiwg-ft - mailman3.cels.anl.gov

Re: [Mpi3-ft] MPI_ANY_SOURCE ... again...
by Bronevetsky, Greg 26 Jan '12

26 Jan '12

I agree that the "MPI_ANY_SOURCE returns on error" semantics is better for users. However, if this is going to be a sticking point for the rest of the forum, it is not actually that difficult to fake this functionality on top of the "MPI_ANY_SOURCE blocks" semantics. Josh, you correctly pointed out that the MPI implementation should be able to leverage its own out of band failure detectors to implement the "returns" functionality but if that is the case, why can't the vendor provide an optional layer to the user that will do exactly the same thing but without messing with the MPI forum? What I'm proposing is that vendors or any other software developers provide a failure notification layer that sends an MPI message on a pre-defined communicator to the process. They would also provide a PMPI layer that wraps MPI_Receive(MPI_ANY_SOURCE) so that it alternates between testing for the arrival of messages that match this operation and a failure notification message. If the former arrives first, the wrapper returns normally. If the latter arrives first, the original MPI_Receive(MPI_ANY_SOURCE) is cancelled and the call returns with an error. Conveniently, since the failure notifier and the PMPI layer are orthogonal, we can connect the application to any failure detector, making it possible to provide these for systems where the vendors are lazy. Greg Bronevetsky Lawrence Livermore National Lab (925) 424-5756 bronevetsky(a)llnl.gov http://greg.bronevetsky.com > -----Original Message----- > From: mpi3-ft-bounces(a)lists.mpi-forum.org [mailto:mpi3-ft- > bounces(a)lists.mpi-forum.org] On Behalf Of Martin Schulz > Sent: Wednesday, January 25, 2012 4:30 PM > To: MPI 3.0 Fault Tolerance and Dynamic Process Control working Group > Subject: Re: [Mpi3-ft] MPI_ANY_SOURCE ... again... > > Hi Josh, all, > > I agree, I think the current approach is fine. Blocking is likely to be more > problematic in many cases, IMHO. However, I am still a bit worried about > splitting the semantics for P2P and Collective routines. I don't see a reason > why a communicator after a collective call to validate wouldn't support > ANY_SOURCE. If it is split, though, any intermediate layer trying to replace > collectives with P2P solutions (and there are plenty of tuning frameworks out > there that try exactly that) have a hard time maintaing the same semantics in > an error case. > > Martin > > > On Jan 25, 2012, at 3:06 PM, Howard Pritchard wrote: > > > Hi Josh, > > > > Cray is okay with the semantics described in the current FTWG proposal > > attached to the ticket. > > > > We plan to just leverage the out-of-band system fault detector > > software that currently kills jobs if a node goes down that the job > > was running on. > > > > Howard > > > > Josh Hursey wrote: > >> We really need to make a decision on semantics for MPI_ANY_SOURCE. > >> > >> During the plenary session the MPI Forum had a problem with the > >> current proposed semantics. The current proposal states (roughly) > >> that MPI_ANY_SOURCE return when a failure emerges in the > >> communicator. The MPI Forum read this as a strong requirement for > >> -progress- (something the MPI standard tries to stay away from). > >> > >> The alternative proposal is that a receive on MPI_ANY_SOURCE will > >> block until completed with a message. This means that it will -not- > >> return when a new failure has been encountered (even if the calling > >> process is the only process left alive in the communicator). This > >> does get around the concern about progress, but puts a large burden on > the end user. > >> > >> > >> There are a couple good use cases for MPI_ANY_SOURCE (grumble, > >> grumble) > >> - Manager/Worker applications, and easy load balancing when multiple > >> incoming messages are expected. This blocking behavior makes the use > >> of MPI_ANY_SOURCE dangerous for fault tolerant applications, and > >> opens up another opportunity for deadlock. > >> > >> For applications that want to use MPI_ANY_SOURCE and be fault > >> tolerant they will need to build their own failure detector on top of > >> MPI using directed point-to-point messages. A basic implementation > >> might post MPI_Irecv()'s to each worker process with an unused tag, > >> then poll on Testany(). If any of these requests complete in error > >> (MPI_ERR_PROC_FAIL_STOP) then the target has failed and the > >> application can take action. This user-level failure detector can > >> (should) be implemented in a third-party library since failure > >> detectors can be difficult to implement in a scalable manner. > >> > >> In reality, the MPI library or the runtime system that supports MPI > >> will already be doing something similar. Even for > >> MPI_ERRORS_ARE_FATAL on MPI_COMM_WORLD, the underlying system > must > >> detect the process failure, and terminate all other processes in > >> MPI_COMM_WORLD. So this represents a -detection- of the failure, and > >> a -notification- of the failure throughout the system (though the > >> notification is an order to terminate). For MPI_ERRORS_RETURN, the > >> MPI will use this detection/notification functionality to reason > >> about the state of the message traffic in the system. So it seems > >> silly to force the user to duplicate this (nontrivial) > >> detection/notification functionality on top of MPI, just to avoid the > progress discussion. > >> > >> > >> So that is a rough summary of the debate. If we are going to move > >> forward, we need to make a decision on MPI_ANY_SOURCE. I would like > >> to make such a decision before/during the next teleconf (Feb. 1). > >> > >> I'm torn on this one, so I look forward to your comments. > >> > >> -- Josh > >> > >> -- > >> Joshua Hursey > >> Postdoctoral Research Associate > >> Oak Ridge National Laboratory > >> http://users.nccs.gov/~jjhursey > >> > > > > > > -- > > Howard Pritchard > > Software Engineering > > Cray, Inc. > > _______________________________________________ > > mpi3-ft mailing list > > mpi3-ft(a)lists.mpi-forum.org > > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-ft > > ______________________________________________________________ > __________ > Martin Schulz, schulzm(a)llnl.gov, http://people.llnl.gov/schulzm CASC @ > Lawrence Livermore National Laboratory, Livermore, USA > > > > > _______________________________________________ > mpi3-ft mailing list > mpi3-ft(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-ft

1 0

Re: [Mpi3-ft] MPI_ANY_SOURCE ... again...
by Josh Hursey 26 Jan '12

26 Jan '12

If we were to change the MPI_ANY_SOURCE semantics to only return when completed/matched (so the suggested proposal modification), would this make life more difficult on an intermediate layer virtualizing the collective operations? I am open to the idea that calling a collective validate() also re-enables the posting of MPI_ANY_SOURCE. The reason we did not do that in the current proposal is because we were uncertain if someone would want to do a validate, but not re-enable the posting of MPI_ANY_SOURCE receives. Since reenabling ANY_SOURCE is a local/quick operation there was not performance argument to be made. Though I think you pose an interesting programability argument. So can you elaborate a bit on your example (I just want to make sure I fully understand)? Would such an intermediate layer be using MPI_ANY_SOURCE in their collective operations and depend on the return-when-new-proc-failure semantic? So the intermediate library would have to reenable ANY_SOURCE, then mask the semantics for user initiated p2p operations over the same communicator. If we added the reenable_any_source semantics with the validate then the intermediate library would not have to virtualize the p2p communication. Is that on the right track? Thanks, Josh On Wed, Jan 25, 2012 at 7:29 PM, Martin Schulz <schulzm(a)llnl.gov> wrote: > Hi Josh, all, > > I agree, I think the current approach is fine. Blocking is likely to be > more problematic in many cases, IMHO. However, I am still a bit worried > about splitting the semantics for P2P and Collective routines. I don't see > a reason why a communicator after a collective call to validate wouldn't > support ANY_SOURCE. If it is split, though, any intermediate layer trying > to replace collectives with P2P solutions (and there are plenty of tuning > frameworks out there that try exactly that) have a hard time maintaing the > same semantics in an error case. > > Martin > > > On Jan 25, 2012, at 3:06 PM, Howard Pritchard wrote: > > > Hi Josh, > > > > Cray is okay with the semantics described in the current > > FTWG proposal attached to the ticket. > > > > We plan to just leverage the out-of-band system fault > > detector software that currently kills jobs if > > a node goes down that the job was running on. > > > > Howard > > > > Josh Hursey wrote: > >> We really need to make a decision on semantics for MPI_ANY_SOURCE. > >> > >> During the plenary session the MPI Forum had a problem with the current > >> proposed semantics. The current proposal states (roughly) that > >> MPI_ANY_SOURCE return when a failure emerges in the communicator. The > >> MPI Forum read this as a strong requirement for -progress- (something > >> the MPI standard tries to stay away from). > >> > >> The alternative proposal is that a receive on MPI_ANY_SOURCE will block > >> until completed with a message. This means that it will -not- return > >> when a new failure has been encountered (even if the calling process is > >> the only process left alive in the communicator). This does get around > >> the concern about progress, but puts a large burden on the end user. > >> > >> > >> There are a couple good use cases for MPI_ANY_SOURCE (grumble, grumble) > >> - Manager/Worker applications, and easy load balancing when > >> multiple incoming messages are expected. This blocking behavior makes > >> the use of MPI_ANY_SOURCE dangerous for fault tolerant applications, and > >> opens up another opportunity for deadlock. > >> > >> For applications that want to use MPI_ANY_SOURCE and be fault tolerant > >> they will need to build their own failure detector on top of MPI using > >> directed point-to-point messages. A basic implementation might post > >> MPI_Irecv()'s to each worker process with an unused tag, then poll on > >> Testany(). If any of these requests complete in error > >> (MPI_ERR_PROC_FAIL_STOP) then the target has failed and the application > >> can take action. This user-level failure detector can (should) be > >> implemented in a third-party library since failure detectors can be > >> difficult to implement in a scalable manner. > >> > >> In reality, the MPI library or the runtime system that supports MPI will > >> already be doing something similar. Even for MPI_ERRORS_ARE_FATAL on > >> MPI_COMM_WORLD, the underlying system must detect the process failure, > >> and terminate all other processes in MPI_COMM_WORLD. So this represents > >> a -detection- of the failure, and a -notification- of the failure > >> throughout the system (though the notification is an order to > >> terminate). For MPI_ERRORS_RETURN, the MPI will use this > >> detection/notification functionality to reason about the state of the > >> message traffic in the system. So it seems silly to force the user to > >> duplicate this (nontrivial) detection/notification functionality on top > >> of MPI, just to avoid the progress discussion. > >> > >> > >> So that is a rough summary of the debate. If we are going to move > >> forward, we need to make a decision on MPI_ANY_SOURCE. I would like to > >> make such a decision before/during the next teleconf (Feb. 1). > >> > >> I'm torn on this one, so I look forward to your comments. > >> > >> -- Josh > >> > >> -- > >> Joshua Hursey > >> Postdoctoral Research Associate > >> Oak Ridge National Laboratory > >> http://users.nccs.gov/~jjhursey > >> > > > > > > -- > > Howard Pritchard > > Software Engineering > > Cray, Inc. > > _______________________________________________ > > mpi3-ft mailing list > > mpi3-ft(a)lists.mpi-forum.org > > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-ft > > ________________________________________________________________________ > Martin Schulz, schulzm(a)llnl.gov, http://people.llnl.gov/schulzm > CASC @ Lawrence Livermore National Laboratory, Livermore, USA > > > > > _______________________________________________ > mpi3-ft mailing list > mpi3-ft(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-ft > > -- Joshua Hursey Postdoctoral Research Associate Oak Ridge National Laboratory http://users.nccs.gov/~jjhursey

1 0

Re: [Mpi3-ft] MPI_ANY_SOURCE ... again...
by Martin Schulz 25 Jan '12

25 Jan '12

Hi Josh, all, I agree, I think the current approach is fine. Blocking is likely to be more problematic in many cases, IMHO. However, I am still a bit worried about splitting the semantics for P2P and Collective routines. I don't see a reason why a communicator after a collective call to validate wouldn't support ANY_SOURCE. If it is split, though, any intermediate layer trying to replace collectives with P2P solutions (and there are plenty of tuning frameworks out there that try exactly that) have a hard time maintaing the same semantics in an error case. Martin On Jan 25, 2012, at 3:06 PM, Howard Pritchard wrote: > Hi Josh, > > Cray is okay with the semantics described in the current > FTWG proposal attached to the ticket. > > We plan to just leverage the out-of-band system fault > detector software that currently kills jobs if > a node goes down that the job was running on. > > Howard > > Josh Hursey wrote: >> We really need to make a decision on semantics for MPI_ANY_SOURCE. >> >> During the plenary session the MPI Forum had a problem with the current >> proposed semantics. The current proposal states (roughly) that >> MPI_ANY_SOURCE return when a failure emerges in the communicator. The >> MPI Forum read this as a strong requirement for -progress- (something >> the MPI standard tries to stay away from). >> >> The alternative proposal is that a receive on MPI_ANY_SOURCE will block >> until completed with a message. This means that it will -not- return >> when a new failure has been encountered (even if the calling process is >> the only process left alive in the communicator). This does get around >> the concern about progress, but puts a large burden on the end user. >> >> >> There are a couple good use cases for MPI_ANY_SOURCE (grumble, grumble) >> - Manager/Worker applications, and easy load balancing when >> multiple incoming messages are expected. This blocking behavior makes >> the use of MPI_ANY_SOURCE dangerous for fault tolerant applications, and >> opens up another opportunity for deadlock. >> >> For applications that want to use MPI_ANY_SOURCE and be fault tolerant >> they will need to build their own failure detector on top of MPI using >> directed point-to-point messages. A basic implementation might post >> MPI_Irecv()'s to each worker process with an unused tag, then poll on >> Testany(). If any of these requests complete in error >> (MPI_ERR_PROC_FAIL_STOP) then the target has failed and the application >> can take action. This user-level failure detector can (should) be >> implemented in a third-party library since failure detectors can be >> difficult to implement in a scalable manner. >> >> In reality, the MPI library or the runtime system that supports MPI will >> already be doing something similar. Even for MPI_ERRORS_ARE_FATAL on >> MPI_COMM_WORLD, the underlying system must detect the process failure, >> and terminate all other processes in MPI_COMM_WORLD. So this represents >> a -detection- of the failure, and a -notification- of the failure >> throughout the system (though the notification is an order to >> terminate). For MPI_ERRORS_RETURN, the MPI will use this >> detection/notification functionality to reason about the state of the >> message traffic in the system. So it seems silly to force the user to >> duplicate this (nontrivial) detection/notification functionality on top >> of MPI, just to avoid the progress discussion. >> >> >> So that is a rough summary of the debate. If we are going to move >> forward, we need to make a decision on MPI_ANY_SOURCE. I would like to >> make such a decision before/during the next teleconf (Feb. 1). >> >> I'm torn on this one, so I look forward to your comments. >> >> -- Josh >> >> -- >> Joshua Hursey >> Postdoctoral Research Associate >> Oak Ridge National Laboratory >> http://users.nccs.gov/~jjhursey >> > > > -- > Howard Pritchard > Software Engineering > Cray, Inc. > _______________________________________________ > mpi3-ft mailing list > mpi3-ft(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-ft ________________________________________________________________________ Martin Schulz, schulzm(a)llnl.gov, http://people.llnl.gov/schulzm CASC @ Lawrence Livermore National Laboratory, Livermore, USA

1 0

Re: [Mpi3-ft] MPI_ANY_SOURCE ... again...
by Howard Pritchard 25 Jan '12

25 Jan '12

Hi Josh, Cray is okay with the semantics described in the current FTWG proposal attached to the ticket. We plan to just leverage the out-of-band system fault detector software that currently kills jobs if a node goes down that the job was running on. Howard Josh Hursey wrote: > We really need to make a decision on semantics for MPI_ANY_SOURCE. > > During the plenary session the MPI Forum had a problem with the current > proposed semantics. The current proposal states (roughly) that > MPI_ANY_SOURCE return when a failure emerges in the communicator. The > MPI Forum read this as a strong requirement for -progress- (something > the MPI standard tries to stay away from). > > The alternative proposal is that a receive on MPI_ANY_SOURCE will block > until completed with a message. This means that it will -not- return > when a new failure has been encountered (even if the calling process is > the only process left alive in the communicator). This does get around > the concern about progress, but puts a large burden on the end user. > > > There are a couple good use cases for MPI_ANY_SOURCE (grumble, grumble) > - Manager/Worker applications, and easy load balancing when > multiple incoming messages are expected. This blocking behavior makes > the use of MPI_ANY_SOURCE dangerous for fault tolerant applications, and > opens up another opportunity for deadlock. > > For applications that want to use MPI_ANY_SOURCE and be fault tolerant > they will need to build their own failure detector on top of MPI using > directed point-to-point messages. A basic implementation might post > MPI_Irecv()'s to each worker process with an unused tag, then poll on > Testany(). If any of these requests complete in error > (MPI_ERR_PROC_FAIL_STOP) then the target has failed and the application > can take action. This user-level failure detector can (should) be > implemented in a third-party library since failure detectors can be > difficult to implement in a scalable manner. > > In reality, the MPI library or the runtime system that supports MPI will > already be doing something similar. Even for MPI_ERRORS_ARE_FATAL on > MPI_COMM_WORLD, the underlying system must detect the process failure, > and terminate all other processes in MPI_COMM_WORLD. So this represents > a -detection- of the failure, and a -notification- of the failure > throughout the system (though the notification is an order to > terminate). For MPI_ERRORS_RETURN, the MPI will use this > detection/notification functionality to reason about the state of the > message traffic in the system. So it seems silly to force the user to > duplicate this (nontrivial) detection/notification functionality on top > of MPI, just to avoid the progress discussion. > > > So that is a rough summary of the debate. If we are going to move > forward, we need to make a decision on MPI_ANY_SOURCE. I would like to > make such a decision before/during the next teleconf (Feb. 1). > > I'm torn on this one, so I look forward to your comments. > > -- Josh > > -- > Joshua Hursey > Postdoctoral Research Associate > Oak Ridge National Laboratory > http://users.nccs.gov/~jjhursey > -- Howard Pritchard Software Engineering Cray, Inc.

1 0

[Mpi3-ft] RTS: Collectives and validate
by Josh Hursey 25 Jan '12

25 Jan '12

It has been proposed that one a process fails in a communicator the posting of new collectives is disallowed, and the user must create a new, dense communicator if they want to have access to collective operations again. This is different than the current proposal where after calling MPI_Comm_validate the posting of new collective operations is allowed over the communicator - even though it has 'blanks' in it. We have strong use cases for being able to call collectives over communicators with "blanks" in them. So this functionality is required. However, it was mentioned that a third-party library might be able to 'fake it' by converting a sparse communicator (exposed to the user) into operations over the dense communicator (required by MPI). The point would be to introduce the 'validate/re-enable collectives' semantic as a separate ticket after the RTS proposal is voted in so as to eliminate the need for such a library. There are some problems with the interposition library solution since, for example, with vector collectives the library would need to do some double buffering to adjust the sparse data buffer provided by the user into a dense data buffer that MPI would require. There are other issues, but I wanted to think through this a bit more before elaborating more on this point. One thing I noted on the call was that (from my inspection of the codebase) FT-MPI uses a dense shadow communicator for many of their collective operations. In Open MPI, we experimented with a different method in which we either worked around the failed processes or created a rebalanced communication tree for collectives over a sparse communicator (we published the results of this initial study). The point is that by exposing the MPI library to the sparse communicator (collective over a communicator with 'blanks') the MPI library has more flexibility in how it manages the collective communication. And as part of that flexibility it does not require all the badness that the interposition library would have to go through to provide the same functionality. So since the additional semantics for collectives are not controversial (IMHO) and there is a strong use case then why would we not try to get it right from the beginning? In short, I have reservations about this proposal, but I wanted more time to work through the implications. I'll need to post more on that tomorrow (hopefully). But, as always, comments are welcome in the meantime. -- Josh -- Joshua Hursey Postdoctoral Research Associate Oak Ridge National Laboratory http://users.nccs.gov/~jjhursey

1 0

Re: [Mpi3-ft] MPI_Comm_validate - What's in a name?
by Graham, Richard L. 25 Jan '12

25 Jan '12

Agreed On Jan 25, 2012, at 5:24 PM, Anthony Skjellum wrote: > I think it is a given that point-to-point should remain allowed on a collective with "holes" :-) > Tony > > ----- Original Message ----- > From: "Josh Hursey" <jjhursey(a)open-mpi.org> > To: "MPI 3.0 Fault Tolerance and Dynamic Process Control working Group" <mpi3-ft(a)lists.mpi-forum.org> > Sent: Wednesday, January 25, 2012 4:09:56 PM > Subject: Re: [Mpi3-ft] MPI_Comm_validate - What's in a name? > > > Yes. I am starting that conversation in another email thread (still working on it). I did not want that discussion to distract here. > > > -- Josh > > > On Wed, Jan 25, 2012 at 5:02 PM, Sur, Sayantan < sayantan.sur(a)intel.com > wrote: > > > > > > > Hi Josh, > > > > I think we also had another discussion that Darius initiated regarding this. We discussed that we will allow p2p communication on communicator with ï¿½holesï¿½, but disallow collectives. Since this seemed to satisfy your use cases. We were thinking about bringing in the validate as an ï¿½add-onï¿½ ticket. Are we still on track to do that? > > > > Thanks. > > > > === > > Sayantan Sur, Ph.D. > > Intel Corp. > > > > > > > From: mpi3-ft-bounces(a)lists.mpi-forum.org [mailto: mpi3-ft-bounces(a)lists.mpi-forum.org ] On Behalf Of Josh Hursey > Sent: Wednesday, January 25, 2012 12:56 PM > To: MPI 3.0 Fault Tolerance and Dynamic Process Control working Group > Subject: [Mpi3-ft] MPI_Comm_validate - What's in a name? > > > > > > On the call today it was suggested that we re-evaluate the name MPI_Comm_validate. It was pointed out that 'validate' seems a bit too close to 'invalid' which is probably not the semantic that we are trying to imply with the name. An alternative is 'check', but that is a bit close to 'checkpoint' so might not be the best either. So we are looking for a good name. > > > > > > We started a similar discussion for reenable_any_source, which might lend some ideas: > > > http://lists.mpi-forum.org/mpi3-ft/2011/12/0931.php > > > > > > The semantic behind MPI_Comm_validate [at the moment] are: > > > (1) A fault tolerant synchronization point returning a consistent value (failed group) at all participating processes > > > (2) Allow for the posting of new collective operations on the communicator (a communicator with potential holes in it) > > > > > > Name suggestions are welcome. > > > > > > -- Josh > > > > > > > -- > Joshua Hursey > Postdoctoral Research Associate > Oak Ridge National Laboratory > http://users.nccs.gov/~jjhursey > _______________________________________________ > mpi3-ft mailing list > mpi3-ft(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-ft > > > > > -- > Joshua Hursey > Postdoctoral Research Associate > Oak Ridge National Laboratory > http://users.nccs.gov/~jjhursey > > _______________________________________________ > mpi3-ft mailing list > mpi3-ft(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-ft > > -- > Anthony Skjellum, PhD > Professor and Chair > Dept. of Computer and Information Sciences > Director, UAB Center for Information Assurance and Joint Forensics Research ("The Center") > University of Alabama at Birmingham > +1-(205)934-8657; FAX: +1- (205)934-5473 > > ___________________________________________ > CONFIDENTIALITY: This e-mail and any attachments are confidential and > may be privileged. If you are not a named recipient, please notify the > sender immediately and do not disclose the contents to another person, > use it for any purpose or store or copy the information in any medium. > > Please consider the environment before printing this e-mail > > _______________________________________________ > mpi3-ft mailing list > mpi3-ft(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-ft

1 0

Re: [Mpi3-ft] MPI_Comm_validate - What's in a name?
by Anthony Skjellum 25 Jan '12

25 Jan '12

sorry, I meant communicator with holes :-) ----- Original Message ----- From: "Anthony Skjellum" <tony(a)cis.uab.edu> To: "MPI 3.0 Fault Tolerance and Dynamic Process Control working Group" <mpi3-ft(a)lists.mpi-forum.org> Sent: Wednesday, January 25, 2012 4:24:32 PM Subject: Re: [Mpi3-ft] MPI_Comm_validate - What's in a name? I think it is a given that point-to-point should remain allowed on a collective with "holes" :-) Tony ----- Original Message ----- From: "Josh Hursey" <jjhursey(a)open-mpi.org> To: "MPI 3.0 Fault Tolerance and Dynamic Process Control working Group" <mpi3-ft(a)lists.mpi-forum.org> Sent: Wednesday, January 25, 2012 4:09:56 PM Subject: Re: [Mpi3-ft] MPI_Comm_validate - What's in a name? Yes. I am starting that conversation in another email thread (still working on it). I did not want that discussion to distract here. -- Josh On Wed, Jan 25, 2012 at 5:02 PM, Sur, Sayantan < sayantan.sur(a)intel.com > wrote: Hi Josh, I think we also had another discussion that Darius initiated regarding this. We discussed that we will allow p2p communication on communicator with ‘holes’, but disallow collectives. Since this seemed to satisfy your use cases. We were thinking about bringing in the validate as an “add-on” ticket. Are we still on track to do that? Thanks. === Sayantan Sur, Ph.D. Intel Corp. From: mpi3-ft-bounces(a)lists.mpi-forum.org [mailto: mpi3-ft-bounces(a)lists.mpi-forum.org ] On Behalf Of Josh Hursey Sent: Wednesday, January 25, 2012 12:56 PM To: MPI 3.0 Fault Tolerance and Dynamic Process Control working Group Subject: [Mpi3-ft] MPI_Comm_validate - What's in a name? On the call today it was suggested that we re-evaluate the name MPI_Comm_validate. It was pointed out that 'validate' seems a bit too close to 'invalid' which is probably not the semantic that we are trying to imply with the name. An alternative is 'check', but that is a bit close to 'checkpoint' so might not be the best either. So we are looking for a good name. We started a similar discussion for reenable_any_source, which might lend some ideas: http://lists.mpi-forum.org/mpi3-ft/2011/12/0931.php The semantic behind MPI_Comm_validate [at the moment] are: (1) A fault tolerant synchronization point returning a consistent value (failed group) at all participating processes (2) Allow for the posting of new collective operations on the communicator (a communicator with potential holes in it) Name suggestions are welcome. -- Josh -- Joshua Hursey Postdoctoral Research Associate Oak Ridge National Laboratory http://users.nccs.gov/~jjhursey _______________________________________________ mpi3-ft mailing list mpi3-ft(a)lists.mpi-forum.org http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-ft -- Joshua Hursey Postdoctoral Research Associate Oak Ridge National Laboratory http://users.nccs.gov/~jjhursey _______________________________________________ mpi3-ft mailing list mpi3-ft(a)lists.mpi-forum.org http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-ft -- Anthony Skjellum, PhD Professor and Chair Dept. of Computer and Information Sciences Director, UAB Center for Information Assurance and Joint Forensics Research ("The Center") University of Alabama at Birmingham +1-(205)934-8657; FAX: +1- (205)934-5473 ___________________________________________ CONFIDENTIALITY: This e-mail and any attachments are confidential and may be privileged. If you are not a named recipient, please notify the sender immediately and do not disclose the contents to another person, use it for any purpose or store or copy the information in any medium. Please consider the environment before printing this e-mail _______________________________________________ mpi3-ft mailing list mpi3-ft(a)lists.mpi-forum.org http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-ft -- Anthony Skjellum, PhD Professor and Chair Dept. of Computer and Information Sciences Director, UAB Center for Information Assurance and Joint Forensics Research ("The Center") University of Alabama at Birmingham +1-(205)934-8657; FAX: +1- (205)934-5473 ___________________________________________ CONFIDENTIALITY: This e-mail and any attachments are confidential and may be privileged. If you are not a named recipient, please notify the sender immediately and do not disclose the contents to another person, use it for any purpose or store or copy the information in any medium. Please consider the environment before printing this e-mail

1 0

Re: [Mpi3-ft] MPI_Comm_validate - What's in a name?
by Anthony Skjellum 25 Jan '12

25 Jan '12

I think it is a given that point-to-point should remain allowed on a collective with "holes" :-) Tony ----- Original Message ----- From: "Josh Hursey" <jjhursey(a)open-mpi.org> To: "MPI 3.0 Fault Tolerance and Dynamic Process Control working Group" <mpi3-ft(a)lists.mpi-forum.org> Sent: Wednesday, January 25, 2012 4:09:56 PM Subject: Re: [Mpi3-ft] MPI_Comm_validate - What's in a name? Yes. I am starting that conversation in another email thread (still working on it). I did not want that discussion to distract here. -- Josh On Wed, Jan 25, 2012 at 5:02 PM, Sur, Sayantan < sayantan.sur(a)intel.com > wrote: Hi Josh, I think we also had another discussion that Darius initiated regarding this. We discussed that we will allow p2p communication on communicator with ‘holes’, but disallow collectives. Since this seemed to satisfy your use cases. We were thinking about bringing in the validate as an “add-on” ticket. Are we still on track to do that? Thanks. === Sayantan Sur, Ph.D. Intel Corp. From: mpi3-ft-bounces(a)lists.mpi-forum.org [mailto: mpi3-ft-bounces(a)lists.mpi-forum.org ] On Behalf Of Josh Hursey Sent: Wednesday, January 25, 2012 12:56 PM To: MPI 3.0 Fault Tolerance and Dynamic Process Control working Group Subject: [Mpi3-ft] MPI_Comm_validate - What's in a name? On the call today it was suggested that we re-evaluate the name MPI_Comm_validate. It was pointed out that 'validate' seems a bit too close to 'invalid' which is probably not the semantic that we are trying to imply with the name. An alternative is 'check', but that is a bit close to 'checkpoint' so might not be the best either. So we are looking for a good name. We started a similar discussion for reenable_any_source, which might lend some ideas: http://lists.mpi-forum.org/mpi3-ft/2011/12/0931.php The semantic behind MPI_Comm_validate [at the moment] are: (1) A fault tolerant synchronization point returning a consistent value (failed group) at all participating processes (2) Allow for the posting of new collective operations on the communicator (a communicator with potential holes in it) Name suggestions are welcome. -- Josh -- Joshua Hursey Postdoctoral Research Associate Oak Ridge National Laboratory http://users.nccs.gov/~jjhursey _______________________________________________ mpi3-ft mailing list mpi3-ft(a)lists.mpi-forum.org http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-ft -- Joshua Hursey Postdoctoral Research Associate Oak Ridge National Laboratory http://users.nccs.gov/~jjhursey _______________________________________________ mpi3-ft mailing list mpi3-ft(a)lists.mpi-forum.org http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-ft -- Anthony Skjellum, PhD Professor and Chair Dept. of Computer and Information Sciences Director, UAB Center for Information Assurance and Joint Forensics Research ("The Center") University of Alabama at Birmingham +1-(205)934-8657; FAX: +1- (205)934-5473 ___________________________________________ CONFIDENTIALITY: This e-mail and any attachments are confidential and may be privileged. If you are not a named recipient, please notify the sender immediately and do not disclose the contents to another person, use it for any purpose or store or copy the information in any medium. Please consider the environment before printing this e-mail

1 0

Re: [Mpi3-ft] MPI_Comm_validate - What's in a name?
by Josh Hursey 25 Jan '12

25 Jan '12

Yes. I am starting that conversation in another email thread (still working on it). I did not want that discussion to distract here. -- Josh On Wed, Jan 25, 2012 at 5:02 PM, Sur, Sayantan <sayantan.sur(a)intel.com>wrote: > Hi Josh,**** > > ** ** > > I think we also had another discussion that Darius initiated regarding > this. We discussed that we will allow p2p communication on communicator > with ‘holes’, but disallow collectives. Since this seemed to satisfy your > use cases. We were thinking about bringing in the validate as an “add-on” > ticket. Are we still on track to do that?**** > > ** ** > > Thanks.**** > > ** ** > > ===**** > > Sayantan Sur, Ph.D.**** > > Intel Corp.**** > > ** ** > > *From:* mpi3-ft-bounces(a)lists.mpi-forum.org [mailto: > mpi3-ft-bounces(a)lists.mpi-forum.org] *On Behalf Of *Josh Hursey > *Sent:* Wednesday, January 25, 2012 12:56 PM > *To:* MPI 3.0 Fault Tolerance and Dynamic Process Control working Group > *Subject:* [Mpi3-ft] MPI_Comm_validate - What's in a name?**** > > ** ** > > On the call today it was suggested that we re-evaluate the name > MPI_Comm_validate. It was pointed out that 'validate' seems a bit too close > to 'invalid' which is probably not the semantic that we are trying to imply > with the name. An alternative is 'check', but that is a bit close to > 'checkpoint' so might not be the best either. So we are looking for a good > name.**** > > ** ** > > We started a similar discussion for reenable_any_source, which might lend > some ideas:**** > > http://lists.mpi-forum.org/mpi3-ft/2011/12/0931.php**** > > ** ** > > The semantic behind MPI_Comm_validate [at the moment] are:**** > > (1) A fault tolerant synchronization point returning a consistent value > (failed group) at all participating processes**** > > (2) Allow for the posting of new collective operations on the > communicator (a communicator with potential holes in it)**** > > ** ** > > Name suggestions are welcome.**** > > ** ** > > -- Josh > > **** > > ** ** > > -- > Joshua Hursey > Postdoctoral Research Associate > Oak Ridge National Laboratory > http://users.nccs.gov/~jjhursey**** > > _______________________________________________ > mpi3-ft mailing list > mpi3-ft(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-ft > -- Joshua Hursey Postdoctoral Research Associate Oak Ridge National Laboratory http://users.nccs.gov/~jjhursey

1 0

Re: [Mpi3-ft] MPI_Comm_validate - What's in a name?
by Pavan Balaji 25 Jan '12

25 Jan '12

On 01/25/2012 04:02 PM, Sur, Sayantan wrote: > I think we also had another discussion that Darius initiated regarding > this. We discussed that we will allow p2p communication on communicator > with ï¿½holesï¿½, but disallow collectives. Since this seemed to satisfy > your use cases. We were thinking about bringing in the validate as an > ï¿½add-onï¿½ ticket. Are we still on track to do that? Just as a note -- the first ticket should work for everything that's not collective on comm, which includes P2P, MPI_Comm_create_group, MPI_Intercomm_create, etc. This will allow users to create a new smaller communicator if needed (or a communicator of the same size by spawning more processes). -- Pavan -- Pavan Balaji http://www.mcs.anl.gov/~balaji

1 0