April 2013 - mpiwg-ft - mailman3.cels.anl.gov

Re: [Mpi3-ft] Updated Chapter Document
by Sur, Sayantan 29 Apr '13

29 Apr '13

Thanks, Wesley! Sayantan From: mpi3-ft-bounces(a)lists.mpi-forum.org [mailto:[email protected]] On Behalf Of Wesley Bland Sent: Monday, April 29, 2013 12:12 PM To: MPI3-FT Working Group Subject: [Mpi3-ft] Updated Chapter Document Per the request from Sayantan on the last call, I've updated the ULFM wiki (https://svn.mpi-forum.org/trac/mpi-forum-web/wiki/User_Level_Failure_Mitiga…) page on the forum wiki to contain the newest text. Hopefully we'll be able to keep the text in the wiki up to date, but most likely it will get stale again as we make more small tweaks. However, there is also a PDF attachment on the page that will be easier to keep current. Thanks, Wesley

1 0

[Mpi3-ft] Updated Chapter Document
by Wesley Bland 29 Apr '13

29 Apr '13

Per the request from Sayantan on the last call, I've updated the ULFM wiki (https://svn.mpi-forum.org/trac/mpi-forum-web/wiki/User_Level_Failure_Mitiga…) page on the forum wiki to contain the newest text. Hopefully we'll be able to keep the text in the wiki up to date, but most likely it will get stale again as we make more small tweaks. However, there is also a PDF attachment on the page that will be easier to keep current. Thanks, Wesley

1 0

Re: [Mpi3-ft] FTWG meeting: supplementary document
by Aurélien Bouteiller 17 Apr '13

17 Apr '13

Here are the minutes from today's meeting. Attendees: OSU, Livermore, UTK, ORNL, ANL * ULFM function coverage: * Some RMA and File functions are not completed yet. * Better define RMA as 1-sided rather than non local: it is not clear to RMA people if one-sided should be allowed (or not) to block (example put after a lock). Separating terms gives flexibility. * RMA * first paragraph: major change is that put, etc may return failures immediately (not only in the following sync). This behavior is optional and left to the implementation, in any case the following sync MUST return an error. * The text is implicit, it should state this in clear. * It does not seem to make sense to extend this behavior to comm fn, because somewhere it is already defined that the error is reported in wait/test. Check? * we do not force consistent error reporting on puts. This does not matter because error reporting should only be consistent with regard to completions, that is, the following sync. * Locks: point 2 is unclear. The removal of the specification behavior for "non target process failure" is a problem. My personal take is that we should phrase that "failure of a process that has tried to acquire the lock in the current epoch should raise an error when acquiring the same lock", or something similar. We should not try to "know" which process does hold the lock at a given moment (it is difficult in fact), and simply consider that the failure of anybody that -may- hold the lock raise an error. * free: use the "high quality" wording with regard to local cleanup. * the same wording must appear in file_free * Todo: include the new ops in items 1, 2 for completion of passive targets etc. Todo: update the wiki with the latest version of the document, trim out the old ones. Hear you in 2 weeks, Aurelien Le 17 avr. 2013 ï¿½ 11:34, Wesley Bland <wbland(a)mcs.anl.gov> a ï¿½crit : > It should be mentioned that most of the functions related to RMA and File I/O have not been classified as they might require feedback from the appropriate people. We also still need to write the definitions of each of the classifications of functions. > > On Apr 17, 2013, at 9:06 AM, Aurï¿½lien Bouteiller <bouteill(a)icl.utk.edu> wrote: > >> All, >> >> Wesley and I made a first pass on the systematic investigation of compatibility of all MPI functions with ULFM. Here is the first draft of the finding. We found no real problem functions, 3 functions may need clarification but are properly covered. >> >> Please take a look and make sure we made no mistake in the classification >> >> https://docs.google.com/document/d/1Ir1zCZP2ZwG1NIO8DEStQGGyQHRe8NjNZXyLolS… >> >> >> >> Dï¿½but du message rï¿½expï¿½diï¿½ : >> >>> De : Aurï¿½lien Bouteiller <bouteill(a)icl.utk.edu> >>> Objet : FTWG meeting reminder >>> Date : 17 avril 2013 09:29:48 UTC-04:00 >>> ï¿½ : "MPI 3.0 Fault Tolerance and Dynamic Process Control working Group" <mpi3-ft(a)lists.mpi-forum.org> >>> >>> Dear WG members, >>> >>> This is a reminder that according to our planning, we are having our regular phone meeting today at noon EDT. >>> >>> Agenda: >>> - followup on RMA work (new interface) >>> - Preliminary findings on coverage of "all" functions by ULFM >>> >>> >>> >>> Date: Apr. 17, 2013 >>> Time: Noon EDT/New York >>> Dial-in information: 218-339-4600 >>> Code: 623998# >>> >>> >>> Next Meetings: >>> * May. 1, 2013 >>> * May. 15, 2013 >>> >>> Please contact me if you want to add items to the agenda on these meetings. >>> >>> -- >>> * Dr. Aurï¿½lien Bouteiller >>> * Researcher at Innovative Computing Laboratory >>> * University of Tennessee >>> * 1122 Volunteer Boulevard, suite 309b >>> * Knoxville, TN 37996 >>> * 865 974 9375 >>> >>> >>> >>> >>> >>> >>> >> >> -- >> * Dr. Aurï¿½lien Bouteiller >> * Researcher at Innovative Computing Laboratory >> * University of Tennessee >> * 1122 Volunteer Boulevard, suite 309b >> * Knoxville, TN 37996 >> * 865 974 9375 >> >> >> >> >> >> >> >> >> _______________________________________________ >> mpi3-ft mailing list >> mpi3-ft(a)lists.mpi-forum.org >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-ft > > > _______________________________________________ > mpi3-ft mailing list > mpi3-ft(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-ft -- * Dr. Aurï¿½lien Bouteiller * Researcher at Innovative Computing Laboratory * University of Tennessee * 1122 Volunteer Boulevard, suite 309b * Knoxville, TN 37996 * 865 974 9375

1 0

Re: [Mpi3-ft] MPI RMA FT Notes
by Wesley Bland 17 Apr '13

17 Apr '13

Jim and I have drafted some text based on these notes. Rather than push the entire PDF back to the mailing list, I've pulled out the relevant page and attached it here with diff highlighting. Feel free to take a look before the call to give feedback. I'm sure that some points raised by Aurelien in his email an hour ago will probably require some more modification. Thanks, Wesley On Apr 4, 2013, at 9:03 AM, Jim Dinan <dinan(a)mcs.anl.gov> wrote: > Hello, > > Here are my notes on open issues to investigate in FT for MPI RMA: > > * Separate/Unified memory models -- Is there any difference in which data is corrupted by a failure between these two models? > > * MPI_Win_allocate() allocates the window buffer. Do failures have any impact on how this buffer is managed? Can it be exposed in another window after a failure occurs? > > * MPI_Win_allocate_shared() similar to win_allocate(). > > * What are the semantics of MPI_Win_free() on a revoked window? It is normally synchronizing (to quiesce passive target communication). Is it a local operation when the window has been revoked? > > * Because of the difficulty in managing a distributed lock, it has been proposed that all synchronizations that occur after a failure on a window should fail. Investigate if this semantic is too restrictive. > > ~Jim. > _______________________________________________ > mpi3-ft mailing list > mpi3-ft(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-ft

1 0

Re: [Mpi3-ft] FTWG meeting: supplementary document
by Wesley Bland 17 Apr '13

17 Apr '13

It should be mentioned that most of the functions related to RMA and File I/O have not been classified as they might require feedback from the appropriate people. We also still need to write the definitions of each of the classifications of functions. On Apr 17, 2013, at 9:06 AM, Aurï¿½lien Bouteiller <bouteill(a)icl.utk.edu> wrote: > All, > > Wesley and I made a first pass on the systematic investigation of compatibility of all MPI functions with ULFM. Here is the first draft of the finding. We found no real problem functions, 3 functions may need clarification but are properly covered. > > Please take a look and make sure we made no mistake in the classification > > https://docs.google.com/document/d/1Ir1zCZP2ZwG1NIO8DEStQGGyQHRe8NjNZXyLolS… > > > > Dï¿½but du message rï¿½expï¿½diï¿½ : > >> De : Aurï¿½lien Bouteiller <bouteill(a)icl.utk.edu> >> Objet : FTWG meeting reminder >> Date : 17 avril 2013 09:29:48 UTC-04:00 >> ï¿½ : "MPI 3.0 Fault Tolerance and Dynamic Process Control working Group" <mpi3-ft(a)lists.mpi-forum.org> >> >> Dear WG members, >> >> This is a reminder that according to our planning, we are having our regular phone meeting today at noon EDT. >> >> Agenda: >> - followup on RMA work (new interface) >> - Preliminary findings on coverage of "all" functions by ULFM >> >> >> >> Date: Apr. 17, 2013 >> Time: Noon EDT/New York >> Dial-in information: 218-339-4600 >> Code: 623998# >> >> >> Next Meetings: >> * May. 1, 2013 >> * May. 15, 2013 >> >> Please contact me if you want to add items to the agenda on these meetings. >> >> -- >> * Dr. Aurï¿½lien Bouteiller >> * Researcher at Innovative Computing Laboratory >> * University of Tennessee >> * 1122 Volunteer Boulevard, suite 309b >> * Knoxville, TN 37996 >> * 865 974 9375 >> >> >> >> >> >> >> > > -- > * Dr. Aurï¿½lien Bouteiller > * Researcher at Innovative Computing Laboratory > * University of Tennessee > * 1122 Volunteer Boulevard, suite 309b > * Knoxville, TN 37996 > * 865 974 9375 > > > > > > > > > _______________________________________________ > mpi3-ft mailing list > mpi3-ft(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-ft

1 0

[Mpi3-ft] FTWG meeting: supplementary document
by Aurélien Bouteiller 17 Apr '13

17 Apr '13

All, Wesley and I made a first pass on the systematic investigation of compatibility of all MPI functions with ULFM. Here is the first draft of the finding. We found no real problem functions, 3 functions may need clarification but are properly covered. Please take a look and make sure we made no mistake in the classification https://docs.google.com/document/d/1Ir1zCZP2ZwG1NIO8DEStQGGyQHRe8NjNZXyLolS… Dï¿½but du message rï¿½expï¿½diï¿½ : > De : Aurï¿½lien Bouteiller <bouteill(a)icl.utk.edu> > Objet : FTWG meeting reminder > Date : 17 avril 2013 09:29:48 UTC-04:00 > ï¿½ : "MPI 3.0 Fault Tolerance and Dynamic Process Control working Group" <mpi3-ft(a)lists.mpi-forum.org> > > Dear WG members, > > This is a reminder that according to our planning, we are having our regular phone meeting today at noon EDT. > > Agenda: > - followup on RMA work (new interface) > - Preliminary findings on coverage of "all" functions by ULFM > > > > Date: Apr. 17, 2013 > Time: Noon EDT/New York > Dial-in information: 218-339-4600 > Code: 623998# > > > Next Meetings: > * May. 1, 2013 > * May. 15, 2013 > > Please contact me if you want to add items to the agenda on these meetings. > > -- > * Dr. Aurï¿½lien Bouteiller > * Researcher at Innovative Computing Laboratory > * University of Tennessee > * 1122 Volunteer Boulevard, suite 309b > * Knoxville, TN 37996 > * 865 974 9375 > > > > > > > -- * Dr. Aurï¿½lien Bouteiller * Researcher at Innovative Computing Laboratory * University of Tennessee * 1122 Volunteer Boulevard, suite 309b * Knoxville, TN 37996 * 865 974 9375

1 0

Re: [Mpi3-ft] MPI RMA FT Notes
by Aurélien Bouteiller 17 Apr '13

17 Apr '13

Here are my personal notes on the same issues as discussed by Jim. * What if window is on the NIC (with Win_Separate) ? * Win_allocate: should we destroy the associated allocated memory when revoking the window? * Answer: Revoke doesn't change the state of object. The window should certainly not be deallocated. The extent of the area left "undefined" may be larger than the regular definition (needs to be clarified). * Win_free: Well defined, similar to comm_free, no need to change (but the according text needs to be amended) * Lock: if the process that currently holds the lock dies, there is a possibility that all other processes stall on the lock * the most common distributed implementation is MCS * tail pointer stored on target, updated with 1-sided atomic_cas * use tail to point to myself * overall, the global state forms a distributed linked list * We think there is a practical implementation that is FT for this algorithm (however, it is to be noted that we do not want or plan to enforce a particular algorithm, but knowing that it is a solvable problem is good). * Other proposition is that lock failures are fatal and definitive (that is all ops will raise an error proc-failed as soon as some lock is used on a window where known failure exist). The only option is then to revoke the window (restrictive for users, but simpler on implementors). Aurelien Le 4 avr. 2013 ï¿½ 10:03, Jim Dinan <dinan(a)mcs.anl.gov> a ï¿½crit : > Hello, > > Here are my notes on open issues to investigate in FT for MPI RMA: > > * Separate/Unified memory models -- Is there any difference in which data is corrupted by a failure between these two models? > > * MPI_Win_allocate() allocates the window buffer. Do failures have any impact on how this buffer is managed? Can it be exposed in another window after a failure occurs? > > * MPI_Win_allocate_shared() similar to win_allocate(). > > * What are the semantics of MPI_Win_free() on a revoked window? It is normally synchronizing (to quiesce passive target communication). Is it a local operation when the window has been revoked? > > * Because of the difficulty in managing a distributed lock, it has been proposed that all synchronizations that occur after a failure on a window should fail. Investigate if this semantic is too restrictive. > > ~Jim. > _______________________________________________ > mpi3-ft mailing list > mpi3-ft(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-ft -- * Dr. Aurï¿½lien Bouteiller * Researcher at Innovative Computing Laboratory * University of Tennessee * 1122 Volunteer Boulevard, suite 309b * Knoxville, TN 37996 * 865 974 9375

1 0

[Mpi3-ft] FTWG meeting reminder
by Aurélien Bouteiller 17 Apr '13

17 Apr '13

Dear WG members, This is a reminder that according to our planning, we are having our regular phone meeting today at noon EDT. Agenda: - followup on RMA work (new interface) - Preliminary findings on coverage of "all" functions by ULFM Date: Apr. 17, 2013 Time: Noon EDT/New York Dial-in information: 218-339-4600 Code: 623998# Next Meetings: * May. 1, 2013 * May. 15, 2013 Please contact me if you want to add items to the agenda on these meetings. -- * Dr. Aurï¿½lien Bouteiller * Researcher at Innovative Computing Laboratory * University of Tennessee * 1122 Volunteer Boulevard, suite 309b * Knoxville, TN 37996 * 865 974 9375

1 0

[Mpi3-ft] MPI RMA FT Notes
by Jim Dinan 04 Apr '13

04 Apr '13

Hello, Here are my notes on open issues to investigate in FT for MPI RMA: * Separate/Unified memory models -- Is there any difference in which data is corrupted by a failure between these two models? * MPI_Win_allocate() allocates the window buffer. Do failures have any impact on how this buffer is managed? Can it be exposed in another window after a failure occurs? * MPI_Win_allocate_shared() similar to win_allocate(). * What are the semantics of MPI_Win_free() on a revoked window? It is normally synchronizing (to quiesce passive target communication). Is it a local operation when the window has been revoked? * Because of the difficulty in managing a distributed lock, it has been proposed that all synchronizations that occur after a failure on a window should fail. Investigate if this semantic is too restrictive. ~Jim.

1 0

Re: [Mpi3-ft] FTWG phone meeting
by David Solt 03 Apr '13

03 Apr '13

Hi all, I haven't been able to attend the WG meetings for a while now due to a standing conflict at this time. I just want to let you know that it is not due to a change in interest in FT :). If anything critical comes up, I can probably get out of my other appointment for a week and I will join the FT WG call that week. Thanks, Dave From: Aurélien Bouteiller <bouteill(a)icl.utk.edu> To: "MPI 3.0 Fault Tolerance and Dynamic Process Control working Group" <mpi3-ft(a)lists.mpi-forum.org>, Date: 04/03/2013 04:48 AM Subject: [Mpi3-ft] FTWG phone meeting Sent by: mpi3-ft-bounces(a)lists.mpi-forum.org Dear WG members, This is a reminder that according to our planning, we are having our regular phone meeting. Agenda: - followup on RMA work (new interface) - summary on ULFM ompi prototype bug fixes Date: Mar. 20, 2013 Time: Noon EDT/New York Dial-in information: 218-339-4600 Code: 623998# Next Meetings: * Apr. 17, 2013 * May. 1, 2013 Please contact me if you want to add items to the agenda on these meetings. -- * Dr. Aurélien Bouteiller * Researcher at Innovative Computing Laboratory * University of Tennessee * 1122 Volunteer Boulevard, suite 309b * Knoxville, TN 37996 * 865 974 9375 _______________________________________________ mpi3-ft mailing list mpi3-ft(a)lists.mpi-forum.org http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-ft

1 0