- mpiwg-io - mailman3.cels.anl.gov

Re: [MPI3-IO] Joint IO/FT Teleconf
by Josh Hursey 19 Apr '12

19 Apr '12

It looks like May 1 from 4-5 pm Eastern works best for everyone that responded. Below are the teleconf details: Date: May 1, 2012 Time: 4 pm Eastern/New York Dial-in information: 877-801-8130 Code: 1044056 -- Josh On Tue, Apr 17, 2012 at 10:44 AM, Josh Hursey <jjhursey(a)open-mpi.org> wrote: > The FT working group would like to have a joint teleconf with the IO > working group to discuss the fault tolerance proposal. > ï¿½- https://svn.mpi-forum.org/trac/mpi-forum-web/ticket/323 > ï¿½- https://svn.mpi-forum.org/trac/mpi-forum-web/ticket/326 > > A Doodle Poll link is below to schedule: > ï¿½- http://www.doodle.com/8iu4nryqiahfa3d9 > > If you are interested in participating in this teleconf, fill out the > poll by 2 pm Eastern on Thursday, April 19. > > Thanks, > Josh > > -- > Joshua Hursey > Postdoctoral Research Associate > Oak Ridge National Laboratory > http://users.nccs.gov/~jjhursey -- Joshua Hursey Postdoctoral Research Associate Oak Ridge National Laboratory http://users.nccs.gov/~jjhursey

1 0

[MPI3-IO] Joint IO/FT Teleconf
by Josh Hursey 17 Apr '12

17 Apr '12

The FT working group would like to have a joint teleconf with the IO working group to discuss the fault tolerance proposal. - https://svn.mpi-forum.org/trac/mpi-forum-web/ticket/323 - https://svn.mpi-forum.org/trac/mpi-forum-web/ticket/326 A Doodle Poll link is below to schedule: - http://www.doodle.com/8iu4nryqiahfa3d9 If you are interested in participating in this teleconf, fill out the poll by 2 pm Eastern on Thursday, April 19. Thanks, Josh -- Joshua Hursey Postdoctoral Research Associate Oak Ridge National Laboratory http://users.nccs.gov/~jjhursey

1 0

[MPI3-IO] Ticket 273
by Mohamad Chaarawi 13 Mar '12

13 Mar '12

Hi All, For those who were not able to attend the Forum meeting in Chicago last week, this is an update to inform you about the status of the nonblocking collective I/O ticket. We decided to add nonblocking version of the collective I/O routines to access data with individual file pointers and explicit offsets (and deprecate their split collective counterpart) but not for shared file pointers. The semantics for nonblocking collective shared file pointers were just too weird to have a majority of the forum agreeing upon. Since we don't really have a use case for them and in order stay in the MPI 3.0 bracket we decided to leave them out of this ticket. The current pdf attached to the ticket is the draft that was read at the last meeting and passed its formal reading. We will vote on this ticket at the next meeting in Japan. Thanks, Mohamad

1 0

Re: [MPI3-IO] Updated draft for ticket 273
by Mohamad Chaarawi 24 Feb '12

24 Feb '12

Hi All, After the comments on and off list about the shared file pointer operations, and specifically about when the shared file pointer will be updated, I'm finding it more compelling now to actually maintain the order in which the shared file pointer is updated. This will make the nonblocking collective and independent shared file pointer operations non local, but in theory, as Edgar pointed out to me, all shared file pointer operations can be non local anyhow. So I took out the Advice to Users that mentions that mixing collective and independent operations is undefined, and added this to the consistency semantics section for nonblocking collective I/O operations: All nonblocking collective I/O calls are local and return immediately, irrespective of the status of other processes. The shared file pointer operations are an exception, because updating the shared file pointer might be a non local operation. I guess this would be the "right" thing to do, and the original intent for the split collective routines as Adam pointed out, despite the contradiction in the standard. The updated draft is attached to the ticket url https://svn.mpi-forum.org/trac/mpi-forum-web/ticket/273 Thanks, Mohamad On 02/15/2012 05:39 PM, Quincey Koziol wrote: > Hi Mohamad, > One small comment about the change to lines 38-43 on p484: this > sentence: "All processes, belonging to the group of the communicator > that was used to open the file, must call collective I/O operations > (blocking and nonblocking) in the same order." doesn't need any > commas. Also, I would probably add the same advise to users for > MPI_FILE_IWRITE_ORDERED as you added for MPI_FILE_IREAD_ORDERED. > > Otherwise, looks good to me, :-) > > Quincey > > On Feb 15, 2012, at 2:37 PM, Mohamad Chaarawi wrote: > >> Hi All, >> >> I attached to the ticket >> (https://svn.mpi-forum.org/trac/mpi-forum-web/ticket/273) the updated >> draft with the suggested ticket 0 changes from the reading at the >> last meeting. >> >> For your convenience, here are the changes made to the text including >> the page number: >> >> * >> p:444 We should make table 13.1 have three rows: blocking, nonblocking& split collective, along with marking the split collectives as deprecated in the table (if possible). (ticket 0) >> * >> p:451 (line 21), 456 (line 40), 463 (line 21) We need to change the 'void *' buffer parameters for the "write" routines to be 'const void *'. >> * >> p: 484 (line 46) "Quality" should be "High Quality". >> * >> p: 484 (line 38-43) The sentence ending with "in between" needs editing to indicate what the collective operations are "between". >> o I rephrased this sentence a little and added a reference to >> the collective semantics. >> * >> p: 484 (line 36) "belonging to the communicator" should be "belonging to the group of the communicator". >> * >> p: 484 (line 48) "All calls" should be changed to "All nonblocking collective I/O calls". >> * >> p: 445 (line 25) The sentence "In a nonblocking or split collective operation, the pointer is updated by the call that initiates the I/O, possibly before the access completes." should have a note about the split collectives being deprecated. >> * >> p"462 (line 28) Add advice to users to indicate the result of intermixing nonblocking collective and independent shared file pointer operations is undefined >> >> Please let me know if you have any comments about the changes. >> >> Thanks, >> Mohamad >> >> _______________________________________________ >> MPI3-IO mailing list >> MPI3-IO(a)lists.mpi-forum.org <mailto:[email protected]> >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-io > > > > _______________________________________________ > MPI3-IO mailing list > MPI3-IO(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-io

1 0

Re: [MPI3-IO] shared file pointer
by Mohamad Chaarawi 24 Feb '12

24 Feb '12

Hi Adam, On 2/23/2012 11:52 PM, Moody, Adam T. wrote: > Hi Mohamad, > One detail we may still need to worry about in the collective case... If I remember correctly, an app is required to initiate collectives in the same order on all procs, but those collectives may complete in any order. If we extend this to non-blocking i/o collectives but we don't specify when the pointer is updated, things get confusing. For example, > > MPI_File_iread_ordered(A) > MPI_File_iread_ordered(B) > MPI_Waitany(A or B) > MPI_Waitany(whatever is left between A and B) > > It's possible that the first waitany call will complete B, not A. If an implementation is allowed to update the pointer in the completion call (the waitany), then B will read the field intended for A. In an all collective case, The pointer is still updated in order of the calls made. The calls CAN complete out of order though, but that is not the issue here I guess.. > > On the other hand, if the standard declares that the pointer must be updated when the operation is initiated (the iread_ordered call), then the correct field is read into its corresponding variable regardless of the order in which the operations are completed. I think this is a nice property to strive for, but it's been long enough now that I forget what the current proposal text says in this case. The current text in the draft says that the pointers are advanced in the order of the operations called (in the initiation call), so you will get what you expect when all the calls are collective.. For the mix case (with shared file pointers), it becomes more complicated, as the non-collective call now depends on the progress of the other processes to be able to update its pointer and make progress, which is why we chose to leave the end result of this case as undefined (the Advice to Users added after the iread/iwrite_ordered operations). We do need to make up our mind about this, whether to keep this case as undefined or not, before Monday, which is the deadline for changing the draft. Thanks, Mohamad > -Adam > > ________________________________________ > From: mpi3-io-bounces(a)lists.mpi-forum.org [mpi3-io-bounces(a)lists.mpi-forum.org] On Behalf Of Mohamad Chaarawi [chaarawi(a)hdfgroup.org] > Sent: Thursday, February 23, 2012 1:48 PM > To: mpi3-io(a)lists.mpi-forum.org > Subject: Re: [MPI3-IO] shared file pointer > > Hi Adam, > > On 2/22/2012 7:25 PM, Adam T. Moody wrote: >> I will say, though, given that we can now have multiple outstanding >> non-blocking collective I/O calls, it would be really nice to be able >> to do the following: >> >> MPI_File_iread_ordered() >> MPI_File_iread_ordered() >> MPI_File_iread_ordered() >> MPI_Waitall() >> >> This provides a natural way for someone to read in three different >> sections from a file -- just issue all the calls and sit back and >> wait. However, this can only be done if the pointer is updated in the >> initiation call. > I don't see how the current proposal would prevent you from doing this.. > The only ordering that we say is undefined is when you mix collective > and independent operations.. > > Thanks, > Mohamad > >> -Adam >> >> >> Adam T. Moody wrote: >> >>> Hi Mohamad and Dries, >>> Yes, I see your point now about "using the corresponding blocking >>> collective routines when ... the end call is issued". I don't think >>> that's what the standard intended, but you're very right in that the >>> text says two different things. Some statements say the pointer is >>> updated by the call that initiates the operation, i.e., the _begin >>> call, but this says the opposite in that an implementation is allowed >>> to do all the work (including updating of the pointer) in the _end >>> call. Thus, it's not clear whether the pointer will always be >>> updated after returning from the _begin call. >>> -Adam >>> >>> >>> Mohamad Chaarawi wrote: >>> >>> >>> >>>> Hi Adam, >>>> >>>> >>>> >>>> >>>>> This statement says that an app can't know whether the begin call >>>>> will synchronize or not, so a portable app must assume that the >>>>> call does synchronize. However, the earlier statements say that >>>>> regardless of whether the MPI library implements the begin call as >>>>> blocking or non-blocking, the app is always guaranteed that the >>>>> shared file pointer will be updated upon return from the begin call. >>>>> >>>>> >>>> Yes but I agree with Dries that there is a contradiction, and it can >>>> be interpreted by a developer either way, i.e. the pointer can be >>>> either updated in the begin or end call.. >>>> >>>> >>>> >>>> >>>>> With split collectives, the "begin" call that initiates the >>>>> operation *can* block, but with non-blocking collectives (as >>>>> currently defined), the "i" call that initiates the operation >>>>> *never* blocks. It's this difference between split collectives and >>>>> non-blocking collectives that causes the difficulty here. To >>>>> efficiently meet the requirements of updating the shared file >>>>> pointer, we'd really like to update the pointer during the "i" >>>>> call, but this would require the "i" call to block. >>>>> >>>>> >>>> I do not have a strong opinion here, as we don't really use this >>>> feature.. But I can see how this could complicate things more to the >>>> user and the developer, which makes me more inclined to keep the >>>> ordering undefined. >>>> That said, we do want to start working on a ticket for new MPI-I/O >>>> features that would actually track order inside the implementation >>>> for nonblocking file access and manipulation routines (more like >>>> queuing).. We discussed that at the last Chicago meeting.. This is >>>> not MPI-3.0 bound though :) >>>> >>>> Thanks, >>>> Mohamad >>>> >>>> >>>> _______________________________________________ >>>> MPI3-IO mailing list >>>> MPI3-IO(a)lists.mpi-forum.org >>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-io >>>> >>>> >>>> >>> >>> >>> >> _______________________________________________ >> MPI3-IO mailing list >> MPI3-IO(a)lists.mpi-forum.org >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-io > _______________________________________________ > MPI3-IO mailing list > MPI3-IO(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-io > > _______________________________________________ > MPI3-IO mailing list > MPI3-IO(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-io

1 0

Re: [MPI3-IO] shared file pointer
by Moody, Adam T. 23 Feb '12

23 Feb '12

Hi Mohamad, One detail we may still need to worry about in the collective case... If I remember correctly, an app is required to initiate collectives in the same order on all procs, but those collectives may complete in any order. If we extend this to non-blocking i/o collectives but we don't specify when the pointer is updated, things get confusing. For example, MPI_File_iread_ordered(A) MPI_File_iread_ordered(B) MPI_Waitany(A or B) MPI_Waitany(whatever is left between A and B) It's possible that the first waitany call will complete B, not A. If an implementation is allowed to update the pointer in the completion call (the waitany), then B will read the field intended for A. On the other hand, if the standard declares that the pointer must be updated when the operation is initiated (the iread_ordered call), then the correct field is read into its corresponding variable regardless of the order in which the operations are completed. I think this is a nice property to strive for, but it's been long enough now that I forget what the current proposal text says in this case. -Adam ________________________________________ From: mpi3-io-bounces(a)lists.mpi-forum.org [mpi3-io-bounces(a)lists.mpi-forum.org] On Behalf Of Mohamad Chaarawi [chaarawi(a)hdfgroup.org] Sent: Thursday, February 23, 2012 1:48 PM To: mpi3-io(a)lists.mpi-forum.org Subject: Re: [MPI3-IO] shared file pointer Hi Adam, On 2/22/2012 7:25 PM, Adam T. Moody wrote: > I will say, though, given that we can now have multiple outstanding > non-blocking collective I/O calls, it would be really nice to be able > to do the following: > > MPI_File_iread_ordered() > MPI_File_iread_ordered() > MPI_File_iread_ordered() > MPI_Waitall() > > This provides a natural way for someone to read in three different > sections from a file -- just issue all the calls and sit back and > wait. However, this can only be done if the pointer is updated in the > initiation call. I don't see how the current proposal would prevent you from doing this.. The only ordering that we say is undefined is when you mix collective and independent operations.. Thanks, Mohamad > -Adam > > > Adam T. Moody wrote: > >> Hi Mohamad and Dries, >> Yes, I see your point now about "using the corresponding blocking >> collective routines when ... the end call is issued". I don't think >> that's what the standard intended, but you're very right in that the >> text says two different things. Some statements say the pointer is >> updated by the call that initiates the operation, i.e., the _begin >> call, but this says the opposite in that an implementation is allowed >> to do all the work (including updating of the pointer) in the _end >> call. Thus, it's not clear whether the pointer will always be >> updated after returning from the _begin call. >> -Adam >> >> >> Mohamad Chaarawi wrote: >> >> >> >>> Hi Adam, >>> >>> >>> >>> >>>> This statement says that an app can't know whether the begin call >>>> will synchronize or not, so a portable app must assume that the >>>> call does synchronize. However, the earlier statements say that >>>> regardless of whether the MPI library implements the begin call as >>>> blocking or non-blocking, the app is always guaranteed that the >>>> shared file pointer will be updated upon return from the begin call. >>>> >>>> >>> Yes but I agree with Dries that there is a contradiction, and it can >>> be interpreted by a developer either way, i.e. the pointer can be >>> either updated in the begin or end call.. >>> >>> >>> >>> >>>> With split collectives, the "begin" call that initiates the >>>> operation *can* block, but with non-blocking collectives (as >>>> currently defined), the "i" call that initiates the operation >>>> *never* blocks. It's this difference between split collectives and >>>> non-blocking collectives that causes the difficulty here. To >>>> efficiently meet the requirements of updating the shared file >>>> pointer, we'd really like to update the pointer during the "i" >>>> call, but this would require the "i" call to block. >>>> >>>> >>> I do not have a strong opinion here, as we don't really use this >>> feature.. But I can see how this could complicate things more to the >>> user and the developer, which makes me more inclined to keep the >>> ordering undefined. >>> That said, we do want to start working on a ticket for new MPI-I/O >>> features that would actually track order inside the implementation >>> for nonblocking file access and manipulation routines (more like >>> queuing).. We discussed that at the last Chicago meeting.. This is >>> not MPI-3.0 bound though :) >>> >>> Thanks, >>> Mohamad >>> >>> >>> _______________________________________________ >>> MPI3-IO mailing list >>> MPI3-IO(a)lists.mpi-forum.org >>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-io >>> >>> >>> >> >> >> >> > > _______________________________________________ > MPI3-IO mailing list > MPI3-IO(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-io _______________________________________________ MPI3-IO mailing list MPI3-IO(a)lists.mpi-forum.org http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-io

1 0

Re: [MPI3-IO] [Mpi3-ft] Fault Tolerance & I/O Discussion
by George Bosilca 23 Feb '12

23 Feb '12

Due to the member-only policy on the I/O list, my previous attempt to send the new FT proposal bounced. This is a re-sent of the same message now that I subscribed. Thanks, george. > Based on the discussion and opinions expressed during this call, the new FT proposal has been extended to include wording about the I/O. The model for I/O is very synchronous, providing a single function related to I/O. This function can be used to invalidate a file, preempting any subsequent operation on it. The file will then have to be closed, and is necessary reopened based on a new communicator (a smaller version of the original one). > > The current version if the FT proposal is attached to this email, and can be accessed in the working group wiki @ https://svn.mpi-forum.org/trac/mpi-forum-web/wiki/User_Level_Failure_Mitiga…. > > We expect to have a final version of the proposal by Monday noon EST, as requested by the MPI_Forum to be able to have a first reading at the next meeting. We are looking forward to your comments. > > george.

1 0

Re: [MPI3-IO] shared file pointer
by Mohamad Chaarawi 23 Feb '12

23 Feb '12

Hi Edgar, On 2/23/2012 3:45 PM, Edgar Gabriel wrote: > I would agree, I would like to add one more comment/question: how does > the code sequence listed above compare to > > MPI_File_iread_shared() > MPI_File_iread_shared() > MPI_File_iread_shared() > MPI_Waitall() > > executed by each process? This is according to my understanding legal in > MPI-2. Yes this is legal in and the implementation should maintain the sequential ordering of the updates to the shared file pointer. > Wouldn't it be difficult to explain to users that the guarantees > given for this scenario are fundamentally different than when using > MPI_File_iread_ordered? This is still fine if we do not mix independent and collective operations. If we do something like this in MPI-2: MPI_File_read_ordered_begin() MPI_File_iread_shared() MPI_File_read_ordered_end() The current text does not imply that the implementation has to update the pointer in the begin call, which means that the order of updating the shared file pointer is not defined. Our proposal does not change this, meaning that if we do something like: MPI_File_iread_ordered() MPI_File_iread_shared() MPI_Waitall() The order in which the file pointer is updated is undefined.. Again, I'm not really against omitting the Advice to Users in the current proposal that says that the order is undefined in this case, if there is a really strong opinion that we should make the bookkeeping order defined when mixing collective and independent operations. Thanks, Mohamad > > Thanks > Edgar

1 0

Re: [MPI3-IO] shared file pointer
by Mohamad Chaarawi 23 Feb '12

23 Feb '12

Hi Adam, On 2/22/2012 7:25 PM, Adam T. Moody wrote: > I will say, though, given that we can now have multiple outstanding > non-blocking collective I/O calls, it would be really nice to be able > to do the following: > > MPI_File_iread_ordered() > MPI_File_iread_ordered() > MPI_File_iread_ordered() > MPI_Waitall() > > This provides a natural way for someone to read in three different > sections from a file -- just issue all the calls and sit back and > wait. However, this can only be done if the pointer is updated in the > initiation call. I don't see how the current proposal would prevent you from doing this.. The only ordering that we say is undefined is when you mix collective and independent operations.. Thanks, Mohamad > -Adam > > > Adam T. Moody wrote: > >> Hi Mohamad and Dries, >> Yes, I see your point now about "using the corresponding blocking >> collective routines when ... the end call is issued". I don't think >> that's what the standard intended, but you're very right in that the >> text says two different things. Some statements say the pointer is >> updated by the call that initiates the operation, i.e., the _begin >> call, but this says the opposite in that an implementation is allowed >> to do all the work (including updating of the pointer) in the _end >> call. Thus, it's not clear whether the pointer will always be >> updated after returning from the _begin call. >> -Adam >> >> >> Mohamad Chaarawi wrote: >> >> >> >>> Hi Adam, >>> >>> >>> >>> >>>> This statement says that an app can't know whether the begin call >>>> will synchronize or not, so a portable app must assume that the >>>> call does synchronize. However, the earlier statements say that >>>> regardless of whether the MPI library implements the begin call as >>>> blocking or non-blocking, the app is always guaranteed that the >>>> shared file pointer will be updated upon return from the begin call. >>>> >>>> >>> Yes but I agree with Dries that there is a contradiction, and it can >>> be interpreted by a developer either way, i.e. the pointer can be >>> either updated in the begin or end call.. >>> >>> >>> >>> >>>> With split collectives, the "begin" call that initiates the >>>> operation *can* block, but with non-blocking collectives (as >>>> currently defined), the "i" call that initiates the operation >>>> *never* blocks. It's this difference between split collectives and >>>> non-blocking collectives that causes the difficulty here. To >>>> efficiently meet the requirements of updating the shared file >>>> pointer, we'd really like to update the pointer during the "i" >>>> call, but this would require the "i" call to block. >>>> >>>> >>> I do not have a strong opinion here, as we don't really use this >>> feature.. But I can see how this could complicate things more to the >>> user and the developer, which makes me more inclined to keep the >>> ordering undefined. >>> That said, we do want to start working on a ticket for new MPI-I/O >>> features that would actually track order inside the implementation >>> for nonblocking file access and manipulation routines (more like >>> queuing).. We discussed that at the last Chicago meeting.. This is >>> not MPI-3.0 bound though :) >>> >>> Thanks, >>> Mohamad >>> >>> >>> _______________________________________________ >>> MPI3-IO mailing list >>> MPI3-IO(a)lists.mpi-forum.org >>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-io >>> >>> >>> >> >> >> >> > > _______________________________________________ > MPI3-IO mailing list > MPI3-IO(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-io

1 0

Re: [MPI3-IO] shared file pointer
by Edgar Gabriel 23 Feb '12

23 Feb '12

On 2/23/2012 8:49 AM, Rob Latham wrote: > On Wed, Feb 22, 2012 at 05:25:04PM -0800, Adam T. Moody wrote: >> I will say, though, given that we can now have multiple outstanding >> non-blocking collective I/O calls, it would be really nice to be >> able to do the following: > >> >> MPI_File_iread_ordered() >> MPI_File_iread_ordered() >> MPI_File_iread_ordered() >> MPI_Waitall() >> >> This provides a natural way for someone to read in three different >> sections from a file -- just issue all the calls and sit back and >> wait. However, this can only be done if the pointer is updated in >> the initiation call. > > Now we're getting somewhere. Setting aside for a moment what the > letter of MPI-2.2 says, Adam's use case is exactly in line with the > spirit of shared file pointers. I would agree, I would like to add one more comment/question: how does the code sequence listed above compare to MPI_File_iread_shared() MPI_File_iread_shared() MPI_File_iread_shared() MPI_Waitall() executed by each process? This is according to my understanding legal in MPI-2. Wouldn't it be difficult to explain to users that the guarantees given for this scenario are fundamentally different than when using MPI_File_iread_ordered? Thanks Edgar > The I/O can happen whenever but the bookeeping -- updating the shared > file pointer -- happens up front. > > ==rob >

1 0