June 2013 - mpiwg-rma - mailman3.cels.anl.gov

Re: [Mpi3-rma] [EXTERNAL] Re: request-based ops
by Pavan Balaji 21 Jun '13

21 Jun '13

Sorry, I'm joining in a bit late in the discussion. Not allowing operations between the 'I' variant of the function and the completion call should be OK. If we later want to change that, it can easily be done in a backward compatible way. Also, if someone really want to bypass that restriction, he/she can create overlapping windows. -- Pavan On 06/20/2013 09:49 AM, Barrett, Brian W wrote: > On 6/20/13 7:39 AM, "Jim Dinan" <james.dinan(a)gmail.com > <mailto:[email protected]>> wrote: > > I agree with the general statement that the nonblocking call marks > the end of the epoch. In the case of MPI_Win_test, it's trying to > close an exposure epoch -- not an access epoch -- so communication > operations from the origin process are not being completed by > synchronization (although, the test/wait synchronization could be > waiting for a self communication, that operation is completed by > start/complete not post/wait). I think an MPI_Win_iwait operation > that uses the semantic you described should be identical to > MPI_Win_test. Unless I'm missing something, which never happens > when discussing RMA. :) > > > You're right; I always get the exposure / access sides of generalized > active target mixed up. With that clarification, I think that > MPI_Win_iwait and MPI_Win_test could be made equivalent fairly easily. > > Brian > > -- > Brian W. Barrett > Scalable System Software Group > Sandia National Laboratories > > > _______________________________________________ > mpi3-rma mailing list > mpi3-rma(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma > -- Pavan Balaji http://www.mcs.anl.gov/~balaji

1 0

Re: [Mpi3-rma] [EXTERNAL] Re: request-based ops
by Barrett, Brian W 20 Jun '13

20 Jun '13

On 6/20/13 7:39 AM, "Jim Dinan" <james.dinan(a)gmail.com<mailto:[email protected]>> wrote: I agree with the general statement that the nonblocking call marks the end of the epoch. In the case of MPI_Win_test, it's trying to close an exposure epoch -- not an access epoch -- so communication operations from the origin process are not being completed by synchronization (although, the test/wait synchronization could be waiting for a self communication, that operation is completed by start/complete not post/wait). I think an MPI_Win_iwait operation that uses the semantic you described should be identical to MPI_Win_test. Unless I'm missing something, which never happens when discussing RMA. :) You're right; I always get the exposure / access sides of generalized active target mixed up. With that clarification, I think that MPI_Win_iwait and MPI_Win_test could be made equivalent fairly easily. Brian -- Brian W. Barrett Scalable System Software Group Sandia National Laboratories

1 0

Re: [Mpi3-rma] [EXTERNAL] Re: request-based ops
by Jim Dinan 20 Jun '13

20 Jun '13

I agree with the general statement that the nonblocking call marks the end of the epoch. In the case of MPI_Win_test, it's trying to close an exposure epoch -- not an access epoch -- so communication operations from the origin process are not being completed by synchronization (although, the test/wait synchronization could be waiting for a self communication, that operation is completed by start/complete not post/wait). I think an MPI_Win_iwait operation that uses the semantic you described should be identical to MPI_Win_test. Unless I'm missing something, which never happens when discussing RMA. :) ~Jim. On Wed, Jun 19, 2013 at 10:34 AM, Barrett, Brian W <bwbarre(a)sandia.gov>wrote: > On 6/19/13 8:58 AM, "Jim Dinan" <james.dinan(a)gmail.com> wrote: > > I think we're in agreement that iflush needs to be defined separately. I > was thinking out loud (without really thinking that hard about it) that > win_test might be usable to achieve iunlock or ifence, but this is > nonsensical because the arguments are different across these functions. > Win_test also only covers win_wait. To be complete, we should also > consider MPI_Win_icomplete, and a few others. Here's a more exhaustive > list: > > MPI_Win_iflush > MPI_Win_iflush_all > MPI_Win_iflush_local > MPI_Win_iflush_local_all > MPI_Win_iunlock > MPI_Win_iunlock_all > MPI_Win_ifence > MPI_Win_icomplete > MPI_Win_iwait (to replace win_test, for completeness?) > > This is a whole other can of worms, but in some cases, the functions that > start an epoch can block (e.g. a lock operation when load/store is possible > for the target). I don't know to what extent we want to look into > nonblocking variants of those routines. We already have nonblocking fence > in the list, which initiates an epoch. > > > I think MPI_WIN_IWAIT would have very different semantics than > MPI_WIN_TEST. Again, I think the only sane way to define non-blocking > synchronization functions is to say essentially that the non-blocking > function starts the synchronization (ends the epoch from the programmers > standpoint) and the successfully completed request ends the synchronization > (starts the next epoch or allows new epochs to be started, depending on the > synchronization). Anything else is nightmarish from either a programmer's > standpoint (non-blocking can block) or an implementor's standpoint (must > suddenly track requests much more fine-grained than today, killing > performance). > > Brian > > -- > Brian W. Barrett > Scalable System Software Group > Sandia National Laboratories > > _______________________________________________ > mpi3-rma mailing list > mpi3-rma(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma >

1 0

Re: [Mpi3-rma] [EXTERNAL] Re: request-based ops
by Barrett, Brian W 19 Jun '13

19 Jun '13

On 6/19/13 8:58 AM, "Jim Dinan" <james.dinan(a)gmail.com> wrote: > I think we're in agreement that iflush needs to be defined separately. I was > thinking out loud (without really thinking that hard about it) that win_test > might be usable to achieve iunlock or ifence, but this is nonsensical because > the arguments are different across these functions. Win_test also only covers > win_wait. To be complete, we should also consider MPI_Win_icomplete, and a > few others. Here's a more exhaustive list: > > MPI_Win_iflush > MPI_Win_iflush_all > MPI_Win_iflush_local > MPI_Win_iflush_local_all > MPI_Win_iunlock > MPI_Win_iunlock_all > MPI_Win_ifence > MPI_Win_icomplete > MPI_Win_iwait (to replace win_test, for completeness?) > > This is a whole other can of worms, but in some cases, the functions that > start an epoch can block (e.g. a lock operation when load/store is possible > for the target). I don't know to what extent we want to look into nonblocking > variants of those routines. We already have nonblocking fence in the list, > which initiates an epoch. I think MPI_WIN_IWAIT would have very different semantics than MPI_WIN_TEST. Again, I think the only sane way to define non-blocking synchronization functions is to say essentially that the non-blocking function starts the synchronization (ends the epoch from the programmers standpoint) and the successfully completed request ends the synchronization (starts the next epoch or allows new epochs to be started, depending on the synchronization). Anything else is nightmarish from either a programmer's standpoint (non-blocking can block) or an implementor's standpoint (must suddenly track requests much more fine-grained than today, killing performance). Brian -- Brian W. Barrett Scalable System Software Group Sandia National Laboratories

1 0

Re: [Mpi3-rma] [EXTERNAL] Re: request-based ops
by Jim Dinan 19 Jun '13

19 Jun '13

Hi Brian, I think we're in agreement that iflush needs to be defined separately. I was thinking out loud (without really thinking that hard about it) that win_test might be usable to achieve iunlock or ifence, but this is nonsensical because the arguments are different across these functions. Win_test also only covers win_wait. To be complete, we should also consider MPI_Win_icomplete, and a few others. Here's a more exhaustive list: MPI_Win_iflush MPI_Win_iflush_all MPI_Win_iflush_local MPI_Win_iflush_local_all MPI_Win_iunlock MPI_Win_iunlock_all MPI_Win_ifence MPI_Win_icomplete MPI_Win_iwait (to replace win_test, for completeness?) This is a whole other can of worms, but in some cases, the functions that start an epoch can block (e.g. a lock operation when load/store is possible for the target). I don't know to what extent we want to look into nonblocking variants of those routines. We already have nonblocking fence in the list, which initiates an epoch. ~Jim. On Tue, Jun 18, 2013 at 4:31 PM, Barrett, Brian W <bwbarre(a)sandia.gov>wrote: > On 6/17/13 9:38 AM, "Jim Dinan" <james.dinan(a)gmail.com> wrote: > > Hi Brian, > > Re: iunlock, ifence -- > > It might make sense to look at defining MPI_Win_test for these > synchronization modes. Some of the semantics might already be defined for > us. > > > I don't think so. MPI_WIN_TEST is only defined for completing an access > epoch of generalized active target. It doesn't actually have the semantics > you want, as it's almost a moving flush. That is, a valid implementation > would is to ask "does my started counter and ack counter match?", over and > over, moving the target. There's no splitting of an operation (the > operation is "am i done?"). But MPI_WIN_FLUSH has much different > semantics; you're splitting an operation into a start/complete, which means > there's operation ordering / completion issues that don't exist with > MPI_WIN_TEST. > > Brian > > -- > Brian W. Barrett > Scalable System Software Group > Sandia National Laboratories > > _______________________________________________ > mpi3-rma mailing list > mpi3-rma(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma >

1 0

Re: [Mpi3-rma] [EXTERNAL] Re: request-based ops
by Barrett, Brian W 18 Jun '13

18 Jun '13

On 6/17/13 9:38 AM, "Jim Dinan" <james.dinan(a)gmail.com> wrote: > Hi Brian, > > Re: iunlock, ifence -- > > It might make sense to look at defining MPI_Win_test for these synchronization > modes. Some of the semantics might already be defined for us. I don't think so. MPI_WIN_TEST is only defined for completing an access epoch of generalized active target. It doesn't actually have the semantics you want, as it's almost a moving flush. That is, a valid implementation would is to ask "does my started counter and ack counter match?", over and over, moving the target. There's no splitting of an operation (the operation is "am i done?"). But MPI_WIN_FLUSH has much different semantics; you're splitting an operation into a start/complete, which means there's operation ordering / completion issues that don't exist with MPI_WIN_TEST. Brian -- Brian W. Barrett Scalable System Software Group Sandia National Laboratories

1 0

Re: [Mpi3-rma] [EXTERNAL] Re: request-based ops
by Jim Dinan 17 Jun '13

17 Jun '13

Hi Brian, Re: iunlock, ifence -- It might make sense to look at defining MPI_Win_test for these synchronization modes. Some of the semantics might already be defined for us. ~Jim. On Mon, Jun 17, 2013 at 9:44 AM, Barrett, Brian W <bwbarre(a)sandia.gov>wrote: > First, I apologize for reply on top; outlook webmail sucks. > > I think I agree with Jeff, option #2 is icky. > > I think I also agree with Jim that my performance concerns could be > overstated, as one could make iflush be a flush and return a completed > request (to be test/waited on later). There's no locking concerns, as you > have to have the lock in order to call flush (in my implementation, > anyway). But it would limit other implementations which queued rma ops, > and that would be unfortunate. > > Since Pavan's on a make everything non-blocking kick, perhaps it's > instructive to think about the ifence or iunlock (which I don't like, but I > think is useful in this conversation). In both, I think it's a relatively > clean semantic to not allow communication operations between the operation > and the test/wait, similar to how we wouldn't allow communication from > another thread during fence / unlock, as the access epoch closes when the > synchronization call starts. In that case, why would we put different > semantics in for flush? > > Or, another way of thinking about it is that for all non-blocking > communication operations, we don't allow the user to modify state between > the non-blocking operation and the test/wait. In most cases, the state is > the user buffer, but in this case, the state is the operation state. So > again, I think there's good precedent for the easy to define, easy to > understand rule that you can't start new communication operations between > the iflush and the test/wait on that window, just like you couldn't modify > the user buffer for isend/ibcast/etc. > > Brian > > -- > Brian W. Barrett > Scalable System Software Group > Sandia National Laboratories > ________________________________________ > From: mpi3-rma-bounces(a)lists.mpi-forum.org [ > mpi3-rma-bounces(a)lists.mpi-forum.org] on behalf of Jeff Hammond [ > jhammond(a)alcf.anl.gov] > Sent: Monday, June 17, 2013 8:14 AM > To: MPI 3.0 Remote Memory Access working group > Subject: Re: [Mpi3-rma] [EXTERNAL] Re: request-based ops > > I do not want to consider interpretation 2 because it is problematic > for some implementations. There are cases where it requires the > flush to happen when the request is waited upon, in which case there > is absolutely no benefit over doing a blocking flush. > > Actually, now that I think about it, every implementation has to issue > or re-issue a flush at the time the wait is called if there have been > an RMA operations issued (to the relevant targets) since the time the > first fence may have been issued. Hence, an implementation that > issues the iflush eagerly may do twice the work whereas one that does > not is providing zero benefit over the blocking flush. > > The only case where 2 is going to be effective and efficient is the > case where it reduces to 1 because no new RMA operations are issued > between the iflush and the wait. > > Jeff > > On Mon, Jun 17, 2013 at 8:52 AM, Jim Dinan <james.dinan(a)gmail.com> wrote: > > It seems like there are two possible semantics for which operations are > > complete by a call to MPI_Win_iflush: > > > > (1) Completes all operations issued by the origin process on the given > > window before MPI_Win_iflush was called. > > (2) Completes all operations issued by the origin process on the given > > window before MPI_Win_iflush completed. > > > > So far, we've just been looking at #1, but I think that #2 is worth > > considering. Option #2 allows an implementation that just checks if the > > counters are equal. This avoids the issue where #1 can't be implemented > in > > terms of #2, because issuing an unbounded number of operations while > testing > > on the iflush request can cause the iflush to not complete indefinitely. > > > > Option #2 does not directly provide the functionality that Jeff was > looking > > for, but this could be implemented using two windows. Issue a bunch of > > operations on win1 and iflush on win2, when win2 has been flushed, > switch to > > issuing operations on win2 and iflushing win1. > > > > ~Jim. > > > > On Mon, Jun 17, 2013 at 7:38 AM, Jim Dinan <james.dinan(a)gmail.com> > wrote: > >> > >> Sorry, I should have been more specific. An implementation of iflush > that > >> waits for the completion of all messages should be valid. Such an > >> implementation would compare counters and return true if they are the > same. > >> This implementation could have the issue I mentioned in the previous > >> message, where the user continuously issuing operations can prevent > iflush > >> from completing. > >> > >> Jim. > >> > >> On Jun 16, 2013 10:13 AM, "Pavan Balaji" <balaji(a)mcs.anl.gov> wrote: > >>> > >>> > >>> On 06/16/2013 10:02 AM, Jim Dinan wrote: > >>>> > >>>> If the channel is unordered, a message after the iflush can increment > >>>> the counter, while one before the iflush has not yet completed. So, > the > >>>> counter is not enough to mark a particular point in time. > >>> > >>> > >>> Ah, good point. > >>> > >>>> An implementation of iflush as flush should still be valid, right? > Just > >>> > >>> > >>> No. You cannot do this if the user only uses TEST. > >>> > >>> MPI_WIN_IFLUSH(&req); > >>> while (MPI_TEST(req) is not done); > >>> > >>> -- Pavan > >>> > >>> -- > >>> Pavan Balaji > >>> http://www.mcs.anl.gov/~balaji > > > > > > > > _______________________________________________ > > mpi3-rma mailing list > > mpi3-rma(a)lists.mpi-forum.org > > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma > > > > -- > Jeff Hammond > Argonne Leadership Computing Facility > University of Chicago Computation Institute > jhammond(a)alcf.anl.gov / (630) 252-5381 > http://www.linkedin.com/in/jeffhammond > https://wiki.alcf.anl.gov/parts/index.php/User:Jhammond > ALCF docs: http://www.alcf.anl.gov/user-guides > _______________________________________________ > mpi3-rma mailing list > mpi3-rma(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma > > > _______________________________________________ > mpi3-rma mailing list > mpi3-rma(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma >

1 0

Re: [Mpi3-rma] [EXTERNAL] Re: request-based ops
by Barrett, Brian W 17 Jun '13

17 Jun '13

First, I apologize for reply on top; outlook webmail sucks. I think I agree with Jeff, option #2 is icky. I think I also agree with Jim that my performance concerns could be overstated, as one could make iflush be a flush and return a completed request (to be test/waited on later). There's no locking concerns, as you have to have the lock in order to call flush (in my implementation, anyway). But it would limit other implementations which queued rma ops, and that would be unfortunate. Since Pavan's on a make everything non-blocking kick, perhaps it's instructive to think about the ifence or iunlock (which I don't like, but I think is useful in this conversation). In both, I think it's a relatively clean semantic to not allow communication operations between the operation and the test/wait, similar to how we wouldn't allow communication from another thread during fence / unlock, as the access epoch closes when the synchronization call starts. In that case, why would we put different semantics in for flush? Or, another way of thinking about it is that for all non-blocking communication operations, we don't allow the user to modify state between the non-blocking operation and the test/wait. In most cases, the state is the user buffer, but in this case, the state is the operation state. So again, I think there's good precedent for the easy to define, easy to understand rule that you can't start new communication operations between the iflush and the test/wait on that window, just like you couldn't modify the user buffer for isend/ibcast/etc. Brian -- Brian W. Barrett Scalable System Software Group Sandia National Laboratories ________________________________________ From: mpi3-rma-bounces(a)lists.mpi-forum.org [mpi3-rma-bounces(a)lists.mpi-forum.org] on behalf of Jeff Hammond [jhammond(a)alcf.anl.gov] Sent: Monday, June 17, 2013 8:14 AM To: MPI 3.0 Remote Memory Access working group Subject: Re: [Mpi3-rma] [EXTERNAL] Re: request-based ops I do not want to consider interpretation 2 because it is problematic for some implementations. There are cases where it requires the flush to happen when the request is waited upon, in which case there is absolutely no benefit over doing a blocking flush. Actually, now that I think about it, every implementation has to issue or re-issue a flush at the time the wait is called if there have been an RMA operations issued (to the relevant targets) since the time the first fence may have been issued. Hence, an implementation that issues the iflush eagerly may do twice the work whereas one that does not is providing zero benefit over the blocking flush. The only case where 2 is going to be effective and efficient is the case where it reduces to 1 because no new RMA operations are issued between the iflush and the wait. Jeff On Mon, Jun 17, 2013 at 8:52 AM, Jim Dinan <james.dinan(a)gmail.com> wrote: > It seems like there are two possible semantics for which operations are > complete by a call to MPI_Win_iflush: > > (1) Completes all operations issued by the origin process on the given > window before MPI_Win_iflush was called. > (2) Completes all operations issued by the origin process on the given > window before MPI_Win_iflush completed. > > So far, we've just been looking at #1, but I think that #2 is worth > considering. Option #2 allows an implementation that just checks if the > counters are equal. This avoids the issue where #1 can't be implemented in > terms of #2, because issuing an unbounded number of operations while testing > on the iflush request can cause the iflush to not complete indefinitely. > > Option #2 does not directly provide the functionality that Jeff was looking > for, but this could be implemented using two windows. Issue a bunch of > operations on win1 and iflush on win2, when win2 has been flushed, switch to > issuing operations on win2 and iflushing win1. > > ~Jim. > > On Mon, Jun 17, 2013 at 7:38 AM, Jim Dinan <james.dinan(a)gmail.com> wrote: >> >> Sorry, I should have been more specific. An implementation of iflush that >> waits for the completion of all messages should be valid. Such an >> implementation would compare counters and return true if they are the same. >> This implementation could have the issue I mentioned in the previous >> message, where the user continuously issuing operations can prevent iflush >> from completing. >> >> Jim. >> >> On Jun 16, 2013 10:13 AM, "Pavan Balaji" <balaji(a)mcs.anl.gov> wrote: >>> >>> >>> On 06/16/2013 10:02 AM, Jim Dinan wrote: >>>> >>>> If the channel is unordered, a message after the iflush can increment >>>> the counter, while one before the iflush has not yet completed. So, the >>>> counter is not enough to mark a particular point in time. >>> >>> >>> Ah, good point. >>> >>>> An implementation of iflush as flush should still be valid, right? Just >>> >>> >>> No. You cannot do this if the user only uses TEST. >>> >>> MPI_WIN_IFLUSH(&req); >>> while (MPI_TEST(req) is not done); >>> >>> -- Pavan >>> >>> -- >>> Pavan Balaji >>> http://www.mcs.anl.gov/~balaji > > > > _______________________________________________ > mpi3-rma mailing list > mpi3-rma(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma -- Jeff Hammond Argonne Leadership Computing Facility University of Chicago Computation Institute jhammond(a)alcf.anl.gov / (630) 252-5381 http://www.linkedin.com/in/jeffhammond https://wiki.alcf.anl.gov/parts/index.php/User:Jhammond ALCF docs: http://www.alcf.anl.gov/user-guides _______________________________________________ mpi3-rma mailing list mpi3-rma(a)lists.mpi-forum.org http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma

1 0

Re: [Mpi3-rma] [EXTERNAL] Re: request-based ops
by Jeff Hammond 17 Jun '13

17 Jun '13

I do not want to consider interpretation 2 because it is problematic for some implementations. There are cases where it requires the flush to happen when the request is waited upon, in which case there is absolutely no benefit over doing a blocking flush. Actually, now that I think about it, every implementation has to issue or re-issue a flush at the time the wait is called if there have been an RMA operations issued (to the relevant targets) since the time the first fence may have been issued. Hence, an implementation that issues the iflush eagerly may do twice the work whereas one that does not is providing zero benefit over the blocking flush. The only case where 2 is going to be effective and efficient is the case where it reduces to 1 because no new RMA operations are issued between the iflush and the wait. Jeff On Mon, Jun 17, 2013 at 8:52 AM, Jim Dinan <james.dinan(a)gmail.com> wrote: > It seems like there are two possible semantics for which operations are > complete by a call to MPI_Win_iflush: > > (1) Completes all operations issued by the origin process on the given > window before MPI_Win_iflush was called. > (2) Completes all operations issued by the origin process on the given > window before MPI_Win_iflush completed. > > So far, we've just been looking at #1, but I think that #2 is worth > considering. Option #2 allows an implementation that just checks if the > counters are equal. This avoids the issue where #1 can't be implemented in > terms of #2, because issuing an unbounded number of operations while testing > on the iflush request can cause the iflush to not complete indefinitely. > > Option #2 does not directly provide the functionality that Jeff was looking > for, but this could be implemented using two windows. Issue a bunch of > operations on win1 and iflush on win2, when win2 has been flushed, switch to > issuing operations on win2 and iflushing win1. > > ~Jim. > > On Mon, Jun 17, 2013 at 7:38 AM, Jim Dinan <james.dinan(a)gmail.com> wrote: >> >> Sorry, I should have been more specific. An implementation of iflush that >> waits for the completion of all messages should be valid. Such an >> implementation would compare counters and return true if they are the same. >> This implementation could have the issue I mentioned in the previous >> message, where the user continuously issuing operations can prevent iflush >> from completing. >> >> Jim. >> >> On Jun 16, 2013 10:13 AM, "Pavan Balaji" <balaji(a)mcs.anl.gov> wrote: >>> >>> >>> On 06/16/2013 10:02 AM, Jim Dinan wrote: >>>> >>>> If the channel is unordered, a message after the iflush can increment >>>> the counter, while one before the iflush has not yet completed. So, the >>>> counter is not enough to mark a particular point in time. >>> >>> >>> Ah, good point. >>> >>>> An implementation of iflush as flush should still be valid, right? Just >>> >>> >>> No. You cannot do this if the user only uses TEST. >>> >>> MPI_WIN_IFLUSH(&req); >>> while (MPI_TEST(req) is not done); >>> >>> -- Pavan >>> >>> -- >>> Pavan Balaji >>> http://www.mcs.anl.gov/~balaji > > > > _______________________________________________ > mpi3-rma mailing list > mpi3-rma(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma -- Jeff Hammond Argonne Leadership Computing Facility University of Chicago Computation Institute jhammond(a)alcf.anl.gov / (630) 252-5381 http://www.linkedin.com/in/jeffhammond https://wiki.alcf.anl.gov/parts/index.php/User:Jhammond ALCF docs: http://www.alcf.anl.gov/user-guides

1 0

Re: [Mpi3-rma] [EXTERNAL] Re: request-based ops
by Jim Dinan 17 Jun '13

17 Jun '13

It seems like there are two possible semantics for which operations are complete by a call to MPI_Win_iflush: (1) Completes all operations issued by the origin process on the given window before MPI_Win_iflush was called. (2) Completes all operations issued by the origin process on the given window before MPI_Win_iflush completed. So far, we've just been looking at #1, but I think that #2 is worth considering. Option #2 allows an implementation that just checks if the counters are equal. This avoids the issue where #1 can't be implemented in terms of #2, because issuing an unbounded number of operations while testing on the iflush request can cause the iflush to not complete indefinitely. Option #2 does not directly provide the functionality that Jeff was looking for, but this could be implemented using two windows. Issue a bunch of operations on win1 and iflush on win2, when win2 has been flushed, switch to issuing operations on win2 and iflushing win1. ~Jim. On Mon, Jun 17, 2013 at 7:38 AM, Jim Dinan <james.dinan(a)gmail.com> wrote: > Sorry, I should have been more specific. An implementation of iflush that > waits for the completion of all messages should be valid. Such an > implementation would compare counters and return true if they are the > same. This implementation could have the issue I mentioned in the previous > message, where the user continuously issuing operations can prevent iflush > from completing. > > Jim. > On Jun 16, 2013 10:13 AM, "Pavan Balaji" <balaji(a)mcs.anl.gov> wrote: > >> >> On 06/16/2013 10:02 AM, Jim Dinan wrote: >> >>> If the channel is unordered, a message after the iflush can increment >>> the counter, while one before the iflush has not yet completed. So, the >>> counter is not enough to mark a particular point in time. >>> >> >> Ah, good point. >> >> An implementation of iflush as flush should still be valid, right? Just >>> >> >> No. You cannot do this if the user only uses TEST. >> >> MPI_WIN_IFLUSH(&req); >> while (MPI_TEST(req) is not done); >> >> -- Pavan >> >> -- >> Pavan Balaji >> http://www.mcs.anl.gov/~balaji >> >

1 0