Hi Brian, Re: iunlock, ifence -- It might make sense to look at defining MPI_Win_test for these synchronization modes. Some of the semantics might already be defined for us. ~Jim. On Mon, Jun 17, 2013 at 9:44 AM, Barrett, Brian W <[email protected]>wrote:
First, I apologize for reply on top; outlook webmail sucks.
I think I agree with Jeff, option #2 is icky.
I think I also agree with Jim that my performance concerns could be overstated, as one could make iflush be a flush and return a completed request (to be test/waited on later). There's no locking concerns, as you have to have the lock in order to call flush (in my implementation, anyway). But it would limit other implementations which queued rma ops, and that would be unfortunate.
Since Pavan's on a make everything non-blocking kick, perhaps it's instructive to think about the ifence or iunlock (which I don't like, but I think is useful in this conversation). In both, I think it's a relatively clean semantic to not allow communication operations between the operation and the test/wait, similar to how we wouldn't allow communication from another thread during fence / unlock, as the access epoch closes when the synchronization call starts. In that case, why would we put different semantics in for flush?
Or, another way of thinking about it is that for all non-blocking communication operations, we don't allow the user to modify state between the non-blocking operation and the test/wait. In most cases, the state is the user buffer, but in this case, the state is the operation state. So again, I think there's good precedent for the easy to define, easy to understand rule that you can't start new communication operations between the iflush and the test/wait on that window, just like you couldn't modify the user buffer for isend/ibcast/etc.
Brian
-- Brian W. Barrett Scalable System Software Group Sandia National Laboratories ________________________________________ From: [email protected] [ [email protected]] on behalf of Jeff Hammond [ [email protected]] Sent: Monday, June 17, 2013 8:14 AM To: MPI 3.0 Remote Memory Access working group Subject: Re: [Mpi3-rma] [EXTERNAL] Re: request-based ops
I do not want to consider interpretation 2 because it is problematic for some implementations. There are cases where it requires the flush to happen when the request is waited upon, in which case there is absolutely no benefit over doing a blocking flush.
Actually, now that I think about it, every implementation has to issue or re-issue a flush at the time the wait is called if there have been an RMA operations issued (to the relevant targets) since the time the first fence may have been issued. Hence, an implementation that issues the iflush eagerly may do twice the work whereas one that does not is providing zero benefit over the blocking flush.
The only case where 2 is going to be effective and efficient is the case where it reduces to 1 because no new RMA operations are issued between the iflush and the wait.
Jeff
On Mon, Jun 17, 2013 at 8:52 AM, Jim Dinan <[email protected]> wrote:
It seems like there are two possible semantics for which operations are complete by a call to MPI_Win_iflush:
(1) Completes all operations issued by the origin process on the given window before MPI_Win_iflush was called. (2) Completes all operations issued by the origin process on the given window before MPI_Win_iflush completed.
So far, we've just been looking at #1, but I think that #2 is worth considering. Option #2 allows an implementation that just checks if the counters are equal. This avoids the issue where #1 can't be implemented in terms of #2, because issuing an unbounded number of operations while testing on the iflush request can cause the iflush to not complete indefinitely.
Option #2 does not directly provide the functionality that Jeff was looking for, but this could be implemented using two windows. Issue a bunch of operations on win1 and iflush on win2, when win2 has been flushed, switch to issuing operations on win2 and iflushing win1.
~Jim.
On Mon, Jun 17, 2013 at 7:38 AM, Jim Dinan <[email protected]> wrote:
Sorry, I should have been more specific. An implementation of iflush
that
waits for the completion of all messages should be valid. Such an implementation would compare counters and return true if they are the same. This implementation could have the issue I mentioned in the previous message, where the user continuously issuing operations can prevent iflush from completing.
Jim.
On Jun 16, 2013 10:13 AM, "Pavan Balaji" <[email protected]> wrote:
On 06/16/2013 10:02 AM, Jim Dinan wrote:
If the channel is unordered, a message after the iflush can increment the counter, while one before the iflush has not yet completed. So,
the
counter is not enough to mark a particular point in time.
Ah, good point.
An implementation of iflush as flush should still be valid, right? Just
No. You cannot do this if the user only uses TEST.
MPI_WIN_IFLUSH(&req); while (MPI_TEST(req) is not done);
-- Pavan
-- Pavan Balaji http://www.mcs.anl.gov/~balaji
_______________________________________________ mpi3-rma mailing list [email protected] http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma
-- Jeff Hammond Argonne Leadership Computing Facility University of Chicago Computation Institute [email protected] / (630) 252-5381 http://www.linkedin.com/in/jeffhammond https://wiki.alcf.anl.gov/parts/index.php/User:Jhammond ALCF docs: http://www.alcf.anl.gov/user-guides _______________________________________________ mpi3-rma mailing list [email protected] http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma
_______________________________________________ mpi3-rma mailing list [email protected] http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma