...

I propose the following additions. I will be at the Forum this week and can discuss them in person with any interested parties. If you want to poop on my ideas, doing so in person will probably go over better. I assure you that the outcome will not be different.

Unless you include overall personal happiness in the outcome.

...

(1) Define semantics of shared memory windows in separate model.

In Section 11.2.3 it says "MPI does not define semantics for accessing shared memory windows in the separate memory model."

I would like to try to define these semantics in MPI 3.1. Shouldn't it at least be possible to use MPI_WIN_(UN)LOCK with MPI_LOCK_EXCLUSIVE here?

I believe that supporting RMA for weakly- or non- coherent memory architectures is vital for future systems. I agree. We had this discussion and decided to postpone it because we had other more pressing issues. Now may be the time. A discussion would be good.

Yes, there was no urgency. I'm glad the standard left this issue wide open instead of defining something restrictive.

...

(2) Add info keys for restricted usage of RMA ops.

Add info keys to windows to restrict usage to subsets of RMA, particular MPI_FETCH_AND_OP and/or MPI_COMPARE_AND_SWAP.

As noted on a recent discussion involving Hubert, Jim and me on this list, this has potential performance benefits. Yes, this should probably be an assertion though. Info keys are too slow to interpret (unless you want to define it on a per-window and not per-operation basis). We shall discuss.

I would like it to be per-window. It would help if there was a way to collectively change info keys but perhaps someone will say that just creating N windows with different keys on to of the same memory is valid here if one barriers/syncs between epochs.

...

(3) Unify and add info keys across window types.

Unify info keys across different window creation mechanisms. As I noted in my previous email, it is not clear if "accumulate_ordering, accumulate_ops, same_size, alloc_shared_noncontig" are valid for MPI_WIN_ALLOCATE and MPI_WIN_CREATE_DYNAMIC. That's what was intended. Maybe we should update the wording to reflect this.

...

Furthermore, I would like the "same_size" info key to be defined for MPI_WIN_CREATE as well since it may eliminate an unnecessary collective (but not all of them, so I don't mind if people don't like this suggestion).

Finally, I would like an info key for "only_one_rank_gives_nonzero_size" (probably needs a better name) since this means that one needs only to do a bcast and not an allreduce or allgather during window creation/allocation. This usage pattern is common for certain types of load-balancing counters (not necessarily globally shared, although GA's NXTVAL is this way) and perhaps also for some implementations of mutexes. We should discuss.

Yes, there are perhaps generalizations of this that I haven't thought of. Best, Jeff -- Jeff Hammond Argonne Leadership Computing Facility University of Chicago Computation Institute [email protected] / (630) 252-5381 http://www.linkedin.com/in/jeffhammond https://wiki.alcf.anl.gov/parts/index.php/User:Jhammond