PAMI can do RDMA against both registered memory segments and virtual addresses today, but the performance difference is noticeable on BGQ. Other implementations have different performance characteristics, so it's a solved problem in some sense. I just do not like having to assume the existence of the necessary hardware features that enable virtual-to-physical translation in the network. I'm not too worried about Cray solving these problems in the future, so if you're sure that OFED and Portals4 are now or will soon be able to deal with the issues created by dynamic windows, then I am less concerned about this issue. I think Pavan's proposal is the right one to focus on at this point since it's a simple concept and addresses the common case anyways. Best, Jeff On Mon, Dec 2, 2013 at 9:50 AM, Barrett, Brian W <[email protected]> wrote:

The idea behind dynamic windows was to make them possible to implement today and push the vendors to support a more rational programming model tomorrow. Like Jim, I'm not sure things are bleak enough that we should change.

In OFED, I think it would be very difficult today. But with Mellanox's work to support dynamic registration cache in hardware, I think things are trending in the right direction there.

For Portals 4, if an implementation supports the BIND_INACCESSIBLE feature, and I think most implementations will, supporting dynamic windows is straight-forward.

PAMI and DMAPP are probably more difficult, but a firehose-like setup should be possible until hardware catches up.

Brian

-- Brian W. Barrett Scalable System Software Group Sandia National Laboratories ________________________________ From: mpiwg-rma [[email protected]] on behalf of Jeff Hammond [[email protected]] Sent: Sunday, December 01, 2013 10:16 AM To: MPI WG Remote Memory Access working group Cc: MPI WG Remote Memory Access working group Subject: [EXTERNAL] Re: [mpiwg-rma] making dynamic windows suck less

How do you do pure (i.e. active-message and rendezvous-free) RDMA with just a virtual address (MPI_Aint) in (1) PAMI, (2) OFED, (3) DMAPP, (4) Portals4?

Jeff

Sent from my iPhone

On Dec 1, 2013, at 10:40 AM, Jim Dinan <[email protected]> wrote:

Can the MPI_Win_memobj be an MPI_Aint?

I don't share this pessimism with respect to dynamic windows; I think we need to prove that these issues exist before we try to fix them.

~Jim.

On Wed, Nov 27, 2013 at 5:46 PM, Jeff Hammond <[email protected]> wrote:

...

Lots of people think - I take position on their correctness - that dynamic windows are doomed to substandard performance. I can certainly see some potential for this on networks that require memory registration.

Can we try to add an opaque object that can encapsulate memory registration such that dynamic windows 2.0 would not have so many problems? If one were to send-recv not just the virtual address but MPI_Win_memobj (to be named properly later), then maybe we could deal with the shortcomings of IB, etc. w.r.t. RDMA and memory registration.

This is really just a thought. I have no concrete proposal. Maybe people will have ideas in December.

Jeff

-- Jeff Hammond [email protected] _______________________________________________ mpiwg-rma mailing list [email protected] http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma

_______________________________________________ mpiwg-rma mailing list [email protected] http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma

_______________________________________________ mpiwg-rma mailing list [email protected] http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma

-- Jeff Hammond [email protected]