Do you really think it is just a couple of ifs? If this is not MPI_COMM_WORLD... if this is a derived datatype but not contiguous locally... if this is a derived datatype but not contiguous remotely... if these are not the same datatypes... if this is not heterogeneous... You also get into the trade-offs (that we obviously have to make even with two routines as well)... so, should contiguous derived datatypes be fast or slow? Well, obviously the NIC based agent can handle them if you do the math in the MPI library, but that math takes instructions. One of the things that was put on the table in Chicago was "should we have specialized functions for each native datatype?" Ultimately, it is all about what your performance targets are in terms of latency and issue rate. On one end of the spectrum, you have MPI_Isend(). On the other you have the speed of a native processor store. Which end of the spectrum do we want to be on? If we want broader penetration for RMA, we need to move toward the latter. So, yeah, "a handful of ifs" may be an issue... Particularly on any architecture where a given thread isn't all that speedy... Keith From: [email protected] [mailto:[email protected]] On Behalf Of Richard Treumann Sent: Wednesday, September 16, 2009 3:05 PM To: MPI 3.0 Remote Memory Access working group Subject: Re: [Mpi3-rma] non-contiguous support in RMA & one-sided pack/unpack (?) Hi Jeff I do not understand the need for two distinct origin side operations if you will assume the agency infrastructure must always be standing by ready to be activated. The MPI implementation will be able to determine at the origin side if the parameters passed into a complex RMA fit the capabilities of its NIC. It can decide whether to use the intrinsic NIC mode or the software agent mode. If it has a NIC that does integer accumulate but not floating point accumulate, the implementation can pick between NIC mode and agent mode. When a new NIC that does floating point accumulate comes along, the implementation can exploit it with no need for application rewrite. On the other hand, the RAW RMA can be used by by applications that want to exploit NIC capabilities that are widely available today and do not want to drag in overheads for an agent waiting on the sidelines. An agent that they may know will never be needed. If someday in the future the typical NIC has certain additional capabilities we can add a richer RAW operation to the MPI standard. Can you explain to me what you expect to have always available as a software agent that will be there at no significant cost in memory or harm to performance of non-RMA operations? Are you thinking we need two distinct routines just to avoid a couple "if" tests to decide inside libmpi whether to use NIC based or agent based RMA? Dick Dick Treumann - MPI Team IBM Systems & Technology Group Dept X2ZA / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601 Tele (845) 433-7846 Fax (845) 433-8363 [cid:[email protected]]Jeff Hammond ---09/16/2009 04:37:33 PM---I was having the same thoughts during my hideous commute today. Clearly, future systems may have ver From: Jeff Hammond <[email protected]> To: "MPI 3.0 Remote Memory Access working group" <[email protected]> Date: 09/16/2009 04:37 PM Subject: Re: [Mpi3-rma] non-contiguous support in RMA & one-sided pack/unpack (?) Sent by: [email protected] ________________________________ <snip> I'm not sure I understand all this talk about assertions upon initialization, since I was hoping that Raw_xfer and general-purpose xfer (Gen_xfer) would both always be available but that the former would be much faster in certain contexts, since implementing the Raw version close to hardware doesn't require much work and would not interfere with how Gen_xfer operates. Am I missing something? <snip>