It's rather a trivial change w/ MPICH2 code :) mostly adding the const keyword; not much of a code change. The change was about propagating the const to the lower function that didn't have it yet; the compiler helps you a lot with that. The necessary cast for the lower level function was already there; I had to add two more casts to deal with the MPI_User_function and MPI_Comm_spawn. Thanks, .Erez -----Original Message----- From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Underwood, Keith D Sent: Friday, March 13, 2009 12:53 PM To: MPI 2.2 Subject: Re: [Mpi-22] Ticket #46: Addconst Keyword to the C bindings- implementation Because the semantics of sockets requires a copy in the kernel? Because changes like this should be made available from the bottom up? Because they figured out how to do it without casting away const somewhere in the middle of the stack? I must admit that I'm impressed by the rate at which you can write correct code if you made this change in 2.5h while in discussions without just casting it away as soon as you entered the library... If you cast it away immediately or already had a heavy dose of const in your library to start with, then that isn't really a fair metric. Keith

-----Original Message----- From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_lists.mpi- forum.org] On Behalf Of Erez Haba Sent: Friday, March 13, 2009 1:43 PM To: MPI 2.2 Subject: Re: [Mpi-22] Ticket #46: Addconst Keyword to the C bindings- implementation

P.S.

Take a look at sockets send function http://www.opengroup.org/onlinepubs/000095399/functions/send.html

why do you think that they bothered adding const to the sockets send function?

Thanks, .Erez

-----Original Message----- From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_lists.mpi- forum.org] On Behalf Of Erez Haba Sent: Friday, March 13, 2009 12:40 PM To: MPI 2.2 Subject: Re: [Mpi-22] Ticket #46: Addconst Keyword to the C bindings- implementation

I've implemented this complete change during the last MPI-forum meeting. It took me about 2.5h hours to implement while participating in the discussion. So I would argue that it's not a big implementation item for any MPI. (look at the diff last change, out of the 10 lines in the diff, only 1 line is changed)

I would argue that casting away the const to adapt your buffer to your lower level interconnect interface is perfectly fine. The semantic for your lower level interconnect interface is simple, clear and does not modify the send buffer. (for the cases where actually you need to cast the const away)

Thanks, .Erez

-----Original Message----- From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_lists.mpi- forum.org] On Behalf Of Barrett, Brian W Sent: Friday, March 13, 2009 11:36 AM To: MPI 2.2 Subject: Re: [Mpi-22] Ticket #46: Addconst Keyword to the C bindings- implementation

All -

I spent some time looking at what it would take to implement this proposal in Open MPI as well as the changes made in the MPICH2 implementation. I think both raise some concerns for this proposal, particularly for MPI 2.2.

The diff for the proposal was almost 10,000 lines long, which is a little frightening for 2.2 change. However, the places the constness is cast away for the send buffer raises some issues. In particular, for eager, contiguous sends, the send buffer constness is cast away in MPID_Send, which is the second call in the send call stack. Since there has been much discussion about how the constness of a buffer shouldn't be cast away at the upper layers of the MPI implementation, it seems problematic that the reference implementation does just that. It seems problematic not because of the choice made by the implementors of the patch, but because it is not clear that the is really a better implementation option. Of course, the choice of how to handle non-contiguous data raises some issues for Open MPI's internals.

Open MPI's low layer communication interface is based on a iovec-like interface. Like most interfaces, the iovec-like interface is shared between the send and receive calls. The iovec structure can not be const, meaning that we are faced with two choices. We can either cast away the const interface before this low-level interface or we can create two data structures, one for send and one fore receive. The const cast option obviously has its problems. The two data structure option is certainly not appealing for us from a code maintainability stand-point. This is further complicated by the fact that almost none of the transports Open MPI currently supports can be used without casting away the constness.

I understand that it would be good to promote the use of const for send buffers in lower-level middleware, as well as the fact that not all MPI implementations have such middleware. However, many interfaces have a similar iovec issue to Open MPI, and they'll be unlike to change. The fact that we will permanently be casting away const is slightly bothersome to me. I think given these issues and the large size of the change, pushing this proposal to 3.0 would be appropriate.

Brian

________________________________________ From: mpi-22-bounces_at_[hidden] [mpi-22-bounces_at_lists.mpi- forum.org] On Behalf Of Erez Haba [erezh_at_[hidden]] Sent: Friday, March 13, 2009 10:40 AM To: MPI 2.2 Subject: Re: [Mpi-22] Ticket #46: Addconst Keyword to the C bindings- implementation

Hi Jeff,

The point of this ticket is to present a consistent const correct interface to the programmer and hide all the idiosyncrasies of the down-level interfaces.

That said, to answer your questions:

No casting is required for: * shared memory * sockets (TCP) when using 'send()' (single buffer passed in)

Casting is required for interfaces that did not define their io vector buffer member as const. (an interface would need to define two io vector structures, one with const buffer for send and one with non-const for receive; most of them don't).

Thus, casting is required when using the scatter/gather io vector with * sockets (TC) * Network Direct (IB and iWarp)

With MPICH2 implementation, casting out the const for the send buffer when using these interfaces was there all along as (most of) MPICH2 internal functions already used const to pass the send buffer. No new cast was required in this full const implementation.

Again, casting away the const to use the lower level interface does not violate any const rules as long as the semantic is preserved.

Thanks, .Erez

-----Original Message----- From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_lists.mpi- forum.org] On Behalf Of Jeff Squyres Sent: Friday, March 13, 2009 6:05 AM To: MPI 2.2 Subject: Re: [Mpi-22] Ticket #46: Addconst Keyword to the C bindings- implementation

Do you end up casting away the const when invoking the Windows network interfaces?

On Linux, I think the following interfaces would require casting the const away:

- shared memory (even using Linux vmsplice()) - MX - TCP - OpenFabrics (both verbs and udapl)

These interfaces do not require casting away const:

- SCTP (I *think* -- I'm not an SCTP expert, but a quick look in sctp.h showed a "const" argument for the sctp_sendmsg() function)

I can't check these offhand:

- Portals - Elan

On Mar 12, 2009, at 6:06 PM, Erez Haba wrote:

...

I urge you, please read the ticket proposal again. It discusses the merit of this proposal.

As for IB; yes you'd copy small messages (which const is fine with) and some implement RDMA read or write for large messages; you might need to cast the const away if your IB interface does not expose const SGE's.

As for the compiler; no you are not violating the contact; (otherwise we wouldn't have casts); as long as you guarantee that you are not changing the buffer. It's a way for the programmer to use old, non const correct, interfaces.

.Erez

-----Original Message----- From: mpi-22-bounces_at_[hidden] [mailto:mpi-22- bounces_at_[hidden] ] On Behalf Of Underwood, Keith D Sent: Thursday, March 12, 2009 2:03 PM To: MPI 2.2 Subject: Re: [Mpi-22] Ticket #46: Add const Keyword to the C bindings- implementation

I didn't think that typical MPI implementations over IB copied the data for large messages (as opposed to the bounce buffers for small messages).

If you are going to have to cast away the const, what is the point of putting it in? In fact, if you have to cast away the const, aren't you violating your contract with the compiler? Should this change be rolled out from the bottom up? i.e. common low-level APIs first and then MPI?

Keith

...

-----Original Message----- From: mpi-22-bounces_at_[hidden] [mailto:mpi-22- bounces_at_lists.mpi- forum.org] On Behalf Of Erez Haba Sent: Thursday, March 12, 2009 2:10 PM To: MPI 2.2 Subject: Re: [Mpi-22] Ticket #46: Add const Keyword to the C bindings- implementation

Possibly you'll have to cast. There are few interfaces that take const for send buffer (like sockets). In other cases you copy the buffer (like with IB) which keeps the const property.

Thanks, .Erez

-----Original Message----- From: mpi-22-bounces_at_[hidden] [mailto:mpi-22- bounces_at_lists.mpi- forum.org] On Behalf Of Underwood, Keith D Sent: Thursday, March 12, 2009 12:05 PM To: MPI 2.2 Subject: Re: [Mpi-22] Ticket #46: Add const Keyword to the C bindings- implementation

It would seem like const would eventually have to be cast away for most lower level interfaces. Portals, Elan, IB Verbs, and as best I can tell MX don't have const on their send buffers...

Keith

...

-----Original Message----- From: mpi-22-bounces_at_[hidden] [mailto:mpi-22- bounces_at_lists.mpi- forum.org] On Behalf Of Erez Haba Sent: Thursday, March 12, 2009 12:44 PM To: MPI 2.2 Subject: Re: [Mpi-22] Ticket #46: Add const Keyword to the C bindings- implementation

No, it's not being cast away (although it is a quick way for you to implement it). In MPICH2 implementation it trickles down to the lower functions. There are very few places where the const is being cast away because ticket #46 does not to break backward compatibility. For example, when the user function is called (const was not added to the MPI_User_function) the implementation must cast away const from the send buffer before calling the user function.

Thanks, .Erez

-----Original Message----- From: mpi-22-bounces_at_[hidden] [mailto:mpi-22- bounces_at_lists.mpi- forum.org] On Behalf Of Jeff Squyres Sent: Thursday, March 12, 2009 11:34 AM To: MPI 2.2 Subject: Re: [Mpi-22] Ticket #46: Add const Keyword to the C bindings- implementation

More importantly, how is const handled? Is it just cast away inside the MPI library?

On Mar 12, 2009, at 2:34 PM, Underwood, Keith D wrote:

...

Since the main purpose of an implementation for something like adding const is to see how invasive the change is, would it be possible to get a diff against the trees that these were created from?

Thanks, Keith

From: mpi-22-bounces_at_[hidden] [mailto:mpi-22- bounces_at_[hidden] ] On Behalf Of Erez Haba Sent: Thursday, March 12, 2009 12:24 PM To: MPI 2.2 Subject: [Mpi-22] Ticket #46: Add const Keyword to the C bindings - implementation

Implementation for ticket #46 is now available from ANL (thanks Pavan)

tarballs

...

http://www.mcs.anl.gov/research/projects/mpich2/downloads/tarballs/nightl y /

...

const

...

thanks, .Erez

_______________________________________________ mpi-22 mailing list mpi-22_at_[hidden] http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22

-- Jeff Squyres Cisco Systems

_______________________________________________ mpi-22 mailing list mpi-22_at_[hidden] http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22

_______________________________________________ mpi-22 mailing list mpi-22_at_[hidden] http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22

_______________________________________________ mpi-22 mailing list mpi-22_at_[hidden] http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22

_______________________________________________ mpi-22 mailing list mpi-22_at_[hidden] http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22

_______________________________________________ mpi-22 mailing list mpi-22_at_[hidden] http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22

_______________________________________________ mpi-22 mailing list mpi-22_at_[hidden] http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22

-- Jeff Squyres Cisco Systems

_______________________________________________ mpi-22 mailing list mpi-22_at_[hidden] http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22

_______________________________________________ mpi-22 mailing list mpi-22_at_[hidden] http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22

_______________________________________________ mpi-22 mailing list mpi-22_at_[hidden] http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22

_______________________________________________ mpi-22 mailing list mpi-22_at_[hidden] http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22

_______________________________________________ mpi-22 mailing list mpi-22_at_[hidden] http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22

_______________________________________________ mpi-22 mailing list mpi-22_at_[hidden] http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22