- Mpi-22 - mailman3.cels.anl.gov

Re: [Mpi-22] Ticket #46: Add const Keywordto theC bindings-implementation
by unknown＠example.com 23 Mar '09

23 Mar '09

I have a stupid question: (*) Is it not required to write tests for all these interfaces (and internal protocols) where the (const) send buffer is declared and allocated as const also in the test program, so that the compiler is able to put the send buffer in special memory regions ? All other tests seem to be only compile tests (and not run tests) whether the internal interfaces are correctly declared. I think that the compiler doesn't have any possibility for optimization within the MPI library (and generating runtime problems with the const interface) since send buffer regions could be used as not const at any other location. Hubert Jeff Squyres wrote: > On Mar 20, 2009, at 4:32 PM, Pavan Balaji wrote: > >> > The testing that I'm talking about would be writing tests to ensure >> that >> > const was put in the right place in all the interfaces -- being >> able to >> > repeatably prove that (i.e., regression testing). Be creative -- >> > perhaps you can use the MPI's mpi.h with a test code that implements >> > MPI_Send with const in the wrong parameter location. It should >> fail to >> > compile, right? Or perhaps it has const in the right parameter >> location >> > but then doesn't cast it away and intentionally modifies it. That >> > should also fail, right? Those are good QA tests. >> >> Thanks. With these tests are you testing the MPI implementation or the >> test program itself? If the application (or test program) uses a const >> in the wrong place, the compiler will throw a warning/error. And if the >> user decides to ignore it, there's nothing the MPI implementation can do >> about it. >> > > I was giving a hypothetical test, probably with a big bias of my own > MPI implementation. In Open MPI, the same mpi.h is used internally as > externally. So a valid test could be to compile an application (or > even just an object file) that "implements" MPI_Send, correctly and/or > incorrectly. Since the mpi.h header file is the same as the one that > is used internally, you can know that if your test MPI_Send > "implementation" is wrong and still compiles ok, then your mpi.h is > wrong. If your test is wrong and it doesn't compile, that's good > (although it obviously doesn't necessarily mean that the mpi.h is > correct). > >> > What's the *need* to make this 2.2? Why such a strong push to get >> this >> > in 2.2? What's the rush? As Brian noted, any MPI implementation can >> > have const in their C bindings today, regardless of what happens in >> 2.2. >> >> Inertia :-). Since the proposal is already in 2.2, I think the onus is >> to show an argument to *move* it to 3.0, rather than to keep it in 2.2. >> > > > By this logic, non-blocking collectives should be in 2.2 as well -- > they have plenty of inertia! :-) > > I think the onus should be to be careful and to get it right. We do > strive for consensus in the Forum whenever possible. The procedural > questions are being addressed (thanks, Erez!), but at least two > technical questions remain: > > - I (and others) have given examples why I (we) think this fails the > "trivial" 2.2 implementation criteria > - I (and others) have cited the disparity of lack of "const" in the > callback function typedefs > > --Jeff Squyres > Cisco Systems > > _______________________________________________ > mpi-22 mailing list > mpi-22_at_[hidden] > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 > *

1 0

Re: [Mpi-22] Ticket #46: Add const Keywordto the C bindings-implementation
by unknown＠example.com 21 Mar '09

21 Mar '09

One reason to deal with this issue in 2.2 versus 3 is that if we decide one way or the other now itself (whether const should or should not be in the MPI interface), it helps all other interfaces being defined in MPI 3 to know now itself whether they need to include const in the right places. If we delay until 3, the new interfaces will have to go back and add const at the last minute, or keep 2 sets of interfaces concurrently active until the end -- with const and without const. Regarding implementation, if you start with the mpi.h provided in the ticket and let the compiler catch you, it can be done (although it is still tedious). How far down you propagate const depends on how much time you are willing to spend. Rajeev

1 0

Re: [Mpi-22] Ticket #46: Add const Keywordto theC bindings-implementation
by unknown＠example.com 20 Mar '09

20 Mar '09

On Mar 20, 2009, at 4:32 PM, Pavan Balaji wrote: > > The testing that I'm talking about would be writing tests to > ensure that > > const was put in the right place in all the interfaces -- being > able to > > repeatably prove that (i.e., regression testing). Be creative -- > > perhaps you can use the MPI's mpi.h with a test code that implements > > MPI_Send with const in the wrong parameter location. It should > fail to > > compile, right? Or perhaps it has const in the right parameter > location > > but then doesn't cast it away and intentionally modifies it. That > > should also fail, right? Those are good QA tests. > > Thanks. With these tests are you testing the MPI implementation or the > test program itself? If the application (or test program) uses a const > in the wrong place, the compiler will throw a warning/error. And if > the > user decides to ignore it, there's nothing the MPI implementation > can do > about it. > I was giving a hypothetical test, probably with a big bias of my own MPI implementation. In Open MPI, the same mpi.h is used internally as externally. So a valid test could be to compile an application (or even just an object file) that "implements" MPI_Send, correctly and/or incorrectly. Since the mpi.h header file is the same as the one that is used internally, you can know that if your test MPI_Send "implementation" is wrong and still compiles ok, then your mpi.h is wrong. If your test is wrong and it doesn't compile, that's good (although it obviously doesn't necessarily mean that the mpi.h is correct). > > What's the *need* to make this 2.2? Why such a strong push to get > this > > in 2.2? What's the rush? As Brian noted, any MPI implementation > can > > have const in their C bindings today, regardless of what happens > in 2.2. > > Inertia :-). Since the proposal is already in 2.2, I think the onus is > to show an argument to *move* it to 3.0, rather than to keep it in > 2.2. > By this logic, non-blocking collectives should be in 2.2 as well -- they have plenty of inertia! :-) I think the onus should be to be careful and to get it right. We do strive for consensus in the Forum whenever possible. The procedural questions are being addressed (thanks, Erez!), but at least two technical questions remain: - I (and others) have given examples why I (we) think this fails the "trivial" 2.2 implementation criteria - I (and others) have cited the disparity of lack of "const" in the callback function typedefs -- Jeff Squyres Cisco Systems

1 0

Re: [Mpi-22] Ticket #46: Add const Keyword to theCbindings-implementation
by unknown＠example.com 20 Mar '09

20 Mar '09

There is no test suite requirement, only the availability of an implementation that can be examined. Rich ----- Original Message ----- From: mpi-22-bounces_at_[hidden] <mpi-22-bounces_at_[hidden]> To: MPI 2.2 <mpi-22_at_[hidden]> Sent: Fri Mar 20 16:26:11 2009 Subject: Re: [Mpi-22] Ticket #46: Add const Keyword to theCbindings-implementation Is a test suite is part of the requirement to accept this ticket? It if so, we will work it through and make one available. -----Original Message----- From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Jeff Squyres Sent: Friday, March 20, 2009 12:42 PM To: MPI 2.2 Subject: Re: [Mpi-22] Ticket #46: Add const Keyword to theC bindings-implementation On Mar 19, 2009, at 9:20 PM, Pavan Balaji wrote: > > - testing that const is in the right places in the interface is > > *exactly* the right kinds of tests to write. > > Maybe I'm missing something here, but can you give some examples of > tests that would fail with this change that a new and strict compiler > will not be able to catch? Obviously I'm referring to cases which > would > work correctly if this ticket wasn't accepted, but will not work if > it were. > Clearly, an implementor who passed a const variable through a non- const interface would get either a compiler warning or error. If the implementor disregards those messages, well, then I guess the battle is lost. ;-) The testing that I'm talking about would be writing tests to ensure that const was put in the right place in all the interfaces -- being able to repeatably prove that (i.e., regression testing). Be creative -- perhaps you can use the MPI's mpi.h with a test code that implements MPI_Send with const in the wrong parameter location. It should fail to compile, right? Or perhaps it has const in the right parameter location but then doesn't cast it away and intentionally modifies it. That should also fail, right? Those are good QA tests. > Unless there are such examples, all this ticket requires is letting > the > compiler find the problems for you. In fact, some tickets in 2.1 > required more QA, as a compiler was useless in catching any real > problems with them and new tests had to be written (e.g., the change > with respect to 0-dim cartesian topologies, or any other trivial > change). > Sure, I wrote a few tests for this case for Open MPI. We try to write tests for most new major MPI functionality. But the 0-dim cartesian topologies creates a few tests, not 90+ tests. > So, what's the motivation to move this to 3.0? > What's the *need* to make this 2.2? Why such a strong push to get this in 2.2? What's the rush? As Brian noted, any MPI implementation can have const in their C bindings today, regardless of what happens in 2.2. -- Jeff Squyres Cisco Systems _______________________________________________ mpi-22 mailing list mpi-22_at_[hidden] http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 _______________________________________________ mpi-22 mailing list mpi-22_at_[hidden] http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22

1 0

Re: [Mpi-22] Ticket #46: Add const Keyword to the C bindings-implementation
by unknown＠example.com 20 Mar '09

20 Mar '09

There was some confusion about what exactly I implemented in those 2.5h in the last forum meeting and how I implemented it. I would like to clarify this confusion and assure you that I am no super-dev of any sort. What I did was simple, I walked all the functions in mpi.h and added (copied) the const from ticket #45. There was scripting, awk or perl involved it was all manual. Once I was done with the header I compiled; the compiler gave me a bunch of errors, saying that the impl does not match the function prototype. In my environment I can quickly go from a compiler error to the source code. I fixed the MPI function impl to copy the same const to the function parameter and compiled again. The compiler gave me a bunch of other errors about internal functions and/or variable assignment. Again I fixed those and recompiled. In very few places I had to do something sophisticated like casting, or changing a local pointer variable to be a pointer to a const. that's it! I repeated that process until I had no errors. Indeed that was just a proof of concept; I used only one compiler in that case and only one platform. Which is why it was only 2.5h, and which is why it took longer (2d) for Pavan to implement over multiple compilers, platforms, test and submit it to the branch. Hope that this helps, .Erez

1 0

Re: [Mpi-22] Ticket #46: Add const Keyword to theC bindings-implementation
by unknown＠example.com 20 Mar '09

20 Mar '09

Hi Jeff, > The testing that I'm talking about would be writing tests to ensure that > const was put in the right place in all the interfaces -- being able to > repeatably prove that (i.e., regression testing). Be creative -- > perhaps you can use the MPI's mpi.h with a test code that implements > MPI_Send with const in the wrong parameter location. It should fail to > compile, right? Or perhaps it has const in the right parameter location > but then doesn't cast it away and intentionally modifies it. That > should also fail, right? Those are good QA tests. Thanks. With these tests are you testing the MPI implementation or the test program itself? If the application (or test program) uses a const in the wrong place, the compiler will throw a warning/error. And if the user decides to ignore it, there's nothing the MPI implementation can do about it. > What's the *need* to make this 2.2? Why such a strong push to get this > in 2.2? What's the rush? As Brian noted, any MPI implementation can > have const in their C bindings today, regardless of what happens in 2.2. Inertia :-). Since the proposal is already in 2.2, I think the onus is to show an argument to *move* it to 3.0, rather than to keep it in 2.2. -- Pavan -- Pavan Balaji http://www.mcs.anl.gov/~balaji

1 0

Re: [Mpi-22] Ticket #46: Add const Keyword to theC bindings-implementation
by unknown＠example.com 20 Mar '09

20 Mar '09

Is a test suite is part of the requirement to accept this ticket? It if so, we will work it through and make one available. -----Original Message----- From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Jeff Squyres Sent: Friday, March 20, 2009 12:42 PM To: MPI 2.2 Subject: Re: [Mpi-22] Ticket #46: Add const Keyword to theC bindings-implementation On Mar 19, 2009, at 9:20 PM, Pavan Balaji wrote: > > - testing that const is in the right places in the interface is > > *exactly* the right kinds of tests to write. > > Maybe I'm missing something here, but can you give some examples of > tests that would fail with this change that a new and strict compiler > will not be able to catch? Obviously I'm referring to cases which > would > work correctly if this ticket wasn't accepted, but will not work if > it were. > Clearly, an implementor who passed a const variable through a non- const interface would get either a compiler warning or error. If the implementor disregards those messages, well, then I guess the battle is lost. ;-) The testing that I'm talking about would be writing tests to ensure that const was put in the right place in all the interfaces -- being able to repeatably prove that (i.e., regression testing). Be creative -- perhaps you can use the MPI's mpi.h with a test code that implements MPI_Send with const in the wrong parameter location. It should fail to compile, right? Or perhaps it has const in the right parameter location but then doesn't cast it away and intentionally modifies it. That should also fail, right? Those are good QA tests. > Unless there are such examples, all this ticket requires is letting > the > compiler find the problems for you. In fact, some tickets in 2.1 > required more QA, as a compiler was useless in catching any real > problems with them and new tests had to be written (e.g., the change > with respect to 0-dim cartesian topologies, or any other trivial > change). > Sure, I wrote a few tests for this case for Open MPI. We try to write tests for most new major MPI functionality. But the 0-dim cartesian topologies creates a few tests, not 90+ tests. > So, what's the motivation to move this to 3.0? > What's the *need* to make this 2.2? Why such a strong push to get this in 2.2? What's the rush? As Brian noted, any MPI implementation can have const in their C bindings today, regardless of what happens in 2.2. -- Jeff Squyres Cisco Systems _______________________________________________ mpi-22 mailing list mpi-22_at_[hidden] http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22

1 0

Re: [Mpi-22] Ticket #46: Add const Keyword to theC bindings-implementation
by unknown＠example.com 20 Mar '09

20 Mar '09

On Mar 19, 2009, at 9:20 PM, Pavan Balaji wrote: > > - testing that const is in the right places in the interface is > > *exactly* the right kinds of tests to write. > > Maybe I'm missing something here, but can you give some examples of > tests that would fail with this change that a new and strict compiler > will not be able to catch? Obviously I'm referring to cases which > would > work correctly if this ticket wasn't accepted, but will not work if > it were. > Clearly, an implementor who passed a const variable through a non- const interface would get either a compiler warning or error. If the implementor disregards those messages, well, then I guess the battle is lost. ;-) The testing that I'm talking about would be writing tests to ensure that const was put in the right place in all the interfaces -- being able to repeatably prove that (i.e., regression testing). Be creative -- perhaps you can use the MPI's mpi.h with a test code that implements MPI_Send with const in the wrong parameter location. It should fail to compile, right? Or perhaps it has const in the right parameter location but then doesn't cast it away and intentionally modifies it. That should also fail, right? Those are good QA tests. > Unless there are such examples, all this ticket requires is letting > the > compiler find the problems for you. In fact, some tickets in 2.1 > required more QA, as a compiler was useless in catching any real > problems with them and new tests had to be written (e.g., the change > with respect to 0-dim cartesian topologies, or any other trivial > change). > Sure, I wrote a few tests for this case for Open MPI. We try to write tests for most new major MPI functionality. But the 0-dim cartesian topologies creates a few tests, not 90+ tests. > So, what's the motivation to move this to 3.0? > What's the *need* to make this 2.2? Why such a strong push to get this in 2.2? What's the rush? As Brian noted, any MPI implementation can have const in their C bindings today, regardless of what happens in 2.2. -- Jeff Squyres Cisco Systems

1 0

Re: [Mpi-22] Ticket #46: Add const Keyword to the C bindings-implementation
by unknown＠example.com 19 Mar '09

19 Mar '09

Hi Jeff, > - testing that const is in the right places in the interface is > *exactly* the right kinds of tests to write. Maybe I'm missing something here, but can you give some examples of tests that would fail with this change that a new and strict compiler will not be able to catch? Obviously I'm referring to cases which would work correctly if this ticket wasn't accepted, but will not work if it were. Unless there are such examples, all this ticket requires is letting the compiler find the problems for you. In fact, some tickets in 2.1 required more QA, as a compiler was useless in catching any real problems with them and new tests had to be written (e.g., the change with respect to 0-dim cartesian topologies, or any other trivial change). Now, to keep the compiler quiet, whether the implementation directly discards the const at the top-level or carries it on to the lowest possible level (e.g., network interface) is a separate discussion and IMO really a quality of implementation issue. And, obviously, pushing this to 3.0 would not ensure that implementations will make an effort to do better. So, what's the motivation to move this to 3.0? Thanks, -- Pavan -- Pavan Balaji http://www.mcs.anl.gov/~balaji

1 0

Re: [Mpi-22] Ticket #46: Add const Keyword to the C bindings-implementation
by unknown＠example.com 19 Mar '09

19 Mar '09

Thanks Brian, My recollection of the last meeting is that opening tickets #129 & #130 was the forum recommendation. I don't recall anyone saying that we can't vote on #46 if there are any amendments in new tickets. However, I do see your point and I will prepare a new proposal that captures tickets #46, #129 and #130 for the next meeting, where we'll get the forum feedback if we want to proceed with the new ticket or ticket #46. (as it does reset the clock and requires some more work from the forum). I am okay with it either way. Thanks, .Erez -----Original Message----- From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Barrett, Brian W Sent: Thursday, March 19, 2009 3:37 PM To: MPI 2.2 Subject: Re: [Mpi-22] Ticket #46: Add const Keyword to the C bindings-implementation No, there was an objection (rightly) to modifying a proposal that had been voted on. I'm saying remove #46, #129, and #130 from consideration, and start an entirely new proposal from day 1 that is complete. Otherwise, we're setting a horrible precedent that it's ok to vote in things that are known to be wrong, because we can just fix them later. In fact, I believe there was suggestion to do exactly that at the last meeting, but you chose to add new proposals instead. Brian On 3/19/09 16:32 , "Erez Haba" <erezh_at_[hidden]> wrote: > Since we already passed first vote, there was an objection to that in the last > meeting. > > -----Original Message----- > From: mpi-22-bounces_at_[hidden] > [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Barrett, Brian W > Sent: Thursday, March 19, 2009 3:28 PM > To: MPI 2.2 > Subject: Re: [Mpi-22] Ticket #46: Add const Keyword to the C > bindings-implementation > > On 3/19/09 16:00 , "Erez Haba" <erezh_at_[hidden]> wrote: > >> As to rushing this in, I'm curious, this ticket is there for over a year; >> already implemented twice and tested by two different bodies. Is this really >> rushing it in? >> >> As to the procedural issues. This ticket is one of the first tickets that we >> had in the system, and of-course as the mpi forum body we did learn a thing >> or >> two in that process, unfortunately, first implemented with that ticket. So >> yes >> there would be issues. I think that we can resolve these 'issues' but puting >> dependency on passing this ticket given that we accept the other two >> corrections to the ticket. > > We're not too late to submit a new proposal to 2.2, as I understand it. Why > not do the right thing, pull #46, #129, and #130, bundle them into a new > proposal, get comments, and remove the procedural issues? I think the > hesitancy to do this only adds to my worry about this proposal, and is part > of the view people have of "rushing it it". > > Brian > > -- > Brian W. Barrett > Dept. 1423: Scalable System Software > Sandia National Laboratories > > > > _______________________________________________ > mpi-22 mailing list > mpi-22_at_[hidden] > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 > > > _______________________________________________ > mpi-22 mailing list > mpi-22_at_[hidden] > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 > -- Brian W. Barrett Dept. 1423: Scalable System Software Sandia National Laboratories _______________________________________________ mpi-22 mailing list mpi-22_at_[hidden] http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22

1 0