March 2009 - Mpi-22 - mailman3.cels.anl.gov

Re: [Mpi-22] Ticket #46: Add const Keyword to theC bindings-implementation
by unknown＠example.com 20 Mar '09

20 Mar '09

Hi Jeff, > The testing that I'm talking about would be writing tests to ensure that > const was put in the right place in all the interfaces -- being able to > repeatably prove that (i.e., regression testing). Be creative -- > perhaps you can use the MPI's mpi.h with a test code that implements > MPI_Send with const in the wrong parameter location. It should fail to > compile, right? Or perhaps it has const in the right parameter location > but then doesn't cast it away and intentionally modifies it. That > should also fail, right? Those are good QA tests. Thanks. With these tests are you testing the MPI implementation or the test program itself? If the application (or test program) uses a const in the wrong place, the compiler will throw a warning/error. And if the user decides to ignore it, there's nothing the MPI implementation can do about it. > What's the *need* to make this 2.2? Why such a strong push to get this > in 2.2? What's the rush? As Brian noted, any MPI implementation can > have const in their C bindings today, regardless of what happens in 2.2. Inertia :-). Since the proposal is already in 2.2, I think the onus is to show an argument to *move* it to 3.0, rather than to keep it in 2.2. -- Pavan -- Pavan Balaji http://www.mcs.anl.gov/~balaji

1 0

Re: [Mpi-22] Ticket #46: Add const Keyword to theC bindings-implementation
by unknown＠example.com 20 Mar '09

20 Mar '09

Is a test suite is part of the requirement to accept this ticket? It if so, we will work it through and make one available. -----Original Message----- From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Jeff Squyres Sent: Friday, March 20, 2009 12:42 PM To: MPI 2.2 Subject: Re: [Mpi-22] Ticket #46: Add const Keyword to theC bindings-implementation On Mar 19, 2009, at 9:20 PM, Pavan Balaji wrote: > > - testing that const is in the right places in the interface is > > *exactly* the right kinds of tests to write. > > Maybe I'm missing something here, but can you give some examples of > tests that would fail with this change that a new and strict compiler > will not be able to catch? Obviously I'm referring to cases which > would > work correctly if this ticket wasn't accepted, but will not work if > it were. > Clearly, an implementor who passed a const variable through a non- const interface would get either a compiler warning or error. If the implementor disregards those messages, well, then I guess the battle is lost. ;-) The testing that I'm talking about would be writing tests to ensure that const was put in the right place in all the interfaces -- being able to repeatably prove that (i.e., regression testing). Be creative -- perhaps you can use the MPI's mpi.h with a test code that implements MPI_Send with const in the wrong parameter location. It should fail to compile, right? Or perhaps it has const in the right parameter location but then doesn't cast it away and intentionally modifies it. That should also fail, right? Those are good QA tests. > Unless there are such examples, all this ticket requires is letting > the > compiler find the problems for you. In fact, some tickets in 2.1 > required more QA, as a compiler was useless in catching any real > problems with them and new tests had to be written (e.g., the change > with respect to 0-dim cartesian topologies, or any other trivial > change). > Sure, I wrote a few tests for this case for Open MPI. We try to write tests for most new major MPI functionality. But the 0-dim cartesian topologies creates a few tests, not 90+ tests. > So, what's the motivation to move this to 3.0? > What's the *need* to make this 2.2? Why such a strong push to get this in 2.2? What's the rush? As Brian noted, any MPI implementation can have const in their C bindings today, regardless of what happens in 2.2. -- Jeff Squyres Cisco Systems _______________________________________________ mpi-22 mailing list mpi-22_at_[hidden] http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22

1 0

Re: [Mpi-22] Ticket #46: Add const Keyword to theC bindings-implementation
by unknown＠example.com 20 Mar '09

20 Mar '09

On Mar 19, 2009, at 9:20 PM, Pavan Balaji wrote: > > - testing that const is in the right places in the interface is > > *exactly* the right kinds of tests to write. > > Maybe I'm missing something here, but can you give some examples of > tests that would fail with this change that a new and strict compiler > will not be able to catch? Obviously I'm referring to cases which > would > work correctly if this ticket wasn't accepted, but will not work if > it were. > Clearly, an implementor who passed a const variable through a non- const interface would get either a compiler warning or error. If the implementor disregards those messages, well, then I guess the battle is lost. ;-) The testing that I'm talking about would be writing tests to ensure that const was put in the right place in all the interfaces -- being able to repeatably prove that (i.e., regression testing). Be creative -- perhaps you can use the MPI's mpi.h with a test code that implements MPI_Send with const in the wrong parameter location. It should fail to compile, right? Or perhaps it has const in the right parameter location but then doesn't cast it away and intentionally modifies it. That should also fail, right? Those are good QA tests. > Unless there are such examples, all this ticket requires is letting > the > compiler find the problems for you. In fact, some tickets in 2.1 > required more QA, as a compiler was useless in catching any real > problems with them and new tests had to be written (e.g., the change > with respect to 0-dim cartesian topologies, or any other trivial > change). > Sure, I wrote a few tests for this case for Open MPI. We try to write tests for most new major MPI functionality. But the 0-dim cartesian topologies creates a few tests, not 90+ tests. > So, what's the motivation to move this to 3.0? > What's the *need* to make this 2.2? Why such a strong push to get this in 2.2? What's the rush? As Brian noted, any MPI implementation can have const in their C bindings today, regardless of what happens in 2.2. -- Jeff Squyres Cisco Systems

1 0

Re: [Mpi-22] Ticket #46: Add const Keyword to the C bindings-implementation
by unknown＠example.com 19 Mar '09

19 Mar '09

Hi Jeff, > - testing that const is in the right places in the interface is > *exactly* the right kinds of tests to write. Maybe I'm missing something here, but can you give some examples of tests that would fail with this change that a new and strict compiler will not be able to catch? Obviously I'm referring to cases which would work correctly if this ticket wasn't accepted, but will not work if it were. Unless there are such examples, all this ticket requires is letting the compiler find the problems for you. In fact, some tickets in 2.1 required more QA, as a compiler was useless in catching any real problems with them and new tests had to be written (e.g., the change with respect to 0-dim cartesian topologies, or any other trivial change). Now, to keep the compiler quiet, whether the implementation directly discards the const at the top-level or carries it on to the lowest possible level (e.g., network interface) is a separate discussion and IMO really a quality of implementation issue. And, obviously, pushing this to 3.0 would not ensure that implementations will make an effort to do better. So, what's the motivation to move this to 3.0? Thanks, -- Pavan -- Pavan Balaji http://www.mcs.anl.gov/~balaji

1 0

Re: [Mpi-22] Ticket #46: Add const Keyword to the C bindings-implementation
by unknown＠example.com 19 Mar '09

19 Mar '09

Thanks Brian, My recollection of the last meeting is that opening tickets #129 & #130 was the forum recommendation. I don't recall anyone saying that we can't vote on #46 if there are any amendments in new tickets. However, I do see your point and I will prepare a new proposal that captures tickets #46, #129 and #130 for the next meeting, where we'll get the forum feedback if we want to proceed with the new ticket or ticket #46. (as it does reset the clock and requires some more work from the forum). I am okay with it either way. Thanks, .Erez -----Original Message----- From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Barrett, Brian W Sent: Thursday, March 19, 2009 3:37 PM To: MPI 2.2 Subject: Re: [Mpi-22] Ticket #46: Add const Keyword to the C bindings-implementation No, there was an objection (rightly) to modifying a proposal that had been voted on. I'm saying remove #46, #129, and #130 from consideration, and start an entirely new proposal from day 1 that is complete. Otherwise, we're setting a horrible precedent that it's ok to vote in things that are known to be wrong, because we can just fix them later. In fact, I believe there was suggestion to do exactly that at the last meeting, but you chose to add new proposals instead. Brian On 3/19/09 16:32 , "Erez Haba" <erezh_at_[hidden]> wrote: > Since we already passed first vote, there was an objection to that in the last > meeting. > > -----Original Message----- > From: mpi-22-bounces_at_[hidden] > [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Barrett, Brian W > Sent: Thursday, March 19, 2009 3:28 PM > To: MPI 2.2 > Subject: Re: [Mpi-22] Ticket #46: Add const Keyword to the C > bindings-implementation > > On 3/19/09 16:00 , "Erez Haba" <erezh_at_[hidden]> wrote: > >> As to rushing this in, I'm curious, this ticket is there for over a year; >> already implemented twice and tested by two different bodies. Is this really >> rushing it in? >> >> As to the procedural issues. This ticket is one of the first tickets that we >> had in the system, and of-course as the mpi forum body we did learn a thing >> or >> two in that process, unfortunately, first implemented with that ticket. So >> yes >> there would be issues. I think that we can resolve these 'issues' but puting >> dependency on passing this ticket given that we accept the other two >> corrections to the ticket. > > We're not too late to submit a new proposal to 2.2, as I understand it. Why > not do the right thing, pull #46, #129, and #130, bundle them into a new > proposal, get comments, and remove the procedural issues? I think the > hesitancy to do this only adds to my worry about this proposal, and is part > of the view people have of "rushing it it". > > Brian > > -- > Brian W. Barrett > Dept. 1423: Scalable System Software > Sandia National Laboratories > > > > _______________________________________________ > mpi-22 mailing list > mpi-22_at_[hidden] > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 > > > _______________________________________________ > mpi-22 mailing list > mpi-22_at_[hidden] > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 > -- Brian W. Barrett Dept. 1423: Scalable System Software Sandia National Laboratories _______________________________________________ mpi-22 mailing list mpi-22_at_[hidden] http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22

1 0

Re: [Mpi-22] Ticket #46: Add const Keyword to the C bindings-implementation
by unknown＠example.com 19 Mar '09

19 Mar '09

Let's not invoke emotion here; there are valid tech and procedural questions that have been raised multiple times. Its always the burden of the proposer to answer such questions - this is no different than any other proposal. Indeed, I think torsten's nbc proposal received *much* more scrutiny than this. - This may have been one of the first / oldest tickets, but it sat there for a long time with no action from you until we're nearly out of time for mpi 22. Using the "its a year old" argument isn't valid. - an open source implementation is *required* for mpi 22. - testing that const is in the right places in the interface is *exactly* the right kinds of tests to write. -jms Sent from my PDA. No type good. ----- Original Message ----- From: mpi-22-bounces_at_[hidden] <mpi-22-bounces_at_[hidden]> To: MPI 2.2 <mpi-22_at_[hidden]> Sent: Thu Mar 19 18:00:33 2009 Subject: Re: [Mpi-22] Ticket #46: Add const Keyword to the C bindings-implementation Terry, I'm truly sorry that you find the information that I provide as unreliable. I know that I'm not part of the 'trusted' OpenMPI clan and I wish that there was something I could do to convince you otherwise. If you recall the last forum meeting, I did a complete implementation of the 'const' ticket during that meeting. (and did let everyone know about it). That was an implementation from scratch that did trickle down the constness to all the internal functions of MSMPI. Indeed I did not do any runtime testing at that meeting, it did happen later. However, since MSMPI is not an open source project, I asked the friends at ANL to re-implement it over MPICH2 and make the source available. Since you might not trust my word, you may ask Rajeev and other friends at ANL about this. The tests conducted were checking for regression. Hence, any program that compiled and run correctly before the const change did compile and run correctly after the const change; with multiple compiler and multiple operating systems. We did not conduct any direct tests that are checking the 'constness' of the interface. I don't really know how you check for constness? Do you check that the buffer was not modified? Well that would be testing ticket #45 (send buffer access) and not ticket #46. Do you check that you pass const in and you don't get a compiler error? that relatively simple and I don't think it buys you much (except it verifies that you put the const in all the right places on the interface API). As to rushing this in, I'm curious, this ticket is there for over a year; already implemented twice and tested by two different bodies. Is this really rushing it in? As to the procedural issues. This ticket is one of the first tickets that we had in the system, and of-course as the mpi forum body we did learn a thing or two in that process, unfortunately, first implemented with that ticket. So yes there would be issues. I think that we can resolve these 'issues' but puting dependency on passing this ticket given that we accept the other two corrections to the ticket. Thanks, .Erez -----Original Message----- From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Terry Dontje Sent: Thursday, March 19, 2009 9:15 AM To: MPI 2.2 Subject: Re: [Mpi-22] Ticket #46: Add const Keywordto the Cbindings-implementation I am not Jeff and I do not play him on tv but I felt the need to interject my 2 cents. Erez Haba wrote: > Jeff, I don't understand why you ignore all the work that has been done with this ticket and suggest your own metric. Was the information I provided unreliable or untrue? > > Possibly but it definitely is biased IMO. > As I said, it took me about 2.5h to implement the entire change on the MSMPI code base; yes there were many internal functions that had const in it, but also many that did not. I don't think that by any measure this is a significant work for any implementer. > > Isn't MSMPI based on MPICH? If so how much of the previous work done to support const did you leverage or did you truly reinvent the same wheel MPICH developers did? I guess I find it pretty amazing the applying 10,000 lines of changes and then validating them took you 2.5 hours. > More than that, this interface change was tested using the ANL test suite and the Intel complete test suite, without a single compile or runtime error over multiple platforms and compilers. Again, this was not a significant work item; and I believe that OpenMPI has a regression suite that crosses platform and compilers which is easy to run (as you described to me many times). > > > Do the ANL tests and any others really test const-ness without changes. Or are you saying the ANL tests already do explicit testing of const-ness on the right parameters? > With this information that I have already provided before, I don't understand your claims; unless you think that they are biased and unreliable. > > I would say your information is a useful datapoint but I am curious if what you did really was the full conversion of your MPI implementation to handle this change. > Hence this ticket completely meets MPI 2.2 requirements in scope and backward compatibility. > > For MPICH based implementations I agree it does. For Open MPI I think it is still questionable. I guess I am still interested in the reasoning why shoving this ticket into 2.2 is such a high priority? --td > Thanks, > .Erez > > -----Original Message----- > From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Jeff Squyres > Sent: Thursday, March 19, 2009 5:20 AM > To: Bronis R. de Supinski; MPI 2.2 > Subject: Re: [Mpi-22] Ticket #46: Add const Keywordto the Cbindings-implementation > > On Mar 19, 2009, at 12:03 AM, Bronis R. de Supinski wrote: > > >> I have been quiet on this debate but I am to the point >> that I feel a need to speak up. I think the "significant >> changes" reason is a canard (I think Brian and Jeff >> believe it but it is still a red herring for the most >> part). The changes to support it are not significant. >> You can implement it by casting it away early. >> >> > > > The changes *are* significant. I count 90 top-level MPI API functions > that are affected by this proposal. > > The bar for 2.2 is "trivial" implementation efforts, no? This is not > trivial. The MPICH diff was ~10,000 lines. Apparently, they chose to > go above and beyond the requirement and propagate const to lower > layers (vs. casting it away immediately), but still -- if a 2.2 change > can inspire a 10,000 line diff, that just seems like it doesn't meet > the spirit of the "trivial" implementation requirement. > > Indeed, in doing the "simple" cast-it-away-immediately approach > implementation, editing and testing each of the 90 functions will > require a good amount of effort. Sure, the first-wave addition of > "const" across all of the affected functions is probably relatively > easy -- one code monkey and several hours. But how long will it take > you to find the one place where you put "const" in the wrong > location? How long will it take you to write new tests to verify the > const-ness in all the right places? To be clear: it's a "not > significant" change to a code monkey. But try to tell your QA guys > that this is not significant -- they'll freak out. > > Hence, if you take the overall scope of both the change and the > additional testing to validate this change, this is a large scale > implementation effort. I just don't see what the rush is to get this > into 2.2. > > I think Brian makes several good arguments, one of which is: those who > want const can put it in today. It hypothetically shouldn't break any > existing user codes (this is the code monkey in me talking, not the QA > rep), and those implementors who want it for stronger contracts can > have it. > > _______________________________________________ mpi-22 mailing list mpi-22_at_[hidden] http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 _______________________________________________ mpi-22 mailing list mpi-22_at_[hidden] http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 *

1 0

Re: [Mpi-22] Ticket #46: Add const Keyword to the C bindings-implementation
by unknown＠example.com 19 Mar '09

19 Mar '09

No, there was an objection (rightly) to modifying a proposal that had been voted on. I'm saying remove #46, #129, and #130 from consideration, and start an entirely new proposal from day 1 that is complete. Otherwise, we're setting a horrible precedent that it's ok to vote in things that are known to be wrong, because we can just fix them later. In fact, I believe there was suggestion to do exactly that at the last meeting, but you chose to add new proposals instead. Brian On 3/19/09 16:32 , "Erez Haba" <erezh_at_[hidden]> wrote: > Since we already passed first vote, there was an objection to that in the last > meeting. > > -----Original Message----- > From: mpi-22-bounces_at_[hidden] > [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Barrett, Brian W > Sent: Thursday, March 19, 2009 3:28 PM > To: MPI 2.2 > Subject: Re: [Mpi-22] Ticket #46: Add const Keyword to the C > bindings-implementation > > On 3/19/09 16:00 , "Erez Haba" <erezh_at_[hidden]> wrote: > >> As to rushing this in, I'm curious, this ticket is there for over a year; >> already implemented twice and tested by two different bodies. Is this really >> rushing it in? >> >> As to the procedural issues. This ticket is one of the first tickets that we >> had in the system, and of-course as the mpi forum body we did learn a thing >> or >> two in that process, unfortunately, first implemented with that ticket. So >> yes >> there would be issues. I think that we can resolve these 'issues' but puting >> dependency on passing this ticket given that we accept the other two >> corrections to the ticket. > > We're not too late to submit a new proposal to 2.2, as I understand it. Why > not do the right thing, pull #46, #129, and #130, bundle them into a new > proposal, get comments, and remove the procedural issues? I think the > hesitancy to do this only adds to my worry about this proposal, and is part > of the view people have of "rushing it it". > > Brian > > -- > Brian W. Barrett > Dept. 1423: Scalable System Software > Sandia National Laboratories > > > > _______________________________________________ > mpi-22 mailing list > mpi-22_at_[hidden] > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 > > > _______________________________________________ > mpi-22 mailing list > mpi-22_at_[hidden] > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 > -- Brian W. Barrett Dept. 1423: Scalable System Software Sandia National Laboratories

1 0

Re: [Mpi-22] Ticket #46: Add const Keyword to the C bindings-implementation
by unknown＠example.com 19 Mar '09

19 Mar '09

That is incorrect. There was an objection to changing #46 and claiming that it already had its first vote. Brian is proposing that you bundle the fixes in a new proposal and have a first reading again. It sounded like there were yet more additions that he was proposing. Keith >-----Original Message----- >From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_lists.mpi- >forum.org] On Behalf Of Erez Haba >Sent: Thursday, March 19, 2009 4:32 PM >To: MPI 2.2 >Subject: Re: [Mpi-22] Ticket #46: Add const Keyword to the C bindings- >implementation > >Since we already passed first vote, there was an objection to that in the >last meeting. > >-----Original Message----- >From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_lists.mpi- >forum.org] On Behalf Of Barrett, Brian W >Sent: Thursday, March 19, 2009 3:28 PM >To: MPI 2.2 >Subject: Re: [Mpi-22] Ticket #46: Add const Keyword to the C bindings- >implementation > >On 3/19/09 16:00 , "Erez Haba" <erezh_at_[hidden]> wrote: > >> As to rushing this in, I'm curious, this ticket is there for over a year; >> already implemented twice and tested by two different bodies. Is this >really >> rushing it in? >> >> As to the procedural issues. This ticket is one of the first tickets that >we >> had in the system, and of-course as the mpi forum body we did learn a >thing or >> two in that process, unfortunately, first implemented with that ticket. >So yes >> there would be issues. I think that we can resolve these 'issues' but >puting >> dependency on passing this ticket given that we accept the other two >> corrections to the ticket. > >We're not too late to submit a new proposal to 2.2, as I understand it. >Why >not do the right thing, pull #46, #129, and #130, bundle them into a new >proposal, get comments, and remove the procedural issues? I think the >hesitancy to do this only adds to my worry about this proposal, and is part >of the view people have of "rushing it it". > >Brian > >-- > Brian W. Barrett > Dept. 1423: Scalable System Software > Sandia National Laboratories > > > >_______________________________________________ >mpi-22 mailing list >mpi-22_at_[hidden] >http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 > > >_______________________________________________ >mpi-22 mailing list >mpi-22_at_[hidden] >http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22

1 0

Re: [Mpi-22] Ticket #46: Add const Keyword to the C bindings-implementation
by unknown＠example.com 19 Mar '09

19 Mar '09

Since we already passed first vote, there was an objection to that in the last meeting. -----Original Message----- From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Barrett, Brian W Sent: Thursday, March 19, 2009 3:28 PM To: MPI 2.2 Subject: Re: [Mpi-22] Ticket #46: Add const Keyword to the C bindings-implementation On 3/19/09 16:00 , "Erez Haba" <erezh_at_[hidden]> wrote: > As to rushing this in, I'm curious, this ticket is there for over a year; > already implemented twice and tested by two different bodies. Is this really > rushing it in? > > As to the procedural issues. This ticket is one of the first tickets that we > had in the system, and of-course as the mpi forum body we did learn a thing or > two in that process, unfortunately, first implemented with that ticket. So yes > there would be issues. I think that we can resolve these 'issues' but puting > dependency on passing this ticket given that we accept the other two > corrections to the ticket. We're not too late to submit a new proposal to 2.2, as I understand it. Why not do the right thing, pull #46, #129, and #130, bundle them into a new proposal, get comments, and remove the procedural issues? I think the hesitancy to do this only adds to my worry about this proposal, and is part of the view people have of "rushing it it". Brian -- Brian W. Barrett Dept. 1423: Scalable System Software Sandia National Laboratories _______________________________________________ mpi-22 mailing list mpi-22_at_[hidden] http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22

1 0

Re: [Mpi-22] Ticket #46: Add const Keyword to the C bindings-implementation
by unknown＠example.com 19 Mar '09

19 Mar '09

On 3/19/09 16:00 , "Erez Haba" <erezh_at_[hidden]> wrote: > As to rushing this in, I'm curious, this ticket is there for over a year; > already implemented twice and tested by two different bodies. Is this really > rushing it in? > > As to the procedural issues. This ticket is one of the first tickets that we > had in the system, and of-course as the mpi forum body we did learn a thing or > two in that process, unfortunately, first implemented with that ticket. So yes > there would be issues. I think that we can resolve these 'issues' but puting > dependency on passing this ticket given that we accept the other two > corrections to the ticket. We're not too late to submit a new proposal to 2.2, as I understand it. Why not do the right thing, pull #46, #129, and #130, bundle them into a new proposal, get comments, and remove the procedural issues? I think the hesitancy to do this only adds to my worry about this proposal, and is part of the view people have of "rushing it it". Brian -- Brian W. Barrett Dept. 1423: Scalable System Software Sandia National Laboratories

1 0