- Mpi3-bwcompat - mailman3.cels.anl.gov

Re: [Mpi3-bwcompat] Notes from ATL Forum meeting
by unknown＠example.com 05 Feb '10

05 Feb '10

On Jan 22, 2010, at 1:50 PM, Jeff Squyres (jsquyres) wrote: > * What to do about the proposed MPI-3 explicit Fortran bindings (the > so-called "F03 bindings")? We could use function overloading and > forego the suffix approach that is necessary in C. But this was > deemed to be a bad idea -- it seemed better to keep function name > parity with C. So have both an MPI_File_write with int count and > MPI_File_writeL with MPI_Count count in the "F03" bindings. One more point here: there should be no "integer" version of MPI_FILE_WRITE in the F03 bindings because they're new. This follows the same precedent of "no int versions of non-blocking collectives". There should only be MPI_FILE_WRITE<suffix> in the F03 bindings that takes an argument of the MPI_COUNT variety (exact type tbd; probably a KIND sort of thing). -- Jeff Squyres <jsquyres_at_[hidden]> Cisco.com - http://www.cisco.com For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/

1 0

ABI compat list from Intel?
by unknown＠example.com 05 Feb '10

05 Feb '10

Alexander -- I seem to recall you mentioning that you had a list of 10-12 items that affect ABI from the MPI-2.x series and/or MPI-3 proposals. Did you ever share that list with the group? -- Jeff Squyres <jsquyres_at_[hidden]> Cisco.com - http://www.cisco.com For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/

1 0

Re: [Mpi3-bwcompat] Notes from ATL Forum meeting
by unknown＠example.com 26 Jan '10

26 Jan '10

On Jan 26, 2010, at 9:38 AM, Jeff Squyres wrote: > On Jan 26, 2010, at 8:39 AM, Quincey Koziol wrote: > >> I think we need to pick a "preferred vector" through the name combinations for each version of the standard and use a feature-neutral suffix for the changed routines' names. Either a totally abstract sequential suffix like: (nothing)->2->3->4, or a standard version specific suffix: (nothing)->3->31->32->...->4->41->... > > I think the single digits would be fine. > > We have a precedent of keeping compatibility between minor MPI revs and only breaking compatibility if something is really broken. I.e., if we make MPI_Foo3 and then find out that it's broken, I think the gut reaction of the Forum would be to fix MPI_Foo3, not make MPI_Foo31. I agree, I was just trying to make the sequences look [more] different. :-) The main difference is that the abstract sequence will never skip a value, whereas the version specific sequence could skip values (if a symbol was changed in version 3 of the standard, was the same in version 4 and changed again in version 5, etc.) Quincey

1 0

Re: [Mpi3-bwcompat] Notes from ATL Forum meeting
by unknown＠example.com 26 Jan '10

26 Jan '10

On Jan 26, 2010, at 8:39 AM, Quincey Koziol wrote: > I think we need to pick a "preferred vector" through the name combinations for each version of the standard and use a feature-neutral suffix for the changed routines' names. Either a totally abstract sequential suffix like: (nothing)->2->3->4, or a standard version specific suffix: (nothing)->3->31->32->...->4->41->... I think the single digits would be fine. We have a precedent of keeping compatibility between minor MPI revs and only breaking compatibility if something is really broken. I.e., if we make MPI_Foo3 and then find out that it's broken, I think the gut reaction of the Forum would be to fix MPI_Foo3, not make MPI_Foo31. -- Jeff Squyres jsquyres_at_[hidden]

1 0

Re: [Mpi3-bwcompat] Notes from ATL Forum meeting
by unknown＠example.com 26 Jan '10

26 Jan '10

Hi Jeff, On Jan 22, 2010, at 12:50 PM, Jeff Squyres wrote: > We had a lively meeting in ATL for the backwards compatibility issues. I took a set of slides from Dave, beefed them up a bit with my own and a bunch of Quincey's thoughts, presented them, and took some of the feedback from the session and put it back in the slides before posting them here (I was rushed a bit when making the slides; they were a little unclear on some points): > > http://meetings.mpi-forum.org/secretary/2010/01/slides.php > > We had a good 1.5-2 hour discussion on the whole set of issues. The exact discussion isn't important; here's the key takeaways (also on the wiki: https://svn.mpi-forum.org/trac/mpi-forum-web/wiki/BackCompatMeetings) Looks like a good summary, thanks for taking notes! > * Make new functions with a suffix (e.g., MPI_File_write_L -- exact > suffix TBD) that take MPI_Count count parameters. I think we (our working group) need to press for a "preferred vector" through the "name dimensions" as we make changes to the MPI API routine names, otherwise we are going to end up with a combinatorial explosion of names that should be implemented (or very confusing names). (Not as bad as the combinatorial explosion for the F90 wrappers, but along the same lines) For example, if we use "_L" as the suffix for the MPI_Count change, and then need another suffix (say "_A") for another change, are we going to have all four combinations of routine names? (MPI_Foo, MPI_Foo_L, MPI_Foo_A and MPI_Foo_L_A) What would we do when we made another change and added another suffix, have eight variants of the MPI_Foo routine? I think we need to pick a "preferred vector" through the name combinations for each version of the standard and use a feature-neutral suffix for the changed routines' names. Either a totally abstract sequential suffix like: (nothing)->2->3->4, or a standard version specific suffix: (nothing)->3->31->32->...->4->41->... What you you guys think? Quincey

1 0

Notes from ATL Forum meeting
by unknown＠example.com 22 Jan '10

22 Jan '10

We had a lively meeting in ATL for the backwards compatibility issues. I took a set of slides from Dave, beefed them up a bit with my own and a bunch of Quincey's thoughts, presented them, and took some of the feedback from the session and put it back in the slides before posting them here (I was rushed a bit when making the slides; they were a little unclear on some points): http://meetings.mpi-forum.org/secretary/2010/01/slides.php We had a good 1.5-2 hour discussion on the whole set of issues. The exact discussion isn't important; here's the key takeaways (also on the wiki: https://svn.mpi-forum.org/trac/mpi-forum-web/wiki/BackCompatMeetings) * A strong technical case has been made to allow >2B counts for all the MPI I/O functions (e.g., MPI_File_write). Rationale: 1. it is common to use MPI_BYTE when accessing files because you need to know when bytes (not elements) hit disk (Quincey can explain more) 1. files larger than 2B bytes are quite common these days * A strong case has ''not'' been made for any other set of MPI functions (MPI_Send, etc.). There were people in the room who felt ''very strongly'' that: 1. only a very small number of people are asking for this 1. some argue that you shouldn't be sending more than 2B items over (today's) networks anyway 1. if you really want to send more than 2B items, you can make a trivial datatype for it '''NOTE:''' the door is not shut to allow large counts for the rest of MPI, but no one has made a compelling technical case that we ''need'' to do it. * We will allow large counts in C bindings for MPI_File functions by: * Create a type named MPI_Count. The MPI spec will not define the size of the type; it's just an integer of some size. * Preserve existing functions with int count parameters (e.g., MPI_File_write). * Make new functions with a suffix (e.g., MPI_File_write_L -- exact suffix TBD) that take MPI_Count count parameters. * Note that some non-IO-related functions will also need to be extended (e.g., MPI_Get_elements --> MPI_Get_elementsL). There may need to be a new error class defined (e.g., MPI_ERR_OVERFLOW) for the case when MPI_Get_elements is called and the answer is more than 2B. With this approach, existing MPI codes continue to work fine. They will need to be modified to use large counts. '''NOTE:''' This sets a precedent to "extend" with new functions if we do this to other parts of MPI someday (e.g., MPI_SendL). But it is not necessarily a binding principle for future Forum decisions. * What to do about the C++ bindings? Nothing -- C++ has been deprecated. Specifically: nothing new will happen to C++ bindings in MPI-3. * What to do about the proposed MPI-3 explicit Fortran bindings (the so-called "F03 bindings")? We could use function overloading and forego the suffix approach that is necessary in C. But this was deemed to be a bad idea -- it seemed better to keep function name parity with C. So have both an MPI_File_write with int count and MPI_File_writeL with MPI_Count count in the "F03" bindings. * What about the F90 bindings? This is a harder question. '''NOTE:''' There was no resolution on this from the ATL meeting. I'm slowly coming to the opinion that we should deprecate the F90 bindings because they're so broken. * What about the existing implicit Fortran interface (i.e., mpif.h)? Do we add large count capabilities to it? Do we add any other MPI-3 capabilities to it? Or should we push users to the new explicit Fortran interface ("use mpi3")? There was no consensus in the room about what to do here -- it seems like we need to pose these questions to a larger set of audiences. There is a spectrum of 4 choices: 1. Deprecate mpif.h -- ''not'' remove it, of course. This strongly encourages Fortran MPI developers to move to "use mpi3". 1. Don't deprecate, but don't add any MPI-3 things to mpif.h. This is only slightly less-strong than a); it's really a PR move so that some people don't freak out at the word "deprecate". 1. Add all of MPI-3 ''except'' large counts to mpif.h. 1. Add all of MPI-3 ''including'' large counts to mpif.h. There were strong feelings for each of these options in ATL (even a) -- the rationale being that there ''are'' no Fortran 77 compilers any more, and there haven't been for many years. Even the J3/Fortran language committee thinks we're crazy for continuing to maintain the mpif.h implicit interfaces). So these questions need to go out to a larger group to think about what we should do here. '''NOTE:''' I sent these questions to the MPI3 Fortran WG mailing list last night. We'll see where the discussion goes. * Finally, there is the question of what we should do for new MPI-3 functions that have a count argument (e.g., MPI_Ibcast). There are two options: 1. Have MPI_Ibcast with an int count and an MPI_IbcastL with an MPI_Count count argument. 1. Have only MPI_Ibcast with an MPI_Count count argument. '''NOTE:''' This came up right at the end and there was virtually no discussion about it. I'm personally leaning towards b) here -- new API functions should use our latest/greatest conventions. -- Jeff Squyres jsquyres_at_[hidden]

1 0

Re: [Mpi3-bwcompat] Stealing the working group
by unknown＠example.com 07 Jan '10

07 Jan '10

Forgot about our friends MPI_Aint, MPI_Fint and MPI_Offset. I guess at MPI-1 time, MPI_Aint was much like we are proposing MPI_Count to be... up to the implementation in C and always an integer in fortran77. MPI-2 fixed that "mistake?", but we want MPI_Count to be like MPI_Aint was for MPI_Type_hvector and friends during the MPI-1 timeframe. > I don't think we want to go there. YMMV if you mix types -- I think that's a C issue, not an MPI issue. I agree, we already could have that issue with MPI_Aint, so we are not introducing anything new. I'll ask Rich about notes and I'll send another copy of the MPI_Count slides later. Thanks, Dave -----Original Message----- From: mpi3-bwcompat-bounces_at_[hidden] [mailto:mpi3-bwcompat-bounces_at_[hidden]] On Behalf Of Jeff Squyres Sent: Thursday, January 07, 2010 3:21 PM To: MPI-3 backwards compatability WG Subject: Re: [Mpi3-bwcompat] Stealing the working group On Jan 7, 2010, at 4:12 PM, Solt, David George wrote: > Ok, more reason to do MPI_Count and only change C, F2003 (and C++ of course!). One reason I was leaning towards not using MPI_Count and simply changing the type to 'long int' and 'integer*8/KIND=8' is to avoid any concerns that people may have that you won't get warnings when you pass integers where MPI_Count is expected. The prototype I did showed that it was not an issue, but there may still be psychological barriers that we are depending too much on compiler behaviors. Another reason is that we now have to explain that MPI_Count is not an opaque object, so what is the benefit of making it a type? Nah -- we already have MPI_Aint, MPI_Fint, and MPI_Offset. Those are all declared to be integer types that are not opaque. > In fact, I wonder if we really want to be very clear how big the count is. We don't want the user to have some code that gets compiler warnings with one MPI and not with another one because one MPI defines it as an int and another defines it as a 'long int'. Then my code: > > long int x = 1; > MPI_Count y; > y=x; > > compiles with no warnings with one MPI and gives me warnings with another MPI because MPI_Count now isn't big enough. I don't think we want to go there. YMMV if you mix types -- I think that's a C issue, not an MPI issue. > >That being said, there were some technical issues raised at the Portland meeting -- I can't remember many of them now, but one sticks out: what about functions that take a count as an OUT argument? Those can't just seamlessly be recompiled (unless MPI_Count === int). > > Oh yeah. I need to address that in the slides. Let me see exactly how many of these there are. If it is not too bad, maybe we can get by with some new interfaces for getting back long counts and otherwise the current extraction routines will need to return an error on truncation. I don't think that most of these are used very commonly and an application that wants to pass large counts already has to change its code to avoid its own application level count overflows, so requiring a code change here should be acceptible. If you can think of any of the other issues let me know. Someone took notes at the Portland meeting -- it would be worth digging them up. I'm quite sure that we're not remembering all the points that were brought up (there was a lot of useless FUD, but there were some good technical points, too). -- Jeff Squyres jsquyres_at_[hidden] _______________________________________________ Mpi3-bwcompat mailing list Mpi3-bwcompat_at_[hidden] http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-bwcompat

1 0

Re: [Mpi3-bwcompat] Stealing the working group
by unknown＠example.com 07 Jan '10

07 Jan '10

On Jan 7, 2010, at 4:12 PM, Solt, David George wrote: > Ok, more reason to do MPI_Count and only change C, F2003 (and C++ of course!). One reason I was leaning towards not using MPI_Count and simply changing the type to 'long int' and 'integer*8/KIND=8' is to avoid any concerns that people may have that you won't get warnings when you pass integers where MPI_Count is expected. The prototype I did showed that it was not an issue, but there may still be psychological barriers that we are depending too much on compiler behaviors. Another reason is that we now have to explain that MPI_Count is not an opaque object, so what is the benefit of making it a type? Nah -- we already have MPI_Aint, MPI_Fint, and MPI_Offset. Those are all declared to be integer types that are not opaque. > In fact, I wonder if we really want to be very clear how big the count is. We don't want the user to have some code that gets compiler warnings with one MPI and not with another one because one MPI defines it as an int and another defines it as a 'long int'. Then my code: > > long int x = 1; > MPI_Count y; > y=x; > > compiles with no warnings with one MPI and gives me warnings with another MPI because MPI_Count now isn't big enough. I don't think we want to go there. YMMV if you mix types -- I think that's a C issue, not an MPI issue. > >That being said, there were some technical issues raised at the Portland meeting -- I can't remember many of them now, but one sticks out: what about functions that take a count as an OUT argument? Those can't just seamlessly be recompiled (unless MPI_Count === int). > > Oh yeah. I need to address that in the slides. Let me see exactly how many of these there are. If it is not too bad, maybe we can get by with some new interfaces for getting back long counts and otherwise the current extraction routines will need to return an error on truncation. I don't think that most of these are used very commonly and an application that wants to pass large counts already has to change its code to avoid its own application level count overflows, so requiring a code change here should be acceptible. If you can think of any of the other issues let me know. Someone took notes at the Portland meeting -- it would be worth digging them up. I'm quite sure that we're not remembering all the points that were brought up (there was a lot of useless FUD, but there were some good technical points, too). -- Jeff Squyres jsquyres_at_[hidden]

1 0

Re: [Mpi3-bwcompat] Stealing the working group
by unknown＠example.com 07 Jan '10

07 Jan '10

>> Thanks for the suggestions. I'm reworking the last proposals a bit. My goal is to come up with something that allows for most interfaces (all but f77) to have a way to do large counts, leave f77 alone, and still have some consistency across languages and across what we already do. >> >> 1) Function overloading as available in Fortran90 and newer right? > >I'd vote for the F2003 MPI bindings and later. I don't really want to touch either the F77 or F90 bindings. If we are going to introduce a new TYPE, then I like this better because of my complaint about handling MPI_Count different from MPI_Request, etc. >> 2) One concern I have with going to MPI_Count is that I think that MPI_Send(buf, 10, ....) should always be legal. If we introduce a type that is up >to the implementation, then that may not be legal any longer? Do we specify in the standard that MPI_Count MUST be defined by the implementation to be >something that can be cast to/from an integer constant? > >Can you give an example where it won't be legal? Ignore me. I confused myself. Since we don't have any other types right now that allow the user to set it directly without a set function, I was thinking that MPI_Count is an opaque object like our other types (except status). But obviously we want to be able to do: MPI_Count x; x = 10; And not MPI_Count x; MPI_Count_create(&x); MPI_Set_count(&x, 10); ... MPI_Count_free(&x); Ha, ha. This is a ridiculous, of course, but I lost track of myself when I was thinking about passing constants to functions that take an MPI_Count. >> 3) Another concern with introducing MPI_Count TYPE in Fortran90/newer is that it is inconsistent with the way other types are handled (MPI_Comm, MPI_Request, MPI_Info, etc.) It seems a bit strange to me that we are going to pick out this one thing and make it a TYPE in all newer Fortrans even though everything else is declared as an INTEGER. > >In the F2003 bindings, we have opaque types for all the MPI handles -- they're no longer integers. > >:-) Ok, more reason to do MPI_Count and only change C, F2003 (and C++ of course!). One reason I was leaning towards not using MPI_Count and simply changing the type to 'long int' and 'integer*8/KIND=8' is to avoid any concerns that people may have that you won't get warnings when you pass integers where MPI_Count is expected. The prototype I did showed that it was not an issue, but there may still be psychological barriers that we are depending too much on compiler behaviors. Another reason is that we now have to explain that MPI_Count is not an opaque object, so what is the benefit of making it a type? In fact, I wonder if we really want to be very clear how big the count is. We don't want the user to have some code that gets compiler warnings with one MPI and not with another one because one MPI defines it as an int and another defines it as a 'long int'. Then my code: long int x = 1; MPI_Count y; y=x; compiles with no warnings with one MPI and gives me warnings with another MPI because MPI_Count now isn't big enough. >That being said, there were some technical issues raised at the Portland meeting -- I can't remember many of them now, but one sticks out: what about functions that take a count as an OUT argument? Those can't just seamlessly be recompiled (unless MPI_Count === int). Oh yeah. I need to address that in the slides. Let me see exactly how many of these there are. If it is not too bad, maybe we can get by with some new interfaces for getting back long counts and otherwise the current extraction routines will need to return an error on truncation. I don't think that most of these are used very commonly and an application that wants to pass large counts already has to change its code to avoid its own application level count overflows, so requiring a code change here should be acceptible. If you can think of any of the other issues let me know. -- Jeff Squyres jsquyres_at_[hidden] _______________________________________________ Mpi3-bwcompat mailing list Mpi3-bwcompat_at_[hidden] http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-bwcompat

1 0

Re: [Mpi3-bwcompat] Stealing the working group
by unknown＠example.com 07 Jan '10

07 Jan '10

On Jan 7, 2010, at 3:24 PM, Solt, David George wrote: > Thanks for the suggestions. I'm reworking the last proposals a bit. My goal is to come up with something that allows for most interfaces (all but f77) to have a way to do large counts, leave f77 alone, and still have some consistency across languages and across what we already do. > > 1) Function overloading as available in Fortran90 and newer right? I'd vote for the F2003 MPI bindings and later. I don't really want to touch either the F77 or F90 bindings. > 2) One concern I have with going to MPI_Count is that I think that MPI_Send(buf, 10, ....) should always be legal. If we introduce a type that is up to the implementation, then that may not be legal any longer? Do we specify in the standard that MPI_Count MUST be defined by the implementation to be something that can be cast to/from an integer constant? Can you give an example where it won't be legal? > 3) Another concern with introducing MPI_Count TYPE in Fortran90/newer is that it is inconsistent with the way other types are handled (MPI_Comm, MPI_Request, MPI_Info, etc.) It seems a bit strange to me that we are going to pick out this one thing and make it a TYPE in all newer Fortrans even though everything else is declared as an INTEGER. In the F2003 bindings, we have opaque types for all the MPI handles -- they're no longer integers. :-) That being said, there were some technical issues raised at the Portland meeting -- I can't remember many of them now, but one sticks out: what about functions that take a count as an OUT argument? Those can't just seamlessly be recompiled (unless MPI_Count === int). -- Jeff Squyres jsquyres_at_[hidden]

1 0