June 2011 - mpiwg-fortran - mailman3.cels.anl.gov

Re: [MPI3 Fortran] MPI function symbol naming convention for tools
by Jeff Squyres 25 Jun '11

25 Jun '11

These emails are taking a lot of time -- perhaps you, Craig, and I should get on the phone in the immediate future and just hash this out. A 30-min phone call can save a week's worth of excurciatiatingly-long, detailed emails... On Jun 24, 2011, at 1:40 PM, Martin Schulz wrote: > Btw. what will be the naming conventions for the "pmpi" > counterparts? Will there also be a pmpi_send_f and a > pmpi_send_f08? Yes. > I assume those will be Fortran calls, > i.e., take Fortran handles? They will accept Fortran handle parameters (including potentially descriptors), but they can be implemented in Fortran or C (or Cobol, if you really want to ;-) ). > Is there a way to call a C > equivalent for the f08 call (i.e., using descriptors, but > not having to map handles back)? I'm not sure what this means. > The latter would > be good, otherwise we'll have to translate F-handles > to C-handles, run the tools, convert them back to > F-handles, before the implementation takes them and > converts them back to C-handles. If you intercept MPI_SEND from Fortran today with a C function (before any of MPI's proposed f08 stuff), you have to convert handles back and forth from INTEGER with MPI_<foo>_c2f and MPI_<foo>_f2c. This will be no different. > I was thinking about buffers that are represented by descriptors > rather than pointers. Choice buffer was the wrong word. Ah; perhaps my definition is wrong -- I was still calling those "choice buffers," regardless of how the buffer appeared to the back-end implementation function (i.e., whether it was passed as a single pointer or a descriptor). > So, when > you call an MPI_Send(buf,1,MPI_INT, ...) through the F08 > interface, then we will intercept a call to mpi_send_f08 where > the first argument is a descriptor with a single entry pointing > to a single integer, correct? That's the goal, yes. >> As Craig noted in his proposal, some MPI symbols in the "use mpi_f08" interface will be the same as those in the mpif.h/"use mpi" interface -- i.e., those without choice buffers and those without MPI handles. E.g., MPI_WTICK, MPI_FINALIZE, etc. > > Oh, wait - that's even more complicated - you are saying that > there will be no mpi_finalize_f08 call since it doesn't have > choice buffers or handles (do you mean status objects?)? > I though the complete API would be available as _f08. Craig proposed it this way to avoid redundant / unnecessary symbols. Why have mpi_finalize_f and mpi_finalize_f08 when they'd be exactly the same? If it could be done via weak symbols, maybe, then you're right that it would be less confusing for both the MPI implementor and the tool author to have the full sets of mpi_<foo>_f and mpi_<foo>_f08...? Craig -- what say you to this idea? >>>> Keep in mind: >>>> >>>> - even if Craig's proposal is accepted, it'll take at least 6 months for this to become part of MPI-3 >>>> - even if Craig's proposal becomes part of MPI-3, implementations will likely take a long time before they support this new stuff >>>> - implementations *could* support the old symbols for a while if they wanted to (e.g., MPI_SEND, mpi_send, mpi_send_, mpi_send__) >>> >>> How? Each routine can only be mapped to a single >>> symbol. Hence, either the mpi_send calls mpi_send_ >>> or mpi_send_f. >> >> Weak symbols. > > No, weak symbols won't be sufficient - either the routine > calls mpi_send_ or mpi_send_f - it can't call both. I don't > think supporting the old symbols is possible. Whether > that's a deal breaker or not will be up to the forum, though > (it may well not be). It doesn't matter what the application calls -- weak symbols are provided so that the application can call any one of the weak symbols and they all resolve to 1 real symbol. Hence, if the app/tool only uses symbol A, but the implementation exports A, B, C, and D, then there's no harm. Specifically: if a tool calls the "old" form (e.g., MPI_SEND) but an implementation now provides the "new" form (mpi_send_f), if the implementation also provides a weak symbol for MPI_SEND, then the tool will still work. >> Also, there is always the point that if your application / tool uses some MPI-2.2-but-not-MPI-3.0 feature, you can always use an older MPI(-2.x) implementation. > > Will those still be around - are you planning to have a maintained > Open MPI 2.2 branch around? Open MPI provides downloads for all old versions, and our SVN is fully open (including the tags for all prior releases). We do stop supporting versions at some point, however. E.g., Open MPI v1.2.x (and prior) is no longer supported. -- Jeff Squyres jsquyres(a)cisco.com For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/

1 0

Re: [MPI3 Fortran] MPI function symbol naming convention for tools
by Bill Long 25 Jun '11

25 Jun '11

On 6/24/11 11:54 AM, Martin Schulz wrote: > > This is probably my lack of knowledge in Fortran compilers, but > is this all even possible with regular compilers: I would assume > that you can use an MPI 3.0 implementation with Craig's proposal > (case A) with an older F77 or F90 compiler, right? If so, one would > use "use mpi" or "mpif.h", i.e., case A.1. Can you do the name > mapping from mpi_send to mpi_send_f - I thought this was a new > feature from F08? > If you still have an old f77 compiler on you system (from decades ago), it will not be adequate. If you have a current version of a Fortran compiler available, you are in much better shape. The meaning of "Fortran" changes over time, and the compilers progress accordingly, with some time delay. The name mapping of a call to MPI_send into a call to mpi_send_f is accomplished with generic interfaces. These were part of Fortran 90 (20 year old technology now) - F08 is not required for this. Other details of the interfaces do require more recent standards, however. >> >> But that being said, what's the problem with testing for the optional arguments? You're in a tool already, meaning that performance overhead is already assumed to be somewhere between (nonzero<--> high), so what's 1 or 2 more "if" statements? > > Perhaps I didn't understand the proposal: as I understand Craig's > proposal, any routine that would have an optional status argument > would now have two corresponding C routines. Supporting that > in the tools is not a big deal, but having such naming exceptions > and identifying which routines fall into this category can be hard > for wrapper code generators (most tools generate the Fortran > interface through a series of scripts). > Handling optional arguments would not necessarily require a second C function, assuming the argument is passed by address (pointer). If the argument is missing in the call, the called function will just see a NULL value for the corresponding formal parameter. This already "just works" with many compilers, and is being written into a proposed TR on enhanced interoperability with C. Cheers, Bill -- Bill Long longb(a)cray.com Fortran Technical Support & voice: 651-605-9024 Bioinformatics Software Development fax: 651-605-9142 Cray Inc./Cray Plaza, Suite 210/380 Jackson St./St. Paul, MN 55101

1 0

Re: [MPI3 Fortran] [Mpi3-tools] MPI function symbol naming convention for tools
by Fab Tillier 25 Jun '11

25 Jun '11

Jeff Squyres wrote on Fri, 24 Jun 2011 at 09:46:10 > On Jun 24, 2011, at 12:20 PM, Hubert Ritzdorf wrote: > >> There are (compiled) libraries and object files containing the corresponding >> external references to MPI_SEND, mpi_send__, mpi_send_ or mpi_send. >> Therefore, you cannot simply change the interface of the MPI_Send routine >> without breaking these libraries. You can also not simply expect that they will >> be recompiled. > > I think it is reasonable to ask applications / libraries / etc. to recompile with > MPI-3. This has been the position of the Forum. There's a difference between recompiling to take advantage of MPI-3 features, and recompiling because MPI-3 has broken backward compatibility. I don't think it's reasonable to ask applications that don't take advantage of MPI-3 features to recompile - applications that may have to go through a certification process for no end-user benefit. I don't think we want MPI-3 to require implementations to break backward compatibility for Fortran codes like this. Merging from your other mail to make it clear: > if (Craig's proposal is accepted) { > if (implementation has implemented Craig's proposal) { > // call this "Case A" > if (app uses mpif.h or "use mpi" interface) { > tool will find mpi_send_f Right here, you just broke backward compatibility. > // ...and others listed in Craig's proposal > } Now, if your "Case A" is for recompiled apps, while still providing the old upper case, lower case, one and two underscore symbols, then there's no problem. From what I gather, this is not what you're proposing though, but it's quite likely I've missed something... Cheers, -Fab

1 0

Re: [MPI3 Fortran] MPI function symbol naming convention for tools
by Martin Schulz 24 Jun '11

24 Jun '11

Hi Jeff, Thanks for the clarifications again - I think I am starting to understand the nuances. A few more comments and questions below. I'll try to write this up and will send this to a few MPI tool developers who are not on our lists, so we can get more feedback. On Jun 24, 2011, at 10:16 AM, Jeff Squyres wrote: > On Jun 24, 2011, at 12:54 PM, Martin Schulz wrote: > >>> if (Craig's proposal is accepted) { >>> if (implementation has implemented Craig's proposal) { >>> // call this "Case A" >>> if (app uses mpif.h or "use mpi" interface) { >>> tool will find mpi_send_f >>> // ...and others listed in Craig's proposal >>> } >>> if (app uses "use mpi_f08" interface) { >>> tool will find mpi_send_f08 >>> // ...and others listed in Craig's proposal >> >> So, every send will go through this symbol, > > Every call to MPI_Send in the "use mpi_f08" interface will go through this symbol. Sure, I meant - once Craig's proposal gets implemented, every call will go through the new symbols (either _f or _f08). Btw. what will be the naming conventions for the "pmpi" counterparts? Will there also be a pmpi_send_f and a pmpi_send_f08? I assume those will be Fortran calls, i.e., take Fortran handles? Is there a way to call a C equivalent for the f08 call (i.e., using descriptors, but not having to map handles back)? The latter would be good, otherwise we'll have to translate F-handles to C-handles, run the tools, convert them back to F-handles, before the implementation takes them and converts them back to C-handles. > >> even if no >> choice buffer is used and a single pointer is sufficient? >> I.e., a MPI_Send(buf,1,MPI_INT, ...) as well? > > I'm not quite sure how you would have a send without a choice buffer...? > > In my mind, "choice buffer" means that anything can go in that position: a scalar, an array, a derived type instance, ...etc. "Choice" means that the type is variable, even if it's scalar. > > Do our definitions differ? I was thinking about buffers that are represented by descriptors rather than pointers. Choice buffer was the wrong word. So, when you call an MPI_Send(buf,1,MPI_INT, ...) through the F08 interface, then we will intercept a call to mpi_send_f08 where the first argument is a descriptor with a single entry pointing to a single integer, correct? > > As Craig noted in his proposal, some MPI symbols in the "use mpi_f08" interface will be the same as those in the mpif.h/"use mpi" interface -- i.e., those without choice buffers and those without MPI handles. E.g., MPI_WTICK, MPI_FINALIZE, etc. Oh, wait - that's even more complicated - you are saying that there will be no mpi_finalize_f08 call since it doesn't have choice buffers or handles (do you mean status objects?)? I though the complete API would be available as _f08. > >> This is probably my lack of knowledge in Fortran compilers, but >> is this all even possible with regular compilers: I would assume >> that you can use an MPI 3.0 implementation with Craig's proposal >> (case A) with an older F77 or F90 compiler, right? > > There is no such thing as an F77 compiler any more (i.e., a compiler that only supports up to the Fortran 77 specification). There hasn't been for (at least) 10-15+ years. Also, note that MPI has *NEVER* supported Fortran 77. It is a popular misconception that it did (easiest point to mention: Fortran 77 limited symbol names to 6 characters (!), which MPI has never obeyed). > > I suspect there aren't any F90 compilers left (i.e., compilers that only support up to the Fortran 90 specification), either (maybe g77, because that hasn't changed in years...?). > > Example: pgif90 is NOT a Fortran-90-only compiler. It supports lots of things beyond what was specified in Fortran 90. That's many Fortran compilers have stopped using "f90" and "f77" in their executable names and started moving towards "pgifortran" and "ifort" and the like. > >> If so, one would >> use "use mpi" or "mpif.h", i.e., case A.1. Can you do the name >> mapping from mpi_send to mpi_send_f - I thought this was a new >> feature from F08? > > Yes, the mpif.h and "use mpi" interfaces can support the behind-the-scenes name remapping. OK, good. > > Remember -- this is not stuff that users will see. Their old (correct) codes that compiled with mpif.h will still continue to compile and run with Craig's proposal. Sure, that's fine and good. > >>> Keep in mind: >>> >>> - even if Craig's proposal is accepted, it'll take at least 6 months for this to become part of MPI-3 >>> - even if Craig's proposal becomes part of MPI-3, implementations will likely take a long time before they support this new stuff >>> - implementations *could* support the old symbols for a while if they wanted to (e.g., MPI_SEND, mpi_send, mpi_send_, mpi_send__) >> >> How? Each routine can only be mapped to a single >> symbol. Hence, either the mpi_send calls mpi_send_ >> or mpi_send_f. > > Weak symbols. No, weak symbols won't be sufficient - either the routine calls mpi_send_ or mpi_send_f - it can't call both. I don't think supporting the old symbols is possible. Whether that's a deal breaker or not will be up to the forum, though (it may well not be). > >>> But that being said, what's the problem with testing for the optional arguments? You're in a tool already, meaning that performance overhead is already assumed to be somewhere between (nonzero <--> high), so what's 1 or 2 more "if" statements? >> >> Perhaps I didn't understand the proposal: as I understand Craig's >> proposal, any routine that would have an optional status argument >> would now have two corresponding C routines. > > It would have 2 corresponding symbols; they don't have to be implemented in C. Sure. > >> Supporting that >> in the tools is not a big deal, but having such naming exceptions >> and identifying which routines fall into this category can be hard >> for wrapper code generators (most tools generate the Fortran >> interface through a series of scripts). > > Is that really a criteria to prevent moving the Fortran MPI interface forward? I didn't say prevent - I just want to know all the implications that this will have. > > Indeed, implementing array subsections (dope vectors) is likely to be a royal PITA for MPI implementors. But it's clearly something that users want/need. > > My point: implementors saying "it's haaarrdd!" is not (always) a good excuse to prevent doing something. ;-) But if its unnecessarily hard and we can get the same functionality easier, then why not? :) The idea of having different interface names depending on the routine and its functionality makes me a bit worried. Since we won't have a full MPI 3.0 implementation anytime (soon) - as with MPI 2.2 - tools often scan the header file and create a custom version of the tool for the MPI implementation. Having too many special cases and having to deduce the names with that, could be a problem. > >> Not if the above decision tree is true - if I use MPI 2.2 compliant >> code (no MPI-3 features) and I use MPI 2.2 tools, but compile it >> against an MPI-3 implementation with Craig's proposal, I won't >> be able to intercept any Fortran calls anymore, since the >> implementation now calls the _f routines. > > The Forum has clearly said many times that MPI-3 is free to break ABI backwards compatibility (and possibly even source code compatibility, although I would not say that everyone on the Forum agrees with that -- in fact, it's a pretty unpopular view). > Sure - and I am personally probably fine with that. > So using an MPI-2.2 tool or application with an MPI-3 implementation without recompiling that tool/application (and updating/adapting tools who are looking for specific implementation features like symbols) -- yes, I can see how that would be problematic. Recompiling is fine, but won't solve the issue here - things still won't work anymore. I understand that this may be necessary here and we may have to live with it (I'll try to get more feedback from the tools community on the implications), but we need to make this part very clear to tool writers and users. > >> This basically breaks >> backwards compatibility. I realize that we said that this may be >> the case for MPI, but its certainly an issue we need to be aware >> off. Also, we do have applications, as far as I know, that use the >> PMPI interface for themselves - hence, you would not only >> break tools, but also some applications. > > The Forum will need to decide if this is important. > > My (current) feeling is that given the time lapse before this still will be necessary, there's plenty of time for tools and applications to adapt. Put differently (and more harshly): application and tool writer laziness is not a good reason to inhibit progress. Sure :) > > Also, there is always the point that if your application / tool uses some MPI-2.2-but-not-MPI-3.0 feature, you can always use an older MPI(-2.x) implementation. Will those still be around - are you planning to have a maintained Open MPI 2.2 branch around? Martin > > -- > Jeff Squyres > jsquyres(a)cisco.com > For corporate legal information go to: > http://www.cisco.com/web/about/doing_business/legal/cri/ > ________________________________________________________________________ Martin Schulz, schulzm(a)llnl.gov, http://people.llnl.gov/schulzm CASC @ Lawrence Livermore National Laboratory, Livermore, USA

1 0

Re: [MPI3 Fortran] MPI function symbol naming convention for tools
by Jeff Squyres 24 Jun '11

24 Jun '11

On Jun 24, 2011, at 12:54 PM, Martin Schulz wrote: >> if (Craig's proposal is accepted) { >> if (implementation has implemented Craig's proposal) { >> // call this "Case A" >> if (app uses mpif.h or "use mpi" interface) { >> tool will find mpi_send_f >> // ...and others listed in Craig's proposal >> } >> if (app uses "use mpi_f08" interface) { >> tool will find mpi_send_f08 >> // ...and others listed in Craig's proposal > > So, every send will go through this symbol, Every call to MPI_Send in the "use mpi_f08" interface will go through this symbol. > even if no > choice buffer is used and a single pointer is sufficient? > I.e., a MPI_Send(buf,1,MPI_INT, ...) as well? I'm not quite sure how you would have a send without a choice buffer...? In my mind, "choice buffer" means that anything can go in that position: a scalar, an array, a derived type instance, ...etc. "Choice" means that the type is variable, even if it's scalar. Do our definitions differ? As Craig noted in his proposal, some MPI symbols in the "use mpi_f08" interface will be the same as those in the mpif.h/"use mpi" interface -- i.e., those without choice buffers and those without MPI handles. E.g., MPI_WTICK, MPI_FINALIZE, etc. > This is probably my lack of knowledge in Fortran compilers, but > is this all even possible with regular compilers: I would assume > that you can use an MPI 3.0 implementation with Craig's proposal > (case A) with an older F77 or F90 compiler, right? There is no such thing as an F77 compiler any more (i.e., a compiler that only supports up to the Fortran 77 specification). There hasn't been for (at least) 10-15+ years. Also, note that MPI has *NEVER* supported Fortran 77. It is a popular misconception that it did (easiest point to mention: Fortran 77 limited symbol names to 6 characters (!), which MPI has never obeyed). I suspect there aren't any F90 compilers left (i.e., compilers that only support up to the Fortran 90 specification), either (maybe g77, because that hasn't changed in years...?). Example: pgif90 is NOT a Fortran-90-only compiler. It supports lots of things beyond what was specified in Fortran 90. That's many Fortran compilers have stopped using "f90" and "f77" in their executable names and started moving towards "pgifortran" and "ifort" and the like. > If so, one would > use "use mpi" or "mpif.h", i.e., case A.1. Can you do the name > mapping from mpi_send to mpi_send_f - I thought this was a new > feature from F08? Yes, the mpif.h and "use mpi" interfaces can support the behind-the-scenes name remapping. Remember -- this is not stuff that users will see. Their old (correct) codes that compiled with mpif.h will still continue to compile and run with Craig's proposal. >> Keep in mind: >> >> - even if Craig's proposal is accepted, it'll take at least 6 months for this to become part of MPI-3 >> - even if Craig's proposal becomes part of MPI-3, implementations will likely take a long time before they support this new stuff >> - implementations *could* support the old symbols for a while if they wanted to (e.g., MPI_SEND, mpi_send, mpi_send_, mpi_send__) > > How? Each routine can only be mapped to a single > symbol. Hence, either the mpi_send calls mpi_send_ > or mpi_send_f. Weak symbols. >> But that being said, what's the problem with testing for the optional arguments? You're in a tool already, meaning that performance overhead is already assumed to be somewhere between (nonzero <--> high), so what's 1 or 2 more "if" statements? > > Perhaps I didn't understand the proposal: as I understand Craig's > proposal, any routine that would have an optional status argument > would now have two corresponding C routines. It would have 2 corresponding symbols; they don't have to be implemented in C. > Supporting that > in the tools is not a big deal, but having such naming exceptions > and identifying which routines fall into this category can be hard > for wrapper code generators (most tools generate the Fortran > interface through a series of scripts). Is that really a criteria to prevent moving the Fortran MPI interface forward? Indeed, implementing array subsections (dope vectors) is likely to be a royal PITA for MPI implementors. But it's clearly something that users want/need. My point: implementors saying "it's haaarrdd!" is not (always) a good excuse to prevent doing something. ;-) > Not if the above decision tree is true - if I use MPI 2.2 compliant > code (no MPI-3 features) and I use MPI 2.2 tools, but compile it > against an MPI-3 implementation with Craig's proposal, I won't > be able to intercept any Fortran calls anymore, since the > implementation now calls the _f routines. The Forum has clearly said many times that MPI-3 is free to break ABI backwards compatibility (and possibly even source code compatibility, although I would not say that everyone on the Forum agrees with that -- in fact, it's a pretty unpopular view). So using an MPI-2.2 tool or application with an MPI-3 implementation without recompiling that tool/application (and updating/adapting tools who are looking for specific implementation features like symbols) -- yes, I can see how that would be problematic. > This basically breaks > backwards compatibility. I realize that we said that this may be > the case for MPI, but its certainly an issue we need to be aware > off. Also, we do have applications, as far as I know, that use the > PMPI interface for themselves - hence, you would not only > break tools, but also some applications. The Forum will need to decide if this is important. My (current) feeling is that given the time lapse before this still will be necessary, there's plenty of time for tools and applications to adapt. Put differently (and more harshly): application and tool writer laziness is not a good reason to inhibit progress. Also, there is always the point that if your application / tool uses some MPI-2.2-but-not-MPI-3.0 feature, you can always use an older MPI(-2.x) implementation. -- Jeff Squyres jsquyres(a)cisco.com For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/

1 0

Re: [MPI3 Fortran] MPI function symbol naming convention for tools
by Martin Schulz 24 Jun '11

24 Jun '11

Hi Jeff, Thanks for the clarifications - that's definitely more clear now. A few additional questions/comments inline below: On Jun 24, 2011, at 7:00 AM, Jeff Squyres wrote: > On Jun 23, 2011, at 10:51 PM, Martin Schulz wrote: > >>>> I read Craig's email as if the existing interface (without using mpi_f08) >>>> would get the new symbol as well. If I read you right, this is not the case. >>> >>> That *is* the case. That's why I started my sentence with "So in one sense..." :-) >> >> Now, I am confused - this seems to contradict the above. >> If I compile a F77 only program that uses mpif.h and has a >> call to MPI_Recv, which symbol(s) do I get? > > if (Craig's proposal is accepted) { > if (implementation has implemented Craig's proposal) { > // call this "Case A" > if (app uses mpif.h or "use mpi" interface) { > tool will find mpi_send_f > // ...and others listed in Craig's proposal > } > if (app uses "use mpi_f08" interface) { > tool will find mpi_send_f08 > // ...and others listed in Craig's proposal So, every send will go through this symbol, even if no choice buffer is used and a single pointer is sufficient? I.e., a MPI_Send(buf,1,MPI_INT, ...) as well? > } > } else { > // implementation has NOT implemented Craig's proposal > // call this "Case B" > if (app uses mpif.h interface) { > tool will find MPI_SEND, mpi_send, mpi_send_, or mpi_send__ Yes, that's the common case covered by today's tools. > } > if (app uses "use mpi" interface) { > I doubt tools know how to find the symbols for functions > with choice buffers, because nothing is standardized > and there's (potentially) lots of function overloading I agree - I don't know of any tool supporting this. > } > } > } else { > // same as "Case B" > } > This is probably my lack of knowledge in Fortran compilers, but is this all even possible with regular compilers: I would assume that you can use an MPI 3.0 implementation with Craig's proposal (case A) with an older F77 or F90 compiler, right? If so, one would use "use mpi" or "mpif.h", i.e., case A.1. Can you do the name mapping from mpi_send to mpi_send_f - I thought this was a new feature from F08? > Keep in mind: > > - even if Craig's proposal is accepted, it'll take at least 6 months for this to become part of MPI-3 > - even if Craig's proposal becomes part of MPI-3, implementations will likely take a long time before they support this new stuff > - implementations *could* support the old symbols for a while if they wanted to (e.g., MPI_SEND, mpi_send, mpi_send_, mpi_send__) How? Each routine can only be mapped to a single symbol. Hence, either the mpi_send calls mpi_send_ or mpi_send_f. > > My point: I'm not too worried about tools needing to update to get the new symbol conventions because a) implementations have to update, too, and b) I think standardized symbol conventions put the MPI spec in a better place for the future -- *future* tools will have an easier time than current tools. > >> Sure, the addition is not hard - the more complicated >> part will be the routines for the optional parameters. >> We can no longer deduce them automatically from >> any header file and will always have to special code >> them. Do we really need them? Why not pass a >> STATUS_IGNORE argument? > > I think that's under debate at the moment. > > But that being said, what's the problem with testing for the optional arguments? You're in a tool already, meaning that performance overhead is already assumed to be somewhere between (nonzero <--> high), so what's 1 or 2 more "if" statements? Perhaps I didn't understand the proposal: as I understand Craig's proposal, any routine that would have an optional status argument would now have two corresponding C routines. Supporting that in the tools is not a big deal, but having such naming exceptions and identifying which routines fall into this category can be hard for wrapper code generators (most tools generate the Fortran interface through a series of scripts). > >>> And, more importantly, I believe there's plenty of time for existing tools to adapt (e.g., to add intercepting the 5th symbol) -- maybe a year or more. OMPI will have a first prototype implementation soon; I imagine the others will follow over time (assuming that this proposal, or something like it, is passed into the MPI-3 spec over the course of the next 6 months). >> >> In the meantime, they will be labeled as MPI 2.2 tools and will >> work with any MPI 2.2 compliant code. As long as that still >> works (and I see no reason why not) then I think things are fine. > > This is what I'm saying. Not if the above decision tree is true - if I use MPI 2.2 compliant code (no MPI-3 features) and I use MPI 2.2 tools, but compile it against an MPI-3 implementation with Craig's proposal, I won't be able to intercept any Fortran calls anymore, since the implementation now calls the _f routines. This basically breaks backwards compatibility. I realize that we said that this may be the case for MPI, but its certainly an issue we need to be aware off. Also, we do have applications, as far as I know, that use the PMPI interface for themselves - hence, you would not only break tools, but also some applications. > >>> Also note that there is no standardized nomenclature for intercepting "use mpi" symbols, so I'll bet you a high-quality beer that there are no portable tools that properly instrument "use mpi" interfaces. OMPI's "use mpi" implementation, for example, *sometimes* calls the corresponding mpif.h symbol, but not always. Heck, I'll even bet you another HQ beer that there's *no* MPI implementation that supports explicit Fortran interfaces for all MPI functions in its "use mpi" interfaces that has a corresponding interception / tracing tool -- specifically because of the lack of standardization of how to do it. >>> >>> Craig's proposal gives you a way to reliably intercept the "use mpi" symbols. >> >> Well, not quite - we still have to play the tricks for older MPIs >> and for the parts of the codes that don't use the module. Those >> will use the old naming convention, right? > > I'm not sure I can parse your statement -- it doesn't seem to be a logical result of my text...? > > The portion of my text that you were replying to was only speaking about the "use mpi" interface -- not mpif.h. See above for discussions about mpif.h. > > As for "use mpi": there is no naming convention for "use mpi" symbols right now. Therefore, there's no commonality between MPI implementations. Are there any tools that reliably, portably intercept all "use mpi" symbols right now? I was under the impression that there were none, but perhaps I'm wrong...? I think this is true. Martin > > -- > Jeff Squyres > jsquyres(a)cisco.com > For corporate legal information go to: > http://www.cisco.com/web/about/doing_business/legal/cri/ > ________________________________________________________________________ Martin Schulz, schulzm(a)llnl.gov, http://people.llnl.gov/schulzm CASC @ Lawrence Livermore National Laboratory, Livermore, USA

1 0

Re: [MPI3 Fortran] MPI function symbol naming convention for tools
by Jeff Squyres 24 Jun '11

24 Jun '11

On Jun 24, 2011, at 12:20 PM, Hubert Ritzdorf wrote: > There are (compiled) libraries and object files containing the corresponding external references to MPI_SEND, mpi_send__, mpi_send_ or mpi_send. Therefore, you cannot simply change the interface of the MPI_Send routine without breaking these libraries. You can also not simply expect that they will be recompiled. I think it is reasonable to ask applications / libraries / etc. to recompile with MPI-3. This has been the position of the Forum. > In addition, the mpif.h include file doesn't have an interface block (and will still generate the external references above) It's actually not specified, right? Meaning that an implementation *could* use an interface block if they wanted to. > and Fortran 90 module doesn't have an optional error argument in the interface block even if you expect that the Fortran 90 module/compiler supports some NO_ARG_CHECK attribute. We didn't add optional arguments to any of the existing MPI Fortran interfaces -- optional arguments are only (possibly) being added in the new F08 bindings. -- Jeff Squyres jsquyres(a)cisco.com For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/

1 0

Re: [MPI3 Fortran] MPI function symbol naming convention for tools
by Hubert Ritzdorf 24 Jun '11

24 Jun '11

There are (compiled) libraries and object files containing the corresponding external references to MPI_SEND, mpi_send__, mpi_send_ or mpi_send. Therefore, you cannot simply change the interface of the MPI_Send routine without breaking these libraries. You can also not simply expect that they will be recompiled. In addition, the mpif.h include file doesn't have an interface block (and will still generate the external references above) and Fortran 90 module doesn't have an optional error argument in the interface block even if you expect that the Fortran 90 module/compiler supports some NO_ARG_CHECK attribute. Therefore, you cannot simply change the name of the Fortran MPI_Send routine to mpi_send_f. Hubert Ritzdorf NEC Germany >-----Original Message----- >From: mpi3-fortran-bounces(a)lists.mpi-forum.org [mailto:mpi3-fortran- >bounces(a)lists.mpi-forum.org] On Behalf Of Jeff Squyres >Sent: Friday, June 24, 2011 4:00 PM >To: Martin Schulz >Cc: MPI3 Tools; MPI-3 Fortran working group >Subject: Re: [MPI3 Fortran] MPI function symbol naming convention for tools > >On Jun 23, 2011, at 10:51 PM, Martin Schulz wrote: > >>>> I read Craig's email as if the existing interface (without using >mpi_f08) >>>> would get the new symbol as well. If I read you right, this is not the >case. >>> >>> That *is* the case. That's why I started my sentence with "So in one >sense..." :-) >> >> Now, I am confused - this seems to contradict the above. >> If I compile a F77 only program that uses mpif.h and has a >> call to MPI_Recv, which symbol(s) do I get? > >if (Craig's proposal is accepted) { > if (implementation has implemented Craig's proposal) { > // call this "Case A" > if (app uses mpif.h or "use mpi" interface) { > tool will find mpi_send_f > // ...and others listed in Craig's proposal > } > if (app uses "use mpi_f08" interface) { > tool will find mpi_send_f08 > // ...and others listed in Craig's proposal > } > } else { > // implementation has NOT implemented Craig's proposal > // call this "Case B" > if (app uses mpif.h interface) { > tool will find MPI_SEND, mpi_send, mpi_send_, or mpi_send__ > } > if (app uses "use mpi" interface) { > I doubt tools know how to find the symbols for functions > with choice buffers, because nothing is standardized > and there's (potentially) lots of function overloading > } > } >} else { > // same as "Case B" >} > >Keep in mind: > >- even if Craig's proposal is accepted, it'll take at least 6 months for >this to become part of MPI-3 >- even if Craig's proposal becomes part of MPI-3, implementations will >likely take a long time before they support this new stuff >- implementations *could* support the old symbols for a while if they >wanted to (e.g., MPI_SEND, mpi_send, mpi_send_, mpi_send__) > >My point: I'm not too worried about tools needing to update to get the new >symbol conventions because a) implementations have to update, too, and b) I >think standardized symbol conventions put the MPI spec in a better place >for the future -- *future* tools will have an easier time than current >tools. > >> Sure, the addition is not hard - the more complicated >> part will be the routines for the optional parameters. >> We can no longer deduce them automatically from >> any header file and will always have to special code >> them. Do we really need them? Why not pass a >> STATUS_IGNORE argument? > >I think that's under debate at the moment. > >But that being said, what's the problem with testing for the optional >arguments? You're in a tool already, meaning that performance overhead is >already assumed to be somewhere between (nonzero <--> high), so what's 1 or >2 more "if" statements? > >>> And, more importantly, I believe there's plenty of time for existing >tools to adapt (e.g., to add intercepting the 5th symbol) -- maybe a year >or more. OMPI will have a first prototype implementation soon; I imagine >the others will follow over time (assuming that this proposal, or something >like it, is passed into the MPI-3 spec over the course of the next 6 >months). >> >> In the meantime, they will be labeled as MPI 2.2 tools and will >> work with any MPI 2.2 compliant code. As long as that still >> works (and I see no reason why not) then I think things are fine. > >This is what I'm saying. > >>> Also note that there is no standardized nomenclature for intercepting >"use mpi" symbols, so I'll bet you a high-quality beer that there are no >portable tools that properly instrument "use mpi" interfaces. OMPI's "use >mpi" implementation, for example, *sometimes* calls the corresponding >mpif.h symbol, but not always. Heck, I'll even bet you another HQ beer >that there's *no* MPI implementation that supports explicit Fortran >interfaces for all MPI functions in its "use mpi" interfaces that has a >corresponding interception / tracing tool -- specifically because of the >lack of standardization of how to do it. >>> >>> Craig's proposal gives you a way to reliably intercept the "use mpi" >symbols. >> >> Well, not quite - we still have to play the tricks for older MPIs >> and for the parts of the codes that don't use the module. Those >> will use the old naming convention, right? > >I'm not sure I can parse your statement -- it doesn't seem to be a logical >result of my text...? > >The portion of my text that you were replying to was only speaking about >the "use mpi" interface -- not mpif.h. See above for discussions about >mpif.h. > >As for "use mpi": there is no naming convention for "use mpi" symbols right >now. Therefore, there's no commonality between MPI implementations. Are >there any tools that reliably, portably intercept all "use mpi" symbols >right now? I was under the impression that there were none, but perhaps >I'm wrong...? > >-- >Jeff Squyres >jsquyres(a)cisco.com >For corporate legal information go to: >http://www.cisco.com/web/about/doing_business/legal/cri/ > > >_______________________________________________ >mpi3-fortran mailing list >mpi3-fortran(a)lists.mpi-forum.org >http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-fortran > > > Click >https://www.mailcontrol.com/sr/usm!Nh46RiHTndxI!oX7UqpixUKymCqxTDACcy67S9iI >adHPAjIZPDMvmfJgurQ+nNutHdm6V!yDEGNPdlyNkQ== to report this email as spam.

1 0

Re: [MPI3 Fortran] MPI function symbol naming convention for tools
by Jeff Squyres 24 Jun '11

24 Jun '11

On Jun 23, 2011, at 10:51 PM, Martin Schulz wrote: >>> I read Craig's email as if the existing interface (without using mpi_f08) >>> would get the new symbol as well. If I read you right, this is not the case. >> >> That *is* the case. That's why I started my sentence with "So in one sense..." :-) > > Now, I am confused - this seems to contradict the above. > If I compile a F77 only program that uses mpif.h and has a > call to MPI_Recv, which symbol(s) do I get? if (Craig's proposal is accepted) { if (implementation has implemented Craig's proposal) { // call this "Case A" if (app uses mpif.h or "use mpi" interface) { tool will find mpi_send_f // ...and others listed in Craig's proposal } if (app uses "use mpi_f08" interface) { tool will find mpi_send_f08 // ...and others listed in Craig's proposal } } else { // implementation has NOT implemented Craig's proposal // call this "Case B" if (app uses mpif.h interface) { tool will find MPI_SEND, mpi_send, mpi_send_, or mpi_send__ } if (app uses "use mpi" interface) { I doubt tools know how to find the symbols for functions with choice buffers, because nothing is standardized and there's (potentially) lots of function overloading } } } else { // same as "Case B" } Keep in mind: - even if Craig's proposal is accepted, it'll take at least 6 months for this to become part of MPI-3 - even if Craig's proposal becomes part of MPI-3, implementations will likely take a long time before they support this new stuff - implementations *could* support the old symbols for a while if they wanted to (e.g., MPI_SEND, mpi_send, mpi_send_, mpi_send__) My point: I'm not too worried about tools needing to update to get the new symbol conventions because a) implementations have to update, too, and b) I think standardized symbol conventions put the MPI spec in a better place for the future -- *future* tools will have an easier time than current tools. > Sure, the addition is not hard - the more complicated > part will be the routines for the optional parameters. > We can no longer deduce them automatically from > any header file and will always have to special code > them. Do we really need them? Why not pass a > STATUS_IGNORE argument? I think that's under debate at the moment. But that being said, what's the problem with testing for the optional arguments? You're in a tool already, meaning that performance overhead is already assumed to be somewhere between (nonzero <--> high), so what's 1 or 2 more "if" statements? >> And, more importantly, I believe there's plenty of time for existing tools to adapt (e.g., to add intercepting the 5th symbol) -- maybe a year or more. OMPI will have a first prototype implementation soon; I imagine the others will follow over time (assuming that this proposal, or something like it, is passed into the MPI-3 spec over the course of the next 6 months). > > In the meantime, they will be labeled as MPI 2.2 tools and will > work with any MPI 2.2 compliant code. As long as that still > works (and I see no reason why not) then I think things are fine. This is what I'm saying. >> Also note that there is no standardized nomenclature for intercepting "use mpi" symbols, so I'll bet you a high-quality beer that there are no portable tools that properly instrument "use mpi" interfaces. OMPI's "use mpi" implementation, for example, *sometimes* calls the corresponding mpif.h symbol, but not always. Heck, I'll even bet you another HQ beer that there's *no* MPI implementation that supports explicit Fortran interfaces for all MPI functions in its "use mpi" interfaces that has a corresponding interception / tracing tool -- specifically because of the lack of standardization of how to do it. >> >> Craig's proposal gives you a way to reliably intercept the "use mpi" symbols. > > Well, not quite - we still have to play the tricks for older MPIs > and for the parts of the codes that don't use the module. Those > will use the old naming convention, right? I'm not sure I can parse your statement -- it doesn't seem to be a logical result of my text...? The portion of my text that you were replying to was only speaking about the "use mpi" interface -- not mpif.h. See above for discussions about mpif.h. As for "use mpi": there is no naming convention for "use mpi" symbols right now. Therefore, there's no commonality between MPI implementations. Are there any tools that reliably, portably intercept all "use mpi" symbols right now? I was under the impression that there were none, but perhaps I'm wrong...? -- Jeff Squyres jsquyres(a)cisco.com For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/

1 0

Re: [MPI3 Fortran] MPI function symbol naming convention for tools
by Martin Schulz 23 Jun '11

23 Jun '11

Hi Jeff, On Jun 23, 2011, at 4:07 AM, Jeff Squyres wrote: > On Jun 23, 2011, at 3:39 AM, Martin Schulz wrote: > >> That makes more sense now - just to clarify: the new names only >> apply if an application uses "use mpi_f08", otherwise the old >> interface applies. Is this correct? > > It's almost that simple, but not quite. > > An MPI application can use any of all of the interfaces. In one .f90 file, for example, there could be a subroutine that uses mpif.h, another subroutine that uses "use mpi", and another that uses "use mpi_f08". Sure, there may be mixed mode programs. > > So tools will need to be expanded to cover all of the MPI Fortran interfaces; if you only catch the old names, you might miss some run-time information. Also that makes sense and is what I expected. What I meant was more that existing tools work without modifications on programs that do not use mpi_f08 at all in any part. Is this correct? > >>>> So in one sense, we're not breaking tools because the use mpi_f08 interface is new. >> >> I read Craig's email as if the existing interface (without using mpi_f08) >> would get the new symbol as well. If I read you right, this is not the case. > > That *is* the case. That's why I started my sentence with "So in one sense..." :-) Now, I am confused - this seems to contradict the above. If I compile a F77 only program that uses mpif.h and has a call to MPI_Recv, which symbol(s) do I get? > >>>> In another sense, if we mandate specific symbols for mpif.h/use mpi, we might break some tools, but put ourselves in a better place for the future. Since a tool already has to sense which back-end symbol to use mpif.h/use mpi (lower case, caps, 1 underscore, 2 underscores -- none of which are mandated by the Fortran spec, by the way; it's just common convention), is it really a hardship to add another symbol name to check during the tools configuration/building mechanism? >> >> I am more worrying about existing tools - those would all >> have to be changed. > > They're all going to have to be changed anyway. > > However, it's not as bad as it sounds. Tools today must intercept the mpif.h symbols by finding the right symbol to intercept: > > MPI_SEND > mpi_send > mpi_send_ > mpi_send__ > > In Craig's proposal, you now have to look for one more form of the symbol: > > mpi_send_f > > Hence, if you add supporting the one new symbol, you'll cover all old MPI implementations and all new ones. Sure, the addition is not hard - the more complicated part will be the routines for the optional parameters. We can no longer deduce them automatically from any header file and will always have to special code them. Do we really need them? Why not pass a STATUS_IGNORE argument? > > And, more importantly, I believe there's plenty of time for existing tools to adapt (e.g., to add intercepting the 5th symbol) -- maybe a year or more. OMPI will have a first prototype implementation soon; I imagine the others will follow over time (assuming that this proposal, or something like it, is passed into the MPI-3 spec over the course of the next 6 months). In the meantime, they will be labeled as MPI 2.2 tools and will work with any MPI 2.2 compliant code. As long as that still works (and I see no reason why not) then I think things are fine. > > Also note that there is no standardized nomenclature for intercepting "use mpi" symbols, so I'll bet you a high-quality beer that there are no portable tools that properly instrument "use mpi" interfaces. OMPI's "use mpi" implementation, for example, *sometimes* calls the corresponding mpif.h symbol, but not always. Heck, I'll even bet you another HQ beer that there's *no* MPI implementation that supports explicit Fortran interfaces for all MPI functions in its "use mpi" interfaces that has a corresponding interception / tracing tool -- specifically because of the lack of standardization of how to do it. > > Craig's proposal gives you a way to reliably intercept the "use mpi" symbols. Well, not quite - we still have to play the tricks for older MPIs and for the parts of the codes that don't use the module. Those will use the old naming convention, right? > > The "use mpi_f08" stuff is new, so tools obviously have to be expanded to instrument those symbols (and new types). But I don't think that's the main point of your question. Sure, we'll have to work on this, but that was to be expected. Martin > > -- > Jeff Squyres > jsquyres(a)cisco.com > For corporate legal information go to: > http://www.cisco.com/web/about/doing_business/legal/cri/ > ________________________________________________________________________ Martin Schulz, schulzm(a)llnl.gov, http://people.llnl.gov/schulzm CASC @ Lawrence Livermore National Laboratory, Livermore, USA

1 0