September 2011 - mpiwg-tools - mailman3.cels.anl.gov

Re: [Mpi3-tools] Questions on the F2008 profiling interface issues
by Marc-Andre Hermanns 30 Sep '11

30 Sep '11

Martin, > I was in Dresden the last few days (visiting the Vampir group, in > particular Andreas Knuepfer and Tobias Hilbrich who are responsible for > their MPI wrappers) and we sat down to talk about the issues around the > profiling interface in ticket #229. > > Here is a quick summary with a few questions that came up: > > * With the new Fortran symbol naming variants, there are 6 or 8 or 10 > versions for every MPI subroutine. Tools could intercept all versions by > providing own symbols for all versions. If it is clear which name the symbols have, additional symbol names should be easy to generate, so at this stage I think it doesn't matter much, whether we have 6, 8, or 10 different schemes to cover. Right? > Now the problem: Within the wrapper function a tool needs to call the > correct PMPI call which usually is the same prefix (either '_f08', > '_f', '__' etc.) as the MPI call it is in. A mapping to matching C > functions (as most tools do it now) is not possible due to problems > with calling conventions for callback routines. As far as I know our wrappers call the C routines, because we jump through all sorts of hoops to get a proper Fortran to C conversion and back. So in this regard, we need to fix this. Interestingly this has not popped up yet. If we write Fortran wrappers in Fortran, how would a Fortran wrapper compiled with Fortran 77 have to call the C measurement code inside the wrapper? If the Fortran compiler implicitly adds underscores to the calls, do I have to provide a special Fortran interface to my measurement system then? > * Would the following solution work? All wrappers for Fortran are written > in Fortran instead of C. > [...] > For every MPI function, there are 6 or 8 variants and all of them > need to be provided by a tool. This would probably mean that we have > to compile wrappers with all three calling conventions (mpif.h, use > mpi_f and use mpi_f08) and then link them together? Don't we need to compile with all three calling conventions on all possible compilers on a given system? We currently provide all 4 name-mangling schemes with every install of Scalasca, because we had some issues with applications using two different Fortran compilers for different sub-modules. (This came up in a project some years back, and ever since then we provide all schemes in the same library and it seems to work.) > * As discussed at the forum, if we want to keep our tools in C (which is > probably still the easier and better option), we need to know the symbols > on the "p" side and we need a portable way to figure this out. We talked > about the solution of two groups of routines - one for which we have _f08 > symbols for one for which we don't. Rolf, how will you add this to the > standard, in particular how will you form those groups. I started to get some doubts about this approach. One of the issues was that some choice-buffer routines had names longer than the proposed limit. So they need to be mangled differently, especially, their symbol needs to be shortened. And as I understood it, the current idea is to leave it to the MPI implementation to define the shortened name. It is one thing, finding out which suffix is used. Finding how function names are mangled without any form of restricting to possible schemes seems to be another ball game? Am I missing something? I think, as it was pointed out in Santorini with the Fortran Bindings in general, we (or at least I) don't have enough Fortran expertise to think this through all to the end and consider all implications. I think it would be a great benefit to have people of the Fortran WG to join on the call on Monday. I hope this is possible. Cheers, Marc-Andre -- Marc-Andre Hermanns German Research School for Simulation Sciences GmbH c/o Laboratory for Parallel Programming 52056 Aachen | Germany Tel +49 241 80 99753 Fax +49 241 80 6 99753 Web www.grs-sim.de Members: Forschungszentrum Jülich GmbH | RWTH Aachen University Registered in the commercial register of the local court of Düren (Amtsgericht Düren) under registration number HRB 5268 Registered office: Jülich Executive board: Prof. Marek Behr Ph.D. | Dr. Norbert Drewes

1 0

Re: [Mpi3-tools] Questions on the F2008 profiling interface issues
by Supalov, Alexander 29 Sep '11

29 Sep '11

Thanks. Should probably go to the Fortran list as well (CC). Don't you guys think this is getting out of hand? E.g., prescribing (rather than recommending) a certain implementation path is certainly a novum. -----Original Message----- From: mpi3-tools-bounces(a)lists.mpi-forum.org [mailto:[email protected]] On Behalf Of Schulz, Martin Sent: Friday, September 30, 2011 4:36 AM To: Rasmussen, Craig E; Rolf Rabenseifner; Jeff Squyres Cc: Andreas Knï¿½pfer; Todd Gamblin; Tobias Hilbrich; MPI3 Tools Subject: [Mpi3-tools] Questions on the F2008 profiling interface issues Hi Rolf, Craig, and Jeff, I was in Dresden the last few days (visiting the Vampir group, in particular Andreas Knuepfer and Tobias Hilbrich who are responsible for their MPI wrappers) and we sat down to talk about the issues around the profiling interface in ticket #229. Here is a quick summary with a few questions that came up: * With the new Fortran symbol naming variants, there are 6 or 8 or 10 versions for every MPI subroutine. Tools could intercept all versions by providing own symbols for all versions. Now the problem: Within the wrapper function a tool needs to call the correct PMPI call which usually is the same prefix (either '_f08', '_f', '__' etc.) as the MPI call it is in. A mapping to matching C functions (as most tools do it now) is not possible due to problems with calling conventions for callback routines. * This problem is only there for Fortran, since in C we only have one interface and we are guaranteed that this is always there. * Would the following solution work? All wrappers for Fortran are written in Fortran instead of C. For every MPI function, there are 6 or 8 variants and all of them need to be provided by a tool. This would probably mean that we have to compile wrappers with all three calling conventions (mpif.h, use mpi_f and use mpi_f08) and then link them together? Then within the Fortran wrappers, one could safely call the plain Fortran name of the PMPI function. Based on the specification of the standard, the MPI implementation would then provide a matching Fortran "P" call that the Fortran compiler then needs to transform into the correct symbol name with one of the prefixes, the same way it works for the regular "MPI" routines. Does this sound right or are we missing something? Does this separate compilation of tool wrappers using all three methods and then linking them together really work? * I think you mentioned this before, but I forgot - why is the BINDC option in Chapter 16 coupled to the SUBARRAYS flag? Why are those not separate issue? * As discussed at the forum, if we want to keep our tools in C (which is probably still the easier and better option), we need to know the symbols on the "p" side and we need a portable way to figure this out. We talked about the solution of two groups of routines - one for which we have _f08 symbols for one for which we don't. Rolf, how will you add this to the standard, in particular how will you form those groups. * Once we have the groups, is the following correct? ** If the symbol pmpi_foo exists, then pmpi_bar exists for any bar ** If the symbol pmpi_foo_ exists, then pmpi_bar_ exists for any bar ** If the symbol pmpi_foo__ exists, then pmpi_bar__ exists for any bar ** If the symbol PMPI_Foo_f exists, then PMPI_Bar_f exists for any bar ** If the symbol pmpi_foo_f08 exists, then pmpi_bar_f08 exists for any bar in the group of _f08 routines defined in the standard, but does not exist for any other bar ** If the symbol pmpi_foo_f08_ exists, then pmpi_bar_f08_ exists for any bar in the group of _f08 routines defined in the standard, but does not exist for any other bar ** If the symbol pmpi_foo_f08__ exists, then pmpi_bar_f08__ exists for any bar in the group of _f08 routines defined in the standard, but does not exist for any other bar ** If the symbol PMPI_Foo_f08 exists, then PMPI_Bar_f08 exists for any bar in the group of _f08 routines defined in the standard, but does not exist for any other bar Is this correct and complete or am I missing any symbols. Does this all sound correct? We just wanted to make sure we understand all the nuances and have things right before the discussion on Monday. Thanks! Martin -- ________________________________________________________________________ Martin Schulz, schulzm(a)llnl.gov, http://people.llnl.gov/schulzm CASC @ Lawrence Livermore National Laboratory, Livermore, USA _______________________________________________ Mpi3-tools mailing list Mpi3-tools(a)lists.mpi-forum.org http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-tools -------------------------------------------------------------------------------------- Intel GmbH Dornacher Strasse 1 85622 Feldkirchen/Muenchen, Deutschland Sitz der Gesellschaft: Feldkirchen bei Muenchen Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer Registergericht: Muenchen HRB 47456 Ust.-IdNr./VAT Registration No.: DE129385895 Citibank Frankfurt a.M. (BLZ 502 109 00) 600119052

1 0

[Mpi3-tools] Questions on the F2008 profiling interface issues
by Schulz, Martin 29 Sep '11

29 Sep '11

Hi Rolf, Craig, and Jeff, I was in Dresden the last few days (visiting the Vampir group, in particular Andreas Knuepfer and Tobias Hilbrich who are responsible for their MPI wrappers) and we sat down to talk about the issues around the profiling interface in ticket #229. Here is a quick summary with a few questions that came up: * With the new Fortran symbol naming variants, there are 6 or 8 or 10 versions for every MPI subroutine. Tools could intercept all versions by providing own symbols for all versions. Now the problem: Within the wrapper function a tool needs to call the correct PMPI call which usually is the same prefix (either '_f08', '_f', '__' etc.) as the MPI call it is in. A mapping to matching C functions (as most tools do it now) is not possible due to problems with calling conventions for callback routines. * This problem is only there for Fortran, since in C we only have one interface and we are guaranteed that this is always there. * Would the following solution work? All wrappers for Fortran are written in Fortran instead of C. For every MPI function, there are 6 or 8 variants and all of them need to be provided by a tool. This would probably mean that we have to compile wrappers with all three calling conventions (mpif.h, use mpi_f and use mpi_f08) and then link them together? Then within the Fortran wrappers, one could safely call the plain Fortran name of the PMPI function. Based on the specification of the standard, the MPI implementation would then provide a matching Fortran "P" call that the Fortran compiler then needs to transform into the correct symbol name with one of the prefixes, the same way it works for the regular "MPI" routines. Does this sound right or are we missing something? Does this separate compilation of tool wrappers using all three methods and then linking them together really work? * I think you mentioned this before, but I forgot - why is the BINDC option in Chapter 16 coupled to the SUBARRAYS flag? Why are those not separate issue? * As discussed at the forum, if we want to keep our tools in C (which is probably still the easier and better option), we need to know the symbols on the "p" side and we need a portable way to figure this out. We talked about the solution of two groups of routines - one for which we have _f08 symbols for one for which we don't. Rolf, how will you add this to the standard, in particular how will you form those groups. * Once we have the groups, is the following correct? ** If the symbol pmpi_foo exists, then pmpi_bar exists for any bar ** If the symbol pmpi_foo_ exists, then pmpi_bar_ exists for any bar ** If the symbol pmpi_foo__ exists, then pmpi_bar__ exists for any bar ** If the symbol PMPI_Foo_f exists, then PMPI_Bar_f exists for any bar ** If the symbol pmpi_foo_f08 exists, then pmpi_bar_f08 exists for any bar in the group of _f08 routines defined in the standard, but does not exist for any other bar ** If the symbol pmpi_foo_f08_ exists, then pmpi_bar_f08_ exists for any bar in the group of _f08 routines defined in the standard, but does not exist for any other bar ** If the symbol pmpi_foo_f08__ exists, then pmpi_bar_f08__ exists for any bar in the group of _f08 routines defined in the standard, but does not exist for any other bar ** If the symbol PMPI_Foo_f08 exists, then PMPI_Bar_f08 exists for any bar in the group of _f08 routines defined in the standard, but does not exist for any other bar Is this correct and complete or am I missing any symbols. Does this all sound correct? We just wanted to make sure we understand all the nuances and have things right before the discussion on Monday. Thanks! Martin -- ________________________________________________________________________ Martin Schulz, schulzm(a)llnl.gov, http://people.llnl.gov/schulzm CASC @ Lawrence Livermore National Laboratory, Livermore, USA

1 0

[Mpi3-tools] MPI-3 Tools TelCon 10/3 - PMPI Wrappers for F2008
by Schulz, Martin 27 Sep '11

27 Sep '11

Hi all, With the MPI_T ticket progressing, it has been a while since we had an MPI-3 tools TelCon. However, during the MPI forum last week a topic came up that may be an issue for tools and Marc-Andre, who was at the forum as well, and I thought that this would be worth discussing in this group. I also cc-ed a few tool developers that we know work on MPI wrappers - if I missed anyone, please feel free to forward this. The MPI forum is currently discussing the inclusion of new bindings for Fortran 2008, as well as an update of the existing F90 bindings. This has an impact on the linker symbols that need to be intercepted by PMPI tools. In particular, tools will have to intercept up to 6 symbols for each MPI routine in order to get full coverage. Further, it will not be guaranteed that all "P" routines are always present in each installation, since this depends on a combination of compiler and linker options. The complete ticket is available at: https://svn.mpi-forum.org/trac/mpi-forum-web/ticket/229 The most relevant parts are in Chapter 14 and 16. Marc-Andre and I believe that we have reached a workable compromise (it won't make it easier for tools, but it will be at least feasible), but it would be good to have further reviews of this issue as well as a discussion on this topic to make sure we haven't missed anything. Change suggestions for this ticket need to be made two weeks in advance of the next forum, which is already end of next week. Hence, unfortunately there isn't much time. We will cover this topic at the next MPI-3 tools TelCon, which is scheduled for 10/3 at 8am PDT, 11am EDT, 5pm MSZ. Anybody is welcome to join. Here is the link (it is also on the MPI-3 tools Wiki): Monday, October 3, 2011, 11am US Eastern time * Webex meeting: https://cisco.webex.com/ciscosales/j.php?ED=175249492&UID=0&PW=NNzdlNTk4YWI x&RT=MiMxMQ%3D%3D <https://cisco.webex.com/ciscosales/j.php?ED=175249492&UID=0&PW=NNzdlNTk4YW Ix&RT=MiMxMQ%3D%3D> * Webex password: tools * Agenda: PMPI Wrappers for Fortran 2008 I apologize to the German participants since I know this is a holiday, but we need to get the feedback in time for the next forum meeting. If you can't make it, but have comments, please send email to this list. Thanks! Martin -- ________________________________________________________________________ Martin Schulz, schulzm(a)llnl.gov, http://people.llnl.gov/schulzm CASC @ Lawrence Livermore National Laboratory, Livermore, USA

1 0

[Mpi3-tools] MPI Tool Information Interface & MPI_Get_library_version passed reading
by Schulz, Martin 23 Sep '11

23 Sep '11

Hi all, Just a quick update from the MPI forum meeting in Santorini: We read both ticket #266 (MPI_T - in a quick, change only version of the reading after a quick vote in the forum) and ticket #204 (the new version call) and both passed. The MPI_T ticket had 2 or 3 very minor cosmetic change requests. I will make those changes and then post both documents in the ticket system for a vote at the next forum. Martin -- ________________________________________________________________________ Martin Schulz, schulzm(a)llnl.gov, http://people.llnl.gov/schulzm CASC @ Lawrence Livermore National Laboratory, Livermore, USA

1 0

[Mpi3-tools] Last Chance: Changes to the MPI_T*_ document
by Martin Schulz 07 Sep '11

07 Sep '11

Hi all, Any more comments on the changes to the MPI_T document? I plan on uploading it tomorrow afternoon to make the two week deadline. Thanks, Martin Begin forwarded message: > From: Martin Schulz <schulzm(a)llnl.gov> > Date: August 27, 2011 6:29:04 PM PDT > To: MPI3 Tools <mpi3-tools(a)lists.mpi-forum.org> > Subject: Re: [Mpi3-tools] Changes to the MPI_T*_ document > Reply-To: Martin Schulz <schulzm(a)llnl.gov> > > Hi all, > > I created a new version of the MPI_T document based on > the feedback from the MPI forum plus some additional > comments. The new draft is at: > > https://svn.mpi-forum.org/trac/mpi-forum-web/attachment/wiki/MPI3Tools/mpit… > > Please take a look at it and see if any more changes are > necessary. To meet the two week rule, we have until 9/8 > to post the document to the MPI forum. > > I don't think we need another tools TelCon until then (but > please speak up if you think we do or if there are any > other topics to cover), but I still would like feedback > on the questions that I sent around in the last email - > so far I have only heard from Bronis; additional feedback > would be helpful. > > > A) Naming: MPI_T or a different name? > ============================== > > I have changed all names to be MPI_T_<lowercase>, but > I have not made any further name changes. Bronis's vote > was for keeping just the "T" and that would be the easiest and > least disruptive to both the document and the existing > prototype(s). Does anybody have a good idea or a > strong opinion about this? > > > B) Types: which types to support? > ========================== > > The new document has the following list: > MPI_INT > MPI_UNSIGNED > MPI_UNSIGNED_LONG > MPI_UNSIGNED_LONG_LONG > MPI_COUNT > MPI_CHAR > MPI_DOUBLE > > With the following rationale included in the text: > > The \MPI/ tool information interface relies mainly on unsigned > datatypes for integer values since most variables are expected > to represent counters or resource sizes. \const{MPI\_INT} is > provided for additional flexibility and is expected to be used > mainly for control variables. > > Providing all basic datatypes, in particular providing all signed > and unsigned variants of integer types, would lead to a larger > number of types, which tools need to interpret. This would > cause unnecessary complexity in the implementation of tools > based on the \MPI/ tool information interface. > > > C) Signal safety > ============= > > I liked Bronis's version of the text and included it in the document. > I'll also start a separate email with Brian to get his feedback. > > > If there are any other pending issues with this document, please > let me know. I really hope we can make the next first reading the > last reading, but for this it would be extremely helpful if more > people would look over the document! > > Thanks! > > Martin > > > > > On Aug 17, 2011, at 4:37 PM, Bronis R. de Supinski wrote: > >> >> Martin: >> >> Re: >>> - Naming: People didn't like MPI_T_ and wanted something >>> more expressive. MPI_Tool_ would be the obvious choice >>> and that would be fine with me. Does anyone have a better >>> suggestion? We can't spend more than four characters on >>> it, though, or we will run into problems with the max. name >>> size (even after shortening some names). Also, I would like >>> this to be the final name change :). >> >> I still feel that MPI_T is the best choice. No matter >> what prefix you choose, someone will not like it. >> >> [snip] >> >>> - We had some lengthy discussion on the thread and async >>> signal safety. Below is a suggestion for a rewrite - once I get >>> some feedback from our group, I will forward this to Brian, >>> since he had the most issues with the current wording. >>> >>> \begin{implementors} >>> Sampling based tools rely on the ability to call the MPI >>> tool information interface, in particular routines to start, stop, >>> read, write and reset performance variables, from any program >>> context, including asynchronous contexts such as signal handlers. >>> MPI implementations should strive, if possible in their particular >> >> OK through the above. >> >>> environment, to enable this behavior for all or a subset of the >> >> Which "behavior"? You do not define any behavior above. You >> define support for usage scenarios. Change to: >> >> environment, to enable these usage scenarios for all or a subset of the >> >>> routines mentioned above. If implementing only a subset, the >>> read, write, and reset routines are typically the most critical >> >>> for sampling based tools. If an MPI implementation >>> is not able to provide such a behavior or if the behavior is only >>> guaranteed under certain circumstances (e.g., for certain signals), >>> this should be clearly documented, e.g., through the description >>> return by {\mpifunc MPI\_T\_Get\_info}. >> >> "such a"? UGH! Poor antecedent for "this". "return by"? Huh? >> >> change to: >> >> for sampling based tools. An MPI implementation should clearly >> document any restrictions on the program contexts in which >> the MPI tool information interface can be used. Restrictions >> might include guaranteeing usage outside of all signals or >> outside a specific set of signals. Any restrictions could be >> documented, for example, through the description returned by >> {\mpifunc MPI\_T\_Get\_info}. >> >>> \end{implementors} >> >> Bronis >> > > ________________________________________________________________________ > Martin Schulz, schulzm(a)llnl.gov, http://people.llnl.gov/schulzm > CASC @ Lawrence Livermore National Laboratory, Livermore, USA > > > ________________________________________________________________________ Martin Schulz, schulzm(a)llnl.gov, http://people.llnl.gov/schulzm CASC @ Lawrence Livermore National Laboratory, Livermore, USA

1 0