December 2014 - mpiwg-tools - mailman3.cels.anl.gov

next telecon
by Mohror, Kathryn 16 Dec '14

16 Dec '14

Hi all, Because of the upcoming holidays, we'll have our next meeting on January 8 2015. Talk to you then! And Happy Holidays! Kathryn _________________________________________________________________ Kathryn Mohror, kathryn(a)llnl.gov, http://scalability.llnl.gov/ Scalability Team @ Lawrence Livermore National Laboratory, Livermore, CA, USA

1 0

Re: [mpiwg-tools] Per the QMPI/PMPI+1/whatever discussion yesterday
by Martin Schulz 11 Dec '14

11 Dec '14

Hi Jeff, all, On 12/10/14, 7:59 AM, "Jeff Squyres (jsquyres)" <jsquyres(a)cisco.com> wrote: >On Dec 9, 2014, at 11:10 AM, Martin Schulz <schulzm(a)llnl.gov> wrote: > >> Is this intended as temporary internal name or for the standard at the >> end. > >Name for the standard. > >> For the former, it¹s fine, but I don¹t think it¹s a good idea to have >> a special name space in the MPI standard. The forum already hated the >>idea >> of MPIT and we had to change it to MPI_T. Also, most of these functions >> are also intended for users, so why shouldn¹t they be normal MPI >>functions? > >That's a fair point, and I don't have a good answer. > >It's just a subjective feeling that a "QMPI_" namespace is good to be >separate to indicate that you're talking with tools, not really the MPI >implementation. The two or three routines are not really directed at the tool, they inform the MPI implementation what tools want to do with it. Either way, we can discuss once we have a concrete API proposal. >> We conquered (I think?) the biggest problem we had with multiple tools, >> the specification of which tools to load (this was the main reason why I >> suggested to limit the proposal). I think the dependencies are easy, >>which >> leaves ordering. Perhaps we should have at least some support for this >> already in the proposal. > >Well, hmm. There's still multi-tool problems, though -- e.g., ordering, >pre/post side effects, replacement (i.e., tool A not getting called for >MPI_Send because tool B decided to implement MPI_Send as >MPI_Isend/MPI_Wait, etc.). > >But perhaps you could convince me. :-) > >Do you want to sketch something out and come back with some ideas? Yes, I’ll do that, but need to think about it a bit more. Martin > >-- >Jeff Squyres >jsquyres(a)cisco.com >For corporate legal information go to: >http://www.cisco.com/web/about/doing_business/legal/cri/ > >_______________________________________________ >mpiwg-tools mailing list >mpiwg-tools(a)lists.mpi-forum.org >http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-tools

1 0

Re: [mpiwg-tools] Per the QMPI/PMPI+1/whatever discussion yesterday
by Jeff Squyres (jsquyres) 10 Dec '14

10 Dec '14

On Dec 9, 2014, at 11:10 AM, Martin Schulz <schulzm(a)llnl.gov> wrote: > Is this intended as temporary internal name or for the standard at the > end. Name for the standard. > For the former, it¹s fine, but I don¹t think it¹s a good idea to have > a special name space in the MPI standard. The forum already hated the idea > of MPIT and we had to change it to MPI_T. Also, most of these functions > are also intended for users, so why shouldn¹t they be normal MPI functions? That's a fair point, and I don't have a good answer. It's just a subjective feeling that a "QMPI_" namespace is good to be separate to indicate that you're talking with tools, not really the MPI implementation. > We conquered (I think?) the biggest problem we had with multiple tools, > the specification of which tools to load (this was the main reason why I > suggested to limit the proposal). I think the dependencies are easy, which > leaves ordering. Perhaps we should have at least some support for this > already in the proposal. Well, hmm. There's still multi-tool problems, though -- e.g., ordering, pre/post side effects, replacement (i.e., tool A not getting called for MPI_Send because tool B decided to implement MPI_Send as MPI_Isend/MPI_Wait, etc.). But perhaps you could convince me. :-) Do you want to sketch something out and come back with some ideas? -- Jeff Squyres jsquyres(a)cisco.com For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/

1 0

Re: [mpiwg-tools] Per the QMPI/PMPI+1/whatever discussion yesterday
by Jeff Squyres (jsquyres) 10 Dec '14

10 Dec '14

On Dec 9, 2014, at 8:38 AM, Bland, Wesley B. <wbland(a)anl.gov> wrote: > Q* seems fine to me. I don’t really care a lot about that end of the name, but I think perhaps the QMPI_Tool_get name isn’t right anymore since we’ve changed the definition of the function from the beginning of the discussion. Now that it’s aimed at providing the tool(s) that is(are) initialized after MPI_Init, perhaps it should be something like QMPI_Tool_get_initialized(). Fair point. How about QMPI_Get_initialized_tools()? -- Jeff Squyres jsquyres(a)cisco.com For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/

1 0

Re: [mpiwg-tools] Per the QMPI/PMPI+1/whatever discussion yesterday
by Martin Schulz 09 Dec '14

09 Dec '14

On 12/9/14, 8:38 AM, "Bland, Wesley B." <wbland(a)anl.gov> wrote: > >> On Dec 9, 2014, at 5:31 AM, Jeff Squyres (jsquyres) >><jsquyres(a)cisco.com> wrote: >> >> This morning, I have a bunch of follow-on thoughts to the >>QPMI/PMPI+1/whatever tools WG face-to-face discussion yesterday. We >>were all getting tired at the end, and I don't think we documented >>everything well. Here are my thoughts this morning, in no particular >>order: >> >> 1. New names: QMPI_Tool_add / QMPI_Tool_get. In general, using the >>"QMPI_" prefix should be sufficient for distinguishing any new functions >>we create here from both the PMPI_ world and from the MPI_T_ world. >>E.g., even though the 2 names I propose here include the word "tool", I >>think the QMPI_ prefix is sufficient to distinguish them from the MPI_T_ >>tool genre. > >Q* seems fine to me. I don¹t really care a lot about that end of the >name, but I think perhaps the QMPI_Tool_get name isn¹t right anymore >since we¹ve changed the definition of the function from the beginning of >the discussion. Now that it¹s aimed at providing the tool(s) that is(are) >initialized after MPI_Init, perhaps it should be something like >QMPI_Tool_get_initialized(). Is this intended as temporary internal name or for the standard at the end. For the former, it¹s fine, but I don¹t think it¹s a good idea to have a special name space in the MPI standard. The forum already hated the idea of MPIT and we had to change it to MPI_T. Also, most of these functions are also intended for users, so why shouldn¹t they be normal MPI functions? >> >> 2. Things we didn't document yesterday: >> >> - Need to decide a free-memory model for QMPI_Tool_get (i.e., how is >>the argv that is returned via QMPI_Tool_get returned). >> - Need to specify that the strings passed in to QMPI_Tool_add are safe >>to be freed after QMPI_Tool_add returns (i.e., imply that the MPI >>implementation will make a copy of the string if it needs it) >> - Need Fortran bindings for the QMPI_* functions. >> - In the union of fields passed to the tool-provided QMPI_multiplex >>function for each, some functions will need to pass an enum to indicate >>which MPI binding the intercept came from (e.g, C, mpif.h, use mpi, use >>mpi_f08) We should also talk more about the union type and should try to spell it out for a few functions. Another concern: we need to find a way to properly represent (and distinguish) Fortran buffer descriptors. >> - In all cases, the tool-provided QMPI_multiplex function will be >>passed the C versions of MPI handles. I.e., if a Fortran program calls >>MPI_Send which ends up invoking a tool's QMPI_multiplex function, the >>QMPI_multiplex function will receive C versions of the communicator and >>datatype handles. >> >> 3. Random thought -- do we need QMPI_Control? This would be analogous >>to MPI_Control -- it would be used for communicating directly with >>tools. Its first argument could be a (const char*) that corresponds to >>the strings that you get back from QMPI_Tool_get (i.e., allowing an app >>to communicate with a specific tool). > >What is the benefit of having this in lieu of just having tool specific >functions? If you (the application) know enough about the tool to know >what to pass to QMPI_Control, why not just call a function directly from >the tool¹s library? We definitely need a QMPI_Qcontrol - two reasons: for one, some of the information provided should be tool independent (where to start or stop profiling or how to mark phases is application characteristics and not tool specific); second, we need to be able to leave these annotations in the code and be able to link the application without the tool. Only f the tool is attached, the Qcontrol gets called. This could be as simple as including the function in the list of wrappable functions. > >> >> 4. We haven't crisply laid out the goals for QMPI/PMPI+1/whatever. >>That is -- to paraphrase Kathryn's sentiments from yesterday: how is >>this interface *better* than the current PMPI world? I think the goals >>are: >> >> - Avoid relying on linker tricks (although this is kinda weak -- linker >>tricks work just fine in many cases, and, indeed, we're not outlawing >>linker tricks like LD_PRELOAD) >> - Solve all language bindings issues (e.g., allow tools to be written >>in C) >> - *Eventually* allow supporting N tools simultaneously >> --> Although this is just the first step in supporting that eventual >>goal We conquered (I think?) the biggest problem we had with multiple tools, the specification of which tools to load (this was the main reason why I suggested to limit the proposal). I think the dependencies are easy, which leaves ordering. Perhaps we should have at least some support for this already in the proposal. >> >> And again paraphrasing Kathryn (who was channeling Adam): given that >>the plan is that MPI-4 won't support N simultaneous tools, are these >>goals enough "better" than the current PMPI to be worth doing? >> >> I think we need a crisp, irrefutable answer for that question before >>going before the entire Forum. > >This seems like a good start. We should also try not to disallow multiple >tools as things are now. We can just stay that it¹s undefined behavior, >and the implementation is free to support it if desired with random >ordering (or something). I agree - there is no reason not to support an ordered list of tools at this point already. Martin > >Thanks, >Wesley > >> >> -- >> Jeff Squyres >> jsquyres(a)cisco.com >> For corporate legal information go to: >>http://www.cisco.com/web/about/doing_business/legal/cri/ >> >> _______________________________________________ >> mpiwg-tools mailing list >> mpiwg-tools(a)lists.mpi-forum.org >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-tools > >_______________________________________________ >mpiwg-tools mailing list >mpiwg-tools(a)lists.mpi-forum.org >http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-tools

1 0

Re: [mpiwg-tools] Per the QMPI/PMPI+1/whatever discussion yesterday
by Bland, Wesley B. 09 Dec '14

09 Dec '14

> On Dec 9, 2014, at 5:31 AM, Jeff Squyres (jsquyres) <jsquyres(a)cisco.com> wrote: > > This morning, I have a bunch of follow-on thoughts to the QPMI/PMPI+1/whatever tools WG face-to-face discussion yesterday. We were all getting tired at the end, and I don't think we documented everything well. Here are my thoughts this morning, in no particular order: > > 1. New names: QMPI_Tool_add / QMPI_Tool_get. In general, using the "QMPI_" prefix should be sufficient for distinguishing any new functions we create here from both the PMPI_ world and from the MPI_T_ world. E.g., even though the 2 names I propose here include the word "tool", I think the QMPI_ prefix is sufficient to distinguish them from the MPI_T_ tool genre. Q* seems fine to me. I don’t really care a lot about that end of the name, but I think perhaps the QMPI_Tool_get name isn’t right anymore since we’ve changed the definition of the function from the beginning of the discussion. Now that it’s aimed at providing the tool(s) that is(are) initialized after MPI_Init, perhaps it should be something like QMPI_Tool_get_initialized(). > > 2. Things we didn't document yesterday: > > - Need to decide a free-memory model for QMPI_Tool_get (i.e., how is the argv that is returned via QMPI_Tool_get returned). > - Need to specify that the strings passed in to QMPI_Tool_add are safe to be freed after QMPI_Tool_add returns (i.e., imply that the MPI implementation will make a copy of the string if it needs it) > - Need Fortran bindings for the QMPI_* functions. > - In the union of fields passed to the tool-provided QMPI_multiplex function for each, some functions will need to pass an enum to indicate which MPI binding the intercept came from (e.g, C, mpif.h, use mpi, use mpi_f08) > - In all cases, the tool-provided QMPI_multiplex function will be passed the C versions of MPI handles. I.e., if a Fortran program calls MPI_Send which ends up invoking a tool's QMPI_multiplex function, the QMPI_multiplex function will receive C versions of the communicator and datatype handles. > > 3. Random thought -- do we need QMPI_Control? This would be analogous to MPI_Control -- it would be used for communicating directly with tools. Its first argument could be a (const char*) that corresponds to the strings that you get back from QMPI_Tool_get (i.e., allowing an app to communicate with a specific tool). What is the benefit of having this in lieu of just having tool specific functions? If you (the application) know enough about the tool to know what to pass to QMPI_Control, why not just call a function directly from the tool’s library? > > 4. We haven't crisply laid out the goals for QMPI/PMPI+1/whatever. That is -- to paraphrase Kathryn's sentiments from yesterday: how is this interface *better* than the current PMPI world? I think the goals are: > > - Avoid relying on linker tricks (although this is kinda weak -- linker tricks work just fine in many cases, and, indeed, we're not outlawing linker tricks like LD_PRELOAD) > - Solve all language bindings issues (e.g., allow tools to be written in C) > - *Eventually* allow supporting N tools simultaneously > --> Although this is just the first step in supporting that eventual goal > > And again paraphrasing Kathryn (who was channeling Adam): given that the plan is that MPI-4 won't support N simultaneous tools, are these goals enough "better" than the current PMPI to be worth doing? > > I think we need a crisp, irrefutable answer for that question before going before the entire Forum. This seems like a good start. We should also try not to disallow multiple tools as things are now. We can just stay that it’s undefined behavior, and the implementation is free to support it if desired with random ordering (or something). Thanks, Wesley > > -- > Jeff Squyres > jsquyres(a)cisco.com > For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/ > > _______________________________________________ > mpiwg-tools mailing list > mpiwg-tools(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-tools

1 0

Per the QMPI/PMPI+1/whatever discussion yesterday
by Jeff Squyres (jsquyres) 09 Dec '14

09 Dec '14

This morning, I have a bunch of follow-on thoughts to the QPMI/PMPI+1/whatever tools WG face-to-face discussion yesterday. We were all getting tired at the end, and I don't think we documented everything well. Here are my thoughts this morning, in no particular order: 1. New names: QMPI_Tool_add / QMPI_Tool_get. In general, using the "QMPI_" prefix should be sufficient for distinguishing any new functions we create here from both the PMPI_ world and from the MPI_T_ world. E.g., even though the 2 names I propose here include the word "tool", I think the QMPI_ prefix is sufficient to distinguish them from the MPI_T_ tool genre. 2. Things we didn't document yesterday: - Need to decide a free-memory model for QMPI_Tool_get (i.e., how is the argv that is returned via QMPI_Tool_get returned). - Need to specify that the strings passed in to QMPI_Tool_add are safe to be freed after QMPI_Tool_add returns (i.e., imply that the MPI implementation will make a copy of the string if it needs it) - Need Fortran bindings for the QMPI_* functions. - In the union of fields passed to the tool-provided QMPI_multiplex function for each, some functions will need to pass an enum to indicate which MPI binding the intercept came from (e.g, C, mpif.h, use mpi, use mpi_f08) - In all cases, the tool-provided QMPI_multiplex function will be passed the C versions of MPI handles. I.e., if a Fortran program calls MPI_Send which ends up invoking a tool's QMPI_multiplex function, the QMPI_multiplex function will receive C versions of the communicator and datatype handles. 3. Random thought -- do we need QMPI_Control? This would be analogous to MPI_Control -- it would be used for communicating directly with tools. Its first argument could be a (const char*) that corresponds to the strings that you get back from QMPI_Tool_get (i.e., allowing an app to communicate with a specific tool). 4. We haven't crisply laid out the goals for QMPI/PMPI+1/whatever. That is -- to paraphrase Kathryn's sentiments from yesterday: how is this interface *better* than the current PMPI world? I think the goals are: - Avoid relying on linker tricks (although this is kinda weak -- linker tricks work just fine in many cases, and, indeed, we're not outlawing linker tricks like LD_PRELOAD) - Solve all language bindings issues (e.g., allow tools to be written in C) - *Eventually* allow supporting N tools simultaneously --> Although this is just the first step in supporting that eventual goal And again paraphrasing Kathryn (who was channeling Adam): given that the plan is that MPI-4 won't support N simultaneous tools, are these goals enough "better" than the current PMPI to be worth doing? I think we need a crisp, irrefutable answer for that question before going before the entire Forum. -- Jeff Squyres jsquyres(a)cisco.com For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/

1 0

Fwd: (Forward to others) Meeting invitation: MPI Forum tools WG
by Jeff Squyres (jsquyres) 08 Dec '14

08 Dec '14

Here's the webex info for the tools WG meeting today: 2-5:30pm US Pacific time. Join if you can / want to. I'm anticipating that we'll be a bit late for the meeting today (e.g., 2:15pm), because 2pm is right when the MPI Forum itself starts. Begin forwarded message: From: Jeff Squyres <messenger(a)webex.com<mailto:[email protected]>> Subject: (Forward to others) Meeting invitation: MPI Forum tools WG Date: December 8, 2014 at 12:57:07 PM PST To: <jsquyres(a)cisco.com<mailto:[email protected]>> Reply-To: <jsquyres(a)cisco.com<mailto:[email protected]>> You can forward this invitation to others. Hello , Jeff Squyres invites you to attend this online meeting. Topic: MPI Forum tools WG Date: Monday, December 8, 2014 Time: 5:00 pm, Eastern Standard Time (New York, GMT-05:00) Meeting Number: 202 232 565 Meeting Password: tools ------------------------------------------------------- To join the online meeting (Now from mobile devices!) ------------------------------------------------------- 1. Go to https://cisco.webex.com/ciscosales/j.php?MTID=m49a64c34ced520e82a14bca7d5a8… 2. Enter your name and email address. 3. Enter the meeting password: tools 4. Click "Join Now". To view in other time zones or languages, please click the link: https://cisco.webex.com/ciscosales/j.php?MTID=maddf14a91245c7dd3e91f23cf9bf… ---------------------------------------------------------------- ALERT � PLEASE READ: DO NOT DIAL THE TOLL FREE NUMBERS FROM WITHIN THE (408) OR (919) AREA CODES ---------------------------------------------------------------- Please dial the local access number for your area from the list below: - San Jose/Milpitas (408) area: 525-6800 - RTP (919) area: 392-3330 Dialing the WebEx toll free numbers from within 408 or 919 area codes is not enabled (non-Cisco phones). � If you dial the toll-free numbers within the 408 or 919 area codes you will be instructed to hang up and dial the local access number.� Please use the call-back option whenever possible and otherwise dial local numbers only. The affected toll free numbers are: (866) 432-9903 for the San Jose/Milpitas area and (866) 349-3520 for the RTP area. ------------------------------------------------------- To join the teleconference only ------------------------------------------------------- 1. Dial into Cisco WebEx (view all Global Access Numbers at http://cisco.com/en/US/about/doing_business/conferencing/index.html 2. Follow the prompts to enter the Meeting Number (listed above) or Access Code followed by the # sign. San Jose, CA: +1.408.525.6800 RTP: +1.919.392.3330 US/Canada: +1.866.432.9903 United Kingdom: +44.20.8824.0117 India: +91.80.4350.1111 Germany: +49.619.6773.9002 Japan: +81.3.5763.9394 China: +86.10.8515.5666 ------------------------------------------------------- For assistance ------------------------------------------------------- 1. Go to https://cisco.webex.com/ciscosales/mc 2. On the left navigation bar, click "Support". You can contact me at: jsquyres(a)cisco.com<mailto:[email protected]> 1-408-525 0971 Add this meeting to your calendar: https://cisco.webex.com/ciscosales/j.php?MTID=m9eb84980b4029af375b908ab71c1… http://www.webex.com<http://www.webex.com/> CCP:+14085256800x202232565# IMPORTANT NOTICE: This WebEx service includes a feature that allows audio and any documents and other materials exchanged or viewed during the session to be recorded. By joining this session, you automatically consent to such recordings. If you do not consent to the recording, discuss your concerns with the meeting host prior to the start of the recording or do not join the session. Please note that any such recordings may be subject to discovery in the event of litigation. -- Jeff Squyres jsquyres(a)cisco.com<mailto:[email protected]> For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/

1 0