September 2009 - mpiwg-tools - mailman3.cels.anl.gov

Re: [Mpi3-tools] MPI3 tools WG Mission Statement
by Chris Gottbrath 02 Sep '09

02 Sep '09

Marc-Andre, I think adding debuggers explicitly for clarity is good. I do think the original 'correctness tool' language covered debuggers. The taxonomy, as I see it, is something along the following lines. Software development tools includes: Correctness tools and performance tools (and other categories) Correctness tools includes: Debuggers (such as TV) and correctness checking tools (such as Marmot) Cheers, Chris Chris Gottbrath, 508-652-7735 or 774-270-3155 Director of Product Management, TotalView Technologies chris.gottbrath(a)totalviewtech.com -- Learn how to radically simplify your debugging: http://www.totalviewtech.com/support/white_papers.html?id=163 ********** This transmission contains confidential and/or legally privileged information from TotalView Technologies intended only for the use of the individual(s) to which it is addressed. If you are not the intended recipient, you are hereby notified that any disclosure, copying or distribution of this information or the taking of any action in reliance on the contents of this transmission is strictly prohibited. If you have received this transmission in error, please notify us immediately. ********** On Sep 2, 2009, at 2:46 PM, Marc-Andre Hermanns wrote: > Hi Martin, > >> The goal of this working group is define interfaces that can be used >> by both performance and correctness tools to gain information about >> internals of the MPI library, interactions between an application and >> the MPI library, as well as the system environment an MPI application >> runs in. This is intended as an extension to the existing and widely >> used PMPI interface. These efforts will help in providing reliable >> and portable interfaces for MPI tools with new functionality >> currently not covered by the MPI standard. > > Does 'correctness' tools include debuggers? It immediately triggered > things like "Marmot" and not directly Totalview, DDT, etc., as I would > usually not refer to a debugger as a correctness checker (but then, > I am > not a native speaker). > > To make that a little clearer I would propose the following change: > > The goal of this working group is to define interfaces that can be > used > by third party tools to obtain internal information about the MPI > library, as well as the system environment an MPI application runs in. > These tools include performance analysis tools, correctness checkers > and > debuggers. These interfaces are complementary to the existing PMPI > interface. These efforts will help in providing reliable and portable > interfaces for MPI tools with new functionality currently not > covered by > the MPI standard. > >> In particular, this WG is aiming at the standardization of the >> following interfaces: >> - A scalable version of a process acquisition interface to identify >> and locate all >> processes which are part of an MPI job. >> - An interface to look for and locate tool DLLs. >> - An interface to inspect message queues. >> - An interface to understand progress in collective communication. >> - An interface to query additional semantic information for opaque >> MPI >> handles. >> - A performance information interface that allows MPI implementations >> to export >> additional low level performance information including, but not >> limited to, the >> operating state of the MPI library. > > The time frame that was discussed today for the MPI 3.x effort was 3+1 > years. That is 3 years until 3.0 is ready, and an additional year for > something like 3.1 (Jeff, feel free to correct me on this). I don't > have > the experience yet, to estimate whether it is doable in this time > frame, > but the list seems to be quite long. All the items on the list seems > valid points of interest, though. > > Best regards, > Marc-Andre > -- > Marc-Andre Hermanns > Juelich Supercomputing Centre > Institute for Advanced Simulation > Forschungszentrum Juelich GmbH > D-52425 Juelich > Germany > > Phone : +49-2461-61-2054 > Fax : +49-2461-61-6656 > eMail : m.a.hermanns(a)fz-juelich.de > WWW : http://www.fz-juelich.de/jsc/ > > JSC is the coordinator of the > John von Neumann Institute for Computing > and member of the > Gauss Centre for Supercomputing > > Sitz der Gesellschaft: Juelich > Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 > Vorsitzende des Aufsichtsrats: MinDir'in Baerbel Brumme-Bothe > Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender), > Dr. Ulrich Krafft (stellv. Vorsitzender), > Prof. Dr. Harald Bolt, > Prof. Dr. Sebastian M. Schmidt > _______________________________________________ > Mpi3-tools mailing list > Mpi3-tools(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-tools

1 0

Re: [Mpi3-tools] MPI3 tools WG Mission Statement
by Martin Schulz 02 Sep '09

02 Sep '09

On Sep 2, 2009, at 1:46 PM, Marc-Andre Hermanns wrote: > Hi Martin, > >> The goal of this working group is define interfaces that can be used >> by both performance and correctness tools to gain information about >> internals of the MPI library, interactions between an application and >> the MPI library, as well as the system environment an MPI application >> runs in. This is intended as an extension to the existing and widely >> used PMPI interface. These efforts will help in providing reliable >> and portable interfaces for MPI tools with new functionality >> currently not covered by the MPI standard. > > Does 'correctness' tools include debuggers? It immediately triggered > things like "Marmot" and not directly Totalview, DDT, etc., as I would > usually not refer to a debugger as a correctness checker (but then, > I am > not a native speaker). This was intended to cover both types of tools: correctness checkers and debuggers. > > To make that a little clearer I would propose the following change: > > The goal of this working group is to define interfaces that can be > used > by third party tools to obtain internal information about the MPI > library, as well as the system environment an MPI application runs in. > These tools include performance analysis tools, correctness checkers > and > debuggers. These interfaces are complementary to the existing PMPI > interface. These efforts will help in providing reliable and portable > interfaces for MPI tools with new functionality currently not > covered by > the MPI standard. Works for me. > >> In particular, this WG is aiming at the standardization of the >> following interfaces: >> - A scalable version of a process acquisition interface to identify >> and locate all >> processes which are part of an MPI job. >> - An interface to look for and locate tool DLLs. >> - An interface to inspect message queues. >> - An interface to understand progress in collective communication. >> - An interface to query additional semantic information for opaque >> MPI >> handles. >> - A performance information interface that allows MPI implementations >> to export >> additional low level performance information including, but not >> limited to, the >> operating state of the MPI library. > > The time frame that was discussed today for the MPI 3.x effort was 3+1 > years. That is 3 years until 3.0 is ready, and an additional year for > something like 3.1 (Jeff, feel free to correct me on this). I don't > have > the experience yet, to estimate whether it is doable in this time > frame, > but the list seems to be quite long. All the items on the list seems > valid points of interest, though. Most of these items have already been discussed and we have concrete proposals on the table and they are in the process of being written up. The only remaining item is the process acquisition interface. Nevertheless, this is a lot of work and I am worried about this new March deadline for getting this done to be included in the MPI 3 standard. We should be able to get some of these items through, but it will a lot of work to make this work for all items (or even just the ones for which we have concrete ideas), in particular, since this schedule seems not to allow for any substantial feedback from the forum - basically, it seems that if there are even small objections that require a rewrite, the proposal already ends up in 3.1 or later. Has this been discussed? Martin > > Best regards, > Marc-Andre > -- > Marc-Andre Hermanns > Juelich Supercomputing Centre > Institute for Advanced Simulation > Forschungszentrum Juelich GmbH > D-52425 Juelich > Germany > > Phone : +49-2461-61-2054 > Fax : +49-2461-61-6656 > eMail : m.a.hermanns(a)fz-juelich.de > WWW : http://www.fz-juelich.de/jsc/ > > JSC is the coordinator of the > John von Neumann Institute for Computing > and member of the > Gauss Centre for Supercomputing > > Sitz der Gesellschaft: Juelich > Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 > Vorsitzende des Aufsichtsrats: MinDir'in Baerbel Brumme-Bothe > Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender), > Dr. Ulrich Krafft (stellv. Vorsitzender), > Prof. Dr. Harald Bolt, > Prof. Dr. Sebastian M. Schmidt > _______________________________________________ > Mpi3-tools mailing list > Mpi3-tools(a)lists.mpi-forum.org > http://*lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-tools _______________________________________________________________________ Martin Schulz, schulzm(a)llnl.gov, http://people.llnl.gov/schulzm CASC @ Lawrence Livermore National Laboratory, Livermore, USA

1 0

Re: [Mpi3-tools] MPI3 tools WG Mission Statement
by Martin Schulz 02 Sep '09

02 Sep '09

Hi Bronis and Rajeev, Thanks for the quick feedback. All: I have posted this draft on the Tools WG Wiki and I hope we can finalize this tomorrow during the WG meeting. Thanks, Martin On Sep 2, 2009, at 1:32 PM, Bronis R. de Supinski wrote: > > I agree. I have no edits to suggest for this informal document. > > On Wed, 2 Sep 2009, Rajeev Thakur wrote: > >> This looks very good. >> >> Rajeev >> >>> -----Original Message----- >>> From: mpi3-tools-bounces(a)lists.mpi-forum.org >>> [mailto:[email protected]] On Behalf Of >>> Martin Schulz >>> Sent: Wednesday, September 02, 2009 10:02 AM >>> To: mpi3-tools(a)lists.mpi-forum.org >>> Subject: [Mpi3-tools] MPI3 tools WG Mission Statement >>> >>> Hi all, >>> >>> Below is a first draft of a mission statement for the tools >>> WG that we >>> were asked to >>> provide. I would like to discuss this with the whole group at >>> tomorrow's WG meeting. >>> Please feel free to also comment via email if you can't make >>> it to the >>> meeting or >>> if have any urgent comments (seems the MPI Forum seems to be >>> reviewing >>> some >>> of these already). >>> >>> Thanks, >>> >>> Martin >>> >>> >>> The goal of this working group is define interfaces that can be used >>> by both performance >>> and correctness tools to gain information about internals of the MPI >>> library, interactions >>> between an application and the MPI library, as well as the system >>> environment an MPI >>> application runs in. This is intended as an extension to the >>> existing >>> and widely used >>> PMPI interface. These efforts will help in providing reliable and >>> portable interfaces for >>> MPI tools with new functionality currently not covered by the MPI >>> standard. >>> >>> In particular, this WG is aiming at the standardization of the >>> following interfaces: >>> - A scalable version of a process acquisition interface to identify >>> and locate all >>> processes which are part of an MPI job. >>> - An interface to look for and locate tool DLLs. >>> - An interface to inspect message queues. >>> - An interface to understand progress in collective communication. >>> - An interface to query additional semantic information for >>> opaque MPI >>> handles. >>> - A performance information interface that allows MPI >>> implementations >>> to export >>> additional low level performance information including, but not >>> limited to, the >>> operating state of the MPI library. >>> >>> >>> ______________________________________________________________ >>> _________ >>> Martin Schulz, schulzm(a)llnl.gov, http://**people.llnl.gov/schulzm >>> CASC @ Lawrence Livermore National Laboratory, Livermore, USA >>> >>> >>> >>> >>> >>> >>> >>> _______________________________________________ >>> Mpi3-tools mailing list >>> Mpi3-tools(a)lists.mpi-forum.org >>> http://**lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-tools >>> >> >> _______________________________________________ >> Mpi3-tools mailing list >> Mpi3-tools(a)lists.mpi-forum.org >> http://**lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-tools >> >> > _______________________________________________ > Mpi3-tools mailing list > Mpi3-tools(a)lists.mpi-forum.org > http://*lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-tools > _______________________________________________________________________ Martin Schulz, schulzm(a)llnl.gov, http://people.llnl.gov/schulzm CASC @ Lawrence Livermore National Laboratory, Livermore, USA

1 0

Re: [Mpi3-tools] MPI3 tools WG Mission Statement
by Marc-Andre Hermanns 02 Sep '09

02 Sep '09

Hi Martin, > The goal of this working group is define interfaces that can be used > by both performance and correctness tools to gain information about > internals of the MPI library, interactions between an application and > the MPI library, as well as the system environment an MPI application > runs in. This is intended as an extension to the existing and widely > used PMPI interface. These efforts will help in providing reliable > and portable interfaces for MPI tools with new functionality > currently not covered by the MPI standard. Does 'correctness' tools include debuggers? It immediately triggered things like "Marmot" and not directly Totalview, DDT, etc., as I would usually not refer to a debugger as a correctness checker (but then, I am not a native speaker). To make that a little clearer I would propose the following change: The goal of this working group is to define interfaces that can be used by third party tools to obtain internal information about the MPI library, as well as the system environment an MPI application runs in. These tools include performance analysis tools, correctness checkers and debuggers. These interfaces are complementary to the existing PMPI interface. These efforts will help in providing reliable and portable interfaces for MPI tools with new functionality currently not covered by the MPI standard. > In particular, this WG is aiming at the standardization of the > following interfaces: > - A scalable version of a process acquisition interface to identify > and locate all > processes which are part of an MPI job. > - An interface to look for and locate tool DLLs. > - An interface to inspect message queues. > - An interface to understand progress in collective communication. > - An interface to query additional semantic information for opaque MPI > handles. > - A performance information interface that allows MPI implementations > to export > additional low level performance information including, but not > limited to, the > operating state of the MPI library. The time frame that was discussed today for the MPI 3.x effort was 3+1 years. That is 3 years until 3.0 is ready, and an additional year for something like 3.1 (Jeff, feel free to correct me on this). I don't have the experience yet, to estimate whether it is doable in this time frame, but the list seems to be quite long. All the items on the list seems valid points of interest, though. Best regards, Marc-Andre -- Marc-Andre Hermanns Juelich Supercomputing Centre Institute for Advanced Simulation Forschungszentrum Juelich GmbH D-52425 Juelich Germany Phone : +49-2461-61-2054 Fax : +49-2461-61-6656 eMail : m.a.hermanns(a)fz-juelich.de WWW : http://www.fz-juelich.de/jsc/ JSC is the coordinator of the John von Neumann Institute for Computing and member of the Gauss Centre for Supercomputing Sitz der Gesellschaft: Juelich Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 Vorsitzende des Aufsichtsrats: MinDir'in Baerbel Brumme-Bothe Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender), Dr. Ulrich Krafft (stellv. Vorsitzender), Prof. Dr. Harald Bolt, Prof. Dr. Sebastian M. Schmidt

1 0

Re: [Mpi3-tools] MPI3 tools WG Mission Statement
by Bronis R. de Supinski 02 Sep '09

02 Sep '09

I agree. I have no edits to suggest for this informal document. On Wed, 2 Sep 2009, Rajeev Thakur wrote: > This looks very good. > > Rajeev > > > -----Original Message----- > > From: mpi3-tools-bounces(a)lists.mpi-forum.org > > [mailto:[email protected]] On Behalf Of > > Martin Schulz > > Sent: Wednesday, September 02, 2009 10:02 AM > > To: mpi3-tools(a)lists.mpi-forum.org > > Subject: [Mpi3-tools] MPI3 tools WG Mission Statement > > > > Hi all, > > > > Below is a first draft of a mission statement for the tools > > WG that we > > were asked to > > provide. I would like to discuss this with the whole group at > > tomorrow's WG meeting. > > Please feel free to also comment via email if you can't make > > it to the > > meeting or > > if have any urgent comments (seems the MPI Forum seems to be > > reviewing > > some > > of these already). > > > > Thanks, > > > > Martin > > > > > > The goal of this working group is define interfaces that can be used > > by both performance > > and correctness tools to gain information about internals of the MPI > > library, interactions > > between an application and the MPI library, as well as the system > > environment an MPI > > application runs in. This is intended as an extension to the > > existing > > and widely used > > PMPI interface. These efforts will help in providing reliable and > > portable interfaces for > > MPI tools with new functionality currently not covered by the MPI > > standard. > > > > In particular, this WG is aiming at the standardization of the > > following interfaces: > > - A scalable version of a process acquisition interface to identify > > and locate all > > processes which are part of an MPI job. > > - An interface to look for and locate tool DLLs. > > - An interface to inspect message queues. > > - An interface to understand progress in collective communication. > > - An interface to query additional semantic information for > > opaque MPI > > handles. > > - A performance information interface that allows MPI > > implementations > > to export > > additional low level performance information including, but not > > limited to, the > > operating state of the MPI library. > > > > > > ______________________________________________________________ > > _________ > > Martin Schulz, schulzm(a)llnl.gov, http://*people.llnl.gov/schulzm > > CASC @ Lawrence Livermore National Laboratory, Livermore, USA > > > > > > > > > > > > > > > > _______________________________________________ > > Mpi3-tools mailing list > > Mpi3-tools(a)lists.mpi-forum.org > > http://*lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-tools > > > > _______________________________________________ > Mpi3-tools mailing list > Mpi3-tools(a)lists.mpi-forum.org > http://*lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-tools > >

1 0

Re: [Mpi3-tools] MPI3 tools WG Mission Statement
by Rajeev Thakur 02 Sep '09

02 Sep '09

This looks very good. Rajeev > -----Original Message----- > From: mpi3-tools-bounces(a)lists.mpi-forum.org > [mailto:[email protected]] On Behalf Of > Martin Schulz > Sent: Wednesday, September 02, 2009 10:02 AM > To: mpi3-tools(a)lists.mpi-forum.org > Subject: [Mpi3-tools] MPI3 tools WG Mission Statement > > Hi all, > > Below is a first draft of a mission statement for the tools > WG that we > were asked to > provide. I would like to discuss this with the whole group at > tomorrow's WG meeting. > Please feel free to also comment via email if you can't make > it to the > meeting or > if have any urgent comments (seems the MPI Forum seems to be > reviewing > some > of these already). > > Thanks, > > Martin > > > The goal of this working group is define interfaces that can be used > by both performance > and correctness tools to gain information about internals of the MPI > library, interactions > between an application and the MPI library, as well as the system > environment an MPI > application runs in. This is intended as an extension to the > existing > and widely used > PMPI interface. These efforts will help in providing reliable and > portable interfaces for > MPI tools with new functionality currently not covered by the MPI > standard. > > In particular, this WG is aiming at the standardization of the > following interfaces: > - A scalable version of a process acquisition interface to identify > and locate all > processes which are part of an MPI job. > - An interface to look for and locate tool DLLs. > - An interface to inspect message queues. > - An interface to understand progress in collective communication. > - An interface to query additional semantic information for > opaque MPI > handles. > - A performance information interface that allows MPI > implementations > to export > additional low level performance information including, but not > limited to, the > operating state of the MPI library. > > > ______________________________________________________________ > _________ > Martin Schulz, schulzm(a)llnl.gov, http://people.llnl.gov/schulzm > CASC @ Lawrence Livermore National Laboratory, Livermore, USA > > > > > > > > _______________________________________________ > Mpi3-tools mailing list > Mpi3-tools(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-tools >

1 0

[Mpi3-tools] MPI3 tools WG Mission Statement
by Martin Schulz 02 Sep '09

02 Sep '09

Hi all, Below is a first draft of a mission statement for the tools WG that we were asked to provide. I would like to discuss this with the whole group at tomorrow's WG meeting. Please feel free to also comment via email if you can't make it to the meeting or if have any urgent comments (seems the MPI Forum seems to be reviewing some of these already). Thanks, Martin The goal of this working group is define interfaces that can be used by both performance and correctness tools to gain information about internals of the MPI library, interactions between an application and the MPI library, as well as the system environment an MPI application runs in. This is intended as an extension to the existing and widely used PMPI interface. These efforts will help in providing reliable and portable interfaces for MPI tools with new functionality currently not covered by the MPI standard. In particular, this WG is aiming at the standardization of the following interfaces: - A scalable version of a process acquisition interface to identify and locate all processes which are part of an MPI job. - An interface to look for and locate tool DLLs. - An interface to inspect message queues. - An interface to understand progress in collective communication. - An interface to query additional semantic information for opaque MPI handles. - A performance information interface that allows MPI implementations to export additional low level performance information including, but not limited to, the operating state of the MPI library. _______________________________________________________________________ Martin Schulz, schulzm(a)llnl.gov, http://people.llnl.gov/schulzm CASC @ Lawrence Livermore National Laboratory, Livermore, USA

1 0

Re: [Mpi3-tools] WG mission statement / bullets
by Jeff Squyres 02 Sep '09

02 Sep '09

On Sep 2, 2009, at 4:56 PM, Martin Schulz wrote: > > Can you publish the list of the WG goals on the wiki ASAP? We're > > looking for them to review here in Helsinki and can't find them. > > I was told we need to have them ready for the meeting in Portland, not > Helsinki. I have a draft almost ready, which I will send out shortly, > but I wanted to discuss it with the whole group during the meeting > tomorrow > before bringing it to the forum. > I think the goal was to review them now (Helsinki meeting) and then present them in Portland at the BOF. I don't know if that was conveyed clearly or not. -- Jeff Squyres jsquyres(a)cisco.com

1 0

Re: [Mpi3-tools] WG mission statement / bullets
by Martin Schulz 02 Sep '09

02 Sep '09

Hi Jeff, On Sep 2, 2009, at 5:49 AM, Jeff Squyres wrote: > Martin -- > > Can you publish the list of the WG goals on the wiki ASAP? We're > looking for them to review here in Helsinki and can't find them. I was told we need to have them ready for the meeting in Portland, not Helsinki. I have a draft almost ready, which I will send out shortly, but I wanted to discuss it with the whole group during the meeting tomorrow before bringing it to the forum. Martin > > Thanks! > > -- > Jeff Squyres > jsquyres(a)cisco.com > > _______________________________________________ > Mpi3-tools mailing list > Mpi3-tools(a)lists.mpi-forum.org > http://*lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-tools > _______________________________________________________________________ Martin Schulz, schulzm(a)llnl.gov, http://people.llnl.gov/schulzm CASC @ Lawrence Livermore National Laboratory, Livermore, USA

1 0

[Mpi3-tools] WG mission statement / bullets
by Jeff Squyres 02 Sep '09

02 Sep '09

Martin -- Can you publish the list of the WG goals on the wiki ASAP? We're looking for them to review here in Helsinki and can't find them. Thanks! -- Jeff Squyres jsquyres(a)cisco.com

1 0