March 2012 - mpiwg-tools - mailman3.cels.anl.gov

[Mpi3-tools] Next TelCon, Monday April 2nd, 8am PDT
by Martin Schulz 30 Mar '12

30 Mar '12

Hi all, Just as a reminder - our next MPI-3.x tools TelCon is next Monday, April 2nd, at 8am PDT, 11am EDT, 5pm MSZ. As the main point on the agenda, Jeff will lead the continuation of the discussion on the MPI handles interface. Login information for WebEx is on the Wiki. Martin ________________________________________________________________________ Martin Schulz, schulzm(a)llnl.gov, http://people.llnl.gov/schulzm CASC @ Lawrence Livermore National Laboratory, Livermore, USA

1 0

Re: [Mpi3-tools] Next TelCon 3/19 - topic debugging interfaces
by Martin Schulz 16 Mar '12

16 Mar '12

Hi all, Just as a note that I forgot in the last email - the US already switched to daylight savings time. Hence, this call will actually be at 8am PDT and there is currently an 8 hour difference between the US westcoast and Europe (instead of 9). Martin On Mar 16, 2012, at 12:04 AM, Martin Schulz wrote: > Just as a reminder for our TelCon on Monday. > > Martin > > > Begin forwarded message: > >> From: "Schulz, Martin" <schulz6(a)llnl.gov> >> Date: March 8, 2012 9:08:43 PM PST >> To: MPI3 Tools <mpi3-tools(a)lists.mpi-forum.org> >> Subject: Next TelCon 3/19 - topic debugging interfaces >> Reply-To: Martin Schulz <schulzm(a)llnl.gov> >> >> Hi all, >> >> Our next TelCon for the MPI tools WG is scheduled for 3/19 at 8am PST. The login information is on the Wiki at: >> >> https://svn.mpi-forum.org/trac/mpi-forum-web/wiki/MPI3Tools >> >> Jeff will go over the various debugging interface proposals, which we are planning to turn into officially MPI forum blessed side documents (as we did for MPIR). In particular, he plans to address the handle API proposal, which is described at: >> >> https://svn.mpi-forum.org/trac/mpi-forum-web/wiki/MPI3Tools/handleapi >> >> It would be great if everybody could familiarize themselves with this proposal (hence the early reminder). >> >> Thanks! >> >> Martin >> >> >> >> ________________________________________________________________________ >> Martin Schulz, schulzm(a)llnl.gov, http://people.llnl.gov/schulzm >> CASC @ Lawrence Livermore National Laboratory, Livermore, USA >> >> >> > > ________________________________________________________________________ > Martin Schulz, schulzm(a)llnl.gov, http://people.llnl.gov/schulzm > CASC @ Lawrence Livermore National Laboratory, Livermore, USA > > > ________________________________________________________________________ Martin Schulz, schulzm(a)llnl.gov, http://people.llnl.gov/schulzm CASC @ Lawrence Livermore National Laboratory, Livermore, USA

1 0

[Mpi3-tools] Fwd: Next TelCon 3/19 - topic debugging interfaces
by Martin Schulz 16 Mar '12

16 Mar '12

Just as a reminder for our TelCon on Monday. Martin Begin forwarded message: > From: "Schulz, Martin" <schulz6(a)llnl.gov> > Date: March 8, 2012 9:08:43 PM PST > To: MPI3 Tools <mpi3-tools(a)lists.mpi-forum.org> > Subject: Next TelCon 3/19 - topic debugging interfaces > Reply-To: Martin Schulz <schulzm(a)llnl.gov> > > Hi all, > > Our next TelCon for the MPI tools WG is scheduled for 3/19 at 8am PST. The login information is on the Wiki at: > > https://svn.mpi-forum.org/trac/mpi-forum-web/wiki/MPI3Tools > > Jeff will go over the various debugging interface proposals, which we are planning to turn into officially MPI forum blessed side documents (as we did for MPIR). In particular, he plans to address the handle API proposal, which is described at: > > https://svn.mpi-forum.org/trac/mpi-forum-web/wiki/MPI3Tools/handleapi > > It would be great if everybody could familiarize themselves with this proposal (hence the early reminder). > > Thanks! > > Martin > > > > ________________________________________________________________________ > Martin Schulz, schulzm(a)llnl.gov, http://people.llnl.gov/schulzm > CASC @ Lawrence Livermore National Laboratory, Livermore, USA > > > ________________________________________________________________________ Martin Schulz, schulzm(a)llnl.gov, http://people.llnl.gov/schulzm CASC @ Lawrence Livermore National Laboratory, Livermore, USA

1 0

[Mpi3-tools] Next TelCon 3/19 - topic debugging interfaces
by Schulz, Martin 08 Mar '12

08 Mar '12

Hi all, Our next TelCon for the MPI tools WG is scheduled for 3/19 at 8am PST. The login information is on the Wiki at: https://svn.mpi-forum.org/trac/mpi-forum-web/wiki/MPI3Tools Jeff will go over the various debugging interface proposals, which we are planning to turn into officially MPI forum blessed side documents (as we did for MPIR). In particular, he plans to address the handle API proposal, which is described at: https://svn.mpi-forum.org/trac/mpi-forum-web/wiki/MPI3Tools/handleapi It would be great if everybody could familiarize themselves with this proposal (hence the early reminder). Thanks! Martin ________________________________________________________________________ Martin Schulz, schulzm(a)llnl.gov, http://people.llnl.gov/schulzm CASC @ Lawrence Livermore National Laboratory, Livermore, USA

1 0

Re: [Mpi3-tools] Pending proposals: DLL, msgq, MPI handles
by Jeffrey Squyres 05 Mar '12

05 Mar '12

On Mar 4, 2012, at 11:51 PM, Martin Schulz wrote: >> Sidenote: The ultimate goal would actually be to put all 3 of the above proposals into a single document (akin to the MPIR document) that is hosted on www.mpi-forum.org. The DLL proposal can be applied to both the message Q and MPI handle proposals, for example. > > Do you really want a single document or three separate documents? Or should this even be combined with the MPIR document into a single document for debugging interfaces? Do you remember the reading/voting procedure for these side documents? We were thinking 2 total documents: 1. MPIR (as it stands now). 2. A new document that contains the DLL stuff, message queue spec, and MPI handle debug spec. The reason is that the message queue + MPI handle debug stuff are both standalone plugins that will be loaded by the debugger. This is a bit different than MPIR. And the DLL-location stuff fits in well with these two. Hence, the organization of the document can be something like: a. intro b. DLL locating c. message queue d. MPI handle debugging And the DLL-locating stuff can apply to the later two chapters. -- Jeff Squyres jsquyres(a)cisco.com For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/

1 0

Re: [Mpi3-tools] Pending proposals: DLL, msgq, MPI handles
by Martin Schulz 05 Mar '12

05 Mar '12

Hi Jeff, all, Thanks for the notes - a few comments inline: On Feb 29, 2012, at 12:59 PM, Jeffrey Squyres wrote: > John D. and I chatted about the pending proposals at the call on Tuesday. > > Short version > ============= > > 1. We need people to look at the MPI handles proposal to determine whether David L. and I should spend time advancing it (URL's below). > > 2. Ultimate goal: a combined document (akin to the MPIR doc) for the DLL proposal, documentation of the message Q debugging, and the MPI handles debugging stuff. > > More details > ============ > > 1. Message Q: unlike MPIR, the message Q stuff is a pretty stable interface, so it hasn't received much effort. But it really should be formally documented and put in a document on www.mpi-forum.org. It probably would not be difficult -- perhaps the white paper that was written a few years ago would be a good starting point. Or the .h file. Or both. Whatever. > > Neither John nor I have the cycles to do the initial writing, but can be available for proofing, etc. > > *** Martin: can you write this up? To be honest, I don't have enough cycles for this at the moment. Also, it would be better if someone could write this up who has actually used that API before - I know the highlights about the API, but I never used it. Are there any volunteers that are familiar enough with the API and could help write it up? > > 2. We talked about the DLL proposal for quite a while. John made some good points: > > - Whether dlopen() succeeds or not is not necessarily a sufficient test to know whether a DLL is a good match for both the host (i.e., the tool) and the target (i.e., the MPI process). The dlopen test will probably always work on Linux, but it may not work "everywhere." Regardless, even if dlopen works, it doesn't tell you anything about the *target* that the plugin is intended for. This was a bad assumption on my part when writing the proposal -- I assumed that host==target, which may not be true. > > There are three things that must match for a DLL to be usable by the tool: > > 2a. Tool details. Things like the endian, pointer size, processor architecture, ...and possibly other stuff. These are the types of things that will make dlopen() succeed or not in the tool. > > 2b. Target details. Same as above: endian, pointer size, processor architecture, ...etc. But these don't have to do with the tool -- they have to do with the MPI process being analyzed by the tool. For example, the tool may be 32 bit and the target MPI process may be 64 bit. > > 2c. MPI configuration details. Some vendors ship multiple libmpi binaries (e.g., one that is threaded and one that is not). Others allow libmpi to be build multiple different ways (e.g., Open MPI can be built with or without heterogeneous support). These different flavors of libmpi may require a different DLL to probe for the information contained within the target MPI process. Hence, the MPI implementation may include multiple DLLs; which one is a match for the MPI target process configuration details can be determined at run-time (see below). > > John suggests associating 2a and 2b metadata with the DLL filename. So instead of having an argv-style array of filenames, have an array of structs with meta data and the filename. /usr/include/elf.h on Linux provides a lot of inspiration here. Perhaps the struct could look like this (pseudocode): > > struct dll_info { > char *filename; > bool big_endian; > uint32_t pointer_size; > enum operating_system_t os; > enum processor_architecture_t cpu; > }; > > (elf.h actually supplies values like OS and processor; this would keep MPI documents out of the business of needing to track this stuff) > > At run-time, the tool can then scan the array of structs to find one or more matches for 2a and 2b info. The tool can dlopen each of these and invoke a well-known function to let the DLL determine if it matches the MPI target process image. The function can return "yes, I match the target image MPI configuration details" or "no, I do not match." > > --> I will amend the proposal with all of the above. This all sounds good to me. > > 3. We also revived/reviewed the MPI handle debugging proposal. If you recall, the description and header file specifying the interface is here: > > https://bitbucket.org/jsquyres/mpi3-tools-handles2/src/tip/ompi/debuggers/M… > https://bitbucket.org/jsquyres/mpi3-tools-handles2/src/tip/ompi/debuggers/m… > > --> David L. and I want to press ahead with this proposal, but we need some more "yeah, that sounds good -- keep working on it" approval from the group before doing so. Can some people read these, refresh themselves, and give us your opinions? Will do. It would be good if others could do the same and then we could pick it up at the next call. > > ...and/or provide specific feedback? :-) > > ----- > > Sidenote: The ultimate goal would actually be to put all 3 of the above proposals into a single document (akin to the MPIR document) that is hosted on www.mpi-forum.org. The DLL proposal can be applied to both the message Q and MPI handle proposals, for example. Do you really want a single document or three separate documents? Or should this even be combined with the MPIR document into a single document for debugging interfaces? Do you remember the reading/voting procedure for these side documents? Martin > > -- > Jeff Squyres > jsquyres(a)cisco.com > For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/ > > > _______________________________________________ > Mpi3-tools mailing list > Mpi3-tools(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-tools ________________________________________________________________________ Martin Schulz, schulzm(a)llnl.gov, http://people.llnl.gov/schulzm CASC @ Lawrence Livermore National Laboratory, Livermore, USA

1 0