May 2009 - mpiwg-tools - mailman3.cels.anl.gov

Re: [Mpi3-tools] Meaning of meaning of unique_id inmqs_communicator?
by John DelSignore 28 May '09

28 May '09

Sure, if I ever find the time to actually work on it. Cheers, John D. Jeff Squyres wrote: > John -- > > Can this be part of the text that you write up for standardizing the > message queues stuff? > > > On May 11, 2009, at 3:21 PM, Geoffrey R. Paulsen wrote: > >> >> >> Regarding Ashley Pittman's message: >> > >> > > I'm not sure I understand the difference between (b) and (c). >> > >> > The difference between (b) and (c) is best explained with reference to >> > MPI_COMM_SELF, in (b) it can have the same id in every process, in (c) >> > it has to have different values everywhere. This has a practical >> > implication, in (c) you can say "Communicator X in this job is...", >> with >> > (b) to be able to describe the communicator you have to also include >> > some information about the communicator members, for example the global >> > rank of local rank zero. >> > >> > In my opinion (c) would make the most logical sense and be easiest for >> > tools to work with however I know from my time working with MPI >> > libraries that (b) is a much more sensible option in reality. >> > >> > Ashley Pittman. >> >> Thank you for that description. I've been able to verify that HP-MPI >> indeed only supports the (b) and not (c). >> >> I agree that from a tools standpoint, (c) would be much more useful, >> but that from an MPI point of view, it might require some semimajor >> changes to the way that we create communicators. >> >> Currently MPI_Comm_Create() does not block for an entire job, just >> for the ranks involved, which would make it tricky to ensure uniqueness >> across an entire job of the communicator id. >> >> I don't think this would be impossible, but should be well thought >> out. >> >> Geoff Paulsen >> >> _______________________________________________ >> Mpi3-tools mailing list >> Mpi3-tools(a)lists.mpi-forum.org >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-tools >> > > > --Jeff Squyres > Cisco Systems > > _______________________________________________ > Mpi3-tools mailing list > Mpi3-tools(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-tools > >

1 0

Re: [Mpi3-tools] Meaning of meaning of unique_id inmqs_communicator?
by Jeff Squyres 27 May '09

27 May '09

John -- Can this be part of the text that you write up for standardizing the message queues stuff? On May 11, 2009, at 3:21 PM, Geoffrey R. Paulsen wrote: > > > Regarding Ashley Pittman's message: > > > > > I'm not sure I understand the difference between (b) and (c). > > > > The difference between (b) and (c) is best explained with > reference to > > MPI_COMM_SELF, in (b) it can have the same id in every process, in > (c) > > it has to have different values everywhere. This has a practical > > implication, in (c) you can say "Communicator X in this job > is...", with > > (b) to be able to describe the communicator you have to also include > > some information about the communicator members, for example the > global > > rank of local rank zero. > > > > In my opinion (c) would make the most logical sense and be easiest > for > > tools to work with however I know from my time working with MPI > > libraries that (b) is a much more sensible option in reality. > > > > Ashley Pittman. > > Thank you for that description. I've been able to verify that HP-MPI > indeed only supports the (b) and not (c). > > I agree that from a tools standpoint, (c) would be much more useful, > but that from an MPI point of view, it might require some semimajor > changes to the way that we create communicators. > > Currently MPI_Comm_Create() does not block for an entire job, just > for the ranks involved, which would make it tricky to ensure > uniqueness > across an entire job of the communicator id. > > I don't think this would be impossible, but should be well thought > out. > > Geoff Paulsen > > _______________________________________________ > Mpi3-tools mailing list > Mpi3-tools(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-tools > -- Jeff Squyres Cisco Systems

1 0

Re: [Mpi3-tools] Meeting Notes from 5/11, next meetings
by Marty Itzkowitz 27 May '09

27 May '09

On 5/27/09 12:42 AM, Supalov, Alexander wrote: > Hi everybody, > > I added my comments to the proposal for lack of feedback per email. Please consider at F2F. Slight correction to item 5: > > Reads "Likewise, the MPI-stall can probably be fairly well statistically approximated by mapping PC to the progress engine routines and its dependants." > > Should read "Likewise, the MPI-stall can probably be fairly well statistically approximated by mapping PC to the progress engine routines and its dependants, except memcpy and equivalents, and wire access routines". > Yes, that's true, but it implies that the tools would know a lot of detail about the MPI implementation. We wanted to avoid that, since every MPI implementation (and the Sun profiling tools support seven different versions) is different. In the Sun MPI profiler we also (by default, although it can be changed) suppress all call frames below the first user call into MPI, to avoid overwhelming the user with details that he or she can do nothing about. Doing so removed more than half the functions from the function list data for a run. Marty

1 0

Re: [Mpi3-tools] Meeting Notes from 5/11, next meetings
by Supalov, Alexander 27 May '09

27 May '09

Hi everybody, I added my comments to the proposal for lack of feedback per email. Please consider at F2F. Slight correction to item 5: Reads "Likewise, the MPI-stall can probably be fairly well statistically approximated by mapping PC to the progress engine routines and its dependants." Should read "Likewise, the MPI-stall can probably be fairly well statistically approximated by mapping PC to the progress engine routines and its dependants, except memcpy and equivalents, and wire access routines". Best regards. Alexander -----Original Message----- From: Supalov, Alexander Sent: Tuesday, May 12, 2009 3:42 PM To: 'mpi3-tools(a)lists.mpi-forum.org' Subject: RE: [Mpi3-tools] Meeting Notes from 5/11, next meetings Hi, I missed this meeting due to a conflict with another one, sorry. How can one comment on the proposals offline? Briefly on the MPI state proposal: 1. The implementation impact and the associated runtime overheads seem to be rather high. 2. The syntax of the proposed interface is OpenMPI specific. 3. The proposed interface discovery method may not work on all platforms/linkage models. 4. The comparison to the API tracing is skewed in favor of sampling. If one can sample one representative process, one can just as well trace one representative process at comparable or lower overheads. Message matching is not required in this case anyway. 5. Finally, and most importantly, one may be able to get comparable information thru sampling without changing the MPI implementation. For example, the inside-MPI/outside-MPI data can be deduced from the program counter mapping to the program address space occupied by the MPI library and its dependants. Likewise, the MPI-stall can probably be fairly well statistically approximated by mapping PC to the progress engine routines and its dependants. They will be implementation specific, but the set of pertinent address ranges should be possible to specify in any given case. Any existing sampler should help here. Best regards. Alexander -----Original Message----- From: mpi3-tools-bounces(a)lists.mpi-forum.org [mailto:[email protected]] On Behalf Of Martin Schulz Sent: Monday, May 11, 2009 7:00 PM To: mpi3-tools(a)lists.mpi-forum.org Subject: [Mpi3-tools] Meeting Notes from 5/11, next meetings Hi all, The meetings notes from today are posted on the Wiki - please let me know if I missed anything, in particular from the first 15min. Here are the dates for the next TelCons: - TelCon: Monday, May 18th, 2009 - Canceled - TelCon: Monday, June 1st, 2009 - Pavan is leading a discussion on PMI, notes will be posted on Wiki - Face to Face: Tuesday, June 9, 2009 (as part of the MPI Forum Meeting in Menlo Park, CA) - Continue discussion on MPI State Interface, focus on implementation impact - Continue discussion on inclusion of PMI - First proposal on Handle Information Interface - TelCon: Monday, June 15th, 2009 - Agenda: tbd (we will decide on this on 6/9) Also, I saw that Rich already put up an agenda, which lists the tools WG for 6/9 from 1-3pm PT. I am not sure if we can change this at this point, but I will try. Martin _______________________________________________________________________ Martin Schulz, schulzm(a)llnl.gov, http://people.llnl.gov/schulz6 CASC @ Lawrence Livermore National Laboratory, Livermore, USA _______________________________________________ Mpi3-tools mailing list Mpi3-tools(a)lists.mpi-forum.org http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-tools --------------------------------------------------------------------- Intel GmbH Dornacher Strasse 1 85622 Feldkirchen/Muenchen Germany Sitz der Gesellschaft: Feldkirchen bei Muenchen Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer Registergericht: Muenchen HRB 47456 Ust.-IdNr. VAT Registration No.: DE129385895 Citibank Frankfurt (BLZ 502 109 00) 600119052 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.

1 0

[Mpi3-tools] MPI handles interface
by Jeff Squyres 18 May '09

18 May '09

Per feedback from the last time we discussed the handles interface, I changed the communicator query interface to return an opaque handle that can subsequently be queried for all the communicator's specific information. This design allows (but does not require) the DLL to probe/obtain all the communicator information from the MPI process once and cache it locally (presumably on the opaque handle) such that if the debugger queries more than just one piece of information, queries can be satisfied locally without needing to probe the MPI process again. I also removed the f2c and cxx2c query functions, replacing them with a bitmask on the main _query() function indicating what the type is of the communicator being queried. Note that I have *only* updated communicator stuff; I have not yet updated the other types, pending feedback from this group. Any further feedback before I actually go implement this interface? Please reply by the end of this week (COB, Friday, 22 May, 2009). ===== Here's the new file (interesting stuff starts around line 675): http://www.open-mpi.org/hg/hgwebdir.cgi/jsquyres/mpi3-tools-handles/file/ti… Here's the commit showing the diff of the changes I made: http://www.open-mpi.org/hg/hgwebdir.cgi/jsquyres/mpi3-tools-handles/rev/f6f… -- Jeff Squyres Cisco Systems

1 0

Re: [Mpi3-tools] Meeting Notes from 5/11, next meetings
by Supalov, Alexander 12 May '09

12 May '09

Hi, I missed this meeting due to a conflict with another one, sorry. How can one comment on the proposals offline? Briefly on the MPI state proposal: 1. The implementation impact and the associated runtime overheads seem to be rather high. 2. The syntax of the proposed interface is OpenMPI specific. 3. The proposed interface discovery method may not work on all platforms/linkage models. 4. The comparison to the API tracing is skewed in favor of sampling. If one can sample one representative process, one can just as well trace one representative process at comparable or lower overheads. Message matching is not required in this case anyway. 5. Finally, and most importantly, one may be able to get comparable information thru sampling without changing the MPI implementation. For example, the inside-MPI/outside-MPI data can be deduced from the program counter mapping to the program address space occupied by the MPI library and its dependants. Likewise, the MPI-stall can probably be fairly well statistically approximated by mapping PC to the progress engine routines and its dependants. They will be implementation specific, but the set of pertinent address ranges should be possible to specify in any given case. Any existing sampler should help here. Best regards. Alexander -----Original Message----- From: mpi3-tools-bounces(a)lists.mpi-forum.org [mailto:[email protected]] On Behalf Of Martin Schulz Sent: Monday, May 11, 2009 7:00 PM To: mpi3-tools(a)lists.mpi-forum.org Subject: [Mpi3-tools] Meeting Notes from 5/11, next meetings Hi all, The meetings notes from today are posted on the Wiki - please let me know if I missed anything, in particular from the first 15min. Here are the dates for the next TelCons: - TelCon: Monday, May 18th, 2009 - Canceled - TelCon: Monday, June 1st, 2009 - Pavan is leading a discussion on PMI, notes will be posted on Wiki - Face to Face: Tuesday, June 9, 2009 (as part of the MPI Forum Meeting in Menlo Park, CA) - Continue discussion on MPI State Interface, focus on implementation impact - Continue discussion on inclusion of PMI - First proposal on Handle Information Interface - TelCon: Monday, June 15th, 2009 - Agenda: tbd (we will decide on this on 6/9) Also, I saw that Rich already put up an agenda, which lists the tools WG for 6/9 from 1-3pm PT. I am not sure if we can change this at this point, but I will try. Martin _______________________________________________________________________ Martin Schulz, schulzm(a)llnl.gov, http://people.llnl.gov/schulz6 CASC @ Lawrence Livermore National Laboratory, Livermore, USA _______________________________________________ Mpi3-tools mailing list Mpi3-tools(a)lists.mpi-forum.org http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-tools --------------------------------------------------------------------- Intel GmbH Dornacher Strasse 1 85622 Feldkirchen/Muenchen Germany Sitz der Gesellschaft: Feldkirchen bei Muenchen Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer Registergericht: Muenchen HRB 47456 Ust.-IdNr. VAT Registration No.: DE129385895 Citibank Frankfurt (BLZ 502 109 00) 600119052 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.

1 0

Re: [Mpi3-tools] Meaning of meaning of unique_id in mqs_communicator?
by Geoffrey R. Paulsen 11 May '09

11 May '09

Regarding Ashley Pittman's message: > > > I'm not sure I understand the difference between (b) and (c). > > The difference between (b) and (c) is best explained with reference to > MPI_COMM_SELF, in (b) it can have the same id in every process, in (c) > it has to have different values everywhere. This has a practical > implication, in (c) you can say "Communicator X in this job is...", with > (b) to be able to describe the communicator you have to also include > some information about the communicator members, for example the global > rank of local rank zero. > > In my opinion (c) would make the most logical sense and be easiest for > tools to work with however I know from my time working with MPI > libraries that (b) is a much more sensible option in reality. > > Ashley Pittman. Thank you for that description. I've been able to verify that HP-MPI indeed only supports the (b) and not (c). I agree that from a tools standpoint, (c) would be much more useful, but that from an MPI point of view, it might require some semimajor changes to the way that we create communicators. Currently MPI_Comm_Create() does not block for an entire job, just for the ranks involved, which would make it tricky to ensure uniqueness across an entire job of the communicator id. I don't think this would be impossible, but should be well thought out. Geoff Paulsen

1 0

Re: [Mpi3-tools] Meeting Notes from 5/11, next meetings
by Pavan Balaji 11 May '09

11 May '09

> - Pavan is leading a discussion on PMI, notes will be posted on Wiki I've also uploaded the initial PMI description to the wiki. Thanks, -- Pavan -- Pavan Balaji http://www.mcs.anl.gov/~balaji

1 0

[Mpi3-tools] Meeting Notes from 5/11, next meetings
by Martin Schulz 11 May '09

11 May '09

Hi all, The meetings notes from today are posted on the Wiki - please let me know if I missed anything, in particular from the first 15min. Here are the dates for the next TelCons: - TelCon: Monday, May 18th, 2009 - Canceled - TelCon: Monday, June 1st, 2009 - Pavan is leading a discussion on PMI, notes will be posted on Wiki - Face to Face: Tuesday, June 9, 2009 (as part of the MPI Forum Meeting in Menlo Park, CA) - Continue discussion on MPI State Interface, focus on implementation impact - Continue discussion on inclusion of PMI - First proposal on Handle Information Interface - TelCon: Monday, June 15th, 2009 - Agenda: tbd (we will decide on this on 6/9) Also, I saw that Rich already put up an agenda, which lists the tools WG for 6/9 from 1-3pm PT. I am not sure if we can change this at this point, but I will try. Martin _______________________________________________________________________ Martin Schulz, schulzm(a)llnl.gov, http://people.llnl.gov/schulz6 CASC @ Lawrence Livermore National Laboratory, Livermore, USA

1 0

Re: [Mpi3-tools] Meaning of meaning of unique_id in mqs_communicator?
by Ashley Pittman 11 May '09

11 May '09

On Mon, 2009-05-11 at 09:54 -0500, Geoffrey R. Paulsen wrote: > Ashley, > > In reading HP-MPI source code, which has implemented the MSQ interface > quite a while back, I believe that we return a value which conforms to > the definition (b) below. Thank you, that's good to know. > I'm not sure I understand the difference between (b) and (c). The difference between (b) and (c) is best explained with reference to MPI_COMM_SELF, in (b) it can have the same id in every process, in (c) it has to have different values everywhere. This has a practical implication, in (c) you can say "Communicator X in this job is...", with (b) to be able to describe the communicator you have to also include some information about the communicator members, for example the global rank of local rank zero. In my opinion (c) would make the most logical sense and be easiest for tools to work with however I know from my time working with MPI libraries that (b) is a much more sensible option in reality. Ashley Pittman.

1 0