October 2010 - mpiwg-tools - mailman3.cels.anl.gov

Re: [Mpi3-tools] Next MPI-3 Tools WG TelCon on 11/1
by Martin Schulz 29 Oct '10

29 Oct '10

Correction (thanks to Marty): the US is next week still on daylight savings time, which means we have an 8 hour time difference for that one week (from CA to central Europe). Hence, we'll have the TelCon at 8am PT, 11am CT, 4pm(!) MSZ. Hopefully this will work for everyone. Martin On Oct 29, 2010, at 4:19 PM, Martin Schulz wrote: > Hi all, > > Our next tools TelCon is planned for Monday 11/1 at the usual > time at 8am PT, 11am CT, 5pm MZ (assuming I got this right > that we all change from daylight savings time to normal time > this weekend). > > The Webex link is: > https://cisco.webex.com/ciscosales/j.php?ED=150743227&UID=0&PW=NZTRhZjI2NGF… > Password: tools > The information is also on the Wiki. > > The agenda is to continue the MPIT discussions - we made > one full pass over the document (mainly driven by Jeff's > comments), but if someone has additional comments or > questions, we can revisit items. > > Otherwise, I would like to pick up the one remaining topic > that we have put aside so far: there were some concerns > about implementations allocating and deallocating large > number of variables and how the current discovery > mechanism would cope with that. How can we support > some kind of ID reuse without causing stale information > if the variables are in use? > > Further, we had some discussion on how to keep the > interface extensible. > > Any discussion on these points would be helpful, since > I believe that these are the two remaining issues before > I can put together a (final?) new draft. > > Thanks, > > Martin > > PS: The current draft (as of early October) is on the Wik. > > ________________________________________________________________________ > Martin Schulz, schulzm(a)llnl.gov, http://people.llnl.gov/schulzm > CASC @ Lawrence Livermore National Laboratory, Livermore, USA > > > ________________________________________________________________________ Martin Schulz, schulzm(a)llnl.gov, http://people.llnl.gov/schulzm CASC @ Lawrence Livermore National Laboratory, Livermore, USA

1 0

Re: [Mpi3-tools] Tools WG status report for SC 2010
by Martin Schulz 29 Oct '10

29 Oct '10

Hi Bronis, all, Thanks for the feedback - I have to admit, though, I have no idea how to represent this graphically in a useful way. If anyone has suggestions, please feel free to draft something. Also, Rich limited each WG to 2-3 slides, so adding two more full slides wouldn't work. It would have to be sketches on the current slides. The other points, that this presentation needs to be more targeted towards the SC audience and be more technical is certainly right. I have attached a modified version - I will ask Rich for a few more days, so that we can discuss this a bit more (incl. on Monday's TelCon). Thanks, Martin On Oct 27, 2010, at 4:45 AM, Bronis R. de Supinski wrote: > > Martin: > > Re: >> As some/most of you have probably seen, Rich asked for 2-3 >> slide overview/status update from each WG for his presentation >> on the MPI-3 efforts during the SC MPI BoF. Attached is a first >> draft for our activities in the tools WG. Please let me know if >> have any comments or suggestions. Rich needs this by Friday, >> so please answer as soon as possible. > > At a high level, these slides are rather dry. I have > found the BoF presentations to be somewhat boring in > the past and these slides won't change that. I think > it would be good to have two more slides that graphically > illustrate the main thrusts. The first could show how > MPIT will work (and, at least to some extent, what it > is good for). The second could illustrate the MPIR > acquisition interface and why we need to extend it > before we incorporate it into the standard. > > At a slightly lower level, what these slides are missing > is related to what you need to show graphically. You > say MPIR is not ideal but you provide nothing that > backs up that statement other than "has scalability > problems". I think you need to show it has those problems > and you need to state that it cannot support dynamic > processes (and that one can expect dynamic processes > will become important to fault tolerance so they are > likely to be used in the future even if they are not > today). Similarly, you don't give any real feel for > what MPIT is or how one would use it and why. Covering > those issues are of much more interest to a general > audience than a status report on something they > do not know enough about to evaluate. > > Bronis > > _______________________________________________ > Mpi3-tools mailing list > Mpi3-tools(a)lists.mpi-forum.org > http://BLOCKEDlists.mpi-forum.org/mailman/listinfo.cgi/mpi3-tools > ________________________________________________________________________ Martin Schulz, schulzm(a)llnl.gov, http://people.llnl.gov/schulzm CASC @ Lawrence Livermore National Laboratory, Livermore, USA

1 0

[Mpi3-tools] Next MPI-3 Tools WG TelCon on 11/1
by Martin Schulz 29 Oct '10

29 Oct '10

Hi all, Our next tools TelCon is planned for Monday 11/1 at the usual time at 8am PT, 11am CT, 5pm MZ (assuming I got this right that we all change from daylight savings time to normal time this weekend). The Webex link is: https://cisco.webex.com/ciscosales/j.php?ED=150743227&UID=0&PW=NZTRhZjI2NGF… Password: tools The information is also on the Wiki. The agenda is to continue the MPIT discussions - we made one full pass over the document (mainly driven by Jeff's comments), but if someone has additional comments or questions, we can revisit items. Otherwise, I would like to pick up the one remaining topic that we have put aside so far: there were some concerns about implementations allocating and deallocating large number of variables and how the current discovery mechanism would cope with that. How can we support some kind of ID reuse without causing stale information if the variables are in use? Further, we had some discussion on how to keep the interface extensible. Any discussion on these points would be helpful, since I believe that these are the two remaining issues before I can put together a (final?) new draft. Thanks, Martin PS: The current draft (as of early October) is on the Wik. ________________________________________________________________________ Martin Schulz, schulzm(a)llnl.gov, http://people.llnl.gov/schulzm CASC @ Lawrence Livermore National Laboratory, Livermore, USA

1 0

Re: [Mpi3-tools] Question about communicator for control variables
by Martin Schulz 29 Oct '10

29 Oct '10

Hi Andreas, On Oct 29, 2010, at 12:59 PM, Andreas Kuntze wrote: > Hello all, > > in the current MPIT draft, the MPIT_CTRLVAR_WRITE function takes a > communicator as an argument. If I understood the description correctly, you > have to pass the communicator returned by MPIT_CTRLVAR_GETINFO. Can someone > explain to me, for what reason you have to pass it, if you can execute > collective write operations on one communicator only for each control > variable? The idea was that the control variable set operation can have a global meaning, in which case it has to be set on all processors of a certain communicator. Passing this communicator into the routine made that explicit (I agree, we could have just made this implicit by using the communicator that is returned by the info call). We discussed this at length at the last face-to-face meeting and actually decided to drop it. The problem that many saw in this was that communicators don't exist before MPI_Init, but we want to be able to call this routine at that time. The compromise was to still return a flag during the info call that says that a variable has a global/collective meaning, but put the burden on the user to ensure that the variable is set consistently across all tasks. This change (along with the other comments I got in the last few meetings), will be reflected in the next MPIT draft. Martin > > Thanks, > > Andreas > _______________________________________________ > Mpi3-tools mailing list > Mpi3-tools(a)lists.mpi-forum.org > http://BLOCKEDlists.mpi-forum.org/mailman/listinfo.cgi/mpi3-tools > ________________________________________________________________________ Martin Schulz, schulzm(a)llnl.gov, http://people.llnl.gov/schulzm CASC @ Lawrence Livermore National Laboratory, Livermore, USA

1 0

[Mpi3-tools] Question about communicator for control variables
by Andreas Kuntze 29 Oct '10

29 Oct '10

Hello all, in the current MPIT draft, the MPIT_CTRLVAR_WRITE function takes a communicator as an argument. If I understood the description correctly, you have to pass the communicator returned by MPIT_CTRLVAR_GETINFO. Can someone explain to me, for what reason you have to pass it, if you can execute collective write operations on one communicator only for each control variable? Thanks, Andreas

1 0

Re: [Mpi3-tools] Tools WG status report for SC 2010
by Bronis R. de Supinski 27 Oct '10

27 Oct '10

Martin: Re: > As some/most of you have probably seen, Rich asked for 2-3 > slide overview/status update from each WG for his presentation > on the MPI-3 efforts during the SC MPI BoF. Attached is a first > draft for our activities in the tools WG. Please let me know if > have any comments or suggestions. Rich needs this by Friday, > so please answer as soon as possible. At a high level, these slides are rather dry. I have found the BoF presentations to be somewhat boring in the past and these slides won't change that. I think it would be good to have two more slides that graphically illustrate the main thrusts. The first could show how MPIT will work (and, at least to some extent, what it is good for). The second could illustrate the MPIR acquisition interface and why we need to extend it before we incorporate it into the standard. At a slightly lower level, what these slides are missing is related to what you need to show graphically. You say MPIR is not ideal but you provide nothing that backs up that statement other than "has scalability problems". I think you need to show it has those problems and you need to state that it cannot support dynamic processes (and that one can expect dynamic processes will become important to fault tolerance so they are likely to be used in the future even if they are not today). Similarly, you don't give any real feel for what MPIT is or how one would use it and why. Covering those issues are of much more interest to a general audience than a status report on something they do not know enough about to evaluate. Bronis

1 0

Re: [Mpi3-tools] MPIT profiling
by Martin Schulz 27 Oct '10

27 Oct '10

Hi Jeff, [cut some text] > >>> I claim that this is inconsistent with the MPIT's goal of allowing multiple tools to be active in a single executable -- what if multiple tools want to use the PMPIT interface? You can't do that without using an external thingy (like PNMPI -- which would need to be expanded to support the MPIT interface). >> >> I don't agree with this - the MPIT interface could be used inside >> the MPI application, e.g., to read and document configurations >> or to set specific runtime parameters. Further, individual application >> libraries/components could include their own measurement calls >> (for self monitoring and tuning). This means that there could be >> multiple "users" of the MPIT interface even without writing a single >> PMPI tool. > > Did you mean PMPIT? Yes, I was seeing them as one. > >> By allowing PMPIT, we would enable tools that add to the offering >> of MPIT for all the above users. > > But you only allow *one* tool per process to do this. That's the inconstancy: there could be multiple consumers of the MPIT interface, but you're only allowing *one* tool to use the "profiling" side of the MPIT interface (or whatever you want to call it). > > In short: if we allow multiple consumers of the MPIT interface, why not also allow multiple users of the "profiling" side of the MPIT interface? Personally, I would love to do that - basically standardizing PnMPIT, but this would make the interface much more complex and does add some overhead in any case - I don't think there is a way around that. I doubt we would get a majority for this (perhaps I am wrong about that?). Also, it would either no longer be the same approach that we have done with MPI/PMPI or we would have to change that as well - that probably will find even less acceptance. Don't we have the same issue with MPI as well, though: multiple components, libraries, communicators using the MPI interface, but only one tool can use the PMPI interface. > >>> Meaning: since supporting multiple tools in a single executable is an explicit goal, we should either allow MPIT profiling by multiple tools within a single process or we shouldn't allow it at all. >> >> I don't think this is necessary and covered by the current proposal. >>> From a tool developers perspective, we can do everything we want/ >> need to do with the current proposal. > > Aren't we going to be in exactly the same sucky position we're in right now with PMPI -- that you can only have *one* tool that provides this functionality per process? I wouldn't call it sucky - there is a solution around it that solves the problem without having to change the standard and without causing overhead in the default case (and I got a few papers out of it ;) ). Sure, having it in the MPI standard would be a bit nicer/cleaner, but at what cost? I don't think this would be an easy interface to define. Martin > >>> 1. Martin: Tools adding their own control / performance variables. My suggestion here is that maybe we can have a registration functionality for this (i.e., an MPIT function to register new control/performance variables). I have not thought this through deeply; my only point is that if we know tool writers want to do something (and Martin says that the tool writers do want to do it), then we should provide them a direct way to do it. Not an indirect way. >> >> We discussed this during the tools workshop and then >> decided to drop it. The requirements for such a registration >> seemed to get out of hand quickly (who tracks the variables, >> who handles starts and stops, who manages the sessions - >> and all this for variables that we don't know yet), while the >> PMPIT interface had all the necessary features for tool >> developers. >> >> Additionally, even with a registration function there was a very >> clear opinion towards includinh the PMPIT interface. The question >> was basically why not do it, since the PMPI interface has proven to >> be so widely successful and used for so much more than >> originally anticipated. How can we ensure that we cover >> everything with a registration function? This may not be the only >> use case in the future (e.g., we would need interception for FT >> layers that restore state on restart, and I am sure we can find >> more use cases). > > ...doesn't these future uses also anticipate the need to have more than one user of the profiling side of the MPIT interface? > > -- > Jeff Squyres > jsquyres(a)cisco.com > For corporate legal information go to: > http://BLOCKEDwww.BLOCKEDcisco.com/web/about/doing_business/legal/cri/ > > > _______________________________________________ > Mpi3-tools mailing list > Mpi3-tools(a)lists.mpi-forum.org > http://BLOCKEDlists.mpi-forum.org/mailman/listinfo.cgi/mpi3-tools > ________________________________________________________________________ Martin Schulz, schulzm(a)llnl.gov, http://people.llnl.gov/schulzm CASC @ Lawrence Livermore National Laboratory, Livermore, USA

1 0

[Mpi3-tools] Tools WG status report for SC 2010
by Martin Schulz 27 Oct '10

27 Oct '10

Hi all, As some/most of you have probably seen, Rich asked for 2-3 slide overview/status update from each WG for his presentation on the MPI-3 efforts during the SC MPI BoF. Attached is a first draft for our activities in the tools WG. Please let me know if have any comments or suggestions. Rich needs this by Friday, so please answer as soon as possible. Thanks! Martin ________________________________________________________________________ Martin Schulz, schulzm(a)llnl.gov, http://people.llnl.gov/schulzm CASC @ Lawrence Livermore National Laboratory, Livermore, USA

1 0

Re: [Mpi3-tools] MPIT profiling
by Jeff Squyres 22 Oct '10

22 Oct '10

On Oct 22, 2010, at 11:58 AM, Martin Schulz wrote: >> My take on section 1.2.8 of the MPIT document is that it's weird/inconsistent that we're saying that you can use a PMPIT interface to profile the MPIT interface, and that it basically behaves exactly like PMPI. > > I don't think profiling is the right word here - we want to intercept > the MPIT interface and alter its functionality. The fact that the > PMPI interface is known as the profiling interface is actually > kind of unfortunate (I realize this describes the initial purpose); > it is used for so much more by today's tools. Ok. >> I claim that this is inconsistent with the MPIT's goal of allowing multiple tools to be active in a single executable -- what if multiple tools want to use the PMPIT interface? You can't do that without using an external thingy (like PNMPI -- which would need to be expanded to support the MPIT interface). > > I don't agree with this - the MPIT interface could be used inside > the MPI application, e.g., to read and document configurations > or to set specific runtime parameters. Further, individual application > libraries/components could include their own measurement calls > (for self monitoring and tuning). This means that there could be > multiple "users" of the MPIT interface even without writing a single > PMPI tool. Did you mean PMPIT? > By allowing PMPIT, we would enable tools that add to the offering > of MPIT for all the above users. But you only allow *one* tool per process to do this. That's the inconstancy: there could be multiple consumers of the MPIT interface, but you're only allowing *one* tool to use the "profiling" side of the MPIT interface (or whatever you want to call it). In short: if we allow multiple consumers of the MPIT interface, why not also allow multiple users of the "profiling" side of the MPIT interface? >> Meaning: since supporting multiple tools in a single executable is an explicit goal, we should either allow MPIT profiling by multiple tools within a single process or we shouldn't allow it at all. > > I don't think this is necessary and covered by the current proposal. >> From a tool developers perspective, we can do everything we want/ > need to do with the current proposal. Aren't we going to be in exactly the same sucky position we're in right now with PMPI -- that you can only have *one* tool that provides this functionality per process? >> 1. Martin: Tools adding their own control / performance variables. My suggestion here is that maybe we can have a registration functionality for this (i.e., an MPIT function to register new control/performance variables). I have not thought this through deeply; my only point is that if we know tool writers want to do something (and Martin says that the tool writers do want to do it), then we should provide them a direct way to do it. Not an indirect way. > > We discussed this during the tools workshop and then > decided to drop it. The requirements for such a registration > seemed to get out of hand quickly (who tracks the variables, > who handles starts and stops, who manages the sessions - > and all this for variables that we don't know yet), while the > PMPIT interface had all the necessary features for tool > developers. > > Additionally, even with a registration function there was a very > clear opinion towards includinh the PMPIT interface. The question > was basically why not do it, since the PMPI interface has proven to > be so widely successful and used for so much more than > originally anticipated. How can we ensure that we cover > everything with a registration function? This may not be the only > use case in the future (e.g., we would need interception for FT > layers that restore state on restart, and I am sure we can find > more use cases). ...doesn't these future uses also anticipate the need to have more than one user of the profiling side of the MPIT interface? -- Jeff Squyres jsquyres(a)cisco.com For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/

1 0

Re: [Mpi3-tools] MPIT profiling
by Martin Schulz 22 Oct '10

22 Oct '10

Hi Jeff, all, On Oct 18, 2010, at 9:13 AM, Jeff Squyres wrote: > We ran out of time before finishing the discussion today... > > My take on section 1.2.8 of the MPIT document is that it's weird/inconsistent that we're saying that you can use a PMPIT interface to profile the MPIT interface, and that it basically behaves exactly like PMPI. I don't think profiling is the right word here - we want to intercept the MPIT interface and alter its functionality. The fact that the PMPI interface is known as the profiling interface is actually kind of unfortunate (I realize this describes the initial purpose); it is used for so much more by today's tools. > > I claim that this is inconsistent with the MPIT's goal of allowing multiple tools to be active in a single executable -- what if multiple tools want to use the PMPIT interface? You can't do that without using an external thingy (like PNMPI -- which would need to be expanded to support the MPIT interface). I don't agree with this - the MPIT interface could be used inside the MPI application, e.g., to read and document configurations or to set specific runtime parameters. Further, individual application libraries/components could include their own measurement calls (for self monitoring and tuning). This means that there could be multiple "users" of the MPIT interface even without writing a single PMPI tool. By allowing PMPIT, we would enable tools that add to the offering of MPIT for all the above users. Additionally, I think adding the PMPIT interface option keeps the MPIT interface in line with the MPI interface, which I think is important to gain acceptance. This was also the feedback from the tools community that feels *very* strongly about maintaining the profiling interface for all of the MPI standard, incl. MPIT. > > Meaning: since supporting multiple tools in a single executable is an explicit goal, we should either allow MPIT profiling by multiple tools within a single process or we shouldn't allow it at all. I don't think this is necessary and covered by the current proposal. >From a tool developers perspective, we can do everything we want/ need to do with the current proposal. > > Background: > > Two use cases for profiling MPIT cited on the call were: > > 1. Martin: Tools adding their own control / performance variables. My suggestion here is that maybe we can have a registration functionality for this (i.e., an MPIT function to register new control/performance variables). I have not thought this through deeply; my only point is that if we know tool writers want to do something (and Martin says that the tool writers do want to do it), then we should provide them a direct way to do it. Not an indirect way. We discussed this during the tools workshop and then decided to drop it. The requirements for such a registration seemed to get out of hand quickly (who tracks the variables, who handles starts and stops, who manages the sessions - and all this for variables that we don't know yet), while the PMPIT interface had all the necessary features for tool developers. Additionally, even with a registration function there was a very clear opinion towards includinh the PMPIT interface. The question was basically why not do it, since the PMPI interface has proven to be so widely successful and used for so much more than originally anticipated. How can we ensure that we cover everything with a registration function? This may not be the only use case in the future (e.g., we would need interception for FT layers that restore state on restart, and I am sure we can find more use cases). Martin > > 2. Marty: It is worthwhile to be able to profile the impact of MPIT itself -- e.g., see if MPIT itself is performing poorly. I extrapolate this to mean that it can be useful to help quantify the at least part of the MPIT's own overhead (as I typed that sentence, I'm not sure how much of MPIT's overhead will be measurable by simply timing how long its individual function calls...? I guess there's other metrics that can be measured, too...?). > > -- > Jeff Squyres > jsquyres(a)cisco.com > For corporate legal information go to: > http://BLOCKEDwww.BLOCKEDcisco.com/web/about/doing_business/legal/cri/ > > > _______________________________________________ > Mpi3-tools mailing list > Mpi3-tools(a)lists.mpi-forum.org > http://BLOCKEDlists.mpi-forum.org/mailman/listinfo.cgi/mpi3-tools > ________________________________________________________________________ Martin Schulz, schulzm(a)llnl.gov, http://people.llnl.gov/schulzm CASC @ Lawrence Livermore National Laboratory, Livermore, USA

1 0