Re: [Mpi3-tools] lightweight event log
Hi Marc-Andre, On Jul 2, 2009, at 1:58 AM, Marc-Andre Hermanns wrote:
Hi Bill,
The idea is that the implementation may be instrumented with the ability to record timestamped events. Like the "variable" interface, the idea here is to provide a way to expose whatever the implementation is prepared to make available, such as information that the developers and tuners of that implementation have found valuable. So in answer to your questoins:
1) what will be logged? Probably timestamps, some indication of location, and some context (like message length). I would not expect all routine parameters to be logged, though an implementation could choose to do thiat
Sometimes tools need all parameter information, though. Especially for higher level analysis, one might need more than the above information.
The idea is to log low-level events below the MPI call interface. Hence, we can't assume a fixed list of parameters anymore. Each implementation can have its own set of information here. What Bill is suggesting is basically only a way for a tool to gather some context around the event and we will try to suggest some commonly existing parameters, but as with the performance variables, we don't want to demand any particular structure. Note, though, that you (as a tool writer) can always combine this logging interface with a PMPI logger and can grab any cleanly defined logging information from there as well.
2) it is lightweight for two reasons: (1) the data logged may be as small and simple as the implementation chooses and (2) the data may be logged in an internal, time and space efficient format. A separate set of routines are provided to interpret the logged data. For example, the timestamps may be stored in an internal format (such as the bits from a clock register) and converted to seconds only when the user evaluates the logged data.
Combining your answers to point 1 and 2, I envision to pre-construct memory-layouts (like or even with MPI datatypes) to the relevant information, so that a tool can basically first tell the implementation what it want to be logged, then define handles for the data layout, and
I am not sure if it's worth having the user pick what should be recorded and what not, since this will be very limited anyway.
finally, retrieve the desired information through the defined handles at the location where they are needed.
I admit I didn't evaluate the implementation overhead for something like that yet, and I am aware that flexibility usually comes with a runtime cost.
3) A performance tool can examine the log files but (as part of the light-weight part) there is no option to invoke a method or routine at every point a log record could be created.
"Users" here means users with some degree of sophistication. Basically, if you can use PAPI and really understand what the various counters mean, then you should be able to use this interface.
I am uncertain whether this logging facility would/should be complementing or competing with existing tracing tools.
Some MPI implementations already have tracing facilities. To my knowledge mpich2 has MPE and OpenMPI has vampirtrace. Is this new logging facility to be provided by these or is it envisioned to be a new infrastructure. Would this not create competition between measurement infrastructures within MPI implementations?
No, it is certainly not intended to be an alternative to existing MPI tracers. This logging facility would run at a much lower level and only record selected MPI internal events, mostly for debugging or very low-level performance analysis in conjunction with a high-level tracer such as VT.
Martin mentioned a ring buffer to be queried? What happens when the buffer is full? If some logging information is lost, what can the user derive from an incomplete log?
The user will be able to size the buffer and query whether there was an overflow, but if there is too much data, it will be lost. The user is responsible for querying this on time. Anything else has the danger of significantly increasing the critical path. Martin
Best regards, Marc-Andre
On Jun 29, 2009, at 7:42 AM, Marc-Andre Hermanns wrote:
Hi Martin,
I am interested in how this logging facility is meant. Do you have a more detailed description of it somewhere?
- What do you anticipate to be logged (despite of timestamps)? - What will make it lightweight? - Is there any way that a performance tool might hook into?
"[...] Users can then read and evaluate this event log at a later time. [...]"
Who are these users? Application developers or tool developers?
Best regards, Marc-Andre
Martin Schulz wrote:
Hi all,
As discussed during the last meeting, our next MPI 3 tools WG TelCon is on 6/29 at the usual time (8am PDT, 11am EDT, 5pm MSZ). The dial-in numbers are as usual:
* Toll Free Dial In Number for Offsite Participants: 866-914-3976 * International Access/Caller Paid Dial In Number: 925-424-8105 * Dial In Number for LLNL Onsite Participants: 4-8105 * Access Security Code: 860609 #
So far, the agenda is to talk about a proposal for a general performance information interface. More information on this proposal is at:
https://svn.mpi-forum.org/trac/mpi-forum-web/wiki/MPI3Tools/pimpi
If anybody wants to bring up another agenda item, please let me know.
Martin
_______________________________________________________________________ Martin Schulz, [email protected], http:// people.llnl.gov/schulz6 CASC @ Lawrence Livermore National Laboratory, Livermore, USA
_______________________________________________ Mpi3-tools mailing list [email protected] http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-tools
-- Marc-Andre Hermanns Juelich Supercomputing Centre Institute for Advanced Simulation Forschungszentrum Juelich GmbH D-52425 Juelich Germany
Phone : +49-2461-61-2054 Fax : +49-2461-61-6656 eMail : [email protected] WWW : http://www.fz-juelich.de/jsc/
JSC is the coordinator of the John von Neumann Institute for Computing and member of the Gauss Centre for Supercomputing
Sitz der Gesellschaft: Juelich Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 Vorsitzende des Aufsichtsrats: MinDir'in Baerbel Brumme-Bothe Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender), Dr. Ulrich Krafft (stellv. Vorsitzender), Prof. Dr. Harald Bolt, Prof. Dr. Sebastian M. Schmidt <smime.p7s><ATT00001.txt>
William Gropp Deputy Director for Research Institute for Advanced Computing Applications and Technologies Paul and Cynthia Saylor Professor of Computer Science University of Illinois Urbana-Champaign
_______________________________________________ Mpi3-tools mailing list [email protected] http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-tools
-- Marc-Andre Hermanns Juelich Supercomputing Centre Institute for Advanced Simulation Forschungszentrum Juelich GmbH D-52425 Juelich Germany
Phone : +49-2461-61-2054 Fax : +49-2461-61-6656 eMail : [email protected] WWW : http://www.fz-juelich.de/jsc/
JSC is the coordinator of the John von Neumann Institute for Computing and member of the Gauss Centre for Supercomputing
Sitz der Gesellschaft: Juelich Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 Vorsitzende des Aufsichtsrats: MinDir'in Baerbel Brumme-Bothe Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender), Dr. Ulrich Krafft (stellv. Vorsitzender), Prof. Dr. Harald Bolt, Prof. Dr. Sebastian M. Schmidt _______________________________________________ Mpi3-tools mailing list [email protected] http://*lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-tools
_______________________________________________________________________ Martin Schulz, [email protected], http:// people.llnl.gov/schulz6 CASC @ Lawrence Livermore National Laboratory, Livermore, USA
participants (1)
-
Martin Schulz