July 2010 - mpiwg-tools - mailman3.cels.anl.gov

Re: [Mpi3-tools] Discussion Topics for the MPIT Interface
by Fab Tillier 30 Jul '10

30 Jul '10

Martin Schulz wrote on Sun, 25 Jul 2010 at 14:34:02 > String Interfaces > ---------------------- > > For many API calls, the MPIT interface relies on being able to return > strings. The only other call (afaik) is MPI_Get_processor_name, which > uses the following method: > > Define a constant for maximum string length > OUT parameter for string buffer (written by the routine), which must > be at least as large as the constant > OUT parameter for the number of characters written by the routine > > In one of the presentations at the forum, some commented that this is > not ideal since it can easily lead to buffer overflow. That was me. > The suggested > alternative was to include an IN parameter (or make the length an IN/OUT > parameter) that allows the caller to specify how long the buffer is. I prefer a single in/out parameter, but either way would work. > If this is set to 0, the routine would not copy the string, but instead > return the size of the string it intends to write. Tools can then > allocate an appropriate size buffer. > > For MPIT this could be especially useful, since we intend to return > description texts, which can be long. > > With this we have the following options (please add if there are more): > > - Stay compatible with the rest of the standard and use the techniques > from MPI_Get_processor_name I'm strongly opposed to this one, as it requires a recompile of tools for every targeted MPI implementation. > - Adopt the new interface for all MPIT calls only I think this is probably the best path forward. > - Adopt the new interface for all MPIT calls and propose a new routine > for MPI_Get_processor_name in MPI-3 with the new interface > - Include both interfaces into MPIT (and propose a matching second > routine for MPI_Get_processor_name) I don't see a reason to support both - if someone wants to used fixed size buffers, they only need to initialize the value of the in/out length parameter - no need to dynamically query and allocate buffers if they don't want to (though they will need to handle errors if their buffer is too small.) > I personally think that we (for now) should stay away of changing > MPI_Get_processor_name and hence would favor one of the first two > solutions. There's more than just MPI_Get_processor_name: MPI_Comm_get_name, MPI_Type_get_name, and MPI_Win_get_name were the first to pop into my mind. There may be others, I haven't done an exhaustive check. For now, I would recommend creating a placeholder ticket for these functions in the existing standard, and move forward with the MPIT work using the second option. We'll have to deprecate the existing functions and create new ones to replace them. I haven't had time to thoroughly read the MPIT interface definition, but it would be great if we could define it such that there is a clear ABI definition (per target OS at least.) Fixing the string size issue goes a long way towards this. For Windows specifically, one aspect that's missing is being explicit as to the function calling convention (I don't know if this is an issue for other operating systems.) Cheers, -Fab

1 0

Re: [Mpi3-tools] Next MPI-3 Tools WG meeting Monday 8/26 + Schedule fornext meetings
by Jeff Squyres 26 Jul '10

26 Jul '10

On Jul 25, 2010, at 5:23 PM, Martin Schulz wrote: > Here is the proposed schedule for the upcoming meetings after tomorrow: > > 8/9 - Final discussion of the MPIR document > 8/23 - Discussion of the completed/integrated MPIT document > 9/6 - Labor day - no meeting > 9/16-9/18 - MPI forum meeting in Stuttgart > First reading of the MPIR document > 9/20 - Feedback from the MPI forum, MPIT discussions Webex links for these meetings are now up on the wiki. -- Jeff Squyres jsquyres(a)cisco.com For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/

1 0

[Mpi3-tools] Discussion Topics for the MPIT Interface
by Martin Schulz 25 Jul '10

25 Jul '10

Hi all, As listed in the previous email, tomorrow's discussion will be around the MPIT interface. I am in the process of putting together a final proposal (based on the last version we had a while ago and the various additions and changes we discussed in the meantime). During this process, I came across a few open issues that I would like to get some feedback/discussion. Here is a list of issues (details below) that I would like to go through on the TelCon: - MPIT Session management - String interfaces and compatibility with rest of the standard - Use of handles for environment variables - Routines to control environment variables - How to integrate MPIT into the standard In particular the first two issues have quite an impact on the API and the MPIT document. it would be great if we could find consensus for them. If anyone has anything else, please send it out to the group. Thanks and talk to you on Monday, Martin MPIT Sessions --------------------- The current version of the MPIT interface is global, i.e., initialized once and all counters are set/reset globally. This works for one tool, but many said that this may not be sufficient. E.g., libraries may want to monitor some data, which would then conflict with other libraries or tools. We already adjusted the init and finalize routines such that they can be called multiple times, but that is only a partial solution. I think we need to keep this feature, though, but the question is whether we want to go beyond that. One idea was to have explicit session - instead of an init call, one would "create" an MPIT session and all subsequent calls to MPIT would take a session handle as the first argument. All counters allocated and probed during a session would then only be relative to the session in which they are called. This would certainly complicate the interface, but would also make it more general and more usable. Opinions? Does anyone see any obvious problems with going down this route? String Interfaces ---------------------- For many API calls, the MPIT interface relies on being able to return strings. The only other call (afaik) is MPI_Get_processor_name, which uses the following method: Define a constant for maximum string length OUT parameter for string buffer (written by the routine), which must be at least as large as the constant OUT parameter for the number of characters written by the routine In one of the presentations at the forum, some commented that this is not ideal since it can easily lead to buffer overflow. The suggested alternative was to include an IN parameter (or make the length an IN/OUT parameter) that allows the caller to specify how long the buffer is. If this is set to 0, the routine would not copy the string, but instead return the size of the string it intends to write. Tools can then allocate an appropriate size buffer. For MPIT this could be especially useful, since we intend to return description texts, which can be long. With this we have the following options (please add if there are more): - Stay compatible with the rest of the standard and use the techniques from MPI_Get_processor_name - Adopt the new interface for all MPIT calls only - Adopt the new interface for all MPIT calls and propose a new routine for MPI_Get_processor_name in MPI-3 with the new interface - Include both interfaces into MPIT (and propose a matching second routine for MPI_Get_processor_name) I personally think that we (for now) should stay away of changing MPI_Get_processor_name and hence would favor one of the first two solutions. Handles for Environment Variables ------------------------------------------------ Performance variables are accessed through handles to improve access speed. Initially we thought that we won't need this for environment variables, However, some have commented that this may also be useful and that we should add this. Are there any good use cases? If we decide to add this, should there be separate handles for performance and environment variables (since the query routines return significantly different information) or not (to reuse the actual data read and set routines)? Ability to Control Environment Variables ------------------------------------------------------ The API includes the ability to set environment variables (if the MPI library enables this). There is actually quite a bit of interest in this feature from the tools and application developer community (we had some discussions on this at SciDAC 2010). To make this more efficient, one suggestion was to allow "batch" updates - the idea behind this is that each single update may need global synchronization and/or may be costly. In this cases, the MPI implementation could delay the settings until an "apply call". One proposal for this would include: - Return information on whether a variable is delayed by MPI to a global apply call (as part of the query call) - A new API call, which is globally synchronizing (perhaps restricted to a communicator?), which then tries to apply all changes made since the last apply call. Variations are certainly possible. Question to the MPI implementors: would this be useful or do you not expect changes to environment to be costly/globally synchronizing? Integration into the standard -------------------------------------- Since we removed the debugging APIs from the proposal, we are proposing only one addition. We could do this one of two ways: - Add a new chapter just for MPIT - Create a shared chapter for tools (with PMPI and MPIT) I think, based on forum feedback, we still have a majority for the second option, which would also be my preference. Any other opinions on this? Decided Issues / pending integration into the document --------------------------------------------------------------------------- - Hierarchy and Group interface Proposed by Dave and discussed separately - Textual updates - Complete list of error codes / reorganization into a single table (there are probably more - don't have the full list in front of me at the moment) ________________________________________________________________________ Martin Schulz, schulzm(a)llnl.gov, http://people.llnl.gov/schulzm CASC @ Lawrence Livermore National Laboratory, Livermore, USA

1 0

[Mpi3-tools] Next MPI-3 Tools WG meeting Monday 8/26 + Schedule for next meetings
by Martin Schulz 25 Jul '10

25 Jul '10

Hi all, Just as a reminder, the next MPI tools TelCon is scheduled for tomorrow (Monday) at the usual time at 8am PDT, 11am EDT, 5pm MSZ. Jeff has again created a Webex meeting for us and the link for that is: https://cisco.webex.com/ciscosales/j.php?ED=144317932&UID=0&PW=NYjk1YmJhZGE… Password: tools The main agenda item for tomorrow is the discussion of open issues for the MPIT document. I will send out a separate email with the details shortly. Unfortunately I will only be able to make the first 20min of the call, but Kathryn offered to lead the rest of the call and to take notes (Thanks!). Here is the proposed schedule for the upcoming meetings after tomorrow: 8/9 - Final discussion of the MPIR document 8/23 - Discussion of the completed/integrated MPIT document 9/6 - Labor day - no meeting 9/16-9/18 - MPI forum meeting in Stuttgart First reading of the MPIR document 9/20 - Feedback from the MPI forum, MPIT discussions I updated the Wiki with this information. Martin PS: Jeff could you please create new Webex meetings for 8/9, 8/23, and 9/20? Thanks! ________________________________________________________________________ Martin Schulz, schulzm(a)llnl.gov, http://people.llnl.gov/schulzm CASC @ Lawrence Livermore National Laboratory, Livermore, USA

1 0

[Mpi3-tools] [Fwd: Re: Reminder: next tools WG Meeting Monday 7/19 (tomorrow) - topic: MPIR]
by John DelSignore 19 Jul '10

19 Jul '10

Second attempt, after changing my email address.

1 0

Re: [Mpi3-tools] Reminder: next tools WG Meeting Monday 7/19 (tomorrow) - topic: MPIR
by Marc-André Hermanns 19 Jul '10

19 Jul '10

Hi all, as I am quite inexperienced with the implementation and use of debugging interfaces I have been more silent about the MPIR discussion, yet, I wanted to raise some questions. I apologize when the answers might seem obvious. ======= Page 7/8, Par. 1: …after it has set MPIR_debug_state to MPIR_DEBUG_SPAWNED and … Should the variable have an initial state? MPIR_NULL? ======= Page 8, Par. 3: …, and avoid defining MPIR_debug_gate in the MPI processes. What does avoid mean? Is it still ok to do so? If it is ok, why should it be avoided? When it is not ok, why not prohibit it? ======= Page 11, Par. 3 + 6: If MPIR_Breakpoint exists the tool should consider this a starter process … if MPIR_i_am_starter ist not defined …. why are there two variables/symbols that seem to do the same thing? ======= Page 18, Par. 10: …is a symbol of any kind, preferably an int, …. If it can be any kind, why prefer an int? Why not require it to be an int? ======= In general: - Some symbols are evaluated for existence or non-existence, yet others have a state (1 or 0). Why? Best regards, Marc-Andre On 18.07.2010, at 23:27, Martin Schulz wrote: > Just as a reminder: the next MPI tools WG meeting will be tomorrow morning at 8am PDT, > 11am EDT, 5pm MSZ. The main item on the agenda is the discussion on the MPIR > document that John has put together. > > The link to the Webex session is on the tools wiki and below: > https://cisco.webex.com/ciscosales/j.php?ED=144317932&UID=0&PW=NYjk1YmJhZGE… > Password: tools > > The latest version of the MPIR document is attached. > > Martin > > <MPIR Process Acquisition Interface 2010-06-11.pdf> > ________________________________________________________________________ > Martin Schulz, schulzm(a)llnl.gov, http://people.llnl.gov/schulzm > CASC @ Lawrence Livermore National Laboratory, Livermore, USA > > > > _______________________________________________ > Mpi3-tools mailing list > Mpi3-tools(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-tools -- Marc-Andre Hermanns German Research School for Simulation Sciences GmbH c/o Laboratory for Parallel Programming 52056 Aachen | Germany Tel +49 241 80 99753 Fax +49 241 80 6 99753 Web www.grs-sim.de Members: Forschungszentrum Jülich GmbH | RWTH Aachen University Registered in the commercial register of the local court of Düren (Amtsgericht Düren) under registration number HRB 5268 Registered office: Jülich Executive board: Prof. Dr. Heiner Müller-Krumbhaar | Prof. Marek Behr, Ph.D.

1 0

[Mpi3-tools] Reminder: next tools WG Meeting Monday 7/19 (tomorrow) - topic: MPIR
by Martin Schulz 18 Jul '10

18 Jul '10

Just as a reminder: the next MPI tools WG meeting will be tomorrow morning at 8am PDT, 11am EDT, 5pm MSZ. The main item on the agenda is the discussion on the MPIR document that John has put together. The link to the Webex session is on the tools wiki and below: https://cisco.webex.com/ciscosales/j.php?ED=144317932&UID=0&PW=NYjk1YmJhZGE… Password: tools The latest version of the MPIR document is attached. Martin ________________________________________________________________________ Martin Schulz, schulzm(a)llnl.gov, http://people.llnl.gov/schulzm CASC @ Lawrence Livermore National Laboratory, Livermore, USA

1 0

[Mpi3-tools] Fwd: (Forward to attendees) Meeting changed: MPI-3 Tools WG meeting
by Jeff Squyres 14 Jul '10

14 Jul '10

Webexes scheduled for the 19th and 26th. URLs are also on the wiki for convenience: https://svn.mpi-forum.org/trac/mpi-forum-web/wiki/MPI3Tools Begin forwarded message: > From: Jeff Squyres <messenger(a)webex.com> > Date: July 14, 2010 3:10:00 PM EDT > To: jsquyres(a)cisco.com > Subject: (Forward to attendees) Meeting changed: MPI-3 Tools WG meeting > Reply-To: jsquyres(a)cisco.com > > **** You can forward this email invitation to attendees **** > > Hello , > > Jeff Squyres changed the meeting information. > > Topic: MPI-3 Tools WG meeting > Date: Every Monday, from Monday, July 19, 2010 to Monday, July 26, 2010 > Time: 11:00 am, Eastern Daylight Time (New York, GMT-04:00) > Meeting Number: 201 866 209 > Meeting Password: tools, > > > ------------------------------------------------------- > To join the online meeting (Now from iPhones and other Smartphones too!) > ------------------------------------------------------- > 1. Go to https://cisco.webex.com/ciscosales/j.php?ED=144317932&UID=0&PW=NYjk1YmJhZGE… > 2. Enter your name and email address. > 3. Enter the meeting password: tools, > 4. Click "Join Now". > 5. Follow the instructions that appear on your screen. > > To view in other time zones or languages, please click the link: > https://cisco.webex.com/ciscosales/j.php?ED=144317932&UID=0&PW=NYjk1YmJhZGE… > > ---------------------------------------------------------------- > ALERT:Toll-Free Dial Restrictions for (408) and (919) Area Codes > ---------------------------------------------------------------- > > The affected toll free numbers are: (866) 432-9903 for the San Jose/Milpitas area and (866) 349-3520 for the RTP area. > > Please dial the local access number for your area from the list below: > - San Jose/Milpitas (408) area: 525-6800 > - RTP (919) area: 392-3330 > > ------------------------------------------------------- > To join the teleconference only > ------------------------------------------------------- > 1. Dial into Cisco WebEx (view all Global Access Numbers at > http://cisco.com/en/US/about/doing_business/conferencing/index.html > 2. Follow the prompts to enter the Meeting Number (listed above) or Access Code followed by the # sign. > > San Jose, CA: +1.408.525.6800 RTP: +1.919.392.3330 > > US/Canada: +1.866.432.9903 United Kingdom: +44.20.8824.0117 > > India: +91.80.4350.1111 Germany: +49.619.6773.9002 > > Japan: +81.3.5763.9394 China: +86.10.8515.5666 > > ------------------------------------------------------- > For assistance > ------------------------------------------------------- > 1. Go to https://cisco.webex.com/ciscosales/mc > 2. On the left navigation bar, click "Support". > > You can contact me at: > jsquyres(a)cisco.com > 1-408-525 0971 > > To update this meeting to your calendar program (for example Microsoft Outlook), click this link: > https://cisco.webex.com/ciscosales/j.php?ED=144317932&UID=0&ICS=UMI&LD=1&RD… > > > WebEx will automatically setup Meeting Manager for Windows the first time you join a meeting. To save time, you can setup prior to the meeting by clicking this link: > https://cisco.webex.com/ciscosales/meetingcenter/mcsetup.php > > > > > http://www.webex.com > > CCP:+14085256800x201866209# > > IMPORTANT NOTICE: This WebEx service includes a feature that allows audio and any documents and other materials exchanged or viewed during the session to be recorded. By joining this session, you automatically consent to such recordings. If you do not consent to the recording, do not join the session. -- Jeff Squyres jsquyres(a)cisco.com For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/

1 0

[Mpi3-tools] Tools WG meeting on 7/12 moved to 7/19
by Martin Schulz 11 Jul '10

11 Jul '10

Hi all, In order to have a critical mass of people involved in the MPIR discussion, let's move the next tools WG meeting to 7/19 (instead of 7/12). This means there will be no meeting on 7/12. Here is the schedule for the next two meetings 7/19 MPIR discussion 7/26 MPIT discussion All of this at the usual time (8am PDT, 11am EDT, 5pm MSZ) - I'll update the tools Wiki tonight. Hope this works for everyone, Martin ________________________________________________________________________ Martin Schulz, schulzm(a)llnl.gov, http://people.llnl.gov/schulzm CASC @ Lawrence Livermore National Laboratory, Livermore, USA

1 0