January 2011 - mpiwg-tools - mailman3.cels.anl.gov

[Mpi3-tools] Draft of the MPIT Specification for the Presentation at February MPI Forum
by Martin Schulz 27 Jan '11

27 Jan '11

Hi all, As probably many of you know, the MPI-3 WG on tools has been (and is) working on an extension to MPI that allows tools (as well as adaptive runtime systems or applications) access to MPI internals. We have named this interface MPIT. The WG has converged on a draft for MPIT that we plan to introduce into the MPI forum process for inclusion into MPI-3. Following the new rules for MPI-3 standardization, I will be presenting this draft at the next MPI forum meeting on Tuesday afternoon. To help this discussion, the current version is attached to this email. As part of this proposal, we also propose to create an overarching chapter on tools, which includes both the existing (unmodified) PMPI interface and the new MPIT specification. This is reflected in the attached draft document. I will upload it to the Tools WG Wiki as well as the MPI SVN at the following URL (once the I can get back onto the servers, which seem to be down at the moment): https://svn.mpi-forum.org/svn/mpi-forum-docs/trunk/working-groups/mpi-3/too… Let me know if you have any questions or comments ahead of the presentation! Thanks, Martin ________________________________________________________________________ Martin Schulz, schulzm(a)llnl.gov, http://people.llnl.gov/schulzm CASC @ Lawrence Livermore National Laboratory, Livermore, USA

1 0

Re: [Mpi3-tools] reviewing the document
by Martin Schulz 26 Jan '11

26 Jan '11

Hi Dave, On Jan 26, 2011, at 7:32 AM, Dave Goodell wrote: > On Jan 26, 2011, at 8:09 AM CST, Jeff Squyres wrote: > >> On Jan 25, 2011, at 6:58 PM, Dave Goodell wrote: >> >>>> I'm wondering if there are really two options here: >>>> >>>> 1. Have a real casting operating (somewhat akin to Martin's initial proposal): have a casting function that returns the OUT value (vs. an int error code). Then you can nest the output of the casting function as an argument to MPI_Perfvar_handle_allocate(), for example. >>>> >>>> 2. Have 9 different versions of MPI_Perfvar_handle_allocate() and MPI_Controlvar_handle_allocate() -- one for each MPI handle type. >>>> >>>> I assume most people will recoil from #2. But... hmm. It *is* the safest way to go. But ugly (because it's lots of functions). Hmm... >>> >>> #1 is cute, but it doesn't match MPI error handling conventions, and I'd rather not begin passing the error code back as an OUT arg for just this small set of conversion routines. >> >> MPI_Wtick, MPI_Wtime, and the f2c / c2f routines return their OUT values. Indeed, the f2c / c2f routines could be considered "casting-like" operators. > > That's a good point, I always forget about f2c/c2f routines because they are just little integer type casting macros in MPICH2. Maybe it's not quite the departure from convention that I claimed. > > Actually, I think I'm in favor of #1 if the implementation is allowed to implement them as macros. They'll be no-ops for us, but you can have full functions if you like to add some type checking from the compiler. But if you want to allocate/deallocate memory in the conversion routines in order to create a new generic handle then this system won't work. Is that what you want? Or is a "user comment" cast of the pointer value all that you need? I think allowing the implementation to be macros is problematic. MPI objects are often intercepted by tools and in many cases also replaced. In this case, the tool must be able to intercept also these conversion routines to make sure any replaced object is also passed on to the MPIT routines. Martin > > -Dave > > > _______________________________________________ > Mpi3-tools mailing list > Mpi3-tools(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-tools ________________________________________________________________________ Martin Schulz, schulzm(a)llnl.gov, http://people.llnl.gov/schulzm CASC @ Lawrence Livermore National Laboratory, Livermore, USA

1 0

Re: [Mpi3-tools] reviewing the document
by Dave Goodell 26 Jan '11

26 Jan '11

On Jan 26, 2011, at 8:09 AM CST, Jeff Squyres wrote: > On Jan 25, 2011, at 6:58 PM, Dave Goodell wrote: > >>> I'm wondering if there are really two options here: >>> >>> 1. Have a real casting operating (somewhat akin to Martin's initial proposal): have a casting function that returns the OUT value (vs. an int error code). Then you can nest the output of the casting function as an argument to MPI_Perfvar_handle_allocate(), for example. >>> >>> 2. Have 9 different versions of MPI_Perfvar_handle_allocate() and MPI_Controlvar_handle_allocate() -- one for each MPI handle type. >>> >>> I assume most people will recoil from #2. But... hmm. It *is* the safest way to go. But ugly (because it's lots of functions). Hmm... >> >> #1 is cute, but it doesn't match MPI error handling conventions, and I'd rather not begin passing the error code back as an OUT arg for just this small set of conversion routines. > > MPI_Wtick, MPI_Wtime, and the f2c / c2f routines return their OUT values. Indeed, the f2c / c2f routines could be considered "casting-like" operators. That's a good point, I always forget about f2c/c2f routines because they are just little integer type casting macros in MPICH2. Maybe it's not quite the departure from convention that I claimed. Actually, I think I'm in favor of #1 if the implementation is allowed to implement them as macros. They'll be no-ops for us, but you can have full functions if you like to add some type checking from the compiler. But if you want to allocate/deallocate memory in the conversion routines in order to create a new generic handle then this system won't work. Is that what you want? Or is a "user comment" cast of the pointer value all that you need? -Dave

1 0

Re: [Mpi3-tools] reviewing the document
by Jeff Squyres 26 Jan '11

26 Jan '11

On Jan 25, 2011, at 6:58 PM, Dave Goodell wrote: >> 3. MPICH may not need real handles for MPIT_MPI_Resource instance (because they're bit-mapped integers), but pointer-driven MPI implementations like Open MPI (may) need at least 2 values: the handle and the type of the handle. Hence, you need to be able to free MPIT_MPI_Resource handles. > > Technically, even if you have an Open MPI-esque implementation, you could implement your MPI object structs s.t. they have a common header containing RTTI and then gently cheat on C's strict-aliasing rules. This would avoid the allocation issues. However, I'm guessing that things aren't already implemented this way for you and such a change could be fairly intrusive. You're completely right; I typed my message in haste last night and meant to say pretty much exactly what you did, but forgot. I mentioned to Martin multiple times that I need to check OMPI and see if we did what you mentioned (have some kind of member right up at the top of all MPI object such that we could figure out its type if we were given just a pointer). I'm not sure that we did, nor what the consequences would be if we grafted such a scheme into the current code base (i.e., I just haven't thought about it yet). >> [snip] >> So why have it? > > I agree, I don't really see the justification for the N routines, as I indicated on the concalls. :-) >> I'm wondering if there are really two options here: >> >> 1. Have a real casting operating (somewhat akin to Martin's initial proposal): have a casting function that returns the OUT value (vs. an int error code). Then you can nest the output of the casting function as an argument to MPI_Perfvar_handle_allocate(), for example. >> >> 2. Have 9 different versions of MPI_Perfvar_handle_allocate() and MPI_Controlvar_handle_allocate() -- one for each MPI handle type. >> >> I assume most people will recoil from #2. But... hmm. It *is* the safest way to go. But ugly (because it's lots of functions). Hmm... > > #1 is cute, but it doesn't match MPI error handling conventions, and I'd rather not begin passing the error code back as an OUT arg for just this small set of conversion routines. MPI_Wtick, MPI_Wtime, and the f2c / c2f routines return their OUT values. Indeed, the f2c / c2f routines could be considered "casting-like" operators. > #2 is horrifying, we will be laughed at for years to come if we attempt something like this. Maybe. But consider that overloading languages like C++ effectively do the same thing; they just have a nice language-built-in-function that all N functions *look* like 1 function. But they're really still implemented (and possibly described) as N different functions, with (potentially) different implementations and logic. > My vote is still for one of the following: > > 3. No conversion routines, just a (void*) arg and assume that the implementation will provide any reasonable amount of runtime type checking that it can. This would work fine for MPICH2 at least, but I think that the resource args to the *var_handle_allocate routines isn't very self-explanatory in this case. I'm not sure that that's a good or a bad assumption (that's not a snarky comment -- I'm honestly not sure). As mentioned above, I have to think about whether we can do that in OMPI or not, for example. > 4. A single conversion routine that takes a (void*) and a constant indicating the type of the input pointer. Like this: > > int MPIT_Convert_object(void *object_ptr, int obj_type, MPIT_MPI_Resource *resource); That would certainly be safer than #3, even if the obj_type is effectively redundant (other than perhaps an additional error/consistency check) in some implementations like mpich2. -- Jeff Squyres jsquyres(a)cisco.com For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/

1 0

Re: [Mpi3-tools] reviewing the document
by Martin Schulz 26 Jan '11

26 Jan '11

Hi all, On Jan 25, 2011, at 3:58 PM, Dave Goodell wrote: > On Jan 25, 2011, at 5:35 PM CST, Jeff Squyres wrote: > >> 1. There's no such thing as an "MPI resource". The spec defines "MPI handles" and "MPI objects"; we need to couch our definitions in those terms. I submitted some suggested text to Martin for section 1.2.2, but did not have time to apply it to the rest of the document. > > Agreed. I will try to make it through the whole document to make that change. The more I think about it, the more I am convinced we need this fixed before sending it out to everyone. > >> 2. There's currently no way to free MPIT_MPI_Resource handles (see #3 before saying that you don't need this :-) ). >> >> 3. MPICH may not need real handles for MPIT_MPI_Resource instance (because they're bit-mapped integers), but pointer-driven MPI implementations like Open MPI (may) need at least 2 values: the handle and the type of the handle. Hence, you need to be able to free MPIT_MPI_Resource handles. > > Technically, even if you have an Open MPI-esque implementation, you could implement your MPI object structs s.t. they have a common header containing RTTI and then gently cheat on C's strict-aliasing rules. This would avoid the allocation issues. However, I'm guessing that things aren't already implemented this way for you and such a change could be fairly intrusive. Yes, Jeff and I talked about this as well and he wasn't sure if this is the case right now in Open MPI. Even if it would, other implementations may not have done it that way and it would be a major pain for them to change. > >> 4. I'm really re-thinking the whole idea that the functions in 1.2.2 provide compile-time type safety. Specifically: I don't think that they do. Sure, you're protected from this kind of bozo error: >> >> MPIT_MPI_Resource_communicator(MPI_INT, ...); >> >> but you're not protected from this kind of IMNSHO-more-insidious error: >> >> MPIT_MPI_Resource_communicator(MPI_COMM_WORLD, &foo); >> MPIT_Perfvar_handle_allocate(x, y, foo, &handle); >> >> where (x, y) refers to a performance variable associated with MPI datatypes. >> >> Sure, you'll get a run-time error, but what did we really gain with the conversion function there? The conversion function is acting like a cast -- nothing more. And just like a real casting operator, it (effectively) tells the compiler "don't worry, I know what I'm doing -- don't complain about types here." I.e., we give up the compile time type safety. You can still hose yourself w.r.t. the foo MPIT handle and get no protection from the compiler. >> >> So why have it? > > I agree, I don't really see the justification for the N routines, as I indicated on the concalls. > >> I'm wondering if there are really two options here: >> >> 1. Have a real casting operating (somewhat akin to Martin's initial proposal): have a casting function that returns the OUT value (vs. an int error code). Then you can nest the output of the casting function as an argument to MPI_Perfvar_handle_allocate(), for example. >> >> 2. Have 9 different versions of MPI_Perfvar_handle_allocate() and MPI_Controlvar_handle_allocate() -- one for each MPI handle type. >> >> I assume most people will recoil from #2. But... hmm. It *is* the safest way to go. But ugly (because it's lots of functions). Hmm... > > #1 is cute, but it doesn't match MPI error handling conventions, and I'd rather not begin passing the error code back as an OUT arg for just this small set of conversion routines. > > #2 is horrifying, we will be laughed at for years to come if we attempt something like this. There is one other alternative, although, I have to say upfront that I am not a fan of it: we could provide a unified handle allocation routine for both control and performance variables. In this case, we would have to pass an additional argument that specifies whether the index is a control or performance variable - and that's the part I think is ugly. However, in this case we would only need one routine per object type, not two. > > My vote is still for one of the following: > > 3. No conversion routines, just a (void*) arg and assume that the implementation will provide any reasonable amount of runtime type checking that it can. This would work fine for MPICH2 at least, but I think that the resource args to the *var_handle_allocate routines isn't very self-explanatory in this case. > > 4. A single conversion routine that takes a (void*) and a constant indicating the type of the input pointer. Like this: > > int MPIT_Convert_object(void *object_ptr, int obj_type, MPIT_MPI_Resource *resource); > > (rename resource appropriately once you decide what you want to call it) This at least makes it somewhat clear to new users what the handle_allocate routines are expecting. If we provide any kind of conversion routine that returns a new, generic object we will have to deal with a matching free routine. This will also open questions like, what happens if the object is freed, but not the generic resource derived from it? I think this can get messy. Based on all this discussion, I am starting to lean back towards just having a void* in the handle allocation routines. If we can't get full type safety anyway without being laughed at, then we might as well take the simplest approach. Martin > > -Dave > > > _______________________________________________ > Mpi3-tools mailing list > Mpi3-tools(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-tools ________________________________________________________________________ Martin Schulz, schulzm(a)llnl.gov, http://people.llnl.gov/schulzm CASC @ Lawrence Livermore National Laboratory, Livermore, USA

1 0

Re: [Mpi3-tools] reviewing the document
by Dave Goodell 25 Jan '11

25 Jan '11

On Jan 25, 2011, at 5:35 PM CST, Jeff Squyres wrote: > 1. There's no such thing as an "MPI resource". The spec defines "MPI handles" and "MPI objects"; we need to couch our definitions in those terms. I submitted some suggested text to Martin for section 1.2.2, but did not have time to apply it to the rest of the document. Agreed. > 2. There's currently no way to free MPIT_MPI_Resource handles (see #3 before saying that you don't need this :-) ). > > 3. MPICH may not need real handles for MPIT_MPI_Resource instance (because they're bit-mapped integers), but pointer-driven MPI implementations like Open MPI (may) need at least 2 values: the handle and the type of the handle. Hence, you need to be able to free MPIT_MPI_Resource handles. Technically, even if you have an Open MPI-esque implementation, you could implement your MPI object structs s.t. they have a common header containing RTTI and then gently cheat on C's strict-aliasing rules. This would avoid the allocation issues. However, I'm guessing that things aren't already implemented this way for you and such a change could be fairly intrusive. > 4. I'm really re-thinking the whole idea that the functions in 1.2.2 provide compile-time type safety. Specifically: I don't think that they do. Sure, you're protected from this kind of bozo error: > > MPIT_MPI_Resource_communicator(MPI_INT, ...); > > but you're not protected from this kind of IMNSHO-more-insidious error: > > MPIT_MPI_Resource_communicator(MPI_COMM_WORLD, &foo); > MPIT_Perfvar_handle_allocate(x, y, foo, &handle); > > where (x, y) refers to a performance variable associated with MPI datatypes. > > Sure, you'll get a run-time error, but what did we really gain with the conversion function there? The conversion function is acting like a cast -- nothing more. And just like a real casting operator, it (effectively) tells the compiler "don't worry, I know what I'm doing -- don't complain about types here." I.e., we give up the compile time type safety. You can still hose yourself w.r.t. the foo MPIT handle and get no protection from the compiler. > > So why have it? I agree, I don't really see the justification for the N routines, as I indicated on the concalls. > I'm wondering if there are really two options here: > > 1. Have a real casting operating (somewhat akin to Martin's initial proposal): have a casting function that returns the OUT value (vs. an int error code). Then you can nest the output of the casting function as an argument to MPI_Perfvar_handle_allocate(), for example. > > 2. Have 9 different versions of MPI_Perfvar_handle_allocate() and MPI_Controlvar_handle_allocate() -- one for each MPI handle type. > > I assume most people will recoil from #2. But... hmm. It *is* the safest way to go. But ugly (because it's lots of functions). Hmm... #1 is cute, but it doesn't match MPI error handling conventions, and I'd rather not begin passing the error code back as an OUT arg for just this small set of conversion routines. #2 is horrifying, we will be laughed at for years to come if we attempt something like this. My vote is still for one of the following: 3. No conversion routines, just a (void*) arg and assume that the implementation will provide any reasonable amount of runtime type checking that it can. This would work fine for MPICH2 at least, but I think that the resource args to the *var_handle_allocate routines isn't very self-explanatory in this case. 4. A single conversion routine that takes a (void*) and a constant indicating the type of the input pointer. Like this: int MPIT_Convert_object(void *object_ptr, int obj_type, MPIT_MPI_Resource *resource); (rename resource appropriately once you decide what you want to call it) This at least makes it somewhat clear to new users what the handle_allocate routines are expecting. -Dave

1 0

[Mpi3-tools] reviewing the document
by Jeff Squyres 25 Jan '11

25 Jan '11

I committed a bunch of punctuation, grammar, and minor word changes in r735. I only got up to section 1.2.2 (more below). I'm sorry, but the change bars have made the latex completely unreadable; I was not able to provide even more change bars. I only got up to section 1.2.2 because Martin and I spent a long time on the phone talking about the conversion functions. I'm still not quite sure what to do. Here's some of the points we talked about (some of which have no resolution): 1. There's no such thing as an "MPI resource". The spec defines "MPI handles" and "MPI objects"; we need to couch our definitions in those terms. I submitted some suggested text to Martin for section 1.2.2, but did not have time to apply it to the rest of the document. 2. There's currently no way to free MPIT_MPI_Resource handles (see #3 before saying that you don't need this :-) ). 3. MPICH may not need real handles for MPIT_MPI_Resource instance (because they're bit-mapped integers), but pointer-driven MPI implementations like Open MPI (may) need at least 2 values: the handle and the type of the handle. Hence, you need to be able to free MPIT_MPI_Resource handles. 4. I'm really re-thinking the whole idea that the functions in 1.2.2 provide compile-time type safety. Specifically: I don't think that they do. Sure, you're protected from this kind of bozo error: MPIT_MPI_Resource_communicator(MPI_INT, ...); but you're not protected from this kind of IMNSHO-more-insidious error: MPIT_MPI_Resource_communicator(MPI_COMM_WORLD, &foo); MPIT_Perfvar_handle_allocate(x, y, foo, &handle); where (x, y) refers to a performance variable associated with MPI datatypes. Sure, you'll get a run-time error, but what did we really gain with the conversion function there? The conversion function is acting like a cast -- nothing more. And just like a real casting operator, it (effectively) tells the compiler "don't worry, I know what I'm doing -- don't complain about types here." I.e., we give up the compile time type safety. You can still hose yourself w.r.t. the foo MPIT handle and get no protection from the compiler. So why have it? I'm wondering if there are really two options here: 1. Have a real casting operating (somewhat akin to Martin's initial proposal): have a casting function that returns the OUT value (vs. an int error code). Then you can nest the output of the casting function as an argument to MPI_Perfvar_handle_allocate(), for example. 2. Have 9 different versions of MPI_Perfvar_handle_allocate() and MPI_Controlvar_handle_allocate() -- one for each MPI handle type. I assume most people will recoil from #2. But... hmm. It *is* the safest way to go. But ugly (because it's lots of functions). Hmm... -- Jeff Squyres jsquyres(a)cisco.com For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/

1 0

[Mpi3-tools] New version of MPIT draft / Request for comments by PM on 1/26
by Martin Schulz 24 Jan '11

24 Jan '11

Hi all, As discussed in the TelCon this morning, here is the latest draft with the changes from this morning and Marc-Andre's email. The changes are also in SVN (merged with Dave's changes) and the document is available on the Wiki. I would like to send the final version of the draft out to the MPI forum by Wednesday (late) evening - hence, please send any further comments by Wednesday afternoon. For those of you with SVN access, please add minor and grammar changes directly to the document. Any change should be marked with the \chgjand{} macro. Thanks! Martin PS: The next step for MPIT is a presentation to the MPI forum on 2/8 from 1pm to 4pm PST. If anyone, who can't make it to the forum meeting, would like to listen in on the plenary presentation and discussion, please let Jeff and me know and we will try to setup an audio conference. ________________________________________________________________________ Martin Schulz, schulzm(a)llnl.gov, http://people.llnl.gov/schulzm CASC @ Lawrence Livermore National Laboratory, Livermore, USA

1 0

[Mpi3-tools] Dave's changes to MPIT
by Dave Goodell 24 Jan '11

24 Jan '11

1 0

[Mpi3-tools] MPI-3 Tools WG
by Jeff Squyres (jsquyres) 24 Jan '11

24 Jan '11

----- WebEx Invite ----- Meeting Number: 201034484 Password: tools ------------------------ To join the meeting online ------------------------ 1. Go to https://cisco.webex.com/cisco/j.php?ED=155386812&UID=0&PW=NMTJhNWQxOTdj&RT=… 2. Enter your name and email address. 3. Enter the meeting password: tools 4. Click "Join". 5. If the meeting includes a teleconference, follow the instructions that appear on your screen. ------------------------ To join the teleconference only ------------------------ 1. Dial into Cisco WebEx - view all Global Access Numbers at http://cisco.com/en/US/about/doing_business/conferencing/index.html 2. Follow the prompts to enter the Meeting Number (listed above) or Access Code followed by the # sign. San Jose, CA: +1.408.525.6800 RTP: +1.919.392.3330 US/Canada: +1.866.432.9903 United Kingdom: +44.20.8824.0117 India: +91.80.4350.1111 Germany: +49.619.6773.9002 Japan: +81.3.5763.9394 China: +86.10.8515.5666 Access code: 201034484 Sign up for a free trial of WebEx http://www.webex.com/go/mcemfreetrial IMPORTANT NOTICE: This WebEx service includes a feature that allows audio and any documents and other materials exchanged or viewed during the session to be recorded. By joining this session, you automatically consent to such recordings. If you do not consent to the recording, do not join the session.

1 0