November 2013 - mpiwg-tools - mailman3.cels.anl.gov

Telecon tomorrow
by Kathryn Mohror 13 Nov '13

13 Nov '13

Hi all, As a reminder, we have our regular telecon tomorrow at 11am EST and 8am PST. The call in information is here: https://svn.mpi-forum.org/trac/mpi-forum-web/wiki/MPI3Tools Agenda items: - Figure out what time we need at the forum meeting, tickets to vote on, etc. - Tickets #377, #384, #383, #400 (#392) - Discuss the "multiple device" support issue further Talk to you soon! Kathryn ______________________________________________________________ Kathryn Mohror, kathryn(a)llnl.gov, http://people.llnl.gov/mohror1 CASC @ Lawrence Livermore National Laboratory, Livermore, CA, USA

1 0

Re: [mpiwg-tools] Open MPI pvar scheme
by Schulz, Martin 12 Nov '13

12 Nov '13

On Nov 12, 2013, at 12:51 PM, "Jeff Squyres (jsquyres)" <jsquyres(a)cisco.com> wrote: > On Nov 11, 2013, at 7:03 PM, "Schulz, Martin" <schulzm(a)llnl.gov> wrote: > >>>> I think it would be better to programmatically express such relationships -- i.e., something new for MPI-4. >> >> I agree with that, but I see it more as an MPI 3.1 feature. Is it really a major concept? > > I thought it was "more" than a minor/bug fix concept... and therefore should wait until 4. We need to finally discuss and decide that (as the forum) - this will definitely be on the agenda in December. >>> Er... I don't understand. Why can't the enum strings change? I think they can change at any time. >> >> So that's a good question - I was assuming (at least that's how I always thought about it) that the same rules as for variables and all their metadata apply here - once read, a description cannot change and an enumeration type can worst thing grow. It is correct, though, that we didn't specify that explicitly - perhaps we need to do that. The only text that comes close is "The second datatype class, enumeration datatypes, describes variables with a fixed set of discrete values.". > > Mmm. Yes. This *implies* that they don't change, but one could argue different meanings of "fixed set". I know, this is ambiguous. > >> Having strings change underneath would be a real problem for the string interface, since it makes the assumption that you can run queries against a string more than once and get the same string. > > It sounds like you are introducing a new requirement, then: you want non-mutable strings, but the ability to have things change over time (e.g., new NICs added or old NICs dropped). Well, I think we have two separate issues here: - Should strings be allowed to change? In particular, should we allow enumerations to change on the fly? I think the answer to this should be no, since this would create race conditions and it would prevent tools from caching any data (that's why we did it for the variable metadata and for the types it seems even more relevant) - Do we need a mechanism to describe changing variable arrays? This is more a requirement question - how is this in your case? Will you support changing number of NICs and how are you planning on doing that? > >>> I think what you might be saying is that the *count* on a handle can't change -- right? I.e., MPI_T currently has no way of saying, "There were N values when you called pvar_handle_alloc() for this pvar, but now this pvar has M values." >> >> Yes, but the count can change when you get a new handle. > > That would be fairly hellish for an implementation: session/handle X has N values, but simultaneously, session/handle Y of the same pvar -- which just happened to be created after session/handle X -- has M values. > > Meaning: the implementation has to track two entirely different views/counts of the same pvar. Well, wouldn't that be up to the implementation? If it doesn't make sense for a particular case, then an MPI can always return the same count. I agree, if an MPI does do it that way, it won't be easy. > >> Yes, that's what I meant with "doesn't have to be at the moment" - any variable that has the potential to have a count that is not 1 is affected. > > ...which is *any* variable, right? Yes - what I meant was that if there is a variable that a particular MPI implementation always decides to return 1 for, then you will probably never need a matching enumeration variable. However, just because it returns 1 once, it doesn't mean that it is guaranteed that it will stay that way for the rest of the run or during the next run. Hence, matching enumeration variables make sense even for variables that have a count of 1. Martin > > -- > Jeff Squyres > jsquyres(a)cisco.com > For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/ > ________________________________________________________________________ Martin Schulz, schulzm(a)llnl.gov, http://people.llnl.gov/schulzm CASC @ Lawrence Livermore National Laboratory, Livermore, USA

1 0

Re: [mpiwg-tools] Open MPI pvar scheme
by Jeff Squyres (jsquyres) 12 Nov '13

12 Nov '13

On Nov 11, 2013, at 7:03 PM, "Schulz, Martin" <schulzm(a)llnl.gov> wrote: >>> I think it would be better to programmatically express such relationships -- i.e., something new for MPI-4. > > I agree with that, but I see it more as an MPI 3.1 feature. Is it really a major concept? I thought it was "more" than a minor/bug fix concept... and therefore should wait until 4. >> Er... I don't understand. Why can't the enum strings change? I think they can change at any time. > > So that's a good question - I was assuming (at least that's how I always thought about it) that the same rules as for variables and all their metadata apply here - once read, a description cannot change and an enumeration type can worst thing grow. It is correct, though, that we didn't specify that explicitly - perhaps we need to do that. The only text that comes close is "The second datatype class, enumeration datatypes, describes variables with a fixed set of discrete values.". Mmm. Yes. This *implies* that they don't change, but one could argue different meanings of "fixed set". > Having strings change underneath would be a real problem for the string interface, since it makes the assumption that you can run queries against a string more than once and get the same string. It sounds like you are introducing a new requirement, then: you want non-mutable strings, but the ability to have things change over time (e.g., new NICs added or old NICs dropped). >> I think what you might be saying is that the *count* on a handle can't change -- right? I.e., MPI_T currently has no way of saying, "There were N values when you called pvar_handle_alloc() for this pvar, but now this pvar has M values." > > Yes, but the count can change when you get a new handle. That would be fairly hellish for an implementation: session/handle X has N values, but simultaneously, session/handle Y of the same pvar -- which just happened to be created after session/handle X -- has M values. Meaning: the implementation has to track two entirely different views/counts of the same pvar. > Yes, that's what I meant with "doesn't have to be at the moment" - any variable that has the potential to have a count that is not 1 is affected. ...which is *any* variable, right? -- Jeff Squyres jsquyres(a)cisco.com For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/

1 0

Re: [mpiwg-tools] Open MPI pvar scheme
by Schulz, Martin 11 Nov '13

11 Nov '13

Hi Jeff, On Nov 11, 2013, at 3:46 PM, "Jeff Squyres (jsquyres)" <jsquyres(a)cisco.com> wrote: > On Nov 11, 2013, at 6:37 PM, "Schulz, Martin" <schulzm(a)llnl.gov> wrote: > >> Just to clarify: are you proposing that we add this officially to the standard (that particular naming scheme) or is this supposed to be an informal agreement? > > The latter. Oh, I see. >> I personally would like to see the former, since it will be important to have something tools can rely on. > > I think it would be better to programmatically express such relationships -- i.e., something new for MPI-4. I agree with that, but I see it more as an MPI 3.1 feature. Is it really a major concept? > >> Assuming you meant that (that's how I understood it), here is one concern: this scheme is not very flexible - once a name is set and the enumeration variable is provided, this cannot be changed anymore. Isn't this even too restrictive for your case (where NICs can come and go in, e.g., a hot-pluggable system)? > > Er... I don't understand. Why can't the enum strings change? I think they can change at any time. So that's a good question - I was assuming (at least that's how I always thought about it) that the same rules as for variables and all their metadata apply here - once read, a description cannot change and an enumeration type can worst thing grow. It is correct, though, that we didn't specify that explicitly - perhaps we need to do that. The only text that comes close is "The second datatype class, enumeration datatypes, describes variables with a fixed set of discrete values.". Having strings change underneath would be a real problem for the string interface, since it makes the assumption that you can run queries against a string more than once and get the same string. > > I think what you might be saying is that the *count* on a handle can't change -- right? I.e., MPI_T currently has no way of saying, "There were N values when you called pvar_handle_alloc() for this pvar, but now this pvar has M values." Yes, but the count can change when you get a new handle. > Right? > > If so, that's really a different (but still potentially valid) issue. > >> In general, the situation can appear in any scenario in which we have a variable that can (doesn't have to at the moment) return a value of more than 1 for the count. Depending on the variable, it can vary greatly in what that means and this can be very dynamic. I think a more dynamic lookup mechanism (e.g., query an enumeration once you have a count and a handle) would be better. > > I don't think this is only for pvars with a count>1. What if N=1 and M=3 in the previous example? Yes, that's what I meant with "doesn't have to be at the moment" - any variable that has the potential to have a count that is not 1 is affected. Martin > > -- > Jeff Squyres > jsquyres(a)cisco.com > For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/ > ________________________________________________________________________ Martin Schulz, schulzm(a)llnl.gov, http://people.llnl.gov/schulzm CASC @ Lawrence Livermore National Laboratory, Livermore, USA

1 0

ticket #383: increase level of consistency
by Kathryn Mohror 11 Nov '13

11 Nov '13

Hi all, I looked at this to try to figure out how to shoehorn in text for this ticket and settled on an advice to implementors near the front of the document that reads: Although this interface is flexible, implementations should strive for consistency in naming and definitions as much as possible. In particular, variables with the same name should have the same meaning across processes in a single job and across runs. I attached the document as well. I'm not sure I'm happy with it, so suggestions are welcome. Kathryn ______________________________________________________________ Kathryn Mohror, kathryn(a)llnl.gov, http://people.llnl.gov/mohror1 CASC @ Lawrence Livermore National Laboratory, Livermore, CA, USA

1 0

Re: [mpiwg-tools] Open MPI pvar scheme
by Jeff Squyres (jsquyres) 11 Nov '13

11 Nov '13

On Nov 11, 2013, at 6:37 PM, "Schulz, Martin" <schulzm(a)llnl.gov> wrote: > Just to clarify: are you proposing that we add this officially to the standard (that particular naming scheme) or is this supposed to be an informal agreement? The latter. > I personally would like to see the former, since it will be important to have something tools can rely on. I think it would be better to programmatically express such relationships -- i.e., something new for MPI-4. > Assuming you meant that (that's how I understood it), here is one concern: this scheme is not very flexible - once a name is set and the enumeration variable is provided, this cannot be changed anymore. Isn't this even too restrictive for your case (where NICs can come and go in, e.g., a hot-pluggable system)? Er... I don't understand. Why can't the enum strings change? I think they can change at any time. I think what you might be saying is that the *count* on a handle can't change -- right? I.e., MPI_T currently has no way of saying, "There were N values when you called pvar_handle_alloc() for this pvar, but now this pvar has M values." Right? If so, that's really a different (but still potentially valid) issue. > In general, the situation can appear in any scenario in which we have a variable that can (doesn't have to at the moment) return a value of more than 1 for the count. Depending on the variable, it can vary greatly in what that means and this can be very dynamic. I think a more dynamic lookup mechanism (e.g., query an enumeration once you have a count and a handle) would be better. I don't think this is only for pvars with a count>1. What if N=1 and M=3 in the previous example? -- Jeff Squyres jsquyres(a)cisco.com For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/

1 0

Re: [mpiwg-tools] Open MPI pvar scheme
by Schulz, Martin 11 Nov '13

11 Nov '13

Hi Jeff, On Nov 11, 2013, at 2:53 PM, "Jeff Squyres (jsquyres)" <jsquyres(a)cisco.com> wrote: > On Nov 11, 2013, at 5:34 PM, Kathryn Mohror <kathryn(a)llnl.gov> wrote: > >>> The answer we're currently using is what we discussed on the tools call: <prefix> is a state variable with enumeration, and all <prefix><name> pvars are expected to use the mapping of that state variable. For exmple, btl_usnic_num_sends will follow the pattern established by the btl_usnic pvar state variable. >> >> I guess the problem with this (and maybe I'm missing something) is that it will be hard for a tool to know if a particular state variable with an enumeration describes other variables that have the prefix in their names. In other words, how do I know that this state variable is special and not just that some pvars happen to have the same prefix, but there's no relationship between them? > > I think what we're trying to say is this: > > - for a given pvar category X > - if there's a state pvar Y (which therefore has an associated enumeration) > - then any pvar in category X that has a name that begins with Y is related to state pvar Y > > This has the restriction that you shouldn't give a state pvar a name that is a prefix of some other pvar in the same category (i.e., so that you don't get incorrect relationship implications). > > Since this convention is limited to a single category, it may not be too onerous. Just to clarify: are you proposing that we add this officially to the standard (that particular naming scheme) or is this supposed to be an informal agreement? I personally would like to see the former, since it will be important to have something tools can rely on. Assuming you meant that (that's how I understood it), here is one concern: this scheme is not very flexible - once a name is set and the enumeration variable is provided, this cannot be changed anymore. Isn't this even too restrictive for your case (where NICs can come and go in, e.g., a hot-pluggable system)? In general, the situation can appear in any scenario in which we have a variable that can (doesn't have to at the moment) return a value of more than 1 for the count. Depending on the variable, it can vary greatly in what that means and this can be very dynamic. I think a more dynamic lookup mechanism (e.g., query an enumeration once you have a count and a handle) would be better. Martin > > -- > Jeff Squyres > jsquyres(a)cisco.com > For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/ > > _______________________________________________ > mpiwg-tools mailing list > mpiwg-tools(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-tools ________________________________________________________________________ Martin Schulz, schulzm(a)llnl.gov, http://people.llnl.gov/schulzm CASC @ Lawrence Livermore National Laboratory, Livermore, USA

1 0

Re: [mpiwg-tools] Open MPI pvar scheme
by Jeff Squyres (jsquyres) 11 Nov '13

11 Nov '13

On Nov 11, 2013, at 5:34 PM, Kathryn Mohror <kathryn(a)llnl.gov> wrote: >> The answer we're currently using is what we discussed on the tools call: <prefix> is a state variable with enumeration, and all <prefix><name> pvars are expected to use the mapping of that state variable. For exmple, btl_usnic_num_sends will follow the pattern established by the btl_usnic pvar state variable. > > I guess the problem with this (and maybe I'm missing something) is that it will be hard for a tool to know if a particular state variable with an enumeration describes other variables that have the prefix in their names. In other words, how do I know that this state variable is special and not just that some pvars happen to have the same prefix, but there's no relationship between them? I think what we're trying to say is this: - for a given pvar category X - if there's a state pvar Y (which therefore has an associated enumeration) - then any pvar in category X that has a name that begins with Y is related to state pvar Y This has the restriction that you shouldn't give a state pvar a name that is a prefix of some other pvar in the same category (i.e., so that you don't get incorrect relationship implications). Since this convention is limited to a single category, it may not be too onerous. -- Jeff Squyres jsquyres(a)cisco.com For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/

1 0

Re: [mpiwg-tools] Open MPI pvar scheme
by Kathryn Mohror 11 Nov '13

11 Nov '13

On Nov 6, 2013, at 7:45 AM, Jeff Squyres (jsquyres) <jsquyres(a)cisco.com> wrote: > We got essentially the same feedback from the OMPI developer community that the tools working group gave: how would a tool find the state pvar to associate the other pvar array slots? > > The answer we're currently using is what we discussed on the tools call: <prefix> is a state variable with enumeration, and all <prefix><name> pvars are expected to use the mapping of that state variable. For exmple, btl_usnic_num_sends will follow the pattern established by the btl_usnic pvar state variable. > > Any other ideas for a convention? I guess the problem with this (and maybe I'm missing something) is that it will be hard for a tool to know if a particular state variable with an enumeration describes other variables that have the prefix in their names. In other words, how do I know that this state variable is special and not just that some pvars happen to have the same prefix, but there's no relationship between them? Maybe we could have another variable type? I know you guys are trying to get a release out the door and it doesn't help that we don't have more concrete ideas on this yet. Everybody: Please do your homework and try to think of more use cases for this so we come up with a good generic solution. Kathryn > > -- > Jeff Squyres > jsquyres(a)cisco.com > For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/ > > _______________________________________________ > mpiwg-tools mailing list > mpiwg-tools(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-tools ______________________________________________________________ Kathryn Mohror, kathryn(a)llnl.gov, http://people.llnl.gov/mohror1 CASC @ Lawrence Livermore National Laboratory, Livermore, CA, USA

1 0

Open MPI pvar scheme
by Jeff Squyres (jsquyres) 06 Nov '13

06 Nov '13

We got essentially the same feedback from the OMPI developer community that the tools working group gave: how would a tool find the state pvar to associate the other pvar array slots? The answer we're currently using is what we discussed on the tools call: <prefix> is a state variable with enumeration, and all <prefix><name> pvars are expected to use the mapping of that state variable. For exmple, btl_usnic_num_sends will follow the pattern established by the btl_usnic pvar state variable. Any other ideas for a convention? -- Jeff Squyres jsquyres(a)cisco.com For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/

1 0