February 2016 - mpi-forum - mailman3.cels.anl.gov

Re: [Mpi-forum] Question about MPI_Info set on communicators
by Schulz Martin 17 Feb '16

17 Feb '16

> >> >See my reply to Jim: if we're supposed to provide the value of *the >> >hint*, that to me sounds like the *user's input* (as opposed to what >>the >> >system chose to do with that hint). >> >> I agree, hints are given by the user to express what an application >>does. >> Those should never be changed by the MPI library (or no are no longer >> (user) hints). > >Even if the MPI implementation should never change the value of these >hints, my original question still stands, because MPI_COMM_GET_INFO's use >of the phrase "actually used by the system" is still ambiguous (MPI-3.1 >p250): > >----- >// Assume that the MPI implementation understands the "foo" info key >hint, but >// is unable to apply it after a communicator is created. >MPI_Info_set(myinfo, "foo", "yes"); >MPI_Comm_set_info(comm, myinfo); >MPI_Comm_get_info(comm, myinfo_returned); >------ > >So I still don't have a definitive answer as to what is in >myinfo_returned: > >1. "foo" = "yes", because that's what the user set, and the MPI >implementation understood the "foo" hint key (regardless of what it chose >to do with that hint) > --> This is what I think should happen > >2. "foo" = "no", because the MPI implementation wasn't able to use the >"foo" hint > --> There's at least 2 explicit votes against this behavior (JeffSq, >Martin) > >3. no "foo" key is set, because the MPI implementation wasn't able to use >the "foo" hint > --> I'm not a big fan of this, because you can't tell the difference >between "foo is not understood by the implementation" and "foo is >understood by the implementation, but this hint can't be used right now". I would say, #3 is correct - the system doesn¹t use/apply the hint, I.e., the behavior is the same as if the hint wasn¹t supplied. I agree, this is not ideal since knowing what an implementation supports would be good, but the standard never makes that distinction. > >========= > >Regardless of what is returned by MPI_COMM_GET_INFO, what happens in >MPI_COMM_DUP? > >The "actually used by the system" ambiguous phrase only applies to >MPI_COMM_GET_INFO. MPI_COMM_DUP is defined in MPI-3.1 p238: > > "MPI_COMM_DUP duplicates the existing communicator comm with ... info >hints." > >It doesn't have any scoping language about *which* hints are copied -- it >pretty much implies that *all* hints are copied. > >As such, I think that "foo"="yes" should be propagated to the new >communicator. Yes, I agree - even if the hint is not used and hence not returned by MPI_COMM_GET_INFO, it¹s still set on the communicator. The user set it and expects it to be set and so it should be part of the duplication. >Which is another reason that I think that MPI_COMM_GET_INFO should also >return "foo"="yes" in myinfo_returned (i.e., so that MPI_COMM_GET_INFO >and MPI_COMM_DUP both behave the same way). I agree, that¹s what it should do - GET and SET should be matching, but that¹s not how the standard is written (sadly). The idea behavior would be three calls, IMHO: MPI_COMM_GET_INFO - return all hints set MPI_COMM_GET_INFO_UNDERSTOOD - return all hints that could be used by an implementation MPI_COMM_GET_INFO_USED - return all hints used (that¹s the current behavior of MPI_COMM_GET_INFO) doing this, though, throws into the usual backwards compatibility problems, though. Martin > >Thoughts? > >-- >Jeff Squyres >jsquyres(a)cisco.com >For corporate legal information go to: >http://www.cisco.com/web/about/doing_business/legal/cri/

1 0

Re: [Mpi-forum] Question about MPI_Info set on communicators
by Jeff Squyres (jsquyres) 17 Feb '16

17 Feb '16

-----Original Message----- From: Schulz Martin <schulzm(a)llnl.gov> Reply: Main MPI Forum mailing list <mpi-forum(a)lists.mpi-forum.org> Date: February 17, 2016 at 5:42:34 PM To: Main MPI Forum mailing list <mpi-forum(a)lists.mpi-forum.org>, Jeff Hammond <jeff.science(a)gmail.com> Subject: Re: [Mpi-forum] Question about MPI_Info set on communicators > >See my reply to Jim: if we're supposed to provide the value of *the > >hint*, that to me sounds like the *user's input* (as opposed to what the > >system chose to do with that hint). > > I agree, hints are given by the user to express what an application does. > Those should never be changed by the MPI library (or no are no longer > (user) hints). Even if the MPI implementation should never change the value of these hints, my original question still stands, because MPI_COMM_GET_INFO's use of the phrase "actually used by the system" is still ambiguous (MPI-3.1 p250): ----- // Assume that the MPI implementation understands the "foo" info key hint, but // is unable to apply it after a communicator is created. MPI_Info_set(myinfo, "foo", "yes"); MPI_Comm_set_info(comm, myinfo); MPI_Comm_get_info(comm, myinfo_returned); ------ So I still don't have a definitive answer as to what is in myinfo_returned: 1. "foo" = "yes", because that's what the user set, and the MPI implementation understood the "foo" hint key (regardless of what it chose to do with that hint) --> This is what I think should happen 2. "foo" = "no", because the MPI implementation wasn't able to use the "foo" hint --> There's at least 2 explicit votes against this behavior (JeffSq, Martin) 3. no "foo" key is set, because the MPI implementation wasn't able to use the "foo" hint --> I'm not a big fan of this, because you can't tell the difference between "foo is not understood by the implementation" and "foo is understood by the implementation, but this hint can't be used right now". ========= Regardless of what is returned by MPI_COMM_GET_INFO, what happens in MPI_COMM_DUP? The "actually used by the system" ambiguous phrase only applies to MPI_COMM_GET_INFO. MPI_COMM_DUP is defined in MPI-3.1 p238: "MPI_COMM_DUP duplicates the existing communicator comm with ... info hints." It doesn't have any scoping language about *which* hints are copied -- it pretty much implies that *all* hints are copied. As such, I think that "foo"="yes" should be propagated to the new communicator. Which is another reason that I think that MPI_COMM_GET_INFO should also return "foo"="yes" in myinfo_returned (i.e., so that MPI_COMM_GET_INFO and MPI_COMM_DUP both behave the same way). Thoughts? -- Jeff Squyres jsquyres(a)cisco.com For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/

1 0

Re: [Mpi-forum] Question about MPI_Info set on communicators
by Schulz Martin 17 Feb '16

17 Feb '16

Another case of email eaten by the server (and I was wondering why you ignored me :) - old email attached below - new comments inline. Martin ________________________________________________________________________ Martin Schulz, schulzm(a)llnl.gov, http://scalability.llnl.gov/ CASC @ Lawrence Livermore National Laboratory, Livermore, USA On 2/16/16, 12:49 PM, "mpi-forum on behalf of Jeff Squyres (jsquyres)" <mpi-forum-bounces(a)lists.mpi-forum.org on behalf of jsquyres(a)cisco.com> wrote: >-----Original Message----- >From: Jeff Hammond <jeff.science(a)gmail.com> >Reply: Main MPI Forum mailing list <mpi-forum(a)lists.mpi-forum.org> >Date: February 16, 2016 at 2:37:57 PM >To: Main MPI Forum mailing list <mpi-forum(a)lists.mpi-forum.org> >Subject: Re: [Mpi-forum] Question about MPI_Info set on communicators > >> > - "yes" because the user asked for it (i.e., that's the hint, >>regardless >> > of what the implementation dos with it. >> >> This answer contradicts the most reasonable interpretation of "hints >> actually used by the system" > >See my reply to Jim: if we're supposed to provide the value of *the >hint*, that to me sounds like the *user's input* (as opposed to what the >system chose to do with that hint). I agree, hints are given by the user to express what an application does. Those should never be changed by the MPI library (or no are no longer (user) hints). > >> > - "no" because the implementation could not provide it >> > - no "foo" key at all because the implementation did not use the hint >> >> I think these are equivalent from a user perspective. In both cases, the >> user is not getting the "magical_powers"="yes" they requested. >> >> However, my reading of MPI 3.1 6.4.4 p250 says that the keys should be >> defined, but set to "no". > >Ok. I don¹t read this into the text - it just says that any hint used by the system is returned, those are not used are not. In this case, I would argue that the MPI library does not use the hint and hence it should not be returned. In any case, an MPI library can¹t change a hint - it is not changing what the application wants to tell the library. >> In any case, we need to improve the standard text unless Bill swoops in >>to >> tell us we are all idiots and points out text that makes the >>interpretation >> unambiguous :-) > >I agree it needs to be clarified. It would be nice if we could clarify >it in a way that is compatible with the 3.0 and 3.1 text (so that we >don't have to have apps distinguish between 3.0/3.1 behavior and >beyond-3.1 behavior). I think it¹s a bit strange (I.e., get and set info are not pairs), but it's expressed in the text. I don¹t think we say anywhere in the text (at least I couldn¹t find it) that MPI is supposed to change user defined hints. Back to the old example (that¹s the missing old email): I don¹t think that these two sequences should result in the same behavior. >> MPI_Comm_get_info(comm, myinfo_returned); >> MPI_Comm_dup_with_info(comm, &newcomm, myinfo_returned); >> >> and >> MPI_Comm_dup(comm, &newcomm); MPI_Comm_get_info states that a new info object is returned that only contains the hints that are actually used by the communicator. Which hints given by the user are used is, I assume, up to the implementation, but the set can be smaller and not the same with the hints given by the user. When I duplicate a communicator, as a user, I would expect all hints that I gave to be copied as well (I.e., the ones that are set on the communicator), not just the used ones. Back to Jeff¹s example I think the critical part is actually the use of MPI_Comm_get_info: MPI_Info_set(myinfo, "use_shared_memory", "yes"); MPI_Comm_set_info(comm, myinfo); MPI_Comm_get_info(comm, myinfo_returned); // Assume here that there was not enough shared memory available to use shared-memory for // communication on this comm, so MPI_Info_get() tells us that use_shared_memory is "no². This should not return ³no² for ³use_shared_memory² (MPI should never change a user defined hint why would this be the case, they are user set properties), instead ³use_shared_memory² should simply not be present in the returned info object. Nevertheless, the hint is still set on the communicator (why would I, as a user, expect that MPI removes a hint) and so, if you dup the communicator, it should get duplicated as well. If the hint now becomes active (as at the end in Jeff¹s example) MPI_Comm_get_info should include it in the returned info object. > >-- >Jeff Squyres >jsquyres(a)cisco.com >For corporate legal information go to: >http://www.cisco.com/web/about/doing_business/legal/cri/ >_______________________________________________ >mpi-forum mailing list >mpi-forum(a)lists.mpi-forum.org >http://secure-web.cisco.com/1tOmvKI1yyCS_mGO20j9isCL5ySWKbNPro0WSEvMJRHaTU >23ypla94_M6gQIGroi6BcOFxLpS5rhu-eI1U3bJl7F0yFSBGQ8R-wRofvFw3aTFAMW6lejH1LL >jDWR4K73IRuA0jxBGhkwe67EmwghcKKbntizOt3WrysjB2jsUI0Lnp2HnXqDgTu_j9iq8c8tkv >GIM0_IOvsjtTB2bz6tqXNFEy_DEJdGsJiBT253R_5pyyEEQCdvy6ThzPFGrpe_pYyRF19iRSZu >OiDtMUpOMBsjThStyjgJSXv-4-rFlr-Z1vEbddhAf9is8Vf9PDIjqQqUkOigEAMSAlMtzUSbeQ >40HMdowaOKVuSOQ9kiQMjI/http%3A%2F%2Flists.mpi-forum.org%2Fmailman%2Flistin >fo.cgi%2Fmpi-forum

1 0

MPI Forum Next Week
by Schulz Martin 17 Feb '16

17 Feb '16

Hi all, First of all, sorry for the sparse communication. I just realized (with Jeff¹s help) yesterday that the MPI forum email server has been quietly deleting my emails about the forum, incl. agenda items, etc. - since I did get quite some input, though, I figured they went through so, thanks for self organizing :). The agenda (which is pretty much final at this point) is on the website please take a look to check if nothing is missing. Also, note that we have quite a full plenary program and as a consequence we may run a little longer on Wednesday as normal, but I want to make sure we get through the program. Also, the registration link is on the logistics page - please don¹t forget to register. Thanks and see you the week after next! Martin ________________________________________________________________________ Martin Schulz, schulzm(a)llnl.gov, http://scalability.llnl.gov/ CASC @ Lawrence Livermore National Laboratory, Livermore, USA

1 0

Re: [Mpi-forum] Question about MPI_Info set on communicators
by Jeff Squyres (jsquyres) 16 Feb '16

16 Feb '16

-----Original Message----- From: Jeff Hammond <jeff.science(a)gmail.com> Reply: Main MPI Forum mailing list <mpi-forum(a)lists.mpi-forum.org> Date: February 16, 2016 at 2:37:57 PM To: Main MPI Forum mailing list <mpi-forum(a)lists.mpi-forum.org> Subject: Re: [Mpi-forum] Question about MPI_Info set on communicators > > - "yes" because the user asked for it (i.e., that's the hint, regardless > > of what the implementation dos with it. > > This answer contradicts the most reasonable interpretation of "hints > actually used by the system" See my reply to Jim: if we're supposed to provide the value of *the hint*, that to me sounds like the *user's input* (as opposed to what the system chose to do with that hint). > > - "no" because the implementation could not provide it > > - no "foo" key at all because the implementation did not use the hint > > I think these are equivalent from a user perspective. In both cases, the > user is not getting the "magical_powers"="yes" they requested. > > However, my reading of MPI 3.1 6.4.4 p250 says that the keys should be > defined, but set to "no". Ok. > In any case, we need to improve the standard text unless Bill swoops in to > tell us we are all idiots and points out text that makes the interpretation > unambiguous :-) I agree it needs to be clarified. It would be nice if we could clarify it in a way that is compatible with the 3.0 and 3.1 text (so that we don't have to have apps distinguish between 3.0/3.1 behavior and beyond-3.1 behavior). -- Jeff Squyres jsquyres(a)cisco.com For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/

1 0

Re: [Mpi-forum] Question about MPI_Info set on communicators
by Jeff Squyres (jsquyres) 16 Feb '16

16 Feb '16

I'll reply more to jeff Gannon when I'm not on a phone - can't reply inline easily... Jim: yes, it says the current value - but of what? The hint? Because if it's the current value of *the hint*, I'd argue that that's the input from the user. Further, I'd claim that the implementation cannot change the value of the hint - it can only choose what to do with a given hint. Specifically: *What the implementation chose to do with a hint* is a different value (i.e., an output value, which may be more suitable to MPI-t). Sent from my phone. No type good. On Feb 16, 2016, at 3:05 PM, Jim Dinan <james.dinan(a)gmail.com<mailto:[email protected]>> wrote: The text says that the "current setting" is returned, not the setting requested in a previous operation that set info on the communicator. It is not specified whether #2 or #3 should occur, but as long as "no" is the default value of the info key in question, the value of the setting is still unambiguously (if inconveniently) conveyed to the user. ~Jim. On Tue, Feb 16, 2016 at 9:50 AM, Jeff Squyres (jsquyres) <jsquyres(a)cisco.com<mailto:[email protected]>> wrote: Jim -- Based on your answer, I'm now a bit confused. I now need to back up from my original question and clarify what MPI_COMM_INFO_GET is supposed to return in this scenario: ----- MPI_Info_set(myinfo, "use_shared_memory", "yes"); MPI_Comm_set_info(comm, my info); // MPI implementation recognizes the "use_shared_memory" hint, but is // unable to use shared memory with this communicator MPI_Comm_get_info(comm, myinfo_returned); ----- What value is in myinfo_returned for use_shared_memory? MPI 3.1 6.4.4 p250 says: "MPI_COMM_GET_INFO returns a new info object containing the hints of the communicator associated with comm. The current setting of all hints actually used by the system related to this communicator is returned in info_used." "...all hints actually used by the system..." is actually a bit ambiguous. In this case, "use_shared_memory" is *recognized* by the MPI implementation. But it wasn't able to be *used* (e.g., because no shared memory was available). So is the value in info_used supposed to be: 1. The user's requested hint (i.e., "yes"), or 2. What MPI is actually using (i.e., "no"), or 3. Since this hint wasn't "used" by the system, then the "use_shared_memory" key shouldn't be returned in info_used ? Indeed, one could make the argument that MPI_COMM_GET_INFO is supposed to return exactly the value(s) that the user requested (if it is *recognized* by the implementation), and MPI_T pvars and/or cvars should be queried to obtain the values indicating what the implementation chose to do with those hints. Thoughts? On February 15, 2016 at 8:25:44 PM, Jim Dinan (james.dinan(a)gmail.com<mailto:[email protected]>) wrote: > "Same" should mean that these call sequences produce the same newcomm, > right? > > MPI_Comm_get_info(comm, myinfo_returned); > MPI_Comm_dup_with_info(comm, &newcomm, myinfo_returned); > > and > > MPI_Comm_dup(comm, &newcomm); > > I think this is an argument for "no". > > I'm not sure if the question is asking whether the MPI implementation can > change communicator info values dynamically. I'm not sure the spec > actually addresses that possibility. If this is allowed, newcomm would > still only get "yes" if the implementation had also toggled the info key on > the parent communicator. > > ~Jim. > > On Mon, Feb 15, 2016 at 5:34 PM, Jeff Squyres (jsquyres) > > wrote: > > > This question has come up in the Open MPI community: > > > > ----- > > MPI_Info_set(myinfo, "use_shared_memory", "yes"); > > MPI_Comm_set_info(comm, myinfo); > > MPI_Comm_get_info(comm, myinfo_returned); > > // Assume here that there was not enough shared memory available to use > > shared-memory for > > // communication on this comm, so MPI_Info_get() tells us that > > use_shared_memory is "no". > > > > // ...time passes, and now assume there is more shared-memory available > > MPI_Comm_dup_with_info(comm, &newcom); > > MPI_comm_get_info(newcom, myinfo_returned); > > ----- > > > > Should newcom have "use_shared_memory" set to "yes" or "no"? > > > > Arguments for "yes": > > - the user originally asked for "yes" > > - now there is shared memory available > > > > Arguments for "no": > > - the current value for "used_shared_memory" on the parent communicator is > > "no" > > > > Which should it be? > > > > MPI-3.1 6.4.2 p238 says: > > > > "...Returns in newcomer a new communicator with the same group or groups, > > same topology, same info hints, any copied cached information, but a new > > context (see Section 6.7.1)." > > > > So the specific question here is: what exactly does "Same info hints" > > mean? The user-specific hint, or the current value of the hint? > > > > -- > > Jeff Squyres > > jsquyres(a)cisco.com<mailto:[email protected]> > > For corporate legal information go to: > > http://www.cisco.com/web/about/doing_business/legal/cri/ > > _______________________________________________ > > mpi-forum mailing list > > mpi-forum(a)lists.mpi-forum.org<mailto:[email protected]> > > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum > _______________________________________________ > mpi-forum mailing list > mpi-forum(a)lists.mpi-forum.org<mailto:[email protected]> > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum -- Jeff Squyres jsquyres(a)cisco.com<mailto:[email protected]> For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/

1 0

Re: [Mpi-forum] Question about MPI_Info set on communicators
by Jim Dinan 16 Feb '16

16 Feb '16

The text says that the "current setting" is returned, not the setting requested in a previous operation that set info on the communicator. It is not specified whether #2 or #3 should occur, but as long as "no" is the default value of the info key in question, the value of the setting is still unambiguously (if inconveniently) conveyed to the user. ~Jim. On Tue, Feb 16, 2016 at 9:50 AM, Jeff Squyres (jsquyres) <jsquyres(a)cisco.com > wrote: > Jim -- > > Based on your answer, I'm now a bit confused. I now need to back up from > my original question and clarify what MPI_COMM_INFO_GET is supposed to > return in this scenario: > > ----- > MPI_Info_set(myinfo, "use_shared_memory", "yes"); > MPI_Comm_set_info(comm, my info); > // MPI implementation recognizes the "use_shared_memory" hint, but is > // unable to use shared memory with this communicator > MPI_Comm_get_info(comm, myinfo_returned); > ----- > > What value is in myinfo_returned for use_shared_memory? > > MPI 3.1 6.4.4 p250 says: > > "MPI_COMM_GET_INFO returns a new info object containing the hints of the > communicator associated with comm. The current setting of all hints > actually used by the system related to this communicator is returned in > info_used." > > "...all hints actually used by the system..." is actually a bit ambiguous. > > In this case, "use_shared_memory" is *recognized* by the MPI > implementation. But it wasn't able to be *used* (e.g., because no shared > memory was available). So is the value in info_used supposed to be: > > 1. The user's requested hint (i.e., "yes"), or > 2. What MPI is actually using (i.e., "no"), or > 3. Since this hint wasn't "used" by the system, then the > "use_shared_memory" key shouldn't be returned in info_used > > ? > > Indeed, one could make the argument that MPI_COMM_GET_INFO is supposed to > return exactly the value(s) that the user requested (if it is *recognized* > by the implementation), and MPI_T pvars and/or cvars should be queried to > obtain the values indicating what the implementation chose to do with those > hints. > > Thoughts? > > > > On February 15, 2016 at 8:25:44 PM, Jim Dinan (james.dinan(a)gmail.com) > wrote: > > "Same" should mean that these call sequences produce the same newcomm, > > right? > > > > MPI_Comm_get_info(comm, myinfo_returned); > > MPI_Comm_dup_with_info(comm, &newcomm, myinfo_returned); > > > > and > > > > MPI_Comm_dup(comm, &newcomm); > > > > I think this is an argument for "no". > > > > I'm not sure if the question is asking whether the MPI implementation can > > change communicator info values dynamically. I'm not sure the spec > > actually addresses that possibility. If this is allowed, newcomm would > > still only get "yes" if the implementation had also toggled the info key > on > > the parent communicator. > > > > ~Jim. > > > > On Mon, Feb 15, 2016 at 5:34 PM, Jeff Squyres (jsquyres) > > wrote: > > > > > This question has come up in the Open MPI community: > > > > > > ----- > > > MPI_Info_set(myinfo, "use_shared_memory", "yes"); > > > MPI_Comm_set_info(comm, myinfo); > > > MPI_Comm_get_info(comm, myinfo_returned); > > > // Assume here that there was not enough shared memory available to use > > > shared-memory for > > > // communication on this comm, so MPI_Info_get() tells us that > > > use_shared_memory is "no". > > > > > > // ...time passes, and now assume there is more shared-memory available > > > MPI_Comm_dup_with_info(comm, &newcom); > > > MPI_comm_get_info(newcom, myinfo_returned); > > > ----- > > > > > > Should newcom have "use_shared_memory" set to "yes" or "no"? > > > > > > Arguments for "yes": > > > - the user originally asked for "yes" > > > - now there is shared memory available > > > > > > Arguments for "no": > > > - the current value for "used_shared_memory" on the parent > communicator is > > > "no" > > > > > > Which should it be? > > > > > > MPI-3.1 6.4.2 p238 says: > > > > > > "...Returns in newcomer a new communicator with the same group or > groups, > > > same topology, same info hints, any copied cached information, but a > new > > > context (see Section 6.7.1)." > > > > > > So the specific question here is: what exactly does "Same info hints" > > > mean? The user-specific hint, or the current value of the hint? > > > > > > -- > > > Jeff Squyres > > > jsquyres(a)cisco.com > > > For corporate legal information go to: > > > http://www.cisco.com/web/about/doing_business/legal/cri/ > > > _______________________________________________ > > > mpi-forum mailing list > > > mpi-forum(a)lists.mpi-forum.org > > > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum > > _______________________________________________ > > mpi-forum mailing list > > mpi-forum(a)lists.mpi-forum.org > > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum > > -- > Jeff Squyres > jsquyres(a)cisco.com > For corporate legal information go to: > http://www.cisco.com/web/about/doing_business/legal/cri/ >

1 0

Re: [Mpi-forum] Question about MPI_Info set on communicators
by Jeff Hammond 16 Feb '16

16 Feb '16

On Tue, Feb 16, 2016 at 11:30 AM, Jeff Squyres (jsquyres) < jsquyres(a)cisco.com> wrote: > Forget the use-shared-memory hint; it was just an example. Substitute in > any "foo" info key hint name(assuming the implementation understands the > "foo" key hint name); the original question remains: what should the value > be if the implementation cannot provide the requested hint? > > If the implementation can _never_ provide that hint, then it should be set to "no" and your question about what happens later is not interesting, because the result there will be "no" whether the implementation remembers that you tried but failed to set it to "yes" or not, because the answer is still "no" and always will be. > - "yes" because the user asked for it (i.e., that's the hint, regardless > of what the implementation dos with it) > This answer contradicts the most reasonable interpretation of "hints actually used by the system" > - "no" because the implementation could not provide it > - no "foo" key at all because the implementation did not use the hint > I think these are equivalent from a user perspective. In both cases, the user is not getting the "magical_powers"="yes" they requested. However, my reading of MPI 3.1 6.4.4 p250 says that the keys should be defined, but set to "no". In any case, we need to improve the standard text unless Bill swoops in to tell us we are all idiots and points out text that makes the interpretation unambiguous :-) Jeff > > Sent from my phone. No type good. > > On Feb 16, 2016, at 2:16 PM, Jeff Hammond <jeff.science(a)gmail.com> wrote: > > > > On Tue, Feb 16, 2016 at 6:50 AM, Jeff Squyres (jsquyres) < > jsquyres(a)cisco.com> wrote: > >> Jim -- >> >> Based on your answer, I'm now a bit confused. I now need to back up from >> my original question and clarify what MPI_COMM_INFO_GET is supposed to >> return in this scenario: >> >> ----- >> MPI_Info_set(myinfo, "use_shared_memory", "yes"); >> MPI_Comm_set_info(comm, my info); >> // MPI implementation recognizes the "use_shared_memory" hint, but is >> // unable to use shared memory with this communicator >> MPI_Comm_get_info(comm, myinfo_returned); >> ----- >> >> What value is in myinfo_returned for use_shared_memory? >> >> MPI 3.1 6.4.4 p250 says: >> >> "MPI_COMM_GET_INFO returns a new info object containing the hints of the >> communicator associated with comm. The current setting of all hints >> actually used by the system related to this communicator is returned in >> info_used." >> >> "...all hints actually used by the system..." is actually a bit ambiguous. >> >> In this case, "use_shared_memory" is *recognized* by the MPI >> implementation. But it wasn't able to be *used* (e.g., because no shared >> memory was available). So is the value in info_used supposed to be: >> >> > How does it know how much shared memory is required? I don't see any > reason for MPI_Comm_set_info to fail on account of lack of shared memory. > If zero bytes of shared memory are available, how does MPI_Comm_set_info > know that this is not enough when the user actually needs? > > >> 1. The user's requested hint (i.e., "yes"), or >> > > I'm changing my answer to this one, in this case. For MPI_Alloc_mem and > MPI_Win_allocate(_shared), my answer will be different. > > The only reasonable behavior is for MPI_Comm_set_info to succeed and set > "use_shared_memory"="yes". Subsequent operations may behave differently > depending on how much shared memory is available. I would expect the user > to probe that using MPI_T. > > Jeff > > >> 2. What MPI is actually using (i.e., "no"), or >> 3. Since this hint wasn't "used" by the system, then the >> "use_shared_memory" key shouldn't be returned in info_used >> >> ? >> >> Indeed, one could make the argument that MPI_COMM_GET_INFO is supposed to >> return exactly the value(s) that the user requested (if it is *recognized* >> by the implementation), and MPI_T pvars and/or cvars should be queried to >> obtain the values indicating what the implementation chose to do with those >> hints. >> >> Thoughts? >> >> >> >> On February 15, 2016 at 8:25:44 PM, Jim Dinan (james.dinan(a)gmail.com) >> wrote: >> > "Same" should mean that these call sequences produce the same newcomm, >> > right? >> > >> > MPI_Comm_get_info(comm, myinfo_returned); >> > MPI_Comm_dup_with_info(comm, &newcomm, myinfo_returned); >> > >> > and >> > >> > MPI_Comm_dup(comm, &newcomm); >> > >> > I think this is an argument for "no". >> > >> > I'm not sure if the question is asking whether the MPI implementation >> can >> > change communicator info values dynamically. I'm not sure the spec >> > actually addresses that possibility. If this is allowed, newcomm would >> > still only get "yes" if the implementation had also toggled the info >> key on >> > the parent communicator. >> > >> > ~Jim. >> > >> > On Mon, Feb 15, 2016 at 5:34 PM, Jeff Squyres (jsquyres) > > wrote: >> > >> > > This question has come up in the Open MPI community: >> > > >> > > ----- >> > > MPI_Info_set(myinfo, "use_shared_memory", "yes"); >> > > MPI_Comm_set_info(comm, myinfo); >> > > MPI_Comm_get_info(comm, myinfo_returned); >> > > // Assume here that there was not enough shared memory available to >> use >> > > shared-memory for >> > > // communication on this comm, so MPI_Info_get() tells us that >> > > use_shared_memory is "no". >> > > >> > > // ...time passes, and now assume there is more shared-memory >> available >> > > MPI_Comm_dup_with_info(comm, &newcom); >> > > MPI_comm_get_info(newcom, myinfo_returned); >> > > ----- >> > > >> > > Should newcom have "use_shared_memory" set to "yes" or "no"? >> > > >> > > Arguments for "yes": >> > > - the user originally asked for "yes" >> > > - now there is shared memory available >> > > >> > > Arguments for "no": >> > > - the current value for "used_shared_memory" on the parent >> communicator is >> > > "no" >> > > >> > > Which should it be? >> > > >> > > MPI-3.1 6.4.2 p238 says: >> > > >> > > "...Returns in newcomer a new communicator with the same group or >> groups, >> > > same topology, same info hints, any copied cached information, but a >> new >> > > context (see Section 6.7.1)." >> > > >> > > So the specific question here is: what exactly does "Same info hints" >> > > mean? The user-specific hint, or the current value of the hint? >> > > >> > > -- >> > > Jeff Squyres >> > > jsquyres(a)cisco.com >> > > For corporate legal information go to: >> > > http://www.cisco.com/web/about/doing_business/legal/cri/ >> > > _______________________________________________ >> > > mpi-forum mailing list >> > > mpi-forum(a)lists.mpi-forum.org >> > > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum >> > _______________________________________________ >> > mpi-forum mailing list >> > mpi-forum(a)lists.mpi-forum.org >> > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum >> >> -- >> Jeff Squyres >> jsquyres(a)cisco.com >> For corporate legal information go to: >> http://www.cisco.com/web/about/doing_business/legal/cri/ >> _______________________________________________ >> mpi-forum mailing list >> mpi-forum(a)lists.mpi-forum.org >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum >> > > > > -- > Jeff Hammond > jeff.science(a)gmail.com > http://jeffhammond.github.io/ > > Received: from xch-rtp-019.cisco.com (64.101.220.159) by > xch-rcd-019.cisco.com > (173.37.102.29) with Microsoft SMTP Server (TLS) id 15.0.1104.5 via Mailbox > Transport; Tue, 16 Feb 2016 13:16:12 -0600 > Received: from xch-rtp-004.cisco.com (64.101.220.144) by > XCH-RTP-019.cisco.com <http://xch-rtp-019.cisco.com> > (64.101.220.159) with Microsoft SMTP Server (TLS) id 15.0.1104.5; Tue, 16 > Feb > 2016 14:16:11 -0500 > Received: from rcdn-iport-1.cisco.com (173.37.86.72) by mail.cisco.com > (64.101.220.144) with Microsoft SMTP Server (TLS) id 15.0.1104.5 via > Frontend > Transport; Tue, 16 Feb 2016 14:16:11 -0500 > Received: from rcdn-core-5.cisco.com ([173.37.93.156]) > by rcdn-iport-1.cisco.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 16 > Feb 2016 19:16:09 +0000 > Received: from alln-inbound-e.cisco.com (alln-inbound-e.cisco.com > [173.37.147.235]) > by rcdn-core-5.cisco.com (8.14.5/8.14.5) with ESMTP id u1GJG9s3016172; > Tue, 16 Feb 2016 19:16:10 GMT > Received-SPF: None (alln-inbound-e.cisco.com: no sender > authenticity information available from domain of > mpi-forum-bounces(a)lists.mpi-forum.org) identity=mailfrom; > client-ip=129.79.39.208; receiver=alln-inbound-e.cisco.com; > envelope-from="mpi-forum-bounces(a)lists.mpi-forum.org"; > x-sender="mpi-forum-bounces(a)lists.mpi-forum.org"; > x-conformance=spf_only > Received-SPF: None (alln-inbound-e.cisco.com: no sender > authenticity information available from domain of > postmaster(a)lion.crest.iu.edu) identity=helo; > client-ip=129.79.39.208; receiver=alln-inbound-e.cisco.com; > envelope-from="mpi-forum-bounces(a)lists.mpi-forum.org"; > x-sender="postmaster(a)lion.crest.iu.edu"; > x-conformance=spf_only > Authentication-Results: alln-inbound-e.cisco.com; spf=None > smtp.mailfrom=mpi-forum-bounces(a)lists.mpi-forum.org; spf=None > smtp.helo=postmaster(a)lion.crest.iu.edu; dkim=hardfail (body hash did not > verify [final]) header.i=(a)gmail.com; dkim=hardfail (body hash did not > verify [final]) header.i=(a)gmail.com; dmarc=fail (p=none dis=none) d= > gmail.com > X-from-outside-Cisco: 129.79.39.208 > IronPort-PHdr: > =?us-ascii?q?9a23=3A3koMHBVMN/yf5qkGiy0ZpEKb+w/V8LGtZVwlr6E/?= > > =?us-ascii?q?grcLSJyIuqrYYRSBt8tkgFKBZ4jH8fUM07OQ6PC/HzBbqs/Y4TgrS99laVwssY?= > > =?us-ascii?q?0uhQsuAcqIWwXQDcXBSGgEJvlET0Jv5HqhMEJYS47UblzWpWCuv3ZJQk2sfTR8?= > > =?us-ascii?q?Kum9IIPOlcP/j7n0oM2MJVkTz2PkMPtbF1afk0b4joEum4xsK6I8mFPig0BjXK?= > > =?us-ascii?q?Bo/15uPk+ZhB3m5829r9ZJ+iVUvO89pYYbCf2pN4xxd7FTDSwnPmYp/4Wr8ECb?= > > =?us-ascii?q?FUrcrkYaSXgcxxpUHxCXq1b+X4ztqW3/rus41i6AIMiwSqo7HjGr7qNuQRmvjz?= > > =?us-ascii?q?8bNjk/6yfchc03ibpa5RymvQF6kLPSe5yfYf93f6fBeoEfXWsEUMFLSikEGI6l?= > > =?us-ascii?q?coYUE8IFPP1EtM/4v1pLoxykGAzqA/ngnSRVjH371rFvzuI6DAvd1xYhFd9djH?= > > =?us-ascii?q?OBidz0Mu83acH94q7S0DCLavpX3Tb758DMcws96aWJRbM1f83L1EwHEwLekk7W?= > > =?us-ascii?q?rJTkej2SzPkI9W+B4Lwzb++3j38brFRUqzmpjuIUq8HngZgIzxjB/CR9wYA6bd?= > > =?us-ascii?q?m5U1IzNdu8Ed5esDuHMqNyQ9g+WCdtoCd/xLocpJv9ezIFnscJ3RnaPt6KfYbA?= > > =?us-ascii?q?whvmUeGXLX8shn9rdvSkjBe/90Ot4uL5X8iwll1Nq3wWwZH3qnkR2kmLuYC8Qf?= > =?us-ascii?q?xn8xLkgG7X2g=3D=3D?= > X-IronPort-Anti-Spam-Filtered: true > X-IronPort-Anti-Spam-Result: > =?us-ascii?q?A0GWAACSdMNWmdAnT4FeBhMBAQEBDwEBA?= > > =?us-ascii?q?QEGAQEBAYNSRyYGhDWkGIFJI4UhilQBDYEmGyIbAQmHJD8UAQEBAQEBAQECDgE?= > > =?us-ascii?q?BAQEBCAsJCSEugi0JATgGBAEBBAEBAQEBAQEBASMBAQEBAQEBAQEBAQEBAQEBA?= > > =?us-ascii?q?QEBAQEBAQEBAQEBAQEBAQEBAQYCDR4PFgEaAQEBAQIBAQEBDxEdAQUKDBIMAgE?= > > =?us-ascii?q?BAQECBgEBBQULDRUNBAQCAgMBGgMBAQUECAEFARwGEwUPDodiAQMKCA6eJIExP?= > > =?us-ascii?q?jGLNIFpgleBYINOChknDVGECAEBAQEBAQEDAQEBAQEVAQUKBIUQhSiCN4I6gkG?= > > =?us-ascii?q?BOgWOHIhCIYENhEaFH3SBc4ImjE1EhjuGBREePwNaDwEBgkEcgWlJAYhbAQEB?= > X-IPAS-Result: > =?us-ascii?q?A0GWAACSdMNWmdAnT4FeBhMBAQEBDwEBAQEGAQEBAYNSRyY?= > > =?us-ascii?q?GhDWkGIFJI4UhilQBDYEmGyIbAQmHJD8UAQEBAQEBAQECDgEBAQEBCAsJCSEug?= > > =?us-ascii?q?i0JATgGBAEBBAEBAQEBAQEBASMBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQE?= > > =?us-ascii?q?BAQEBAQEBAQYCDR4PFgEaAQEBAQIBAQEBDxEdAQUKDBIMAgEBAQECBgEBBQULD?= > > =?us-ascii?q?RUNBAQCAgMBGgMBAQUECAEFARwGEwUPDodiAQMKCA6eJIExPjGLNIFpgleBYIN?= > > =?us-ascii?q?OChknDVGECAEBAQEBAQEDAQEBAQEVAQUKBIUQhSiCN4I6gkGBOgWOHIhCIYENh?= > =?us-ascii?q?EaFH3SBc4ImjE1EhjuGBREePwNaDwEBgkEcgWlJAYhbAQEB?= > X-IronPort-AV: E=Sophos;i="5.22,456,1449532800"; > d="scan'208,217";a="194686767" > X-Amp-Result: Clean > X-Amp-File-Uploaded: False > X-IronPort-Outbreak-Status: No, level 0, Unknown - Unknown > Received: from crest.iu.edu (HELO lion.crest.iu.edu) ([129.79.39.208]) > by alln-inbound-e.cisco.com with ESMTP; 16 Feb 2016 19:16:08 +0000 > Received: by lion.crest.iu.edu (Postfix, from userid 493) > id 8E3BD260779; Tue, 16 Feb 2016 14:16:08 -0500 (EST) > X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on > lion.crest.iu.edu > X-Spam-Level: > X-Spam-Status: No, score=-2.5 required=5.0 > tests=BAYES_00,DKIM_ADSP_CUSTOM_MED, > DKIM_SIGNED,FREEMAIL_FROM,HTML_MESSAGE,RCVD_IN_DNSWL_LOW,T_DKIM_INVALID > autolearn=unavailable version=3.3.1 > Received: from crest.iu.edu (localhost [127.0.0.1]) > by lion.crest.iu.edu (Postfix) with ESMTP id 220D12608A4; > Tue, 16 Feb 2016 14:16:06 -0500 (EST) > X-Original-To: mpi-forum(a)lists.mpi-forum.org > Delivered-To: mpi-forum(a)lists.mpi-forum.org > Received: by lion.crest.iu.edu (Postfix, from userid 493) > id C3B4326079D; Tue, 16 Feb 2016 14:16:04 -0500 (EST) > Received: from mail-ob0-f177.google.com (mail-ob0-f177.google.com > [209.85.214.177]) > by lion.crest.iu.edu (Postfix) with ESMTP id 0CF2D260779 > for <mpi-forum(a)lists.mpi-forum.org>; > Tue, 16 Feb 2016 14:16:04 -0500 (EST) > Received: by mail-ob0-f177.google.com with SMTP id gc3so172260092obb.3 > for <mpi-forum(a)lists.mpi-forum.org>; > Tue, 16 Feb 2016 11:16:04 -0800 (PST) > DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; > s=20120113; > h=mime-version:in-reply-to:references:from:date:message-id:subject:to > :cc:content-type; > bh=KEQt98rYsl7/LCgT0QeppyTHXsG6OQ4HlXJPO5cSx6g=; > b=ZjDP3X63meN3mEnca6KBi6EBk9pci2/5zBnyPW9jXQQoUqv8q6f6xnTqfe6kAJIANL > BrdVhlIVnOzqmSuQluiCJ+wif1RFZsh9I1q2Rc3JmktVNu9mlkj8GABpnIV76JsN3+Nh > TE4bBFG6d/yEpONEhBmHYb0M9CPNfGwjyHcU/iwIIdp/NJ2aCFQd2DT3iTjubsfcQ9YC > NtZrfSHLBJ7W7uXA4xh502P+h5n3Zw4cK5T3jvnX2dS8Ya7q2RgvaB9hZ0O573Z8KPGX > NckpbsAI93Z+2WOYKjYD+RahNtTisu/3pg/YUYPp32fT3ClY6+y/lCvz+v3MAbEr5QwM > i3hw== > X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; > d=1e100.net; s=20130820; > h=x-gm-message-state:mime-version:in-reply-to:references:from:date > :message-id:subject:to:cc:content-type; > bh=KEQt98rYsl7/LCgT0QeppyTHXsG6OQ4HlXJPO5cSx6g=; > b=f0LWIMla+RNtd30P6me/Due6bIOkINSUFd72cgxq4quY2u5vwQ2vGAidpQmR6GxaVX > Va0CxATjTEdQthxE645JcglMkoa3Qr3/NyZSn3NdxXcGBNSt4O21JWy2JMhyrRq/ZOdU > ZUUWR3E/0sndIX6N2CVn8TmcDe+V6oK5seuP1/s0DBArBabm7UvX8i4WpZ87Rbro8ZZz > bGaz6H5r2BNGfvqTs/dJpFgngDtZmQhfs9bBQqTAZnZF2BT0Hr5mYXslbpVpeeFP10ZB > js+ZIBksZ3r0nNfSp2PBSI80LJmHIYwzv1Uo4Dv+qMXMS/r5JI3ZDgdYBaajWn3Ed4Hg > Wurw== > X-Gm-Message-State: > AG10YORIYeBZwr2gNbdXFNnEMipCzyF4XmxjsJ5xCcL1SPpPAt3FYho5f7woQq6eyHH8eU3A5UOCDzVNozyUTg== > X-Received: by 10.182.81.227 with SMTP id d3mr8859463oby.78.1455650163481; > Tue, 16 Feb 2016 11:16:03 -0800 (PST) > Received: by 10.76.160.99 with HTTP; Tue, 16 Feb 2016 11:15:43 -0800 (PST) > In-Reply-To: <etPan.56c3373a.c1948d0.7e1e@JSQUYRES-M-H05C> > References: <etPan.56c25272.501e2675.7e1e@JSQUYRES-M-H05C> > <CAOoEU4Hp-m7RVfAD42ueQLcryUyAP4zY_D-prbHHyPwBdi=6Hg(a)mail.gmail.com> > <etPan.56c3373a.c1948d0.7e1e@JSQUYRES-M-H05C> > From: Jeff Hammond <jeff.science(a)gmail.com> > Date: Tue, 16 Feb 2016 11:15:43 -0800 > Message-ID: < > CAGKz=u+pZtXqgiNV+tgCiv+tttMBUREc6CMYG=BVcC7eqYXcpw(a)mail.gmail.com> > To: Main MPI Forum mailing list <mpi-forum(a)lists.mpi-forum.org> > Subject: Re: [Mpi-forum] Question about MPI_Info set on communicators > X-BeenThere: mpi-forum(a)lists.mpi-forum.org > X-Mailman-Version: 2.1.15 > Precedence: list > Reply-To: Main MPI Forum mailing list <mpi-forum(a)lists.mpi-forum.org> > List-Id: Main MPI Forum mailing list <mpi-forum.lists.mpi-forum.org> > List-Unsubscribe: < > http://lists.mpi-forum.org/mailman/options.cgi/mpi-forum>, > <mailto:[email protected]?subject=unsubscribe > <mpi-forum-request(a)lists.mpi-forum.org?subject=unsubscribe>> > List-Archive: <http://lists.mpi-forum.org/MailArchives/mpi-forum/> > List-Post: <mailto:[email protected] > <mpi-forum(a)lists.mpi-forum.org>> > List-Help: <mailto:[email protected]?subject=help > <mpi-forum-request(a)lists.mpi-forum.org?subject=help>> > List-Subscribe: <http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum > >, > <mailto:[email protected]?subject=subscribe > <mpi-forum-request(a)lists.mpi-forum.org?subject=subscribe>> > Content-Type: multipart/mixed; > boundary="===============5365574427901088839==" > Errors-To: mpi-forum-bounces(a)lists.mpi-forum.org > Sender: mpi-forum <mpi-forum-bounces(a)lists.mpi-forum.org> > Return-Path: mpi-forum-bounces(a)lists.mpi-forum.org > X-MS-Exchange-Organization-AuthSource: XCH-RTP-004.cisco.com > <http://xch-rtp-004.cisco.com> > X-MS-Exchange-Organization-AuthAs: Internal > X-MS-Exchange-Organization-AuthMechanism: 10 > X-MS-Exchange-Organization-Network-Message-Id: > b0530982-0953-4081-be2c-08d33705a182 > X-MS-Exchange-Organization-AVStamp-Enterprise: 1.0 > MIME-Version: 1.0 > > --===============5365574427901088839== > Content-Type: multipart/alternative; boundary=047d7b2e4058e96442052be7f662 > > --047d7b2e4058e96442052be7f662 > Content-Type: text/plain; charset=UTF-8 > > > On Tue, Feb 16, 2016 at 6:50 AM, Jeff Squyres (jsquyres) < > jsquyres(a)cisco.com > > wrote: > > > Jim -- > > > Based on your answer, I'm now a bit confused. I now need to back up from > > my original question and clarify what MPI_COMM_INFO_GET is supposed to > > return in this scenario: > > > ----- > > MPI_Info_set(myinfo, "use_shared_memory", "yes"); > > MPI_Comm_set_info(comm, my info); > > // MPI implementation recognizes the "use_shared_memory" hint, but is > > // unable to use shared memory with this communicator > > MPI_Comm_get_info(comm, myinfo_returned); > > ----- > > > What value is in myinfo_returned for use_shared_memory? > > > MPI 3.1 6.4.4 p250 says: > > > "MPI_COMM_GET_INFO returns a new info object containing the hints of the > > communicator associated with comm. The current setting of all hints > > actually used by the system related to this communicator is returned in > > info_used." > > > "...all hints actually used by the system..." is actually a bit ambiguous. > > > In this case, "use_shared_memory" is *recognized* by the MPI > > implementation. But it wasn't able to be *used* (e.g., because no shared > > memory was available). So is the value in info_used supposed to be: > > > > How does it know how much shared memory is required? I don't see any > reason for MPI_Comm_set_info to fail on account of lack of shared memory. > If zero bytes of shared memory are available, how does MPI_Comm_set_info > know that this is not enough when the user actually needs? > > > 1. The user's requested hint (i.e., "yes"), or > > > > I'm changing my answer to this one, in this case. For MPI_Alloc_mem and > MPI_Win_allocate(_shared), my answer will be different. > > The only reasonable behavior is for MPI_Comm_set_info to succeed and set > "use_shared_memory"="yes". Subsequent operations may behave differently > depending on how much shared memory is available. I would expect the user > to probe that using MPI_T. > > Jeff > > > 2. What MPI is actually using (i.e., "no"), or > > 3. Since this hint wasn't "used" by the system, then the > > "use_shared_memory" key shouldn't be returned in info_used > > > ? > > > Indeed, one could make the argument that MPI_COMM_GET_INFO is supposed to > > return exactly the value(s) that the user requested (if it is *recognized* > > by the implementation), and MPI_T pvars and/or cvars should be queried to > > obtain the values indicating what the implementation chose to do with those > > hints. > > > Thoughts? > > > > > On February 15, 2016 at 8:25:44 PM, Jim Dinan (james.dinan(a)gmail.com) > > wrote: > > "Same" should mean that these call sequences produce the same newcomm, > > right? > > > MPI_Comm_get_info(comm, myinfo_returned); > > MPI_Comm_dup_with_info(comm, &newcomm, myinfo_returned); > > > and > > > MPI_Comm_dup(comm, &newcomm); > > > I think this is an argument for "no". > > > I'm not sure if the question is asking whether the MPI implementation can > > change communicator info values dynamically. I'm not sure the spec > > actually addresses that possibility. If this is allowed, newcomm would > > still only get "yes" if the implementation had also toggled the info key > > on > > the parent communicator. > > > ~Jim. > > > On Mon, Feb 15, 2016 at 5:34 PM, Jeff Squyres (jsquyres) > > wrote: > > > This question has come up in the Open MPI community: > > > ----- > > MPI_Info_set(myinfo, "use_shared_memory", "yes"); > > MPI_Comm_set_info(comm, myinfo); > > MPI_Comm_get_info(comm, myinfo_returned); > > // Assume here that there was not enough shared memory available to use > > shared-memory for > > // communication on this comm, so MPI_Info_get() tells us that > > use_shared_memory is "no". > > > // ...time passes, and now assume there is more shared-memory available > > MPI_Comm_dup_with_info(comm, &newcom); > > MPI_comm_get_info(newcom, myinfo_returned); > > ----- > > > Should newcom have "use_shared_memory" set to "yes" or "no"? > > > Arguments for "yes": > > - the user originally asked for "yes" > > - now there is shared memory available > > > Arguments for "no": > > - the current value for "used_shared_memory" on the parent > > communicator is > > "no" > > > Which should it be? > > > MPI-3.1 6.4.2 p238 says: > > > "...Returns in newcomer a new communicator with the same group or > > groups, > > same topology, same info hints, any copied cached information, but a > > new > > context (see Section 6.7.1)." > > > So the specific question here is: what exactly does "Same info hints" > > mean? The user-specific hint, or the current value of the hint? > > > -- > > Jeff Squyres > > jsquyres(a)cisco.com > > For corporate legal information go to: > > http://www.cisco.com/web/about/doing_business/legal/cri/ > > _______________________________________________ > > mpi-forum mailing list > > mpi-forum(a)lists.mpi-forum.org > > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum > > _______________________________________________ > > mpi-forum mailing list > > mpi-forum(a)lists.mpi-forum.org > > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum > > > -- > > Jeff Squyres > > jsquyres(a)cisco.com > > For corporate legal information go to: > > http://www.cisco.com/web/about/doing_business/legal/cri/ > > _______________________________________________ > > mpi-forum mailing list > > mpi-forum(a)lists.mpi-forum.org > > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum > > > > > > -- > Jeff Hammond > jeff.science(a)gmail.com > http://jeffhammond.github.io/ > > --047d7b2e4058e96442052be7f662 > Content-Type: text/html; charset=UTF-8 > Content-Transfer-Encoding: quoted-printable > > <meta http-equiv=3D"Content-Type" content=3D"text/html; > charset=3Dutf-8"><d= > iv dir=3D"ltr"><br><div class=3D"gmail_extra"><br><div > class=3D"gmail_quote= > ">On Tue, Feb 16, 2016 at 6:50 AM, Jeff Squyres (jsquyres) <span > dir=3D"ltr= > "><<a href=3D"mailto:[email protected] <jsquyres(a)cisco.com>" > target=3D"_blank">jsquyres@cisc= > o.com</a>></span> wrote:<br><blockquote class=3D"gmail_quote" > style=3D"m= > argin:0px 0px 0px > 0.8ex;border-left-width:1px;border-left-color:rgb(204,204= > ,204);border-left-style:solid;padding-left:1ex">Jim --<br> > <br> > Based on your answer, I'm now a bit confused.  I now need to back up > f= > rom my original question and clarify what MPI_COMM_INFO_GET is supposed to > = > return in this scenario:<br> > <span class=3D""><br> > -----<br> > MPI_Info_set(myinfo, "use_shared_memory", > "yes"); = > <br> > MPI_Comm_set_info(comm, my info);<br> > </span>// MPI implementation recognizes the "use_shared_memory" > h= > int, but is<br> > // unable to use shared memory with this communicator<br> > MPI_Comm_get_info(comm, myinfo_returned); <br> > -----<br> > <br> > What value is in myinfo_returned for use_shared_memory?<br> > <br> > MPI 3.1 6.4.4 p250 says:<br> > <br> > "MPI_COMM_GET_INFO returns a new info object containing the hints of > t= > he communicator associated with comm. The current setting of all hints > actu= > ally used by the system related to this communicator is returned in > info_us= > ed."<br> > <br> > "...all hints actually used by the system..." is actually a bit > a= > mbiguous.<br> > <br> > In this case, "use_shared_memory" is *recognized* by the MPI > impl= > ementation.  But it wasn't able to be *used* (e.g., because no shared > = > memory was available).  So is the value in info_used supposed to > = > be:<br> > <br></blockquote><div><br></div><div>How does it know how much shared > memor= > y is required?  I don't see any reason for MPI_Comm_set_info to fail > o= > n account of lack of shared memory.  If zero bytes of shared memory > ar= > e available, how does MPI_Comm_set_info know that this is not enough when > t= > he user actually needs?</div><div> </div><blockquote > class=3D"gmail_qu= > ote" style=3D"margin:0px 0px 0px > 0.8ex;border-left-width:1px;border-left-co= > lor:rgb(204,204,204);border-left-style:solid;padding-left:1ex"> > 1. The user's requested hint (i.e., "yes"), > or<br></blockquote><d= > iv><br></div><div>I'm changing my answer to this one, in this case.  > F= > or MPI_Alloc_mem and MPI_Win_allocate(_shared), my answer will be > different= > .</div><div><br></div><div>The only reasonable behavior is for > MPI_Comm_set= > _info to succeed and set > "use_shared_memory"=3D"yes".&n= > bsp; Subsequent operations may behave differently depending on how much > sha= > red memory is available.  I would expect the user to probe that using > = > MPI_T.</div><div><br></div><div>Jeff</div><div> </div><blockquote > clas= > s=3D"gmail_quote" style=3D"margin:0px 0px 0px > 0.8ex;border-left-width:1px;b= > > order-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"= > > > 2. What MPI is actually using (i.e., "no"), or<br> > 3. Since this hint wasn't "used" by the system, then the > "us= > e_shared_memory" key shouldn't be returned in info_used<br> > <br> > ?<br> > <br> > Indeed, one could make the argument that MPI_COMM_GET_INFO is supposed to > r= > eturn exactly the value(s) that the user requested (if it is *recognized* > b= > y the implementation), and MPI_T pvars and/or cvars should be queried to > ob= > tain the values indicating what the implementation chose to do with those > h= > ints.<br> > <br> > Thoughts?<br> > <span class=3D"im"><br> > <br> > <br> > On February 15, 2016 at 8:25:44 PM, Jim Dinan (<a href=3D"mailto: > james.dina= > n(a)gmail.com">james.dinan(a)gmail.com</a>) wrote:<br> > > "Same" should mean that these call sequences produce the > sam= > e newcomm,<br> > > right?<br> > ><br> > > MPI_Comm_get_info(comm, myinfo_returned);<br> > > MPI_Comm_dup_with_info(comm, &newcomm, myinfo_returned);<br> > ><br> > > and<br> > ><br> > > MPI_Comm_dup(comm, &newcomm);<br> > ><br> > > I think this is an argument for "no".<br> > ><br> > > I'm not sure if the question is asking whether the MPI implementation > = > can<br> > > change communicator info values dynamically. I'm not sure the spec<br> > > actually addresses that possibility. If this is allowed, newcomm > would= > <br> > > still only get "yes" if the implementation had also toggled > = > the info key on<br> > > the parent communicator.<br> > ><br> > > ~Jim.<br> > ><br> > </span><div class=3D""><div class=3D"h5">> On Mon, Feb 15, 2016 at 5:34 > = > PM, Jeff Squyres (jsquyres) > > wrote:<br> > ><br> > > > This question has come up in the Open MPI community:<br> > > ><br> > > > -----<br> > > > MPI_Info_set(myinfo, "use_shared_memory", > "yes&quo= > t;);<br> > > > MPI_Comm_set_info(comm, myinfo);<br> > > > MPI_Comm_get_info(comm, myinfo_returned);<br> > > > // Assume here that there was not enough shared memory available > = > to use<br> > > > shared-memory for<br> > > > // communication on this comm, so MPI_Info_get() tells us > that<br= > > > > > use_shared_memory is "no".<br> > > ><br> > > > // ...time passes, and now assume there is more shared-memory > ava= > ilable<br> > > > MPI_Comm_dup_with_info(comm, &newcom);<br> > > > MPI_comm_get_info(newcom, myinfo_returned);<br> > > > -----<br> > > ><br> > > > Should newcom have "use_shared_memory" set to > "yes= > " or "no"?<br> > > ><br> > > > Arguments for "yes":<br> > > > - the user originally asked for "yes"<br> > > > - now there is shared memory available<br> > > ><br> > > > Arguments for "no":<br> > > > - the current value for "used_shared_memory" on the > par= > ent communicator is<br> > > > "no"<br> > > ><br> > > > Which should it be?<br> > > ><br> > > > MPI-3.1 6.4.2 p238 says:<br> > > ><br> > > > "...Returns in newcomer a new communicator with the same > gro= > up or groups,<br> > > > same topology, same info hints, any copied cached information, > bu= > t a new<br> > > > context (see Section 6.7.1)."<br> > > ><br> > > > So the specific question here is: what exactly does "Same > in= > fo hints"<br> > > > mean? The user-specific hint, or the current value of the > hint?<b= > r> > > ><br> > > > --<br> > > > Jeff Squyres<br> > > > <a href=3D"mailto:[email protected] <jsquyres(a)cisco.com>"> > jsquyres(a)cisco.com</a><br> > > > For corporate legal information go to:<br> > > > <a href=3D" > http://www.cisco.com/web/about/doing_business/legal/cr= > i/" rel=3D"noreferrer" target=3D"_blank"> > http://www.cisco.com/web/about/doi= > ng_business/legal/cri/</a><br> > > > _______________________________________________<br> > > > mpi-forum mailing list<br> > > > <a href=3D"mailto:[email protected] > <mpi-forum(a)lists.mpi-forum.org>">mpi-forum(a)lists.= > mpi-forum.org</a><br> > > > <a href=3D" > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-fo= > rum" rel=3D"noreferrer" target=3D"_blank"> > http://lists.mpi-forum.org/mailma= > n/listinfo.cgi/mpi-forum</a><br> > > _______________________________________________<br> > > mpi-forum mailing list<br> > > <a href=3D"mailto:[email protected] > <mpi-forum(a)lists.mpi-forum.org>">mpi-forum(a)lists.mpi-f= > orum.org</a><br> > > <a href=3D"http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum" > = > rel=3D"noreferrer" target=3D"_blank"> > http://lists.mpi-forum.org/mailman/lis= > tinfo.cgi/mpi-forum</a><br> > <br> > --<br> > Jeff Squyres<br> > <a href=3D"mailto:[email protected] <jsquyres(a)cisco.com>"> > jsquyres(a)cisco.com</a><br> > For corporate legal information go to: <a href=3D" > http://www.cisco.com/web/= > about/doing_business/legal/cri/" rel=3D"noreferrer" > target=3D"_blank">http:= > //www.cisco.com/web/about/doing_business/legal/cri/</a><br> > _______________________________________________<br> > mpi-forum mailing list<br> > <a href=3D"mailto:[email protected] > <mpi-forum(a)lists.mpi-forum.org>">mpi-forum(a)lists.mpi-forum.= > org</a><br> > <a href=3D"http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum" rel= > =3D"noreferrer" target=3D"_blank"> > http://lists.mpi-forum.org/mailman/listin= > fo.cgi/mpi-forum</a></div></div></blockquote></div><br><br > clear=3D"all"><d= > iv><br></div>-- <br><div class=3D"gmail_signature">Jeff Hammond<br><a href= > =3D"mailto:[email protected] <jeff.science(a)gmail.com>" > target=3D"_blank">jeff.science(a)gmail.com= > </a><br><a href=3D"http://jeffhammond.github.io/" target=3D"_blank"> > http://= > jeffhammond.github.io/</a></div> > </div></div> > > --047d7b2e4058e96442052be7f662-- > > --===============5365574427901088839== > Content-Type: text/plain; charset="us-ascii" > Content-Transfer-Encoding: 7bit > Content-Disposition: inline > > _______________________________________________ > mpi-forum mailing list > mpi-forum(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum > --===============5365574427901088839==-- > > > _______________________________________________ > mpi-forum mailing list > mpi-forum(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum > -- Jeff Hammond jeff.science(a)gmail.com http://jeffhammond.github.io/

1 0

Re: [Mpi-forum] Question about MPI_Info set on communicators
by Jeff Squyres (jsquyres) 16 Feb '16

16 Feb '16

Forget the use-shared-memory hint; it was just an example. Substitute in any "foo" info key hint name(assuming the implementation understands the "foo" key hint name); the original question remains: what should the value be if the implementation cannot provide the requested hint? - "yes" because the user asked for it (i.e., that's the hint, regardless of what the implementation dos with it) - "no" because the implementation could not provide it - no "foo" key at all because the implementation did not use the hint Sent from my phone. No type good. On Feb 16, 2016, at 2:16 PM, Jeff Hammond <jeff.science(a)gmail.com<mailto:[email protected]>> wrote: On Tue, Feb 16, 2016 at 6:50 AM, Jeff Squyres (jsquyres) <jsquyres(a)cisco.com<mailto:[email protected]>> wrote: Jim -- Based on your answer, I'm now a bit confused. I now need to back up from my original question and clarify what MPI_COMM_INFO_GET is supposed to return in this scenario: ----- MPI_Info_set(myinfo, "use_shared_memory", "yes"); MPI_Comm_set_info(comm, my info); // MPI implementation recognizes the "use_shared_memory" hint, but is // unable to use shared memory with this communicator MPI_Comm_get_info(comm, myinfo_returned); ----- What value is in myinfo_returned for use_shared_memory? MPI 3.1 6.4.4 p250 says: "MPI_COMM_GET_INFO returns a new info object containing the hints of the communicator associated with comm. The current setting of all hints actually used by the system related to this communicator is returned in info_used." "...all hints actually used by the system..." is actually a bit ambiguous. In this case, "use_shared_memory" is *recognized* by the MPI implementation. But it wasn't able to be *used* (e.g., because no shared memory was available). So is the value in info_used supposed to be: How does it know how much shared memory is required? I don't see any reason for MPI_Comm_set_info to fail on account of lack of shared memory. If zero bytes of shared memory are available, how does MPI_Comm_set_info know that this is not enough when the user actually needs? 1. The user's requested hint (i.e., "yes"), or I'm changing my answer to this one, in this case. For MPI_Alloc_mem and MPI_Win_allocate(_shared), my answer will be different. The only reasonable behavior is for MPI_Comm_set_info to succeed and set "use_shared_memory"="yes". Subsequent operations may behave differently depending on how much shared memory is available. I would expect the user to probe that using MPI_T. Jeff 2. What MPI is actually using (i.e., "no"), or 3. Since this hint wasn't "used" by the system, then the "use_shared_memory" key shouldn't be returned in info_used ? Indeed, one could make the argument that MPI_COMM_GET_INFO is supposed to return exactly the value(s) that the user requested (if it is *recognized* by the implementation), and MPI_T pvars and/or cvars should be queried to obtain the values indicating what the implementation chose to do with those hints. Thoughts? On February 15, 2016 at 8:25:44 PM, Jim Dinan (james.dinan(a)gmail.com<mailto:[email protected]>) wrote: > "Same" should mean that these call sequences produce the same newcomm, > right? > > MPI_Comm_get_info(comm, myinfo_returned); > MPI_Comm_dup_with_info(comm, &newcomm, myinfo_returned); > > and > > MPI_Comm_dup(comm, &newcomm); > > I think this is an argument for "no". > > I'm not sure if the question is asking whether the MPI implementation can > change communicator info values dynamically. I'm not sure the spec > actually addresses that possibility. If this is allowed, newcomm would > still only get "yes" if the implementation had also toggled the info key on > the parent communicator. > > ~Jim. > > On Mon, Feb 15, 2016 at 5:34 PM, Jeff Squyres (jsquyres) > > wrote: > > > This question has come up in the Open MPI community: > > > > ----- > > MPI_Info_set(myinfo, "use_shared_memory", "yes"); > > MPI_Comm_set_info(comm, myinfo); > > MPI_Comm_get_info(comm, myinfo_returned); > > // Assume here that there was not enough shared memory available to use > > shared-memory for > > // communication on this comm, so MPI_Info_get() tells us that > > use_shared_memory is "no". > > > > // ...time passes, and now assume there is more shared-memory available > > MPI_Comm_dup_with_info(comm, &newcom); > > MPI_comm_get_info(newcom, myinfo_returned); > > ----- > > > > Should newcom have "use_shared_memory" set to "yes" or "no"? > > > > Arguments for "yes": > > - the user originally asked for "yes" > > - now there is shared memory available > > > > Arguments for "no": > > - the current value for "used_shared_memory" on the parent communicator is > > "no" > > > > Which should it be? > > > > MPI-3.1 6.4.2 p238 says: > > > > "...Returns in newcomer a new communicator with the same group or groups, > > same topology, same info hints, any copied cached information, but a new > > context (see Section 6.7.1)." > > > > So the specific question here is: what exactly does "Same info hints" > > mean? The user-specific hint, or the current value of the hint? > > > > -- > > Jeff Squyres > > jsquyres(a)cisco.com<mailto:[email protected]> > > For corporate legal information go to: > > http://www.cisco.com/web/about/doing_business/legal/cri/ > > _______________________________________________ > > mpi-forum mailing list > > mpi-forum(a)lists.mpi-forum.org<mailto:[email protected]> > > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum > _______________________________________________ > mpi-forum mailing list > mpi-forum(a)lists.mpi-forum.org<mailto:[email protected]> > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum -- Jeff Squyres jsquyres(a)cisco.com<mailto:[email protected]> For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/ _______________________________________________ mpi-forum mailing list mpi-forum(a)lists.mpi-forum.org<mailto:[email protected]> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum -- Jeff Hammond jeff.science(a)gmail.com<mailto:[email protected]> http://jeffhammond.github.io/ Received: from xch-rtp-019.cisco.com<http://xch-rtp-019.cisco.com> (64.101.220.159) by xch-rcd-019.cisco.com<http://xch-rcd-019.cisco.com> (173.37.102.29) with Microsoft SMTP Server (TLS) id 15.0.1104.5 via Mailbox Transport; Tue, 16 Feb 2016 13:16:12 -0600 Received: from xch-rtp-004.cisco.com<http://xch-rtp-004.cisco.com> (64.101.220.144) by XCH-RTP-019.cisco.com<http://xch-rtp-019.cisco.com> (64.101.220.159) with Microsoft SMTP Server (TLS) id 15.0.1104.5; Tue, 16 Feb 2016 14:16:11 -0500 Received: from rcdn-iport-1.cisco.com<http://rcdn-iport-1.cisco.com> (173.37.86.72) by mail.cisco.com<http://mail.cisco.com> (64.101.220.144) with Microsoft SMTP Server (TLS) id 15.0.1104.5 via Frontend Transport; Tue, 16 Feb 2016 14:16:11 -0500 Received: from rcdn-core-5.cisco.com<http://rcdn-core-5.cisco.com> ([173.37.93.156]) by rcdn-iport-1.cisco.com<http://rcdn-iport-1.cisco.com> with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 16 Feb 2016 19:16:09 +0000 Received: from alln-inbound-e.cisco.com<http://alln-inbound-e.cisco.com> (alln-inbound-e.cisco.com<http://alln-inbound-e.cisco.com> [173.37.147.235]) by rcdn-core-5.cisco.com<http://rcdn-core-5.cisco.com> (8.14.5/8.14.5) with ESMTP id u1GJG9s3016172; Tue, 16 Feb 2016 19:16:10 GMT Received-SPF: None (alln-inbound-e.cisco.com<http://alln-inbound-e.cisco.com>: no sender authenticity information available from domain of mpi-forum-bounces(a)lists.mpi-forum.org<mailto:[email protected]>) identity=mailfrom; client-ip=129.79.39.208; receiver=alln-inbound-e.cisco.com<http://alln-inbound-e.cisco.com>; envelope-from="mpi-forum-bounces(a)lists.mpi-forum.org<mailto:[email protected]>"; x-sender="mpi-forum-bounces(a)lists.mpi-forum.org<mailto:[email protected]>"; x-conformance=spf_only Received-SPF: None (alln-inbound-e.cisco.com<http://alln-inbound-e.cisco.com>: no sender authenticity information available from domain of postmaster(a)lion.crest.iu.edu<mailto:[email protected]>) identity=helo; client-ip=129.79.39.208; receiver=alln-inbound-e.cisco.com<http://alln-inbound-e.cisco.com>; envelope-from="mpi-forum-bounces(a)lists.mpi-forum.org<mailto:[email protected]>"; x-sender="postmaster(a)lion.crest.iu.edu<mailto:[email protected]>"; x-conformance=spf_only Authentication-Results: alln-inbound-e.cisco.com<http://alln-inbound-e.cisco.com>; spf=None smtp.mailfrom=mpi-forum-bounces(a)lists.mpi-forum.org<mailto:[email protected]>; spf=None smtp.helo=postmaster(a)lion.crest.iu.edu<mailto:[email protected]>; dkim=hardfail (body hash did not verify [final]) header.i=(a)gmail.com<mailto:[email protected]>; dkim=hardfail (body hash did not verify [final]) header.i=(a)gmail.com<mailto:[email protected]>; dmarc=fail (p=none dis=none) d=gmail.com<http://gmail.com> X-from-outside-Cisco: 129.79.39.208 IronPort-PHdr: =?us-ascii?q?9a23=3A3koMHBVMN/yf5qkGiy0ZpEKb+w/V8LGtZVwlr6E/?= =?us-ascii?q?grcLSJyIuqrYYRSBt8tkgFKBZ4jH8fUM07OQ6PC/HzBbqs/Y4TgrS99laVwssY?= =?us-ascii?q?0uhQsuAcqIWwXQDcXBSGgEJvlET0Jv5HqhMEJYS47UblzWpWCuv3ZJQk2sfTR8?= =?us-ascii?q?Kum9IIPOlcP/j7n0oM2MJVkTz2PkMPtbF1afk0b4joEum4xsK6I8mFPig0BjXK?= =?us-ascii?q?Bo/15uPk+ZhB3m5829r9ZJ+iVUvO89pYYbCf2pN4xxd7FTDSwnPmYp/4Wr8ECb?= =?us-ascii?q?FUrcrkYaSXgcxxpUHxCXq1b+X4ztqW3/rus41i6AIMiwSqo7HjGr7qNuQRmvjz?= =?us-ascii?q?8bNjk/6yfchc03ibpa5RymvQF6kLPSe5yfYf93f6fBeoEfXWsEUMFLSikEGI6l?= =?us-ascii?q?coYUE8IFPP1EtM/4v1pLoxykGAzqA/ngnSRVjH371rFvzuI6DAvd1xYhFd9djH?= =?us-ascii?q?OBidz0Mu83acH94q7S0DCLavpX3Tb758DMcws96aWJRbM1f83L1EwHEwLekk7W?= =?us-ascii?q?rJTkej2SzPkI9W+B4Lwzb++3j38brFRUqzmpjuIUq8HngZgIzxjB/CR9wYA6bd?= =?us-ascii?q?m5U1IzNdu8Ed5esDuHMqNyQ9g+WCdtoCd/xLocpJv9ezIFnscJ3RnaPt6KfYbA?= =?us-ascii?q?whvmUeGXLX8shn9rdvSkjBe/90Ot4uL5X8iwll1Nq3wWwZH3qnkR2kmLuYC8Qf?= =?us-ascii?q?xn8xLkgG7X2g=3D=3D?= X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: =?us-ascii?q?A0GWAACSdMNWmdAnT4FeBhMBAQEBDwEBA?= =?us-ascii?q?QEGAQEBAYNSRyYGhDWkGIFJI4UhilQBDYEmGyIbAQmHJD8UAQEBAQEBAQECDgE?= =?us-ascii?q?BAQEBCAsJCSEugi0JATgGBAEBBAEBAQEBAQEBASMBAQEBAQEBAQEBAQEBAQEBA?= =?us-ascii?q?QEBAQEBAQEBAQEBAQEBAQEBAQYCDR4PFgEaAQEBAQIBAQEBDxEdAQUKDBIMAgE?= =?us-ascii?q?BAQECBgEBBQULDRUNBAQCAgMBGgMBAQUECAEFARwGEwUPDodiAQMKCA6eJIExP?= =?us-ascii?q?jGLNIFpgleBYINOChknDVGECAEBAQEBAQEDAQEBAQEVAQUKBIUQhSiCN4I6gkG?= =?us-ascii?q?BOgWOHIhCIYENhEaFH3SBc4ImjE1EhjuGBREePwNaDwEBgkEcgWlJAYhbAQEB?= X-IPAS-Result: =?us-ascii?q?A0GWAACSdMNWmdAnT4FeBhMBAQEBDwEBAQEGAQEBAYNSRyY?= =?us-ascii?q?GhDWkGIFJI4UhilQBDYEmGyIbAQmHJD8UAQEBAQEBAQECDgEBAQEBCAsJCSEug?= =?us-ascii?q?i0JATgGBAEBBAEBAQEBAQEBASMBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQE?= =?us-ascii?q?BAQEBAQEBAQYCDR4PFgEaAQEBAQIBAQEBDxEdAQUKDBIMAgEBAQECBgEBBQULD?= =?us-ascii?q?RUNBAQCAgMBGgMBAQUECAEFARwGEwUPDodiAQMKCA6eJIExPjGLNIFpgleBYIN?= =?us-ascii?q?OChknDVGECAEBAQEBAQEDAQEBAQEVAQUKBIUQhSiCN4I6gkGBOgWOHIhCIYENh?= =?us-ascii?q?EaFH3SBc4ImjE1EhjuGBREePwNaDwEBgkEcgWlJAYhbAQEB?= X-IronPort-AV: E=Sophos;i="5.22,456,1449532800"; d="scan'208,217";a="194686767" X-Amp-Result: Clean X-Amp-File-Uploaded: False X-IronPort-Outbreak-Status: No, level 0, Unknown - Unknown Received: from crest.iu.edu<http://crest.iu.edu> (HELO lion.crest.iu.edu<http://lion.crest.iu.edu>) ([129.79.39.208]) by alln-inbound-e.cisco.com<http://alln-inbound-e.cisco.com> with ESMTP; 16 Feb 2016 19:16:08 +0000 Received: by lion.crest.iu.edu<http://lion.crest.iu.edu> (Postfix, from userid 493) id 8E3BD260779; Tue, 16 Feb 2016 14:16:08 -0500 (EST) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on lion.crest.iu.edu<http://lion.crest.iu.edu> X-Spam-Level: X-Spam-Status: No, score=-2.5 required=5.0 tests=BAYES_00,DKIM_ADSP_CUSTOM_MED, DKIM_SIGNED,FREEMAIL_FROM,HTML_MESSAGE,RCVD_IN_DNSWL_LOW,T_DKIM_INVALID autolearn=unavailable version=3.3.1 Received: from crest.iu.edu<http://crest.iu.edu> (localhost [127.0.0.1]) by lion.crest.iu.edu<http://lion.crest.iu.edu> (Postfix) with ESMTP id 220D12608A4; Tue, 16 Feb 2016 14:16:06 -0500 (EST) X-Original-To: mpi-forum(a)lists.mpi-forum.org<mailto:[email protected]> Delivered-To: mpi-forum(a)lists.mpi-forum.org<mailto:[email protected]> Received: by lion.crest.iu.edu<http://lion.crest.iu.edu> (Postfix, from userid 493) id C3B4326079D; Tue, 16 Feb 2016 14:16:04 -0500 (EST) Received: from mail-ob0-f177.google.com<http://mail-ob0-f177.google.com> (mail-ob0-f177.google.com<http://mail-ob0-f177.google.com> [209.85.214.177]) by lion.crest.iu.edu<http://lion.crest.iu.edu> (Postfix) with ESMTP id 0CF2D260779 for <mpi-forum(a)lists.mpi-forum.org<mailto:[email protected]>>; Tue, 16 Feb 2016 14:16:04 -0500 (EST) Received: by mail-ob0-f177.google.com<http://mail-ob0-f177.google.com> with SMTP id gc3so172260092obb.3 for <mpi-forum(a)lists.mpi-forum.org<mailto:[email protected]>>; Tue, 16 Feb 2016 11:16:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com<http://gmail.com>; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=KEQt98rYsl7/LCgT0QeppyTHXsG6OQ4HlXJPO5cSx6g=; b=ZjDP3X63meN3mEnca6KBi6EBk9pci2/5zBnyPW9jXQQoUqv8q6f6xnTqfe6kAJIANL BrdVhlIVnOzqmSuQluiCJ+wif1RFZsh9I1q2Rc3JmktVNu9mlkj8GABpnIV76JsN3+Nh TE4bBFG6d/yEpONEhBmHYb0M9CPNfGwjyHcU/iwIIdp/NJ2aCFQd2DT3iTjubsfcQ9YC NtZrfSHLBJ7W7uXA4xh502P+h5n3Zw4cK5T3jvnX2dS8Ya7q2RgvaB9hZ0O573Z8KPGX NckpbsAI93Z+2WOYKjYD+RahNtTisu/3pg/YUYPp32fT3ClY6+y/lCvz+v3MAbEr5QwM i3hw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net<http://1e100.net>; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-type; bh=KEQt98rYsl7/LCgT0QeppyTHXsG6OQ4HlXJPO5cSx6g=; b=f0LWIMla+RNtd30P6me/Due6bIOkINSUFd72cgxq4quY2u5vwQ2vGAidpQmR6GxaVX Va0CxATjTEdQthxE645JcglMkoa3Qr3/NyZSn3NdxXcGBNSt4O21JWy2JMhyrRq/ZOdU ZUUWR3E/0sndIX6N2CVn8TmcDe+V6oK5seuP1/s0DBArBabm7UvX8i4WpZ87Rbro8ZZz bGaz6H5r2BNGfvqTs/dJpFgngDtZmQhfs9bBQqTAZnZF2BT0Hr5mYXslbpVpeeFP10ZB js+ZIBksZ3r0nNfSp2PBSI80LJmHIYwzv1Uo4Dv+qMXMS/r5JI3ZDgdYBaajWn3Ed4Hg Wurw== X-Gm-Message-State: AG10YORIYeBZwr2gNbdXFNnEMipCzyF4XmxjsJ5xCcL1SPpPAt3FYho5f7woQq6eyHH8eU3A5UOCDzVNozyUTg== X-Received: by 10.182.81.227 with SMTP id d3mr8859463oby.78.1455650163481; Tue, 16 Feb 2016 11:16:03 -0800 (PST) Received: by 10.76.160.99 with HTTP; Tue, 16 Feb 2016 11:15:43 -0800 (PST) In-Reply-To: <etPan.56c3373a.c1948d0.7e1e@JSQUYRES-M-H05C> References: <etPan.56c25272.501e2675.7e1e@JSQUYRES-M-H05C> <CAOoEU4Hp-m7RVfAD42ueQLcryUyAP4zY_D-prbHHyPwBdi=6Hg(a)mail.gmail.com<mailto:CAOoEU4Hp-m7RVfAD42ueQLcryUyAP4zY_D-prbHHyPwBdi=6Hg@mail.gmail.com>> <etPan.56c3373a.c1948d0.7e1e@JSQUYRES-M-H05C> From: Jeff Hammond <jeff.science(a)gmail.com<mailto:[email protected]>> Date: Tue, 16 Feb 2016 11:15:43 -0800 Message-ID: <CAGKz=u+pZtXqgiNV+tgCiv+tttMBUREc6CMYG=BVcC7eqYXcpw(a)mail.gmail.com<mailto:CAGKz=u+pZtXqgiNV+tgCiv+tttMBUREc6CMYG=BVcC7eqYXcpw@mail.gmail.com>> To: Main MPI Forum mailing list <mpi-forum(a)lists.mpi-forum.org<mailto:[email protected]>> Subject: Re: [Mpi-forum] Question about MPI_Info set on communicators X-BeenThere: mpi-forum(a)lists.mpi-forum.org<mailto:[email protected]> X-Mailman-Version: 2.1.15 Precedence: list Reply-To: Main MPI Forum mailing list <mpi-forum(a)lists.mpi-forum.org<mailto:[email protected]>> List-Id: Main MPI Forum mailing list <mpi-forum.lists.mpi-forum.org<http://mpi-forum.lists.mpi-forum.org>> List-Unsubscribe: <http://lists.mpi-forum.org/mailman/options.cgi/mpi-forum>, <mailto:[email protected]?subject=unsubscribe> List-Archive: <http://lists.mpi-forum.org/MailArchives/mpi-forum/> List-Post: <mailto:[email protected]> List-Help: <mailto:[email protected]?subject=help> List-Subscribe: <http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum>, <mailto:[email protected]?subject=subscribe> Content-Type: multipart/mixed; boundary="===============5365574427901088839==" Errors-To: mpi-forum-bounces(a)lists.mpi-forum.org<mailto:[email protected]> Sender: mpi-forum <mpi-forum-bounces(a)lists.mpi-forum.org<mailto:[email protected]>> Return-Path: mpi-forum-bounces(a)lists.mpi-forum.org<mailto:[email protected]> X-MS-Exchange-Organization-AuthSource: XCH-RTP-004.cisco.com<http://xch-rtp-004.cisco.com> X-MS-Exchange-Organization-AuthAs: Internal X-MS-Exchange-Organization-AuthMechanism: 10 X-MS-Exchange-Organization-Network-Message-Id: b0530982-0953-4081-be2c-08d33705a182 X-MS-Exchange-Organization-AVStamp-Enterprise: 1.0 MIME-Version: 1.0 --===============5365574427901088839== Content-Type: multipart/alternative; boundary=047d7b2e4058e96442052be7f662 --047d7b2e4058e96442052be7f662 Content-Type: text/plain; charset=UTF-8 On Tue, Feb 16, 2016 at 6:50 AM, Jeff Squyres (jsquyres) <jsquyres(a)cisco.com<mailto:[email protected]> wrote: Jim -- Based on your answer, I'm now a bit confused. I now need to back up from my original question and clarify what MPI_COMM_INFO_GET is supposed to return in this scenario: ----- MPI_Info_set(myinfo, "use_shared_memory", "yes"); MPI_Comm_set_info(comm, my info); // MPI implementation recognizes the "use_shared_memory" hint, but is // unable to use shared memory with this communicator MPI_Comm_get_info(comm, myinfo_returned); ----- What value is in myinfo_returned for use_shared_memory? MPI 3.1 6.4.4 p250 says: "MPI_COMM_GET_INFO returns a new info object containing the hints of the communicator associated with comm. The current setting of all hints actually used by the system related to this communicator is returned in info_used." "...all hints actually used by the system..." is actually a bit ambiguous. In this case, "use_shared_memory" is *recognized* by the MPI implementation. But it wasn't able to be *used* (e.g., because no shared memory was available). So is the value in info_used supposed to be: How does it know how much shared memory is required? I don't see any reason for MPI_Comm_set_info to fail on account of lack of shared memory. If zero bytes of shared memory are available, how does MPI_Comm_set_info know that this is not enough when the user actually needs? 1. The user's requested hint (i.e., "yes"), or I'm changing my answer to this one, in this case. For MPI_Alloc_mem and MPI_Win_allocate(_shared), my answer will be different. The only reasonable behavior is for MPI_Comm_set_info to succeed and set "use_shared_memory"="yes". Subsequent operations may behave differently depending on how much shared memory is available. I would expect the user to probe that using MPI_T. Jeff 2. What MPI is actually using (i.e., "no"), or 3. Since this hint wasn't "used" by the system, then the "use_shared_memory" key shouldn't be returned in info_used ? Indeed, one could make the argument that MPI_COMM_GET_INFO is supposed to return exactly the value(s) that the user requested (if it is *recognized* by the implementation), and MPI_T pvars and/or cvars should be queried to obtain the values indicating what the implementation chose to do with those hints. Thoughts? On February 15, 2016 at 8:25:44 PM, Jim Dinan (james.dinan(a)gmail.com<mailto:[email protected]>) wrote: "Same" should mean that these call sequences produce the same newcomm, right? MPI_Comm_get_info(comm, myinfo_returned); MPI_Comm_dup_with_info(comm, &newcomm, myinfo_returned); and MPI_Comm_dup(comm, &newcomm); I think this is an argument for "no". I'm not sure if the question is asking whether the MPI implementation can change communicator info values dynamically. I'm not sure the spec actually addresses that possibility. If this is allowed, newcomm would still only get "yes" if the implementation had also toggled the info key on the parent communicator. ~Jim. On Mon, Feb 15, 2016 at 5:34 PM, Jeff Squyres (jsquyres) > > wrote: This question has come up in the Open MPI community: ----- MPI_Info_set(myinfo, "use_shared_memory", "yes"); MPI_Comm_set_info(comm, myinfo); MPI_Comm_get_info(comm, myinfo_returned); // Assume here that there was not enough shared memory available to use shared-memory for // communication on this comm, so MPI_Info_get() tells us that use_shared_memory is "no". // ...time passes, and now assume there is more shared-memory available MPI_Comm_dup_with_info(comm, &newcom); MPI_comm_get_info(newcom, myinfo_returned); ----- Should newcom have "use_shared_memory" set to "yes" or "no"? Arguments for "yes": - the user originally asked for "yes" - now there is shared memory available Arguments for "no": - the current value for "used_shared_memory" on the parent communicator is "no" Which should it be? MPI-3.1 6.4.2 p238 says: "...Returns in newcomer a new communicator with the same group or groups, same topology, same info hints, any copied cached information, but a new context (see Section 6.7.1)." So the specific question here is: what exactly does "Same info hints" mean? The user-specific hint, or the current value of the hint? -- Jeff Squyres jsquyres(a)cisco.com<mailto:[email protected]> For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/ _______________________________________________ mpi-forum mailing list mpi-forum(a)lists.mpi-forum.org<mailto:[email protected]> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum _______________________________________________ mpi-forum mailing list mpi-forum(a)lists.mpi-forum.org<mailto:[email protected]> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum -- Jeff Squyres jsquyres(a)cisco.com<mailto:[email protected]> For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/ _______________________________________________ mpi-forum mailing list mpi-forum(a)lists.mpi-forum.org<mailto:[email protected]> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum -- Jeff Hammond jeff.science(a)gmail.com<mailto:[email protected]> http://jeffhammond.github.io/ --047d7b2e4058e96442052be7f662 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable <meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3Dutf-8"><d= iv dir=3D"ltr"><br><div class=3D"gmail_extra"><br><div class=3D"gmail_quote= ">On Tue, Feb 16, 2016 at 6:50 AM, Jeff Squyres (jsquyres) <span dir=3D"ltr= "><<a href=3D"mailto:[email protected]" target=3D"_blank">jsquyres@cisc= o.com<http://o.com></a>></span> wrote:<br><blockquote class=3D"gmail_quote" style=3D"m= argin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204= ,204);border-left-style:solid;padding-left:1ex">Jim --<br> <br> Based on your answer, I'm now a bit confused.  I now need to back up f= rom my original question and clarify what MPI_COMM_INFO_GET is supposed to = return in this scenario:<br> <span class=3D""><br> -----<br> MPI_Info_set(myinfo, "use_shared_memory", "yes"); = <br> MPI_Comm_set_info(comm, my info);<br> </span>// MPI implementation recognizes the "use_shared_memory" h= int, but is<br> // unable to use shared memory with this communicator<br> MPI_Comm_get_info(comm, myinfo_returned); <br> -----<br> <br> What value is in myinfo_returned for use_shared_memory?<br> <br> MPI 3.1 6.4.4 p250 says:<br> <br> "MPI_COMM_GET_INFO returns a new info object containing the hints of t= he communicator associated with comm. The current setting of all hints actu= ally used by the system related to this communicator is returned in info_us= ed."<br> <br> "...all hints actually used by the system..." is actually a bit a= mbiguous.<br> <br> In this case, "use_shared_memory" is *recognized* by the MPI impl= ementation.  But it wasn't able to be *used* (e.g., because no shared = memory was available).  So is the value in info_used supposed to = be:<br> <br></blockquote><div><br></div><div>How does it know how much shared memor= y is required?  I don't see any reason for MPI_Comm_set_info to fail o= n account of lack of shared memory.  If zero bytes of shared memory ar= e available, how does MPI_Comm_set_info know that this is not enough when t= he user actually needs?</div><div> </div><blockquote class=3D"gmail_qu= ote" style=3D"margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-co= lor:rgb(204,204,204);border-left-style:solid;padding-left:1ex"> 1. The user's requested hint (i.e., "yes"), or<br></blockquote><d= iv><br></div><div>I'm changing my answer to this one, in this case.  F= or MPI_Alloc_mem and MPI_Win_allocate(_shared), my answer will be different= .</div><div><br></div><div>The only reasonable behavior is for MPI_Comm_set= _info to succeed and set "use_shared_memory"=3D"yes".&n= bsp; Subsequent operations may behave differently depending on how much sha= red memory is available.  I would expect the user to probe that using = MPI_T.</div><div><br></div><div>Jeff</div><div> </div><blockquote clas= s=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left-width:1px;b= order-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"= 2. What MPI is actually using (i.e., "no"), or<br> 3. Since this hint wasn't "used" by the system, then the "us= e_shared_memory" key shouldn't be returned in info_used<br> <br> ?<br> <br> Indeed, one could make the argument that MPI_COMM_GET_INFO is supposed to r= eturn exactly the value(s) that the user requested (if it is *recognized* b= y the implementation), and MPI_T pvars and/or cvars should be queried to ob= tain the values indicating what the implementation chose to do with those h= ints.<br> <br> Thoughts?<br> <span class=3D"im"><br> <br> <br> On February 15, 2016 at 8:25:44 PM, Jim Dinan (<a href=3D"mailto:james.dina= n(a)gmail.com<mailto:[email protected]>">james.dinan(a)gmail.com<mailto:[email protected]></a>) wrote:<br> > "Same" should mean that these call sequences produce the sam= e newcomm,<br> > right?<br> ><br> > MPI_Comm_get_info(comm, myinfo_returned);<br> > MPI_Comm_dup_with_info(comm, &newcomm, myinfo_returned);<br> ><br> > and<br> ><br> > MPI_Comm_dup(comm, &newcomm);<br> ><br> > I think this is an argument for "no".<br> ><br> > I'm not sure if the question is asking whether the MPI implementation = can<br> > change communicator info values dynamically. I'm not sure the spec<br> > actually addresses that possibility. If this is allowed, newcomm would= <br> > still only get "yes" if the implementation had also toggled = the info key on<br> > the parent communicator.<br> ><br> > ~Jim.<br> ><br> </span><div class=3D""><div class=3D"h5">> On Mon, Feb 15, 2016 at 5:34 = PM, Jeff Squyres (jsquyres) > > wrote:<br> ><br> > > This question has come up in the Open MPI community:<br> > ><br> > > -----<br> > > MPI_Info_set(myinfo, "use_shared_memory", "yes&quo= t;);<br> > > MPI_Comm_set_info(comm, myinfo);<br> > > MPI_Comm_get_info(comm, myinfo_returned);<br> > > // Assume here that there was not enough shared memory available = to use<br> > > shared-memory for<br> > > // communication on this comm, so MPI_Info_get() tells us that<br= > > use_shared_memory is "no".<br> > ><br> > > // ...time passes, and now assume there is more shared-memory ava= ilable<br> > > MPI_Comm_dup_with_info(comm, &newcom);<br> > > MPI_comm_get_info(newcom, myinfo_returned);<br> > > -----<br> > ><br> > > Should newcom have "use_shared_memory" set to "yes= " or "no"?<br> > ><br> > > Arguments for "yes":<br> > > - the user originally asked for "yes"<br> > > - now there is shared memory available<br> > ><br> > > Arguments for "no":<br> > > - the current value for "used_shared_memory" on the par= ent communicator is<br> > > "no"<br> > ><br> > > Which should it be?<br> > ><br> > > MPI-3.1 6.4.2 p238 says:<br> > ><br> > > "...Returns in newcomer a new communicator with the same gro= up or groups,<br> > > same topology, same info hints, any copied cached information, bu= t a new<br> > > context (see Section 6.7.1)."<br> > ><br> > > So the specific question here is: what exactly does "Same in= fo hints"<br> > > mean? The user-specific hint, or the current value of the hint?<b= r> > ><br> > > --<br> > > Jeff Squyres<br> > > <a href=3D"mailto:[email protected]">jsquyres(a)cisco.com<mailto:[email protected]></a><br> > > For corporate legal information go to:<br> > > <a href=3D"http://www.cisco.com/web/about/doing_business/legal/cr= i/" rel=3D"noreferrer" target=3D"_blank">http://www.cisco.com/web/about/doi= ng_business/legal/cri/</a><br> > > _______________________________________________<br> > > mpi-forum mailing list<br> > > <a href=3D"mailto:[email protected]">mpi-forum(a)lists.= mpi-forum.org<http://mpi-forum.org></a><br> > > <a href=3D"http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-fo= rum" rel=3D"noreferrer" target=3D"_blank">http://lists.mpi-forum.org/mailma= n/listinfo.cgi/mpi-forum</a><br> > _______________________________________________<br> > mpi-forum mailing list<br> > <a href=3D"mailto:[email protected]">mpi-forum(a)lists.mpi<mailto:[email protected]>-f= orum.org<http://orum.org></a><br> > <a href=3D"http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum" = rel=3D"noreferrer" target=3D"_blank">http://lists.mpi-forum.org/mailman/lis= tinfo.cgi/mpi-forum</a><br> <br> --<br> Jeff Squyres<br> <a href=3D"mailto:[email protected]">jsquyres(a)cisco.com<mailto:[email protected]></a><br> For corporate legal information go to: <a href=3D"http://www.cisco.com/web/= about/doing_business/legal/cri/" rel=3D"noreferrer" target=3D"_blank">http:= //www.cisco.com/web/about/doing_business/legal/cri/<http://www.cisco.com/web/about/doing_business/legal/cri/></a><br> _______________________________________________<br> mpi-forum mailing list<br> <a href=3D"mailto:[email protected]">mpi-forum(a)lists.mpi<mailto:[email protected]>-forum.= org</a><br> <a href=3D"http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum" rel= =3D"noreferrer" target=3D"_blank">http://lists.mpi-forum.org/mailman/listin= fo.cgi/mpi-forum</a></div></div></blockquote></div><br><br clear=3D"all"><d= iv><br></div>-- <br><div class=3D"gmail_signature">Jeff Hammond<br><a href= =3D"mailto:[email protected]" target=3D"_blank">jeff.science(a)gmail.com<mailto:[email protected]>= </a><br><a href=3D"http://jeffhammond.github.io/" target=3D"_blank">http://= jeffhammond.github.io/<http://jeffhammond.github.io/></a></div> </div></div> --047d7b2e4058e96442052be7f662-- --===============5365574427901088839== Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ mpi-forum mailing list mpi-forum(a)lists.mpi-forum.org<mailto:[email protected]> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum --===============5365574427901088839==--

1 0

Re: [Mpi-forum] Question about MPI_Info set on communicators
by Jeff Hammond 16 Feb '16

16 Feb '16

On Tue, Feb 16, 2016 at 6:50 AM, Jeff Squyres (jsquyres) <jsquyres(a)cisco.com > wrote: > Jim -- > > Based on your answer, I'm now a bit confused. I now need to back up from > my original question and clarify what MPI_COMM_INFO_GET is supposed to > return in this scenario: > > ----- > MPI_Info_set(myinfo, "use_shared_memory", "yes"); > MPI_Comm_set_info(comm, my info); > // MPI implementation recognizes the "use_shared_memory" hint, but is > // unable to use shared memory with this communicator > MPI_Comm_get_info(comm, myinfo_returned); > ----- > > What value is in myinfo_returned for use_shared_memory? > > MPI 3.1 6.4.4 p250 says: > > "MPI_COMM_GET_INFO returns a new info object containing the hints of the > communicator associated with comm. The current setting of all hints > actually used by the system related to this communicator is returned in > info_used." > > "...all hints actually used by the system..." is actually a bit ambiguous. > > In this case, "use_shared_memory" is *recognized* by the MPI > implementation. But it wasn't able to be *used* (e.g., because no shared > memory was available). So is the value in info_used supposed to be: > > How does it know how much shared memory is required? I don't see any reason for MPI_Comm_set_info to fail on account of lack of shared memory. If zero bytes of shared memory are available, how does MPI_Comm_set_info know that this is not enough when the user actually needs? > 1. The user's requested hint (i.e., "yes"), or > I'm changing my answer to this one, in this case. For MPI_Alloc_mem and MPI_Win_allocate(_shared), my answer will be different. The only reasonable behavior is for MPI_Comm_set_info to succeed and set "use_shared_memory"="yes". Subsequent operations may behave differently depending on how much shared memory is available. I would expect the user to probe that using MPI_T. Jeff > 2. What MPI is actually using (i.e., "no"), or > 3. Since this hint wasn't "used" by the system, then the > "use_shared_memory" key shouldn't be returned in info_used > > ? > > Indeed, one could make the argument that MPI_COMM_GET_INFO is supposed to > return exactly the value(s) that the user requested (if it is *recognized* > by the implementation), and MPI_T pvars and/or cvars should be queried to > obtain the values indicating what the implementation chose to do with those > hints. > > Thoughts? > > > > On February 15, 2016 at 8:25:44 PM, Jim Dinan (james.dinan(a)gmail.com) > wrote: > > "Same" should mean that these call sequences produce the same newcomm, > > right? > > > > MPI_Comm_get_info(comm, myinfo_returned); > > MPI_Comm_dup_with_info(comm, &newcomm, myinfo_returned); > > > > and > > > > MPI_Comm_dup(comm, &newcomm); > > > > I think this is an argument for "no". > > > > I'm not sure if the question is asking whether the MPI implementation can > > change communicator info values dynamically. I'm not sure the spec > > actually addresses that possibility. If this is allowed, newcomm would > > still only get "yes" if the implementation had also toggled the info key > on > > the parent communicator. > > > > ~Jim. > > > > On Mon, Feb 15, 2016 at 5:34 PM, Jeff Squyres (jsquyres) > > wrote: > > > > > This question has come up in the Open MPI community: > > > > > > ----- > > > MPI_Info_set(myinfo, "use_shared_memory", "yes"); > > > MPI_Comm_set_info(comm, myinfo); > > > MPI_Comm_get_info(comm, myinfo_returned); > > > // Assume here that there was not enough shared memory available to use > > > shared-memory for > > > // communication on this comm, so MPI_Info_get() tells us that > > > use_shared_memory is "no". > > > > > > // ...time passes, and now assume there is more shared-memory available > > > MPI_Comm_dup_with_info(comm, &newcom); > > > MPI_comm_get_info(newcom, myinfo_returned); > > > ----- > > > > > > Should newcom have "use_shared_memory" set to "yes" or "no"? > > > > > > Arguments for "yes": > > > - the user originally asked for "yes" > > > - now there is shared memory available > > > > > > Arguments for "no": > > > - the current value for "used_shared_memory" on the parent > communicator is > > > "no" > > > > > > Which should it be? > > > > > > MPI-3.1 6.4.2 p238 says: > > > > > > "...Returns in newcomer a new communicator with the same group or > groups, > > > same topology, same info hints, any copied cached information, but a > new > > > context (see Section 6.7.1)." > > > > > > So the specific question here is: what exactly does "Same info hints" > > > mean? The user-specific hint, or the current value of the hint? > > > > > > -- > > > Jeff Squyres > > > jsquyres(a)cisco.com > > > For corporate legal information go to: > > > http://www.cisco.com/web/about/doing_business/legal/cri/ > > > _______________________________________________ > > > mpi-forum mailing list > > > mpi-forum(a)lists.mpi-forum.org > > > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum > > _______________________________________________ > > mpi-forum mailing list > > mpi-forum(a)lists.mpi-forum.org > > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum > > -- > Jeff Squyres > jsquyres(a)cisco.com > For corporate legal information go to: > http://www.cisco.com/web/about/doing_business/legal/cri/ > _______________________________________________ > mpi-forum mailing list > mpi-forum(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum > -- Jeff Hammond jeff.science(a)gmail.com http://jeffhammond.github.io/

1 0