February 2016 - mpi-forum - mailman3.cels.anl.gov

Re: [Mpi-forum] Question about MPI_Info set on communicators
by George Bosilca 18 Feb '16

18 Feb '16

OK, we'll do this. George. On Thu, Feb 18, 2016 at 3:28 PM, Jim Dinan <james.dinan(a)gmail.com> wrote: > We don't have meeting time scheduled for this at the moment, and the > agenda already looks full. We could back-fill whenever there is a free > slot, if we prepare a few slides for discussion. > > ~Jim. > > On Thu, Feb 18, 2016 at 12:56 PM, Jeff Squyres (jsquyres) < > jsquyres(a)cisco.com> wrote: > >> Jim: >> >> Is there a WG session in Chicago where we can discuss this stuff? >> >> I ask because this issue has stalled the Open MPI implementation effort >> to actually start doing interesting/useful things with MPI_COMM_SET_INFO. >> >> >> -----Original Message----- >> From: Jim Dinan <james.dinan(a)gmail.com> >> Reply: Main MPI Forum mailing list <mpi-forum(a)lists.mpi-forum.org> >> Date: February 18, 2016 at 11:04:02 AM >> To: Main MPI Forum mailing list <mpi-forum(a)lists.mpi-forum.org> >> Subject: Re: [Mpi-forum] Question about MPI_Info set on communicators >> >> > This is the semantic that we have been working with in the info >> assertions >> > proposal. But, I do see the ambiguity. >> > >> > Several things have come up that would impact the info assertions >> proposal: >> > >> > 1. If comm_get_info + dup_with_info is not equivalent to comm_dup, then >> we >> > are losing functionality by removing info propagation in dup. We may >> need >> > a new get_info routine to support both types of queries. >> > >> > 2. If comm_get_info does not return values indicating the current >> > configuration/behavior of the MPI library then we may need to adjust the >> > language we added to comm_set_info indicating that users should check >> > whether changes to info key values were accepted by the MPI library. >> > >> > ~Jim. >> > >> > On Wed, Feb 17, 2016 at 8:50 PM, Thakur, Rajeev wrote: >> > >> > > In the I/O chapter, an implementation is allowed to ignore the hints >> > > provided by the user. MPI_File_get_info returns the current value of >> hints >> > > actually used in the implementation. So suppose a user passes the >> info key >> > > “striping_unit" with value 10, but the system on which the file is >> created >> > > doesn’t support user control of striping unit, the implementation can >> > > ignore the user-provided value and, in MPI_File_get_info, return the >> > > default striping unit used for the file (which could be 8, say). >> > > >> > > Rajeev >> > > >> > > >> > > >> > > > On Feb 17, 2016, at 5:57 PM, Schulz Martin wrote: >> > > > >> > > >> >> > > >>>> See my reply to Jim: if we're supposed to provide the value of >> *the >> > > >>>> hint*, that to me sounds like the *user's input* (as opposed to >> what >> > > >>> the >> > > >>>> system chose to do with that hint). >> > > >>> >> > > >>> I agree, hints are given by the user to express what an >> application >> > > >>> does. >> > > >>> Those should never be changed by the MPI library (or no are no >> longer >> > > >>> (user) hints). >> > > >> >> > > >> Even if the MPI implementation should never change the value of >> these >> > > >> hints, my original question still stands, because >> MPI_COMM_GET_INFO's >> > > use >> > > >> of the phrase "actually used by the system" is still ambiguous >> (MPI-3.1 >> > > >> p250): >> > > >> >> > > >> ----- >> > > >> // Assume that the MPI implementation understands the "foo" info >> key >> > > >> hint, but >> > > >> // is unable to apply it after a communicator is created. >> > > >> MPI_Info_set(myinfo, "foo", "yes"); >> > > >> MPI_Comm_set_info(comm, myinfo); >> > > >> MPI_Comm_get_info(comm, myinfo_returned); >> > > >> ------ >> > > >> >> > > >> So I still don't have a definitive answer as to what is in >> > > >> myinfo_returned: >> > > >> >> > > >> 1. "foo" = "yes", because that's what the user set, and the MPI >> > > >> implementation understood the "foo" hint key (regardless of what it >> > > chose >> > > >> to do with that hint) >> > > >> --> This is what I think should happen >> > > >> >> > > >> 2. "foo" = "no", because the MPI implementation wasn't able to use >> the >> > > >> "foo" hint >> > > >> --> There's at least 2 explicit votes against this behavior >> (JeffSq, >> > > >> Martin) >> > > >> >> > > >> 3. no "foo" key is set, because the MPI implementation wasn't able >> to >> > > use >> > > >> the "foo" hint >> > > >> --> I'm not a big fan of this, because you can't tell the >> difference >> > > >> between "foo is not understood by the implementation" and "foo is >> > > >> understood by the implementation, but this hint can't be used right >> > > now". >> > > > >> > > > I would say, #3 is correct - the system doesnąt use/apply the hint, >> I.e., >> > > > the behavior is the same as if the hint wasnąt supplied. I agree, >> this is >> > > > not ideal since knowing what an implementation supports would be >> good, >> > > but >> > > > the standard never makes that distinction. >> > > > >> > > >> >> > > >> ========= >> > > >> >> > > >> Regardless of what is returned by MPI_COMM_GET_INFO, what happens >> in >> > > >> MPI_COMM_DUP? >> > > >> >> > > >> The "actually used by the system" ambiguous phrase only applies to >> > > >> MPI_COMM_GET_INFO. MPI_COMM_DUP is defined in MPI-3.1 p238: >> > > >> >> > > >> "MPI_COMM_DUP duplicates the existing communicator comm with ... >> info >> > > >> hints." >> > > >> >> > > >> It doesn't have any scoping language about *which* hints are >> copied -- >> > > it >> > > >> pretty much implies that *all* hints are copied. >> > > >> >> > > >> As such, I think that "foo"="yes" should be propagated to the new >> > > >> communicator. >> > > > >> > > > Yes, I agree - even if the hint is not used and hence not returned >> by >> > > > MPI_COMM_GET_INFO, itąs still set on the communicator. The user set >> it >> > > and >> > > > expects it to be set and so it should be part of the duplication. >> > > > >> > > >> Which is another reason that I think that MPI_COMM_GET_INFO should >> also >> > > >> return "foo"="yes" in myinfo_returned (i.e., so that >> MPI_COMM_GET_INFO >> > > >> and MPI_COMM_DUP both behave the same way). >> > > > >> > > > I agree, thatąs what it should do - GET and SET should be matching, >> but >> > > > thatąs not how the standard is written (sadly). >> > > > >> > > > The idea behavior would be three calls, IMHO: >> > > > >> > > > MPI_COMM_GET_INFO - return all hints set >> > > > MPI_COMM_GET_INFO_UNDERSTOOD - return all hints that could be used >> by an >> > > > implementation >> > > > MPI_COMM_GET_INFO_USED - return all hints used (thatąs the current >> > > > behavior of MPI_COMM_GET_INFO) >> > > > >> > > > doing this, though, throws into the usual backwards compatibility >> > > > problems, though. >> > > > >> > > > Martin >> > > > >> > > > >> > > > >> > > >> >> > > >> Thoughts? >> > > >> >> > > >> -- >> > > >> Jeff Squyres >> > > >> jsquyres(a)cisco.com >> > > >> For corporate legal information go to: >> > > >> http://www.cisco.com/web/about/doing_business/legal/cri/ >> > > > >> > > > >> > > > _______________________________________________ >> > > > mpi-forum mailing list >> > > > mpi-forum(a)lists.mpi-forum.org >> > > > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum >> > > >> > > _______________________________________________ >> > > mpi-forum mailing list >> > > mpi-forum(a)lists.mpi-forum.org >> > > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum >> > > >> > _______________________________________________ >> > mpi-forum mailing list >> > mpi-forum(a)lists.mpi-forum.org >> > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum >> >> -- >> Jeff Squyres >> jsquyres(a)cisco.com >> For corporate legal information go to: >> http://www.cisco.com/web/about/doing_business/legal/cri/ >> > > > _______________________________________________ > mpi-forum mailing list > mpi-forum(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum >

1 0

Re: [Mpi-forum] Question about MPI_Info set on communicators
by Jim Dinan 18 Feb '16

18 Feb '16

We don't have meeting time scheduled for this at the moment, and the agenda already looks full. We could back-fill whenever there is a free slot, if we prepare a few slides for discussion. ~Jim. On Thu, Feb 18, 2016 at 12:56 PM, Jeff Squyres (jsquyres) < jsquyres(a)cisco.com> wrote: > Jim: > > Is there a WG session in Chicago where we can discuss this stuff? > > I ask because this issue has stalled the Open MPI implementation effort to > actually start doing interesting/useful things with MPI_COMM_SET_INFO. > > > -----Original Message----- > From: Jim Dinan <james.dinan(a)gmail.com> > Reply: Main MPI Forum mailing list <mpi-forum(a)lists.mpi-forum.org> > Date: February 18, 2016 at 11:04:02 AM > To: Main MPI Forum mailing list <mpi-forum(a)lists.mpi-forum.org> > Subject: Re: [Mpi-forum] Question about MPI_Info set on communicators > > > This is the semantic that we have been working with in the info > assertions > > proposal. But, I do see the ambiguity. > > > > Several things have come up that would impact the info assertions > proposal: > > > > 1. If comm_get_info + dup_with_info is not equivalent to comm_dup, then > we > > are losing functionality by removing info propagation in dup. We may need > > a new get_info routine to support both types of queries. > > > > 2. If comm_get_info does not return values indicating the current > > configuration/behavior of the MPI library then we may need to adjust the > > language we added to comm_set_info indicating that users should check > > whether changes to info key values were accepted by the MPI library. > > > > ~Jim. > > > > On Wed, Feb 17, 2016 at 8:50 PM, Thakur, Rajeev wrote: > > > > > In the I/O chapter, an implementation is allowed to ignore the hints > > > provided by the user. MPI_File_get_info returns the current value of > hints > > > actually used in the implementation. So suppose a user passes the info > key > > > “striping_unit" with value 10, but the system on which the file is > created > > > doesn’t support user control of striping unit, the implementation can > > > ignore the user-provided value and, in MPI_File_get_info, return the > > > default striping unit used for the file (which could be 8, say). > > > > > > Rajeev > > > > > > > > > > > > > On Feb 17, 2016, at 5:57 PM, Schulz Martin wrote: > > > > > > > >> > > > >>>> See my reply to Jim: if we're supposed to provide the value of > *the > > > >>>> hint*, that to me sounds like the *user's input* (as opposed to > what > > > >>> the > > > >>>> system chose to do with that hint). > > > >>> > > > >>> I agree, hints are given by the user to express what an application > > > >>> does. > > > >>> Those should never be changed by the MPI library (or no are no > longer > > > >>> (user) hints). > > > >> > > > >> Even if the MPI implementation should never change the value of > these > > > >> hints, my original question still stands, because > MPI_COMM_GET_INFO's > > > use > > > >> of the phrase "actually used by the system" is still ambiguous > (MPI-3.1 > > > >> p250): > > > >> > > > >> ----- > > > >> // Assume that the MPI implementation understands the "foo" info key > > > >> hint, but > > > >> // is unable to apply it after a communicator is created. > > > >> MPI_Info_set(myinfo, "foo", "yes"); > > > >> MPI_Comm_set_info(comm, myinfo); > > > >> MPI_Comm_get_info(comm, myinfo_returned); > > > >> ------ > > > >> > > > >> So I still don't have a definitive answer as to what is in > > > >> myinfo_returned: > > > >> > > > >> 1. "foo" = "yes", because that's what the user set, and the MPI > > > >> implementation understood the "foo" hint key (regardless of what it > > > chose > > > >> to do with that hint) > > > >> --> This is what I think should happen > > > >> > > > >> 2. "foo" = "no", because the MPI implementation wasn't able to use > the > > > >> "foo" hint > > > >> --> There's at least 2 explicit votes against this behavior (JeffSq, > > > >> Martin) > > > >> > > > >> 3. no "foo" key is set, because the MPI implementation wasn't able > to > > > use > > > >> the "foo" hint > > > >> --> I'm not a big fan of this, because you can't tell the difference > > > >> between "foo is not understood by the implementation" and "foo is > > > >> understood by the implementation, but this hint can't be used right > > > now". > > > > > > > > I would say, #3 is correct - the system doesnąt use/apply the hint, > I.e., > > > > the behavior is the same as if the hint wasnąt supplied. I agree, > this is > > > > not ideal since knowing what an implementation supports would be > good, > > > but > > > > the standard never makes that distinction. > > > > > > > >> > > > >> ========= > > > >> > > > >> Regardless of what is returned by MPI_COMM_GET_INFO, what happens in > > > >> MPI_COMM_DUP? > > > >> > > > >> The "actually used by the system" ambiguous phrase only applies to > > > >> MPI_COMM_GET_INFO. MPI_COMM_DUP is defined in MPI-3.1 p238: > > > >> > > > >> "MPI_COMM_DUP duplicates the existing communicator comm with ... > info > > > >> hints." > > > >> > > > >> It doesn't have any scoping language about *which* hints are copied > -- > > > it > > > >> pretty much implies that *all* hints are copied. > > > >> > > > >> As such, I think that "foo"="yes" should be propagated to the new > > > >> communicator. > > > > > > > > Yes, I agree - even if the hint is not used and hence not returned by > > > > MPI_COMM_GET_INFO, itąs still set on the communicator. The user set > it > > > and > > > > expects it to be set and so it should be part of the duplication. > > > > > > > >> Which is another reason that I think that MPI_COMM_GET_INFO should > also > > > >> return "foo"="yes" in myinfo_returned (i.e., so that > MPI_COMM_GET_INFO > > > >> and MPI_COMM_DUP both behave the same way). > > > > > > > > I agree, thatąs what it should do - GET and SET should be matching, > but > > > > thatąs not how the standard is written (sadly). > > > > > > > > The idea behavior would be three calls, IMHO: > > > > > > > > MPI_COMM_GET_INFO - return all hints set > > > > MPI_COMM_GET_INFO_UNDERSTOOD - return all hints that could be used > by an > > > > implementation > > > > MPI_COMM_GET_INFO_USED - return all hints used (thatąs the current > > > > behavior of MPI_COMM_GET_INFO) > > > > > > > > doing this, though, throws into the usual backwards compatibility > > > > problems, though. > > > > > > > > Martin > > > > > > > > > > > > > > > >> > > > >> Thoughts? > > > >> > > > >> -- > > > >> Jeff Squyres > > > >> jsquyres(a)cisco.com > > > >> For corporate legal information go to: > > > >> http://www.cisco.com/web/about/doing_business/legal/cri/ > > > > > > > > > > > > _______________________________________________ > > > > mpi-forum mailing list > > > > mpi-forum(a)lists.mpi-forum.org > > > > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum > > > > > > _______________________________________________ > > > mpi-forum mailing list > > > mpi-forum(a)lists.mpi-forum.org > > > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum > > > > > _______________________________________________ > > mpi-forum mailing list > > mpi-forum(a)lists.mpi-forum.org > > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum > > -- > Jeff Squyres > jsquyres(a)cisco.com > For corporate legal information go to: > http://www.cisco.com/web/about/doing_business/legal/cri/ >

1 0

Re: [Mpi-forum] Question about MPI_Info set on communicators
by Jeff Squyres (jsquyres) 18 Feb '16

18 Feb '16

Jim: Is there a WG session in Chicago where we can discuss this stuff? I ask because this issue has stalled the Open MPI implementation effort to actually start doing interesting/useful things with MPI_COMM_SET_INFO. -----Original Message----- From: Jim Dinan <james.dinan(a)gmail.com> Reply: Main MPI Forum mailing list <mpi-forum(a)lists.mpi-forum.org> Date: February 18, 2016 at 11:04:02 AM To: Main MPI Forum mailing list <mpi-forum(a)lists.mpi-forum.org> Subject: Re: [Mpi-forum] Question about MPI_Info set on communicators > This is the semantic that we have been working with in the info assertions > proposal. But, I do see the ambiguity. > > Several things have come up that would impact the info assertions proposal: > > 1. If comm_get_info + dup_with_info is not equivalent to comm_dup, then we > are losing functionality by removing info propagation in dup. We may need > a new get_info routine to support both types of queries. > > 2. If comm_get_info does not return values indicating the current > configuration/behavior of the MPI library then we may need to adjust the > language we added to comm_set_info indicating that users should check > whether changes to info key values were accepted by the MPI library. > > ~Jim. > > On Wed, Feb 17, 2016 at 8:50 PM, Thakur, Rajeev wrote: > > > In the I/O chapter, an implementation is allowed to ignore the hints > > provided by the user. MPI_File_get_info returns the current value of hints > > actually used in the implementation. So suppose a user passes the info key > > “striping_unit" with value 10, but the system on which the file is created > > doesn’t support user control of striping unit, the implementation can > > ignore the user-provided value and, in MPI_File_get_info, return the > > default striping unit used for the file (which could be 8, say). > > > > Rajeev > > > > > > > > > On Feb 17, 2016, at 5:57 PM, Schulz Martin wrote: > > > > > >> > > >>>> See my reply to Jim: if we're supposed to provide the value of *the > > >>>> hint*, that to me sounds like the *user's input* (as opposed to what > > >>> the > > >>>> system chose to do with that hint). > > >>> > > >>> I agree, hints are given by the user to express what an application > > >>> does. > > >>> Those should never be changed by the MPI library (or no are no longer > > >>> (user) hints). > > >> > > >> Even if the MPI implementation should never change the value of these > > >> hints, my original question still stands, because MPI_COMM_GET_INFO's > > use > > >> of the phrase "actually used by the system" is still ambiguous (MPI-3.1 > > >> p250): > > >> > > >> ----- > > >> // Assume that the MPI implementation understands the "foo" info key > > >> hint, but > > >> // is unable to apply it after a communicator is created. > > >> MPI_Info_set(myinfo, "foo", "yes"); > > >> MPI_Comm_set_info(comm, myinfo); > > >> MPI_Comm_get_info(comm, myinfo_returned); > > >> ------ > > >> > > >> So I still don't have a definitive answer as to what is in > > >> myinfo_returned: > > >> > > >> 1. "foo" = "yes", because that's what the user set, and the MPI > > >> implementation understood the "foo" hint key (regardless of what it > > chose > > >> to do with that hint) > > >> --> This is what I think should happen > > >> > > >> 2. "foo" = "no", because the MPI implementation wasn't able to use the > > >> "foo" hint > > >> --> There's at least 2 explicit votes against this behavior (JeffSq, > > >> Martin) > > >> > > >> 3. no "foo" key is set, because the MPI implementation wasn't able to > > use > > >> the "foo" hint > > >> --> I'm not a big fan of this, because you can't tell the difference > > >> between "foo is not understood by the implementation" and "foo is > > >> understood by the implementation, but this hint can't be used right > > now". > > > > > > I would say, #3 is correct - the system doesnąt use/apply the hint, I.e., > > > the behavior is the same as if the hint wasnąt supplied. I agree, this is > > > not ideal since knowing what an implementation supports would be good, > > but > > > the standard never makes that distinction. > > > > > >> > > >> ========= > > >> > > >> Regardless of what is returned by MPI_COMM_GET_INFO, what happens in > > >> MPI_COMM_DUP? > > >> > > >> The "actually used by the system" ambiguous phrase only applies to > > >> MPI_COMM_GET_INFO. MPI_COMM_DUP is defined in MPI-3.1 p238: > > >> > > >> "MPI_COMM_DUP duplicates the existing communicator comm with ... info > > >> hints." > > >> > > >> It doesn't have any scoping language about *which* hints are copied -- > > it > > >> pretty much implies that *all* hints are copied. > > >> > > >> As such, I think that "foo"="yes" should be propagated to the new > > >> communicator. > > > > > > Yes, I agree - even if the hint is not used and hence not returned by > > > MPI_COMM_GET_INFO, itąs still set on the communicator. The user set it > > and > > > expects it to be set and so it should be part of the duplication. > > > > > >> Which is another reason that I think that MPI_COMM_GET_INFO should also > > >> return "foo"="yes" in myinfo_returned (i.e., so that MPI_COMM_GET_INFO > > >> and MPI_COMM_DUP both behave the same way). > > > > > > I agree, thatąs what it should do - GET and SET should be matching, but > > > thatąs not how the standard is written (sadly). > > > > > > The idea behavior would be three calls, IMHO: > > > > > > MPI_COMM_GET_INFO - return all hints set > > > MPI_COMM_GET_INFO_UNDERSTOOD - return all hints that could be used by an > > > implementation > > > MPI_COMM_GET_INFO_USED - return all hints used (thatąs the current > > > behavior of MPI_COMM_GET_INFO) > > > > > > doing this, though, throws into the usual backwards compatibility > > > problems, though. > > > > > > Martin > > > > > > > > > > > >> > > >> Thoughts? > > >> > > >> -- > > >> Jeff Squyres > > >> jsquyres(a)cisco.com > > >> For corporate legal information go to: > > >> http://www.cisco.com/web/about/doing_business/legal/cri/ > > > > > > > > > _______________________________________________ > > > mpi-forum mailing list > > > mpi-forum(a)lists.mpi-forum.org > > > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum > > > > _______________________________________________ > > mpi-forum mailing list > > mpi-forum(a)lists.mpi-forum.org > > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum > > > _______________________________________________ > mpi-forum mailing list > mpi-forum(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum -- Jeff Squyres jsquyres(a)cisco.com For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/

1 0

Re: [Mpi-forum] Question about MPI_Info set on communicators
by Jim Dinan 18 Feb '16

18 Feb '16

This is the semantic that we have been working with in the info assertions proposal. But, I do see the ambiguity. Several things have come up that would impact the info assertions proposal: 1. If comm_get_info + dup_with_info is not equivalent to comm_dup, then we are losing functionality by removing info propagation in dup. We may need a new get_info routine to support both types of queries. 2. If comm_get_info does not return values indicating the current configuration/behavior of the MPI library then we may need to adjust the language we added to comm_set_info indicating that users should check whether changes to info key values were accepted by the MPI library. ~Jim. On Wed, Feb 17, 2016 at 8:50 PM, Thakur, Rajeev <thakur(a)mcs.anl.gov> wrote: > In the I/O chapter, an implementation is allowed to ignore the hints > provided by the user. MPI_File_get_info returns the current value of hints > actually used in the implementation. So suppose a user passes the info key > “striping_unit" with value 10, but the system on which the file is created > doesn’t support user control of striping unit, the implementation can > ignore the user-provided value and, in MPI_File_get_info, return the > default striping unit used for the file (which could be 8, say). > > Rajeev > > > > > On Feb 17, 2016, at 5:57 PM, Schulz Martin <schulzm(a)llnl.gov> wrote: > > > >> > >>>> See my reply to Jim: if we're supposed to provide the value of *the > >>>> hint*, that to me sounds like the *user's input* (as opposed to what > >>> the > >>>> system chose to do with that hint). > >>> > >>> I agree, hints are given by the user to express what an application > >>> does. > >>> Those should never be changed by the MPI library (or no are no longer > >>> (user) hints). > >> > >> Even if the MPI implementation should never change the value of these > >> hints, my original question still stands, because MPI_COMM_GET_INFO's > use > >> of the phrase "actually used by the system" is still ambiguous (MPI-3.1 > >> p250): > >> > >> ----- > >> // Assume that the MPI implementation understands the "foo" info key > >> hint, but > >> // is unable to apply it after a communicator is created. > >> MPI_Info_set(myinfo, "foo", "yes"); > >> MPI_Comm_set_info(comm, myinfo); > >> MPI_Comm_get_info(comm, myinfo_returned); > >> ------ > >> > >> So I still don't have a definitive answer as to what is in > >> myinfo_returned: > >> > >> 1. "foo" = "yes", because that's what the user set, and the MPI > >> implementation understood the "foo" hint key (regardless of what it > chose > >> to do with that hint) > >> --> This is what I think should happen > >> > >> 2. "foo" = "no", because the MPI implementation wasn't able to use the > >> "foo" hint > >> --> There's at least 2 explicit votes against this behavior (JeffSq, > >> Martin) > >> > >> 3. no "foo" key is set, because the MPI implementation wasn't able to > use > >> the "foo" hint > >> --> I'm not a big fan of this, because you can't tell the difference > >> between "foo is not understood by the implementation" and "foo is > >> understood by the implementation, but this hint can't be used right > now". > > > > I would say, #3 is correct - the system doesnąt use/apply the hint, I.e., > > the behavior is the same as if the hint wasnąt supplied. I agree, this is > > not ideal since knowing what an implementation supports would be good, > but > > the standard never makes that distinction. > > > >> > >> ========= > >> > >> Regardless of what is returned by MPI_COMM_GET_INFO, what happens in > >> MPI_COMM_DUP? > >> > >> The "actually used by the system" ambiguous phrase only applies to > >> MPI_COMM_GET_INFO. MPI_COMM_DUP is defined in MPI-3.1 p238: > >> > >> "MPI_COMM_DUP duplicates the existing communicator comm with ... info > >> hints." > >> > >> It doesn't have any scoping language about *which* hints are copied -- > it > >> pretty much implies that *all* hints are copied. > >> > >> As such, I think that "foo"="yes" should be propagated to the new > >> communicator. > > > > Yes, I agree - even if the hint is not used and hence not returned by > > MPI_COMM_GET_INFO, itąs still set on the communicator. The user set it > and > > expects it to be set and so it should be part of the duplication. > > > >> Which is another reason that I think that MPI_COMM_GET_INFO should also > >> return "foo"="yes" in myinfo_returned (i.e., so that MPI_COMM_GET_INFO > >> and MPI_COMM_DUP both behave the same way). > > > > I agree, thatąs what it should do - GET and SET should be matching, but > > thatąs not how the standard is written (sadly). > > > > The idea behavior would be three calls, IMHO: > > > > MPI_COMM_GET_INFO - return all hints set > > MPI_COMM_GET_INFO_UNDERSTOOD - return all hints that could be used by an > > implementation > > MPI_COMM_GET_INFO_USED - return all hints used (thatąs the current > > behavior of MPI_COMM_GET_INFO) > > > > doing this, though, throws into the usual backwards compatibility > > problems, though. > > > > Martin > > > > > > > >> > >> Thoughts? > >> > >> -- > >> Jeff Squyres > >> jsquyres(a)cisco.com > >> For corporate legal information go to: > >> http://www.cisco.com/web/about/doing_business/legal/cri/ > > > > > > _______________________________________________ > > mpi-forum mailing list > > mpi-forum(a)lists.mpi-forum.org > > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum > > _______________________________________________ > mpi-forum mailing list > mpi-forum(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum >

1 0

Re: [Mpi-forum] use of communicator in MPI_Pack and MPI_Unpack
by Michael Rezny 17 Feb '16

17 Feb '16

Hi Rajeev, thank you for the clarification. It is most helpful. kindest regards Mike On 18 February 2016 at 13:55, Thakur, Rajeev <thakur(a)mcs.anl.gov> wrote: > Your understanding is correct in that the communicator you pass needs to > contain the processes to which the message is sent or received from. The > standard is saying it in a simple way. > > But I would suggest reconsidering whether you really need pack and unpack > in the first place. They are rarely used functions, and there is an > associated performance penalty. You could try sending the messages > directly, e.g., by using derived datatypes if the data is noncontiguous. > > Rajeev > > > > On Feb 17, 2016, at 8:38 PM, Michael Rezny <michael.rezny(a)monash.edu> > wrote: > > > > Hi Rajeev, > > thanks for the prompt response. > > > > I am attempting to ensure that our code conforms to the standard. > > > > Would it be correct to say that, for the MPI standard, the communicator > used in MPI_Pack must > > be a superset of the ranks in the communicator used subsequently by MPI > calls that use the packed buffer? > > > > And likewise, is it correct to state that MPI_Unpack must use a > communicator that is a superset of the communicator used to > > receive or create the buffer that is input to MPI_Unpack? > > > > If that is the case, then would MPI_COMM_WORLD always bea valid option > for use as a communicator > > in MPI_Pack and MPI_Unpack? > > > > Then, as you said, in a homogeneous system, there should be no > disadvantage in specifying > > MPI_COMM_WORLD. > > > > Perhaps the standard and man pages for MPI_Pack could be restated: > > > The comm argument must include all (be a superset) of all the > communicator"s" that will be > > > subsequently used for sending the packed message. > > > > And something similar for MPI_Unpack. > > > > kindest regards > > Mike > > > > On 18 February 2016 at 12:57, Thakur, Rajeev <thakur(a)mcs.anl.gov> wrote: > > You can use MPI_COMM_WORLD as the communicator. If you are running on a > homogeneous system, there shouldn’t be any performance issue in using > MPI_COMM_WORLD as the communicator either. > > > > Rajeev > > > > > On Feb 17, 2016, at 6:09 PM, Michael Rezny <michael.rezny(a)monash.edu> > wrote: > > > > > > Hi, > > > I have a question regarding the use of a communicator in MPI_Pack and > MPI_Unpack. > > > I have googled and search the MPI mailing lists but haven't found an > answer. > > > > > > The OpenMPI man page for MPI_Pack states: > > > The comm argument is the communicator that will be > > > subsequently used for sending the packed message. > > > > > > And something similar for MPI_Unpack. > > > > > > Which seems to reflect what is defined in the MPI standards. > > > > > > Basically, what I would like to do is to have an MPI rank as a coupler. > > > That is, receiving a packed message from one application and > > > passing it to another application. > > > > > > Comm_A would be the ranks of application A plus the coupler and > > > Comm_B would be the ranks of application B plus the coupler. > > > > > > In this case, MPI_COMM_WORLD = Comm_A U Comm_B. > > > > > > I would like to have a solution that does not have the coupler > unpacking > > > the buffer received from application A and then packing it to send it > to application B > > > > > > I just found a very nice explanation of the need for a communicator in > Pack / Unpack on page 135 of Using MPI Portable Parallel Programming with > the Message-Passing Interface > > > second edition by Gropp, Lusk, and Skjellum. > > > > > > If I understand this correctly, it would be legal to use > MPI_COMM_WORLD as the communicator > > > for MPI_Pack and MPI_Unpack. The only downside is that one might not > get the optimal performance. > > > > > > The alternative we are using is to use MPI_Pack_external and > MPI_Unpack_external > > > which requires no communicator. But the downside is that on > homogeneous little-endian HPCs, > > > this will always entail byte swapping on packing and unpacking. > > > > > > So my question is: What is the correct and optimal way to implement > passing packed messages > > > in the coupling example described above? > > > > > > kindest regards > > > Mike > > > _______________________________________________ > > > mpi-forum mailing list > > > mpi-forum(a)lists.mpi-forum.org > > > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum > > > > _______________________________________________ > > mpi-forum mailing list > > mpi-forum(a)lists.mpi-forum.org > > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum > > > > _______________________________________________ > > mpi-forum mailing list > > mpi-forum(a)lists.mpi-forum.org > > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum > > _______________________________________________ > mpi-forum mailing list > mpi-forum(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum

1 0

Re: [Mpi-forum] use of communicator in MPI_Pack and MPI_Unpack
by Thakur, Rajeev 17 Feb '16

17 Feb '16

Your understanding is correct in that the communicator you pass needs to contain the processes to which the message is sent or received from. The standard is saying it in a simple way. But I would suggest reconsidering whether you really need pack and unpack in the first place. They are rarely used functions, and there is an associated performance penalty. You could try sending the messages directly, e.g., by using derived datatypes if the data is noncontiguous. Rajeev > On Feb 17, 2016, at 8:38 PM, Michael Rezny <michael.rezny(a)monash.edu> wrote: > > Hi Rajeev, > thanks for the prompt response. > > I am attempting to ensure that our code conforms to the standard. > > Would it be correct to say that, for the MPI standard, the communicator used in MPI_Pack must > be a superset of the ranks in the communicator used subsequently by MPI calls that use the packed buffer? > > And likewise, is it correct to state that MPI_Unpack must use a communicator that is a superset of the communicator used to > receive or create the buffer that is input to MPI_Unpack? > > If that is the case, then would MPI_COMM_WORLD always bea valid option for use as a communicator > in MPI_Pack and MPI_Unpack? > > Then, as you said, in a homogeneous system, there should be no disadvantage in specifying > MPI_COMM_WORLD. > > Perhaps the standard and man pages for MPI_Pack could be restated: > > The comm argument must include all (be a superset) of all the communicator"s" that will be > > subsequently used for sending the packed message. > > And something similar for MPI_Unpack. > > kindest regards > Mike > > On 18 February 2016 at 12:57, Thakur, Rajeev <thakur(a)mcs.anl.gov> wrote: > You can use MPI_COMM_WORLD as the communicator. If you are running on a homogeneous system, there shouldn’t be any performance issue in using MPI_COMM_WORLD as the communicator either. > > Rajeev > > > On Feb 17, 2016, at 6:09 PM, Michael Rezny <michael.rezny(a)monash.edu> wrote: > > > > Hi, > > I have a question regarding the use of a communicator in MPI_Pack and MPI_Unpack. > > I have googled and search the MPI mailing lists but haven't found an answer. > > > > The OpenMPI man page for MPI_Pack states: > > The comm argument is the communicator that will be > > subsequently used for sending the packed message. > > > > And something similar for MPI_Unpack. > > > > Which seems to reflect what is defined in the MPI standards. > > > > Basically, what I would like to do is to have an MPI rank as a coupler. > > That is, receiving a packed message from one application and > > passing it to another application. > > > > Comm_A would be the ranks of application A plus the coupler and > > Comm_B would be the ranks of application B plus the coupler. > > > > In this case, MPI_COMM_WORLD = Comm_A U Comm_B. > > > > I would like to have a solution that does not have the coupler unpacking > > the buffer received from application A and then packing it to send it to application B > > > > I just found a very nice explanation of the need for a communicator in Pack / Unpack on page 135 of Using MPI Portable Parallel Programming with the Message-Passing Interface > > second edition by Gropp, Lusk, and Skjellum. > > > > If I understand this correctly, it would be legal to use MPI_COMM_WORLD as the communicator > > for MPI_Pack and MPI_Unpack. The only downside is that one might not get the optimal performance. > > > > The alternative we are using is to use MPI_Pack_external and MPI_Unpack_external > > which requires no communicator. But the downside is that on homogeneous little-endian HPCs, > > this will always entail byte swapping on packing and unpacking. > > > > So my question is: What is the correct and optimal way to implement passing packed messages > > in the coupling example described above? > > > > kindest regards > > Mike > > _______________________________________________ > > mpi-forum mailing list > > mpi-forum(a)lists.mpi-forum.org > > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum > > _______________________________________________ > mpi-forum mailing list > mpi-forum(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum > > _______________________________________________ > mpi-forum mailing list > mpi-forum(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum

1 0

Re: [Mpi-forum] use of communicator in MPI_Pack and MPI_Unpack
by Michael Rezny 17 Feb '16

17 Feb '16

Hi Rajeev, thanks for the prompt response. I am attempting to ensure that our code conforms to the standard. Would it be correct to say that, for the MPI standard, the communicator used in MPI_Pack must be a superset of the ranks in the communicator used subsequently by MPI calls that use the packed buffer? And likewise, is it correct to state that MPI_Unpack must use a communicator that is a superset of the communicator used to receive or create the buffer that is input to MPI_Unpack? If that is the case, then would MPI_COMM_WORLD always bea valid option for use as a communicator in MPI_Pack and MPI_Unpack? Then, as you said, in a homogeneous system, there should be no disadvantage in specifying MPI_COMM_WORLD. Perhaps the standard and man pages for MPI_Pack could be restated: > The comm argument must include all (be a superset) of all the communicator"s" that will be > subsequently used for sending the packed message. And something similar for MPI_Unpack. kindest regards Mike On 18 February 2016 at 12:57, Thakur, Rajeev <thakur(a)mcs.anl.gov> wrote: > You can use MPI_COMM_WORLD as the communicator. If you are running on a > homogeneous system, there shouldn’t be any performance issue in using > MPI_COMM_WORLD as the communicator either. > > Rajeev > > > On Feb 17, 2016, at 6:09 PM, Michael Rezny <michael.rezny(a)monash.edu> > wrote: > > > > Hi, > > I have a question regarding the use of a communicator in MPI_Pack and > MPI_Unpack. > > I have googled and search the MPI mailing lists but haven't found an > answer. > > > > The OpenMPI man page for MPI_Pack states: > > The comm argument is the communicator that will be > > subsequently used for sending the packed message. > > > > And something similar for MPI_Unpack. > > > > Which seems to reflect what is defined in the MPI standards. > > > > Basically, what I would like to do is to have an MPI rank as a coupler. > > That is, receiving a packed message from one application and > > passing it to another application. > > > > Comm_A would be the ranks of application A plus the coupler and > > Comm_B would be the ranks of application B plus the coupler. > > > > In this case, MPI_COMM_WORLD = Comm_A U Comm_B. > > > > I would like to have a solution that does not have the coupler unpacking > > the buffer received from application A and then packing it to send it to > application B > > > > I just found a very nice explanation of the need for a communicator in > Pack / Unpack on page 135 of Using MPI Portable Parallel Programming with > the Message-Passing Interface > > second edition by Gropp, Lusk, and Skjellum. > > > > If I understand this correctly, it would be legal to use MPI_COMM_WORLD > as the communicator > > for MPI_Pack and MPI_Unpack. The only downside is that one might not get > the optimal performance. > > > > The alternative we are using is to use MPI_Pack_external and > MPI_Unpack_external > > which requires no communicator. But the downside is that on homogeneous > little-endian HPCs, > > this will always entail byte swapping on packing and unpacking. > > > > So my question is: What is the correct and optimal way to implement > passing packed messages > > in the coupling example described above? > > > > kindest regards > > Mike > > _______________________________________________ > > mpi-forum mailing list > > mpi-forum(a)lists.mpi-forum.org > > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum > > _______________________________________________ > mpi-forum mailing list > mpi-forum(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum

1 0

Re: [Mpi-forum] use of communicator in MPI_Pack and MPI_Unpack
by Thakur, Rajeev 17 Feb '16

17 Feb '16

You can use MPI_COMM_WORLD as the communicator. If you are running on a homogeneous system, there shouldn’t be any performance issue in using MPI_COMM_WORLD as the communicator either. Rajeev > On Feb 17, 2016, at 6:09 PM, Michael Rezny <michael.rezny(a)monash.edu> wrote: > > Hi, > I have a question regarding the use of a communicator in MPI_Pack and MPI_Unpack. > I have googled and search the MPI mailing lists but haven't found an answer. > > The OpenMPI man page for MPI_Pack states: > The comm argument is the communicator that will be > subsequently used for sending the packed message. > > And something similar for MPI_Unpack. > > Which seems to reflect what is defined in the MPI standards. > > Basically, what I would like to do is to have an MPI rank as a coupler. > That is, receiving a packed message from one application and > passing it to another application. > > Comm_A would be the ranks of application A plus the coupler and > Comm_B would be the ranks of application B plus the coupler. > > In this case, MPI_COMM_WORLD = Comm_A U Comm_B. > > I would like to have a solution that does not have the coupler unpacking > the buffer received from application A and then packing it to send it to application B > > I just found a very nice explanation of the need for a communicator in Pack / Unpack on page 135 of Using MPI Portable Parallel Programming with the Message-Passing Interface > second edition by Gropp, Lusk, and Skjellum. > > If I understand this correctly, it would be legal to use MPI_COMM_WORLD as the communicator > for MPI_Pack and MPI_Unpack. The only downside is that one might not get the optimal performance. > > The alternative we are using is to use MPI_Pack_external and MPI_Unpack_external > which requires no communicator. But the downside is that on homogeneous little-endian HPCs, > this will always entail byte swapping on packing and unpacking. > > So my question is: What is the correct and optimal way to implement passing packed messages > in the coupling example described above? > > kindest regards > Mike > _______________________________________________ > mpi-forum mailing list > mpi-forum(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum

1 0

Re: [Mpi-forum] Question about MPI_Info set on communicators
by Thakur, Rajeev 17 Feb '16

17 Feb '16

In the I/O chapter, an implementation is allowed to ignore the hints provided by the user. MPI_File_get_info returns the current value of hints actually used in the implementation. So suppose a user passes the info key “striping_unit" with value 10, but the system on which the file is created doesn’t support user control of striping unit, the implementation can ignore the user-provided value and, in MPI_File_get_info, return the default striping unit used for the file (which could be 8, say). Rajeev > On Feb 17, 2016, at 5:57 PM, Schulz Martin <schulzm(a)llnl.gov> wrote: > >> >>>> See my reply to Jim: if we're supposed to provide the value of *the >>>> hint*, that to me sounds like the *user's input* (as opposed to what >>> the >>>> system chose to do with that hint). >>> >>> I agree, hints are given by the user to express what an application >>> does. >>> Those should never be changed by the MPI library (or no are no longer >>> (user) hints). >> >> Even if the MPI implementation should never change the value of these >> hints, my original question still stands, because MPI_COMM_GET_INFO's use >> of the phrase "actually used by the system" is still ambiguous (MPI-3.1 >> p250): >> >> ----- >> // Assume that the MPI implementation understands the "foo" info key >> hint, but >> // is unable to apply it after a communicator is created. >> MPI_Info_set(myinfo, "foo", "yes"); >> MPI_Comm_set_info(comm, myinfo); >> MPI_Comm_get_info(comm, myinfo_returned); >> ------ >> >> So I still don't have a definitive answer as to what is in >> myinfo_returned: >> >> 1. "foo" = "yes", because that's what the user set, and the MPI >> implementation understood the "foo" hint key (regardless of what it chose >> to do with that hint) >> --> This is what I think should happen >> >> 2. "foo" = "no", because the MPI implementation wasn't able to use the >> "foo" hint >> --> There's at least 2 explicit votes against this behavior (JeffSq, >> Martin) >> >> 3. no "foo" key is set, because the MPI implementation wasn't able to use >> the "foo" hint >> --> I'm not a big fan of this, because you can't tell the difference >> between "foo is not understood by the implementation" and "foo is >> understood by the implementation, but this hint can't be used right now". > > I would say, #3 is correct - the system doesnąt use/apply the hint, I.e., > the behavior is the same as if the hint wasnąt supplied. I agree, this is > not ideal since knowing what an implementation supports would be good, but > the standard never makes that distinction. > >> >> ========= >> >> Regardless of what is returned by MPI_COMM_GET_INFO, what happens in >> MPI_COMM_DUP? >> >> The "actually used by the system" ambiguous phrase only applies to >> MPI_COMM_GET_INFO. MPI_COMM_DUP is defined in MPI-3.1 p238: >> >> "MPI_COMM_DUP duplicates the existing communicator comm with ... info >> hints." >> >> It doesn't have any scoping language about *which* hints are copied -- it >> pretty much implies that *all* hints are copied. >> >> As such, I think that "foo"="yes" should be propagated to the new >> communicator. > > Yes, I agree - even if the hint is not used and hence not returned by > MPI_COMM_GET_INFO, itąs still set on the communicator. The user set it and > expects it to be set and so it should be part of the duplication. > >> Which is another reason that I think that MPI_COMM_GET_INFO should also >> return "foo"="yes" in myinfo_returned (i.e., so that MPI_COMM_GET_INFO >> and MPI_COMM_DUP both behave the same way). > > I agree, thatąs what it should do - GET and SET should be matching, but > thatąs not how the standard is written (sadly). > > The idea behavior would be three calls, IMHO: > > MPI_COMM_GET_INFO - return all hints set > MPI_COMM_GET_INFO_UNDERSTOOD - return all hints that could be used by an > implementation > MPI_COMM_GET_INFO_USED - return all hints used (thatąs the current > behavior of MPI_COMM_GET_INFO) > > doing this, though, throws into the usual backwards compatibility > problems, though. > > Martin > > > >> >> Thoughts? >> >> -- >> Jeff Squyres >> jsquyres(a)cisco.com >> For corporate legal information go to: >> http://www.cisco.com/web/about/doing_business/legal/cri/ > > > _______________________________________________ > mpi-forum mailing list > mpi-forum(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum

1 0

use of communicator in MPI_Pack and MPI_Unpack
by Michael Rezny 17 Feb '16

17 Feb '16

Hi, I have a question regarding the use of a communicator in MPI_Pack and MPI_Unpack. I have googled and search the MPI mailing lists but haven't found an answer. The OpenMPI man page for MPI_Pack states: The comm argument is the communicator that will be subsequently used for sending the packed message. And something similar for MPI_Unpack. Which seems to reflect what is defined in the MPI standards. Basically, what I would like to do is to have an MPI rank as a coupler. That is, receiving a packed message from one application and passing it to another application. Comm_A would be the ranks of application A plus the coupler and Comm_B would be the ranks of application B plus the coupler. In this case, MPI_COMM_WORLD = Comm_A U Comm_B. I would like to have a solution that does not have the coupler unpacking the buffer received from application A and then packing it to send it to application B I just found a very nice explanation of the need for a communicator in Pack / Unpack on page 135 of Using MPI Portable Parallel Programming with the Message-Passing Interface second edition by Gropp, Lusk, and Skjellum. If I understand this correctly, it would be legal to use MPI_COMM_WORLD as the communicator for MPI_Pack and MPI_Unpack. The only downside is that one might not get the optimal performance. The alternative we are using is to use MPI_Pack_external and MPI_Unpack_external which requires no communicator. But the downside is that on homogeneous little-endian HPCs, this will always entail byte swapping on packing and unpacking. So my question is: What is the correct and optimal way to implement passing packed messages in the coupling example described above? kindest regards Mike

1 0