January 2016 - mpi-forum - mailman3.cels.anl.gov

Re: [Mpi-forum] MPI_DATATYPE_NULL when count=0
by Schulz Martin 13 Jan '16

13 Jan '16

Hi Jeff, I agree the fix is simple. I just wanted to point out that this can break existing codes, in particular if we only fix this for some routines. In this case, if we make the change consistently (incl. all type creation functions) it¹s probably not an issue at all. As for the removed functions that code was written pre MPI 2.1. It¹s true legacy code :) (that¹s also why its not in the public repo and not usually tested), but the problem is still there with the new functions as well. Martin ________________________________________________________________________ Martin Schulz, schulzm(a)llnl.gov, http://scalability.llnl.gov/ CASC @ Lawrence Livermore National Laboratory, Livermore, USA From: mpi-forum <mpi-forum-bounces(a)lists.mpi-forum.org> on behalf of Jeff Hammond <jeff.science(a)gmail.com> Reply-To: Main mailing list <mpi-forum(a)lists.mpi-forum.org> Date: Wednesday, January 13, 2016 at 1:27 PM To: Main mailing list <mpi-forum(a)lists.mpi-forum.org> Subject: Re: [Mpi-forum] MPI_DATATYPE_NULL when count=0 I'm just guessing, but it seems like a trivial change to not stuff nothing into the struct is sufficient: #define DT_GENERALALLOC(b,c,dt,pb,var) \ { int err,lens[2]; MPI_Aint addr[2]; MPI_Datatype types[2]; \ types[0]=MPI_CHAR; lens[0]=piggyback_size; err=MPI_Address((void*) pb, &(addr[0])); \ if (err!=MPI_SUCCESS) { return err; } \ likely_if (c>0) { types[1]=dt; lens[1]=c; err=MPI_Address((void*) b, &(addr[1])); \ if (err!=MPI_SUCCESS) { return err; } \ err=MPI_Type_struct(2,lens,addr,types,&var); \ if (err!=MPI_SUCCESS) { return err; } \ } else { var = types[0]; } err=MPI_Type_commit(&var); \ if (err!=MPI_SUCCESS) { return err; } \ } However, I'm not sure it is prudent to discuss this code in this context, since it is not compliant with MPI 3.0 ;-) B.2.2 Changes in MPI-3.0 1. 2. Section 2.6.1 on page 17, Section 16.2 on page 604 and all other chapters. 3. The C++ bindings were removed from the standard. See errata in Section B.2.1 on page 798 for the latest changes to the MPI C++ binding defined in MPI-2.2. 4. This change may affect backward compatibility. 5. 6. Section 2.6.1 on page 17, Section 15.1 on page 599 and Section 16.1 on page 603. The deprecated functions MPI_TYPE_HVECTOR, MPI_TYPE_HINDEXED, MPI_TYPE_STRUCT, MPI_ADDRESS, MPI_TYPE_EXTENT, MPI_TYPE_LB, MPI_TYPE_UB, MPI_ERRHANDLER_CREATE (and its callback function prototype MPI_Handler_function), MPI_ERRHANDLER_SET, MPI_ERRHANDLER_GET, the deprecated special datatype handles MPI_LB, MPI_UB, and the constants MPI_COMBINER_HINDEXED_INTEGER, MPI_COMBINER_HVECTOR_INTEGER, MPI_COMBINER_STRUCT_INTEGER were removed from the standard. 7. 8. This change may affect backward compatibility. Best, Jeff On Wed, Jan 13, 2016 at 12:54 PM, Schulz Martin <schulzm(a)llnl.gov> wrote: > Hi Jeff, > > Unfortunately, I don¹t have the code on a public repo (it is part of our > internal PnMPI branch), but the critical macro is below. Nothing too > surprising it basically takes the existing datatype and adds one field, > creates a new datatype and then commits it. The following communication then > uses this datatype together with MPI_BOTTOM. > > I agree, this code could be optimized for such special cases and adding an if > statement to check for zero counts is doable. However, my point was that the > proposed change does change the standard so that existing code breaks. > > Also, it would seem strange that you then could use MPI_TYPE_NULL for some > operations with count zero and not for others. For this particular code, if we > would define what it means to have MPI_TYPE_NULL for MPI_Type_struct with a > zero count (it should probably just be ignored, although we would have to > think through if there are any consequences for the extent calculations), the > code would work just fine. I am not sure, if this would cover all cases, > though (it might, if we are carefully defining this for all routines that take > datatypes as arguments). > > Martin > > > #define DT_GENERALALLOC(b,c,dt,pb,var) \ > { int err,lens[2]; MPI_Aint addr[2]; MPI_Datatype types[2]; \ > types[0]=MPI_CHAR; lens[0]=piggyback_size; err=MPI_Address((void*) pb, > &(addr[0])); \ > if (err!=MPI_SUCCESS) { return err; } \ > types[1]=dt; lens[1]=c; err=MPI_Address((void*) b, &(addr[1])); \ > if (err!=MPI_SUCCESS) { return err; } \ > err=MPI_Type_struct(2,lens,addr,types,&var); \ > if (err!=MPI_SUCCESS) { return err; } \ > err=MPI_Type_commit(&var); \ > if (err!=MPI_SUCCESS) { return err; } \ > } > > > > > ________________________________________________________________________ > Martin Schulz, schulzm(a)llnl.gov, http://scalability.llnl.gov/ > CASC @ Lawrence Livermore National Laboratory, Livermore, USA > > > From: mpi-forum <mpi-forum-bounces(a)lists.mpi-forum.org> on behalf of Jeff > Hammond <jeff.science(a)gmail.com> > Reply-To: Main mailing list <mpi-forum(a)lists.mpi-forum.org> > Date: Wednesday, January 13, 2016 at 12:09 PM > > To: Main mailing list <mpi-forum(a)lists.mpi-forum.org> > Subject: Re: [Mpi-forum] MPI_DATATYPE_NULL when count=0 > > Can you provide the specifics of a tool that does this today? I'd like to > read the source code to see what changes are actually required. > > How can you piggyback an MPI collective call by hacking the datatype if > count=0? You already have to check if count=0 and bump it to count=1 if you > are going to send a piggy for a null transfer, so how hard is it to replace > MPI_DATATYPE_NULL with MPI_BYTE? If you are not sending a piggy for a null > transfer, then the issue is moot. > > If nothing else, the overhead of implementing the trivial branch that solves > this issue will be negligible compared to firing up the datatype engine. > > Jeff > > On Wed, Jan 13, 2016 at 11:41 AM, Schulz Martin <schulzm(a)llnl.gov> wrote: >> In general, I agree that allowing any argument for datatype if the count is >> zero makes sense. However, such a change (even if it is just allowing more) >> can cause backwards compatibility problems: e.g., any PMPI tool or other kind >> of internal wrapper library can currently have the assumption that it is only >> passed valid datatypes and hence can use the datatype without having to >> check. We, for example, use this to implement piggybacking (by creating a new >> struct datatype that contains the original one and adds a piggyback field). >> If we allowed arbitrary datatypes, we would brake such codes, unless we then >> also allowed it for datatype creation. >> >> If we wanted to change this, we probably need to be very careful and make >> sure we are consistent across the standard and touch all routines that take >> datatype arguments. Just adding the exception for P2P, or even for all >> communication routines, seems insufficient. >> >> Martin >> >> >> ________________________________________________________________________ >> >> ________________________________________________________________________ >> Martin Schulz, schulzm(a)llnl.gov, http://scalability.llnl.gov/ >> CASC @ Lawrence Livermore National Laboratory, Livermore, USA >> >> >> From: mpi-forum <mpi-forum-bounces(a)lists.mpi-forum.org> on behalf of Jeff >> Hammond <jeff.science(a)gmail.com> >> Reply-To: Main mailing list <mpi-forum(a)lists.mpi-forum.org> >> Date: Wednesday, January 13, 2016 at 6:24 AM >> To: Main mailing list <mpi-forum(a)lists.mpi-forum.org> >> >> Subject: Re: [Mpi-forum] MPI_DATATYPE_NULL when count=0 >> >> That is precisely what I want: when count=0, the buffer and the datatype >> arguments are ignored. There may be a good reason for MPI_DATATYPE_NULL to >> be invalid when it is actually relevant, but when count=0, the datatype >> argument is not relevant, because there are zero of them. >> >> We already state explicitly that arguments are ignored for MPI_IN_PLACE and >> MPI_NO_OP: >> >> * The ³in place² option for intracommunicators is specified by passing >> MPI_IN_PLACE as the value of recvbuf at the root. In such a case, recvcount >> and recvtype are ignored..." >> >> * "When MPI_NO_OP is specified as the operation, the origin_addr, >> origin_count, and origin_datatype arguments are ignored." >> >> I see no reason why count=0 is different from these cases. >> >> Jeff >> >> On Wed, Jan 13, 2016 at 5:47 AM, William Gropp <wgropp(a)illinois.edu> wrote: >>> > >>> > True, MPI_BOTTOM need not be null, but it might be, so the standard >>> doesn¹t prohibit it. >>> > >>> > A carefully worded exception to the general rule for NULL handles would be >>> ok. I would still make it narrow; for example, just for the use of >>> MPI_DATATYPE_NULL in communication operations with a zero count. >>> > >>> > Bill >>> > >>> > William Gropp >>> > Director, Parallel Computing Institute >>> > Thomas M. Siebel Chair in Computer Science >>> > Chief Scientist, NCSA >>> > University of Illinois Urbana-Champaign >>> > >>> > >>> > >>> > >>> > >>> > On Jan 12, 2016, at 11:22 PM, Fab Tillier <ftillier(a)microsoft.com> wrote: >>> > >>> > Don¹t datatypes with absolute addresses rely on MPI_BOTTOM, and a portable >>> program can¹t assume that MPI_BOTTOM == NULL? >>> > >>> > I think from an ease of use perspective, if I have a count of zero, having >>> to pick an arbitrary committed datatype is non-intuitive. If count is zero, >>> there is no buffer or datatype, and there should be no requirement such >>> non-existent datatype be committed. >>> > >>> > Cheers, >>> > -Fab >>> > >>> > From: mpi-forum [mailto:[email protected]] On Behalf >>> Of William Gropp >>> > Sent: Wednesday, 13 January 2016 4:52 p.m. >>> > To: Main MPI Forum mailing list <mpi-forum(a)lists.mpi-forum.org> >>> > Subject: Re: [Mpi-forum] MPI_DATATYPE_NULL when count=0 >>> > >>> > Yes, NULL buffers are valid - they aren¹t an MPI object. There is no >>> prohibition about passing them; only if they are used to form an invalid >>> address (note that a datatype with absolute addresses relies on NULL as the >>> buffer address). >>> > >>> > Bill >>> > >>> > >>> > William Gropp >>> > Director, Parallel Computing Institute >>> > Thomas M. Siebel Chair in Computer Science >>> > Chief Scientist, NCSA >>> > University of Illinois Urbana-Champaign >>> > >>> > >>> > >>> > >>> > On Jan 12, 2016, at 9:49 PM, Jeff Hammond <jeff.science(a)gmail.com> wrote: >>> > >>> > >>> > Who wrote Example 4.23 of MPI 3.1? >>> > >>> > I propose to add the exception rather than delete that example. I do not >>> want to have to use real datatypes when sending nothing. >>> > >>> > Is NULL a valid buffer handle? Can I pass null pointers when count=0? >>> > >>> > >>> > Jeff >>> > >>> > >>> > On Tuesday, January 12, 2016, William Gropp <wgropp(a)illinois.edu> wrote: >>> > >>> > The standard says that the null handles are invalid for input unless >>> explicitly permitted. There¹s no exemption for a datatype argument where >>> the associated count is zero. >>> > >>> > Bill >>> > >>> > >>> > William Gropp >>> > Director, Parallel Computing Institute >>> > Thomas M. Siebel Chair in Computer Science >>> > Chief Scientist, NCSA >>> > University of Illinois Urbana-Champaign >>> > >>> > >>> > >>> > >>> > On Jan 12, 2016, at 9:26 PM, Jeff Hammond <jeff.science(a)gmail.com> wrote: >>> > >>> > >>> > Is it allowed to communicate messages with >>> (buffer=NULL,count=0,type=MPI_DATATYPE_NULL)? >>> > >>> > George thinks MPI_DATATYPE_NULL cannot be used even when count=0, which I >>> think is ridiculous, particularly when one makes the analogy to buffer=NULL. >>> > >>> > See >>> http://secure-web.cisco.com/1uIwjXiVB0UCnFFG4b2DNhy-WFSpBHoZVan-ksg1oo1XCSjc >>> KasJCWljyye8xZYsJX2A72ZewdK6RzVNcE2jWKnCqVVFTmp9Oprj6gbb1orgQeDMDduaWKuVGKXK >>> bRsPnX2cJZeGodMtDUQx5HPmA46Qmql4kBAvZDR2KnWoFOQKuotirK6FQIazgUFBlC9CPP8fkF0T >>> FKw4OIzxQJ_Tp2Xe-mRb3swzYiungQAHNUVs53lxZSOLwP432qi3ovJ8cFw4G6XCXEkE37Xx2rGY >>> hrsHiZQy03zUFRmYy13QckBzIrtfY1zG5piXPVIOCfIoFfvm4o-tUjgyZSzspCUye23lLO2G1ic7 >>> fjL5OsxTrokI/http%3A%2F%2Fwww.open-mpi.org%2Fcommunity%2Flists%2Fusers%2F201 >>> 6%2F01%2F28255.php for the thread in question. >> >>> > >>> > There is example code in MPI that uses this behavior. George and I agree >>> that example text is non-normative, but I cannot imagine how this example >>> came to exist without the Forum believing it to be valid. >>> > >>> > Jeff >>> > >>> > -- >>> > Jeff Hammond >>> > jeff.science(a)gmail.com >>> > http://jeffhammond.github.io/ >>> > _______________________________________________ >>> > mpi-forum mailing list >>> > mpi-forum(a)lists.mpi-forum.org >>> > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum >>> > >>> > >>> > >>> > >>> > >>> > -- >>> > Jeff Hammond >>> > jeff.science(a)gmail.com >>> > http://jeffhammond.github.io/ >>> > _______________________________________________ >>> > mpi-forum mailing list >>> > mpi-forum(a)lists.mpi-forum.org >>> > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum >>> > >>> > >>> > _______________________________________________ >>> > mpi-forum mailing list >>> > mpi-forum(a)lists.mpi-forum.org >>> > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum >>> > >>> > >>> > >>> > _______________________________________________ >>> > mpi-forum mailing list >>> > mpi-forum(a)lists.mpi-forum.org >>> > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum >> >> >> >> >> -- >> Jeff Hammond >> jeff.science(a)gmail.com >> http://jeffhammond.github.io/ >> >> _______________________________________________ >> mpi-forum mailing list >> mpi-forum(a)lists.mpi-forum.org >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum > > > > -- > Jeff Hammond > jeff.science(a)gmail.com > http://jeffhammond.github.io/ > > _______________________________________________ > mpi-forum mailing list > mpi-forum(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum -- Jeff Hammond jeff.science(a)gmail.com http://jeffhammond.github.io/

1 0

Re: [Mpi-forum] MPI_DATATYPE_NULL when count=0
by Fab Tillier 13 Jan '16

13 Jan '16

I think rather than explicitly allowing MPI_DATATYPE_NULL, we should clarify that the buffer and datatype parameters are “ignored” when count is zero. Alternatively, we could document them as “significant” only if count is non-zero. I believe passing MPI_DATATYPE_NULL is standard-compliant in the following scenarios given the current standard text: - sendtype in MPI_Gather with MPI_IN_PLACE: standard says sendtype is “ignored” - recvtype in MPI_Gather on non-root processes: standard says only sendtype (of the datatype parameters) is “significant” We already have precedent for ignoring parameters, and it seems straight forward to extend that to situations where count is zero. That said, if we fix this in the standard, implementations (including tools that masquerade as implementations) that don’t comply will need to change to be compliant. It does not affect backward compatibility for MPI applications though. Cheers, -Fab From: mpi-forum [mailto:[email protected]] On Behalf Of Jeff Hammond Sent: Thursday, 14 January 2016 10:27 a.m. To: Main MPI Forum mailing list <mpi-forum(a)lists.mpi-forum.org> Subject: Re: [Mpi-forum] MPI_DATATYPE_NULL when count=0 I'm just guessing, but it seems like a trivial change to not stuff nothing into the struct is sufficient: #define DT_GENERALALLOC(b,c,dt,pb,var) \ { int err,lens[2]; MPI_Aint addr[2]; MPI_Datatype types[2]; \ types[0]=MPI_CHAR; lens[0]=piggyback_size; err=MPI_Address((void*) pb, &(addr[0])); \ if (err!=MPI_SUCCESS) { return err; } \ likely_if (c>0) { types[1]=dt; lens[1]=c; err=MPI_Address((void*) b, &(addr[1])); \ if (err!=MPI_SUCCESS) { return err; } \ err=MPI_Type_struct(2,lens,addr,types,&var); \ if (err!=MPI_SUCCESS) { return err; } \ } else { var = types[0]; } err=MPI_Type_commit(&var); \ if (err!=MPI_SUCCESS) { return err; } \ } However, I'm not sure it is prudent to discuss this code in this context, since it is not compliant with MPI 3.0 ;-) B.2.2 Changes in MPI-3.0 1. Section 2.6.1 on page 17, Section 16.2 on page 604 and all other chapters. The C++ bindings were removed from the standard. See errata in Section B.2.1 on page 798 for the latest changes to the MPI C++ binding defined in MPI-2.2. This change may affect backward compatibility. 2. Section 2.6.1 on page 17, Section 15.1 on page 599 and Section 16.1 on page 603. The deprecated functions MPI_TYPE_HVECTOR, MPI_TYPE_HINDEXED, MPI_TYPE_STRUCT, MPI_ADDRESS, MPI_TYPE_EXTENT, MPI_TYPE_LB, MPI_TYPE_UB, MPI_ERRHANDLER_CREATE (and its callback function prototype MPI_Handler_function), MPI_ERRHANDLER_SET, MPI_ERRHANDLER_GET, the deprecated special datatype handles MPI_LB, MPI_UB, and the constants MPI_COMBINER_HINDEXED_INTEGER, MPI_COMBINER_HVECTOR_INTEGER, MPI_COMBINER_STRUCT_INTEGER were removed from the standard. This change may affect backward compatibility. Best, Jeff On Wed, Jan 13, 2016 at 12:54 PM, Schulz Martin <schulzm(a)llnl.gov<mailto:[email protected]>> wrote: Hi Jeff, Unfortunately, I don’t have the code on a public repo (it is part of our internal PnMPI branch), but the critical macro is below. Nothing too surprising – it basically takes the existing datatype and adds one field, creates a new datatype and then commits it. The following communication then uses this datatype together with MPI_BOTTOM. I agree, this code could be optimized for such special cases and adding an if statement to check for zero counts is doable. However, my point was that the proposed change does change the standard so that existing code breaks. Also, it would seem strange that you then could use MPI_TYPE_NULL for some operations with count zero and not for others. For this particular code, if we would define what it means to have MPI_TYPE_NULL for MPI_Type_struct with a zero count (it should probably just be ignored, although we would have to think through if there are any consequences for the extent calculations), the code would work just fine. I am not sure, if this would cover all cases, though (it might, if we are carefully defining this for all routines that take datatypes as arguments). Martin #define DT_GENERALALLOC(b,c,dt,pb,var) \ { int err,lens[2]; MPI_Aint addr[2]; MPI_Datatype types[2]; \ types[0]=MPI_CHAR; lens[0]=piggyback_size; err=MPI_Address((void*) pb, &(addr[0])); \ if (err!=MPI_SUCCESS) { return err; } \ types[1]=dt; lens[1]=c; err=MPI_Address((void*) b, &(addr[1])); \ if (err!=MPI_SUCCESS) { return err; } \ err=MPI_Type_struct(2,lens,addr,types,&var); \ if (err!=MPI_SUCCESS) { return err; } \ err=MPI_Type_commit(&var); \ if (err!=MPI_SUCCESS) { return err; } \ } ________________________________________________________________________ Martin Schulz, schulzm(a)llnl.gov<mailto:[email protected]>, http://scalability.llnl.gov/<https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fscalability…> CASC @ Lawrence Livermore National Laboratory, Livermore, USA From: mpi-forum <mpi-forum-bounces(a)lists.mpi-forum.org<mailto:[email protected]>> on behalf of Jeff Hammond <jeff.science(a)gmail.com<mailto:[email protected]>> Reply-To: Main mailing list <mpi-forum(a)lists.mpi-forum.org<mailto:[email protected]>> Date: Wednesday, January 13, 2016 at 12:09 PM To: Main mailing list <mpi-forum(a)lists.mpi-forum.org<mailto:[email protected]>> Subject: Re: [Mpi-forum] MPI_DATATYPE_NULL when count=0 Can you provide the specifics of a tool that does this today? I'd like to read the source code to see what changes are actually required. How can you piggyback an MPI collective call by hacking the datatype if count=0? You already have to check if count=0 and bump it to count=1 if you are going to send a piggy for a null transfer, so how hard is it to replace MPI_DATATYPE_NULL with MPI_BYTE? If you are not sending a piggy for a null transfer, then the issue is moot. If nothing else, the overhead of implementing the trivial branch that solves this issue will be negligible compared to firing up the datatype engine. Jeff On Wed, Jan 13, 2016 at 11:41 AM, Schulz Martin <schulzm(a)llnl.gov<mailto:[email protected]>> wrote: In general, I agree that allowing any argument for datatype if the count is zero makes sense. However, such a change (even if it is just allowing more) can cause backwards compatibility problems: e.g., any PMPI tool or other kind of internal wrapper library can currently have the assumption that it is only passed valid datatypes and hence can use the datatype without having to check. We, for example, use this to implement piggybacking (by creating a new struct datatype that contains the original one and adds a piggyback field). If we allowed arbitrary datatypes, we would brake such codes, unless we then also allowed it for datatype creation. If we wanted to change this, we probably need to be very careful and make sure we are consistent across the standard and touch all routines that take datatype arguments. Just adding the exception for P2P, or even for all communication routines, seems insufficient. Martin ________________________________________________________________________ ________________________________________________________________________ Martin Schulz, schulzm(a)llnl.gov<mailto:[email protected]>, http://scalability.llnl.gov/<https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fscalability…> CASC @ Lawrence Livermore National Laboratory, Livermore, USA From: mpi-forum <mpi-forum-bounces(a)lists.mpi-forum.org<mailto:[email protected]>> on behalf of Jeff Hammond <jeff.science(a)gmail.com<mailto:[email protected]>> Reply-To: Main mailing list <mpi-forum(a)lists.mpi-forum.org<mailto:[email protected]>> Date: Wednesday, January 13, 2016 at 6:24 AM To: Main mailing list <mpi-forum(a)lists.mpi-forum.org<mailto:[email protected]>> Subject: Re: [Mpi-forum] MPI_DATATYPE_NULL when count=0 That is precisely what I want: when count=0, the buffer and the datatype arguments are ignored. There may be a good reason for MPI_DATATYPE_NULL to be invalid when it is actually relevant, but when count=0, the datatype argument is not relevant, because there are zero of them. We already state explicitly that arguments are ignored for MPI_IN_PLACE and MPI_NO_OP: * The “in place” option for intracommunicators is specified by passing MPI_IN_PLACE as the value of recvbuf at the root. In such a case, recvcount and recvtype are ignored..." * "When MPI_NO_OP is specified as the operation, the origin_addr, origin_count, and origin_datatype arguments are ignored." I see no reason why count=0 is different from these cases. Jeff On Wed, Jan 13, 2016 at 5:47 AM, William Gropp <wgropp(a)illinois.edu<mailto:[email protected]>> wrote: > > True, MPI_BOTTOM need not be null, but it might be, so the standard doesn’t prohibit it. > > A carefully worded exception to the general rule for NULL handles would be ok. I would still make it narrow; for example, just for the use of MPI_DATATYPE_NULL in communication operations with a zero count. > > Bill > > William Gropp > Director, Parallel Computing Institute > Thomas M. Siebel Chair in Computer Science > Chief Scientist, NCSA > University of Illinois Urbana-Champaign > > > > > > On Jan 12, 2016, at 11:22 PM, Fab Tillier <ftillier(a)microsoft.com<mailto:[email protected]>> wrote: > > Don’t datatypes with absolute addresses rely on MPI_BOTTOM, and a portable program can’t assume that MPI_BOTTOM == NULL? > > I think from an ease of use perspective, if I have a count of zero, having to pick an arbitrary committed datatype is non-intuitive. If count is zero, there is no buffer or datatype, and there should be no requirement such non-existent datatype be committed. > > Cheers, > -Fab > > From: mpi-forum [mailto:[email protected]<mailto:[email protected]>] On Behalf Of William Gropp > Sent: Wednesday, 13 January 2016 4:52 p.m. > To: Main MPI Forum mailing list <mpi-forum(a)lists.mpi-forum.org<mailto:[email protected]>> > Subject: Re: [Mpi-forum] MPI_DATATYPE_NULL when count=0 > > Yes, NULL buffers are valid - they aren’t an MPI object. There is no prohibition about passing them; only if they are used to form an invalid address (note that a datatype with absolute addresses relies on NULL as the buffer address). > > Bill > > > William Gropp > Director, Parallel Computing Institute > Thomas M. Siebel Chair in Computer Science > Chief Scientist, NCSA > University of Illinois Urbana-Champaign > > > > > On Jan 12, 2016, at 9:49 PM, Jeff Hammond <jeff.science(a)gmail.com<mailto:[email protected]>> wrote: > > > Who wrote Example 4.23 of MPI 3.1? > > I propose to add the exception rather than delete that example. I do not want to have to use real datatypes when sending nothing. > > Is NULL a valid buffer handle? Can I pass null pointers when count=0? > > > Jeff > > > On Tuesday, January 12, 2016, William Gropp <wgropp(a)illinois.edu<mailto:[email protected]>> wrote: > > The standard says that the null handles are invalid for input unless explicitly permitted. There’s no exemption for a datatype argument where the associated count is zero. > > Bill > > > William Gropp > Director, Parallel Computing Institute > Thomas M. Siebel Chair in Computer Science > Chief Scientist, NCSA > University of Illinois Urbana-Champaign > > > > > On Jan 12, 2016, at 9:26 PM, Jeff Hammond <jeff.science(a)gmail.com<mailto:[email protected]>> wrote: > > > Is it allowed to communicate messages with (buffer=NULL,count=0,type=MPI_DATATYPE_NULL)? > > George thinks MPI_DATATYPE_NULL cannot be used even when count=0, which I think is ridiculous, particularly when one makes the analogy to buffer=NULL. > > See http://secure-web.cisco.com/1uIwjXiVB0UCnFFG4b2DNhy-WFSpBHoZVan-ksg1oo1XCSj…<https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fsecure-web.…> for the thread in question. > > There is example code in MPI that uses this behavior. George and I agree that example text is non-normative, but I cannot imagine how this example came to exist without the Forum believing it to be valid. > > Jeff > > -- > Jeff Hammond > jeff.science(a)gmail.com<mailto:[email protected]> > http://jeffhammond.github.io/<https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fjeffhammond…> > _______________________________________________ > mpi-forum mailing list > mpi-forum(a)lists.mpi-forum.org<mailto:[email protected]> > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum<https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2flists.mpi-f…> > > > > > > -- > Jeff Hammond > jeff.science(a)gmail.com<mailto:[email protected]> > http://jeffhammond.github.io/<https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fjeffhammond…> > _______________________________________________ > mpi-forum mailing list > mpi-forum(a)lists.mpi-forum.org<mailto:[email protected]> > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum<https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2flists.mpi-f…> > > > _______________________________________________ > mpi-forum mailing list > mpi-forum(a)lists.mpi-forum.org<mailto:[email protected]> > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum<https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2flists.mpi-f…> > > > > _______________________________________________ > mpi-forum mailing list > mpi-forum(a)lists.mpi-forum.org<mailto:[email protected]> > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum<https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2flists.mpi-f…> -- Jeff Hammond jeff.science(a)gmail.com<mailto:[email protected]> http://jeffhammond.github.io/<https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fjeffhammond…> _______________________________________________ mpi-forum mailing list mpi-forum(a)lists.mpi-forum.org<mailto:[email protected]> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum<https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2flists.mpi-f…> -- Jeff Hammond jeff.science(a)gmail.com<mailto:[email protected]> http://jeffhammond.github.io/<https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fjeffhammond…> _______________________________________________ mpi-forum mailing list mpi-forum(a)lists.mpi-forum.org<mailto:[email protected]> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum<https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2flists.mpi-f…> -- Jeff Hammond jeff.science(a)gmail.com<mailto:[email protected]> http://jeffhammond.github.io/<https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fjeffhammond…>

1 0

Re: [Mpi-forum] MPI_DATATYPE_NULL when count=0
by Jeff Hammond 13 Jan '16

13 Jan '16

I'm just guessing, but it seems like a trivial change to not stuff nothing into the struct is sufficient: #define DT_GENERALALLOC(b,c,dt,pb,var) \ { int err,lens[2]; MPI_Aint addr[2]; MPI_Datatype types[2]; \ types[0]=MPI_CHAR; lens[0]=piggyback_size; err=MPI_Address((void*) pb, &(addr[0])); \ if (err!=MPI_SUCCESS) { return err; } \ likely_if (c>0) { types[1]=dt; lens[1]=c; err=MPI_Address((void*) b, &(addr[1])); \ if (err!=MPI_SUCCESS) { return err; } \ err=MPI_Type_struct(2,lens,addr,types,&var); \ if (err!=MPI_SUCCESS) { return err; } \ } else { var = types[0]; } err=MPI_Type_commit(&var); \ if (err!=MPI_SUCCESS) { return err; } \ } However, I'm not sure it is prudent to discuss this code in this context, since it is not compliant with MPI 3.0 ;-) B.2.2 Changes in MPI-3.0 1. Section 2.6.1 on page 17, Section 16.2 on page 604 and all other chapters. The C++ bindings were removed from the standard. See errata in Section B.2.1 on page 798 for the latest changes to the MPI C++ binding defined in MPI-2.2. This change may affect backward compatibility. 2. Section 2.6.1 on page 17, Section 15.1 on page 599 and Section 16.1 on page 603. The deprecated functions MPI_TYPE_HVECTOR, MPI_TYPE_HINDEXED, MPI_TYPE_STRUCT, MPI_ADDRESS, MPI_TYPE_EXTENT, MPI_TYPE_LB, MPI_TYPE_UB, MPI_ERRHANDLER_CREATE (and its callback function prototype MPI_Handler_ function), MPI_ERRHANDLER_SET, MPI_ERRHANDLER_GET, the deprecated special datatype handles MPI_LB, MPI_UB, and the constants MPI_COMBINER_ HINDEXED_INTEGER, MPI_COMBINER_HVECTOR_INTEGER, MPI_COMBINER_STRUCT_INTEGER were removed from the standard. This change may affect backward compatibility. Best, Jeff On Wed, Jan 13, 2016 at 12:54 PM, Schulz Martin <schulzm(a)llnl.gov> wrote: > Hi Jeff, > > Unfortunately, I don’t have the code on a public repo (it is part of our > internal PnMPI branch), but the critical macro is below. Nothing too > surprising – it basically takes the existing datatype and adds one field, > creates a new datatype and then commits it. The following communication > then uses this datatype together with MPI_BOTTOM. > > I agree, this code could be optimized for such special cases and adding an > if statement to check for zero counts is doable. However, my point was that > the proposed change does change the standard so that existing code breaks. > > Also, it would seem strange that you then could use MPI_TYPE_NULL for some > operations with count zero and not for others. For this particular code, if > we would define what it means to have MPI_TYPE_NULL for MPI_Type_struct > with a zero count (it should probably just be ignored, although we would > have to think through if there are any consequences for the extent > calculations), the code would work just fine. I am not sure, if this would > cover all cases, though (it might, if we are carefully defining this for > all routines that take datatypes as arguments). > > Martin > > > #define DT_GENERALALLOC(b,c,dt,pb,var) \ > { int err,lens[2]; MPI_Aint addr[2]; MPI_Datatype types[2]; \ > types[0]=MPI_CHAR; lens[0]=piggyback_size; err=MPI_Address((void*) pb, > &(addr[0])); \ > if (err!=MPI_SUCCESS) { return err; } \ > types[1]=dt; lens[1]=c; err=MPI_Address((void*) b, &(addr[1])); \ > if (err!=MPI_SUCCESS) { return err; } \ > err=MPI_Type_struct(2,lens,addr,types,&var); \ > if (err!=MPI_SUCCESS) { return err; } \ > err=MPI_Type_commit(&var); \ > if (err!=MPI_SUCCESS) { return err; } \ > } > > > > > ________________________________________________________________________ > Martin Schulz, schulzm(a)llnl.gov, http://scalability.llnl.gov/ > CASC @ Lawrence Livermore National Laboratory, Livermore, USA > > > From: mpi-forum <mpi-forum-bounces(a)lists.mpi-forum.org> on behalf of Jeff > Hammond <jeff.science(a)gmail.com> > Reply-To: Main mailing list <mpi-forum(a)lists.mpi-forum.org> > Date: Wednesday, January 13, 2016 at 12:09 PM > > To: Main mailing list <mpi-forum(a)lists.mpi-forum.org> > Subject: Re: [Mpi-forum] MPI_DATATYPE_NULL when count=0 > > Can you provide the specifics of a tool that does this today? I'd like to > read the source code to see what changes are actually required. > > How can you piggyback an MPI collective call by hacking the datatype if > count=0? You already have to check if count=0 and bump it to count=1 if > you are going to send a piggy for a null transfer, so how hard is it to > replace MPI_DATATYPE_NULL with MPI_BYTE? If you are not sending a piggy > for a null transfer, then the issue is moot. > > If nothing else, the overhead of implementing the trivial branch that > solves this issue will be negligible compared to firing up the datatype > engine. > > Jeff > > On Wed, Jan 13, 2016 at 11:41 AM, Schulz Martin <schulzm(a)llnl.gov> wrote: > >> In general, I agree that allowing any argument for datatype if the count >> is zero makes sense. However, such a change (even if it is just allowing >> more) can cause backwards compatibility problems: e.g., any PMPI tool or >> other kind of internal wrapper library can currently have the assumption >> that it is only passed valid datatypes and hence can use the datatype >> without having to check. We, for example, use this to implement >> piggybacking (by creating a new struct datatype that contains the original >> one and adds a piggyback field). If we allowed arbitrary datatypes, we >> would brake such codes, unless we then also allowed it for datatype >> creation. >> >> If we wanted to change this, we probably need to be very careful and make >> sure we are consistent across the standard and touch all routines that take >> datatype arguments. Just adding the exception for P2P, or even for all >> communication routines, seems insufficient. >> >> Martin >> >> >> ________________________________________________________________________ >> >> ________________________________________________________________________ >> Martin Schulz, schulzm(a)llnl.gov, http://scalability.llnl.gov/ >> CASC @ Lawrence Livermore National Laboratory, Livermore, USA >> >> >> From: mpi-forum <mpi-forum-bounces(a)lists.mpi-forum.org> on behalf of >> Jeff Hammond <jeff.science(a)gmail.com> >> Reply-To: Main mailing list <mpi-forum(a)lists.mpi-forum.org> >> Date: Wednesday, January 13, 2016 at 6:24 AM >> To: Main mailing list <mpi-forum(a)lists.mpi-forum.org> >> >> Subject: Re: [Mpi-forum] MPI_DATATYPE_NULL when count=0 >> >> That is precisely what I want: when count=0, the buffer and the datatype >> arguments are ignored. There may be a good reason for MPI_DATATYPE_NULL to >> be invalid when it is actually relevant, but when count=0, the datatype >> argument is not relevant, because there are zero of them. >> >> We already state explicitly that arguments are ignored for MPI_IN_PLACE >> and MPI_NO_OP: >> >> * The “in place” option for intracommunicators is specified by passing >> MPI_IN_PLACE as the value of recvbuf at the root. In such a case, recvcount >> and recvtype are ignored..." >> >> * "When MPI_NO_OP is specified as the operation, the origin_addr, >> origin_count, and origin_datatype arguments are ignored." >> >> I see no reason why count=0 is different from these cases. >> >> Jeff >> >> On Wed, Jan 13, 2016 at 5:47 AM, William Gropp <wgropp(a)illinois.edu> >> wrote: >> > >> > True, MPI_BOTTOM need not be null, but it might be, so the standard >> doesn’t prohibit it. >> > >> > A carefully worded exception to the general rule for NULL handles would >> be ok. I would still make it narrow; for example, just for the use of >> MPI_DATATYPE_NULL in communication operations with a zero count. >> > >> > Bill >> > >> > William Gropp >> > Director, Parallel Computing Institute >> > Thomas M. Siebel Chair in Computer Science >> > Chief Scientist, NCSA >> > University of Illinois Urbana-Champaign >> > >> > >> > >> > >> > >> > On Jan 12, 2016, at 11:22 PM, Fab Tillier <ftillier(a)microsoft.com> >> wrote: >> > >> > Don’t datatypes with absolute addresses rely on MPI_BOTTOM, and a >> portable program can’t assume that MPI_BOTTOM == NULL? >> > >> > I think from an ease of use perspective, if I have a count of zero, >> having to pick an arbitrary committed datatype is non-intuitive. If count >> is zero, there is no buffer or datatype, and there should be no requirement >> such non-existent datatype be committed. >> > >> > Cheers, >> > -Fab >> > >> > From: mpi-forum [mailto:[email protected]] On >> Behalf Of William Gropp >> > Sent: Wednesday, 13 January 2016 4:52 p.m. >> > To: Main MPI Forum mailing list <mpi-forum(a)lists.mpi-forum.org> >> > Subject: Re: [Mpi-forum] MPI_DATATYPE_NULL when count=0 >> > >> > Yes, NULL buffers are valid - they aren’t an MPI object. There is no >> prohibition about passing them; only if they are used to form an invalid >> address (note that a datatype with absolute addresses relies on NULL as the >> buffer address). >> > >> > Bill >> > >> > >> > William Gropp >> > Director, Parallel Computing Institute >> > Thomas M. Siebel Chair in Computer Science >> > Chief Scientist, NCSA >> > University of Illinois Urbana-Champaign >> > >> > >> > >> > >> > On Jan 12, 2016, at 9:49 PM, Jeff Hammond <jeff.science(a)gmail.com> >> wrote: >> > >> > >> > Who wrote Example 4.23 of MPI 3.1? >> > >> > I propose to add the exception rather than delete that example. I do >> not want to have to use real datatypes when sending nothing. >> > >> > Is NULL a valid buffer handle? Can I pass null pointers when count=0? >> > >> > >> > Jeff >> > >> > >> > On Tuesday, January 12, 2016, William Gropp <wgropp(a)illinois.edu> >> wrote: >> > >> > The standard says that the null handles are invalid for input unless >> explicitly permitted. There’s no exemption for a datatype argument where >> the associated count is zero. >> > >> > Bill >> > >> > >> > William Gropp >> > Director, Parallel Computing Institute >> > Thomas M. Siebel Chair in Computer Science >> > Chief Scientist, NCSA >> > University of Illinois Urbana-Champaign >> > >> > >> > >> > >> > On Jan 12, 2016, at 9:26 PM, Jeff Hammond <jeff.science(a)gmail.com> >> wrote: >> > >> > >> > Is it allowed to communicate messages with >> (buffer=NULL,count=0,type=MPI_DATATYPE_NULL)? >> > >> > George thinks MPI_DATATYPE_NULL cannot be used even when count=0, which >> I think is ridiculous, particularly when one makes the analogy to >> buffer=NULL. >> > >> > See >> http://secure-web.cisco.com/1uIwjXiVB0UCnFFG4b2DNhy-WFSpBHoZVan-ksg1oo1XCSj… >> for the thread in question. >> >> > >> > There is example code in MPI that uses this behavior. George and I >> agree that example text is non-normative, but I cannot imagine how this >> example came to exist without the Forum believing it to be valid. >> > >> > Jeff >> > >> > -- >> > Jeff Hammond >> > jeff.science(a)gmail.com >> > http://jeffhammond.github.io/ >> > _______________________________________________ >> > mpi-forum mailing list >> > mpi-forum(a)lists.mpi-forum.org >> > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum >> > >> > >> > >> > >> > >> > -- >> > Jeff Hammond >> > jeff.science(a)gmail.com >> > http://jeffhammond.github.io/ >> > _______________________________________________ >> > mpi-forum mailing list >> > mpi-forum(a)lists.mpi-forum.org >> > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum >> > >> > >> > _______________________________________________ >> > mpi-forum mailing list >> > mpi-forum(a)lists.mpi-forum.org >> > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum >> > >> > >> > >> > _______________________________________________ >> > mpi-forum mailing list >> > mpi-forum(a)lists.mpi-forum.org >> > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum >> >> >> >> >> -- >> Jeff Hammond >> jeff.science(a)gmail.com >> http://jeffhammond.github.io/ >> >> _______________________________________________ >> mpi-forum mailing list >> mpi-forum(a)lists.mpi-forum.org >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum >> > > > > -- > Jeff Hammond > jeff.science(a)gmail.com > http://jeffhammond.github.io/ > > _______________________________________________ > mpi-forum mailing list > mpi-forum(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum > -- Jeff Hammond jeff.science(a)gmail.com http://jeffhammond.github.io/

1 0

Re: [Mpi-forum] MPI_DATATYPE_NULL when count=0
by Schulz Martin 13 Jan '16

13 Jan '16

Hi Jeff, Unfortunately, I don¹t have the code on a public repo (it is part of our internal PnMPI branch), but the critical macro is below. Nothing too surprising it basically takes the existing datatype and adds one field, creates a new datatype and then commits it. The following communication then uses this datatype together with MPI_BOTTOM. I agree, this code could be optimized for such special cases and adding an if statement to check for zero counts is doable. However, my point was that the proposed change does change the standard so that existing code breaks. Also, it would seem strange that you then could use MPI_TYPE_NULL for some operations with count zero and not for others. For this particular code, if we would define what it means to have MPI_TYPE_NULL for MPI_Type_struct with a zero count (it should probably just be ignored, although we would have to think through if there are any consequences for the extent calculations), the code would work just fine. I am not sure, if this would cover all cases, though (it might, if we are carefully defining this for all routines that take datatypes as arguments). Martin #define DT_GENERALALLOC(b,c,dt,pb,var) \ { int err,lens[2]; MPI_Aint addr[2]; MPI_Datatype types[2]; \ types[0]=MPI_CHAR; lens[0]=piggyback_size; err=MPI_Address((void*) pb, &(addr[0])); \ if (err!=MPI_SUCCESS) { return err; } \ types[1]=dt; lens[1]=c; err=MPI_Address((void*) b, &(addr[1])); \ if (err!=MPI_SUCCESS) { return err; } \ err=MPI_Type_struct(2,lens,addr,types,&var); \ if (err!=MPI_SUCCESS) { return err; } \ err=MPI_Type_commit(&var); \ if (err!=MPI_SUCCESS) { return err; } \ } ________________________________________________________________________ Martin Schulz, schulzm(a)llnl.gov, http://scalability.llnl.gov/ CASC @ Lawrence Livermore National Laboratory, Livermore, USA From: mpi-forum <mpi-forum-bounces(a)lists.mpi-forum.org> on behalf of Jeff Hammond <jeff.science(a)gmail.com> Reply-To: Main mailing list <mpi-forum(a)lists.mpi-forum.org> Date: Wednesday, January 13, 2016 at 12:09 PM To: Main mailing list <mpi-forum(a)lists.mpi-forum.org> Subject: Re: [Mpi-forum] MPI_DATATYPE_NULL when count=0 Can you provide the specifics of a tool that does this today? I'd like to read the source code to see what changes are actually required. How can you piggyback an MPI collective call by hacking the datatype if count=0? You already have to check if count=0 and bump it to count=1 if you are going to send a piggy for a null transfer, so how hard is it to replace MPI_DATATYPE_NULL with MPI_BYTE? If you are not sending a piggy for a null transfer, then the issue is moot. If nothing else, the overhead of implementing the trivial branch that solves this issue will be negligible compared to firing up the datatype engine. Jeff On Wed, Jan 13, 2016 at 11:41 AM, Schulz Martin <schulzm(a)llnl.gov> wrote: > In general, I agree that allowing any argument for datatype if the count is > zero makes sense. However, such a change (even if it is just allowing more) > can cause backwards compatibility problems: e.g., any PMPI tool or other kind > of internal wrapper library can currently have the assumption that it is only > passed valid datatypes and hence can use the datatype without having to check. > We, for example, use this to implement piggybacking (by creating a new struct > datatype that contains the original one and adds a piggyback field). If we > allowed arbitrary datatypes, we would brake such codes, unless we then also > allowed it for datatype creation. > > If we wanted to change this, we probably need to be very careful and make sure > we are consistent across the standard and touch all routines that take > datatype arguments. Just adding the exception for P2P, or even for all > communication routines, seems insufficient. > > Martin > > > ________________________________________________________________________ > > ________________________________________________________________________ > Martin Schulz, schulzm(a)llnl.gov, http://scalability.llnl.gov/ > CASC @ Lawrence Livermore National Laboratory, Livermore, USA > > > From: mpi-forum <mpi-forum-bounces(a)lists.mpi-forum.org> on behalf of Jeff > Hammond <jeff.science(a)gmail.com> > Reply-To: Main mailing list <mpi-forum(a)lists.mpi-forum.org> > Date: Wednesday, January 13, 2016 at 6:24 AM > To: Main mailing list <mpi-forum(a)lists.mpi-forum.org> > > Subject: Re: [Mpi-forum] MPI_DATATYPE_NULL when count=0 > > That is precisely what I want: when count=0, the buffer and the datatype > arguments are ignored. There may be a good reason for MPI_DATATYPE_NULL to be > invalid when it is actually relevant, but when count=0, the datatype argument > is not relevant, because there are zero of them. > > We already state explicitly that arguments are ignored for MPI_IN_PLACE and > MPI_NO_OP: > > * The ³in place² option for intracommunicators is specified by passing > MPI_IN_PLACE as the value of recvbuf at the root. In such a case, recvcount > and recvtype are ignored..." > > * "When MPI_NO_OP is specified as the operation, the origin_addr, > origin_count, and origin_datatype arguments are ignored." > > I see no reason why count=0 is different from these cases. > > Jeff > > On Wed, Jan 13, 2016 at 5:47 AM, William Gropp <wgropp(a)illinois.edu> wrote: >> > >> > True, MPI_BOTTOM need not be null, but it might be, so the standard doesn¹t >> prohibit it. >> > >> > A carefully worded exception to the general rule for NULL handles would be >> ok. I would still make it narrow; for example, just for the use of >> MPI_DATATYPE_NULL in communication operations with a zero count. >> > >> > Bill >> > >> > William Gropp >> > Director, Parallel Computing Institute >> > Thomas M. Siebel Chair in Computer Science >> > Chief Scientist, NCSA >> > University of Illinois Urbana-Champaign >> > >> > >> > >> > >> > >> > On Jan 12, 2016, at 11:22 PM, Fab Tillier <ftillier(a)microsoft.com> wrote: >> > >> > Don¹t datatypes with absolute addresses rely on MPI_BOTTOM, and a portable >> program can¹t assume that MPI_BOTTOM == NULL? >> > >> > I think from an ease of use perspective, if I have a count of zero, having >> to pick an arbitrary committed datatype is non-intuitive. If count is zero, >> there is no buffer or datatype, and there should be no requirement such >> non-existent datatype be committed. >> > >> > Cheers, >> > -Fab >> > >> > From: mpi-forum [mailto:[email protected]] On Behalf Of >> William Gropp >> > Sent: Wednesday, 13 January 2016 4:52 p.m. >> > To: Main MPI Forum mailing list <mpi-forum(a)lists.mpi-forum.org> >> > Subject: Re: [Mpi-forum] MPI_DATATYPE_NULL when count=0 >> > >> > Yes, NULL buffers are valid - they aren¹t an MPI object. There is no >> prohibition about passing them; only if they are used to form an invalid >> address (note that a datatype with absolute addresses relies on NULL as the >> buffer address). >> > >> > Bill >> > >> > >> > William Gropp >> > Director, Parallel Computing Institute >> > Thomas M. Siebel Chair in Computer Science >> > Chief Scientist, NCSA >> > University of Illinois Urbana-Champaign >> > >> > >> > >> > >> > On Jan 12, 2016, at 9:49 PM, Jeff Hammond <jeff.science(a)gmail.com> wrote: >> > >> > >> > Who wrote Example 4.23 of MPI 3.1? >> > >> > I propose to add the exception rather than delete that example. I do not >> want to have to use real datatypes when sending nothing. >> > >> > Is NULL a valid buffer handle? Can I pass null pointers when count=0? >> > >> > >> > Jeff >> > >> > >> > On Tuesday, January 12, 2016, William Gropp <wgropp(a)illinois.edu> wrote: >> > >> > The standard says that the null handles are invalid for input unless >> explicitly permitted. There¹s no exemption for a datatype argument where the >> associated count is zero. >> > >> > Bill >> > >> > >> > William Gropp >> > Director, Parallel Computing Institute >> > Thomas M. Siebel Chair in Computer Science >> > Chief Scientist, NCSA >> > University of Illinois Urbana-Champaign >> > >> > >> > >> > >> > On Jan 12, 2016, at 9:26 PM, Jeff Hammond <jeff.science(a)gmail.com> wrote: >> > >> > >> > Is it allowed to communicate messages with >> (buffer=NULL,count=0,type=MPI_DATATYPE_NULL)? >> > >> > George thinks MPI_DATATYPE_NULL cannot be used even when count=0, which I >> think is ridiculous, particularly when one makes the analogy to buffer=NULL. >> > >> > See >> http://secure-web.cisco.com/1uIwjXiVB0UCnFFG4b2DNhy-WFSpBHoZVan-ksg1oo1XCSj… >> asJCWljyye8xZYsJX2A72ZewdK6RzVNcE2jWKnCqVVFTmp9Oprj6gbb1orgQeDMDduaWKuVGKXKbR >> sPnX2cJZeGodMtDUQx5HPmA46Qmql4kBAvZDR2KnWoFOQKuotirK6FQIazgUFBlC9CPP8fkF0TFKw >> 4OIzxQJ_Tp2Xe-mRb3swzYiungQAHNUVs53lxZSOLwP432qi3ovJ8cFw4G6XCXEkE37Xx2rGYhrsH >> iZQy03zUFRmYy13QckBzIrtfY1zG5piXPVIOCfIoFfvm4o-tUjgyZSzspCUye23lLO2G1ic7fjL5O >> sxTrokI/http%3A%2F%2Fwww.open-mpi.org%2Fcommunity%2Flists%2Fusers%2F2016%2F01 >> %2F28255.php for the thread in question. >> > >> > There is example code in MPI that uses this behavior. George and I agree >> that example text is non-normative, but I cannot imagine how this example >> came to exist without the Forum believing it to be valid. >> > >> > Jeff >> > >> > -- >> > Jeff Hammond >> > jeff.science(a)gmail.com >> > http://jeffhammond.github.io/ >> > _______________________________________________ >> > mpi-forum mailing list >> > mpi-forum(a)lists.mpi-forum.org >> > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum >> > >> > >> > >> > >> > >> > -- >> > Jeff Hammond >> > jeff.science(a)gmail.com >> > http://jeffhammond.github.io/ >> > _______________________________________________ >> > mpi-forum mailing list >> > mpi-forum(a)lists.mpi-forum.org >> > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum >> > >> > >> > _______________________________________________ >> > mpi-forum mailing list >> > mpi-forum(a)lists.mpi-forum.org >> > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum >> > >> > >> > >> > _______________________________________________ >> > mpi-forum mailing list >> > mpi-forum(a)lists.mpi-forum.org >> > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum > > > > > -- > Jeff Hammond > jeff.science(a)gmail.com > http://jeffhammond.github.io/ > > _______________________________________________ > mpi-forum mailing list > mpi-forum(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum -- Jeff Hammond jeff.science(a)gmail.com http://jeffhammond.github.io/

1 0

Re: [Mpi-forum] MPI_DATATYPE_NULL when count=0
by Jeff Hammond 13 Jan '16

13 Jan '16

Can you provide the specifics of a tool that does this today? I'd like to read the source code to see what changes are actually required. How can you piggyback an MPI collective call by hacking the datatype if count=0? You already have to check if count=0 and bump it to count=1 if you are going to send a piggy for a null transfer, so how hard is it to replace MPI_DATATYPE_NULL with MPI_BYTE? If you are not sending a piggy for a null transfer, then the issue is moot. If nothing else, the overhead of implementing the trivial branch that solves this issue will be negligible compared to firing up the datatype engine. Jeff On Wed, Jan 13, 2016 at 11:41 AM, Schulz Martin <schulzm(a)llnl.gov> wrote: > In general, I agree that allowing any argument for datatype if the count > is zero makes sense. However, such a change (even if it is just allowing > more) can cause backwards compatibility problems: e.g., any PMPI tool or > other kind of internal wrapper library can currently have the assumption > that it is only passed valid datatypes and hence can use the datatype > without having to check. We, for example, use this to implement > piggybacking (by creating a new struct datatype that contains the original > one and adds a piggyback field). If we allowed arbitrary datatypes, we > would brake such codes, unless we then also allowed it for datatype > creation. > > If we wanted to change this, we probably need to be very careful and make > sure we are consistent across the standard and touch all routines that take > datatype arguments. Just adding the exception for P2P, or even for all > communication routines, seems insufficient. > > Martin > > > ________________________________________________________________________ > > ________________________________________________________________________ > Martin Schulz, schulzm(a)llnl.gov, http://scalability.llnl.gov/ > CASC @ Lawrence Livermore National Laboratory, Livermore, USA > > > From: mpi-forum <mpi-forum-bounces(a)lists.mpi-forum.org> on behalf of Jeff > Hammond <jeff.science(a)gmail.com> > Reply-To: Main mailing list <mpi-forum(a)lists.mpi-forum.org> > Date: Wednesday, January 13, 2016 at 6:24 AM > To: Main mailing list <mpi-forum(a)lists.mpi-forum.org> > > Subject: Re: [Mpi-forum] MPI_DATATYPE_NULL when count=0 > > That is precisely what I want: when count=0, the buffer and the datatype > arguments are ignored. There may be a good reason for MPI_DATATYPE_NULL to > be invalid when it is actually relevant, but when count=0, the datatype > argument is not relevant, because there are zero of them. > > We already state explicitly that arguments are ignored for MPI_IN_PLACE > and MPI_NO_OP: > > * The “in place” option for intracommunicators is specified by passing > MPI_IN_PLACE as the value of recvbuf at the root. In such a case, recvcount > and recvtype are ignored..." > > * "When MPI_NO_OP is specified as the operation, the origin_addr, > origin_count, and origin_datatype arguments are ignored." > > I see no reason why count=0 is different from these cases. > > Jeff > > On Wed, Jan 13, 2016 at 5:47 AM, William Gropp <wgropp(a)illinois.edu> > wrote: > > > > True, MPI_BOTTOM need not be null, but it might be, so the standard > doesn’t prohibit it. > > > > A carefully worded exception to the general rule for NULL handles would > be ok. I would still make it narrow; for example, just for the use of > MPI_DATATYPE_NULL in communication operations with a zero count. > > > > Bill > > > > William Gropp > > Director, Parallel Computing Institute > > Thomas M. Siebel Chair in Computer Science > > Chief Scientist, NCSA > > University of Illinois Urbana-Champaign > > > > > > > > > > > > On Jan 12, 2016, at 11:22 PM, Fab Tillier <ftillier(a)microsoft.com> > wrote: > > > > Don’t datatypes with absolute addresses rely on MPI_BOTTOM, and a > portable program can’t assume that MPI_BOTTOM == NULL? > > > > I think from an ease of use perspective, if I have a count of zero, > having to pick an arbitrary committed datatype is non-intuitive. If count > is zero, there is no buffer or datatype, and there should be no requirement > such non-existent datatype be committed. > > > > Cheers, > > -Fab > > > > From: mpi-forum [mailto:[email protected]] On > Behalf Of William Gropp > > Sent: Wednesday, 13 January 2016 4:52 p.m. > > To: Main MPI Forum mailing list <mpi-forum(a)lists.mpi-forum.org> > > Subject: Re: [Mpi-forum] MPI_DATATYPE_NULL when count=0 > > > > Yes, NULL buffers are valid - they aren’t an MPI object. There is no > prohibition about passing them; only if they are used to form an invalid > address (note that a datatype with absolute addresses relies on NULL as the > buffer address). > > > > Bill > > > > > > William Gropp > > Director, Parallel Computing Institute > > Thomas M. Siebel Chair in Computer Science > > Chief Scientist, NCSA > > University of Illinois Urbana-Champaign > > > > > > > > > > On Jan 12, 2016, at 9:49 PM, Jeff Hammond <jeff.science(a)gmail.com> > wrote: > > > > > > Who wrote Example 4.23 of MPI 3.1? > > > > I propose to add the exception rather than delete that example. I do not > want to have to use real datatypes when sending nothing. > > > > Is NULL a valid buffer handle? Can I pass null pointers when count=0? > > > > > > Jeff > > > > > > On Tuesday, January 12, 2016, William Gropp <wgropp(a)illinois.edu> wrote: > > > > The standard says that the null handles are invalid for input unless > explicitly permitted. There’s no exemption for a datatype argument where > the associated count is zero. > > > > Bill > > > > > > William Gropp > > Director, Parallel Computing Institute > > Thomas M. Siebel Chair in Computer Science > > Chief Scientist, NCSA > > University of Illinois Urbana-Champaign > > > > > > > > > > On Jan 12, 2016, at 9:26 PM, Jeff Hammond <jeff.science(a)gmail.com> > wrote: > > > > > > Is it allowed to communicate messages with > (buffer=NULL,count=0,type=MPI_DATATYPE_NULL)? > > > > George thinks MPI_DATATYPE_NULL cannot be used even when count=0, which > I think is ridiculous, particularly when one makes the analogy to > buffer=NULL. > > > > See http://www.open-mpi.org/community/lists/users/2016/01/28255.php for > the thread in question. > > > > There is example code in MPI that uses this behavior. George and I > agree that example text is non-normative, but I cannot imagine how this > example came to exist without the Forum believing it to be valid. > > > > Jeff > > > > -- > > Jeff Hammond > > jeff.science(a)gmail.com > > http://jeffhammond.github.io/ > > _______________________________________________ > > mpi-forum mailing list > > mpi-forum(a)lists.mpi-forum.org > > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum > > > > > > > > > > > > -- > > Jeff Hammond > > jeff.science(a)gmail.com > > http://jeffhammond.github.io/ > > _______________________________________________ > > mpi-forum mailing list > > mpi-forum(a)lists.mpi-forum.org > > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum > > > > > > _______________________________________________ > > mpi-forum mailing list > > mpi-forum(a)lists.mpi-forum.org > > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum > > > > > > > > _______________________________________________ > > mpi-forum mailing list > > mpi-forum(a)lists.mpi-forum.org > > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum > > > > > -- > Jeff Hammond > jeff.science(a)gmail.com > http://jeffhammond.github.io/ > > _______________________________________________ > mpi-forum mailing list > mpi-forum(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum > -- Jeff Hammond jeff.science(a)gmail.com http://jeffhammond.github.io/

1 0

Re: [Mpi-forum] MPI_DATATYPE_NULL when count=0
by Schulz Martin 13 Jan '16

13 Jan '16

In general, I agree that allowing any argument for datatype if the count is zero makes sense. However, such a change (even if it is just allowing more) can cause backwards compatibility problems: e.g., any PMPI tool or other kind of internal wrapper library can currently have the assumption that it is only passed valid datatypes and hence can use the datatype without having to check. We, for example, use this to implement piggybacking (by creating a new struct datatype that contains the original one and adds a piggyback field). If we allowed arbitrary datatypes, we would brake such codes, unless we then also allowed it for datatype creation. If we wanted to change this, we probably need to be very careful and make sure we are consistent across the standard and touch all routines that take datatype arguments. Just adding the exception for P2P, or even for all communication routines, seems insufficient. Martin ________________________________________________________________________ ________________________________________________________________________ Martin Schulz, schulzm(a)llnl.gov, http://scalability.llnl.gov/ CASC @ Lawrence Livermore National Laboratory, Livermore, USA From: mpi-forum <mpi-forum-bounces(a)lists.mpi-forum.org> on behalf of Jeff Hammond <jeff.science(a)gmail.com> Reply-To: Main mailing list <mpi-forum(a)lists.mpi-forum.org> Date: Wednesday, January 13, 2016 at 6:24 AM To: Main mailing list <mpi-forum(a)lists.mpi-forum.org> Subject: Re: [Mpi-forum] MPI_DATATYPE_NULL when count=0 That is precisely what I want: when count=0, the buffer and the datatype arguments are ignored. There may be a good reason for MPI_DATATYPE_NULL to be invalid when it is actually relevant, but when count=0, the datatype argument is not relevant, because there are zero of them. We already state explicitly that arguments are ignored for MPI_IN_PLACE and MPI_NO_OP: * The ³in place² option for intracommunicators is specified by passing MPI_IN_PLACE as the value of recvbuf at the root. In such a case, recvcount and recvtype are ignored..." * "When MPI_NO_OP is specified as the operation, the origin_addr, origin_count, and origin_datatype arguments are ignored." I see no reason why count=0 is different from these cases. Jeff On Wed, Jan 13, 2016 at 5:47 AM, William Gropp <wgropp(a)illinois.edu> wrote: > > True, MPI_BOTTOM need not be null, but it might be, so the standard doesn¹t prohibit it. > > A carefully worded exception to the general rule for NULL handles would be ok. I would still make it narrow; for example, just for the use of MPI_DATATYPE_NULL in communication operations with a zero count. > > Bill > > William Gropp > Director, Parallel Computing Institute > Thomas M. Siebel Chair in Computer Science > Chief Scientist, NCSA > University of Illinois Urbana-Champaign > > > > > > On Jan 12, 2016, at 11:22 PM, Fab Tillier <ftillier(a)microsoft.com> wrote: > > Don¹t datatypes with absolute addresses rely on MPI_BOTTOM, and a portable program can¹t assume that MPI_BOTTOM == NULL? > > I think from an ease of use perspective, if I have a count of zero, having to pick an arbitrary committed datatype is non-intuitive. If count is zero, there is no buffer or datatype, and there should be no requirement such non-existent datatype be committed. > > Cheers, > -Fab > > From: mpi-forum [mailto:[email protected]] On Behalf Of William Gropp > Sent: Wednesday, 13 January 2016 4:52 p.m. > To: Main MPI Forum mailing list <mpi-forum(a)lists.mpi-forum.org> > Subject: Re: [Mpi-forum] MPI_DATATYPE_NULL when count=0 > > Yes, NULL buffers are valid - they aren¹t an MPI object. There is no prohibition about passing them; only if they are used to form an invalid address (note that a datatype with absolute addresses relies on NULL as the buffer address). > > Bill > > > William Gropp > Director, Parallel Computing Institute > Thomas M. Siebel Chair in Computer Science > Chief Scientist, NCSA > University of Illinois Urbana-Champaign > > > > > On Jan 12, 2016, at 9:49 PM, Jeff Hammond <jeff.science(a)gmail.com> wrote: > > > Who wrote Example 4.23 of MPI 3.1? > > I propose to add the exception rather than delete that example. I do not want to have to use real datatypes when sending nothing. > > Is NULL a valid buffer handle? Can I pass null pointers when count=0? > > > Jeff > > > On Tuesday, January 12, 2016, William Gropp <wgropp(a)illinois.edu> wrote: > > The standard says that the null handles are invalid for input unless explicitly permitted. There¹s no exemption for a datatype argument where the associated count is zero. > > Bill > > > William Gropp > Director, Parallel Computing Institute > Thomas M. Siebel Chair in Computer Science > Chief Scientist, NCSA > University of Illinois Urbana-Champaign > > > > > On Jan 12, 2016, at 9:26 PM, Jeff Hammond <jeff.science(a)gmail.com> wrote: > > > Is it allowed to communicate messages with (buffer=NULL,count=0,type=MPI_DATATYPE_NULL)? > > George thinks MPI_DATATYPE_NULL cannot be used even when count=0, which I think is ridiculous, particularly when one makes the analogy to buffer=NULL. > > See http://www.open-mpi.org/community/lists/users/2016/01/28255.php for the thread in question. > > There is example code in MPI that uses this behavior. George and I agree that example text is non-normative, but I cannot imagine how this example came to exist without the Forum believing it to be valid. > > Jeff > > -- > Jeff Hammond > jeff.science(a)gmail.com > http://jeffhammond.github.io/ > _______________________________________________ > mpi-forum mailing list > mpi-forum(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum > > > > > > -- > Jeff Hammond > jeff.science(a)gmail.com > http://jeffhammond.github.io/ > _______________________________________________ > mpi-forum mailing list > mpi-forum(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum > > > _______________________________________________ > mpi-forum mailing list > mpi-forum(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum > > > > _______________________________________________ > mpi-forum mailing list > mpi-forum(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum -- Jeff Hammond jeff.science(a)gmail.com http://jeffhammond.github.io/

1 0

Re: [Mpi-forum] MPI_DATATYPE_NULL when count=0
by Jeff Hammond 13 Jan '16

13 Jan '16

That is precisely what I want: when count=0, the buffer and the datatype arguments are ignored. There may be a good reason for MPI_DATATYPE_NULL to be invalid when it is actually relevant, but when count=0, the datatype argument is not relevant, because there are zero of them. We already state explicitly that arguments are ignored for MPI_IN_PLACE and MPI_NO_OP: * The “in place” option for intracommunicators is specified by passing MPI_IN_PLACE as the value of recvbuf at the root. In such a case, recvcount and recvtype are ignored..." * "When MPI_NO_OP is specified as the operation, the origin_addr, origin_count, and origin_datatype arguments are ignored." I see no reason why count=0 is different from these cases. Jeff On Wed, Jan 13, 2016 at 5:47 AM, William Gropp <wgropp(a)illinois.edu> wrote: > > True, MPI_BOTTOM need not be null, but it might be, so the standard doesn’t prohibit it. > > A carefully worded exception to the general rule for NULL handles would be ok. I would still make it narrow; for example, just for the use of MPI_DATATYPE_NULL in communication operations with a zero count. > > Bill > > William Gropp > Director, Parallel Computing Institute > Thomas M. Siebel Chair in Computer Science > Chief Scientist, NCSA > University of Illinois Urbana-Champaign > > > > > > On Jan 12, 2016, at 11:22 PM, Fab Tillier <ftillier(a)microsoft.com> wrote: > > Don’t datatypes with absolute addresses rely on MPI_BOTTOM, and a portable program can’t assume that MPI_BOTTOM == NULL? > > I think from an ease of use perspective, if I have a count of zero, having to pick an arbitrary committed datatype is non-intuitive. If count is zero, there is no buffer or datatype, and there should be no requirement such non-existent datatype be committed. > > Cheers, > -Fab > > From: mpi-forum [mailto:[email protected]] On Behalf Of William Gropp > Sent: Wednesday, 13 January 2016 4:52 p.m. > To: Main MPI Forum mailing list <mpi-forum(a)lists.mpi-forum.org> > Subject: Re: [Mpi-forum] MPI_DATATYPE_NULL when count=0 > > Yes, NULL buffers are valid - they aren’t an MPI object. There is no prohibition about passing them; only if they are used to form an invalid address (note that a datatype with absolute addresses relies on NULL as the buffer address). > > Bill > > > William Gropp > Director, Parallel Computing Institute > Thomas M. Siebel Chair in Computer Science > Chief Scientist, NCSA > University of Illinois Urbana-Champaign > > > > > On Jan 12, 2016, at 9:49 PM, Jeff Hammond <jeff.science(a)gmail.com> wrote: > > > Who wrote Example 4.23 of MPI 3.1? > > I propose to add the exception rather than delete that example. I do not want to have to use real datatypes when sending nothing. > > Is NULL a valid buffer handle? Can I pass null pointers when count=0? > > > Jeff > > > On Tuesday, January 12, 2016, William Gropp <wgropp(a)illinois.edu> wrote: > > The standard says that the null handles are invalid for input unless explicitly permitted. There’s no exemption for a datatype argument where the associated count is zero. > > Bill > > > William Gropp > Director, Parallel Computing Institute > Thomas M. Siebel Chair in Computer Science > Chief Scientist, NCSA > University of Illinois Urbana-Champaign > > > > > On Jan 12, 2016, at 9:26 PM, Jeff Hammond <jeff.science(a)gmail.com> wrote: > > > Is it allowed to communicate messages with (buffer=NULL,count=0,type=MPI_DATATYPE_NULL)? > > George thinks MPI_DATATYPE_NULL cannot be used even when count=0, which I think is ridiculous, particularly when one makes the analogy to buffer=NULL. > > See http://www.open-mpi.org/community/lists/users/2016/01/28255.php for the thread in question. > > There is example code in MPI that uses this behavior. George and I agree that example text is non-normative, but I cannot imagine how this example came to exist without the Forum believing it to be valid. > > Jeff > > -- > Jeff Hammond > jeff.science(a)gmail.com > http://jeffhammond.github.io/ > _______________________________________________ > mpi-forum mailing list > mpi-forum(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum > > > > > > -- > Jeff Hammond > jeff.science(a)gmail.com > http://jeffhammond.github.io/ > _______________________________________________ > mpi-forum mailing list > mpi-forum(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum > > > _______________________________________________ > mpi-forum mailing list > mpi-forum(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum > > > > _______________________________________________ > mpi-forum mailing list > mpi-forum(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum -- Jeff Hammond jeff.science(a)gmail.com http://jeffhammond.github.io/

1 0

Re: [Mpi-forum] MPI_DATATYPE_NULL when count=0
by William Gropp 13 Jan '16

13 Jan '16

True, MPI_BOTTOM need not be null, but it might be, so the standard doesn’t prohibit it. A carefully worded exception to the general rule for NULL handles would be ok. I would still make it narrow; for example, just for the use of MPI_DATATYPE_NULL in communication operations with a zero count. Bill William Gropp Director, Parallel Computing Institute Thomas M. Siebel Chair in Computer Science Chief Scientist, NCSA University of Illinois Urbana-Champaign On Jan 12, 2016, at 11:22 PM, Fab Tillier <ftillier(a)microsoft.com> wrote: > Don’t datatypes with absolute addresses rely on MPI_BOTTOM, and a portable program can’t assume that MPI_BOTTOM == NULL? > > I think from an ease of use perspective, if I have a count of zero, having to pick an arbitrary committed datatype is non-intuitive. If count is zero, there is no buffer or datatype, and there should be no requirement such non-existent datatype be committed. > > Cheers, > -Fab > > From: mpi-forum [mailto:[email protected]] On Behalf Of William Gropp > Sent: Wednesday, 13 January 2016 4:52 p.m. > To: Main MPI Forum mailing list <mpi-forum(a)lists.mpi-forum.org> > Subject: Re: [Mpi-forum] MPI_DATATYPE_NULL when count=0 > > Yes, NULL buffers are valid - they aren’t an MPI object. There is no prohibition about passing them; only if they are used to form an invalid address (note that a datatype with absolute addresses relies on NULL as the buffer address). > > Bill > > William Gropp > Director, Parallel Computing Institute > Thomas M. Siebel Chair in Computer Science > Chief Scientist, NCSA > University of Illinois Urbana-Champaign > > > > > > On Jan 12, 2016, at 9:49 PM, Jeff Hammond <jeff.science(a)gmail.com> wrote: > > > Who wrote Example 4.23 of MPI 3.1? > > I propose to add the exception rather than delete that example. I do not want to have to use real datatypes when sending nothing. > > Is NULL a valid buffer handle? Can I pass null pointers when count=0? > > > Jeff > > > On Tuesday, January 12, 2016, William Gropp <wgropp(a)illinois.edu> wrote: > The standard says that the null handles are invalid for input unless explicitly permitted. There’s no exemption for a datatype argument where the associated count is zero. > > Bill > > William Gropp > Director, Parallel Computing Institute > Thomas M. Siebel Chair in Computer Science > Chief Scientist, NCSA > University of Illinois Urbana-Champaign > > > > > > On Jan 12, 2016, at 9:26 PM, Jeff Hammond <jeff.science(a)gmail.com> wrote: > > > Is it allowed to communicate messages with (buffer=NULL,count=0,type=MPI_DATATYPE_NULL)? > > George thinks MPI_DATATYPE_NULL cannot be used even when count=0, which I think is ridiculous, particularly when one makes the analogy to buffer=NULL. > > See http://www.open-mpi.org/community/lists/users/2016/01/28255.php for the thread in question. > > There is example code in MPI that uses this behavior. George and I agree that example text is non-normative, but I cannot imagine how this example came to exist without the Forum believing it to be valid. > > Jeff > > -- > Jeff Hammond > jeff.science(a)gmail.com > http://jeffhammond.github.io/ > _______________________________________________ > mpi-forum mailing list > mpi-forum(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum > > > > -- > Jeff Hammond > jeff.science(a)gmail.com > http://jeffhammond.github.io/ > _______________________________________________ > mpi-forum mailing list > mpi-forum(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum > > _______________________________________________ > mpi-forum mailing list > mpi-forum(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum

1 0

Re: [Mpi-forum] MPI_DATATYPE_NULL when count=0
by Fab Tillier 12 Jan '16

12 Jan '16

Don't datatypes with absolute addresses rely on MPI_BOTTOM, and a portable program can't assume that MPI_BOTTOM == NULL? I think from an ease of use perspective, if I have a count of zero, having to pick an arbitrary committed datatype is non-intuitive. If count is zero, there is no buffer or datatype, and there should be no requirement such non-existent datatype be committed. Cheers, -Fab From: mpi-forum [mailto:[email protected]] On Behalf Of William Gropp Sent: Wednesday, 13 January 2016 4:52 p.m. To: Main MPI Forum mailing list <mpi-forum(a)lists.mpi-forum.org> Subject: Re: [Mpi-forum] MPI_DATATYPE_NULL when count=0 Yes, NULL buffers are valid - they aren't an MPI object. There is no prohibition about passing them; only if they are used to form an invalid address (note that a datatype with absolute addresses relies on NULL as the buffer address). Bill William Gropp Director, Parallel Computing Institute Thomas M. Siebel Chair in Computer Science Chief Scientist, NCSA University of Illinois Urbana-Champaign On Jan 12, 2016, at 9:49 PM, Jeff Hammond <jeff.science(a)gmail.com<mailto:[email protected]>> wrote: Who wrote Example 4.23 of MPI 3.1? I propose to add the exception rather than delete that example. I do not want to have to use real datatypes when sending nothing. Is NULL a valid buffer handle? Can I pass null pointers when count=0? Jeff On Tuesday, January 12, 2016, William Gropp <wgropp(a)illinois.edu<mailto:[email protected]>> wrote: The standard says that the null handles are invalid for input unless explicitly permitted. There's no exemption for a datatype argument where the associated count is zero. Bill William Gropp Director, Parallel Computing Institute Thomas M. Siebel Chair in Computer Science Chief Scientist, NCSA University of Illinois Urbana-Champaign On Jan 12, 2016, at 9:26 PM, Jeff Hammond <jeff.science(a)gmail.com<javascript:_e(%7B%7D,'cvml','jeff.science(a)gmail.com');>> wrote: Is it allowed to communicate messages with (buffer=NULL,count=0,type=MPI_DATATYPE_NULL)? George thinks MPI_DATATYPE_NULL cannot be used even when count=0, which I think is ridiculous, particularly when one makes the analogy to buffer=NULL. See http://www.open-mpi.org/community/lists/users/2016/01/28255.php<https://na01.safelinks.protection.outlook.com/?url=https%3a%2f%2furldefense…> for the thread in question. There is example code in MPI that uses this behavior. George and I agree that example text is non-normative, but I cannot imagine how this example came to exist without the Forum believing it to be valid. Jeff -- Jeff Hammond jeff.science(a)gmail.com<javascript:_e(%7B%7D,'cvml','jeff.science(a)gmail.com');> http://jeffhammond.github.io/<https://na01.safelinks.protection.outlook.com/?url=https%3a%2f%2furldefense…> _______________________________________________ mpi-forum mailing list mpi-forum(a)lists.mpi-forum.org<javascript:_e(%7B%7D,'cvml','mpi-forum(a)lists.mpi-forum.org');> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum<https://na01.safelinks.protection.outlook.com/?url=https%3a%2f%2furldefense…> -- Jeff Hammond jeff.science(a)gmail.com<mailto:[email protected]> http://jeffhammond.github.io/<https://na01.safelinks.protection.outlook.com/?url=https%3a%2f%2furldefense…> _______________________________________________ mpi-forum mailing list mpi-forum(a)lists.mpi-forum.org<mailto:[email protected]> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum

1 0

Re: [Mpi-forum] MPI_DATATYPE_NULL when count=0
by William Gropp 12 Jan '16

12 Jan '16

Yes, NULL buffers are valid - they aren’t an MPI object. There is no prohibition about passing them; only if they are used to form an invalid address (note that a datatype with absolute addresses relies on NULL as the buffer address). Bill William Gropp Director, Parallel Computing Institute Thomas M. Siebel Chair in Computer Science Chief Scientist, NCSA University of Illinois Urbana-Champaign On Jan 12, 2016, at 9:49 PM, Jeff Hammond <jeff.science(a)gmail.com> wrote: > Who wrote Example 4.23 of MPI 3.1? > > I propose to add the exception rather than delete that example. I do not want to have to use real datatypes when sending nothing. > > Is NULL a valid buffer handle? Can I pass null pointers when count=0? > > Jeff > > On Tuesday, January 12, 2016, William Gropp <wgropp(a)illinois.edu> wrote: > The standard says that the null handles are invalid for input unless explicitly permitted. There’s no exemption for a datatype argument where the associated count is zero. > > Bill > > William Gropp > Director, Parallel Computing Institute > Thomas M. Siebel Chair in Computer Science > Chief Scientist, NCSA > University of Illinois Urbana-Champaign > > > > > > On Jan 12, 2016, at 9:26 PM, Jeff Hammond <jeff.science(a)gmail.com> wrote: > >> Is it allowed to communicate messages with (buffer=NULL,count=0,type=MPI_DATATYPE_NULL)? >> >> George thinks MPI_DATATYPE_NULL cannot be used even when count=0, which I think is ridiculous, particularly when one makes the analogy to buffer=NULL. >> >> See http://www.open-mpi.org/community/lists/users/2016/01/28255.php for the thread in question. >> >> There is example code in MPI that uses this behavior. George and I agree that example text is non-normative, but I cannot imagine how this example came to exist without the Forum believing it to be valid. >> >> Jeff >> >> -- >> Jeff Hammond >> jeff.science(a)gmail.com >> http://jeffhammond.github.io/ >> _______________________________________________ >> mpi-forum mailing list >> mpi-forum(a)lists.mpi-forum.org >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum > > > > -- > Jeff Hammond > jeff.science(a)gmail.com > http://jeffhammond.github.io/ > _______________________________________________ > mpi-forum mailing list > mpi-forum(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum

1 0