- Mpi3-bwcompat - mailman3.cels.anl.gov

Re: [Mpi3-bwcompat] MPI_Count mixing with int...
by unknown＠example.com 09 Jul '10

09 Jul '10

Rajeev Thakur wrote on Thu, 8 Jul 2010 at 22:06:24 > Yes, that is a long standing problem -- that datatypes use MPI_Aint > (wherever they do), but they really need to use the larger of MPI_Aint > or MPI_Offset. So an 'optimal' implementation would be one where MPI_Aint, MPI_Offset, MPI_Count, and MPI_Size all map to the same underlying type. Ugh, this is getting messy quickly. I think I'll just deal with the MPI_Count and MPI_Size issues, ignoring the MPI_Aint/MPI_Offset issues. Some poor soul a decade from now will likely curse me for it, but I don't want to get into deprecating a bunch of functions (those with Aint) to normalize them too. So to summarize, my plan is as follows: 1. Define MPI_Size (instead of MPI_Count) 2. Use MPI_Size as the type for all size, count, and position variables. 3. Ignore MPI_Aint and MPI_Offset inconsistencies. If anyone has issues with this, please holler sooner than later. Thanks, -Fab > >> -----Original Message----- >> From: Fab Tillier >> Sent: Thursday, July 08, 2010 11:20 PM >> >> Rajeev Thakur wrote on Thu, 8 Jul 2010 at 18:24:16 >> >>> You can define a filetype that is many Gbytes and read only a small >>> part of it. The filetype is specified in MPI_File_set_view and is not >>> passed in the read/write calls. The datatype passed in the read/write >>> calls refers to the buffer passed to that call. >> >> Perfect, thanks for the example! Does this imply that >> MPI_Type_get_extent, which returns an MPI_Aint, might >> overflow the extent parameter on a 32-bit platform on which a >> user defines a many Gbyte filetype? How should we address >> this (there are probably other APIs affected similarly)? >> >> -Fab >> >>>> -----Original Message----- >>>> From: Fab Tillier >>>> Sent: Thursday, July 08, 2010 3:32 PM >>>> >>>> Rajeev Thakur wrote on Thu, 8 Jul 2010 at 12:52:58 >>>> >>>>>> As an aside, can count ever be larger than size_t? >>>>> >>>>> Probably not in the communication or I/O functions since count >>>>> refers to the number of items in the memory buffer, but it could be >>>>> for the datatype constructor functions since datatypes can >>>> represent file layout. >>>> >>>> How would you use a datatype that is larger than your address space? >>>> You can't read or write it, you can't send or receive it, you can't >>>> pack/unpack it... >>>> >>>> Am I missing something and a datatype like this would be useful? >>>> >>>> -Fab

1 0

Re: [Mpi3-bwcompat] MPI_Count mixing with int...
by unknown＠example.com 09 Jul '10

09 Jul '10

Yes, that is a long standing problem -- that datatypes use MPI_Aint (wherever they do), but they really need to use the larger of MPI_Aint or MPI_Offset. Rajeev > -----Original Message----- > From: mpi3-bwcompat-bounces_at_[hidden] > [mailto:mpi3-bwcompat-bounces_at_[hidden]] On Behalf Of Fab Tillier > Sent: Thursday, July 08, 2010 11:20 PM > To: MPI-3 backwards compatability WG > Subject: Re: [Mpi3-bwcompat] MPI_Count mixing with int... > > Rajeev Thakur wrote on Thu, 8 Jul 2010 at 18:24:16 > > > You can define a filetype that is many Gbytes and read only a small > > part of it. The filetype is specified in MPI_File_set_view > and is not > > passed in the read/write calls. The datatype passed in the > read/write > > calls refers to the buffer passed to that call. > > Perfect, thanks for the example! Does this imply that > MPI_Type_get_extent, which returns an MPI_Aint, might > overflow the extent parameter on a 32-bit platform on which a > user defines a many Gbyte filetype? How should we address > this (there are probably other APIs affected similarly)? > > -Fab > > >> -----Original Message----- > >> From: Fab Tillier > >> Sent: Thursday, July 08, 2010 3:32 PM > >> > >> Rajeev Thakur wrote on Thu, 8 Jul 2010 at 12:52:58 > >> > >>>> As an aside, can count ever be larger than size_t? > >>> > >>> Probably not in the communication or I/O functions since count > >>> refers to the number of items in the memory buffer, but > it could be > >>> for the datatype constructor functions since datatypes can > >> represent file layout. > >> > >> How would you use a datatype that is larger than your > address space? > >> You can't read or write it, you can't send or receive it, > you can't > >> pack/unpack it... > >> > >> Am I missing something and a datatype like this would be useful? > >> > >> -Fab > > _______________________________________________ > Mpi3-bwcompat mailing list > Mpi3-bwcompat_at_[hidden] > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-bwcompat >

1 0

Re: [Mpi3-bwcompat] MPI_Count mixing with int...
by unknown＠example.com 08 Jul '10

08 Jul '10

Rajeev Thakur wrote on Thu, 8 Jul 2010 at 18:24:16 > You can define a filetype that is many Gbytes and read only a small part > of it. The filetype is specified in MPI_File_set_view and is not passed > in the read/write calls. The datatype passed in the read/write calls > refers to the buffer passed to that call. Perfect, thanks for the example! Does this imply that MPI_Type_get_extent, which returns an MPI_Aint, might overflow the extent parameter on a 32-bit platform on which a user defines a many Gbyte filetype? How should we address this (there are probably other APIs affected similarly)? -Fab >> -----Original Message----- >> From: Fab Tillier >> Sent: Thursday, July 08, 2010 3:32 PM >> >> Rajeev Thakur wrote on Thu, 8 Jul 2010 at 12:52:58 >> >>>> As an aside, can count ever be larger than size_t? >>> >>> Probably not in the communication or I/O functions since count refers >>> to the number of items in the memory buffer, but it could be for the >>> datatype constructor functions since datatypes can >> represent file layout. >> >> How would you use a datatype that is larger than your address >> space? You can't read or write it, you can't send or receive >> it, you can't pack/unpack it... >> >> Am I missing something and a datatype like this would be useful? >> >> -Fab

1 0

Re: [Mpi3-bwcompat] MPI_Count mixing with int...
by unknown＠example.com 08 Jul '10

08 Jul '10

You can define a filetype that is many Gbytes and read only a small part of it. The filetype is specified in MPI_File_set_view and is not passed in the read/write calls. The datatype passed in the read/write calls refers to the buffer passed to that call. Rajeev > -----Original Message----- > From: mpi3-bwcompat-bounces_at_[hidden] > [mailto:mpi3-bwcompat-bounces_at_[hidden]] On Behalf Of Fab Tillier > Sent: Thursday, July 08, 2010 3:32 PM > To: MPI-3 backwards compatability WG > Subject: Re: [Mpi3-bwcompat] MPI_Count mixing with int... > > Rajeev Thakur wrote on Thu, 8 Jul 2010 at 12:52:58 > > >> As an aside, can count ever be larger than size_t? > > > > Probably not in the communication or I/O functions since > count refers > > to the number of items in the memory buffer, but it could > be for the > > datatype constructor functions since datatypes can > represent file layout. > > How would you use a datatype that is larger than your address > space? You can't read or write it, you can't send or receive > it, you can't pack/unpack it... > > Am I missing something and a datatype like this would be useful? > > -Fab > > _______________________________________________ > Mpi3-bwcompat mailing list > Mpi3-bwcompat_at_[hidden] > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-bwcompat >

1 0

Re: [Mpi3-bwcompat] Meeting schedule
by unknown＠example.com 08 Jul '10

08 Jul '10

I would much prefer a later meeting time, early mornings are not good for me given morning activities of getting the kids out the door and off to their various activities. Something like 9AM PST or later. :) -Fab Solt, David George wrote on Thu, 8 Jul 2010 at 13:41:09 > Our reoccurring backwards compatibility meeting expired. It was set for > every other Friday at 10am EST. I can just fire it up again, but want > to check in and see if we want to keep that time. If you would attend > if the meeting was at a different time or if you attend regularly but > would prefer a different time, please shoot me an e-mail. If I don't > hear from many people I will fire it up again starting next week. > > Dave > > _______________________________________________ > Mpi3-bwcompat mailing list > Mpi3-bwcompat_at_[hidden] > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-bwcompat

1 0

Meeting schedule
by unknown＠example.com 08 Jul '10

08 Jul '10

Our reoccurring backwards compatibility meeting expired. It was set for every other Friday at 10am EST. I can just fire it up again, but want to check in and see if we want to keep that time. If you would attend if the meeting was at a different time or if you attend regularly but would prefer a different time, please shoot me an e-mail. If I don't hear from many people I will fire it up again starting next week. Dave

1 0

Re: [Mpi3-bwcompat] MPI_Count mixing with int...
by unknown＠example.com 08 Jul '10

08 Jul '10

Jeff Squyres wrote on Thu, 8 Jul 2010 at 10:29:37 > On Jul 8, 2010, at 12:26 PM, Fab Tillier wrote: > >>> I agree - we shouldn't conflate counts and sizes, they are >>> totally different things. Defining them both makes sense to me. >> >> But they are the same for the MPI_PACKED datatype, so really >> shouldn't be different. > > ...and let's not forget that MPI_Aint's are used for displacements in > MPI_GEt, MPI_PUT, and MPI_ACCUMULATE. :-\ Should we disentangle > those, too? No, because MPI_Aint is as close as the standard comes to size_t, which is the right type. I would anticipate MPI implementations defining MPI_Size/MPI_Count as either int for back compat, or MPI_Aint. Though I wouldn't be surprised if some platform has some weird MPI_Aint limitation, but in my cosy little world of Windows, MPI_Aint is really size_t. >> As an aside, can count ever be larger than size_t? > > I don't think so, but why do you ask? Because if count could be larger than size_t, but MPI_Size cannot, you have a mismatch that we need to make sure we understand. In this context, MPI_Size is the type used for functions that should have used MPI_Aint but didn't for historical reasons such as MPI_Pack/Unpack/Type_size/Pack_size. -Fab

1 0

Re: [Mpi3-bwcompat] MPI_Count mixing with int...
by unknown＠example.com 08 Jul '10

08 Jul '10

Rajeev Thakur wrote on Thu, 8 Jul 2010 at 12:52:58 >> As an aside, can count ever be larger than size_t? > > Probably not in the communication or I/O functions since count refers to > the number of items in the memory buffer, but it could be for the > datatype constructor functions since datatypes can represent file layout. How would you use a datatype that is larger than your address space? You can't read or write it, you can't send or receive it, you can't pack/unpack it... Am I missing something and a datatype like this would be useful? -Fab

1 0

Re: [Mpi3-bwcompat] MPI_Count mixing with int...
by unknown＠example.com 08 Jul '10

08 Jul '10

Jeff Squyres wrote on Thu, 8 Jul 2010 at 10:34:46 > On Jul 8, 2010, at 1:32 PM, Quincey Koziol wrote: > >>> As an aside, can count ever be larger than size_t? >> >> I would imagine so - on a 32-bit CPU/system with a 64-bit > file system. > > Oh ya, that whole filesystem thing. :-) But even on a 32-bit system, you can't do more than a 32-bit file access at a time. Even if you created a datatype that represented more than 2^31 bytes, you wouldn't be able to use it in any of the functions because you wouldn't be able to transfer that much data at once. > But still -- why do you ask? I am hoping you don't plan to mention > "size_t" in the standard at all... I ask because MPI_Size (for MPI_Type_size, MPI_Pack_size, and the position parameters of MPI_Pack/Unpack) would likely be size_t - how could you use more than the addressable space? I also think that MPI_Count would reasonably be size_t also, since I pretty firmly believe that MPI_Count and MPI_Size should be the same type. -Fab

1 0

Re: [Mpi3-bwcompat] MPI_Count mixing with int...
by unknown＠example.com 08 Jul '10

08 Jul '10

> As an aside, can count ever be larger than size_t? Probably not in the communication or I/O functions since count refers to the number of items in the memory buffer, but it could be for the datatype constructor functions since datatypes can represent file layout. Rajeev

1 0