October 2019 - mpiwg-large-counts - mailman3.cels.anl.gov

Re: [Mpiwg-large-counts] Large Count - the principles for counts, sizes, and byte and nonbyte displacements
by HOLMES Daniel 24 Oct '19

24 Oct '19

Hi Rolf & Jeff, I think this wiki article is instructive on this topic also: https://en.wikipedia.org/wiki/X86_memory_segmentation This seems like a crazy memory addressing system to me personally, but it is a (historic) example of a segmented addressing approach that MPI_Aint can support. The “strange properties” for arithmetic are strange indeed, depending on what the MPI_Aint stores and how. If MPI_Aint was 20 bits long and stores only the address, then it cannot be used to determine uniquely which segment is being used or what the offset is within that segment (there are 4096 possible answers). Does MPI need that more detailed information? Probably - because segments were a way of implementing memory protection, i.e. accessing a segment you did not have permission to access led to a “segmentation fault” error. I do not know enough about these old architectures to say whether an attempt to access the *same byte* using two different segment:offset pairs that produce the *same* address could result in different behaviour. That is, if I have access permissions for segment 3 but not for segment 4, I can access {seg=3,offset=2^16-16} but can I access {segment=4,offset=2^16-32}, which is the same byte? If not, then MPI needs to store segment and offset inside MPI_Aint to be able to check and to set registers correctly. If MPI_Aint is 32 bits long and stores the segment in the first 16 bits and the offset in the last 16 bits, then the 20 bit address can be computed in a single simple instruction and both segment and offset are immediately retrievable. However, doing ordinary arithmetic with this bitwise representation is unwise because it is a compound structure type. Let us subtract 1 from an MPI_Aint of this layout which stores offset of 0 and some non-zero segment. We get offset (2^16-1) in segment (s-1), which is not 1 byte before the previous MPI_Aint because segments overlap. The same happens when adding and overflowing the offset portion - it changes the segment in an incorrect way. Segment++ moves the address forward only 16 bytes, not 2^16 bytes. The wrap-around from the end of the address space back to the beginning is also a source of strange properties for arithmetic. One of the key statements from that wiki page is this: The root of the problem is that no appropriate address-arithmetic instructions suitable for flat addressing of the entire memory range are available.[citation needed] Flat addressing is possible by applying multiple instructions, which however leads to slower programs. Cheers, Dan. — Dr Daniel Holmes PhD Architect (HPC Research) d.holmes(a)epcc.ed.ac.uk<mailto:[email protected]> Phone: +44 (0) 131 651 3465 Mobile: +44 (0) 7940 524 088 Address: Room 2.09, Bayes Centre, 47 Potterrow, Central Area, Edinburgh, EH8 9BT — The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. — On 24 Oct 2019, at 17:27, Jeff Squyres (jsquyres) via mpiwg-large-counts <mpiwg-large-counts(a)lists.mpi-forum.org<mailto:[email protected]>> wrote: On Oct 24, 2019, at 11:15 AM, Rolf Rabenseifner <rabenseifner(a)hlrs.de<mailto:[email protected]>> wrote: For me, it looked like that there was some misunderstanding of the concept that absolute and relative addresses and number of bytes that can be stored in MPI_Aint. ...with the caveat that MPI_Aint -- as it is right now -- does not support modern segmented memory systems (i.e., where you need more than a small number of bits to indicate the segment where the memory lives). I think that changes the conversation entirely, right? -- Jeff Squyres jsquyres(a)cisco.com<mailto:[email protected]> _______________________________________________ mpiwg-large-counts mailing list mpiwg-large-counts(a)lists.mpi-forum.org<mailto:[email protected]> https://lists.mpi-forum.org/mailman/listinfo/mpiwg-large-counts

1 0

Re: [Mpiwg-large-counts] Large Count - the principles for counts, sizes, and byte and nonbyte displacements
by Jeff Squyres (jsquyres) 24 Oct '19

24 Oct '19

On Oct 24, 2019, at 11:15 AM, Rolf Rabenseifner <rabenseifner(a)hlrs.de<mailto:[email protected]>> wrote: For me, it looked like that there was some misunderstanding of the concept that absolute and relative addresses and number of bytes that can be stored in MPI_Aint. ...with the caveat that MPI_Aint -- as it is right now -- does not support modern segmented memory systems (i.e., where you need more than a small number of bits to indicate the segment where the memory lives). I think that changes the conversation entirely, right? -- Jeff Squyres jsquyres(a)cisco.com<mailto:[email protected]>

1 0

Re: [Mpiwg-large-counts] Large Count - the principles for counts, sizes, and byte and nonbyte displacements
by Rolf Rabenseifner 24 Oct '19

24 Oct '19

I was reading web pages about segmented address space. And they show two possibilities (which may go only into the past): - flattened address space - segmented with some high order bits refering to the Segment and low order bits to address within the segment. If I understood correctly, then the major problem behind MPI_Aint_diff and MPI_Aint_add was the following: (my example is with 8 bit MPI_Aint) addr1 01111111 = 127 (signed int) = 127 (unsigned int) addr2 10000001 = -127 (signed int) = 129 (unsigned int) Internally the addreses are viewed by the hardware and OS as unsigned. MPI_Aint is interpreting the same bits as signed int. addr2-addr1 = 129 -127 = 2 (as unsigned int) but in a real application code with "-" operator: = -127 -127 = -254 --> signed int Overflow because 8 bit can express only -128 .. +127 --> detected or automatically corrected with +256 --> -254+256 = 2 The segmented problem was solved very early through the wording on "sequential storage" (unchanged since MPI-1.1 !!! :-) in the section "Correct use of addresses". What I see, that before changing anything, the mpiwg-large-counts WG should really understand MPI-3.1. For me, it looked like that there was some misunderstanding of the concept that absolute and relative addresses and number of bytes that can be stored in MPI_Aint. Best regards Rolf ----- Original Message ----- > From: "Jeff Squyres" <jsquyres(a)cisco.com> > To: "Rolf Rabenseifner" <rabenseifner(a)hlrs.de> > Cc: "mpiwg-large-counts" <mpiwg-large-counts(a)lists.mpi-forum.org> > Sent: Thursday, October 24, 2019 3:28:47 PM > Subject: Re: [Mpiwg-large-counts] Large Count - the principles for counts, sizes, and byte and nonbyte displacements > On Oct 24, 2019, at 2:19 AM, Rolf Rabenseifner <rabenseifner(a)hlrs.de> wrote: >> >>> Does MPI support segmented address spaces or not? >> >> When I understand correctly then MPI is and was always prepared for segmented >> address spaces as long as all integer types are signed integers and the >> segmentation is done through some higher order bits. > > > As I was pondering your replies from yesterday and thinking about how to > respond, this is exactly the same point that I got to. > > I think what I have been failing to elucidate is that this is not how modern > segmented address spaces work. It's not sufficient to have a few bits to > specify which segment to use. Instead, you typically need three things: > > 1. The type of memory segment being described (which may be just a few bits). > In reality, this "type" may correlate to which API you need to use to access > this memory (e.g., main RAM, RDMA, accelerator memory, persistent memory, other > IO memory, ...etc.). > 2. A context of some kind (usually more than a small number of bits -- perhaps > 32 or 64 bits). > 3. A pointer (usually 32 or 64 bits). > > For example, if I'm accessing IO memory of some flavor, I probably need some > kind of context/handle that points to a descriptor with information about my > data segment on the IO peripheral in question (e.g., a NIC), and then a pointer > for the specific place that I want in that specific memory. I use this > context/handle + pointer with some underlying API to get/set that data. > > I think that's the disconnect here: the prior thinking about MPI_Aint may have > assumed "it's either main RAM or it's XYZ memory, and the rest of the bits are > enough to uniquely describe the address." But that doesn't seem sufficient any > more to describe the evolution of complex, segmented memory systems (and be > able to use their supporting APIs). > > Meaning: perhaps a pure integer MPI_Aint may have been sufficient in the past. > But I'm not sure that's useful any more. > > -- > Jeff Squyres > jsquyres(a)cisco.com -- Dr. Rolf Rabenseifner . . . . . . . . . .. email rabenseifner(a)hlrs.de . High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 . University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 . Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner . Nobelstr. 19, D-70550 Stuttgart, Germany . . . . (Office: Room 1.307) .

1 0

Re: [Mpiwg-large-counts] Large Count - the principles for counts, sizes, and byte and nonbyte displacements
by Jeff Squyres (jsquyres) 24 Oct '19

24 Oct '19

On Oct 24, 2019, at 2:19 AM, Rolf Rabenseifner <rabenseifner(a)hlrs.de> wrote: > >> Does MPI support segmented address spaces or not? > > When I understand correctly then MPI is and was always prepared for segmented address spaces as long as all integer types are signed integers and the segmentation is done through some higher order bits. As I was pondering your replies from yesterday and thinking about how to respond, this is exactly the same point that I got to. I think what I have been failing to elucidate is that this is not how modern segmented address spaces work. It's not sufficient to have a few bits to specify which segment to use. Instead, you typically need three things: 1. The type of memory segment being described (which may be just a few bits). In reality, this "type" may correlate to which API you need to use to access this memory (e.g., main RAM, RDMA, accelerator memory, persistent memory, other IO memory, ...etc.). 2. A context of some kind (usually more than a small number of bits -- perhaps 32 or 64 bits). 3. A pointer (usually 32 or 64 bits). For example, if I'm accessing IO memory of some flavor, I probably need some kind of context/handle that points to a descriptor with information about my data segment on the IO peripheral in question (e.g., a NIC), and then a pointer for the specific place that I want in that specific memory. I use this context/handle + pointer with some underlying API to get/set that data. I think that's the disconnect here: the prior thinking about MPI_Aint may have assumed "it's either main RAM or it's XYZ memory, and the rest of the bits are enough to uniquely describe the address." But that doesn't seem sufficient any more to describe the evolution of complex, segmented memory systems (and be able to use their supporting APIs). Meaning: perhaps a pure integer MPI_Aint may have been sufficient in the past. But I'm not sure that's useful any more. -- Jeff Squyres jsquyres(a)cisco.com

1 0

Re: [Mpiwg-large-counts] Large Count - the principles for counts, sizes, and byte and nonbyte displacements
by Rolf Rabenseifner 24 Oct '19

24 Oct '19

Dear Jeff and all, > Does MPI support segmented address spaces or not? When I understand correctly then MPI is and was always prepared for segmented address spaces as long as all integer types are signed integers and the segmentation is done through some higher order bits. Best regards Rolf ----- Rolf Rabenseifner via mpiwg-large-counts <mpiwg-large-counts(a)lists.mpi-forum.org> wrote: > See below > > ----- Original Message ----- > > From: "Jeff Squyres" <jsquyres(a)cisco.com> > > To: "Rolf Rabenseifner" <rabenseifner(a)hlrs.de> > > Cc: "mpiwg-large-counts" <mpiwg-large-counts(a)lists.mpi-forum.org> > > Sent: Wednesday, October 23, 2019 5:12:33 PM > > Subject: Re: [Mpiwg-large-counts] Large Count - the principles for counts, sizes, and byte and nonbyte displacements > > > On Oct 23, 2019, at 10:44 AM, Rolf Rabenseifner <rabenseifner(a)hlrs.de> wrote: > >> > >>> typedef struct { uint64_t context; uint64_t pointer; } MPI_Aint; > >>> > >>> Is that incorrect? > >> > >> I expect, that it is not correct. My apologies for that. > >> > >> In MPI-1.0 - MPI-2.1 all address calculation had to be done with - and + > >> operators. > >> Only for the case, that somebody switches integer overflow detection on, > >> we added MPI_Aint_add and MPI_Aint_diff. > > > > I'm confused with how to reconcile that with MPI-3.1 4.1.12 "Correct Use of > > Addresses", p115:13-15: > > > > "Also, in machines with a segmented address space, addresses are not unique and > > address arithmetic has some peculiar properties. Thus, the use of addresses, > > that is, displacements relative to the start address MPI_BOTTOM, has to be > > restricted." > > > > Doesn't this explicitly state that segmented address spaces are supported? > > Yes, some higher bits may be used as a flag. > If two absolute addresses are within the same array or structure (i.e. variable), > i.e., within the save sequential storage, then a minus operator must still work. > > > >> We asked ourself, whether we want to deprecate the use of + and - operators, > >> i.e., to add this to Table 2.1 on page 18 and the forum decided "no" per straw > >> vote. > >> We definitely never removed the operators + and - from the list of > >> valid operations for MPI_Aint. > > > > I'm not sure how to reconcile your statements with MPI-3.1 2.5.6 "Absolute > > Addresses and Relative Address Displacements" p16:39-43: > > > > "For retrieving absolute addresses or any calculation with absolute addresses, > > one should use the routines and functions provided in Section 4.1.5. Section > > 4.1.12 provides additional rules for the correct use of absolute addresses. For > > expressions with relative displacements or other usage without absolute > > addresses, intrinsic operators (e.g., +, -, *) can be used." > > > > If MPI_Aint is supposed to be used for absolute addresses, this tells me that > > MPI_Aint_add/diff must be used for all mathematical operations. > > The following definitions apply in understanding how to implement an > ISO International Standard and other normative ISO deliverables (TS, PAS, IWA). > > - "shall" indicates a requirement. > - "should" indicates a recommendation. > > > ... any calculation with absolute addresses, > > one ***should*** use the routines and functions provided in Section 4.1.5 > > ... indicates a recommendation, not a must > > > > >> MPI-3.1 on page 17 lines 16-18 clearly tells that any int, MPI:Aint, and > >> MPI_Offset value > >> can be assigned to a MPI_Count variable, i.e., all 4 must be integers of some > >> byte-size. > > > > I think it actually says something subtly different than that. It says: > > > > "The size of the MPI_Count type is determined by the MPI implementation with the > > restriction that it must be minimally capable of encoding any value that may be > > stored in a variable of type int, MPI_Aint, or MPI_Offset in C and of type > > INTEGER, INTEGER (KIND=MPI_ADDRESS_KIND), or INTEGER (KIND=MPI_OFFSET_KIND) in > > Fortran." > > > > Meaning: an MPI_Count must be big enough to hold an MPI_Aint. It does not say > > that you can assign one to the other. In my hypothetical "typedef struct" > > example, this means that MPI_Count would need to be a 128 bit value. > > > > ---- > > > > All this being said, what all this discussion probably means is that there are > > discrepancies in the standard that should be fixed and/or clarified. But I > > think the discussion starts with a fundamental question: > > > > Does MPI support segmented address spaces or not? > > > > -- > > Jeff Squyres > > jsquyres(a)cisco.com > > -- > Dr. Rolf Rabenseifner . . . . . . . . . .. email rabenseifner(a)hlrs.de . > High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 . > University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 . > Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner . > Nobelstr. 19, D-70550 Stuttgart, Germany . . . . (Office: Room 1.307) . > _______________________________________________ > mpiwg-large-counts mailing list > mpiwg-large-counts(a)lists.mpi-forum.org > https://lists.mpi-forum.org/mailman/listinfo/mpiwg-large-counts -- Dr. Rolf Rabenseifner . . . . . . . . . .. email rabenseifner(a)hlrs.de . High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 . University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 . Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner . Nobelstr. 19, D-70550 Stuttgart, Germany . . . . (Office: Room 1.307) .

1 0

Re: [Mpiwg-large-counts] Large Count - the principles for counts, sizes, and byte and nonbyte displacements
by Rolf Rabenseifner 23 Oct '19

23 Oct '19

See below ----- Original Message ----- > From: "Jeff Squyres" <jsquyres(a)cisco.com> > To: "Rolf Rabenseifner" <rabenseifner(a)hlrs.de> > Cc: "mpiwg-large-counts" <mpiwg-large-counts(a)lists.mpi-forum.org> > Sent: Wednesday, October 23, 2019 5:12:33 PM > Subject: Re: [Mpiwg-large-counts] Large Count - the principles for counts, sizes, and byte and nonbyte displacements > On Oct 23, 2019, at 10:44 AM, Rolf Rabenseifner <rabenseifner(a)hlrs.de> wrote: >> >>> typedef struct { uint64_t context; uint64_t pointer; } MPI_Aint; >>> >>> Is that incorrect? >> >> I expect, that it is not correct. My apologies for that. >> >> In MPI-1.0 - MPI-2.1 all address calculation had to be done with - and + >> operators. >> Only for the case, that somebody switches integer overflow detection on, >> we added MPI_Aint_add and MPI_Aint_diff. > > I'm confused with how to reconcile that with MPI-3.1 4.1.12 "Correct Use of > Addresses", p115:13-15: > > "Also, in machines with a segmented address space, addresses are not unique and > address arithmetic has some peculiar properties. Thus, the use of addresses, > that is, displacements relative to the start address MPI_BOTTOM, has to be > restricted." > > Doesn't this explicitly state that segmented address spaces are supported? Yes, some higher bits may be used as a flag. If two absolute addresses are within the same array or structure (i.e. variable), i.e., within the save sequential storage, then a minus operator must still work. >> We asked ourself, whether we want to deprecate the use of + and - operators, >> i.e., to add this to Table 2.1 on page 18 and the forum decided "no" per straw >> vote. >> We definitely never removed the operators + and - from the list of >> valid operations for MPI_Aint. > > I'm not sure how to reconcile your statements with MPI-3.1 2.5.6 "Absolute > Addresses and Relative Address Displacements" p16:39-43: > > "For retrieving absolute addresses or any calculation with absolute addresses, > one should use the routines and functions provided in Section 4.1.5. Section > 4.1.12 provides additional rules for the correct use of absolute addresses. For > expressions with relative displacements or other usage without absolute > addresses, intrinsic operators (e.g., +, -, *) can be used." > > If MPI_Aint is supposed to be used for absolute addresses, this tells me that > MPI_Aint_add/diff must be used for all mathematical operations. The following definitions apply in understanding how to implement an ISO International Standard and other normative ISO deliverables (TS, PAS, IWA). - "shall" indicates a requirement. - "should" indicates a recommendation. > ... any calculation with absolute addresses, > one ***should*** use the routines and functions provided in Section 4.1.5 ... indicates a recommendation, not a must > >> MPI-3.1 on page 17 lines 16-18 clearly tells that any int, MPI:Aint, and >> MPI_Offset value >> can be assigned to a MPI_Count variable, i.e., all 4 must be integers of some >> byte-size. > > I think it actually says something subtly different than that. It says: > > "The size of the MPI_Count type is determined by the MPI implementation with the > restriction that it must be minimally capable of encoding any value that may be > stored in a variable of type int, MPI_Aint, or MPI_Offset in C and of type > INTEGER, INTEGER (KIND=MPI_ADDRESS_KIND), or INTEGER (KIND=MPI_OFFSET_KIND) in > Fortran." > > Meaning: an MPI_Count must be big enough to hold an MPI_Aint. It does not say > that you can assign one to the other. In my hypothetical "typedef struct" > example, this means that MPI_Count would need to be a 128 bit value. > > ---- > > All this being said, what all this discussion probably means is that there are > discrepancies in the standard that should be fixed and/or clarified. But I > think the discussion starts with a fundamental question: > > Does MPI support segmented address spaces or not? > > -- > Jeff Squyres > jsquyres(a)cisco.com -- Dr. Rolf Rabenseifner . . . . . . . . . .. email rabenseifner(a)hlrs.de . High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 . University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 . Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner . Nobelstr. 19, D-70550 Stuttgart, Germany . . . . (Office: Room 1.307) .

1 0

Re: [Mpiwg-large-counts] Large Count - the principles for counts, sizes, and byte and nonbyte displacements
by Jeff Squyres (jsquyres) 23 Oct '19

23 Oct '19

On Oct 23, 2019, at 10:44 AM, Rolf Rabenseifner <rabenseifner(a)hlrs.de> wrote: > >> typedef struct { uint64_t context; uint64_t pointer; } MPI_Aint; >> >> Is that incorrect? > > I expect, that it is not correct. My apologies for that. > > In MPI-1.0 - MPI-2.1 all address calculation had to be done with - and + operators. > Only for the case, that somebody switches integer overflow detection on, > we added MPI_Aint_add and MPI_Aint_diff. I'm confused with how to reconcile that with MPI-3.1 4.1.3 "Correct Use of Addresses", p115:13-15: "Also, in machines with a segmented address space, addresses are not unique and address arithmetic has some peculiar properties. Thus, the use of addresses, that is, displacements relative to the start address MPI_BOTTOM, has to be restricted." Doesn't this explicitly state that segmented address spaces are supported? > We asked ourself, whether we want to deprecate the use of + and - operators, > i.e., to add this to Table 2.1 on page 18 and the forum decided "no" per straw vote. > We definitely never removed the operators + and - from the list of > valid operations for MPI_Aint. I'm not sure how to reconcile your statements with MPI-3.1 2.5.6 "Absolute Addresses and Relative Address Displacements" p16:39-43: "For retrieving absolute addresses or any calculation with absolute addresses, one should use the routines and functions provided in Section 4.1.5. Section 4.1.12 provides additional rules for the correct use of absolute addresses. For expressions with relative displacements or other usage without absolute addresses, intrinsic operators (e.g., +, -, *) can be used." If MPI_Aint is supposed to be used for absolute addresses, this tells me that MPI_Aint_add/diff must be used for all mathematical operations. > MPI-3.1 on page 17 lines 16-18 clearly tells that any int, MPI:Aint, and MPI_Offset value > can be assigned to a MPI_Count variable, i.e., all 4 must be integers of some byte-size. I think it actually says something subtly different than that. It says: "The size of the MPI_Count type is determined by the MPI implementation with the restriction that it must be minimally capable of encoding any value that may be stored in a variable of type int, MPI_Aint, or MPI_Offset in C and of type INTEGER, INTEGER (KIND=MPI_ADDRESS_KIND), or INTEGER (KIND=MPI_OFFSET_KIND) in Fortran." Meaning: an MPI_Count must be big enough to hold an MPI_Aint. It does not say that you can assign one to the other. In my hypothetical "typedef struct" example, this means that MPI_Count would need to be a 128 bit value. ---- All this being said, what all this discussion probably means is that there are discrepancies in the standard that should be fixed and/or clarified. But I think the discussion starts with a fundamental question: Does MPI support segmented address spaces or not? -- Jeff Squyres jsquyres(a)cisco.com

1 0

Re: [Mpiwg-large-counts] Pythonizing + BigCount: virtual meeting on Nov 13?
by Jeff Hammond 23 Oct '19

23 Oct '19

Come on, Squyres. You know we have to call it ExaCounts in order for our publicly funded colleagues to take any interest in this. Jeff Sent from my iPhone > On Oct 23, 2019, at 10:11 AM, Jeff Squyres (jsquyres) via mpiwg-large-counts <mpiwg-large-counts(a)lists.mpi-forum.org> wrote: > > Martin S -- > > The LargeCount / BigCount / WhopperCount / GinormousCount / UseYouFavoriteWordCount WG would like to reserve the 2-hour virtual meeting on Wed Nov 13. This is the Wed before SC, and I'm sure it will be incredibly well attended! ;-) > > We'd like to cover two things: > > 1. The state of LaTeX Pythonizing > (Recall: the Pythonization -- yes, that's totally a word; you must use it in a sentence with a colleague today -- of the MPI LaTeX is a separate effort than the Embiggening -- yes, that's also a word -- of "count" parameters. Specifically: we're doing a comparable-to-MPI-3.1 Pythonization of the MPI bindings in LaTeX without Embiggening or otherwise changing any parameters. Pythonization opens the door to multiple interesting things in the future, including Embiggening "count" parameters). > > 2. The state of BigCount / Embiggening > There was a reading and much discussion at the last Forum meeting about Embiggening. We'd like to get some Forum feedback before the next Forum meeting in December. > > -- > Jeff Squyres > jsquyres(a)cisco.com > > _______________________________________________ > mpiwg-large-counts mailing list > mpiwg-large-counts(a)lists.mpi-forum.org > https://lists.mpi-forum.org/mailman/listinfo/mpiwg-large-counts

1 0

Re: [Mpiwg-large-counts] Large Count - the principles for counts, sizes, and byte and nonbyte displacements
by Rolf Rabenseifner 23 Oct '19

23 Oct '19

Dear Jeff, > typedef struct { uint64_t context; uint64_t pointer; } MPI_Aint; > > Is that incorrect? I expect, that it is not correct. My apologies for that. In MPI-1.0 - MPI-2.1 all address calculation had to be done with - and + operators. Only for the case, that somebody switches integer overflow detection on, we added MPI_Aint_add and MPI_Aint_diff. We asked ourself, whether we want to deprecate the use of + and - operators, i.e., to add this to Table 2.1 on page 18 and the forum decided "no" per straw vote. We definitely never removed the operators + and - from the list of valid operations for MPI_Aint. MPI-3.1 on page 17 lines 16-18 clearly tells that any int, MPI:Aint, and MPI_Offset value can be assigned to a MPI_Count variable, i.e., all 4 must be integers of some byte-size. Best regards Rolf ----- Original Message ----- > From: "Jeff Squyres" <jsquyres(a)cisco.com> > To: "Rolf Rabenseifner" <rabenseifner(a)hlrs.de>, "mpiwg-large-counts" <mpiwg-large-counts(a)lists.mpi-forum.org> > Sent: Wednesday, October 23, 2019 4:19:53 PM > Subject: Re: [Mpiwg-large-counts] Large Count - the principles for counts, sizes, and byte and nonbyte displacements > On Oct 9, 2019, at 1:44 PM, Rolf Rabenseifner via mpiwg-large-counts > <mpiwg-large-counts(a)lists.mpi-forum.org> wrote: > >> I never said that that a number is a displacement. >> MPI_Aint is clearly used for >> - addresses relative to a buffer begin, >> - absolute addresses (returned by MPI_GET_ADDRESS) >> - number of bytes. >> >> MPI-3.1 says >> 2.5.6 Absolute Addresses and Relative Address Displacements >> Some MPI procedures use address arguments that represent an absolute address in >> the calling >> program, or relative displacement arguments that represent differences of two >> absolute >> addresses. The datatype of such arguments is MPI_Aint in C and INTEGER (KIND= >> MPI_ADDRESS_KIND) in Fortran. > > I thought the whole point of MPI_Aint was to support segmented address spaces. > Indeed, the next sentence in MPI-3.1 states: > > "These types must have the same width and encode address values in the same > manner such that address values in one language may be passed directly to > another language without conversion." > > This says to me that MPI_Aint can be encoded, which I take to mean the > following: > > 1. This is exactly the reason why we have MPI_AINT_ADD / MPI_AINT_DIFF. > 2. Presuming I have an MPI_ADDRESS_KIND value in Fortran that represents a 128 > bit integer, the following is valid in C: > > typedef struct { uint64_t context; uint64_t pointer; } MPI_Aint; > > Is that incorrect? > > -- > Jeff Squyres > jsquyres(a)cisco.com -- Dr. Rolf Rabenseifner . . . . . . . . . .. email rabenseifner(a)hlrs.de . High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 . University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 . Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner . Nobelstr. 19, D-70550 Stuttgart, Germany . . . . (Office: Room 1.307) .

1 0

Re: [Mpiwg-large-counts] Large Count - the principles for counts, sizes, and byte and nonbyte displacements
by Jeff Squyres (jsquyres) 23 Oct '19

23 Oct '19

On Oct 9, 2019, at 1:44 PM, Rolf Rabenseifner via mpiwg-large-counts <mpiwg-large-counts(a)lists.mpi-forum.org> wrote: > I never said that that a number is a displacement. > MPI_Aint is clearly used for > - addresses relative to a buffer begin, > - absolute addresses (returned by MPI_GET_ADDRESS) > - number of bytes. > > MPI-3.1 says > 2.5.6 Absolute Addresses and Relative Address Displacements > Some MPI procedures use address arguments that represent an absolute address in the calling > program, or relative displacement arguments that represent differences of two absolute > addresses. The datatype of such arguments is MPI_Aint in C and INTEGER (KIND= > MPI_ADDRESS_KIND) in Fortran. I thought the whole point of MPI_Aint was to support segmented address spaces. Indeed, the next sentence in MPI-3.1 states: "These types must have the same width and encode address values in the same manner such that address values in one language may be passed directly to another language without conversion." This says to me that MPI_Aint can be encoded, which I take to mean the following: 1. This is exactly the reason why we have MPI_AINT_ADD / MPI_AINT_DIFF. 2. Presuming I have an MPI_ADDRESS_KIND value in Fortran that represents a 128 bit integer, the following is valid in C: typedef struct { uint64_t context; uint64_t pointer; } MPI_Aint; Is that incorrect? -- Jeff Squyres jsquyres(a)cisco.com

1 0