I was reading web pages about segmented address space. And they show two possibilities (which may go only into the past): - flattened address space - segmented with some high order bits refering to the Segment and low order bits to address within the segment. If I understood correctly, then the major problem behind MPI_Aint_diff and MPI_Aint_add was the following: (my example is with 8 bit MPI_Aint) addr1 01111111 = 127 (signed int) = 127 (unsigned int) addr2 10000001 = -127 (signed int) = 129 (unsigned int) Internally the addreses are viewed by the hardware and OS as unsigned. MPI_Aint is interpreting the same bits as signed int. addr2-addr1 = 129 -127 = 2 (as unsigned int) but in a real application code with "-" operator: = -127 -127 = -254 --> signed int Overflow because 8 bit can express only -128 .. +127 --> detected or automatically corrected with +256 --> -254+256 = 2 The segmented problem was solved very early through the wording on "sequential storage" (unchanged since MPI-1.1 !!! :-) in the section "Correct use of addresses". What I see, that before changing anything, the mpiwg-large-counts WG should really understand MPI-3.1. For me, it looked like that there was some misunderstanding of the concept that absolute and relative addresses and number of bytes that can be stored in MPI_Aint. Best regards Rolf ----- Original Message -----

From: "Jeff Squyres" <[email protected]> To: "Rolf Rabenseifner" <[email protected]> Cc: "mpiwg-large-counts" <[email protected]> Sent: Thursday, October 24, 2019 3:28:47 PM Subject: Re: [Mpiwg-large-counts] Large Count - the principles for counts, sizes, and byte and nonbyte displacements

On Oct 24, 2019, at 2:19 AM, Rolf Rabenseifner <[email protected]> wrote:

...

...

Does MPI support segmented address spaces or not?

When I understand correctly then MPI is and was always prepared for segmented address spaces as long as all integer types are signed integers and the segmentation is done through some higher order bits.

As I was pondering your replies from yesterday and thinking about how to respond, this is exactly the same point that I got to.

I think what I have been failing to elucidate is that this is not how modern segmented address spaces work. It's not sufficient to have a few bits to specify which segment to use. Instead, you typically need three things:

1. The type of memory segment being described (which may be just a few bits). In reality, this "type" may correlate to which API you need to use to access this memory (e.g., main RAM, RDMA, accelerator memory, persistent memory, other IO memory, ...etc.). 2. A context of some kind (usually more than a small number of bits -- perhaps 32 or 64 bits). 3. A pointer (usually 32 or 64 bits).

For example, if I'm accessing IO memory of some flavor, I probably need some kind of context/handle that points to a descriptor with information about my data segment on the IO peripheral in question (e.g., a NIC), and then a pointer for the specific place that I want in that specific memory. I use this context/handle + pointer with some underlying API to get/set that data.

I think that's the disconnect here: the prior thinking about MPI_Aint may have assumed "it's either main RAM or it's XYZ memory, and the rest of the bits are enough to uniquely describe the address." But that doesn't seem sufficient any more to describe the evolution of complex, segmented memory systems (and be able to use their supporting APIs).

Meaning: perhaps a pure integer MPI_Aint may have been sufficient in the past. But I'm not sure that's useful any more.

-- Jeff Squyres [email protected]

-- Dr. Rolf Rabenseifner . . . . . . . . . .. email [email protected] . High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 . University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 . Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner . Nobelstr. 19, D-70550 Stuttgart, Germany . . . . (Office: Room 1.307) .