See below ----- Original Message -----
From: "Jeff Squyres" <[email protected]> To: "Rolf Rabenseifner" <[email protected]> Cc: "mpiwg-large-counts" <[email protected]> Sent: Wednesday, October 23, 2019 5:12:33 PM Subject: Re: [Mpiwg-large-counts] Large Count - the principles for counts, sizes, and byte and nonbyte displacements
On Oct 23, 2019, at 10:44 AM, Rolf Rabenseifner <[email protected]> wrote:
typedef struct { uint64_t context; uint64_t pointer; } MPI_Aint;
Is that incorrect?
I expect, that it is not correct. My apologies for that.
In MPI-1.0 - MPI-2.1 all address calculation had to be done with - and + operators. Only for the case, that somebody switches integer overflow detection on, we added MPI_Aint_add and MPI_Aint_diff.
I'm confused with how to reconcile that with MPI-3.1 4.1.12 "Correct Use of Addresses", p115:13-15:
"Also, in machines with a segmented address space, addresses are not unique and address arithmetic has some peculiar properties. Thus, the use of addresses, that is, displacements relative to the start address MPI_BOTTOM, has to be restricted."
Doesn't this explicitly state that segmented address spaces are supported?
Yes, some higher bits may be used as a flag. If two absolute addresses are within the same array or structure (i.e. variable), i.e., within the save sequential storage, then a minus operator must still work.
We asked ourself, whether we want to deprecate the use of + and - operators, i.e., to add this to Table 2.1 on page 18 and the forum decided "no" per straw vote. We definitely never removed the operators + and - from the list of valid operations for MPI_Aint.
I'm not sure how to reconcile your statements with MPI-3.1 2.5.6 "Absolute Addresses and Relative Address Displacements" p16:39-43:
"For retrieving absolute addresses or any calculation with absolute addresses, one should use the routines and functions provided in Section 4.1.5. Section 4.1.12 provides additional rules for the correct use of absolute addresses. For expressions with relative displacements or other usage without absolute addresses, intrinsic operators (e.g., +, -, *) can be used."
If MPI_Aint is supposed to be used for absolute addresses, this tells me that MPI_Aint_add/diff must be used for all mathematical operations.
The following definitions apply in understanding how to implement an ISO International Standard and other normative ISO deliverables (TS, PAS, IWA). - "shall" indicates a requirement. - "should" indicates a recommendation.
... any calculation with absolute addresses, one ***should*** use the routines and functions provided in Section 4.1.5
... indicates a recommendation, not a must
MPI-3.1 on page 17 lines 16-18 clearly tells that any int, MPI:Aint, and MPI_Offset value can be assigned to a MPI_Count variable, i.e., all 4 must be integers of some byte-size.
I think it actually says something subtly different than that. It says:
"The size of the MPI_Count type is determined by the MPI implementation with the restriction that it must be minimally capable of encoding any value that may be stored in a variable of type int, MPI_Aint, or MPI_Offset in C and of type INTEGER, INTEGER (KIND=MPI_ADDRESS_KIND), or INTEGER (KIND=MPI_OFFSET_KIND) in Fortran."
Meaning: an MPI_Count must be big enough to hold an MPI_Aint. It does not say that you can assign one to the other. In my hypothetical "typedef struct" example, this means that MPI_Count would need to be a 128 bit value.
----
All this being said, what all this discussion probably means is that there are discrepancies in the standard that should be fixed and/or clarified. But I think the discussion starts with a fundamental question:
Does MPI support segmented address spaces or not?
-- Jeff Squyres [email protected]
-- Dr. Rolf Rabenseifner . . . . . . . . . .. email [email protected] . High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 . University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 . Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner . Nobelstr. 19, D-70550 Stuttgart, Germany . . . . (Office: Room 1.307) .