Large Count - the principles for counts, sizes, and byte and nonbyte displacements
Dear all, To understand how big and large counts should be implemented in MPI-4, it is important to understand the count, displacement and size model of in MPI-3.1. As long as we do not have a common understanding of MPI-3.1, we will have problems to define MPI-4. Therefore here my understanding of MPI-3. We have - number of array elements with elements of a given type (typically represented by an MPI datatype handle). Usual argument name: count Usual description: number of elements in ... buffer (non-negative integer) C-type in MPI-3.1: int C-type in _l: MPI_Count - an index into such an array, i.e., the number of an element. Argument name / descriptions [Routine]: -- array_of_displacements / displacement ..., in multiples of oldtype extent (... integer) [MPI_TYPE_INDEXED] -- sdispls / integer array (of length group size). Entry j specifies the displacement ... [MPI_ALLTOALLV] C-type in MPI-3.1: int C-type in _l: MPI_Count - number of bytes Argument name / descriptions [Routine]: -- size / size of window in bytes (non-negative integer) [MPI_WIN_CREATE] -- outsize / output buffer size, in bytes (integer) [MPI_PACK_EXTERNAL] C-type in MPI-3.1: MPI_Aint C-type in _l: MPI_Aint (Wrong) C-types in MPI-3.1: int [MPI_PACK, MPI_TYPE_SIZE] C-type (corrected) in _l: MPI_Aint [MPI_PACK, MPI_TYPE_SIZE] - smaller number in bytes Some argument names: e.g., disp_unit Description: local unit size for displacements, in bytes (positive integer) C-type in MPI-3.1: int C-type in _l: still int? or MPI_Count? - Position or relative byte displacement within an array of bytes. Such values can be calculated as any sum and product of in, long, long long, and MPI_Aint as long as MPI_Aint value contains a pure integer size value, i.e., an (integer) difference of two absolute addresses within one sequential storage, see MPI-3.1 page 115 line 31, or a MPI datatype extent, retrieved, e.g., with MPI_TYPE_GET_EXTENT. Argument names / description: -- position / current position in buffer, in bytes (integer) [MPI_PACK_EXTERNAL] -- array_of_displacements / byte displacement of each block (array of integer) [MPI_TYPE_CREATE_STRUCT] C-type in MPI-3.1: MPI_Aint C-type in _l: MPI_Aint (Wrong) C-types in MPI-3.1: int [MPI_PACK, MPI_ALLTOALLW] C-type (corrected) in _l: MPI_Aint [MPI_PACK, MPI_ALLTOALLW] - Absolute address values for byte displacements. These values are also valid for all byte displacements in datatype routines and in MPI_NEIGHBOR_ALLTOALLW, provided that they are used in combination with buffer=MPI_BOTTOM. They cannot be used in MPI_ALLTOALLW. With "C-type (corrected) in _l: MPI_Aint [MPI_ALLTOALLW]", they are also usable with MPI_ALLTOALLW. I already looked at the Large/Big Count pdf and saw that in the datatype chapter these rules were broken, for example for the ...PACK/UNPACK... routines. Therefore my clear question, do we agree on the rules above? Already detected bugs in Version from Sep. 13, 2019: - page 127: No idea why you changed the name from MPI_GET_ELEMENTS to MPI_TYPE_GET_ELEMENTS. Should be reverted. - MPI_PACK: outsize and position should be handled identical to those in MPI_PACK_EXTERNAL, i.e., both are MPI_Aint... - MPI_PACK_SIZE: size should be handled identical to that in MPI_PACK_EXTERNAL_SIZE, i.e., MPI_Aint... - MPI_UNPACK: insize and position should be handled identical to those in MPI_UNPACK_EXTERNAL, i.e., both are MPI_Aint... - MPI_Type_contiguous: the large count _l version is missing - MPI_Type_create_darray -- array_of_distribs must be INTEGER, because it holds enumeration values and nothing else. -- array_of_dargs requires significant explanation because it can hold an enumeration (INTEGER) and also large count values which can cause not understandable compiler error reports in Fortran, because using MPI_COUNT_KIND array_of_gsizes values together with an INTEGER enumeration constant, here MPI_DISTRIBUTE_DFLT_DARG, would cause a compiler message like "no matching interface found". Two possible text/interface solutions: - If using the mpi_f08 module and MPI_DISTRIBUTE_DFLT_DARG together with large Count Version of this procedure, i.e., INTEGER(KIND=MPI_COUNT_KIND) array_of_gsizes and array_of_dargs arguments, then one should use INT(MPI_DISTRIBUTE_DFLT_DARG, MPI_COUNT_KIND) instead of MPI_DISTRIBUTE_DFLT_DARG. - overloading with two versions (long,normal) and (long,long) but i would recommend the first solution because it does not require additional MPI library implementation overhead. Before I can continue with reviewing, the principles above must be cleared/discussed/agreed/... Best regards Rolf -- Dr. Rolf Rabenseifner . . . . . . . . . .. email [email protected] . High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 . University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 . Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner . Nobelstr. 19, D-70550 Stuttgart, Germany . . . . (Office: Room 1.307) .
participants (1)
-
Rolf Rabenseifner