mpiwg-large-counts

Download

[email protected]

September 2019

1 participants
1 discussions

Large Count - the principles for counts, sizes, and byte and nonbyte displacements
by Rolf Rabenseifner 23 Sep '19

23 Sep '19

Dear all, To understand how big and large counts should be implemented in MPI-4, it is important to understand the count, displacement and size model of in MPI-3.1. As long as we do not have a common understanding of MPI-3.1, we will have problems to define MPI-4. Therefore here my understanding of MPI-3. We have - number of array elements with elements of a given type (typically represented by an MPI datatype handle). Usual argument name: count Usual description: number of elements in ... buffer (non-negative integer) C-type in MPI-3.1: int C-type in _l: MPI_Count - an index into such an array, i.e., the number of an element. Argument name / descriptions [Routine]: -- array_of_displacements / displacement ..., in multiples of oldtype extent (... integer) [MPI_TYPE_INDEXED] -- sdispls / integer array (of length group size). Entry j specifies the displacement ... [MPI_ALLTOALLV] C-type in MPI-3.1: int C-type in _l: MPI_Count - number of bytes Argument name / descriptions [Routine]: -- size / size of window in bytes (non-negative integer) [MPI_WIN_CREATE] -- outsize / output buffer size, in bytes (integer) [MPI_PACK_EXTERNAL] C-type in MPI-3.1: MPI_Aint C-type in _l: MPI_Aint (Wrong) C-types in MPI-3.1: int [MPI_PACK, MPI_TYPE_SIZE] C-type (corrected) in _l: MPI_Aint [MPI_PACK, MPI_TYPE_SIZE] - smaller number in bytes Some argument names: e.g., disp_unit Description: local unit size for displacements, in bytes (positive integer) C-type in MPI-3.1: int C-type in _l: still int? or MPI_Count? - Position or relative byte displacement within an array of bytes. Such values can be calculated as any sum and product of in, long, long long, and MPI_Aint as long as MPI_Aint value contains a pure integer size value, i.e., an (integer) difference of two absolute addresses within one sequential storage, see MPI-3.1 page 115 line 31, or a MPI datatype extent, retrieved, e.g., with MPI_TYPE_GET_EXTENT. Argument names / description: -- position / current position in buffer, in bytes (integer) [MPI_PACK_EXTERNAL] -- array_of_displacements / byte displacement of each block (array of integer) [MPI_TYPE_CREATE_STRUCT] C-type in MPI-3.1: MPI_Aint C-type in _l: MPI_Aint (Wrong) C-types in MPI-3.1: int [MPI_PACK, MPI_ALLTOALLW] C-type (corrected) in _l: MPI_Aint [MPI_PACK, MPI_ALLTOALLW] - Absolute address values for byte displacements. These values are also valid for all byte displacements in datatype routines and in MPI_NEIGHBOR_ALLTOALLW, provided that they are used in combination with buffer=MPI_BOTTOM. They cannot be used in MPI_ALLTOALLW. With "C-type (corrected) in _l: MPI_Aint [MPI_ALLTOALLW]", they are also usable with MPI_ALLTOALLW. I already looked at the Large/Big Count pdf and saw that in the datatype chapter these rules were broken, for example for the ...PACK/UNPACK... routines. Therefore my clear question, do we agree on the rules above? Already detected bugs in Version from Sep. 13, 2019: - page 127: No idea why you changed the name from MPI_GET_ELEMENTS to MPI_TYPE_GET_ELEMENTS. Should be reverted. - MPI_PACK: outsize and position should be handled identical to those in MPI_PACK_EXTERNAL, i.e., both are MPI_Aint... - MPI_PACK_SIZE: size should be handled identical to that in MPI_PACK_EXTERNAL_SIZE, i.e., MPI_Aint... - MPI_UNPACK: insize and position should be handled identical to those in MPI_UNPACK_EXTERNAL, i.e., both are MPI_Aint... - MPI_Type_contiguous: the large count _l version is missing - MPI_Type_create_darray -- array_of_distribs must be INTEGER, because it holds enumeration values and nothing else. -- array_of_dargs requires significant explanation because it can hold an enumeration (INTEGER) and also large count values which can cause not understandable compiler error reports in Fortran, because using MPI_COUNT_KIND array_of_gsizes values together with an INTEGER enumeration constant, here MPI_DISTRIBUTE_DFLT_DARG, would cause a compiler message like "no matching interface found". Two possible text/interface solutions: - If using the mpi_f08 module and MPI_DISTRIBUTE_DFLT_DARG together with large Count Version of this procedure, i.e., INTEGER(KIND=MPI_COUNT_KIND) array_of_gsizes and array_of_dargs arguments, then one should use INT(MPI_DISTRIBUTE_DFLT_DARG, MPI_COUNT_KIND) instead of MPI_DISTRIBUTE_DFLT_DARG. - overloading with two versions (long,normal) and (long,long) but i would recommend the first solution because it does not require additional MPI library implementation overhead. Before I can continue with reviewing, the principles above must be cleared/discussed/agreed/... Best regards Rolf -- Dr. Rolf Rabenseifner . . . . . . . . . .. email rabenseifner(a)hlrs.de . High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 . University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 . Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner . Nobelstr. 19, D-70550 Stuttgart, Germany . . . . (Office: Room 1.307) .

1 0