---------- Forwarded message ----------
From: Jeff Hammond <jeff.science@gmail.com>
Date: Thu, May 12, 2016 at 10:50 PM
Subject: Re: [mpiwg-rma] Use of MPI_SHORT_INT, etc with MPI_Accumulate
To: MPI WG Remote Memory Access working group <mpiwg-rma@mpi-forum.org>
Cc: MPI WG Remote Memory Access working group <mpiwg-rma@lists.mpi-forum.org>




On Thu, May 12, 2016 at 1:38 PM, Nathan Hjelm <hjelmn@mac.com> wrote:
>
> Ok, so in this case “predefined type” means any type defined by MPI. That is not clear from the text and it is reasonable to assume that non-homogenous (and possibly non-contiguous) datatypes are not allowed.

It is absolutely clear from page 12 of MPI 3.1:

A predefined datatype is a datatype with a predefined (constant) name (such as MPI_INT, MPI_FLOAT_INT, or MPI_PACKED) or a datatype constructed with MPI_TYPE_CREATE_F90_INTEGER, MPI_TYPE_CREATE_F90_REAL, or MPI_TYPE_CREATE_F90_COMPLEX. The former are named whereas the latter are unnamed.

> MPI_MINLOC and MPI_MAXLOC could be be assumed to be only allowed for MPI_2INT, MPI_2REAL, etc for example. 

Except for MPI_2INT, the homogeneous pair types are for Fortran.  If we added this restriction, we might as well remove MAXLOC/MINLOC from RMA altogether.

> Allowing MPI_SHORT_INT, MPI_DOUBLE_INT, etc requires MPI implementations to write special cases to handle these potentially non-contiguous datatypes.

MPI RMA permits the use of user-defined non-contiguous datatypes where all basic components are of the same predefined datatype.  The only difference here is that the atomicity covers the whole pair type.  However, since accumulate atomicity is only assured when the same datatype is used, you can implement e.g. MPI_LONG_DOUBLE_INT using locking and use atomic instructions for other types where it is easier.

> A lot of extra code for what is likely no benefit to end users.

How is it a lot of extra code?  Do you not already have a "stupid" RMA code path for at least some of MPI_C(XX)_LONG_DOUBLE, MPI_C(XX)_DOUBLE_COMPLEX, MPI_C(XX)_LONG_DOUBLE_COMPLEX and/or MPI_PROD?  Remember that the accumulate atomicity rules are very strict, which permits a fast (potentially hardware) implementation for some (op,type) pairs and a slow but functional implementation for others.

>  Does anyone know of a code that makes use of this feature?

I know users that depend upon the symmetry of MPI_Accumulate and MPI_Reduce w.r.t. MAXLOC and MINLOC.  I don't know what state the code is in, but PETSc developers have said they want to use this.

Jeff

> -Nathan
>
>
> > On May 11, 2016, at 8:35 AM, William Gropp <wgropp@illinois.edu> wrote:
> >
> > Read this as (predefined datatype) or (derived datatype where all basic components are of the same predefined datatype).
> >
> > MPI_SHORT_INT is a predefined datatype, and so is valid for accumulate.
> >
> > Bill
> >
> > William Gropp
> > Director, Parallel Computing Institute
> > Thomas M. Siebel Chair in Computer Science
> > Chief Scientist, NCSA
> > University of Illinois Urbana-Champaign
> >
> >
> >
> >
> >
> > On May 11, 2016, at 9:02 AM, Nathan Hjelm <hjelmn@mac.com> wrote:
> >
> >> I have a user who is trying to use MPI_SHORT_INT as the origin and target datatype arguments of MPI_Accumulate. My interpretation of the standard does not allow this type. I am justifying this because of MPI 3-1 § 11.3.4 pp 425:2-8:
> >>
> >> Each datatype argument must be a predefined datatype or a derived datatype, where
> >> all basic components are of the same predefined datatype. Both datatype arguments must
> >> be constructed from the same predefined datatype.
> >>
> >> MPI_SHORT_INT, MPI_LONG_DOUBLE_INT, etc are not in any of the A.1 tables labeled as "predefined datatypes”. They show up in a separate list of datatypes for reduction functions. Since these datatypes are not “predefined datatypes” and are not composite datatypes of the "same predefined datatype” they are not valid for MPI_Accumulate, MPI_Get_accumulate, etc. Am I wrong in my interpretation?
> >>
> >> -Nathan
> >> _______________________________________________
> >> mpiwg-rma mailing list
> >> mpiwg-rma@mpi-forum.org
> >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma
> >
> > _______________________________________________
> > mpiwg-rma mailing list
> > mpiwg-rma@mpi-forum.org
> > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma
>
>
> _______________________________________________
> mpiwg-rma mailing list
> mpiwg-rma@mpi-forum.org
> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma




--
Jeff Hammond
jeff.science@gmail.com
http://jeffhammond.github.io/



--
Jeff Hammond
jeff.science@gmail.com
http://jeffhammond.github.io/