On 05/06/2014 12:19 PM, Jeff Hammond wrote:

Issue #2: cannot use built-in reduce ops.

Once we rule out using built-in ops with our large-count datatypes, we must reimplement all of the reduction operations required. I find this to be nontrivial. I have not yet figured out how to get at the underlying datatype info in a simple manner. It appears that MPI_Type_get_envelope exists for this purpose, but it's a huge pain to have to call this function when all I need to know is the number of built-in datatypes so that I can apply my clever and use MPI_Reduce_local inside of my user-defined operation.

To determine the number of built-in datatypes, yes, one must recursively call MPI_Type_get_envelope and MPI_Type_get_contents. See the ROMIO datatype flattening code for just how much of a pain MPI_Type_get_contents and MPI_Type_get_envelope is. You might find Rob Ross's libmpitypes approach a bit better, but that code is designed for applying functions to the data of a datatype, deliberately bypassing the get_envelope/get_contents information. ==rob -- Rob Latham Mathematics and Computer Science Division Argonne National Lab, IL USA