Hi all,
I've got a problem which I think might be a candidate for a fix in
the 2.2 standard: Is it legal to make MPI calls in the 'delete'
callback for an attribute attached to MPI_COMM_SELF which is being
freed during a call to MPI_Finalize?
Section 4.8 in the 2.0 standard doesn't really say, I think. Here's
the text:
> 4.8 Allowing User Functions at Process Termination
>
> There are times in which it would be convenient to have actions
> happen when an MPI process
> finishes. For example, a routine may do initializations that are
> useful until the MPI job (or
> that part of the job that being terminated in the case of
> dynamically created processes) is
> finished. This can be accomplished in MPI-2 by attaching an
> attribute to MPI_COMM_SELF
> with a callback function. When MPI_FINALIZE is called, it will first
> execute the equivalent
> of an MPI_COMM_FREE on MPI_COMM_SELF. This will cause the delete
> callback function
> to be executed on all keys associated with MPI_COMM_SELF, in an
> arbitrary order. If no
> key has been attached to MPI_COMM_SELF, then no callback is invoked.
> The “freeing” of
> MPI_COMM_SELF occurs before any other parts of MPI are affected.
> Thus, for example,
> calling MPI_FINALIZED will return false in any of these callback
> functions. Once done with
> MPI_COMM_SELF, the order and rest of the actions taken by
> MPI_FINALIZE is not specified.
>
> Advice to implementors. Since attributes can be added from any
> supported language,
> the MPI implementation needs to remember the creating language so
> the correct
> callback is made. (End of advice to implementors.)
Here's my use case: we cache modified file information in the HDF5
library and need to be able to flush that data to the file if the user
calls MPI_Finalize without calling the HDF5 routines to close (or
flush) that file first. It looks like adding an attribute onto
MPI_COMM_SELF is the right thing to do, and the standard seems to
imply that it's possible to make MPI calls from that callback, but
it's not certain to me. (Obviously, the HDF5 library needs to make I/
O calls and possible may need to exchange data or invoke a barrier, etc)
Is it reasonable to add a sentence to the effect that "all MPI
operations (except MPI_Finalize and MPI_Init) are possible from the
attribute's 'delete' callback"?
Here's a copy of section 4.8, with text to that effect worked in:
> 4.8 Allowing User Functions at Process Termination
>
> There are times in which it would be convenient to have actions
> happen when an MPI process
> finishes. For example, a routine may do initializations that are
> useful until the MPI job (or
> that part of the job that being terminated in the case of
> dynamically created processes) is
> finished. This can be accomplished in MPI-2 by attaching an
> attribute to MPI_COMM_SELF
> with a callback function. When MPI_FINALIZE is called, it will first
> execute the equivalent
> of an MPI_COMM_FREE on MPI_COMM_SELF. This will cause the delete
> callback function
> to be executed on all keys associated with MPI_COMM_SELF, in an
> arbitrary order. If no
> key has been attached to MPI_COMM_SELF, then no callback is invoked.
> The “freeing” of
> MPI_COMM_SELF occurs before any other parts of MPI are affected and
> all MPI operations (except MPI_Finalize and MPI_Init) are available
> to the attribute's 'delete' callback. Thus, for example,
> calling MPI_FINALIZED will return false in any of these callback
> functions. Once done with
> MPI_COMM_SELF, the order and rest of the actions taken by
> MPI_FINALIZE is not specified.
>
> Advice to implementors. Since attributes can be added from any
> supported language,
> the MPI implementation needs to remember the creating language so
> the correct
> callback is made. (End of advice to implementors.)
Thanks,
Quincey Koziol
The HDF Group
