Proposal for MPI Initialization time assertions in MPI 2.2 Define two new MPI functions: MPI_INIT_ASSERTED(assertions, threads, provided) [ IN assertions ] assertions about application behavior [ IN threads] desired level of thread support (integer) [ OUT provided] provided level of thread support (integer) int MPI_Init_asserted(int *argc, char *((*argv)[]), int assertions, int threads, int *provided) MPI_INIT_ASSERTED(ASSERTIONS, THREADS, PROVIDED, IERROR) INTEGER ASSERTIONS, THREADS, PROVIDED, IERROR (I will need help for C++ bindings ) The assertions arguments is to be provided as the logical OR of any subset of the assertions defined by the MPI standard. Zero is valid for an application that does not wish to make any assertion. The assertions are constants defined in mpi.h. MPI_QUERY_ASSERTIONS(stated, exploited) [ OUT stated] assertions as give in the MPI_INIT_ASSERTED call [ OUT exploited ] assertions the MPI implementation is exploiting int MPI_Query_assertions(int *stated, int *exploited) MPI_QUERY_ASSERTIONS(STATED, EXPLOITED, IERROR) INTEGER STATED, EXPLOITED, IERROR The outputs, STATED and EXPLOITED, can be checked for specific assertions with logical masks. For example to see if the application has used MPI_NO_SEND_CANCEL, library code could use: MPI_Query_assertions(&stated, &exploited); if (stated & MPI_NO_SEND_CANCEL) .... To confirm that no assertions are being exploited the library could use if (exploited) ..... error ==================================================================== Rationale There are certain semantic guarantees or features required by the MPI standard that can add significant memory overheads or performance costs. If an application does not depend on one of these features and can declare that fact at MPI initialization, the MPI implementation may be able to provide better performance. Assertions on MPI_INIT_ASSERTED let an application declare it does not require a specific feature and allow the implementation to omit support for that feature if there is a savings from doing so. When an application calls MPI_INIT_ASSERTED with a particular assertion the MPI implementation may depend on the application to act as the assertion promises. The behavior of an MPI application that uses or depends on some feature of MPI after asserting it does not is undefined. The application cannot depend on the assertion being exploited. The MPI application must be valid for an MPI implementation that ignores the assertion. Advise to implementors - When it is practical, a high quality MPI implementation will raise an error if an application violates an assertion that was provided at initialization It will not be practical for the MPI implementation to detect all assertion violations. End advise to implementors - The following assertions are available (defined in mpi.h) MPI_NO_EAGER_THROTTLE The MPI standard allows an implementation to send some messages eagerly, meaning the message can complete at the sender before a matching receive is posted. The implementation is required to hold the message data until the matching receive gets posted. If there is no space to buffer the data the send must block. The MPI job must not fail because it has run out of space or lose the data. Providing this guarantee can demand an expensive flow control. In many well structured applications there will never be a need to buffer significant numbers of eager messages but eager sends do help performance. These applications can tell the MPI implementation the guarantee is optional. MPI_NO_REDUCTION_DETERMINISM The MPI standard either requires or strongly urges that an MPI_REDUCE/MPI_ALLREDUCE give exactly the same answer every time. There are applications that do not need a "same answer" promise as long as they can assume they will get a "correct" answer. Maybe they can be provided a faster reduction algorithm. MPI_NO_DATATYPE_HETEROGENEITY MPI_Datatypes that are made from multiple types can be optimized if it is known there will never be a need to translate the data between heterogeneous nodes. A structure made of an MPI_INT followed by an MPI_FLOAT is likely to be 8 contiguous bytes with an extent of 8. An MPI implementation that "knows" it will not need to deal with data conversion can simplify the datatype commit logic and resulting internal representation by discarding the MPI_INT/MPI_FLOAT distinctions and just recording that the type is 8 bytes with a stride of 8. ============================== If we add MPI_INIT_ASSERTED to the MPI standard, MPI_INIT and MPI_INIT_THREAD continue to work as before. MPI_INIT_THREAD is equivalent to MPI_INIT_ASSERTED with zero as the assertion. We can consider deprecating the older initialization calls. Above are the assertions I have decided make most sense for the first pass. There are others that may be justified but a rationale is needed. If MPI 2.2 defines support for the assertion above we can consider others as part of MPI 3.0, Below are some That may be justified but will need a better rationale. MPI_NO_THREAD_CONTENTION (I am not sure this different from asking for MPI_THREAD_SERIALIZED but here is what it would mean) If the MPI library may be called by contending threads it must do something to protect internal state from conflicting updates. Whether this involves mutex locks or some other scheme, it is likely to have a performance cost. This assertion promises that the application will not make conflicting MPI calls. The MPI implementation can bypass any logic that defends state against multiple concurrent calls to MPI. The application may be multi-threaded in other aspects. MPI_NO_SEND_CANCEL To support MPI_CANCEL of a send, the MPI implementation may need to add extra information to every message header. If the application can assert that it does not try to cancel any sends then perhaps less state must be maintained and smaller message headers can be used. MPI_NO_ANY_SOURCE Dick Treumann - MPI Team/TCEM IBM Systems & Technology Group Dept 0lva / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601 Tele (845) 433-7846 Fax (845) 433-8363 *