Hi Martin, This is an interesting argument for disallowing use of MPI_THREAD_SINGLE and MPI_THREAD_FUNNELED with the sessions model (and with the world model in combination with the sessions model). Modern MPI libraries effectively only have MPI_THREAD_SERIALIZED and MPI_THREAD_MULTIPLE anyway. However, a single thread could create a bunch of sessions and initialise the world model and comply with MPI_THREAD_SINGLE and/or MPI_THREAD_FUNNELED. Two libraries might be called by a single thread and thereby be thread compliant. So, prohibiting these thread support levels seems too severe. Library writers need to document the restrictions they place on the main application and all the other libraries that it uses. If a library tells MPI it will only use a particular thread then the caller into that library must comply, etc. An MPI library is free to upgrade later (if it is capable of that) without telling already-initialised sessions (because superset) - except for MPI_THREAD_SINGLE (there can be only one). It is also free to return a lower provided than the requested. The only problem occurs if two distinct threads ask for MPI_THREAD_FUNNELED concurrently and MPI provides MPI_THREAD_FUNNELED to one of them but is not able to upgrade internally - it is then stuck for when the second thread makes its request. One of the main points of the isolation between threads was that no cross-session protection was needed. Thus, session S1 funnelled onto thread T1 and S2 funnelled onto T2 should be easy to implement, even though that scenario is not currently possible with MPI-3.1 and no-one has actually coded it yet. Cheers, Dan. — Dr Daniel Holmes PhD Executive Director Chief Technology Officer CHI Ltd [email protected]
Hi Howard,
I recall we decided to leave the behavior of thread support level when using multiple sessions/world model up to the implementation. However, in the case of MPI_THREAD_FUNNELED we should probably add a blurb about not doing this in the advice to users in the Session Creation and Destruction Methods.
I think the problem also appears if the original MPI_Init_thread uses FUNNELED. We probably need to add something there as well.
My first guess based on the openmpi prototype is that if one session is initialized with funneled, then all subsequent created sessions or mpi init thread will be supported at funnel level.
That kind of makes sense - but what happens if the second Session_Init is then on another thread? The funneled refers to a different thread.
For your thread serialized question I would say the answer is no, unless under the covers the implementation is always working in effectively thread multiple mode.
Hmm, I understand the motivation and implementation implications, but this would kill the isolation then, which is also not good. Also, if two libraries have separate sessions, it is often not even possible to avoid this.
Almost seems like we should deprecate funneled and serialized.
Another item to cover next Monday __.
Martin