On Oct 20 2015, William Gropp wrote:

The standard doesn't explicitly address this.

However, in the spirit of the standard, I would say yes, multiple threads may be used. The behavior of side effects of the function are not defined, nor is it required to apply to elements 0..n-1 in that order in an n-element vector (this is a vector on each process; the is/is-not commutative applies between processes). Rather than try to enumerate and disallow certain actions, I think this is the right solution - the action of the user function on the input parameters is defined; nothing else should be. An implementation might warn users that the function may be called by multiple threads concurrently (e.g., to use atomic memory operations to count invocations or increment "evil").

While I agree in principle, that is disastrous in practice. None of C, Fortran nor POSIX are fit for such an implementation. I can see no sign that C11 is ever going to become established (even C99 took over a decade). Even if it did, its memory model is seriously and incomprehensibly ambiguous because (unlike C++11), it has left much of the old, ambiguous sequence point description in place. And (again unlike C11), it doesn't specify anything about any of the library macros and functions. Fortran doesn't say anything about threading. Its concept of purity is not enough, because there are two separate such concepts and it is the other that is really needed. I.e. a function call does not just not affect anything outside, but nothing outside affects it. It gets a long way there, but provides far too many loopholes. POSIX threading is a well-known disaster area, not least because it contains no description of how its rules relate to C's, because its list of thread-safe functions is quite simply broken, and because (God help us all) it says NOTHING about how to synchronise signalling and state control. And, yes, I have seen that produce logical impossibilities in the field, despite using some of its more heavyweight thread synchronisation. At the very least, it should be possible to ensure that any threading is transparent using MPI_Init_thread. Regards, Nick Maclaren.