Now, to one of your questions. An MPI_ANY_SOURCE
Although I appreciate the discussion, my intent (uh-oh!) in bring this up to let you know I "accept" the problem, yet ask for the capability anyway, but in a manner that keeps it from presenting problems everywhere. Or maybe I'm under-estimating what I was once told: the use of MPI_ANY_SOURCE anywhere means it is a problem everywhere, ie in _every_ function involved in transmitting data? If that is the case, but I still _really_ wanted to use -lmpi_subset, I could do this: suppose a pe knows it will receive data from m pes. It could post numpe non-blocking receives, complete m, discover who they're from, then cancel the rest. Now I'm thinking I've created a bigger problem: when running acros numpes=100k cores, but m is say 10. True? Barring some sort of workaround, excluding codes that "need" MPI_ANY_SOURCE seems to meaningfully reduce the number of codes that could use -lmpi_subset.
3. I (basically) understand the adverse performance effects of allowing promiscuous receives (MPI_ANY_SOURCE). However, this is a powerful capability for many codes, and used only in moderation, eg for setting up communication requirements (such as communication partners in unstructured, semi-structured, and dynamic mesh computations). In this case the sender knows its partner, but the receiver does not. A reduction(sum) is used to let each process know the number of communication partners from which it will receive data, the process posts that many promiscuous receives, which when satisfied lets it from then on specify the sender. So would it be possible to include this capability in a separate function, say the blocking send/recv, but not allow it in the non-blocking version?
Richard -- Richard Barrett Future Technologies Group, Computer Science and Mathematics Division, and Scientific Computing Group, National Center for Computational Science Oak Ridge National Laboratory http://ft.ornl.gov/~rbarrett