Just to follow up, I think that the “right” thing to do with respect to some sort of
 transactional model is to have some sort of standard way to request such
 communications take place – probably at init time.  We have had such an MPI
 implementation running in production for several years on a multi-thousand
 process cluster, and the only thing that needs to be exposed to the users is the
 ability to turn on/off  the functionality – all the rest is taken care of just fine within
 the context of the MPI 2.0 standard, and is 100% standard compliant.
Cool! Can you post the API for this thing? Might be a useful thing to start from. Also, its a prototype implementation.

This does not deal with hints on the network “state”.
Yeah, I'm not sure if such hints are valuable enough to be included in the spec. It really falls under the quality-of-service ideas what we've tossed around, so we should write up a proposal for those and see how much QoS people really want.

Greg Bronevetsky
Post-Doctoral Researcher
1028 Building 451
Lawrence Livermore National Lab
(925) 424-5756
bronevetsky1@llnl.gov