Regarding Ashley Pittman's message:
I'm not sure I understand the difference between (b) and (c).
The difference between (b) and (c) is best explained with reference to MPI_COMM_SELF, in (b) it can have the same id in every process, in (c) it has to have different values everywhere. This has a practical implication, in (c) you can say "Communicator X in this job is...", with (b) to be able to describe the communicator you have to also include some information about the communicator members, for example the global rank of local rank zero.
In my opinion (c) would make the most logical sense and be easiest for tools to work with however I know from my time working with MPI libraries that (b) is a much more sensible option in reality.
Ashley Pittman.
Thank you for that description. I've been able to verify that HP-MPI indeed only supports the (b) and not (c). I agree that from a tools standpoint, (c) would be much more useful, but that from an MPI point of view, it might require some semimajor changes to the way that we create communicators. Currently MPI_Comm_Create() does not block for an entire job, just for the ranks involved, which would make it tricky to ensure uniqueness across an entire job of the communicator id. I don't think this would be impossible, but should be well thought out. Geoff Paulsen