I think the result of your weaker semantics is PVM or some other type of formalised chaos :)
Handling group-members failing to take part could be useful for the process-fail-stop type of fault tolerance but how does a partial group reach consensus that the other members are never going to join in, rather than just assuming they are being
a bit slow? There are ways - but let us first define what keep-alives/timeouts/RAS looks like in MPI and then look at the implications of that on the whole interface. The nearest MPI semantic is soft spawn, I think. We could look at what might be permitted
if the user supplied an info key “soft = true” to any communicator/window/file creation routine. This is, perhaps, a topic for the FT WG.
How many non-members are permitted to join in ad-hoc with no warning? When is the communicator creation finished, if ever? Can new members apply to join even after some (sub)group has reached a consensus and begun using the communicator for communication? The
nearest MPI semantic is connect/accept, I think, but that permits exactly one connection between (the root processes of) two arbitrarily sized pre-existing groups where all members of each group have already reached consensus with all other members of their
group. By induction, that relies on pairwise connections - fixed size of 2 (i.e. exactly one non-member), FCFS, block indefinitely if unmatched.
There is an event/exception mechanism in the PMIx functionality that we are currently using for the prototype implementation - if PMIx discovers actual process-failure then it can callback into all other processes to tell them about it and they can all react
in one of several appropriate ways (fail the operation, form a smaller group, invite spare processes to act as replacements, request more resources to act as replacements). Some of those reactions might lead to resilience rather than fault tolerance but it
is all hidden behind (or forbidden by) the stronger semantics of MPI, which assumes reliability and a priori coordination (information supplied at X is always consistent with information supplied at Y). Again, this is probably a topic for the FT WG.
A question that has always bothered me about intercomms - why only two groups? Why not permit a topology of leaders, each of which coordinates a local group, possibly with its own local topology? Does anyone have a compelling use-case for such generality?
Summary: these are interesting research questions but not Sessions WG topics.
Cheers,
Dan.
—
Dr Daniel Holmes PhD
Applications Consultant in HPC Research
d.holmes@epcc.ed.ac.uk
Phone: +44 (0) 131 651 3465
Mobile: +44 (0) 7940 524 088
Address: Room 2.09, Bayes Centre, 47 Potterrow, Central Area, Edinburgh, EH8 9BT
—
The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336.
—