On Feb 28, 2011, at 1:50 PM, Joshua Hursey wrote:

Reworked block below -----------------------

Looks good

...

...

We have the following states (prefix with MPI_RANK_STATE_): - OK (active) - FAILED (failed, unrecognized) - NULL (failed, recognized)

We could add a few new modifiers (prefix with MPI_RANK_STATE_MOD_): - NEW (since last call to {global|local} validate) - OLD (before last call to {global|local} validate) - RECOGNIZED (-- maybe to replace the NULL state above?

I like this idea.

The idea of or'ing states, or the idea of having a 'Recognized' modifier, or both?

Both. (But I prefer NULL to RECOGNIZED.)

...

...

To determine "L" or "G" they would use the following functions: ---------------------------- MPI_Comm_validate_local(comm, &num_failed) - Local operation - Update L - num_failed = |L| (both recognized and unrecognized)

MPI_Comm_validate_global(comm, &num_failed) - Collective operation - Update G - Update L = G - num_failed = |L| = |G| ----------------------------

Accessors have the following properties: - These are local operations - None of them modify "L" or "G" - Take an or'ed list of states and modifiers to determine 'type' - If incount = 0, then outcount = |L| or |G|, rank_infos ignored

---------------------------- MPI_Comm_get_state_local(comm, type, incount, &outcount, rank_infos[]) - Local operation - Returns the set of processes in "L" that match the 'type' specified - outcount = min(incount, |L|) - MPI_ERR_SIZE if incount != 0 and incount < |L|

MPI_Comm_get_state_global(comm, type, incount, &outcount, rank_infos[]) - Local operation - Returns the set of processes in "G" that match the 'type' specified - outcount = min(incount, |G|) - MPI_ERR_SIZE if incount != 0 and incount < |G| ----------------------------

So an application can do something like: ------------ MPI_Comm_validate_global(comm, &num_failed_start); /* Do work */ MPI_Comm_validate_global(comm, &num_failed_end);

if( num_failed_start < num_failed_end ) { /* something failed */ incount = 0; MPI_Comm_get_state_global(comm, MPI_RANK_STATE_NULL|MPI_RANK_STATE_MOD_NEW, incount, &outcount, NULL); rank_infos = malloc(... * outcount); incount = outcount; MPI_Comm_get_state_global(comm, MPI_RANK_STATE_NULL|MPI_RANK_STATE_MOD_NEW, incount, &outcount, rank_infos); } ------------

Instead of having the 'if incount = 0' rule, we could just introduce a new function like: ---------------------------- MPI_Comm_get_num_state_local(comm, type, &count); MPI_Comm_get_num_state_global(comm, type, &count);

In that case we can even replace num_failed in the comm_validate functions with a flag: new_failures. Then use the above to get the counts.

Or even better, eliminate the second argument from the MPI_Comm_validate_{local|global}, and just pass the communicator to it - similar to MPI_Barrier. Since the accessor functions are always related to the last update call there is no real need (other than shorthand) to have the additional parameter.

I considered that, but that would require two calls to determine if anything failed. Replacing the count with a are_there_new_failures flag would solve that.

My removing the count parameter from the MPI_Comm_validate_{local|global} we get out of the business of deciding which count to return, and let the user specify it explicitly.

The example would now expand out a bit to be: ------------ MPI_Comm_validate_global(comm); MPI_Comm_get_num_state_global(comm, STATE_NULL|MOD_NEW, &num_failed_start); /* Do work */ MPI_Comm_validate_global(comm); MPI_Comm_get_num_state_global(comm, STATE_NULL|MOD_NEW, &num_failed_end);

if( num_failed_start < num_failed_end ) { /* something failed */ incount = num_failed_end; rank_infos = malloc(... * incount); MPI_Comm_get_state_global(comm, MPI_RANK_STATE_NULL|MPI_RANK_STATE_MOD_NEW, incount, &outcount, rank_infos); } ------------

Replacing count with a flag would look like this. So in the common, non-error case you just do a branch. It's not so much a performance thing (validate_global is collective), but a convenience thing to the programmer. MPI_Comm_validate_global(comm, &new_failures); /* Do work */ MPI_Comm_validate_global(comm, &new_failures); if( new_failures ) { /* something failed */ MPI_Comm_get_num_state_global(comm, STATE_NULL|MOD_NEW, &num_failed_end); incount = num_failed_end; rank_infos = malloc(... * incount); MPI_Comm_get_state_global(comm, MPI_RANK_STATE_NULL|MPI_RANK_STATE_MOD_NEW, incount, &outcount, rank_infos); } Hmm. We could combine validate and get_num_state: MPI_Comm_validate_global(comm, count_type, &count) This would let the user decide what count to return. -d