Ignore the last email there were bugs, and the bcast should be in a while loop. -d do { if (root) read_some_data_from_a_file(buffer); do { int val; err = MPI_Bcast(buffer, .... ,root, comm); if (err) { val = FALSE; MPI_Comm_agreement(comm, &val); MPI_Comm_invalidate(comm); MPI_Comm_shrink(comm, &newcomm); MPI_Comm_free(&comm); comm = newcomm; } else { val = TRUE; MPI_Comm_agreement(comm, &val); if (!val) { MPI_Comm_shrink(comm, &newcomm); MPI_Comm_free(&comm); comm = newcomm; } } while (!val); MPI_Comm_size(comm, size); done = do_computation(buffer, size); /* Let's agree for sure if we are done with the computation */ MPI_Comm_agreement(comm, &done); while (!done); MPI_Finalize(); On Apr 6, 2012, at 11:28 AM, Darius Buntinas wrote:
Wouldn't you need to do something like this?
-d
do {
if (root) read_some_data_from_a_file(buffer);
err = MPI_Bcast(buffer, .... ,root, comm); if (err) { int val = FALSE; MPI_Comm_invalidate(comm); MPI_Comm_shrink(comm, &newcomm); MPI_Comm_free(&comm); comm = newcomm; MPI_Comm_agreement(comm, &val); } else { int val = TRUE; MPI_Comm_agreement(comm, &val); if (!val) { MPI_Comm_shrink(comm, &newcomm); MPI_Comm_free(&comm); comm = newcomm; } MPI_Comm_size(comm, size);
done = do_computation(buffer, size);
/* Let's agree for sure if we are done with the computation */
MPI_Comm_agreement(comm, &done);
while (!done);
MPI_Finalize();
On Apr 5, 2012, at 4:50 PM, David Solt wrote:
I have another question about MPI_Comm_agreement:
If I want to do this:
.... do {
if (root) read_some_data_from_a_file(buffer);
err = MPI_Bcast(buffer, .... ,root, comm); if (err) { MPI_Comm_invalidate(comm); MPI_Comm_shrink(comm, &newcomm); MPI_Comm_free(&comm); comm = newcomm; }
MPI_Comm_size(comm, size);
done = do_computation(buffer, size);
/* Let's agree for sure if we are done with the computation */
MPI_Comm_agreement(comm, &done);
while (!done);
MPI_Finalize();
This code can deadlock because some ranks may enter MPI_Comm_agreement while others detect an error in MPI_Bcast and call MPI_Comm_invalidate followed by MPI_Comm_shrink (assume that do_computation is really, really fast). The call to MPI_Comm_invalidate will not allow the processes that have already entered MPI_Comm_agreement to leave that call (P543L45: "Advice to users. MPI_COMM_AGREEMENT maintains its collective behavior even if the comm is invalidated. (End of advice to users.)" ) and MPI_Comm_agreement cannot return an error due to the call to MPI_Comm_invalidate (P545L38: "This function must not return an error due to process failure (error classes MPI_ERR_PROC_FAILED and MPI_ERR_INVALIDATED)...") .
This would not work: .... do {
if (root) read_some_data_from_a_file(buffer);
err = MPI_Bcast(buffer, .... ,root, comm); if (err) { MPI_Comm_invalidate(comm); MPI_Comm_shrink(comm, &newcomm); MPI_Comm_free(&comm); comm = newcomm; }
MPI_Comm_size(comm, size);
done = do_computation(buffer, size);
/* Let's agree for sure if we are done with the computation */
MPI_Barrier(comm); // don't check the error code, this is just to "catch" invalidate messages MPI_Comm_agreement(comm, &done);
while (!done);
MPI_Finalize();
because a rank may enter the barrier, get knocked out by the call to invalidate and then go on to call MPI_Comm_agreement anyway. So we can try the following:
do {
if (root) read_some_data_from_a_file(buffer);
err = MPI_Bcast(buffer, .... ,root, comm); if (err) { MPI_Comm_invalidate(comm); MPI_Comm_shrink(comm, &newcomm); MPI_Comm_free(&comm); comm = newcomm; }
MPI_Comm_size(comm, size);
done = do_computation(buffer, size);
/* Let's agree for sure if we are done with the computation */
err = MPI_Barrier(comm); if (err) { MPI_Comm_invalidate(comm); MPI_Comm_shrink(comm, &newcomm); MPI_Comm_free(&comm); comm = newcomm; } MPI_Comm_agreement(comm, &done);
while (!done);
MPI_Finalize();
But now we have done nothing more than move the problem down a few lines. Some ranks may succeed the MPI_Barrier and go on to MPI_Comm_agreement while others attempt to invalidate/shrink. Is there a solution to this problem? How can one safely use MPI_Comm_agreement and MPI_Comm_shrink in the same application?
Thanks, Dave _______________________________________________ mpi3-ft mailing list [email protected] http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-ft