On Nov 13, 2013, at 4:25 PM, George Bosilca <[email protected]> wrote:

Disconnect doesn’t work at the physical connection level, it is supposed to mark two previously connected world as disconnected in the MPI sense.

Mmm... not quite. It disconnects two *communicators* -- but the *processes* may still be connected (in the MPI sense). See the example in my previous mail -- you have to call COMM_DISCONNECT twice and FILE_CLOSE to actually disconnect. This is supported by MPI-3 10.5.4 (p399:45-47).

In MPI there is this transitive notion of connected processes, which can only be reversed by the usage of disconnect. Basically MPI_Finalize is collective over all connected processes, which is a burden in some cases. Disconnect is here to allow the disruption of the connected notion.

Yeah, I know what disconnect is for. :-)

Thus, Disconnect only has a special meaning when called on a communicator returned by SPAWN, ACCEPT/CONNECT and JOIN. Any other usage is similar to MPI_Comm_Free. This in well explained in 10.5.4 once at the beginning of the section and then again in the first advice to users.

I'm not debating what COMM_DISCONNECT actually does -- I'm just saying that it's actually a poor name choice: per 10.5.4, it *doesn't* always disconnect you. Per the example in my previous mail, the processes involved are connected via 3 things: 1. the communicator created by COMM_SPAWN 2. the communicator created by INTERCOMM_MERGE 3. the communicator created by FILE_OPEN You have to COMM_DISCONNECT the first 2, and FILE_CLOSE then 3rd, and *then* you're actually disconnected. Hence, all I'm saying is that "COMM_DISCONNECT" is actually a bit of a misnomer -- the action it's really performing is something more like MPI_COMM_FREE_WAIT (i.e., do a COMM_FREE, but actually wait for all communication to complete on that communicator before returning). For reference, this was my example:

...

----- MPI_Comm_spawn(..., &child); MPI_Intercomm_merge(child, 0, &child_intra); MPI_File_open(child_intra, ..., &file); MPI_Comm_disconnect(&child_intra); MPI_Comm_disconnect(&child); -----

After giving that example, I mused:

...

I wonder if this is worth an AtoU...

-- Jeff Squyres [email protected] For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/