On Wed, Feb 01, 2012 at 03:16:30PM -0600, Dries Kimpe wrote:

* Torsten Hoefler <[email protected]> [2012-02-01 15:04:41]:

...

...

Torsten? (CC'd) There are obviously no filehandles in NBC. The scan operations match in issue-order and the buffers specified have to be disjunct (as for any nonblocking communication in MPI). I.e., the two NBCs do not influence each other from a correctness perspective.

Meaning that the ordering issue for non-blocking collectives (but not mixed case) is straigtforward, right?

yes

Implementation of MPI_File_Iread_ordered: MPI_Iscan (amount of local data) <make progress> When MPI_Iscan completes: update shared file pointer <atomic: one rank can do this> MPI_File_iread_at Not really --- one would have to "collect" all other (noncollective) operations that update the shared FP and make sure they update it *after* the scan completed.

As long as it is possible to adjust and get the shared file pointer (which is what I'll be proposing in the ticket I've talked about to you guys) it is possible to do this and not have 'undefined' behaviour.

Also note that currently, ROMIO already has this functionality. (As this is how MPI_File_read_shared is implemented).

Because of the MPI_Iscan properties, a second MPI_File_iread_ordered could not proceed before the first one finished the scan phase, hence they would be properly ordered. Well, they *can* complete out of order (not match though!) you need to make sure the order it preserved.

Now, for mixing in indepentent shared FP operations, while not undefined, the ordering would not be determined and will depend on what happens on other nodes and other things.

As in:

(1) MPI_File_iread_ordered ... (2) MPI_File_read_shared

(3) MPI_Wait ()

If (1) can complete the scan before the call to (2), (1) will appear to logically execute before (2). In all other cases, (1) will logically execute before (2).

Dries

-- bash$ :(){ :|:&};: --------------------- http://www.unixer.de/ ----- Torsten Hoefler | Performance Modeling and Simulation Lead Blue Waters Directorate | University of Illinois (UIUC) 1205 W Clark Street | Urbana, IL, 61801 NCSA Building | +01 (217) 244-7736