Just something that can be structured into a tree on which paths or inodes we declare the already existing MPI-IO files. Not necessarily with all the constrains of POSIX. Basically introducing simple inodes, implemented with supported protocols / transports. Axel On 07.01.19 16:02, William Gropp wrote:
Are we assuming a POSIX filesystem (a bad idea in my opinion)? Are we assuming a more restricted set of file system structure operations, along with weaker consistency semantics (matching the MPI-IO approach rather than POSIX)?
Bill
William Gropp Director and Chief Scientist, NCSA Thomas M. Siebel Chair in Computer Science University of Illinois Urbana-Champaign
On Jan 7, 2019, at 8:28 AM, Axel Huebl via mpi-comments <[email protected] <mailto:[email protected]>> wrote:
Dear MPI-Forum,
should we extend the MPI-I/O interfaces for directory operations? Especially a "mkdir -p" equivalent could be useful.
In many scientific apps that create output at scale, an output structure needs to be created on demand.
One can of course decide to do so with e.g. Linux `mkdir` but this can be tricky:
If all ranks do it, one will race the others and it's unnecessary expensive in terms of operations.
If a "root" rank does it, there is no guarantee that e.g. a MPI_Barrier which syncs the "MPI-World" will also cause the "filesystem-world" to be up-to-date. For example a mkdir on a slow NFS local node has bot yet propagated to the other compute nodes.
Comments, feedback and overlooked already existing solutions welcome!
Best regards, Axel Huebl --
Axel Huebl Phone: +49 351 260 3582 Institute of Radiation Physics http://www.hzdr.de/crp Helmholtz-Zentrum Dresden - Rossendorf (HZDR) Bautzner Landstr. 400 | 01328 Dresden | Germany Board of Directors: Prof. Dr. Dr. h. c. Roland Sauerbrey, Dr. Ulrich Breuer Company Registration Number VR 1693, Amtsgericht Dresden _______________________________________________ mpi-comments mailing list [email protected] https://lists.mpi-forum.org/mailman/listinfo/mpi-comments
-- Axel Huebl Phone: +49 351 260 3582 Institute of Radiation Physics http://www.hzdr.de/crp Helmholtz-Zentrum Dresden - Rossendorf (HZDR) Bautzner Landstr. 400 | 01328 Dresden | Germany Board of Directors: Prof. Dr. Dr. h. c. Roland Sauerbrey, Dr. Ulrich Breuer Company Registration Number VR 1693, Amtsgericht Dresden