March 2013 - mpiwg-io - mailman3.cels.anl.gov

Re: [MPI3-IO] I/O working group Notes
by Rajeev Thakur 13 Mar '13

13 Mar '13

Fab is interested in generalized requests for a different use case. Rajeev On Mar 13, 2013, at 3:24 PM, Rob Latham wrote: > On Wed, Mar 13, 2013 at 03:09:22PM -0500, William Gropp wrote: >> Relevant to this is that enhancing generalized requests also came up. Part of the discussion is this: would an enhancement to generalized requests, which would have broader benefit, be a more general solution? >> >> Background: The Forum is increasingly concerned about the creation of routines to solve specific problems rather than stepping back and trying to find a more general and powerful solution. > > With the disclaimer that I am not a Forum participant... > > "rule of three" is a nice guideline for the specific/general approach. > Find three use cases and now you've properly motivated taking the time > to design a general solution. > > We have "rule of one" right now. The only grequest client is I/O. > The only extended grequest client is I/O. > > Torsten's nonblocking collectives could have been case number two. He > found standard grequests unworkable and apparently found it easer to > advocate for nonblocking collectives via the forum than to adopt and > advocate for extended grequests. > > I'd say delete or deprecate grequests if one is concerned with API > bloat to make room for non-blocking I/O routines! In 15 years there > has been but one paper examining grequests (ours). There has been but > one grequest client -- ROMIO (ours) -- and that only because ROMIO was > implemented as a stunt: implement the MPI-IO chapter on top of > (instead of integrated into) an MPI-1 implementation. > > Said differently: improve grequests and MPI will still likely have > zero grequest consumers. improve I/O and MPI will have at the very > least HDF5 as a customer. > > ==rob > > -- > Rob Latham > Mathematics and Computer Science Division > Argonne National Lab, IL USA > _______________________________________________ > MPI3-IO mailing list > MPI3-IO(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-io

1 0

Re: [MPI3-IO] I/O working group Notes
by Rob Latham 13 Mar '13

13 Mar '13

On Wed, Mar 13, 2013 at 03:09:22PM -0500, William Gropp wrote: > Relevant to this is that enhancing generalized requests also came up. Part of the discussion is this: would an enhancement to generalized requests, which would have broader benefit, be a more general solution? > > Background: The Forum is increasingly concerned about the creation of routines to solve specific problems rather than stepping back and trying to find a more general and powerful solution. With the disclaimer that I am not a Forum participant... "rule of three" is a nice guideline for the specific/general approach. Find three use cases and now you've properly motivated taking the time to design a general solution. We have "rule of one" right now. The only grequest client is I/O. The only extended grequest client is I/O. Torsten's nonblocking collectives could have been case number two. He found standard grequests unworkable and apparently found it easer to advocate for nonblocking collectives via the forum than to adopt and advocate for extended grequests. I'd say delete or deprecate grequests if one is concerned with API bloat to make room for non-blocking I/O routines! In 15 years there has been but one paper examining grequests (ours). There has been but one grequest client -- ROMIO (ours) -- and that only because ROMIO was implemented as a stunt: implement the MPI-IO chapter on top of (instead of integrated into) an MPI-1 implementation. Said differently: improve grequests and MPI will still likely have zero grequest consumers. improve I/O and MPI will have at the very least HDF5 as a customer. ==rob -- Rob Latham Mathematics and Computer Science Division Argonne National Lab, IL USA

1 0

Re: [MPI3-IO] I/O working group Notes
by Quincey Koziol 13 Mar '13

13 Mar '13

On Mar 13, 2013, at 3:09 PM, William Gropp <wgropp(a)illinois.edu> wrote: > Relevant to this is that enhancing generalized requests also came up. Part of the discussion is this: would an enhancement to generalized requests, which would have broader benefit, be a more general solution? Yes, that would be a fine direction to go. > Background: The Forum is increasingly concerned about the creation of routines to solve specific problems rather than stepping back and trying to find a more general and powerful solution. Wholeheartedly agree! :-) Quincey > > Bill > > William Gropp > Director, Parallel Computing Institute > Deputy Director for Research > Institute for Advanced Computing Applications and Technologies > Thomas M. Siebel Chair in Computer Science > University of Illinois Urbana-Champaign > > > > > On Mar 13, 2013, at 2:56 PM, Rob Latham wrote: > >> On Wed, Mar 13, 2013 at 02:46:34PM -0500, Quincey Koziol wrote: >>>> Can something be done with generalized request? >>> Someone always brings this up. :-) We should get up to speed on the GRQ routines and know what they can do... >> >> They cannot do much. I am, reluctantly, the worlds expert on >> generalized requests, having written a paper about it 5(!!!) years >> ago. >> >> http://dx.doi.org/10.1007/978-3-540-75416-9_33 >> >> pdf: http://www.mcs.anl.gov/~robl/papers/latham_grequest-enhance.pdf >> >> Standard Generalized requests have a "all progress must happen outside >> of MPI" progress model. Perhaps in this modern surfeit of cores era, >> it is no longer a big deal to spawn a thread ? >> >> The paper I linked proposes generalized request extensions designed to >> play better with posix, pvfs, and NTFS async i/o routines. These >> routines can make progress in the background, but do need someone to >> call a completion function (aio_suspend, PVFS_sys_wait, or >> WaitForMultipleObjectEx). >> >> ==rob >> >> -- >> Rob Latham >> Mathematics and Computer Science Division >> Argonne National Lab, IL USA >> _______________________________________________ >> MPI3-IO mailing list >> MPI3-IO(a)lists.mpi-forum.org >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-io > > _______________________________________________ > MPI3-IO mailing list > MPI3-IO(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-io

1 0

Re: [MPI3-IO] I/O working group Notes
by William Gropp 13 Mar '13

13 Mar '13

Relevant to this is that enhancing generalized requests also came up. Part of the discussion is this: would an enhancement to generalized requests, which would have broader benefit, be a more general solution? Background: The Forum is increasingly concerned about the creation of routines to solve specific problems rather than stepping back and trying to find a more general and powerful solution. Bill William Gropp Director, Parallel Computing Institute Deputy Director for Research Institute for Advanced Computing Applications and Technologies Thomas M. Siebel Chair in Computer Science University of Illinois Urbana-Champaign On Mar 13, 2013, at 2:56 PM, Rob Latham wrote: > On Wed, Mar 13, 2013 at 02:46:34PM -0500, Quincey Koziol wrote: >>> Can something be done with generalized request? >> Someone always brings this up. :-) We should get up to speed on the GRQ routines and know what they can do... > > They cannot do much. I am, reluctantly, the worlds expert on > generalized requests, having written a paper about it 5(!!!) years > ago. > > http://dx.doi.org/10.1007/978-3-540-75416-9_33 > > pdf: http://www.mcs.anl.gov/~robl/papers/latham_grequest-enhance.pdf > > Standard Generalized requests have a "all progress must happen outside > of MPI" progress model. Perhaps in this modern surfeit of cores era, > it is no longer a big deal to spawn a thread ? > > The paper I linked proposes generalized request extensions designed to > play better with posix, pvfs, and NTFS async i/o routines. These > routines can make progress in the background, but do need someone to > call a completion function (aio_suspend, PVFS_sys_wait, or > WaitForMultipleObjectEx). > > ==rob > > -- > Rob Latham > Mathematics and Computer Science Division > Argonne National Lab, IL USA > _______________________________________________ > MPI3-IO mailing list > MPI3-IO(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-io

1 0

Re: [MPI3-IO] I/O working group Notes
by Rob Latham 13 Mar '13

13 Mar '13

On Wed, Mar 13, 2013 at 02:46:34PM -0500, Quincey Koziol wrote: > > Can something be done with generalized request? > Someone always brings this up. :-) We should get up to speed on the GRQ routines and know what they can do... They cannot do much. I am, reluctantly, the worlds expert on generalized requests, having written a paper about it 5(!!!) years ago. http://dx.doi.org/10.1007/978-3-540-75416-9_33 pdf: http://www.mcs.anl.gov/~robl/papers/latham_grequest-enhance.pdf Standard Generalized requests have a "all progress must happen outside of MPI" progress model. Perhaps in this modern surfeit of cores era, it is no longer a big deal to spawn a thread ? The paper I linked proposes generalized request extensions designed to play better with posix, pvfs, and NTFS async i/o routines. These routines can make progress in the background, but do need someone to call a completion function (aio_suspend, PVFS_sys_wait, or WaitForMultipleObjectEx). ==rob -- Rob Latham Mathematics and Computer Science Division Argonne National Lab, IL USA

1 0

Re: [MPI3-IO] I/O working group Notes
by Quincey Koziol 13 Mar '13

13 Mar '13

Hi Mohamad, On Mar 12, 2013, at 8:28 PM, Mohamad Chaarawi <chaarawi(a)hdfgroup.org> wrote: > Hi all, > > I attached the slides that I presented at the Forum, and here are some notes I took when discussing the proposals: Good notes, thanks! > Collective I/O ticket: > Need a more compelling reason to add them to the standard. Maybe a more in-depth study and demand from users with use cases. > Implementation of split collectives hard as is, so to add a harder interface for implementers needs to be justified. > Will not attempt to deprecate the split collective in this ticket. OK, we can provide use cases, drawing from our experiences with HDF5. > Asynchronous MPI Model: > discuss whether it makes sense to introduce such a model in MPI > Can something be done with generalized request? Someone always brings this up. :-) We should get up to speed on the GRQ routines and know what they can do... > a more thorough study should be conducted on what internal MPI features get exposed > Should consider an implementation path for MPI that does not have threads. > No overhead should be introduced, which might not be the case with RMA What did people think of the idea in general? > Independent File View changes: > The file_set_view operation should be used to do optimizations (collectively), so adding an independent version would urge developers to not "do the right thing" with the original routine. > what happens in some cases when all processes set the file view collectively, then 1 process changes its file view independently? How would the other processes know that? The MPI library might need to communicate that change to other processes if the original set file_view did certain optimizations. > For our use case, we can introduce a flag on File open, that says set_view is always independent or collective. We can introduce also another collective routine to allow the user to change set_view from independent to collective and vice versa. I like some of the latter ideas (file open option, etc). They would help take the backward compatibility risk out of the equation. > Grouped I/O operations: > A file_reopen with a sub group is a feasible solution, but more though needs to be taken into consideration.. > What are the consistency semantics for the new file handle that is returned with open_grouped. this needs to be examine more. > What happens if you opened the file twice, but with a different communicator? Can MPI optimize that? > This also affects the file view that is set for the original file handle. How would that play out now? allow grouped file view changes on the new handle, or just allow independent file changes? > do some benchmarking with existing MPI implementations to see performance of Opening file multiple times. OK, sounds positive, we should work out answers to these questions... > Shared FP bug: > emphasize the offset returned should be for where access started. > go over all routines for shared file pointers and indicate why only those three were chosen. We can add a position returned to the ordered operation, it just seemed easy for the user to be able to retrieve it. > generally seemed ok, but needed to emphasize use cases more in proposal. I would _really_like_ the "write_and_get" routine - that would make space allocation in HDF5 _so_ much easier! Getting a use case for this one won't be a problem. :-) Sounds like a good meeting, with some things we can move forward. :-) Quincey

1 0

[MPI3-IO] I/O working group Notes
by Mohamad Chaarawi 12 Mar '13

12 Mar '13

Hi all, I attached the slides that I presented at the Forum, and here are some notes I took when discussing the proposals: Collective I/O ticket: * Need a more compelling reason to add them to the standard. Maybe a more in-depth study and demand from users with use cases. * Implementation of split collectives hard as is, so to add a harder interface for implementers needs to be justified. * Will not attempt to deprecate the split collective in this ticket. Asynchronous MPI Model: * discuss whether it makes sense to introduce such a model in MPI * Can something be done with generalized request. * a more thorough study should be conducted on what internal MPI features get exposed * Should consider an implementation path for MPI that does not have threads. * No overhead should be introduced, which might not be the case with RMA Independent File View changes: * The file_set_view operation should be used to do optimizations (collectively), so adding an independent version would urge developers to not "do the right thing" with the original routine. * what happens in some cases when all processes set the file view collectively, then 1 process changes its file view independently? How would the other processes know that? The MPI library might need to communicate that change to other processes if the original set file_view did certain optimizations. * For our use case, we can introduce a flag on File open, that says set_view is always independent or collective. We can introduce also another collective routine to allow the user to change set_view from independent to collective and vice versa. Grouped I/O operations: * A file_reopen with a sub group is a feasible solution, but more though needs to be taken into consideration.. * What are the consistency semantics for the new file handle that is returned with open_grouped. this needs to be examine more. * What happens if you opened the file twice, but with a different communicator? Can MPI optimize that? * This also affects the file view that is set for the original file handle. How would that play out now? allow grouped file view changes on the new handle, or just allow independent file changes? * do some benchmarking with existing MPI implementations to see performance of Opening file multiple times. Shared FP bug: * emphasize the offset returned should be for where access started. * go over all routines for shared file pointers and indicate why only those three were chosen. We can add a position returned to the ordered operation, it just seemed easy for the user to be able to retrieve it. * generally seemed ok, but needed to emphasize use cases more in proposal. Thanks, Mohamad

1 0

Re: [MPI3-IO] Dial in participants for I/O WG meeting
by Dries Kimpe 07 Mar '13

07 Mar '13

* Mohamad Chaarawi <chaarawi(a)hdfgroup.org> [2013-03-07 14:01:59]: > I don't know. CCing to the MPI I/O mailing list. > Anyone interested in attending by Webex the working group session on Tuesday > morning? Not me. On vacation. Dries

1 0

Re: [MPI3-IO] Dial in participants for I/O WG meeting
by Mohamad Chaarawi 07 Mar '13

07 Mar '13

I don't know. CCing to the MPI I/O mailing list. Anyone interested in attending by Webex the working group session on Tuesday morning? Thanks, Mohamad On 3/7/2013 1:43 PM, Quincey Koziol wrote: > Hi Martin, > Hmm, Mohamad's going to be at the Forum meeting this time, so I'll CC him to check. > > Quincey > > > On Mar 7, 2013, at 11:44 AM, "Schulz, Martin" <schulzm(a)llnl.gov> wrote: > >> Hi Quincey, >> >> Are you planning to have any call-in participants for the I/O WG meeting? If so, please let Jeff and me know - we are trying to determine how many Webex sessions we need. >> >> Thanks, >> >> Martin >> >> ________________________________________________________________________ >> Martin Schulz, schulzm(a)llnl.gov, http://people.llnl.gov/schulzm >> CASC @ Lawrence Livermore National Laboratory, Livermore, USA >> >> >>

1 0