February 2013 - mpiwg-io - mailman3.cels.anl.gov

Re: [MPI3-IO] Topics for discussion for the I/O working group slot
by Rob Ross 26 Feb '13

26 Feb '13

Hi, You might consider a non-collective version (name TBD) that does not allow for changing the data rep (i.e., doesn't have that parameter) and that makes it erroneous to use shared file pointer routines after the call, until after a following collective file set view (this because you can't know if all file views are sync'd or not, and probably they are not). Many hints would be ignored by this non-collective version, but I think that's fine. Rob On Feb 26, 2013, at 8:21 AM, Rob Latham wrote: > On Mon, Feb 25, 2013 at 05:26:28PM -0600, Mohamad Chaarawi wrote: >> Hi Dries, >> >> On 2/25/2013 3:44 PM, Dries Kimpe wrote: >>> - Fix shared file pointer access cfr. the EuroMPI paper? >> >> yes thanks for reminding me :-) >> >>> - independent file view changes >> >> Do you think, implementation wise, that it would be straightforward >> to just remove this restriction? I'm worried how that would effect >> shared file pointer operations in the standard. >> I'm all for having a new routine to change the file view >> independently with restrictions on what comes after that. > > Yes, please. MPI_File_set_view_not_dumb would be a fine name. > > Or one could introduce a crapton of routines that take a file view in > addition to a memory datatype and obviate the need for setting file > view at all (well except for data representation but who actually does > that?) > > ==rob > > -- > Rob Latham > Mathematics and Computer Science Division > Argonne National Lab, IL USA > _______________________________________________ > MPI3-IO mailing list > MPI3-IO(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-io

1 0

Re: [MPI3-IO] Topics for discussion for the I/O working group slot
by Rob Latham 26 Feb '13

26 Feb '13

On Mon, Feb 25, 2013 at 05:26:28PM -0600, Mohamad Chaarawi wrote: > Hi Dries, > > On 2/25/2013 3:44 PM, Dries Kimpe wrote: > >- Fix shared file pointer access cfr. the EuroMPI paper? > > yes thanks for reminding me :-) > > >- independent file view changes > > Do you think, implementation wise, that it would be straightforward > to just remove this restriction? I'm worried how that would effect > shared file pointer operations in the standard. > I'm all for having a new routine to change the file view > independently with restrictions on what comes after that. Yes, please. MPI_File_set_view_not_dumb would be a fine name. Or one could introduce a crapton of routines that take a file view in addition to a memory datatype and obviate the need for setting file view at all (well except for data representation but who actually does that?) ==rob -- Rob Latham Mathematics and Computer Science Division Argonne National Lab, IL USA

1 0

Re: [MPI3-IO] Topics for discussion for the I/O working group slot
by Mohamad Chaarawi 25 Feb '13

25 Feb '13

Hi Quincey/Dries, On 2/25/2013 5:09 PM, Dries Kimpe wrote: > * Quincey Koziol <koziol(a)hdfgroup.org> [2013-02-25 16:08:47]: > >> I was thinking more about this, and what about the "group collective" >> I/O operations that we've kicked around a bit? (Where "group >> collective" is a sub-group of the processes that opened the file) > Or the alternative: a subgroup-open which can quickly generate a handle > for a subgroup given the handle for the larger group. Yes i like that better than copying and modifying the entire I/O operations. I don't imagine the sub-group open is a real file-system open (right?) so it should be lightweight? Thanks, Mohamad > > Dries > > > _______________________________________________ > MPI3-IO mailing list > MPI3-IO(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-io

1 0

Re: [MPI3-IO] Topics for discussion for the I/O working group slot
by Quincey Koziol 25 Feb '13

25 Feb '13

On Feb 25, 2013, at 5:09 PM, Dries Kimpe <dkimpe(a)mcs.anl.gov> wrote: > > * Quincey Koziol <koziol(a)hdfgroup.org> [2013-02-25 16:08:47]: > >> I was thinking more about this, and what about the "group collective" >> I/O operations that we've kicked around a bit? (Where "group >> collective" is a sub-group of the processes that opened the file) > > Or the alternative: a subgroup-open which can quickly generate a handle > for a subgroup given the handle for the larger group. This would be fine, as long as it was fast - I would expect that the subgroup-opened file handle would only exist for a single HDF5 I/O call (read or write). Quincey

1 0

Re: [MPI3-IO] Topics for discussion for the I/O working group slot
by Mohamad Chaarawi 25 Feb '13

25 Feb '13

Hi Dries, On 2/25/2013 3:44 PM, Dries Kimpe wrote: > - Fix shared file pointer access cfr. the EuroMPI paper? yes thanks for reminding me :-) > - independent file view changes Do you think, implementation wise, that it would be straightforward to just remove this restriction? I'm worried how that would effect shared file pointer operations in the standard. I'm all for having a new routine to change the file view independently with restrictions on what comes after that. Thanks, Mohamad

1 0

Re: [MPI3-IO] Topics for discussion for the I/O working group slot
by Dries Kimpe 25 Feb '13

25 Feb '13

* Quincey Koziol <koziol(a)hdfgroup.org> [2013-02-25 16:08:47]: > I was thinking more about this, and what about the "group collective" > I/O operations that we've kicked around a bit? (Where "group > collective" is a sub-group of the processes that opened the file) Or the alternative: a subgroup-open which can quickly generate a handle for a subgroup given the handle for the larger group. Dries

1 0

Re: [MPI3-IO] Topics for discussion for the I/O working group slot
by Quincey Koziol 25 Feb '13

25 Feb '13

I was thinking more about this, and what about the "group collective" I/O operations that we've kicked around a bit? (Where "group collective" is a sub-group of the processes that opened the file) Quincey On Feb 25, 2013, at 3:44 PM, Dries Kimpe <dkimpe(a)mcs.anl.gov> wrote: > - Fix shared file pointer access cfr. the EuroMPI paper? > - independent file view changes > > > Dries > > * Mohamad Chaarawi <chaarawi(a)hdfgroup.org> [2013-02-25 15:41:16]: > >> Hi All, > >> The I/O working group has 1hr and 45 minutes of time allocated at the Forum >> meeting in a couple of weeks. >> HDF doesn't have a lot of things to talk about, other than current status of >> the non-blocking collective I/O ticket and maybe a non-blocking open/close.. >> But I don't imagine those would take more than 1/2 an hr. >> Does anyone have anything they would want to discuss then? > >> Thanks, >> Mohamad >> _______________________________________________ >> MPI3-IO mailing list >> MPI3-IO(a)lists.mpi-forum.org >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-io > _______________________________________________ > MPI3-IO mailing list > MPI3-IO(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-io

1 0

Re: [MPI3-IO] Topics for discussion for the I/O working group slot
by Dries Kimpe 25 Feb '13

25 Feb '13

- Fix shared file pointer access cfr. the EuroMPI paper? - independent file view changes Dries * Mohamad Chaarawi <chaarawi(a)hdfgroup.org> [2013-02-25 15:41:16]: > Hi All, > The I/O working group has 1hr and 45 minutes of time allocated at the Forum > meeting in a couple of weeks. > HDF doesn't have a lot of things to talk about, other than current status of > the non-blocking collective I/O ticket and maybe a non-blocking open/close.. > But I don't imagine those would take more than 1/2 an hr. > Does anyone have anything they would want to discuss then? > Thanks, > Mohamad > _______________________________________________ > MPI3-IO mailing list > MPI3-IO(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-io

1 0

[MPI3-IO] Topics for discussion for the I/O working group slot
by Mohamad Chaarawi 25 Feb '13

25 Feb '13

Hi All, The I/O working group has 1hr and 45 minutes of time allocated at the Forum meeting in a couple of weeks. HDF doesn't have a lot of things to talk about, other than current status of the non-blocking collective I/O ticket and maybe a non-blocking open/close.. But I don't imagine those would take more than 1/2 an hr. Does anyone have anything they would want to discuss then? Thanks, Mohamad

1 0