October 2013 - devel - mailman3.cels.anl.gov

large datatype/count work has landed
by Rob Latham 31 Oct '13

31 Oct '13

I just pushed my 18-patch 4-month old patchset to master. This change touched just about every part of MPICH, so you're likely to have conflicts with whatever work-in-progress branch you've got cooking. I'll be happy to help you work out merge conflicts if it's not obvious what needs to go where. ==rob -- Rob Latham Mathematics and Computer Science Division Argonne National Lab, IL USA

1 0

Re: [mpich-devel] when is MPICH going to be count-safe?
by Rob Latham 25 Oct '13

25 Oct '13

On Fri, Oct 18, 2013 at 03:25:34PM -0400, Jim Dinan wrote: > It would be good to commit the test case too, if you haven't > already added it. While there was already a test making a similar datatype, the type wasn't actually getting used. I just pushed large_type_sendrec and marked it xfail. Thanks for the testcase, Jeff. ==rob -- Rob Latham Mathematics and Computer Science Division Argonne National Lab, IL USA

1 0

Public Key for Git Access
by Ashwin Aji 24 Oct '13

24 Oct '13

Please add the below key for GIT access. ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA1uiZ/2aUmxIHyqhgyBSUE6c1vRh1kd3D17iG/XOOY3KgRc6l dD1BlHgZQymWn9U8+spBFeoA4UfDUMLNVwhnMRq2urLteLEaa3fT7RhKbZikqRH/TuHhpZgqt159 Ef+VfivF1i/cTwMiQnHuAyh+rnBXWyKyNZDkrcIr5ETDJkQwPzYwbLdIRLaGsQum+IwvbhqX71UQ eP/CZ4sWlPfE+9l+ZDoIBVu60Lmv5E78G6MoGMxtUDGgWxt+di6gmK+iAohZtPmOuzpj67jp/xmH 5kr/CKC4Aa+INE3HsSAb4jW5PieazR3Pl8FNUBQdIps1TErqPj/QNlf4RGFmbUIbEQ== aaji(a)fire.cs.vt.edu Thanks, Ashwin

1 0

Re: [mpich-devel] MPI_Pack/MPI_Unpack/MPI_Pack_size and large datatypes
by Pavan Balaji 19 Oct '13

19 Oct '13

On Oct 18, 2013, at 8:15 PM, Rob Latham wrote: > On Fri, Oct 18, 2013 at 04:44:32PM -0500, Pavan Balaji wrote: >> Hi Rob, >> >> I totally agree with adding this into MPICH. >> >> However, with respect to the MPI standard, can one not implement MPIX_Pack_x on top of current MPI-3 using appropriate datatype creation? > > It'd have to be "next to" or "inside", but not "on top of". The usual > trick of "just describe big type with a smaller number of CONTIGs > won't work: > > MPI_Pack(const void *inbuf, int incount, MPI_Datatype datatype, > void * outbuf, int outcount int *position); > > We're ok on inputs: a normal buf, count, datatype tuple. > > How do you work around the output, though? outcount is a count of > bytes, not a count of datatypes. Position is a plain old pointer to > memory. I meant, one can write a higher-level function that takes Aint, which internally uses a combination of MPI_Pack and MPI_Get_elements_x. No? -- Pavan -- Pavan Balaji http://www.mcs.anl.gov/~balaji

1 0

Re: [mpich-devel] MPI_Pack/MPI_Unpack/MPI_Pack_size and large datatypes
by Rob Latham 18 Oct '13

18 Oct '13

On Fri, Oct 18, 2013 at 08:17:56PM -0500, Jeff Hammond wrote: > Let's just deprecate Pack functions instead of fixing them. That's what we did for C++ :-) > > But seriously, does anyone use Pack functions? They are the most straightforward way to deal with datatypes. parallel-netcdf uses pack/unpack . ==rob > Jeff > > ----- Original Message ----- > > From: "Rob Latham" <robl(a)mcs.anl.gov> > > To: devel(a)mpich.org > > Sent: Friday, October 18, 2013 8:15:42 PM > > Subject: Re: [mpich-devel] MPI_Pack/MPI_Unpack/MPI_Pack_size and large datatypes > > > > On Fri, Oct 18, 2013 at 04:44:32PM -0500, Pavan Balaji wrote: > > > Hi Rob, > > > > > > I totally agree with adding this into MPICH. > > > > > > However, with respect to the MPI standard, can one not implement > > > MPIX_Pack_x on top of current MPI-3 using appropriate datatype > > > creation? > > > > It'd have to be "next to" or "inside", but not "on top of". The > > usual > > trick of "just describe big type with a smaller number of CONTIGs > > won't work: > > > > MPI_Pack(const void *inbuf, int incount, MPI_Datatype datatype, > > void * outbuf, int outcount int *position); > > > > We're ok on inputs: a normal buf, count, datatype tuple. > > > > How do you work around the output, though? outcount is a count of > > bytes, not a count of datatypes. Position is a plain old pointer to > > memory. > > > > ==rob > > > > -- > > Rob Latham > > Mathematics and Computer Science Division > > Argonne National Lab, IL USA > > > -- Rob Latham Mathematics and Computer Science Division Argonne National Lab, IL USA

1 0

Re: [mpich-devel] MPI_Pack/MPI_Unpack/MPI_Pack_size and large datatypes
by Jeff Hammond 18 Oct '13

18 Oct '13

Let's just deprecate Pack functions instead of fixing them. That's what we did for C++ :-) But seriously, does anyone use Pack functions? Jeff ----- Original Message ----- > From: "Rob Latham" <robl(a)mcs.anl.gov> > To: devel(a)mpich.org > Sent: Friday, October 18, 2013 8:15:42 PM > Subject: Re: [mpich-devel] MPI_Pack/MPI_Unpack/MPI_Pack_size and large datatypes > > On Fri, Oct 18, 2013 at 04:44:32PM -0500, Pavan Balaji wrote: > > Hi Rob, > > > > I totally agree with adding this into MPICH. > > > > However, with respect to the MPI standard, can one not implement > > MPIX_Pack_x on top of current MPI-3 using appropriate datatype > > creation? > > It'd have to be "next to" or "inside", but not "on top of". The > usual > trick of "just describe big type with a smaller number of CONTIGs > won't work: > > MPI_Pack(const void *inbuf, int incount, MPI_Datatype datatype, > void * outbuf, int outcount int *position); > > We're ok on inputs: a normal buf, count, datatype tuple. > > How do you work around the output, though? outcount is a count of > bytes, not a count of datatypes. Position is a plain old pointer to > memory. > > ==rob > > -- > Rob Latham > Mathematics and Computer Science Division > Argonne National Lab, IL USA > -- Jeff Hammond Argonne Leadership Computing Facility University of Chicago Computation Institute jhammond(a)alcf.anl.gov / (630) 252-5381 http://www.linkedin.com/in/jeffhammond https://wiki.alcf.anl.gov/parts/index.php/User:Jhammond ALCF docs: http://www.alcf.anl.gov/user-guides

1 0

Re: [mpich-devel] MPI_Pack/MPI_Unpack/MPI_Pack_size and large datatypes
by Rob Latham 18 Oct '13

18 Oct '13

On Fri, Oct 18, 2013 at 04:44:32PM -0500, Pavan Balaji wrote: > Hi Rob, > > I totally agree with adding this into MPICH. > > However, with respect to the MPI standard, can one not implement MPIX_Pack_x on top of current MPI-3 using appropriate datatype creation? It'd have to be "next to" or "inside", but not "on top of". The usual trick of "just describe big type with a smaller number of CONTIGs won't work: MPI_Pack(const void *inbuf, int incount, MPI_Datatype datatype, void * outbuf, int outcount int *position); We're ok on inputs: a normal buf, count, datatype tuple. How do you work around the output, though? outcount is a count of bytes, not a count of datatypes. Position is a plain old pointer to memory. ==rob -- Rob Latham Mathematics and Computer Science Division Argonne National Lab, IL USA

1 0

Re: [mpich-devel] when is MPICH going to be count-safe?
by Jeff Hammond 18 Oct '13

18 Oct '13

It's not really in MPICH test format so I have to modify it before commit. And it sounds like Rob did something similar enough already. Jeff ----- Jim Dinan <james.dinan(a)gmail.com> wrote: > All I'm suggesting is that you guys commit the test case that you just ran, > like 5 emails ago. > > > On Fri, Oct 18, 2013 at 5:21 PM, Jeff Hammond <jhammond(a)alcf.anl.gov> wrote: > > > We look forward to your patch :-) > > > > jeff > > > > ----- Original Message ----- > > > From: "Jim Dinan" <james.dinan(a)gmail.com> > > > To: devel(a)mpich.org > > > Sent: Friday, October 18, 2013 3:19:32 PM > > > Subject: Re: [mpich-devel] when is MPICH going to be count-safe? > > > > > > > > > More tests is more better. > > > > > > > > > A test in hand is worth two in the bush. > > > > > > > > > :) > > > > > > > > > ~Jim. > > > > > > > > > > > > On Fri, Oct 18, 2013 at 3:39 PM, Rob Latham < robl(a)mcs.anl.gov > > > > wrote: > > > > > > > > > > > > On Fri, Oct 18, 2013 at 03:25:34PM -0400, Jim Dinan wrote: > > > > (Disclaimer: I haven't looked at the branch in question, so please > > > > ignore > > > > if redundant) It would be good to commit the test case too, if you > > > > haven't > > > > already added it. > > > > > > I added large_type to the test suite a while back. It constructs a > > > type not unlike what Jeff describes. > > > > > > The mere act of committing some of those types makes MPICH blow up, > > > so > > > I don't really know how my feature branch can make things worse than > > > that! :> > > > > > > > > > > > > ==rob > > > > > > -- > > > Rob Latham > > > Mathematics and Computer Science Division > > > Argonne National Lab, IL USA > > > > > > > > > > -- > > Jeff Hammond > > Argonne Leadership Computing Facility > > University of Chicago Computation Institute > > jhammond(a)alcf.anl.gov / (630) 252-5381 > > http://www.linkedin.com/in/jeffhammond > > https://wiki.alcf.anl.gov/parts/index.php/User:Jhammond > > ALCF docs: http://www.alcf.anl.gov/user-guides > > > > -- Jeff Hammond Argonne Leadership Computing Facility University of Chicago Computation Institute jhammond(a)alcf.anl.gov / (630) 252-5381 http://www.linkedin.com/in/jeffhammond https://wiki.alcf.anl.gov/parts/index.php/User:Jhammond ALCF docs: http://www.alcf.anl.gov/user-guides

1 0

Re: [mpich-devel] MPI_Pack/MPI_Unpack/MPI_Pack_size and large datatypes
by Pavan Balaji 18 Oct '13

18 Oct '13

Hi Rob, I totally agree with adding this into MPICH. However, with respect to the MPI standard, can one not implement MPIX_Pack_x on top of current MPI-3 using appropriate datatype creation? -- Pavan On Oct 18, 2013, at 4:40 PM, Rob Latham wrote: > No one on the mpi forum had much to say when I mentioned the oddity > that is MPI_Pack: 'int' used to describe position in a bytestream. > 'int' used to describe a memory address... > > When Jeff brought up MPI_Pack a year ago, the response was "it took us > 3 years to do MPI_Count. we're not touching MPI_Pack". Fair enough. > > Except! the practice of packing mpi datatypes into a contiguous buffer > is rather common in the MPICH code. > > I propose promoting the 'outsize', 'position' and 'size' parameters of > the internal routines (MPIR_Pack_impl, MPIR_Unpack_impl, > MPIR_Pack_size_impl) to MPI_Aint: > > int MPIR_Pack_impl(const void *inbuf, int incount, MPI_Datatype > datatype, void *outbuf, MPI_Aint outcount, MPI_Aint *position); > > void MPIR_Pack_size_impl(int incount, MPI_Datatype datatype, > MPI_Aint *size); > > int MPIR_Unpack_impl(const void *inbuf, MPI_Aint insize, > MPI_Aint *position, void *outbuf, int outcount, > MPI_Datatype datatype); > > I would have used MPI_Count, but Dave and Jim had some concerns about > 128-bit math in places. Is pack/unpack ever on the critical path? > > I've added this change to the > mpich-review/large-count-large-typesize-combo review branch. > > ==rob > > -- > Rob Latham > Mathematics and Computer Science Division > Argonne National Lab, IL USA -- Pavan Balaji http://www.mcs.anl.gov/~balaji

1 0

MPI_Pack/MPI_Unpack/MPI_Pack_size and large datatypes
by Rob Latham 18 Oct '13

18 Oct '13

No one on the mpi forum had much to say when I mentioned the oddity that is MPI_Pack: 'int' used to describe position in a bytestream. 'int' used to describe a memory address... When Jeff brought up MPI_Pack a year ago, the response was "it took us 3 years to do MPI_Count. we're not touching MPI_Pack". Fair enough. Except! the practice of packing mpi datatypes into a contiguous buffer is rather common in the MPICH code. I propose promoting the 'outsize', 'position' and 'size' parameters of the internal routines (MPIR_Pack_impl, MPIR_Unpack_impl, MPIR_Pack_size_impl) to MPI_Aint: int MPIR_Pack_impl(const void *inbuf, int incount, MPI_Datatype datatype, void *outbuf, MPI_Aint outcount, MPI_Aint *position); void MPIR_Pack_size_impl(int incount, MPI_Datatype datatype, MPI_Aint *size); int MPIR_Unpack_impl(const void *inbuf, MPI_Aint insize, MPI_Aint *position, void *outbuf, int outcount, MPI_Datatype datatype); I would have used MPI_Count, but Dave and Jim had some concerns about 128-bit math in places. Is pack/unpack ever on the critical path? I've added this change to the mpich-review/large-count-large-typesize-combo review branch. ==rob -- Rob Latham Mathematics and Computer Science Division Argonne National Lab, IL USA

1 0