September 2011 - mpiwg-hybridpm - mailman3.cels.anl.gov

Re: [Mpi3-hybridpm] Telecon 09/28 at 10am central
by Pavan Balaji 28 Sep '11

28 Sep '11

The meeting minutes are available here: https://svn.mpi-forum.org/trac/mpi-forum-web/wiki/notes-2011-09-28 -- Pavan On 09/26/2011 09:05 PM, Pavan Balaji wrote: > > The next hybrid telecon will be on 09/28 (Wednesday) at 10am central. > The telecon information is available on the wiki. > > https://svn.mpi-forum.org/trac/mpi-forum-web/wiki/MPI3Hybrid > > Here's agenda for the telecon: > > 1. Helper team proposal: Doug will present the proposal one more time > and we'll have a straw vote on whether we want to take the proposal to > the Forum. > > 2. Update from Ron on the Comm_split_type proposal. > > 3. Update from Torsten/Jim on the shared memory windows proposal. > > 4. Feedback on the endpoints presentation at the Forum (unlikely to get > to in the 1 hour telecon). > > -- Pavan > -- Pavan Balaji http://www.mcs.anl.gov/~balaji

1 0

[Mpi3-hybridpm] notes/comments from Santorini
by Steve Oyanagi 27 Sep '11

27 Sep '11

Hi, By chance are there any notes/comments/etc from the Greece hybrid group meetings? Anything the rest of the list who didn't go to Santorini should know? Thanks, - Steve

1 0

Re: [Mpi3-hybridpm] Telecon 09/28 at 10am central
by Douglas Miller 27 Sep '11

27 Sep '11

The proposal is posted on ticket #217, please try to review it prior to the meeting. https://svn.mpi-forum.org/trac/mpi-forum-web/ticket/217 thanks, _______________________________________________ Douglas Miller BlueGene Messaging Development IBM Corp., Rochester, MN USA Bldg 030-2 A401 dougmill(a)us.ibm.com Douglas Miller/Rochester/IBM Pavan Balaji <balaji(a)mcs.anl.g ov> To Sent by: mpi3-hybridpm(a)lists.mpi-forum.org mpi3-hybridpm-bou cc nces(a)lists.mpi-fo rum.org Subject [Mpi3-hybridpm] Telecon 09/28 at 10am central 09/26/2011 09:05 PM Please respond to mpi3-hybridpm@lis ts.mpi-forum.org The next hybrid telecon will be on 09/28 (Wednesday) at 10am central. The telecon information is available on the wiki. https://svn.mpi-forum.org/trac/mpi-forum-web/wiki/MPI3Hybrid Here's agenda for the telecon: 1. Helper team proposal: Doug will present the proposal one more time and we'll have a straw vote on whether we want to take the proposal to the Forum. 2. Update from Ron on the Comm_split_type proposal. 3. Update from Torsten/Jim on the shared memory windows proposal. 4. Feedback on the endpoints presentation at the Forum (unlikely to get to in the 1 hour telecon). -- Pavan -- Pavan Balaji http://www.mcs.anl.gov/~balaji _______________________________________________ Mpi3-hybridpm mailing list Mpi3-hybridpm(a)lists.mpi-forum.org http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-hybridpm

1 0

[Mpi3-hybridpm] Telecon 09/28 at 10am central
by Pavan Balaji 26 Sep '11

26 Sep '11

The next hybrid telecon will be on 09/28 (Wednesday) at 10am central. The telecon information is available on the wiki. https://svn.mpi-forum.org/trac/mpi-forum-web/wiki/MPI3Hybrid Here's agenda for the telecon: 1. Helper team proposal: Doug will present the proposal one more time and we'll have a straw vote on whether we want to take the proposal to the Forum. 2. Update from Ron on the Comm_split_type proposal. 3. Update from Torsten/Jim on the shared memory windows proposal. 4. Feedback on the endpoints presentation at the Forum (unlikely to get to in the 1 hour telecon). -- Pavan -- Pavan Balaji http://www.mcs.anl.gov/~balaji

1 0

Re: [Mpi3-hybridpm] [Mpi3-rma] Fix for win_allocate_shared
by Torsten Hoefler 23 Sep '11

23 Sep '11

On Fri, 2011-09-23 at 12:56 -0500, Pavan Balaji wrote: > On 09/23/2011 12:47 PM, Barrett, Brian W wrote: > > It might not be able to. If non-coherent load/store is byte access > > (or word access), it can provide a separate public and private window > > in separate and meet the semantics. If it provides cacheline access > > to non-coherent space, it will not be able to provide shared memory > > across uncached space; such is the cost of standardization. > > Do we know any non-cache-coherent system that provides byte access (note > that word access is not sufficient)? > > What I'm arguing is that we are not giving any additional flexibility > for non-cache-coherent systems with this new "clarified semantics". In > fact, we are making it more difficult for future MPI versions to > actually define a semantics by saying that "multiple remote stores to > the same window have to work" -- we'll be stuck with this if we say that > in the standard now. I see your point but would like to postpone to the telecon. > Instead, if we leave it undefined, in MPI-3.1 (or 4.0), we might be able > to separate out local load/store and remote load/store operations and > say that multiple remote load/store operations is undefined (or > erroneous or whatever). > > So the options I'm proposing are: > > 1. Either define remote load/stores in SEPARATE. I think we can just define that concurrent accesses are undefined. > (or) > > 2. Leave it as undefined. > > Attempting a half-definition is making it worse than leaving it undefined. It's not a half-definition. It's very clear in that it's not supporting some architectures (that may not expose shared memory windows then). All the Best, Torsten

1 0

Re: [Mpi3-hybridpm] [Mpi3-rma] Fix for win_allocate_shared
by Torsten Hoefler 23 Sep '11

23 Sep '11

On Fri, Sep 23, 2011 at 12:40:43PM -0500, Pavan Balaji wrote: > > On 09/23/2011 12:39 PM, Torsten Hoefler wrote: >> It clarifies semantics for separate (the first part) and the advice to >> implementers makes the semantic restriction explicit. > > What I mean is -- how will an MPI implementation on non-cache-coherent > system be able to expose shared memory with this restriction? Our decision was: Either expose non-cacheable memory or not at all. Well, the decision is not final, it's very easy to add a rule that says that the outcome of conflicting remote/local load/store accesses is undefined. Torsten PS: Sorry for my sluggish response, we're all stuck on an island with internet issues ... ;-) -- bash$ :(){ :|:&};: --------------------- http://www.unixer.de/ ----- Torsten Hoefler | Performance Modeling and Simulation Lead Blue Waters Directorate | University of Illinois (UIUC) 1205 W Clark Street | Urbana, IL, 61801 NCSA Building | +01 (217) 244-7736

1 0

Re: [Mpi3-hybridpm] [Mpi3-rma] Fix for win_allocate_shared
by Pavan Balaji 23 Sep '11

23 Sep '11

Ok, I understand your disagreement. We should discuss this in the telecon. -- Pavan On 09/23/2011 12:59 PM, Barrett, Brian W wrote: > I disagree. It provides a rational target for future architectures and provides the ability of all cache coherent systems to support shared memory, something I'm not convinced the unified model does. > > Brian > > ----- Original Message ----- > From: Pavan Balaji [mailto:[email protected]] > Sent: Friday, September 23, 2011 11:56 AM > To: mpi3-rma(a)lists.mpi-forum.org<mpi3-rma(a)lists.mpi-forum.org> > Cc: Barrett, Brian W; Torsten Hoefler<htor(a)illinois.edu>; mpi3-hybridpm(a)lists.mpi-forum.org<mpi3-hybridpm(a)lists.mpi-forum.org> > Subject: Re: [Mpi3-rma] Fix for win_allocate_shared > > > On 09/23/2011 12:47 PM, Barrett, Brian W wrote: >> It might not be able to. If non-coherent load/store is byte access >> (or word access), it can provide a separate public and private window >> in separate and meet the semantics. If it provides cacheline access >> to non-coherent space, it will not be able to provide shared memory >> across uncached space; such is the cost of standardization. > > Do we know any non-cache-coherent system that provides byte access (note > that word access is not sufficient)? > > What I'm arguing is that we are not giving any additional flexibility > for non-cache-coherent systems with this new "clarified semantics". In > fact, we are making it more difficult for future MPI versions to > actually define a semantics by saying that "multiple remote stores to > the same window have to work" -- we'll be stuck with this if we say that > in the standard now. > > Instead, if we leave it undefined, in MPI-3.1 (or 4.0), we might be able > to separate out local load/store and remote load/store operations and > say that multiple remote load/store operations is undefined (or > erroneous or whatever). > > So the options I'm proposing are: > > 1. Either define remote load/stores in SEPARATE. > > (or) > > 2. Leave it as undefined. > > Attempting a half-definition is making it worse than leaving it undefined. > > -- Pavan > -- Pavan Balaji http://www.mcs.anl.gov/~balaji

1 0

Re: [Mpi3-hybridpm] [Mpi3-rma] Fix for win_allocate_shared
by Pavan Balaji 23 Sep '11

23 Sep '11

On 09/23/2011 12:47 PM, Barrett, Brian W wrote: > It might not be able to. If non-coherent load/store is byte access > (or word access), it can provide a separate public and private window > in separate and meet the semantics. If it provides cacheline access > to non-coherent space, it will not be able to provide shared memory > across uncached space; such is the cost of standardization. Do we know any non-cache-coherent system that provides byte access (note that word access is not sufficient)? What I'm arguing is that we are not giving any additional flexibility for non-cache-coherent systems with this new "clarified semantics". In fact, we are making it more difficult for future MPI versions to actually define a semantics by saying that "multiple remote stores to the same window have to work" -- we'll be stuck with this if we say that in the standard now. Instead, if we leave it undefined, in MPI-3.1 (or 4.0), we might be able to separate out local load/store and remote load/store operations and say that multiple remote load/store operations is undefined (or erroneous or whatever). So the options I'm proposing are: 1. Either define remote load/stores in SEPARATE. (or) 2. Leave it as undefined. Attempting a half-definition is making it worse than leaving it undefined. -- Pavan -- Pavan Balaji http://www.mcs.anl.gov/~balaji

1 0

Re: [Mpi3-hybridpm] [Mpi3-rma] Fix for win_allocate_shared
by Pavan Balaji 23 Sep '11

23 Sep '11

On 09/23/2011 12:39 PM, Torsten Hoefler wrote: > It clarifies semantics for separate (the first part) and the advice to > implementers makes the semantic restriction explicit. What I mean is -- how will an MPI implementation on non-cache-coherent system be able to expose shared memory with this restriction? -- Pavan -- Pavan Balaji http://www.mcs.anl.gov/~balaji

1 0

Re: [Mpi3-hybridpm] [Mpi3-rma] Fix for win_allocate_shared
by Torsten Hoefler 23 Sep '11

23 Sep '11

On Fri, 2011-09-23 at 12:35 -0500, Pavan Balaji wrote: > On 09/23/2011 12:32 PM, Torsten Hoefler wrote: > > On Fri, 2011-09-23 at 12:25 -0500, Pavan Balaji wrote: > >> On 09/23/2011 02:39 AM, Torsten Hoefler wrote: > >>> "The semantic rules for shared memory windows in the MPI_RMA_SEPARATE > >>> model allow concurrent load/store accesses from two distinct processes > >>> to the same target window that is remote to both processes. An > >>> implementation cannot expose shared memory windows on hardware that does > >>> not support this access mode." > >> > >> This seems to suggest that it won't work for non-cache-coherent systems. > >> Didn't we decide against that? > > Brian probably wants to comment on this :-). > > > > Here's my recap: The issue was that many architectures may not be able > > to support UNIFIED. So we don't want to limit the SEPARATE model too > > much in order to still make it useful. If we later, based on our > > experience, find that we need this limitation to support existing > > architectures, then we can add the restriction. However, if we add it > > now, it may limiting the programming model unnecessarily. > > I'm not sure how this is better than my original suggestion of leaving > this as "undefined" for SEPARATE. That way, it is possible to define it > in the later MPI standards. But Jim and you didn't want to do that. It clarifies semantics for separate (the first part) and the advice to implementers makes the semantic restriction explicit. Best, Torsten

1 0