October 2010 - mpiwg-rma - mailman3.cels.anl.gov

Re: [Mpi3-rma] RMA proposal 2 initial text
by Pavan Balaji 31 Oct '10

31 Oct '10

I've uploaded an updated draft of the proposal that includes user-defined accumulate operations as well. -- Pavan On 10/29/2010 12:33 PM, Pavan Balaji wrote: > > Thanks. Updated the description on the wiki page. There are three > changes in this proposal as listed below. > > -- Pavan > > On 10/29/2010 11:49 AM, William Gropp wrote: >> Thanks. I've added this to the Propoal 2 page on the wiki ( https://svn.mpi-forum.org/trac/mpi-forum-web/wiki/mpi3-rma-proposal2 >> ) >> >> Bill >> >> On Oct 27, 2010, at 9:41 PM, Pavan Balaji wrote: >> >>> All, >>> >>> I've attached a draft of proposal 2. Most of the changes that were >>> originally planned for proposal 2 have been merged into proposal 1. I >>> think we might be able to merge some more (or all) features into >>> proposal 1 as well. >>> >>> Here are the main changes: >>> >>> 1. Communication ordering: communication from the same source to the >>> same memory location is ordered now for all epochs (not something >>> specific to lockfree). The user can relax it by setting the >>> "unordered" >>> info argument or a per-epoch MPI_MODE_UNORDERED assertion. >>> >>> 2. New calls for per-operation local completions for buffer reuse. >>> Appropriate test and wait calls are also added. >>> >>> 3. Added a new MPI_WIN_LOCK_WAIT call that blocks till the lock is >>> acquired for using shared memory regions with load/store capabilities >>> within a window. >>> >>> I haven't added the MPI_WIN_ALLFLUSHALL call yet. >>> >>> -- Pavan >>> >>> -- >>> Pavan Balaji >>> http://www.mcs.anl.gov/~balaji >>> <one-side-2.pdf><ATT00001..txt> >> >> William Gropp >> Deputy Director for Research >> Institute for Advanced Computing Applications and Technologies >> Paul and Cynthia Saylor Professor of Computer Science >> University of Illinois Urbana-Champaign >> >> >> >> >> _______________________________________________ >> mpi3-rma mailing list >> mpi3-rma(a)lists.mpi-forum.org >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma > -- Pavan Balaji http://www.mcs.anl.gov/~balaji

1 0

Re: [Mpi3-rma] RMA proposal 2 initial text
by Pavan Balaji 29 Oct '10

29 Oct '10

Thanks. Updated the description on the wiki page. There are three changes in this proposal as listed below. -- Pavan On 10/29/2010 11:49 AM, William Gropp wrote: > Thanks. I've added this to the Propoal 2 page on the wiki ( https://svn.mpi-forum.org/trac/mpi-forum-web/wiki/mpi3-rma-proposal2 > ) > > Bill > > On Oct 27, 2010, at 9:41 PM, Pavan Balaji wrote: > >> All, >> >> I've attached a draft of proposal 2. Most of the changes that were >> originally planned for proposal 2 have been merged into proposal 1. I >> think we might be able to merge some more (or all) features into >> proposal 1 as well. >> >> Here are the main changes: >> >> 1. Communication ordering: communication from the same source to the >> same memory location is ordered now for all epochs (not something >> specific to lockfree). The user can relax it by setting the >> "unordered" >> info argument or a per-epoch MPI_MODE_UNORDERED assertion. >> >> 2. New calls for per-operation local completions for buffer reuse. >> Appropriate test and wait calls are also added. >> >> 3. Added a new MPI_WIN_LOCK_WAIT call that blocks till the lock is >> acquired for using shared memory regions with load/store capabilities >> within a window. >> >> I haven't added the MPI_WIN_ALLFLUSHALL call yet. >> >> -- Pavan >> >> -- >> Pavan Balaji >> http://www.mcs.anl.gov/~balaji >> <one-side-2.pdf><ATT00001..txt> > > William Gropp > Deputy Director for Research > Institute for Advanced Computing Applications and Technologies > Paul and Cynthia Saylor Professor of Computer Science > University of Illinois Urbana-Champaign > > > > > _______________________________________________ > mpi3-rma mailing list > mpi3-rma(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma -- Pavan Balaji http://www.mcs.anl.gov/~balaji

1 0

Re: [Mpi3-rma] RMA proposal 2 initial text
by William Gropp 29 Oct '10

29 Oct '10

Thanks. I've added this to the Propoal 2 page on the wiki ( https://svn.mpi-forum.org/trac/mpi-forum-web/wiki/mpi3-rma-proposal2 ) Bill On Oct 27, 2010, at 9:41 PM, Pavan Balaji wrote: > All, > > I've attached a draft of proposal 2. Most of the changes that were > originally planned for proposal 2 have been merged into proposal 1. I > think we might be able to merge some more (or all) features into > proposal 1 as well. > > Here are the main changes: > > 1. Communication ordering: communication from the same source to the > same memory location is ordered now for all epochs (not something > specific to lockfree). The user can relax it by setting the > "unordered" > info argument or a per-epoch MPI_MODE_UNORDERED assertion. > > 2. New calls for per-operation local completions for buffer reuse. > Appropriate test and wait calls are also added. > > 3. Added a new MPI_WIN_LOCK_WAIT call that blocks till the lock is > acquired for using shared memory regions with load/store capabilities > within a window. > > I haven't added the MPI_WIN_ALLFLUSHALL call yet. > > -- Pavan > > -- > Pavan Balaji > http://www.mcs.anl.gov/~balaji > <one-side-2.pdf><ATT00001..txt> William Gropp Deputy Director for Research Institute for Advanced Computing Applications and Technologies Paul and Cynthia Saylor Professor of Computer Science University of Illinois Urbana-Champaign

1 0

Re: [Mpi3-rma] RMA proposal 2 initial text
by Jeff Hammond 28 Oct '10

28 Oct '10

This sentence is missing a word: "MPI_WIN_CREATE_DYNAMIC creates a window that allows ________ to attach (register) and detach (deregister) process memory locally." It doesn't sound like MPI, but I would use the word "one" there. Perhaps the MPI way is "the programmer" or "the user". Best, Jeff On Wed, Oct 27, 2010 at 9:41 PM, Pavan Balaji <balaji(a)mcs.anl.gov> wrote: > All, > > I've attached a draft of proposal 2. Most of the changes that were > originally planned for proposal 2 have been merged into proposal 1. I think > we might be able to merge some more (or all) features into proposal 1 as > well. > > Here are the main changes: > > 1. Communication ordering: communication from the same source to the same > memory location is ordered now for all epochs (not something specific to > lockfree). The user can relax it by setting the "unordered" info argument or > a per-epoch MPI_MODE_UNORDERED assertion. > > 2. New calls for per-operation local completions for buffer reuse. > Appropriate test and wait calls are also added. > > 3. Added a new MPI_WIN_LOCK_WAIT call that blocks till the lock is acquired > for using shared memory regions with load/store capabilities within a > window. > > I haven't added the MPI_WIN_ALLFLUSHALL call yet. > > ï¿½-- Pavan > > -- > Pavan Balaji > http://www.mcs.anl.gov/~balaji > > _______________________________________________ > mpi3-rma mailing list > mpi3-rma(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma > > -- Jeff Hammond Argonne Leadership Computing Facility jhammond(a)alcf.anl.gov / (630) 252-5381 http://www.linkedin.com/in/jeffhammond

1 0

[Mpi3-rma] RMA proposal 2 initial text
by Pavan Balaji 27 Oct '10

27 Oct '10

All, I've attached a draft of proposal 2. Most of the changes that were originally planned for proposal 2 have been merged into proposal 1. I think we might be able to merge some more (or all) features into proposal 1 as well. Here are the main changes: 1. Communication ordering: communication from the same source to the same memory location is ordered now for all epochs (not something specific to lockfree). The user can relax it by setting the "unordered" info argument or a per-epoch MPI_MODE_UNORDERED assertion. 2. New calls for per-operation local completions for buffer reuse. Appropriate test and wait calls are also added. 3. Added a new MPI_WIN_LOCK_WAIT call that blocks till the lock is acquired for using shared memory regions with load/store capabilities within a window. I haven't added the MPI_WIN_ALLFLUSHALL call yet. -- Pavan -- Pavan Balaji http://www.mcs.anl.gov/~balaji

1 0

Re: [Mpi3-rma] Proposal 1 discussion points for next telecon
by Pavan Balaji 26 Oct '10

26 Oct '10

Bill, On 10/26/2010 02:41 PM, William Gropp wrote: > Names are *always* discussed *only* after the semantics are settled. Agreed. I just wanted to have note of that somewhere, so we don't forget to discuss it later. -- Pavan -- Pavan Balaji http://www.mcs.anl.gov/~balaji

1 0

Re: [Mpi3-rma] Proposal 1 discussion points for next telecon
by William Gropp 26 Oct '10

26 Oct '10

Names are *always* discussed *only* after the semantics are settled. Bill On Oct 25, 2010, at 11:35 PM, Pavan Balaji wrote: > > On 10/25/2010 10:58 PM, Torsten Hoefler wrote: >> Hi Pavan, >>> This proposal doesn't have an atomic GET operation. >>> >>> MPI_ACCUMULATE with MPI_REPLACE is an atomic PUT. >>> >>> MPI_GET_ACCUMULATE with MPI_NO_OP does not work as an atomic GET >>> as it >>> does not take more than 1 count, or non-predefined datatypes. >> Correct, we had count>1 and ddts in an earlier version and changed it >> after heated discussions about buffering. I forgot to add it to the >> discussion items. I have no huge problems to allow ddts and >> counts>1, >> however, I believe Brian and Keith were against it. >> >> We should see if an advice to users/implementers that MPI_NO_OP is a >> special case that doesn't require buffering and that all other >> operations might be really really slow with large data. >> >> Please consider this item 7 on the discussion list! > > Yes, I remember the discussion and it is a valid argument -- it was > with > respect to buffering requirements because of retransmissions (for > reliability). > > Note that I'm not proposing that MPI_GET_ACCUMULATE be made more > generic. I'm just saying that we need a (possible separate) function > to > do atomic GETs. However, if MPI_GET_ACCUMULATE is not as generic as > GET, > it should be given a different name (though, if you want to discuss > names later, that's fine). > > -- Pavan > > -- > Pavan Balaji > http://www.mcs.anl.gov/~balaji > _______________________________________________ > mpi3-rma mailing list > mpi3-rma(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma William Gropp Deputy Director for Research Institute for Advanced Computing Applications and Technologies Paul and Cynthia Saylor Professor of Computer Science University of Illinois Urbana-Champaign

1 0

Re: [Mpi3-rma] Proposal 1 discussion points for next telecon
by Pavan Balaji 25 Oct '10

25 Oct '10

On 10/25/2010 10:58 PM, Torsten Hoefler wrote: > Hi Pavan, >> This proposal doesn't have an atomic GET operation. >> >> MPI_ACCUMULATE with MPI_REPLACE is an atomic PUT. >> >> MPI_GET_ACCUMULATE with MPI_NO_OP does not work as an atomic GET as it >> does not take more than 1 count, or non-predefined datatypes. > Correct, we had count>1 and ddts in an earlier version and changed it > after heated discussions about buffering. I forgot to add it to the > discussion items. I have no huge problems to allow ddts and counts>1, > however, I believe Brian and Keith were against it. > > We should see if an advice to users/implementers that MPI_NO_OP is a > special case that doesn't require buffering and that all other > operations might be really really slow with large data. > > Please consider this item 7 on the discussion list! Yes, I remember the discussion and it is a valid argument -- it was with respect to buffering requirements because of retransmissions (for reliability). Note that I'm not proposing that MPI_GET_ACCUMULATE be made more generic. I'm just saying that we need a (possible separate) function to do atomic GETs. However, if MPI_GET_ACCUMULATE is not as generic as GET, it should be given a different name (though, if you want to discuss names later, that's fine). -- Pavan -- Pavan Balaji http://www.mcs.anl.gov/~balaji

1 0

Re: [Mpi3-rma] Proposal 1 discussion points for next telecon
by Torsten Hoefler 25 Oct '10

25 Oct '10

Hi Sayantan, > When is the next telecon scheduled? I don't know -- I believe Rajeev wanted to schedule a telecon next week or the week after. Thanks, Torsten -- bash$ :(){ :|:&};: --------------------- http://www.unixer.de/ ----- Torsten Hoefler | Performance Modeling and Simulation Lead Blue Waters Directorate | University of Illinois (UIUC) 1205 W Clark Street | Urbana, IL, 61801 NCSA Building | +01 (217) 244-7736

1 0

Re: [Mpi3-rma] Proposal 1 discussion points for next telecon
by Torsten Hoefler 25 Oct '10

25 Oct '10

Hi Pavan, > This proposal doesn't have an atomic GET operation. > > MPI_ACCUMULATE with MPI_REPLACE is an atomic PUT. > > MPI_GET_ACCUMULATE with MPI_NO_OP does not work as an atomic GET as it > does not take more than 1 count, or non-predefined datatypes. Correct, we had count>1 and ddts in an earlier version and changed it after heated discussions about buffering. I forgot to add it to the discussion items. I have no huge problems to allow ddts and counts >1, however, I believe Brian and Keith were against it. We should see if an advice to users/implementers that MPI_NO_OP is a special case that doesn't require buffering and that all other operations might be really really slow with large data. Please consider this item 7 on the discussion list! Thanks, Torsten > -- Pavan > > On 10/25/2010 05:15 PM, Torsten Hoefler wrote: >> Hi RMA Working Group, >> >> Bill and I finished the edits on proposal 1 (see wiki). >> >> We have several points to discuss in the next telecon: >> >> 1) We should work on a more generic info mechanism which solves the info >> attach, detach and query problems we mentioned (however, it should not >> be limited to RMA/windows) >> >> 2) We should discuss the register/deregister interface. Right now it >> has base and size. This has several issues. One possible solution would >> be to return a handle that is needed to free the memory. This has issues >> too. We put both choices into the current draft and should discuss. >> >> 3) CAS needs a query to determine if a datatype is hardware-optimized. >> We think it should use RMA_Query for this and assume that anything that >> returns the RMA unified model is hw optimized >> >> 4) Discuss difference between lock-free synchronization and lock/unlock. >> The semantics of lock-free are different from a lock-all/shared epoch. >> The main difference is that holding a "shared" lock means that the user >> guarantees that there are no conflicting accesses (if there are, then he >> can often "upgrade" to an exclusive lock). So this is consistent with >> the literature on concurrent programming. If we now allow conflicting >> accesses (remember that MPI-2 defines a "conflicting" access on a >> per-window granularity) in lock-shared then this would break this >> semantic property. One would also need to allow this for the single-lock >> epochs and this could cost us several optimizations. >> >> 5) Do we need ordering semantics for any other synchronization mode? I >> would like to keep it a bit separate for now (i.e., in the lockfree >> synch mode), however, it would not be hard to specify it also for other >> modes. Are there use-cases? >> >> 6) We added a sentence about the access granularity to 11.5.5 (lockfree >> synchronization). This is because we have to allow conflicting accesses >> (load/store + put/get etc.) in the same epoch on a window. The >> granularity there is aligned with MPI's access granularity (i.e., >> datatype sizes). We avoided to mention anything in bytes or such. >> >> I uploaded the latest working version to the wiki at >> https://svn.mpi-forum.org/trac/mpi-forum-web/attachment/wiki/mpi3-rma-propo… >> >> Thanks a lot, >> Torsten& Bill >> > > -- > Pavan Balaji > http://www.mcs.anl.gov/~balaji > -- bash$ :(){ :|:&};: --------------------- http://www.unixer.de/ ----- Es ist leichter, einen Atomkern zu spalten als ein Vorurteil [Albert Einstein]

1 0