February 2010 - mpiwg-rma - mailman3.cels.anl.gov

Re: [Mpi3-rma] RMA proposal
by Pavan Balaji 24 Feb '10

24 Feb '10

Based on some additional internal discussion and comments here, there's a new version of the draft put up: http://www.mcs.anl.gov/~balaji/tmp/rma/rma-1.5.pdf -- Pavan On 02/23/2010 12:51 PM, Pavan Balaji wrote: > All, > > I've created a new RMA proposal > (http://www.mcs.anl.gov/~balaji/tmp/rma/rma-1.4.pdf) and would like to > initiate some discussion on it. > > The general idea is similar to the lock/unlock primitives in MPI-2.2, > but the main difference is that it decouples synchronization of requests > from lock/unlock itself. I've added some discussion items at places > where there is no clear answer one way or the other. There are some > examples in the end to give you a feel of how it can be used. > > In order to not confuse with the semantics of MPI-2.2, I've forked off a > new concept called "Portals" which is similar to "Windows". We can > either keep it as a separate chapter (or sub-chapter within RMA); or we > can decide to merge it back into the concept of "Windows" as a new > synchronization primitive. Names and syntax are always open to change. > > Look forward to your comments. > > Thanks, > > -- Pavan > -- Pavan Balaji http://www.mcs.anl.gov/~balaji

1 0

[Mpi3-rma] RMA proposal
by Pavan Balaji 23 Feb '10

23 Feb '10

All, I've created a new RMA proposal (http://www.mcs.anl.gov/~balaji/tmp/rma/rma-1.4.pdf) and would like to initiate some discussion on it. The general idea is similar to the lock/unlock primitives in MPI-2.2, but the main difference is that it decouples synchronization of requests from lock/unlock itself. I've added some discussion items at places where there is no clear answer one way or the other. There are some examples in the end to give you a feel of how it can be used. In order to not confuse with the semantics of MPI-2.2, I've forked off a new concept called "Portals" which is similar to "Windows". We can either keep it as a separate chapter (or sub-chapter within RMA); or we can decide to merge it back into the concept of "Windows" as a new synchronization primitive. Names and syntax are always open to change. Look forward to your comments. Thanks, -- Pavan -- Pavan Balaji http://www.mcs.anl.gov/~balaji

1 0

Re: [Mpi3-rma] MPI 3 RMA Examples needed
by Jeff Hammond 06 Feb '10

06 Feb '10

Sorry for the delay. Maybe it is still Friday somewhere. Many quantum chemistry algorithms use shared-memory semantics where arrays are either (1) get/read-only or (2) acc/rmw-only. I don't see many calls to put. Many collectives could be non-blocking, but I don't know if the performance impact would be significant. In the Fock build procedure of distributed-data SCF/DFT, the code looks approximately like this [Comp. Phys. Comm. 143 (2002) 69ï¿½82]: ============================================================= matrices D and F distributed over processors (partial replication would be good but we can ignore this) R > ~2000 // if not, then use replicated-data (local D, F) algorithm instead N > ~400 // proportional to R D(r,r) is read-only F(r,r) is rmw-only while not converged do { zero F forall i in range(1:R) by chunks // chunks can be 1-14 for atomic shells or 20-100 for whole atoms at a time forall j in range(1:R) by chunks // can dynamic load balance here OR forall k in range(1:R) by chunks forall l in range(1:R) by chunks // can dynamic load balance here get D(i,j), D(i,k), D(i,l), D(j,k), D(j,l), D(k,l) from global D compute V(i,j,k,l) locally compute F(i,j), F(i,k), F(i,l), F(j,k), F(j,l), F(k,l) from D chunks and V accumulate local F(i,j), F(i,k), F(i,l), F(j,k), F(j,l), F(k,l) to global F end forall end forall end forall end forall fence // do some dense linear algebra - using Matlab notation [E,U] = eig(F) D = U(1:N,1:N)*transpose(U(1:N,1:N)) fence } end ============================================================= It is not impossible to implement this code with one-sided. MPQC uses only MPI two-sided calls and the SCF scales as well as NWChem on BlueGene/P, but (1) that is not an apples-to-apples comparison because of many factors, like the routines to compute V (2) BGP is biased in favor of an MPI+Pthreads programming model - , which is what MPQC uses - relative to GA/ARMCI running a process per core. With the amount of memory per node on most machines now, the replicated-to-distributed cutoff moves steadily upwards, especially if one uses shmem on the node. Hence, if it were scalable, using collectives would be fine. However, dynamic-load-balancing (DLB) is a huge issue to do the varying granularity of work associated with the calculation of V. In summary, I am not certain that changes to the existing RMA semantics are necessary to implement the aforementioned algorithm efficiently in MPI-2 except for the need to have RMW to implement DLB. A second class of quantum chemistry methods is far more memory intensive but a more favorable task-size distribution. The formation of a coupled-cluster intermediate or residual looks approximately like one of the following: ============================================================= N > 200 R > 1000 J = 4D array of size O(N^4) which is antisymmetic in index pairs over N and R T = 4D array of size O(N^2 * R^2) which is antisymmetic in index pairs over N and R V = 4D array of size O(N^2 * R^2) which is antisymmetic in index pairs over N and R // trying to compute J(i,j,k,l) += sum(a,b) T(a,b,k,l) * V(i,j,a,b) forall i in range(1:R) by chunks of ~20 forall j in range(1:R) by chunks of ~20 if i<j forall k in range(1:R) by chunks of ~20 forall l in range(1:R) by chunks of ~20 if k<l forall a in range(1:R) by chunks of ~50 forall b in range(1:R) by chunks of ~50 if a<b call dynamic_load_balance get T(a,b,k,l) // this is a 20x20x50x50 patch get V(i,j,a,b) // this is a 20x20x50x50 patch dgemm T(a,b,k,l) and V(i,j,a,b) to form J(i,j,k,l) accumulate J(i,j,k,l) to global J // this is a 20x20x20x20 patch end forall end forall end forall end forall end forall end forall ============================================================= N > 200 R > 1000 J = 2D array of size O(N * R) T = 4D array of size O(N^2 * R^2) which is antisymmetic in index pairs over N and R V = same as T // trying to compute J(j,a) += sum(b,i) T(b,i) * V(i,j,a,b) broadcast T forall j in range(1:R) by chunks of ~20 forall i in range(1:R) by chunks of ~20 if i<j forall a in range(1:R) by chunks of ~50 forall b in range(1:R) by chunks of ~50 if a<b call dynamic_load_balance get V(i,j,a,b) // this is a 20x20x50x50 patch do J(j,a) += sum(b,i) T(b,i) * V(i,j,a,b) for local patches end forall end forall end forall end forall end forall end forall accumulate-reduce J broadcast J // this might be over a different communication than the reduction ============================================================= N > 200 R > 1000 J = 2D array of size O(N^2) T = 4D array of size O(N^2 * R^2) which is antisymmetic in index pairs over N and R V = 4D array of size O(N^2 * R^2) which is antisymmetic in index pairs over N and R // trying to compute J(j,k) += sum(a,b,i) T(a,b,k,i) * V(i,j,a,b) forall j in range(1:R) by chunks of ~20 forall k in range(1:R) by chunks of ~20 forall a in range(1:R) by chunks of ~50 forall b in range(1:R) by chunks of ~50 if a<b forall i in range(1:R) by chunks of ~20 if k<i and i<j call dynamic_load_balance get T(a,b,k,i) // this is a 20x20x50x50 patch get V(i,j,a,b) // this is a 20x20x50x50 patch do J(j,k) += sum(a,b,i) T(a,b,k,i) * V(i,j,a,b) for local patches end forall end forall end forall end forall end forall end forall accumulate-reduce J broadcast J // this might be over a different communication than the reduction ============================================================= I can say with certainty that passive progress is critical to the performance of the above three procedures. The local work inevitably varies and all the processes will be out-of-sync almost immediately, but since the local work can take up to a minute in the worst case, if an incoming remote get or acc is blocked until that call finishes, the overhead is huge. On BGP, lack of passive progress decreased the performance by 40-50% over the entire job. I'm brushing much of the complexity of quantum many-body theory under the rug, but if you look at the NWChem source, the above pseudo-code is fairly well represented there. Of course, there could be better algorithms, but coding them up with the all index permutation symmetry is a pain, hence the TCE project. Best, Jeff On Sun, Jan 31, 2010 at 8:16 AM, William Gropp <wgropp(a)illinois.edu> wrote: > Dear MPI RMA Group, > > We have several partial MPI RMA proposals. ï¿½To move forward, we need > to have a better understanding of the real needs by users, and we will > probably need to make some tough decisions about what we will support > and what we won't (as Marc has noted, some fairly obvious shared > memory operations are very tough in OpenMP, so its clear that being > universal isn't requred). > > What we'd like by this *Friday* are some specific examples of > operations that are hard to achieve in MPI RMA *and* that have a clear > application need. ï¿½What we *don't* want is simply "we should have a > better ï¿½Put", "we need active messages", or "the implementations I've used > are > too slow". ï¿½What we do want is something like the following: > > We've implemented a halo exchange with MPI-RMA, and the construction > of the Memory windows is awkward and limiting, particularly if the > domains are created dynamically, making it hard to create the memory > windows collectively. ï¿½We need either a method that lets us export a > local window or a way to allow all processes to refer to one single > window (something like the MPI_WIN_WORLD proposal). ï¿½Example code can > be found at <url here> ï¿½(or post on wiki). > > or > > We need a fetch and increment (or something similar) to implement a > remote lock that will allow us to make a complex series of remote > updates (and accesses) atomically that are needed for <specific > application description here>. ï¿½As shown in Using MPI-2, while a fetch > and increment is possible in MPI-RMA, it is extremely complex and > awkward. > > We'll take these examples and compare them to the current proposals > and the original MPI RMA in order to evaluate where we are. > > Again, please send us your concrete requirements by Friday, Feb 5th. > Thanks! > > Bill and Rajeev > > William Gropp > Deputy Director for Research > Institute for Advanced Computing Applications and Technologies > Paul and Cynthia Saylor Professor of Computer Science > University of Illinois Urbana-Champaign > > > > > _______________________________________________ > mpi3-rma mailing list > mpi3-rma(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma > -- Jeff Hammond Argonne Leadership Computing Facility jhammond(a)mcs.anl.gov / (630) 252-5381 http://www.linkedin.com/in/jeffhammond

1 0

[Mpi3-rma] Reminder: MPI 3 RMA Examples needed
by Rajeev Thakur 04 Feb '10

04 Feb '10

A reminder to send us your concrete requirements by tomorrow (need specific examples). Thanks to the few who have already responded. Rajeev -----Original Message----- From: mpi3-rma-bounces(a)lists.mpi-forum.org [mailto:[email protected]] On Behalf Of William Gropp Sent: Sunday, January 31, 2010 8:17 AM To: MPI 3.0 Remote Memory Access working group Subject: [Mpi3-rma] MPI 3 RMA Examples needed Dear MPI RMA Group, We have several partial MPI RMA proposals. To move forward, we need to have a better understanding of the real needs by users, and we will probably need to make some tough decisions about what we will support and what we won't (as Marc has noted, some fairly obvious shared memory operations are very tough in OpenMP, so its clear that being universal isn't requred). What we'd like by this *Friday* are some specific examples of operations that are hard to achieve in MPI RMA *and* that have a clear application need. What we *don't* want is simply "we should have a better Put", "we need active messages", or "the implementations I've used are too slow". What we do want is something like the following: We've implemented a halo exchange with MPI-RMA, and the construction of the Memory windows is awkward and limiting, particularly if the domains are created dynamically, making it hard to create the memory windows collectively. We need either a method that lets us export a local window or a way to allow all processes to refer to one single window (something like the MPI_WIN_WORLD proposal). Example code can be found at <url here> (or post on wiki). or We need a fetch and increment (or something similar) to implement a remote lock that will allow us to make a complex series of remote updates (and accesses) atomically that are needed for <specific application description here>. As shown in Using MPI-2, while a fetch and increment is possible in MPI-RMA, it is extremely complex and awkward. We'll take these examples and compare them to the current proposals and the original MPI RMA in order to evaluate where we are. Again, please send us your concrete requirements by Friday, Feb 5th. Thanks! Bill and Rajeev William Gropp Deputy Director for Research Institute for Advanced Computing Applications and Technologies Paul and Cynthia Saylor Professor of Computer Science University of Illinois Urbana-Champaign

1 0

[Mpi3-rma] MPI-3 RMA comments from GA team
by Jeff Hammond 03 Feb '10

03 Feb '10

I posted the example code Bruce is referring to below on the RMA Wiki. Best, Jeff ---------- Forwarded message ---------- From: Palmer, Bruce J <Bruce.Palmer(a)pnl.gov> Date: Tue, Feb 2, 2010 at 11:37 AM Subject: MPI-3 To: "jhammond(a)mcs.anl.gov" <jhammond(a)mcs.anl.gov> Jeff, I accidentally ditched your original message about this, but I wanted to get back to you on your request for things that are difficult to do using a message passing model. A couple of things came to mind: 1) global counters (this would tie in to dynamic load balancing schemes, which would also be difficult to implement in MPI) 2) I think the scatter/gather/scatter-accumulate functionality in GA would be tremendously difficult to reproduce in MPI (this is distinct from the MPI versions of scatter and gather). This is a fairly complicated functionality in GA as well, but it seems like creating a comparable capability in MPI would be extremely unpleasant. The gather/scatter routines seem to be useful for parallel computations on unstructured grids, e.g. updating ghost cells and other kinds of calculations on sparse or unstructured data. 3) We created a code to do a sparse matrix-vector multiply in the global/testing directory called sprsmatmult.F that uses the gather/scatter routines. I don't know if the algorithm is particularly optimal but it is another example of something that would be difficult to do outside a one-sided context. Of course, all of the above assumes the existence of a global address space, but supporting a GAS is much more straightforward using one-sided communication than it would be for message passing. In fact, I'm not really sure how you would get a GAS to work for a message passing model without making most of your operations collectives. Bruce -- Jeff Hammond Argonne Leadership Computing Facility jhammond(a)mcs.anl.gov / (630) 252-5381 http://www.linkedin.com/in/jeffhammond

1 0

Re: [Mpi3-rma] mpi3-rma Digest, Vol 13, Issue 1
by Torsten Hoefler 02 Feb '10

02 Feb '10

On Tue, Feb 02, 2010 at 08:07:18AM -0600, Quincey Koziol wrote: > Hi all, > I have two difficult problems in HDF5 that I need to solve, hopefully with some portion of new RMA-style work: > > - HDF5 files have a single global resource: the space in the file > itself. I'm willing to concede that I may not be able to track > individual pieces of free space in the file when operating in > parallel, but I definitely need to be able to allocate new space at > the end of the file in a "thread-safe" way, without setting aside any > MPI process or thread to manage the "end of file" value. I believe > that an atomic fetch-and-increment would solve this problem, as long > as it could increment by an arbitrary amount (not just '1') and > didn't involve any other MPI processes besides the one requesting the > space in the file. that should work "out of the box" with MPI_Get_accumulate (or MPI_Accumulate_get). You could even do this today with a somewhat more complex protocol, i.e., Peterson's, Dekker's or Lamport's algorithms for mutual exclusion, involving multiple round-trips :-(. You could also implement AM semantics in top of P2P and Iprobes if you don't want an additional thread. > - HDF5 files also have multiple distributed (local, perhaps is a > better word?) resources that need to be operated on in atomic ways > (i.e. without involving other MPI processes). For example, when an > application wants to create a new object in an HDF5 file, the > following operations need to occur (or something similar): > > 1*) <Allocate space for the object in the file> Well, MPI will probably never have an "allocate memory" operation :-). This seems to be a good use-case for user-defined accumulate operations, or even better: active messages! > 2) <Write the information for the new object to the space allocated> > 3*) <Check if an object of the requested name exists in a group in > the file, and if not, modify the group's data structure(s) to include > a link to the new object> Same as 1). > Operations marked with a '*' need to happen atomically and without > other processes being involved. Step 3) seems analogous to a > fetch-and-increment, in an abstract way, but I'm not certain how to > solve it. Would the "Remote Method Invocation" be able to take care > of this sort of operation, which is essentially equivalent to > inserting an object in a hash table? Yes, for user-defined ops or AM's, you would probably need to do your own locking with special locks (to insure message progression and avoid deadlocks). I'd recommend to look at the GASNet documentation if you really want to know why. All the Best, Torsten -- bash$ :(){ :|:&};: --------------------- http://www.unixer.de/ ----- Torsten Hoefler | Research Associate Blue Waters Directorate | University of Illinois 1205 W Clark Street | Urbana, IL, 61801 NCSA Building | +01 (217) 244-7736

1 0

Re: [Mpi3-rma] mpi3-rma Digest, Vol 13, Issue 1
by Quincey Koziol 02 Feb '10

02 Feb '10

Hi all, I have two difficult problems in HDF5 that I need to solve, hopefully with some portion of new RMA-style work: - HDF5 files have a single global resource: the space in the file itself. I'm willing to concede that I may not be able to track individual pieces of free space in the file when operating in parallel, but I definitely need to be able to allocate new space at the end of the file in a "thread-safe" way, without setting aside any MPI process or thread to manage the "end of file" value. I believe that an atomic fetch-and-increment would solve this problem, as long as it could increment by an arbitrary amount (not just '1') and didn't involve any other MPI processes besides the one requesting the space in the file. - HDF5 files also have multiple distributed (local, perhaps is a better word?) resources that need to be operated on in atomic ways (i.e. without involving other MPI processes). For example, when an application wants to create a new object in an HDF5 file, the following operations need to occur (or something similar): 1*) <Allocate space for the object in the file> 2) <Write the information for the new object to the space allocated> 3*) <Check if an object of the requested name exists in a group in the file, and if not, modify the group's data structure(s) to include a link to the new object> Operations marked with a '*' need to happen atomically and without other processes being involved. Step 3) seems analogous to a fetch-and-increment, in an abstract way, but I'm not certain how to solve it. Would the "Remote Method Invocation" be able to take care of this sort of operation, which is essentially equivalent to inserting an object in a hash table? Quincey On Jan 31, 2010, at 11:00 AM, mpi3-rma-request(a)lists.mpi-forum.org wrote: > Message: 1 > Date: Sun, 31 Jan 2010 08:16:48 -0600 > From: William Gropp <wgropp(a)illinois.edu> > Subject: [Mpi3-rma] MPI 3 RMA Examples needed > To: "MPI 3.0 Remote Memory Access working group" > <mpi3-rma(a)lists.mpi-forum.org> > Message-ID: <A696E02D-340B-4F5E-B16E-227B13E0A844(a)illinois.edu> > Content-Type: text/plain; charset="US-ASCII"; format=flowed; delsp=yes > > Dear MPI RMA Group, > > We have several partial MPI RMA proposals. To move forward, we need > to have a better understanding of the real needs by users, and we will > probably need to make some tough decisions about what we will support > and what we won't (as Marc has noted, some fairly obvious shared > memory operations are very tough in OpenMP, so its clear that being > universal isn't requred). > > What we'd like by this *Friday* are some specific examples of > operations that are hard to achieve in MPI RMA *and* that have a clear > application need. What we *don't* want is simply "we should have a > better Put", "we need active messages", or "the implementations I've > used are > too slow". What we do want is something like the following: > > We've implemented a halo exchange with MPI-RMA, and the construction > of the Memory windows is awkward and limiting, particularly if the > domains are created dynamically, making it hard to create the memory > windows collectively. We need either a method that lets us export a > local window or a way to allow all processes to refer to one single > window (something like the MPI_WIN_WORLD proposal). Example code can > be found at <url here> (or post on wiki). > > or > > We need a fetch and increment (or something similar) to implement a > remote lock that will allow us to make a complex series of remote > updates (and accesses) atomically that are needed for <specific > application description here>. As shown in Using MPI-2, while a fetch > and increment is possible in MPI-RMA, it is extremely complex and > awkward. > > We'll take these examples and compare them to the current proposals > and the original MPI RMA in order to evaluate where we are. > > Again, please send us your concrete requirements by Friday, Feb 5th. > Thanks! > > Bill and Rajeev > > William Gropp > Deputy Director for Research > Institute for Advanced Computing Applications and Technologies > Paul and Cynthia Saylor Professor of Computer Science > University of Illinois Urbana-Champaign > > > > > > > ------------------------------ > > _______________________________________________ > mpi3-rma mailing list > mpi3-rma(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma > > > End of mpi3-rma Digest, Vol 13, Issue 1 > ***************************************

1 0

Re: [Mpi3-rma] FW: MPI 3 RMA Examples needed
by Jeff Hammond 01 Feb '10

01 Feb '10

> However, if your problem is static (enough), you could establish > channels upfront (e.g., sparse matrix vector in Krylov subspace > methods). I am not a PDE person, but my intuition is that those sparse matvec solvers often run in conjunction with AMR or FEM where static, one-time initialization is impossible. However, I'll trust the PETSc team and similar-minded folks to figure out those issues when they come. Perhaps the LLNL folks know something about the multi-physics codes which solve continuum equations using FEM and can confirm or deny the possible utility of RMI in those codes at exascale. Jeff -- Jeff Hammond Argonne Leadership Computing Facility jhammond(a)mcs.anl.gov / (630) 252-5381 http://www.linkedin.com/in/jeffhammond

1 0

Re: [Mpi3-rma] FW: MPI 3 RMA Examples needed
by Torsten Hoefler 01 Feb '10

01 Feb '10

Jeff, > While we can debate the sanity of globally-scoped data at exascale, > there will always be science which is not amenable to simple domain > decomposition and thus requires some type of global view. Whether the > data structures are simple ala GA or complex ala Madness, RMI is the > only sensible ways to implement these involve RMI. just as a side-note: what you call RMI is what I'd call Active Message and I fully agree that this is one key technology to reach good scalability for a class of applications (I was mostly thinking of irregular applications but your point is an excellent addition). However, if your problem is static (enough), you could establish channels upfront (e.g., sparse matrix vector in Krylov subspace methods). Best, Torsten -- bash$ :(){ :|:&};: --------------------- http://www.unixer.de/ ----- Torsten Hoefler | Research Associate Blue Waters Directorate | University of Illinois 1205 W Clark Street | Urbana, IL, 61801 NCSA Building | +01 (217) 244-7736

1 0

Re: [Mpi3-rma] MPI 3 RMA Examples needed
by Underwood, Keith D 01 Feb '10

01 Feb '10

That is certainly a good start, but I am concerned about two issues being hard to figure out: 1) IB isn't exactly "the best you can do" in terms of RMA support. 2) What application usage scenario are you going to model? Brian and I prepped some slides for the last meeting (http://meetings.mpi-forum.org/secretary/2010/01/slides.php and then the RMA-put-January-2010 slides) that suggest that the API is not entirely broken from a perspective of arguments passed into Put (we should debate the significant difference between possible MPI_Put rate and possible shmem_put() rate, but it isn't 10x or anything). However, what is more concerning are questions regarding the actual usage model. If, for example, something about the semantic model required you to call one of the synchronization operations too often, that would be relevant to know. Keith > -----Original Message----- > From: Jeff Hammond [mailto:[email protected]] > Sent: Sunday, January 31, 2010 7:15 PM > To: Underwood, Keith D > Cc: MPI 3.0 Remote Memory Access working group > Subject: Re: [Mpi3-rma] MPI 3 RMA Examples needed > > Keith, > > Pavan and I are trying to separate implementation versus semantic > shortcomings by comparing ARMCI and MVAPICH over IB, and I will try to > also include OFED in an apples-to-apples-to-apples comparison of the > simplest operations. Both ARMCI and MVAPICH are heavily tuned for IB > and so I expect the implementation quality to be fairly similar; > hence, performance differences are probably due to semantics. > > We are also working going to see how close we can get to ARMCI with > MPI-2 RMA in an attempt to elucidate the semantic limitations of the > MPI standard. Obviously, ARMCI is not the end-all-be-all in one-sided > APIs, but I cannot get GASNet to function and OpenSHMEM is vaporware > (i.e. I cannot download it nor is it available on any machine I have > access to) so I choose to focus exclusively on ARMCI. > > Jeff > > On Sun, Jan 31, 2010 at 5:47 PM, William Gropp <wgropp(a)illinois.edu> > wrote: > > I agree that performance is an important issue, and we will need to > discuss > > it. ï¿½However, all too many of the current implementations are > unnecessarily > > poor in performance, so to first order, I believe we should > concentrate on > > what our users need (and believe can be implemented efficiently on > some > > platform) rather than the experience people have had with slow and > > unoptimized implementations. > > > > Bill > > > > On Jan 31, 2010, at 5:34 PM, Underwood, Keith D wrote: > > > >> This is an excellent framing of one of the things that will help the > RMA > >> group move forward; however, I do believe we need to get performance > into > >> the process. ï¿½What I do not know is how we separate performance of > the > >> interface from performance of an implementation. ï¿½Just because you > are able > >> to implement something without unbearably awkward code doesn't mean > that > >> code will perform. ï¿½How do we account for what is a semantic problem > (it > >> isn't POSSIBLE to implement this piece of code and make it go fast) > and what > >> is an implementation problem (nobody has bothered to make this piece > of code > >> fast)? ï¿½That input is just as important as what is syntactically > awkward. > >> > >> Keith > >> > >>> -----Original Message----- > >>> From: mpi3-rma-bounces(a)lists.mpi-forum.org [mailto:mpi3-rma- > >>> bounces(a)lists.mpi-forum.org] On Behalf Of William Gropp > >>> Sent: Sunday, January 31, 2010 7:17 AM > >>> To: MPI 3.0 Remote Memory Access working group > >>> Subject: [Mpi3-rma] MPI 3 RMA Examples needed > >>> > >>> Dear MPI RMA Group, > >>> > >>> We have several partial MPI RMA proposals. ï¿½To move forward, we > need > >>> to have a better understanding of the real needs by users, and we > will > >>> probably need to make some tough decisions about what we will > support > >>> and what we won't (as Marc has noted, some fairly obvious shared > >>> memory operations are very tough in OpenMP, so its clear that being > >>> universal isn't requred). > >>> > >>> What we'd like by this *Friday* are some specific examples of > >>> operations that are hard to achieve in MPI RMA *and* that have a > clear > >>> application need. ï¿½What we *don't* want is simply "we should have a > >>> better ï¿½Put", "we need active messages", or "the implementations > I've > >>> used are > >>> too slow". ï¿½What we do want is something like the following: > >>> > >>> We've implemented a halo exchange with MPI-RMA, and the > construction > >>> of the Memory windows is awkward and limiting, particularly if the > >>> domains are created dynamically, making it hard to create the > memory > >>> windows collectively. ï¿½We need either a method that lets us export > a > >>> local window or a way to allow all processes to refer to one single > >>> window (something like the MPI_WIN_WORLD proposal). ï¿½Example code > can > >>> be found at <url here> ï¿½(or post on wiki). > >>> > >>> or > >>> > >>> We need a fetch and increment (or something similar) to implement a > >>> remote lock that will allow us to make a complex series of remote > >>> updates (and accesses) atomically that are needed for <specific > >>> application description here>. ï¿½As shown in Using MPI-2, while a > fetch > >>> and increment is possible in MPI-RMA, it is extremely complex and > >>> awkward. > >>> > >>> We'll take these examples and compare them to the current proposals > >>> and the original MPI RMA in order to evaluate where we are. > >>> > >>> Again, please send us your concrete requirements by Friday, Feb > 5th. > >>> Thanks! > >>> > >>> Bill and Rajeev > >>> > >>> William Gropp > >>> Deputy Director for Research > >>> Institute for Advanced Computing Applications and Technologies > >>> Paul and Cynthia Saylor Professor of Computer Science > >>> University of Illinois Urbana-Champaign > >>> > >>> > >>> > >>> > >>> _______________________________________________ > >>> mpi3-rma mailing list > >>> mpi3-rma(a)lists.mpi-forum.org > >>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma > >> > >> _______________________________________________ > >> mpi3-rma mailing list > >> mpi3-rma(a)lists.mpi-forum.org > >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma > > > > William Gropp > > Deputy Director for Research > > Institute for Advanced Computing Applications and Technologies > > Paul and Cynthia Saylor Professor of Computer Science > > University of Illinois Urbana-Champaign > > > > > > > > > > _______________________________________________ > > mpi3-rma mailing list > > mpi3-rma(a)lists.mpi-forum.org > > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma > > > > > > -- > Jeff Hammond > Argonne Leadership Computing Facility > jhammond(a)mcs.anl.gov / (630) 252-5381 > http://www.linkedin.com/in/jeffhammond

1 0