mpiwg-rma

Download

[email protected]

April 2014

2 participants
2 discussions

Re: [mpiwg-rma] {MAX,MIN}LOC in RMA
by Jim Dinan 09 Apr '14

09 Apr '14

FYI, we added a test case for MPI_Accumulate with MIN/MAX_LOC to the MPICH test suite: http://git.mpich.org/mpich.git/commit/a80bf84aecfc6c6152474623d418b2a3cb71f… Also, the mutex idea I had didn't pan out because of the LOC operators always chose the least loc when the values are equal. Can't think of a way to make this work.. ~Jim. On Thu, Mar 13, 2014 at 3:25 PM, Jim Dinan <james.dinan(a)gmail.com> wrote: > MAX/MIN_LOC operations have an (albeit odd) atomic compare and replace > semantic. They operate on a pair type that has <value, location> > components. The user provides the data for both value and location. The > min/max comparison (i.e. less-than/greater-than) is done on the value. If > the comparison is true, then the remote <value, location> is atomically > replaced with the supplied data. > > These operations can be useful for one-sided reductions. I think PetSC > uses them for this. In MPI-2 (there are, of course better ways in MPI-3) > you can also use these operations to implement a spinlock. The value is a > boolean indicating if the mutex is taken and the location is the rank that > hold the mutex. The mutex initially contains the unlocked value <0, > MPI_PROC_NULL>. You do a MAX_LOC accumulate of <1, me> to try to claim the > mutex then do a get to see if you succeeded (exclusive epochs because of > mixing accumulates and gets and the same op restriction). To unlock, you > write the unlocked value back to the mutex. > > I have a test case that implements this, that's been queued up for a while > waiting to go out to MPICH. Should be able to push it out soon. > > ~Jim. > > > On Thu, Mar 13, 2014 at 3:05 PM, Jeff Hammond <jeff.science(a)gmail.com>wrote: > >> What is the expected behavior of {MAX,MIN}LOC in RMA? I have a hard >> time seeing what the use would be. Does anyone want to use MAXLOC to >> find the MAX of two numbers and write in the window the rank (chosen >> from the set of the initiator and the target) of the pair member that >> is larger? It's not like one can use this to implement a one-sided >> reduction since any time the value in the window is larger, the >> information about the original LOC will be lost. >> >> Can we remove these operations from the set of accumulate operations? >> >> Thanks, >> >> Jeff >> >> -- >> Jeff Hammond >> jeff.science(a)gmail.com >> _______________________________________________ >> mpiwg-rma mailing list >> mpiwg-rma(a)lists.mpi-forum.org >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma >> > >

1 0

Re: [mpiwg-rma] Problems with RMA synchronization in combination with load/store shared memory accesses
by Rolf Rabenseifner 08 Apr '14

08 Apr '14

Jim, I'm now sure, that mpich has a bug with assertions on shared memory windows. in the example, rcv_buf_left and rcv_buf_right are the windows. the only accesses to these rcv_buf_... are stores from remote and fully local loads. Both accesses are done in different epochs surrounded by MPI_Win_fence. According to your interpretation (which is really okay) all fences can use all possible assertions (!!!), except after the remote stores, MPI_MODE_NOSTORE cannot be used. I updated the example and mpich is executing it wrong. Please check it yourself on your installation: halo_1sided_store_win_alloc_shared_w-a-2-cray.c Without the assertions, all works: halo_1sided_store_win_alloc_shared_w-a-2NO-cray.c Could you verify that mpich has a bug? Additionally, I would recommend that we add after MPI-3.0 p451:33 Note that in shared memory windows (allocated with MPI_WIN_ALLOCATE_SHARED), there is no difference between remote store accesses and local store accesses to the window. This would help to understand that "the local window was not updated by stores" does not mean "by local stores", see p452:1 and p452:9. Is it a good idea? Best regards Rolf ----- Original Message ----- > From: "Jim Dinan" <james.dinan(a)gmail.com> > To: "MPI WG Remote Memory Access working group" <mpiwg-rma(a)lists.mpi-forum.org> > Sent: Friday, March 21, 2014 8:14:22 PM > Subject: Re: [mpiwg-rma] Problems with RMA synchronization in combination with load/store shared memory accesses > > > > Rolf, > > > This line is incorrect: MPI_Win_fence(MPI_MODE_NOSTORE + > MPI_MODE_NOPRECEDE, win_ rcv_buf _left ); > > > You need to do a bitwise OR of the assertions (MPI_MODE_NOSTORE | > MPI_MODE_NOPRECEDE). > > In halo_1sided_store_win_alloc_shared.c, you are doing stores within > the epoch, so MPI_MODE_NOSTORE looks like an incorrect assertion on > the closing fence. > > Following the Fence epoch, you are reading from the left/right recv > buffers. That also needs to be done within an RMA epoch, if you are > reading non-local data. > > ~Jim. > > > > On Fri, Feb 21, 2014 at 6:07 AM, Rolf Rabenseifner < > rabenseifner(a)hlrs.de > wrote: > > > Dear member of the RMA group and especially the mpich developers, > > I have real problems with the new shared memory in MPI-3.0, > i.e., the load/stores together with the RMA synchronization > causes wrong execution results. > > The attached > 1sided_halo_C_mpich_problems_rabenseifner.tar.gz or .zip > contains > > - 1sided/halo_1sided_put_win_alloc.c > > The basis that works. It uses MPI_Put and MPI_Win_fence for > duplex left/right halo communication. > > - 1sided/halo_1sided_store_win_alloc_shared.c > > This is the same, but a shared memory window is used and > the MPU_Put is substituted by storing the data in the > neighbors window. Same MPI_Win_fence with same assertions. > > This does not work, although I'm sure that my assertions are > correct. > > Known possibilities: > - I'm wrong and was not able to understand the assertions > on MPI-3.0 p452:8-19. > - I'm wrong because it is invalid to use the MPI_Win_fence > together with the shared memory windows. > - mpich has a bug. > (The first two possibilities are the reason, why I use this > Forum email list) > > - 1sided/halo_1sided_store_win_alloc_shared_w-a-cray.c > > This is a work-around-for Cray that works on our Cray > and does not use MPI_MODE_NOPRECEDE and MPI_MODE_NOSUCCEED. > It also runs on another mpich installation. > > - 1sided/halo_1sided_store_win_alloc_shared_pscw.c > > Here, MPI_Win_fence is substituted by Post-Start-Complete-Wait > and it does not work for any assertions. > > Same possibilities as above. > > - 1sided/halo_1sided_store_win_alloc_shared_query.c > - 1sided/halo_1sided_store_win_alloc_shared_query_w-a-cray.c > > Same as halo_1sided_store_win_alloc_shared.c > but non-contigues windows are used. > Same problems as above. > > - 1sided/halo_1sided_store_win_alloc_shared_othersync.c > > This version uses the synchronization according to > #413 and it is tested and works on two platforms. > > Best regards > Rolf > > -- > Dr. Rolf Rabenseifner . . . . . . . . . .. email rabenseifner(a)hlrs.de > High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 > University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 > Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner > Nobelstr. 19, D-70550 Stuttgart, Germany . . . . (Office: Room 1.307) > > _______________________________________________ > mpiwg-rma mailing list > mpiwg-rma(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma > > > _______________________________________________ > mpiwg-rma mailing list > mpiwg-rma(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma -- Dr. Rolf Rabenseifner . . . . . . . . . .. email rabenseifner(a)hlrs.de High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner Nobelstr. 19, D-70550 Stuttgart, Germany . . . . (Office: Room 1.307)

1 0