May 2017 - mpi-forum - mailman3.cels.anl.gov

Threading guarantees of MPI_User_function
by Phil Ruffwind 25 May '17

25 May '17

Hello all, I have another question about MPI_User_function. Does MPI make any guarantees regarding (a) whether multiple invocations of MPI_User_function could occur simultaneously within a given process for a given MPI_Op, and regarding (b) which thread MPI_User_function gets called on? The answer to (a) affects whether MPI_User_function would need any synchronization (like mutexes or semaphores) if it mutates global state. The answer to (b) affects whether it is possible to use thread-local state inside MPI_User_function. I would appreciate any clarifications. Thanks. Phil

1 0

Brief MPI Forum Mailing List Outage on May 26
by Bland, Wesley 25 May '17

25 May '17

Hi folks, All of the mailing lists for the MPI Forum will have a brief outage tomorrow (Friday, May 26th) at 10:00 AM Central US time while Argonne (our gracious mailing list host) migrates a server and we make a quick DNS change to point to the new location. This should be a brief outage and (hopefully) no incoming messages will be lost during the transition, but be aware that messages sent around that time have a chance of not getting delivered depending on your local DNS cache and timing. If you have any questions or concerns about this, please let me know. Thanks, Wesley

1 0

mpi_scatterv problem in fortran
by Siva Srinivas Kolukula 16 May '17

16 May '17

My aim is given any m*n matrix (rectangular or square) and given any number of processors..the matrix should be scattered into chunks/ sub arrays into each processor. I want to scatter matrix from root to other processors using scatterv. I am creating a communicator topology using *mpi_cart_create*. As an example I have the below code in fortran: PROGRAM SendRecv USE mpi IMPLICIT none integer, PARAMETER :: m = 4, n = 4 integer, DIMENSION(m,n) :: a, b,h integer :: i,j,count integer,allocatable, dimension(:,:):: loc ! local piece of global 2d array INTEGER :: istatus(MPI_STATUS_SIZE),ierr integer, dimension(2) :: sizes, subsizes, starts INTEGER :: ista,iend,jsta,jend,ilen,jlen INTEGER :: iprocs, jprocs, nprocs integer,allocatable,dimension(:):: rcounts, displs INTEGER :: rcounts0,displs0 integer, PARAMETER :: ROOT = 0 integer :: dims(2),coords(2) logical :: periods(2) data periods/2*.false./ integer :: status(MPI_STATUS_SIZE) integer :: comm2d,source,myrank integer :: newtype, resizedtype integer :: comsize,charsize integer(kind=MPI_ADDRESS_KIND) :: extent, begin CALL MPI_INIT(ierr) CALL MPI_COMM_SIZE(MPI_COMM_WORLD, nprocs, ierr) CALL MPI_COMM_RANK(MPI_COMM_WORLD, myrank, ierr) ! Get a new communicator for a decomposition of the domain. dims(1) = 0 dims(2) = 0 CALL MPI_DIMS_CREATE(nprocs,2,dims,ierr) if (myrank.EQ.Root) then print *,nprocs,'processors have been arranged into',dims(1),'X',dims(2),'grid' endif CALL MPI_CART_CREATE(MPI_COMM_WORLD,2,dims,periods,.true., & comm2d,ierr) ! Get my position in this communicator CALL MPI_COMM_RANK(comm2d,myrank,ierr) ! Get the decomposition CALL fnd2ddecomp(comm2d,m,n,ista,iend,jsta,jend) ! print *,ista,jsta,iend,jend ilen = iend - ista + 1 jlen = jend - jsta + 1 CALL MPI_Cart_get(comm2d,2,dims,periods,coords,ierr) iprocs = dims(1) jprocs = dims(2) ! define the global matrix if (myrank==ROOT) then count = 0 do j = 1,n do i = 1,m a(i,j) = count count = count+1 enddo enddo print *, 'global matrix is: ' do 90 i=1,m do 80 j = 1,n write(*,70)a(i,j) 70 format(2x,I5,$) 80 continue print *, ' ' 90 continue endif call MPI_Barrier(MPI_COMM_WORLD, ierr) starts = [0,0] sizes = [m, n] subsizes = [ilen, jlen] call MPI_Type_create_subarray(2, sizes, subsizes, starts, & MPI_ORDER_FORTRAN, MPI_INTEGER, & newtype, ierr) call MPI_Type_size(MPI_INTEGER, charsize, ierr) begin = 0 extent = charsize call MPI_Type_create_resized(newtype, begin, extent, resizedtype, ierr) call MPI_Type_commit(resizedtype, ierr) ! get counts and displacmeents allocate(rcounts(nprocs),displs(nprocs)) rcounts0 = 1 displs0 = (ista-1) + (jsta-1)*m CALL MPI_Allgather(rcounts0,1,MPI_INT,rcounts,1,MPI_INT,MPI_COMM_WORLD,IERR) CALL MPI_Allgather(displs0,1,MPI_INT,displs,1,MPI_INT,MPI_COMM_WORLD,IERR) CALL MPI_Barrier(MPI_COMM_WORLD, ierr) ! scatter data allocate(loc(ilen,jlen)) call MPI_Scatterv(a,rcounts,displs,resizedtype, & loc,ilen*jlen,MPI_INTEGER, & ROOT,MPI_COMM_WORLD,ierr) ! print each processor matrix do source = 0,nprocs-1 if (myrank.eq.source) then print *,'myrank:',source do i=1,ilen do j = 1,jlen write(*,701)loc(i,j) 701 format(2x,I5,$) enddo print *, ' ' enddo endif call MPI_Barrier(MPI_COMM_WORLD, ierr) enddo call MPI_Type_free(newtype,ierr) call MPI_Type_free(resizedtype,ierr) deallocate(rcounts,displs) deallocate(loc) CALL MPI_FINALIZE(ierr) contains subroutine fnd2ddecomp(comm2d,m,n,ista,iend,jsta,jend) integer comm2d integer m,n,ista,jsta,iend,jend integer dims(2),coords(2),ierr logical periods(2) ! Get (i,j) position of a processor from Cartesian topology. CALL MPI_Cart_get(comm2d,2,dims,periods,coords,ierr) ! Decomposition in first (ie. X) direction CALL MPE_DECOMP1D(m,dims(1),coords(1),ista,iend) ! Decomposition in second (ie. Y) direction CALL MPE_DECOMP1D(n,dims(2),coords(2),jsta,jend) end subroutine fnd2ddecomp SUBROUTINE MPE_DECOMP1D(n,numprocs,myid,s,e) integer n,numprocs,myid,s,e,nlocal,deficit nlocal = n / numprocs s = myid * nlocal + 1 deficit = mod(n,numprocs) s = s + min(myid,deficit) ! Give one more slice to processors if (myid .lt. deficit) then nlocal = nlocal + 1 endif e = s + nlocal - 1 if (e .gt. n .or. myid .eq. numprocs-1) e = n end subroutine MPE_DECOMP1D END program SendRecv I am generating a 4x4 matrix, and using scatterv I am sending the blocks of matrices to other processors. Code works fine for 4,2 and 16 processors. But throws a error for three processors. What modifications I have to do make it work for any number of given processors. Global matrix in Root: [ 0 4 8 12 1 5 9 13 2 6 10 14 3 7 11 15 ] For 4 processors each processors gets. Rank =0 : [0 4 1 5] Rank =1 : [8 12 9 13] Rank =2 : [2 6 3 7] Rank =3 : [10 14 11 15] Code works for 4, 2 and 16 processors; in fact it works when sub-arrays are of similar size. It fails for 3 processors. For 3 processors I am expecting: Rank =0 : [0 4 8 12 1 5 9 13] Rank =1 : [2 6 10 14] Rank =2 : [3 7 11 15] But I am getting the following error. Where I am missing? what modifications I have to make to make it work. Fatal error in PMPI_Scatterv: Message truncated, error stack: PMPI_Scatterv(671)................: MPI_Scatterv(sbuf=0x6b58c0, scnts=0xf95d90, displs=0xfafbe0, dtype=USER<resized>, rbuf=0xfafc00, rcount=4, MPI_INTEGER, root=0, MPI_COMM_WORLD) failed MPIR_Scatterv_impl(211)...........: I_MPIR_Scatterv_intra(278)........: Failure during collective I_MPIR_Scatterv_intra(272)........: MPIR_Scatterv(147)................: MPIDI_CH3U_Receive_data_found(131): Message from rank 0 and tag 6 truncated; 32 bytes received but buffer size is 16 Fatal error in PMPI_Scatterv: Message truncated, error stack: PMPI_Scatterv(671)................: MPI_Scatterv(sbuf=0x6b58c0, scnts=0x240bda0, displs=0x240be60, dtype=USER<resized>, rbuf=0x240be80, rcount=4, MPI_INTEGER, root=0, MPI_COMM_WORLD) failed MPIR_Scatterv_impl(211)...........: I_MPIR_Scatterv_intra(278)........: Failure during collective I_MPIR_Scatterv_intra(272)........: MPIR_Scatterv(147)................: MPIDI_CH3U_Receive_data_found(131): Message from rank 0 and tag 6 truncated; 32 bytes received but buffer size is 16 forrtl: error (69): process interrupted (SIGINT) Image PC Routine Line Source a.out 0000000000479165 Unknown Unknown Unknown a.out 0000000000476D87 Unknown Unknown Unknown a.out 000000000044B7C4 Unknown Unknown Unknown a.out 000000000044B5D6 Unknown Unknown Unknown a.out 000000000042DB76 Unknown Unknown Unknown a.out 00000000004053DE Unknown Unknown Unknown libpthread.so.0 00007F2327456790 Unknown Unknown Unknown libc.so.6 00007F2326EFE2F7 Unknown Unknown Unknown libmpi.so.12 00007F2327B899E8 Unknown Unknown Unknown libmpi.so.12 00007F2327C94E39 Unknown Unknown Unknown libmpi.so.12 00007F2327C94B32 Unknown Unknown Unknown libmpi.so.12 00007F2327B6E44A Unknown Unknown Unknown libmpi.so.12 00007F2327B6DD5D Unknown Unknown Unknown libmpi.so.12 00007F2327B6DBDC Unknown Unknown Unknown libmpi.so.12 00007F2327B6DB0C Unknown Unknown Unknown libmpi.so.12 00007F2327B6F932 Unknown Unknown Unknown libmpifort.so.12 00007F2328294B1C Unknown Unknown Unknown a.out 000000000040488B Unknown Unknown Unknown a.out 000000000040385E Unknown Unknown Unknown libc.so.6 00007F2326E4DD5D Unknown Unknown Unknown a.out 0000000000403769 Unknown Unknown Unknown I understand that for a given three processors, the first processor is getting 32 bytes (2X4), the same data is sent to other processors, but their receive count is 16 bytes (1X4), which is leading to error. How I can modify the program? I got the error reason, but I am not able to make it right to work. What modifications I have to make? _ *SAVE WATER ** ~ **SAVE ENERGY**~ **~ **SAVE EARTH *[image: Earth-22-june.gif (7996 bytes)] http://sites.google.com/site/kolukulasivasrinivas/ Siva Srinivas Kolukula, PhD *Scientist - B* Indian Tsunami Early Warning Centre (ITEWC) Advisory Services and Satellite Oceanography Group (ASG) Indian National Centre for Ocean Information Services (INCOIS) "Ocean Valley" Pragathi Nagar (B.O) Nizampet (S.O) Hyderabad - 500 090 Telangana, INDIA Office: 040 23886124 *Cell: +91 9381403232; +91 8977801947*

1 0

Re: [Mpi-forum] June MPI Forum Meeting in Aachen
by Dan Holmes 15 May '17

15 May '17

Hi Martin, Please schedule Point-to-point WG time so we can discuss and progress the two active topics: * Allocating receive and freeing send (ARECV and FSEND) - still in design phase * INFO query - lengthy discussion on git issue #63 <https://github.com/mpi-forum/mpi-issues/issues/63> comments - needs confirmation of resolution I would also like to progress issue #63 formally, if possible. It has had a reading at the last meeting. The discussion via git comments seems to have resolved the objections raised at the March meeting, which clears the way for the issue to progress as is, without any text/semantic changes. So, please schedule a 1st vote for issue #63 (pull request #27). https://github.com/mpi-forum/mpi-issues/issues/63 <https://github.com/mpi-forum/mpi-issues/issues/63> https://github.com/mpi-forum/mpi-standard/pull/27 <https://github.com/mpi-forum/mpi-standard/pull/27> Draft standard PDF, as read at previous meeting: https://github.com/mpi-forum/mpi-standard/files/772368/mpi-report.pdf <https://github.com/mpi-forum/mpi-standard/files/772368/mpi-report.pdf> (Replacement text is highlighted, see pages 251, 252. For removed/replaced text, see git diff from pull request.) If the WG time uncovers continued strong objections, then we may decide during the meeting to withdraw/postpone the vote. Cheers, Dan. > On 13 May 2017, at 11:02, Martin Schulz <schulzm(a)llnl.gov> wrote: > > Hi all, > > The June forum meeting is coming closer - June 14-16 in Aachen, Germany. The logistics page for the meeting is at: > > http://mpi-forum.org/meetings/2017/06/logistics <http://mpi-forum.org/meetings/2017/06/logistics> > > Please register as soon as you can. If you are unsure whether you can travel but are hoping to, or if your approval process hasn’t gotten to the point where you can register, please let me know directly. It would be really helpful to get a mostly correct head count as well as an idea on whether we’ll hit quota sooner than later. > > I started a draft of the agenda at: > > http://mpi-forum.org/meetings/2017/06/agenda <http://mpi-forum.org/meetings/2017/06/agenda> > > Currently, I am planning on 1.5 days of WGs followed by 1.5 days of plenaries. Based on demand, though, I may move this around and extend the WGs to two full days, which would also move the voting block to later in the day. > > Also, please note that the two week mark for announcing readings, erratas and ballots is on May 30th - hence, please send me this information before this date. However, also here earlier is better - no need to wait until May 30th! > > Further, WG chairs: please send me all requests for WG times (how long and possible conflicts). > > Finally, the June meeting is traditionally the meeting where we make decisions about the locations for next year. With the meeting being outside the US this time, which typically influences the attendance significantly, I wanted to bring this up ahead of time via email. If you have any suggestions or particular dislikes, or if you are willing to host a forum meeting at your site, please let me know and I will incorporate this in the discussion in Aachen. As for dates - if we stick with the normal rhythm, this would get us to: > Spring meeting: 3/5 to 3/8 > Summer meeting: 6/4-6/7 > Fall meeting: with EuroMPI - trying to get further information > Winter meeting: 12/3-12/6 > If these dates cause any major conflicts, please let me know as well. > > Thanks! > > Martin > > > _______________________________________________ > mpi-forum mailing list > mpi-forum(a)lists.mpi-forum.org > https://lists.mpi-forum.org/mailman/listinfo/mpi-forum The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336.

1 0

June MPI Forum Meeting in Aachen - PLEASE RESPOND TO ACTION ITEMS
by Martin Schulz 13 May '17

13 May '17

Hi all, The June forum meeting is coming closer - June 14-16 in Aachen, Germany. The logistics page for the meeting is at: http://mpi-forum.org/meetings/2017/06/logistics <http://mpi-forum.org/meetings/2017/06/logistics> Please register as soon as you can. If you are unsure whether you can travel but are hoping to, or if your approval process hasn’t gotten to the point where you can register, please let me know directly. It would be really helpful to get a mostly correct head count as well as an idea on whether we’ll hit quota sooner than later. I started a draft of the agenda at: http://mpi-forum.org/meetings/2017/06/agenda <http://mpi-forum.org/meetings/2017/06/agenda> Currently, I am planning on 1.5 days of WGs followed by 1.5 days of plenaries. Based on demand, though, I may move this around and extend the WGs to two full days, which would also move the voting block to later in the day. Also, please note that the two week mark for announcing readings, erratas and ballots is on May 30th - hence, please send me this information before this date. However, also here earlier is better - no need to wait until May 30th! Further, WG chairs: please send me all requests for WG times (how long and possible conflicts). Finally, the June meeting is traditionally the meeting where we make decisions about the locations for next year. With the meeting being outside the US this time, which typically influences the attendance significantly, I wanted to bring this up ahead of time via email. If you have any suggestions or particular dislikes, or if you are willing to host a forum meeting at your site, please let me know and I will incorporate this in the discussion in Aachen. As for dates - if we stick with the normal rhythm, this would get us to: Spring meeting: 3/5 to 3/8 Summer meeting: 6/4-6/7 Fall meeting: with EuroMPI - trying to get further information Winter meeting: 12/3-12/6 If these dates cause any major conflicts, please let me know as well. Thanks! Martin

1 0