December 2008 - mpiwg-fortran - mailman3.cels.anl.gov

[MPI3 Fortran] Proposed solution for the volatile buffer problem
by Bill Long 30 Dec '08

30 Dec '08

I've thought about the various proposed solutions to the problem of protecting a buffer used for a split send/receive - wait pair of calls. In particular, the need to prevent motion of code involving the buffer across the wait call. Most of them seem to have at least one defect. Here is a possible solution (not entirely original) that I think works, and involves minimal and localized changes in the Fortran spec: Introduce a new subprogram prefix named VOLATILE. A VOLATILE subroutine: 1) must also be BIND(C) 2) may modify any variable accessible in the program unit that calls the subroutine The VOLATILE prefix is not required to match between an interface and the actual definition of the subprogram. Restriction (1) gets around the objection that the concept really makes no sense in the Fortran-Fortran case. Restriction (2) will require that variables be reloaded from memory after a call to a volatile subroutine, and that code motion across the call is effectively blocked. This scheme can be implemented entirely in a module with interfaces for the MPI routines, avoiding user code changes. Vendors understand what "volatile" means for variables, and this usage is a very natural extension. This should be easy to implement. And avoids adding more keywords to Fortran. In practice, the VOLATILE prefix would be added to the interface for the "wait" subroutine. There is some concern about the performance hit from register spills before the CALL, but since you would possibly be waiting for the subroutine to complete anyway, it might not have an noticeable performance hit in actual practice. Cheers, Bill -- Bill Long longb(a)cray.com Fortran Technical Support & voice: 651-605-9024 Bioinformatics Software Development fax: 651-605-9142 Cray Inc., 1340 Mendota Heights Rd., Mendota Heights, MN, 55120

1 0

[MPI3 Fortran] Posting to this list
by Jeff Squyres 23 Dec '08

23 Dec '08

Public service announcement... Please be sure that you post to the list *with exactly the same e-mail address that you subscribed*. That is, if you subscribe "john.doe(a)fun.com", posts form "john.doe(a)host.subdomain.fun.com " will be sent to moderation (and therefore take more time before they are sent on to the list, because a human has to review it). I mention this because it has happened multiple times in the past 2 weeks; sometimes introducing multiple days' worth of delays before people's posts got through to the list. -- Jeff Squyres Cisco Systems

1 0

Re: [MPI3 Fortran] (j3.2006) Effect of volatile
by Michael Ingrassia 22 Dec '08

22 Dec '08

Since Fortran hasn't had END BLOCK statements until now I guess there's no way to survey whether all vendors flush registers and set up barriers to code motion at an END BLOCK. >While this might work as a >practical matter, I don't think the language semantics requires that. Could the semantics require that, in principle ? --Michael I.

1 0

Re: [MPI3 Fortran] Effect of volatile
by Bill Long 22 Dec '08

22 Dec '08

Craig Rasmussen wrote: > At the MPI Forum meeting this week there was a discussion on the > effects of volatile (the subject has come up on the WG5 list as well) > on performance. MPI users are very concerned about performance and > are worried that providing the volatile attribute to an array will > substantially affect performance. However, I think the following > should work with minimal performance implications: > > --------------------- > > buf(1) = 1 > call MPI_Irecv(buf, ...., request,...) > > ! .... do work without buf > > BLOCK > volatile :: buf > call MPI_Wait(request, ...) > END BLOCK > > ! now buf is ready to use > > x = buf(1) > > --------------------- > > I believe that the effects of the volatile statement within the block > is AS IF MPI_Wait were called with buf as a parameter and the dummy > variable was INTENT(INOUT): any registers containing values associated > with buf will be flushed and code motion will be restricted. Volatile > would only be in effect within the block construct and would not have > any performance implications otherwise. At the assignment statement, > x = buf(1), the value assigned to x would have to be retrieved from > memory and could not be obtained from a register. Interesting idea. I'm a bit concerned about the last sentence above. Buf is not (declared to be) volatile at the point of the x = buf(1) statement. Are you suggesting that the volatility specified in the BLOCK "leaks" out to subsequent statements? If so, how far does the leakage extend? To the end of the program unit? If so, you may have lost some of the performance you were hoping to keep. Actually, I would expect that the volatility of buf is only inside the block, and there is no leakage outside the block. While this might work as a practical matter, I don't think the language semantics requires that. The construct also fails to meet the goal of avoiding user source code changes, though I see that as a secondary issue. Cheers, Bill > > I should note that buf can't actually be passed as a parameter to > MPI_Wait as the request handle could be associated with any number of > variables, not just buf. > > Regards, > Craig > > > > > > > > > _______________________________________________ > mpi3-fortran mailing list > mpi3-fortran(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-fortran -- Bill Long longb(a)cray.com Fortran Technical Support & voice: 651-605-9024 Bioinformatics Software Development fax: 651-605-9142 Cray Inc., 1340 Mendota Heights Rd., Mendota Heights, MN, 55120

1 0

Re: [MPI3 Fortran] New information up on the wiki
by Bill Long 19 Dec '08

19 Dec '08

I agree with Steve that any suggestion that Fortran might support varargs should be purged from the MPI spec. Support for a varargs mechanism has not been proposed for the Fortran / C interoperability TR, and I believe that adding such a feature is very unlikely. Cheers, Bill Craig Rasmussen wrote: > As we are meeting next week I thought I put some information out for > comment over the next couple of days. There is a open ticket on > MPI-2-2 regarding the use of C varargs from Fortran. From the ticket: > > The following Fortran errorhandler callback subroutines are prototyped > with varags in the MPI 2.1 spec: > > * COMM_ERRHANDLER_FN > * WIN_ERRHANDLER_FN > * FILE_ERRHANDLER_FN > > Fortran does not have variable length argument lists. This is quite > obviously a typo that dates back to MPI 2.0 but was missed in 2.1. > > The suggestion is to remove the ",..." syntax from the Fortran 2.2 > bindings as it is not part of Fortran. This seems entirely reasonable > to me. > > There is a move afoot to get interoperability with varags in the > Fortran C interoperability standard. This is still up in the air at > the moment and we don't know which way it will go. > > Comments? > > Craig > > _______________________________________________ > mpi3-fortran mailing list > mpi3-fortran(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-fortran -- Bill Long longb(a)cray.com Fortran Technical Support & voice: 651-605-9024 Bioinformatics Software Development fax: 651-605-9142 Cray Inc., 1340 Mendota Heights Rd., Mendota Heights, MN, 55120

1 0

[MPI3 Fortran] Effect of volatile
by Craig Rasmussen 18 Dec '08

18 Dec '08

At the MPI Forum meeting this week there was a discussion on the effects of volatile (the subject has come up on the WG5 list as well) on performance. MPI users are very concerned about performance and are worried that providing the volatile attribute to an array will substantially affect performance. However, I think the following should work with minimal performance implications: --------------------- buf(1) = 1 call MPI_Irecv(buf, ...., request,...) ! .... do work without buf BLOCK volatile :: buf call MPI_Wait(request, ...) END BLOCK ! now buf is ready to use x = buf(1) --------------------- I believe that the effects of the volatile statement within the block is AS IF MPI_Wait were called with buf as a parameter and the dummy variable was INTENT(INOUT): any registers containing values associated with buf will be flushed and code motion will be restricted. Volatile would only be in effect within the block construct and would not have any performance implications otherwise. At the assignment statement, x = buf(1), the value assigned to x would have to be retrieved from memory and could not be obtained from a register. I should note that buf can't actually be passed as a parameter to MPI_Wait as the request handle could be associated with any number of variables, not just buf. Regards, Craig

1 0

Re: [MPI3 Fortran] (j3.2006) (SC22WG5.3823) Please tell me I'm wrong
by Robert Corbett 18 Dec '08

18 Dec '08

Craig Rasmussen wrote: > Without help from the Fortran standard, I don't think there is anyway to > avoid a combinatorial explosion of interfaces (made worse by the F2008 > with 15 dimensions). Consider the simple example code snippet: > > --------- > > subroutine test_recv(message) > real :: message(:,:) > > call MPI_Recv(message, 2, MPI_INTEGER, 1, 1, MPI_COMM_WORLD, > MPI_STATUS_IGNORE) > > end subroutine > > --------- > > For interface declaration: > > subroutine MPI_Recv_wrapper(buf, count, datatype, source, tag, comm, > status, err) > real, dimension(*), intent(out) :: buf > > or: > > subroutine MPI_Recv_wrapper(buf, count, datatype, source, tag, comm, > status, err) > real, dimension(1,1,1,1,1,1,*), intent(out) :: buf > > This give the following error with the Intel compiler: > > fortcom: Error: test_recv_call.f90, line 15: There is no matching > specific subroutine for this generic subroutine call. [MPI_RECV] > call MPI_Recv(message, 2, MPI_INTEGER, dest, 1, MPI_COMM_WORLD, > MPI_STATUS_IGNORE) > --------^ > > On the other hand, if we use the interface: > > subroutine MPI_Recv_wrapper(buf, count, datatype, source, tag, comm, > status, err) > type(C_PTR) :: buf > > There is still a problem as assumed shape arrays are not interoperable. > From gfortran: > > call MPI_Recv(C_LOC(message), 2, MPI_INTEGER, dest, 1, MPI_COMM_WORLD, requ > 1 > Error: Assumed-shape array 'message' at (1) cannot be an argument to the > procedure 'c_loc because it is not C interoperable > > > So there doesn't seem to be anyway to currently do multi-dimensional > interfaces for assumed-shape actuals without a combinatorial explosion > of interfaces. This also seems to be the case for assumed-size actuals > as well. > > Comments? I have given some consideration to the final point. The restriction that C_LOC cannot be applied to an array that is not contiguous seems to be unnecessary for most types on most architectures. I am not sure if it csn work for arrays of type CHARACTER on machines where the smallest addressable unit is larger than a byte, but other than that, I see no need for the restriction. If C_LOC is extended to allow actual arguments that are not contiguous, one further extension is needed to make it useful, namely, a way to access the strides. The intrinsic function SHAPE provides an array of the extents of the array. A function, say C_STRIDE, could provide an array of strides. Given those two extensions, a C function could be written to access the elements of an assumed-shape or deferred-shape array. The form of the prototype for the function might be foo(int rank, void* address, size_t extent[restrict], ptrdiff_t stride[restrict], ...) Bob Corbett

1 0

Re: [MPI3 Fortran] (j3.2006) (SC22WG5.3825) Please tell me I'm wrong
by Craig E. Rasmussen 17 Dec '08

17 Dec '08

> There is no way to have a generic interface and use "sequence > association" argument passing, which is what you are trying to do. That > does indeed require help from WG5. > I was just reading though argument association and came across the sequence association part and assumed this was the relevant text. Thanks, Craig

1 0

Re: [MPI3 Fortran] (j3.2006) (SC22WG5.3823) Please tell me I'm wrong
by Craig E. Rasmussen 17 Dec '08

17 Dec '08

Jim, Thanks for your response. The question was not really how to get around the problem but to ask why the problem exists in the first place. MPI users have been using generic MPI interfaces for years (using F77 style implicit interfaces) so they won't be happy if they have to change their codes in this way. I think I have figured out the reasoning. I can argument associate a 2D array with a 1D dummy specified as real :: dimension(*) :: dummy_array However, generic resolution can't be used to resolve a procedure calling with a 2D actual to a 1D dummy. Is this reasoning correct? Thanks, Craig > Craig > > If your message is a contiguous assumed-shape, then you can solve the > problem without Fortran committee's help by using a conversion function as > follows > > function convertToRank1 (x, n) result (res) > real, target, intent(inout) :: x(n) > integer, intent(in) :: n > > real, pointer :: res(:) > > res => x > end function > > The your subroutine can be rewritten as follows > > subroutine test_recv (message) > real, target :: message(:,:) !<-- assume it is contiguous > > real, pointer :: remappedArray(:) > > remappedArray => convertToRank1(message, size(message)) > > call MP_Recv (remappedArray, 2, MPI_INTEGER, 1, 1, MPI_COMM_WORLD, > MPI_STATUS_IGNORE) > > end subroutine > > > That only solves the problem if the assumed-shape is contiguous. In F03 > we have bounds-remapping for pointers that converts (maps) rank one target > to multi-dimension arrays. I think that can be used if your test_recv() > always expects a rank-one array assumed-shape dummy. You can map that > array into a rank-n array using data pointer assignment to match the > interface of MPI_Recv_wrapper, then you don't have the problems with > generic resolution issues. > > > Cheers, > > Jim Xia > > RL Fortran Compiler Test > IBM Toronto Lab at 8200 Warden Ave, Markham, On, L6G 1C7 > Phone (905) 413-3444 Tie-line 313-3444 > email: jimxia(a)ca.ibm.com > D2/YF7/8200 /MKM > > > > From: > Craig Rasmussen <crasmussen(a)lanl.gov> > To: > MPI-3 Fortran working group <mpi3-fortran(a)lists.mpi-forum.org> > Cc: > WG5 <sc22wg5(a)open-std.org> > Date: > 12/17/2008 10:45 AM > Subject: > (j3.2006) (SC22WG5.3823) Please tell me I'm wrong > > > > Without help from the Fortran standard, I don't think there is anyway to > avoid a combinatorial explosion of interfaces (made worse by the F2008 > with 15 dimensions). Consider the simple example code snippet: > > --------- > > subroutine test_recv(message) > real :: message(:,:) > > call MPI_Recv(message, 2, MPI_INTEGER, 1, 1, MPI_COMM_WORLD, > MPI_STATUS_IGNORE) > > end subroutine > > --------- > > For interface declaration: > > subroutine MPI_Recv_wrapper(buf, count, datatype, source, tag, comm, > status, err) > real, dimension(*), intent(out) :: buf > > or: > > subroutine MPI_Recv_wrapper(buf, count, datatype, source, tag, comm, > status, err) > real, dimension(1,1,1,1,1,1,*), intent(out) :: buf > > This give the following error with the Intel compiler: > > fortcom: Error: test_recv_call.f90, line 15: There is no matching specific > subroutine for this generic subroutine call. [MPI_RECV] > call MPI_Recv(message, 2, MPI_INTEGER, dest, 1, MPI_COMM_WORLD, > MPI_STATUS_IGNORE) > --------^ > > On the other hand, if we use the interface: > > subroutine MPI_Recv_wrapper(buf, count, datatype, source, tag, comm, > status, err) > type(C_PTR) :: buf > > There is still a problem as assumed shape arrays are not interoperable. >>From gfortran: > > call MPI_Recv(C_LOC(message), 2, MPI_INTEGER, dest, 1, MPI_COMM_WORLD, > requ > 1 > Error: Assumed-shape array 'message' at (1) cannot be an argument to the > procedure 'c_loc because it is not C interoperable > > > So there doesn't seem to be anyway to currently do multi-dimensional > interfaces for assumed-shape actuals without a combinatorial explosion of > interfaces. This also seems to be the case for assumed-size actuals as > well. > > Comments? > > Cheers, > Craig > > _______________________________________________ > J3 mailing list > J3(a)j3-fortran.org > http://j3-fortran.org/mailman/listinfo/j3 > > > _______________________________________________ > J3 mailing list > J3(a)j3-fortran.org > http://j3-fortran.org/mailman/listinfo/j3 >

1 0

Re: [MPI3 Fortran] Please tell me I'm wrong
by Aleksandar Donev 17 Dec '08

17 Dec '08

Craig, You should read last-week's WG5 messages concerning the Interop TR (especially mine :-). Reinhold Bader already identified this as a problem. The cause of it is the *generic* interface for MPI_Recv. If you declared the non-generic interface: subroutine MPI_Recv(buf, count, datatype, source, tag, comm, status, err) real, dimension(*), intent(out) ï¿½:: buf then you could do real :: message(:,:) call MPI_Recv(message, 2, MPI_INTEGER, 1, 1, MPI_COMM_WORLD, MPI_STATUS_IGNORE) and it would work as expected. If "message" is not contiguous a copy in/out would occur. There is no way to have a generic interface and use "sequence association" argument passing, which is what you are trying to do. That does indeed require help from WG5. Best, Aleks

1 0