February 2009 - mpiwg-fortran - mailman3.cels.anl.gov

Re: [MPI3 Fortran] MPI buffer arguments - void and call-by-reference
by Craig Rasmussen 23 Feb '09

23 Feb '09

Regarding funding for a new "real" Fortran MPI interface. Because of the economic recession, there may be short term funding to do projects of this type. Is anyone interested in gathering together to see if the Office of Science would be interested in funding this effort? Regards, Craig On Feb 23, 2009, at 9:27 AM, Craig Rasmussen wrote: > > On Feb 19, 2009, at 9:00 AM, Bill Long wrote: > >> Aleksandar Donev wrote: >>> Hi Craig, >>> >>> Since I wasn't in Vegas I will read the papers and try to >>> reconstruct what you guys talked about, and in particular, to >>> understand why "TYPE(*), DIMENSION(*)" was not a possibility >>> (excluding scalar actuals if necessary---handle that by a separate >>> procedure if need be). >>> >>>> interface >>>> subroutine MPI_Send(buf, ...) BIND(C) >>>> TYPE(*), DIMENSION(..), INTENT(in) :: buf >>>> ... >>>> end subroutine >>>> end interface >>>> I think using TYPE(*) and DIMENSION(..) is an excellent solution >>>> for MPI 3.0. The advantages are: >>> I am in agreement that this is a good solution. >>> BUT: >>>> 1. Current user code won't have to change beyond using the MPI3 >>>> module. >>> That makes no sense. If the array is passed by descriptor, there >>> is absolutely no need to also pass its size as an argument. Even >>> the type, at least for the set of predefined intrinsic >>> interoperable types, does not need to be passed. >>> >>> Either we keep the interfaces as close to what they are (pass-by- >>> address), or, we refine them. Otherwise you get something that >>> neither satisfies existing codes/implementations nor does it >>> satisfy demanding users (read: me :-) >> >> I would note (for the MPI folks) that discussions at J3 meetings >> suggest that Aleks is not the only one who would like a "modern" >> MPI interface for Fortran. J3 has no official position on this one >> way or the other. We are working to make such an interface >> possible (as part of a larger scope C interoperability TR), but it >> is up to the MPI group to decide what to support. The hard reality >> is that there is a enormous amount of legacy MPI code out there and >> the "don't make me change my code" crowd is a significant fraction >> (most likely the majority) of MPI/Fortran users. So, I think that >> continuing to support the current style of interface is not really >> optional. The question is whether to support an additional >> interface that is more in tune with modern Fortran, and offers >> definite advantages to authors of new codes. I don't think it is >> either/or, but rather one or two. > > This topic has come up frequently at MPI Forum meetings and in > discussions between Jeff Squyres and myself. The answer to the > question is that there will be a low level interface that follows > the current C standard as closely as possible, but provides type > checking where feasible and solves/provide guidance on the > asynchronous problem. > > There is also interest in providing a high level (real Fortran) > interface that takes advantage of the full features in Fortran. > This would be a separate effort and not part of the MPI > standardization effort. I'll still awaiting news whether there will > be funding for this or not. > > Regards, > Craig > > _______________________________________________ > mpi3-fortran mailing list > mpi3-fortran(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-fortran

1 0

Re: [MPI3 Fortran] MPI buffer arguments - void and call-by-reference
by Craig Rasmussen 23 Feb '09

23 Feb '09

On Feb 19, 2009, at 9:00 AM, Bill Long wrote: > Aleksandar Donev wrote: >> Hi Craig, >> >> Since I wasn't in Vegas I will read the papers and try to >> reconstruct what you guys talked about, and in particular, to >> understand why "TYPE(*), DIMENSION(*)" was not a possibility >> (excluding scalar actuals if necessary---handle that by a separate >> procedure if need be). >> >>> interface >>> subroutine MPI_Send(buf, ...) BIND(C) >>> TYPE(*), DIMENSION(..), INTENT(in) :: buf >>> ... >>> end subroutine >>> end interface >>> I think using TYPE(*) and DIMENSION(..) is an excellent solution >>> for MPI 3.0. The advantages are: >> I am in agreement that this is a good solution. >> BUT: >>> 1. Current user code won't have to change beyond using the MPI3 >>> module. >> That makes no sense. If the array is passed by descriptor, there is >> absolutely no need to also pass its size as an argument. Even the >> type, at least for the set of predefined intrinsic interoperable >> types, does not need to be passed. >> >> Either we keep the interfaces as close to what they are (pass-by- >> address), or, we refine them. Otherwise you get something that >> neither satisfies existing codes/implementations nor does it >> satisfy demanding users (read: me :-) > > I would note (for the MPI folks) that discussions at J3 meetings > suggest that Aleks is not the only one who would like a "modern" MPI > interface for Fortran. J3 has no official position on this one way > or the other. We are working to make such an interface possible (as > part of a larger scope C interoperability TR), but it is up to the > MPI group to decide what to support. The hard reality is that there > is a enormous amount of legacy MPI code out there and the "don't > make me change my code" crowd is a significant fraction (most likely > the majority) of MPI/Fortran users. So, I think that continuing to > support the current style of interface is not really optional. The > question is whether to support an additional interface that is more > in tune with modern Fortran, and offers definite advantages to > authors of new codes. I don't think it is either/or, but rather one > or two. This topic has come up frequently at MPI Forum meetings and in discussions between Jeff Squyres and myself. The answer to the question is that there will be a low level interface that follows the current C standard as closely as possible, but provides type checking where feasible and solves/provide guidance on the asynchronous problem. There is also interest in providing a high level (real Fortran) interface that takes advantage of the full features in Fortran. This would be a separate effort and not part of the MPI standardization effort. I'll still awaiting news whether there will be funding for this or not. Regards, Craig

1 0

Re: [MPI3 Fortran] MPI buffer arguments - void and call-by-reference
by Aleksandar Donev 19 Feb '09

19 Feb '09

On Thursday 19 February 2009 08:00, Bill Long wrote: > So, I think that continuing to support the current style of > interface is not really optional. Am I understanding correctly that the official J3 response is what Nick said: > Saying about the existing interface "this > is non-standard, so check with your compiler documentation whether it > works and how to do that" is merely describing the current situation. > Which is livable with, even if not nice. That is, are we telling the MPI Forum: Live with non-standard extensions and the current interface, or upgrade to a better, standard, safer interface that we will provide tools/features for in the upcoming Interop TR? I am perfectly fine with such a response, at least it will settle things down. Thanks, Aleks

1 0

Re: [MPI3 Fortran] MPI buffer arguments - void and call-by-reference
by N.M. Maclaren 19 Feb '09

19 Feb '09

On Feb 19 2009, Bill Long wrote: > >I would note (for the MPI folks) that discussions at J3 meetings suggest >that Aleks is not the only one who would like a "modern" MPI interface >for Fortran. Dead right! And not just in J3. > The hard reality is that there is a enormous amount of >legacy MPI code out there and the "don't make me change my code" crowd >is a significant fraction (most likely the majority) of MPI/Fortran >users. So, I think that continuing to support the current style of >interface is not really optional. The question is whether to support an >additional interface that is more in tune with modern Fortran, and >offers definite advantages to authors of new codes. I don't think it is >either/or, but rather one or two. I agree. And my view is that attempting to do both in a single call is just making life too difficult. Saying about the existing interface "this is non-standard, so check with your compiler documentation whether it works and how to do that" is merely describing the current situation. Which is livable with, even if not nice. Regards, Nick Maclaren, University of Cambridge Computing Service, New Museums Site, Pembroke Street, Cambridge CB2 3QH, England. Email: nmm1(a)cam.ac.uk Tel.: +44 1223 334761 Fax: +44 1223 334679

1 0

Re: [MPI3 Fortran] MPI buffer arguments - void and call-by-reference
by Bill Long 19 Feb '09

19 Feb '09

Aleksandar Donev wrote: > Hi Craig, > > Since I wasn't in Vegas I will read the papers and try to reconstruct > what you guys talked about, and in particular, to understand why > "TYPE(*), DIMENSION(*)" was not a possibility (excluding scalar > actuals if necessary---handle that by a separate procedure if need be). > >> interface >> subroutine MPI_Send(buf, ...) BIND(C) >> TYPE(*), DIMENSION(..), INTENT(in) :: buf >> ... >> end subroutine >> end interface >> I think using TYPE(*) and DIMENSION(..) is an excellent solution for >> MPI 3.0. The advantages are: > I am in agreement that this is a good solution. > BUT: >> 1. Current user code won't have to change beyond using the MPI3 module. > That makes no sense. If the array is passed by descriptor, there is > absolutely no need to also pass its size as an argument. Even the > type, at least for the set of predefined intrinsic interoperable > types, does not need to be passed. > > Either we keep the interfaces as close to what they are > (pass-by-address), or, we refine them. Otherwise you get something > that neither satisfies existing codes/implementations nor does it > satisfy demanding users (read: me :-) I would note (for the MPI folks) that discussions at J3 meetings suggest that Aleks is not the only one who would like a "modern" MPI interface for Fortran. J3 has no official position on this one way or the other. We are working to make such an interface possible (as part of a larger scope C interoperability TR), but it is up to the MPI group to decide what to support. The hard reality is that there is a enormous amount of legacy MPI code out there and the "don't make me change my code" crowd is a significant fraction (most likely the majority) of MPI/Fortran users. So, I think that continuing to support the current style of interface is not really optional. The question is whether to support an additional interface that is more in tune with modern Fortran, and offers definite advantages to authors of new codes. I don't think it is either/or, but rather one or two. Cheers, Bill > > Best, > Aleks > -- Bill Long longb(a)cray.com Fortran Technical Support & voice: 651-605-9024 Bioinformatics Software Development fax: 651-605-9142 Cray Inc., 1340 Mendota Heights Rd., Mendota Heights, MN, 55120

1 0

Re: [MPI3 Fortran] MPI buffer arguments - void and call-by-reference
by Aleksandar Donev 18 Feb '09

18 Feb '09

Hi Craig, Since I wasn't in Vegas I will read the papers and try to reconstruct what you guys talked about, and in particular, to understand why "TYPE(*), DIMENSION(*)" was not a possibility (excluding scalar actuals if necessary---handle that by a separate procedure if need be). > interface > subroutine MPI_Send(buf, ...) BIND(C) > TYPE(*), DIMENSION(..), INTENT(in) :: buf > ... > end subroutine > end interface > I think using TYPE(*) and DIMENSION(..) is an excellent solution for MPI > 3.0. The advantages are: I am in agreement that this is a good solution. BUT: > 1. Current user code won't have to change beyond using the MPI3 module. That makes no sense. If the array is passed by descriptor, there is absolutely no need to also pass its size as an argument. Even the type, at least for the set of predefined intrinsic interoperable types, does not need to be passed. Either we keep the interfaces as close to what they are (pass-by-address), or, we refine them. Otherwise you get something that neither satisfies existing codes/implementations nor does it satisfy demanding users (read: me :-) Best, Aleks -- Aleksandar Donev, Ph.D. Lawrence Postdoctoral Fellow @ LLNL High Performance Computational Materials Science and Chemistry E-mail: donev1(a)llnl.gov Phone: (925) 424-6816 Fax: (925) 423-0785 Address: P.O.Box 808, L-367, Livermore, CA 94551-9900 Web: http://cherrypit.princeton.edu/donev

1 0

Re: [MPI3 Fortran] MPI buffer arguments - void and call-by-reference
by Bill Long 17 Feb '09

17 Feb '09

Jeff Squyres wrote: > On Feb 17, 2009, at 3:08 PM, Craig Rasmussen wrote: > >> This means that actual argument associated with the dummy argument >> buf, can be an array or scalar of any type. If it is an array, it >> can have any rank. The major impact to the MPI implementation is >> that the buf argument will be passed by descriptor. > > Two knee-jerk questions: > > 1. Is there a standard layout / struct / whatever for this > descriptor? Or does it vary by compiler? (I really hope you answer > "yes" to the first question...) There are actually two descriptors. One is the Fortran descriptor; a pointer to this is passed as the actual argument. The other is the C descriptor. This is defined in a C header file and has a standardized set of components. There is also a function with a standardized interface that converts a Fortran descriptor into the equivalent C descriptor. > > 2. Is the descriptor available via C, or only available via Fortran? > (meaning: does the MPI wrapper function *have* to be written in > Fortran? I really hope you answer "yes, it's available in C"...) The C descriptor definition and the conversion function are available only to C. So you would need to write the wrapper routines in C. Some background: Fortran implementations already use descriptors to pass assumed-shape, allocatable, and pointer arrays. The format of these descriptors varies with vendor, and the internal structure of the descriptors is hidden from the user. The goal here is to provide an equivalent C descriptor that is exposed to the C programmer and has a portable set of components and a portable conversion routine. The conversion routine is supplied by the Fortran vendor and knows about the internals of that vendor's Fortran descriptor. Cheers, Bill > >> I think using TYPE(*) and DIMENSION(..) is an excellent solution for >> MPI 3.0. The advantages are: >> >> 1. Current user code won't have to change beyond using the MPI3 module. > > > Remember that we wanted to make Fortran MPI handles just like C > handles, so users will need to change that as well. It won't just be > s/use mpi/use mpi3/g. > -- Bill Long longb(a)cray.com Fortran Technical Support & voice: 651-605-9024 Bioinformatics Software Development fax: 651-605-9142 Cray Inc., 1340 Mendota Heights Rd., Mendota Heights, MN, 55120

1 0

Re: [MPI3 Fortran] MPI buffer arguments - void and call-by-reference
by Jeff Squyres 17 Feb '09

17 Feb '09

On Feb 17, 2009, at 3:08 PM, Craig Rasmussen wrote: > This means that actual argument associated with the dummy argument > buf, can be an array or scalar of any type. If it is an array, it > can have any rank. The major impact to the MPI implementation is > that the buf argument will be passed by descriptor. Two knee-jerk questions: 1. Is there a standard layout / struct / whatever for this descriptor? Or does it vary by compiler? (I really hope you answer "yes" to the first question...) 2. Is the descriptor available via C, or only available via Fortran? (meaning: does the MPI wrapper function *have* to be written in Fortran? I really hope you answer "yes, it's available in C"...) > I think using TYPE(*) and DIMENSION(..) is an excellent solution for > MPI 3.0. The advantages are: > > 1. Current user code won't have to change beyond using the MPI3 > module. Remember that we wanted to make Fortran MPI handles just like C handles, so users will need to change that as well. It won't just be s/use mpi/use mpi3/g. -- Jeff Squyres Cisco Systems

1 0

Re: [MPI3 Fortran] MPI buffer arguments - void and call-by-reference
by N.M. Maclaren 17 Feb '09

17 Feb '09

On Feb 17 2009, Bill Long wrote: > >>This program directly violates C1232. The same program with VOLATILE >replaced buy ASYNCHRONOUS violates the same constraint. Neither is a >valid Fortran subprogram and the compiler is free to not produce any >object code. True. Careless of me. Sorry. >>> In the cases relevant to the MPI discussion, there is essentially no >>> difference between VOLATILE and ASYNCHRONOUS is regards to code >>> motion across a call site. The caller cannot tell if the callee >>> contains a WAIT statement, and even if it could, would be hard >>> pressed to determine if that particular WAIT statement had anything >>> to do with a particular asynchronous variable. Effectively, all >>> motion of code involving a variable with either the volatile or >>> asynchronous attributes across a call site is suppressed by the >>> optimizer. The difference is that a variable can get the >>> ASYNCHRONOUS attribute implicitly, whereas VOLATILE must be >>> explicitly specified. >> >> That's not true. It is true only for external calls (i.e. to procedures >> of properties unknown to the compiler). > >Which is exactly why I started the paragraph with "In the cases relevant >to the MPI discussion...". All of the calls to MPI routines are >external calls. Before laying our flame bait like "That's not true", it >would really help if you actually read the paragraph to which you are >replying. All right. It is factually incorrect. I also need to point out that your second statement is similar. I had both noted your condition, and had read the whole of your posting. What you seem to be assuming is that the attributes are relevant SOLELY to the actual MPI calls; they aren't, as is immediately obvious by the inspection of a few real MPI programs. Consider the following over-simplified example, using classic methodology, and a fairly stupid calculation: SUBROUTINE Fred REAL, ASYNCHRONOUS :: array(2,10,10,64) REAL :: accumulate(10,10) INTEGER :: n, k CALL Bstart(array(:,:,:,1)) ! Starts to transfer in DO n = 1,64 CALL Bstop(array(:,:,:,n)) ! Waits for the transfer IF (N < 64) CALL Bstart(array(:,:,:,n+1)) accumulate = 0.0 joe: DO k = 1,10 accumulate = accumulate+MATMUL(array(1,:,:,n),array(2,:,:,n)) END DO joe END DO END SUBROUTINE Fred The code of loop 'joe' can be optimised just as if the ASYNCHRONOUS were omitted, but that would NOT be true if it were VOLATILE. Note that, if 'array' does NOT have the ASYNCHRONOUS attribute, there is nothing stopping the compiler from using copy-in/copy-out on the calls to Bstart and Bstop. Also note that subroutine Fred does not refer to MPI directly, and the compiler cannot (in general) know whether MPI is being used or whether Fortran asynchronous I/O is. Both show the identical problem. >> Really? Consider a program that transfers into a buffer, does a >> decomposition/FFT/whatever on it, and transfers it out again. A very >> common paradigm. If the compiler can 'see' the source of the code that >> does the decomposition/FFT/whatever, and can tell that there is nothing >> in it that could change the pending status, it can optimise ASYNCHRONOUS >> just like ordinary data. But it can't do that for VOLATILE. > >That analysis is valid as long as ASYNCHRONOUS is limited to Fortran I/O >as is the case currently. Then the compiler "can tell", as you note >(assuming it can 'see' the source). Remove the restriction that >ASYNCHRONOUS is tied to Fortran I/O and the analysis falls apart. A >good argument for not using ASYNCHRONOUS for MPI routines. Again, that statement is factually incorrect. Exactly HOW is a compiler to tell that an external procedure that is passed a buffer 'array' will not (eventually) call Fortran code that either starts Fortran asynchronous I/O on that array or executes a WAIT statement? The simple fact is that a compiler must assume that ANY external procedure (INCLUDING any MPI calls) may start or stop Fortran asynchronous I/O - as I have pointed out before, there is no reason why MPI non-blocking transfers could not be implemented USING Fortran asynchronous I/O! It is therefore impossible for it to make use of the (current) fact that only Fortran asynchronous I/O affects buffers with the ASYNCHRONOUS attribute OVER AN EXTERNAL PROCEDURE CALL. Yes, it is, if no external procedure calls are involved, which also means that MPI isn't involved. So that is irrelevant. Regards, Nick Maclaren, University of Cambridge Computing Service, New Museums Site, Pembroke Street, Cambridge CB2 3QH, England. Email: nmm1(a)cam.ac.uk Tel.: +44 1223 334761 Fax: +44 1223 334679

1 0

Re: [MPI3 Fortran] MPI buffer arguments - void and call-by-reference
by Bill Long 17 Feb '09

17 Feb '09

Craig Rasmussen wrote: > > > We can also discuss MPI_Isend and MPI_Irecv and what to do with them. > I'm assuming that the use of ASYNCHRONOUS plus TARGET will be > preferred over VOLATILE. Since at this point ASYNCHRONOUS really only > pertains to Fortran IO, MPI Forum could request another name for an > equivalent attribute, perhaps one that combines the semantics of > ASYNCHRONOUS and TARGET. > How about naming the attribute BUFFER? I think this would be easy for users to remember, and read well in the source code: real,buffer :: buf(1000) Cheers, Bill -- Bill Long longb(a)cray.com Fortran Technical Support & voice: 651-605-9024 Bioinformatics Software Development fax: 651-605-9142 Cray Inc., 1340 Mendota Heights Rd., Mendota Heights, MN, 55120

1 0