March 2008 - mpiwg-fortran - mailman3.cels.anl.gov

Re: [MPI3 Fortran] MPI-2.1: Fortran 90 bindings
by Craig Rasmussen 10 Mar '08

10 Mar '08

It is my hope that we can get a formal request from the MPI Forum for something like this to support MPI, although it might be better to change the restrictions on CLOC to do this. The topic has come up a couple of times on the Fortran committee and failed for lack of support. Coming from the MPI Forum (rather than just the few of us on the committee who use MPI) should help our cause. (Aleks, you may want to join this email group, I know you have lots of free time :-) Cheers, Craig On Mar 10, 2008, at 8:20 AM, Lionel, Steve wrote: > Harald Klimach wrote: > >> It would be very nice, if Fortran would allow the assumed >> size notation in a generic interface to consume all ranks. >> Which would mean that you'd have to use assumed shape >> arrays if you want to explicitly use different routines >> for different ranks and both options would be mutually >> exclusive. > > I discussed this with our Fortran standards rep last week. Intel and > many other vendors have extensions that allow for this, but we agreed > that the standard should be amended to allow for the option. I > suggested that as part of the C interoperability that a "VOID" > argument > attribute be added which would serve the same purpose it does in C. > > Absent that, Fortran allows you to pass an element of an array of any > rank to an array argument of any rank, such as: > > Call sub(a(1)) > > Fortran specifies that the remaining elements in the actual argument > array are "sequence associated" with the corresponding elements of the > dummy argument. > > Fortran does NOT allow passing a scalar variable or expression to an > array dummy argument. > > Steve Lionel > Intel > > _______________________________________________ > mpi3-fortran mailing list > mpi3-fortran(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-fortran

1 0

Re: [MPI3 Fortran] MPI-2.1: Fortran 90 bindings
by Lionel, Steve 10 Mar '08

10 Mar '08

Harald Klimach wrote: > It would be very nice, if Fortran would allow the assumed > size notation in a generic interface to consume all ranks. > Which would mean that you'd have to use assumed shape > arrays if you want to explicitly use different routines > for different ranks and both options would be mutually > exclusive. I discussed this with our Fortran standards rep last week. Intel and many other vendors have extensions that allow for this, but we agreed that the standard should be amended to allow for the option. I suggested that as part of the C interoperability that a "VOID" argument attribute be added which would serve the same purpose it does in C. Absent that, Fortran allows you to pass an element of an array of any rank to an array argument of any rank, such as: Call sub(a(1)) Fortran specifies that the remaining elements in the actual argument array are "sequence associated" with the corresponding elements of the dummy argument. Fortran does NOT allow passing a scalar variable or expression to an array dummy argument. Steve Lionel Intel

1 0

Re: [MPI3 Fortran] MPI-2.1: Fortran 90 bindings
by Craig Rasmussen 10 Mar '08

10 Mar '08

On Mar 9, 2008, at 4:33 PM, Jeff Squyres wrote: >> In MPI-2 Page 13 Line 33 it says: >> All MPI names have an MPI_ prefix, and all characters are >> capitals. >> However this statement "and all characters are capitals" >> hasn't any meaning in the Fortran context, right? > > > My understanding (and I'll repeat again: I'm nowhere near a fortran > expert) is that fortran compilers will change subroutine/function/ > variable symbols to either all upper case or all lower case (possibly > with some additional underscores). I don't know if this is a Fortran > standard thing (e.g., if the Fortran standard says that symbols are > case-insensitive), or whether it's an implementation thing. Fortran identifiers are not case sensitive. However you can give a name to a BIND(C) declaration so that the symbol generated is case sensitive and follows the companion C compilers convention on symbol generation. For example, BIND(C, name="MPI_Comm_rank") generated the symbol, _MPI_Comm_rank Cheers, Craig

1 0

Re: [MPI3 Fortran] MPI-2.1: Fortran 90 bindings
by Jeff Squyres 09 Mar '08

09 Mar '08

On Mar 8, 2008, at 2:03 AM, Harald Klimach wrote: >> It is required that the send and receive buffers have >> compatible type >> maps. See MPI-1:4.1: Note that this is the only requirement -- compatible type *maps*. Not compatible *types*. > So this is about derived types? I don't think it is > possible to catch derived types within the scope of F90, > you have to fall back to the implicit F77 interface for > this anyway. > Even so, as you said compatible type maps are required, > which means, as far as I understand it, that there are the > same base types on each side. > Looking at MPI-1: 3.3: "Data type matching and data > conversion", I even think it is required that the two > arguments are of the same type. Not really. You could send N COMPLEX items and receive N*2 REAL items. You can get more creative with C structs and Fortran derived types, with the already-mentioned restriction that you might (* see below) have to fall back to the F77 bindings for Fortran derived types. > Observations so far: > It would be very nice, if Fortran would allow the assumed > size notation in a generic interface to consume all ranks. It was pointed out on the call on Friday that most F90 compilers have some kind of directive that effectively allows a Fortran subroutine/ function parameter to be the equivalent of C's (void*), thereby making much of this conversation moot. We were going to investigate this a bit more deeply; this option was not [widely] available when MPI-2 was written (nor when the Rasmussen/Squyres paper was written a few years ago). > Which would mean that you'd have to use assumed shape > arrays if you want to explicitly use different routines > for different ranks and both options would be mutually > exclusive. > > I think the MPI standard really prohibits different types > in the case where there are 2 choice arguments. Per above, I disagree. :-) > I can't find anywhere in the standard where it is required > that arrays of all ranks are allowed to be passed as > actual arguments. But I can't find where it states that > scalars are allowed either and in MPI-2 it is explicitly > stated that this is an inconsistency of MPI with F90. > Both things (scalars and arrays of different ranks) are > rather a observed behaviour of the F77 binding, which > comes from the implicit interfaces. > Maybe the F90 binding shouldn't exactly rebuild the > behaviour of the F77 binding? This is a point that was explicitly brought up when designing the C++ bindings (that perhaps the C++ bindings should not be effectively a 1- to-1 mapping of the C bindings). The Forum overwhelmingly decided to keep them simple and pretty much the same as the C bindings (they do take advantage of a small number of C++ language features, to be more natural to the C++ developer community). The rationale is that if you design an alternate set of bindings, it is effectively a new and different standard that requires its own, separate maintenance, and require a potentially substantially different implementation (than the C bindings). > Having to use only arrays of rank 1 certainly is quite > inconvinient for the user, but I don't see where it is > required by the standard to support all ranks in the F90 > binding. Open MPI defaults to supporting scalar plus ranks 1-4; you can change this with a configure argument when building Open MPI. We did this because we noticed that in some compilers, supporting 5-7 ranks made compilation of F90 MPI apps "slow" (and users complained). We found "4" to be a "sweet spot". But we had to offer the *possibility* of all 7 for anyone who wanted it. > To allow scalars and all ranks for all types I think in > total there are 8 * 15 * 75 = 9000 subroutines needed. > > If you restrict yourself to the types, that are at least > required by MPI (MPI-1: 3.2.2) and don't support any other > ranks besides rank 1, you're down at: 6*75 = 825 > subroutines. Based on my disagreements above, I think the above formulae are incorrect. > In MPI-2 Page 13 Line 33 it says: > All MPI names have an MPI_ prefix, and all characters are > capitals. > However this statement "and all characters are capitals" > hasn't any meaning in the Fortran context, right? My understanding (and I'll repeat again: I'm nowhere near a fortran expert) is that fortran compilers will change subroutine/function/ variable symbols to either all upper case or all lower case (possibly with some additional underscores). I don't know if this is a Fortran standard thing (e.g., if the Fortran standard says that symbols are case-insensitive), or whether it's an implementation thing. -- Jeff Squyres Cisco Systems

1 0

Re: [MPI3 Fortran] MPI-2.1: Fortran 90 bindings
by Harald Klimach 08 Mar '08

08 Mar '08

Hi, > It is required that the send and receive buffers have >compatible type > maps. See MPI-1:4.1: > So this is about derived types? I don't think it is possible to catch derived types within the scope of F90, you have to fall back to the implicit F77 interface for this anyway. Even so, as you said compatible type maps are required, which means, as far as I understand it, that there are the same base types on each side. Looking at MPI-1: 3.3: "Data type matching and data conversion", I even think it is required that the two arguments are of the same type. Observations so far: It would be very nice, if Fortran would allow the assumed size notation in a generic interface to consume all ranks. Which would mean that you'd have to use assumed shape arrays if you want to explicitly use different routines for different ranks and both options would be mutually exclusive. I think the MPI standard really prohibits different types in the case where there are 2 choice arguments. I can't find anywhere in the standard where it is required that arrays of all ranks are allowed to be passed as actual arguments. But I can't find where it states that scalars are allowed either and in MPI-2 it is explicitly stated that this is an inconsistency of MPI with F90. Both things (scalars and arrays of different ranks) are rather a observed behaviour of the F77 binding, which comes from the implicit interfaces. Maybe the F90 binding shouldn't exactly rebuild the behaviour of the F77 binding? Having to use only arrays of rank 1 certainly is quite inconvinient for the user, but I don't see where it is required by the standard to support all ranks in the F90 binding. To allow scalars and all ranks for all types I think in total there are 8 * 15 * 75 = 9000 subroutines needed. If you restrict yourself to the types, that are at least required by MPI (MPI-1: 3.2.2) and don't support any other ranks besides rank 1, you're down at: 6*75 = 825 subroutines. In MPI-2 Page 13 Line 33 it says: All MPI names have an MPI_ prefix, and all characters are capitals. However this statement "and all characters are capitals" hasn't any meaning in the Fortran context, right? Best regards, Harald

1 0

Re: [MPI3 Fortran] MPI-2.1: Fortran 90 bindings
by Jeff Squyres 07 Mar '08

07 Mar '08

On Mar 7, 2008, at 7:01 AM, Harald Klimach wrote: >> But the type is variable, right? Hence, if you use an explicit size, >> the *real* size of the buffer (in bytes) is sizeof(type)*size. The >> "size" parameter is supposed to be specified in bytes, not increments >> of type size. So I don't think the above binding would yield correct >> results for non-character types. Right? > Hm, ok, perhaps this was a bad example, I never used MPI_BUFFER_ATTACH > and just picked it because it has only two arguments ;) > MPI_SEND would have been more appropiate. I think that would work for scalars (heck, you don't even need the count for a scalar) and for 1D arrays. But it seems problematic for 2D and above. >> Actually, MPI explicitly allows you to mix-n-match the types. :-\ > Could you please point me to, where this is stated? It is required that the send and receive buffers have compatible type maps. See MPI-1:4.1: "The type-matching conditions for the collective operations are more strict than the corresponding conditions between sender and receiver in point-to-point. Namely, for collective operations, the amount of data sent must exactly match the amount of data specified by the receiver. Distinct type maps (the layout in memory, see Sec. Derived datatypes ) between sender and receiver are still allowed." > I can't find it. > All I see is mentioning of that it should be allowed to have different > types on separate calls. Which is also true for functions with one > choice argument. Of course this may be due to my humble understanding > of MPI. > Anyway this may be a detail which needs clarification? -- Jeff Squyres Cisco Systems

1 0

Re: [MPI3 Fortran] MPI-2.1: Fortran 90 bindings
by Harald Klimach 07 Mar '08

07 Mar '08

OK, this was my misunderstanding. The generic interface is the problem, Fortran distinguishes, which Routine to actually use by type, kind and rank. I wrongly assumed that this rank distinguishing isn't in charge for assumed size array, as it is for the non generic interface. Obviously this is not the case, and if you use generic interfaces you have to get along with this. > This is why we have all the dimensions explicitly specified in Open MPI. > Yes, sorry for any confusion I may have introduced. > Either way, the back-end C function gets a pointer. So I think we > were likely a) going above and beyond the spec, and b) fitting > MPI_BUFFER_ATTACH into the same OMPI F90 infrastructure as everything > else. Yes, that was my point, that it may be beyond the standard. If you have to take into account all dimensions separately anyway, this doesn't matter anyway. So my point is void, I think. It is very convenient if you can hand over scalars. > But the type is variable, right? Hence, if you use an explicit size, > the *real* size of the buffer (in bytes) is sizeof(type)*size. The > "size" parameter is supposed to be specified in bytes, not increments > of type size. So I don't think the above binding would yield correct > results for non-character types. Right? Hm, ok, perhaps this was a bad example, I never used MPI_BUFFER_ATTACH and just picked it because it has only two arguments ;) MPI_SEND would have been more appropiate. > Actually, MPI explicitly allows you to mix-n-match the types. :-\ Could you please point me to, where this is stated? I can't find it. All I see is mentioning of that it should be allowed to have different types on separate calls. Which is also true for functions with one choice argument. Of course this may be due to my humble understanding of MPI. Anyway this may be a detail which needs clarification? > Certainly, there are cases that are non-sensical (e.g., sending its > and receiving floats), but you *could* send complexes and receive > twice as many floats, and that would be fine. > -- Dipl.-Ing. Harald Klimach email: klimach(a)hlrs.de HLRS, Uni Stuttgart http://www.hlrs.de/people/klimach Nobelstraße 19 phone: 0711-685 60300 70569 Stuttgart fax: 0711-685 65832

1 0

Re: [MPI3 Fortran] MPI-2.1: Fortran 90 bindings
by Jeff Squyres 06 Mar '08

06 Mar '08

On Mar 6, 2008, at 6:20 PM, Craig Rasmussen wrote: > I should warn everyone that the problem becomes worse for Fortran > 2008 because the number of possible dimensions has been increased to > 15. Sweet. :-) > By the way, I've been officially appointed by the Fortran J3 > standards committee to liaison with the MPI committee (if the MPI > Forum will accept me :-) Now would be a very good time to decide > what should be in new Fortran bindings and if they still don't seem > to work well, you can instruct me to ask the Fortran committee to > make changes to the proposed F2008 standard. I have one suggestion > already (but I want to keep you all in suspense until next week when > I can explain it). ...or the teleconference tomorrow. :-) -- Jeff Squyres Cisco Systems

1 0

Re: [MPI3 Fortran] MPI-2.1: Fortran 90 bindings
by Jeff Squyres 06 Mar '08

06 Mar '08

On Mar 6, 2008, at 4:33 PM, Harald Klimach wrote: > OK, sorry I should have done this earlier! > I just had a look into openmpi, and how it's done there, and found, > that there already assumed size is used. w00t. I wouldn't have know that that was what it was called. :-) > Looking at the standard I found it is also everywhere talking of > assumed > size only. > For example in openmpi-1.2.5/ompi/mpi/f90/mpi_buffer_attach_f90.f90.sh > a construct like this is used: > for rank in $allranks > do > case "$rank" in 0) dim='' ; esac > case "$rank" in 1) dim=', dimension(*)' ; esac > case "$rank" in 2) dim=', dimension(1,*)' ; esac > case "$rank" in 3) dim=', dimension(1,1,*)' ; esac > case "$rank" in 4) dim=', dimension(1,1,1,*)' ; esac > case "$rank" in 5) dim=', dimension(1,1,1,1,*)' ; esac > case "$rank" in 6) dim=', dimension(1,1,1,1,1,*)' ; esac > case "$rank" in 7) dim=', dimension(1,1,1,1,1,1,*)' ; esac > > The only 2 meaningful options are the first two. $rank = 0 provides > the > possibility to provide the routine with a scalar. $rank = 1 enables > the > routine to take an array of any rank. > The rest is just not needed. At least to my understanding. It did not work for us to *only* provide (*); we found that we had to list all the dimensions. Here's a relatively simple example that shows the problem with both gfortran and ifort on my system: ------------------ [17:23] svbu-mpi:~/tmp/f90 % cat module.f90 module my_function_module interface my_function subroutine my_function_scalar(arg) integer, intent(in) :: arg end subroutine my_function_scalar subroutine my_function_1d(arg) integer, dimension(*), intent(in) :: arg end subroutine my_function_1d end interface my_function end module my_function_module [17:23] svbu-mpi:~/tmp/f90 % gfortran -c module.f90 [17:23] svbu-mpi:~/tmp/f90 % cat arrays.f90 !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! ! Subroutines !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! subroutine my_function_scalar(arg) implicit none integer, intent(in) :: arg print *, "This is the scalar my_function" end subroutine my_function_scalar subroutine my_function_1d(arg) implicit none integer, dimension(*), intent(in) :: arg print *, "This is the 1d my_function" end subroutine my_function_1d !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! ! All good. Now try to use them !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! program f90_is_confusing use my_function_module implicit none integer :: a integer, dimension(1) :: b integer, dimension(2,3) :: c call my_function(a) call my_function(b) call my_function(c) end program f90_is_confusing [17:23] svbu-mpi:~/tmp/f90 % gfortran arrays.f90 -I. In file arrays.f90:28 call my_function(c) 1 Error: Generic subroutine 'my_function' at (1) is not an intrinsic subroutine [17:23] svbu-mpi:~/tmp/f90 % ------------------ So it didn't allow calling my_function with a 2D array (c). Just to make sure this is not a gfortran weird-ism, I tried compiling the same stuff with ifort: ------------------ [17:23] svbu-mpi:~/tmp/f90 % ifort --version ifort (IFORT) 10.1 20071116 Copyright (C) 1985-2007 Intel Corporation. All rights reserved. [17:23] svbu-mpi:~/tmp/f90 % ifort module.f90 -c [17:23] svbu-mpi:~/tmp/f90 % ifort arrays.f90 -I. fortcom: Error: arrays.f90, line 28: There is no matching specific subroutine for this generic subroutine call. [MY_FUNCTION] call my_function(c) -------^ compilation aborted for arrays.f90 (code 1) [17:23] svbu-mpi:~/tmp/f90 % ------------------- This is why we have all the dimensions explicitly specified in Open MPI. > I would even doubt, that the first option is really necessary, as the > standard states buffer for MPI_BUFFER_ATTACH to be: > <type> BUFFER(*) > To my reading that would mean it has to be an array. From the options I think we were just covering all possible bases with MPI_BUFFER_ATTACH -- you *could* specify a buffer of size 1. But s/ MPI_BUFFER_ATTACH/MPI_SEND/ and the issue is the same (i.e., it does make sense to pass a scalar to MPI_SEND). > above $rank = 1 would then be the only one, that is really needed. The > MPI-2 standard is talking about this issue on page 285 in the > paragraph > "Problems Due to Strong Typing". As I read it, it is expected to only > use the array. The user can always just use an array of size 1. > However it is very convenient to just put the scalar in the argument > list ;) Either way, the back-end C function gets a pointer. So I think we were likely a) going above and beyond the spec, and b) fitting MPI_BUFFER_ATTACH into the same OMPI F90 infrastructure as everything else. > So it certainly is a nice thing to have the additional interface (but > requires to double the work by the implementor). > > Actually you could even an explicit shape array: > subroutine MPI_BUFFER_ATTACH(buffer, size, ierror) > integer :: size, iError > integer :: buffer(size) > end subroutine MPI_BUFFER_ATTACH > > and call it with any integer array, that fits into size. Just try it > with the sample program I sent in my first mail ;). This would ensure > that the actual argument for buffer isn't larger than size. But the type is variable, right? Hence, if you use an explicit size, the *real* size of the buffer (in bytes) is sizeof(type)*size. The "size" parameter is supposed to be specified in bytes, not increments of type size. So I don't think the above binding would yield correct results for non-character types. Right? > Regarding the functions with two choice parameters: > Is there any function which actually would make it necessary to allow > two different types? > By a brief glance at the standard, I only found constructs like in > MPI_Allreduce: > > MPI_ALLREDUCE(sendbuf, recvbuf, count, datatype, op, comm, ierror) > <type> sendbuf(*), recvbuf(*) > integer count, datatype, op, comm, ierror > > To my understanding this means, that sendbuf and recvbuf have to of > the > same type. It won't make any sense in the programs logic otherwise > anyway. Actually, MPI explicitly allows you to mix-n-match the types. :-\ > If they are really allowed to be of different types, I would expect > them to be on two lines: > <type> sendbuf(*) > <type> recvbuf(*) > But perhaps that's just a bad interpretation by me. > > No Fortran program should want to get different typed values then it > sent. > > Are there really any MPI functions, where it is sensible to use two > different types for the two choice parameters? It's unfortunately not for us to say; the standard allows it, so we have to allow it (because *some* app out there is likely using it in a weird-o way that is valid for their situation). Certainly, there are cases that are non-sensical (e.g., sending its and receiving floats), but you *could* send complexes and receive twice as many floats, and that would be fine. -- Jeff Squyres Cisco Systems

1 0

Re: [MPI3 Fortran] MPI-2.1: Fortran 90 bindings
by Craig Rasmussen 06 Mar '08

06 Mar '08

I should warn everyone that the problem becomes worse for Fortran 2008 because the number of possible dimensions has been increased to 15. By the way, I've been officially appointed by the Fortran J3 standards committee to liaison with the MPI committee (if the MPI Forum will accept me :-) Now would be a very good time to decide what should be in new Fortran bindings and if they still don't seem to work well, you can instruct me to ask the Fortran committee to make changes to the proposed F2008 standard. I have one suggestion already (but I want to keep you all in suspense until next week when I can explain it). Cheers, Craig On Mar 5, 2008, at 7:15 PM, Jeff Squyres wrote: > The current MPI-2 F90 bindings are un-implementable because they > require almost 7 *million* interface functions: with 15 intrinsic > Fortran types, each with 7 possible dimensions, the current F90 > interface *requires*: > > - 50 MPI functions with one choice buffer: 15 * 7 * 50 = 5,250 > functions > - 25 MPI functions with two choice buffers: (15 * 7 * 25)^2 = 6.8M > functions > - ...and a few hundred more MPI functions with no choice buffers > > This is clearly broken.

1 0