March 2008 - mpiwg-fortran - mailman3.cels.anl.gov

Re: [MPI3 Fortran] Proposing changes to Fortran 2008
by Aleksandar Donev 31 Mar '08

31 Mar '08

On Monday 31 March 2008 12:58, Hubert Ritzdorf wrote: > Nevertheless, if the compiler is not sure that it can perform this > optimization the compiler inserts a copy-in/copy-out. Yes, and this happens only in cases where the array in question is "non Fortran 77", for example, it is a section with a stride, it is an assumed-shape array, a pointer array, or some such. In such cases copy in/out could be required. I hear people complaining about Fortran 90 having introduced unnecessary copy in/out. I cannot see how this is true unless you are actually using a feature that was new in Fortran 90. > Such a compiler option turns off copy-in/copy-out for that source > file to be compiled. It works such as an optimization flag. An optimization never affects correctness. What if copy in/out is necessary at run-time, i.e., the actual argument is not contiguous? Does this flag you mention cause those programs to abort with a run-time error? Or they crash mysteriously? > but this doesn't > help the application programmer. He will look for a "portable" solution > and transfer > the first array element (I know, it's not guaranteed but it works). I am trying to understand is what you are actually proposing (desiring). You want a way to specify in an interface block: The actual argument must be contiguous and no copy in/out shall be performed? Should the contiguity check be compile time or run time? Note that this has nothing to do with the TKR-matching business we've been talking about. The issue of copying is separate. Best, Aleks

1 0

Re: [MPI3 Fortran] Proposing changes to Fortran 2008
by Hubert Ritzdorf 31 Mar '08

31 Mar '08

Aleksandar Donev wrote: > >> And I have also seen compiler switches for "don't perform copy-in/copy >> out". >> > Can you give an example please and explain what it does exactly? > When I see it correctly, avoiding copy-in/copy-out is somewhat like optimization for a Fortran compiler. In the beginning of the Fortran 90 compiler developments, many compilers performed a copy-in/copy-out at places where it really was not necessary. With the improvement of the compilers, the number of copy-in/copy-out operations decreased. Nevertheless, if the compiler is not sure that it can perform this optimization the compiler inserts a copy-in/copy-out. Such a compiler option turns off copy-in/copy-out for that source file to be compiled. It works such as an optimization flag. > >> And I see >> Fortran codes transferring scalar arguments such as "my_array (1,1)" >> (nevertheless many >> data items are transferred) in order to avoid copy in/copy out. >> > Why not just "my_array"? If this is trying to get around the rank-matching > requirement than using my_array(1,1) is in fact perfectly legal and OK---you > are relying on sequence association which is exactly what passing the first > element of an array does. But in that case your assertion that they do this > to avoid copy in/out is wrong---they do it to pass a rank-2 array to a rank-1 > buffer dummy. > > Formally, nothing in the standard prevents copy in/out when relying on > sequence association as in the example above. It is just that no compiler in > their sane mind would perform copying when that was completely unnecessary: > The actual argument is already contiguous so why copy it? > > Since the application programmer cannot be "sure" that copy-in/copy out is not performed. If you specify my_array(1,1), the compiler will transfer the address of my_array(1,1). Otherwise, the compiler could decide to perform a copy-in/copy-out (see above, since the compiler is not sure that it is completely unnecessary). I have for example seen that a compiler decided to make a temporary vector for an array-section of a 1-dimensional array: the MPI_Irecv received incorrect data, the application code failed. From the Fortran 90 compilers view, it was correct code (possibly not most efficient), but this doesn't help the application programmer. He will look for a "portable" solution and transfer the first array element (I know, it's not guaranteed but it works). Best regards Hubert

1 0

Re: [MPI3 Fortran] Proposing changes to Fortran 2008
by Supalov, Alexander 27 Mar '08

27 Mar '08

PPS. John apparently retired from the Rutherford Labs but remained active. Please pass my best wishes to him next time you meet. -----Original Message----- From: mpi3-fortran-bounces(a)lists.mpi-forum.org [mailto:[email protected]] On Behalf Of Aleksandar Donev Sent: Thursday, March 27, 2008 9:31 PM To: MPI-3 Fortran working group Subject: Re: [MPI3 Fortran] Proposing changes to Fortran 2008 Hi, MPI_ISend has many more issues than MPI_Send. Which one are we discussing? For MPI_Send, I see no problem with copy in/out, in fact, I see it as desirable because the compiler does the work of copying things like columns of arrays into contig buffers. Is there a problem with copy in/out and the blocking calls? > The C_LOC functionality is not far away from this requirement C_LOC exactly satisfies the requirement of no copy in/out and probably should be used for MPI_ISend. It requires TARGET, which should in fact be required if you are retaining a pointer to an array (even if this pointer is only in C and not visible to the Fortran compiler). > the ASYNCHRONOUS attribute also avoids any copying of the actual argument. In the case of MPI_ISend even with C_LOC, ASYNCHRONOUS (actually VOLATILE, to be exact) is necessary on the array whose values are being modified behind the compiler's back. This is independent of the issue of copying. Best, Aleks P.S. John Reid is still the head of the Fortran committee -- Aleksandar Donev, Ph.D. Lawrence Postdoctoral Fellow @ Lawrence Livermore National Laboratory High Performance Computational Materials Science and Chemistry E-mail: donev1(a)llnl.gov Phone: (925) 424-6816 Fax: (925) 423-0785 Address: P.O.Box 808, L-367, Livermore, CA 94551-9900 Web: http://cherrypit.princeton.edu/donev _______________________________________________ mpi3-fortran mailing list mpi3-fortran(a)lists.mpi-forum.org http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-fortran --------------------------------------------------------------------- Intel GmbH Dornacher Strasse 1 85622 Feldkirchen/Muenchen Germany Sitz der Gesellschaft: Feldkirchen bei Muenchen Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer Registergericht: Muenchen HRB 47456 Ust.-IdNr. VAT Registration No.: DE129385895 Citibank Frankfurt (BLZ 502 109 00) 600119052 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.

1 0

Re: [MPI3 Fortran] Proposing changes to Fortran 2008
by Aleksandar Donev 27 Mar '08

27 Mar '08

On Thursday 27 March 2008 13:49, Hubert Ritzdorf wrote: > Yes, if derived MPI datatypes are used. C_LOC should be used for MPI_Get_address to solve the issue you described. Then the offsets are correct and even if there is copy in/out the base is right and the offsets would be right so there should be no problem. > The requirement of target attribute is not really acceptable from the > MPI standard; for example I would > expect that that MPI_Isend (C_LOC(buf(1000:2000)), ...) would not be > allowed. You can do MPI_Isend (C_LOC(buf(1000)), ...) if you'd like...the C standard guarantees that the address of buf(1000:2000) and buf(1000) are the same. C_LOC(buf(1000:2000)) is not allowed in Fortran 2003, but it is in Fortran 2008. We formalized the concept of "simply-contiguous array" in Fortran 2008 and your buf(1000:2000) is such an array and allowed in C_LOC (I was the one that proposed that, actually). It would be easy for compiler vendors to add this to compilers even without implementing anything else in Fortran 2003. Best, Aleks

1 0

Re: [MPI3 Fortran] Proposing changes to Fortran 2008
by Hubert Ritzdorf 27 Mar '08

27 Mar '08

Aleksandar Donev wrote: > Hi, > > MPI_ISend has many more issues than MPI_Send. Which one are we discussing? > > For MPI_Send, I see no problem with copy in/out, in fact, I see it as > desirable because the compiler does the work of copying things like columns > of arrays into contig buffers. Is there a problem with copy in/out and the > blocking calls? > Yes, if derived MPI datatypes are used. > >> The C_LOC functionality is not far away from this requirement >> > C_LOC exactly satisfies the requirement of no copy in/out and probably should > be used for MPI_ISend. It requires TARGET, which should in fact be required > if you are retaining a pointer to an array (even if this pointer is only in C > and not visible to the Fortran compiler). > The requirement of target attribute is not really acceptable from the MPI standard; for example I would expect that that MPI_Isend (C_LOC(buf(1000:2000)), ...) would not be allowed. Best regards Hubert

1 0

Re: [MPI3 Fortran] Proposing changes to Fortran 2008
by Aleksandar Donev 27 Mar '08

27 Mar '08

On Thursday 27 March 2008 13:03, Hubert Ritzdorf wrote: > I don't believe that the compiler guys have introduced such compiler > switches only for MPI > since also other libraries have such problems. Of course. I use C libraries all the time and I care a lot about finding a good solution to the (multiple and somewhat distinct) issues that MPI birngs out. In fact, to be honest, I don't per se care for MPI itself, rather, generically the interfacing with such libraries written in other languages. > And I have also seen compiler switches for "don't perform copy-in/copy > out". Can you give an example please and explain what it does exactly? > And I see > Fortran codes transferring scalar arguments such as "my_array (1,1)" > (nevertheless many > data items are transferred) ï¿½in order to avoid ï¿½copy in/copy out. Why not just "my_array"? If this is trying to get around the rank-matching requirement than using my_array(1,1) is in fact perfectly legal and OK---you are relying on sequence association which is exactly what passing the first element of an array does. But in that case your assertion that they do this to avoid copy in/out is wrong---they do it to pass a rank-2 array to a rank-1 buffer dummy. Formally, nothing in the standard prevents copy in/out when relying on sequence association as in the example above. It is just that no compiler in their sane mind would perform copying when that was completely unnecessary: The actual argument is already contiguous so why copy it? Best, Aleks

1 0

Re: [MPI3 Fortran] Proposing changes to Fortran 2008
by Aleksandar Donev 27 Mar '08

27 Mar '08

Hi, MPI_ISend has many more issues than MPI_Send. Which one are we discussing? For MPI_Send, I see no problem with copy in/out, in fact, I see it as desirable because the compiler does the work of copying things like columns of arrays into contig buffers. Is there a problem with copy in/out and the blocking calls? > The C_LOC functionality is not far away from this requirement C_LOC exactly satisfies the requirement of no copy in/out and probably should be used for MPI_ISend. It requires TARGET, which should in fact be required if you are retaining a pointer to an array (even if this pointer is only in C and not visible to the Fortran compiler). > the ASYNCHRONOUS attribute also avoids any copying of the actual argument. In the case of MPI_ISend even with C_LOC, ASYNCHRONOUS (actually VOLATILE, to be exact) is necessary on the array whose values are being modified behind the compiler's back. This is independent of the issue of copying. Best, Aleks P.S. John Reid is still the head of the Fortran committee -- Aleksandar Donev, Ph.D. Lawrence Postdoctoral Fellow @ Lawrence Livermore National Laboratory High Performance Computational Materials Science and Chemistry E-mail: donev1(a)llnl.gov Phone: (925) 424-6816 Fax: (925) 423-0785 Address: P.O.Box 808, L-367, Livermore, CA 94551-9900 Web: http://cherrypit.princeton.edu/donev

1 0

Re: [MPI3 Fortran] Proposing changes to Fortran 2008
by Supalov, Alexander 27 Mar '08

27 Mar '08

Hi, I'd side with Hubert. Standard or not, copy in/out is a real problem. If you're looking for someone to blame, John Reid and your sincerely proposed those switches back in (oh) 1996 to fix a problem in an interface that was relying on MPI inside but did not expose MPI features to the application programmers. John ran most of that show and told me that the proposal was enthusiastically picked up by compiler vendors. I believe John is still around, although retired, so you can ask him for more details. Best regards. Alexander -----Original Message----- From: mpi3-fortran-bounces(a)lists.mpi-forum.org [mailto:[email protected]] On Behalf Of Hubert Ritzdorf Sent: Thursday, March 27, 2008 9:03 PM To: MPI-3 Fortran working group Subject: Re: [MPI3 Fortran] Proposing changes to Fortran 2008 Aleksandar Donev wrote: > On Thursday 27 March 2008 07:53, Hubert Ritzdorf wrote: > > >> This is what some vendors already implemented (using different names). >> When I remember (and understood) >> it correctly, you mentioned that they didn't it right. >> > They have not integrated it with the rest of the standard. You can do many > things that apparently work fine until you try them with polymorphic > assumed-shape arrays of a parameterized derived type (yes, Fortran has > those). A standard has to be internally fully self-consistent...it cannot > just say "oh, no one will do that, just make it work". > > >> Since the >> behaviour should be well-defined if an >> interface description is not available and the vendors have to implement >> it, and it should be possible >> to specify it for a single argument. >> > The existing standard says the following about implicit interfaces. A > routine's interface can be omitted, however, the interface exists and there > is a *unique* interface (modulo some attributes that can be omitted). That > is, an MPI_Send routine, which exists somewhere, according to the Fortran > standard, has to either accept an integer buffer or a real buffer. It cannot > accept both. There is a *unique* interface. If the interface is implicit, it > is up to the user to ensure the actual arguments actually match the type/rank > etc. of the dummy. If it is explicit, the compiler helps check it. That is > the ONLY difference between implicit and explicit interfaces. > > Now, compilers often allow users to violate the standard and call MPI_Send > with both a real and an integer buffer array, and they either do not do any > analysis to detect it, or if they do, they offer a compiler switch to > say "close your eyes please". It is however, not conforming to the Fortran > standard, period. > I don't believe that the compiler guys have introduced such compiler switches only for MPI since also other libraries have such problems. And I have also seen compiler switches for "don't perform copy-in/copy out". And I see Fortran codes transferring scalar arguments such as "my_array (1,1)" (nevertheless many data items are transferred) in order to avoid copy in/copy out. > >> There are 2 general problems for MPI with these copy-in/copy-out behaviour: >> > I understand the problems it causes with MPI. But, it also make a lot of > wonderful things work in Fortran. You cannot remove it from Fortran just to > accommodate MPI. Saying "don't do copy in/out" will only work if you specify, > in details, what to do about situations where copy in/out is required. Should > we disallow those situations (i.e., put rules like "the actual argument shall > not be..."). Do some existing Fortran codes rely on copy in/out, for example: > > CALL MPI_Send(my_array(1,:),...) ! Will copy column 1 into a contig buffer > I agree that there may be Fortran application codes which are using this. But this would probably not work for MPI_Isend. And this is the main problem. The application programmers cannot always decide what the compiler is doing and whether the MPI calls will work or not. > Should we now make those programs non-conforming? I am not offering answers > here, I am pointing out that explicit answers are required. > I think, that it must be possible to explicitly define what is performed at the subroutine interface so that the application programmer AND the compiler explicitly knows what the other partner expects. The C_LOC functionality is not far away from this requirement and the ASYNCHRONOUS attribute also avoids any copying of the actual argument. Best regards Hubert --------------------------------------------------------------------- Intel GmbH Dornacher Strasse 1 85622 Feldkirchen/Muenchen Germany Sitz der Gesellschaft: Feldkirchen bei Muenchen Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer Registergericht: Muenchen HRB 47456 Ust.-IdNr. VAT Registration No.: DE129385895 Citibank Frankfurt (BLZ 502 109 00) 600119052 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.

1 0

Re: [MPI3 Fortran] Proposing changes to Fortran 2008
by Hubert Ritzdorf 27 Mar '08

27 Mar '08

Aleksandar Donev wrote: > On Thursday 27 March 2008 07:53, Hubert Ritzdorf wrote: > > >> This is what some vendors already implemented (using different names). >> When I remember (and understood) >> it correctly, you mentioned that they didn't it right. >> > They have not integrated it with the rest of the standard. You can do many > things that apparently work fine until you try them with polymorphic > assumed-shape arrays of a parameterized derived type (yes, Fortran has > those). A standard has to be internally fully self-consistent...it cannot > just say "oh, no one will do that, just make it work". > > >> Since the >> behaviour should be well-defined if an >> interface description is not available and the vendors have to implement >> it, and it should be possible >> to specify it for a single argument. >> > The existing standard says the following about implicit interfaces. A > routine's interface can be omitted, however, the interface exists and there > is a *unique* interface (modulo some attributes that can be omitted). That > is, an MPI_Send routine, which exists somewhere, according to the Fortran > standard, has to either accept an integer buffer or a real buffer. It cannot > accept both. There is a *unique* interface. If the interface is implicit, it > is up to the user to ensure the actual arguments actually match the type/rank > etc. of the dummy. If it is explicit, the compiler helps check it. That is > the ONLY difference between implicit and explicit interfaces. > > Now, compilers often allow users to violate the standard and call MPI_Send > with both a real and an integer buffer array, and they either do not do any > analysis to detect it, or if they do, they offer a compiler switch to > say "close your eyes please". It is however, not conforming to the Fortran > standard, period. > I don't believe that the compiler guys have introduced such compiler switches only for MPI since also other libraries have such problems. And I have also seen compiler switches for "don't perform copy-in/copy out". And I see Fortran codes transferring scalar arguments such as "my_array (1,1)" (nevertheless many data items are transferred) in order to avoid copy in/copy out. > >> There are 2 general problems for MPI with these copy-in/copy-out behaviour: >> > I understand the problems it causes with MPI. But, it also make a lot of > wonderful things work in Fortran. You cannot remove it from Fortran just to > accommodate MPI. Saying "don't do copy in/out" will only work if you specify, > in details, what to do about situations where copy in/out is required. Should > we disallow those situations (i.e., put rules like "the actual argument shall > not be..."). Do some existing Fortran codes rely on copy in/out, for example: > > CALL MPI_Send(my_array(1,:),...) ! Will copy column 1 into a contig buffer > I agree that there may be Fortran application codes which are using this. But this would probably not work for MPI_Isend. And this is the main problem. The application programmers cannot always decide what the compiler is doing and whether the MPI calls will work or not. > Should we now make those programs non-conforming? I am not offering answers > here, I am pointing out that explicit answers are required. > I think, that it must be possible to explicitly define what is performed at the subroutine interface so that the application programmer AND the compiler explicitly knows what the other partner expects. The C_LOC functionality is not far away from this requirement and the ASYNCHRONOUS attribute also avoids any copying of the actual argument. Best regards Hubert

1 0

Re: [MPI3 Fortran] Proposing changes to Fortran 2008
by Aleksandar Donev 27 Mar '08

27 Mar '08

On Thursday 27 March 2008 07:53, Hubert Ritzdorf wrote: > This is what some vendors already implemented (using different names). > When I remember (and understood) > it correctly, you mentioned that they didn't it right. They have not integrated it with the rest of the standard. You can do many things that apparently work fine until you try them with polymorphic assumed-shape arrays of a parameterized derived type (yes, Fortran has those). A standard has to be internally fully self-consistent...it cannot just say "oh, no one will do that, just make it work". > Since the > behaviour should be well-defined if an > interface description is not available and the vendors have to implement > it, and it should be possible > to specify it for a single argument. The existing standard says the following about implicit interfaces. A routine's interface can be omitted, however, the interface exists and there is a *unique* interface (modulo some attributes that can be omitted). That is, an MPI_Send routine, which exists somewhere, according to the Fortran standard, has to either accept an integer buffer or a real buffer. It cannot accept both. There is a *unique* interface. If the interface is implicit, it is up to the user to ensure the actual arguments actually match the type/rank etc. of the dummy. If it is explicit, the compiler helps check it. That is the ONLY difference between implicit and explicit interfaces. Now, compilers often allow users to violate the standard and call MPI_Send with both a real and an integer buffer array, and they either do not do any analysis to detect it, or if they do, they offer a compiler switch to say "close your eyes please". It is however, not conforming to the Fortran standard, period. > There are 2 general problems for MPI with these copy-in/copy-out behaviour: I understand the problems it causes with MPI. But, it also make a lot of wonderful things work in Fortran. You cannot remove it from Fortran just to accommodate MPI. Saying "don't do copy in/out" will only work if you specify, in details, what to do about situations where copy in/out is required. Should we disallow those situations (i.e., put rules like "the actual argument shall not be..."). Do some existing Fortran codes rely on copy in/out, for example: CALL MPI_Send(my_array(1,:),...) ! Will copy column 1 into a contig buffer Should we now make those programs non-conforming? I am not offering answers here, I am pointing out that explicit answers are required. Best, Aleks

1 0