March 2008 - mpiwg-fortran - mailman3.cels.anl.gov

Re: [MPI3 Fortran] Proposing changes to Fortran 2008
by Hubert Ritzdorf 27 Mar '08

27 Mar '08

Hi, Aleksandar Donev wrote: > Hi, > > Not to be argumentative, take this as IMO please. > > On Friday 21 March 2008 05:13, Hubert Ritzdorf wrote: > >> MPI and Fortran 77 have worked without any problems >> > Except for the problem that the MPI Forum chose to completely ignore and > violate ALL existing Fortran standards. This *includes* Fortran 77. All > Fortran versions are strictly typed and passing a REAL array to an INTEGER > argument has never been, nor will it ever be legal. > > >> Thus, an attribute for an argument "Behave such as without interface >> description >> > Then don't supply an interface? Or simply say > > VOID :: buffer > > for the actual just to give it a name. I already commented on this earlier. > This is what some vendors already implemented (using different names). When I remember (and understood) it correctly, you mentioned that they didn't it right. Since the behaviour should be well-defined if an interface description is not available and the vendors have to implement it, and it should be possible to specify it for a single argument. It's fine for me to have such a VOID attribute. > >> and don't perform copy-in/copy-out" >> > What if the actual argument is a strided (non-contiguous) array or, say, an > array constructor? The Fortran committee did not invent copy-in/copy-out > because compilers wanted to do copy frivolously. It is a necessary > implementation method to pass certain types of actuals to certain types of > dummies. You have to explicitly address this and cannot just say "go back to > Fortran 77". New Fortran has strided arrays, array constructors, array > pointers, etc., etc., like it or not. > There are 2 general problems for MPI with these copy-in/copy-out behaviour: (*) In non-blocking MPI communication routines, the buffers may be received or sent after the MPI routine which specifies the buffer has already finished. If the compiler issues a copy-in/copy out of that buffer before/after that MPI routine, this causes incorrect results. I.e. the data is read from or written to memory which is already used by other tasks. Even if the application programmer is aware of this and has ensured that the data is stored in consecutive/contiguous memory addresses, it happens that the compiler performs a copy-in/copy-out before/after these MPI routines. Thus, a major problem for the application programmer is that there is no possibility to specify this to the interface description so that the compiler doesn't perform copy-in/copy out. A second major problem is that the application program may work for a long time with this copy-in/copy-out behaviour for non-blocking MPI routines. But small changes in the run-time behaviour of the application program may cause errors (incorrect results/segmentation violations) in other (far away) parts of the application programs which are very very hard to find/debug. (*) Derived MPI datatypes are defined as sequence of primitive types and byte displacements. The byte displacements (which might be negative) are relative to the buffer specified in the MPI function call. (If the MPI_BOTTOM is specified as MPI buffer, you can see the byte displacements as memory addresses). In order to compute these displacements for general derived datatypes, function MPI_Get_address should be called. Again copy-in before MPI_Get_address or copy-in/copy-out before/after a MPI communication subroutine call might cause incorrect results because the memory layout of the generated MPI derived datatype and the memory layout which was generated by the compiler before/after the MPI communication call might be totally different. Even if the application programmer defines MPI derived datatype correctly, he cannot control the behaviour of the application program since he doesn't know at which places the compiler creates temporary memory in order to perform copy-in/copy-out. Best regards Hubert >

1 0

Re: [MPI3 Fortran] Proposing changes to Fortran 2008
by Aleksandar Donev 25 Mar '08

25 Mar '08

Hi, Not to be argumentative, take this as IMO please. On Friday 21 March 2008 05:13, Hubert Ritzdorf wrote: > MPI and Fortran 77 have worked without any problems Except for the problem that the MPI Forum chose to completely ignore and violate ALL existing Fortran standards. This *includes* Fortran 77. All Fortran versions are strictly typed and passing a REAL array to an INTEGER argument has never been, nor will it ever be legal. > Thus, an attribute for an argument "Behave such as without interface > description Then don't supply an interface? Or simply say VOID :: buffer for the actual just to give it a name. I already commented on this earlier. > and don't perform copy-in/copy-out" What if the actual argument is a strided (non-contiguous) array or, say, an array constructor? The Fortran committee did not invent copy-in/copy-out because compilers wanted to do copy frivolously. It is a necessary implementation method to pass certain types of actuals to certain types of dummies. You have to explicitly address this and cannot just say "go back to Fortran 77". New Fortran has strided arrays, array constructors, array pointers, etc., etc., like it or not. Best, Aleks -- Aleksandar Donev, Ph.D. Lawrence Postdoctoral Fellow @ Lawrence Livermore National Laboratory High Performance Computational Materials Science and Chemistry E-mail: donev1(a)llnl.gov Phone: (925) 424-6816 Fax: (925) 423-0785 Address: P.O.Box 808, L-367, Livermore, CA 94551-9900 Web: http://cherrypit.princeton.edu/donev

1 0

Re: [MPI3 Fortran] Proposing changes to Fortran 2008
by Hubert Ritzdorf 21 Mar '08

21 Mar '08

Hi, I think what is required is an option/attribute for an argument that it should behave such as in Fortran 77 and before: This means (*) Don't check (ignore) the type and rank of the corresponding argument in the calling program (*) Don't perform copy in/copy out for that argument before/after calling the routine (especially important for all non-blocking routines such as MPI_Isend, MPI_Irecv and all possibly newly introduced non-blocking MPI routines) (*) Pass the reference/pointer of the (first) variable to the subroutine; no descriptor This is exactly what was done by the old Fortran 77 compilers (except for character variables on some other compilers (Cray, ...)) This must be still possible, since Fortran must support the old object files. This is also important for all derived MPI datatypes Such an option would be not only beneficial for MPI. I'm also programming Fortran libraries for coupling climate/weather codes and I have the problem also within these Fortran libraries that I cannot give an interface description for some general routines since the number of possibilities by different types and ranks is too large. Therefore, I don't provide an interface description for these routines and Fortran already fulfills the items above except for the copy in/copy out requirement. MPI and Fortran 77 have worked without any problems (except for the character variables for which sometimes a pointer to a structure was passed). The problems with the number of interfaces and copy-in/copy-out of (sub-)vectors/arrays were introduced by Fortran 90. Thus, an attribute for an argument "Behave such as without interface description and don't perform copy-in/copy-out" (or 2 separate attributes "Behave such as without interface description" and "don't perform copy-in/copy-out) would be beneficial for each (FORTRAN) library programmer and would help the compiler guys in the decision, whether a copy-in/copy out has to be performed, and can improve the performance of the Fortran code. Hubert

1 0

Re: [MPI3 Fortran] Proposing changes to Fortran 2008
by Supalov, Alexander 20 Mar '08

20 Mar '08

Hi, The datatypes used in several concurrent operations may describe portions of data that overlap (hole/dataA, dataB/hole). In this case sending the holes will violate the standard I'm afraid. Best regards. Alexander -----Original Message----- From: mpi3-fortran-bounces(a)lists.mpi-forum.org [mailto:[email protected]] On Behalf Of Jeff Squyres Sent: Thursday, March 20, 2008 8:27 PM To: MPI-3 Fortran working group Subject: Re: [MPI3 Fortran] Proposing changes to Fortran 2008 On homogeneous architectures -- where the same hardware and software is used on all sides (to include the compiler) -- yes, this will likely work just fine. And probably most people do this, because it can be quite a bit easier. If your data is dense in memory, that's probably "good enough". If your data is sparse, you could end up sending quite a bit of garbage along with your real data, decreasing the bandwidth available to your app, potentially adding more latency, etc. In heterogeneous situations (where any of the above assumptions is not true), it clearly may not work properly. On Mar 20, 2008, at 1:47 PM, Aleksandar Donev wrote: > On Thursday 20 March 2008 10:35, Jeff Squyres wrote: >> because the >> user application will have had to created an MPI datatype *at run >> time* that will have accurate offsets/types of each member. > Out of curiosity: Have you seen codes that actually do that? > What I would do, to make my life simple, is treated the whole scalar > of > derived type as a blob of memory and ignored what fields are > actually inside, > in what order, and with what padding. On homogeneous architectures > doesn't > this work just fine? > Aleks > > _______________________________________________ > mpi3-fortran mailing list > mpi3-fortran(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-fortran -- Jeff Squyres Cisco Systems _______________________________________________ mpi3-fortran mailing list mpi3-fortran(a)lists.mpi-forum.org http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-fortran --------------------------------------------------------------------- Intel GmbH Dornacher Strasse 1 85622 Feldkirchen/Muenchen Germany Sitz der Gesellschaft: Feldkirchen bei Muenchen Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer Registergericht: Muenchen HRB 47456 Ust.-IdNr. VAT Registration No.: DE129385895 Citibank Frankfurt (BLZ 502 109 00) 600119052 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.

1 0

Re: [MPI3 Fortran] Proposing changes to Fortran 2008
by Jeff Squyres 20 Mar '08

20 Mar '08

On homogeneous architectures -- where the same hardware and software is used on all sides (to include the compiler) -- yes, this will likely work just fine. And probably most people do this, because it can be quite a bit easier. If your data is dense in memory, that's probably "good enough". If your data is sparse, you could end up sending quite a bit of garbage along with your real data, decreasing the bandwidth available to your app, potentially adding more latency, etc. In heterogeneous situations (where any of the above assumptions is not true), it clearly may not work properly. On Mar 20, 2008, at 1:47 PM, Aleksandar Donev wrote: > On Thursday 20 March 2008 10:35, Jeff Squyres wrote: >> because the >> user application will have had to created an MPI datatype *at run >> time* that will have accurate offsets/types of each member. > Out of curiosity: Have you seen codes that actually do that? > What I would do, to make my life simple, is treated the whole scalar > of > derived type as a blob of memory and ignored what fields are > actually inside, > in what order, and with what padding. On homogeneous architectures > doesn't > this work just fine? > Aleks > > _______________________________________________ > mpi3-fortran mailing list > mpi3-fortran(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-fortran -- Jeff Squyres Cisco Systems

1 0

Re: [MPI3 Fortran] Proposing changes to Fortran 2008
by Aleksandar Donev 20 Mar '08

20 Mar '08

On Thursday 20 March 2008 10:35, Jeff Squyres wrote: > because the ï¿½ > user application will have had to created an MPI datatype *at run ï¿½ > time* that will have accurate offsets/types of each member. Out of curiosity: Have you seen codes that actually do that? What I would do, to make my life simple, is treated the whole scalar of derived type as a blob of memory and ignored what fields are actually inside, in what order, and with what padding. On homogeneous architectures doesn't this work just fine? Aleks

1 0

Re: [MPI3 Fortran] Proposing changes to Fortran 2008
by Jeff Squyres 20 Mar '08

20 Mar '08

Ok. But the point here is that you do not have to have an interoperable type to pass it to an MPI function as a choice buffer. Specifically: if you have a derived type in Fortran and the compiler re-orders the fields in memory, that's no problem for MPI because the user application will have had to created an MPI datatype *at run time* that will have accurate offsets/types of each member. So MPI has the true memory layout map, regardless of whether the Fortran compiler re-ordered the members or not. On Mar 20, 2008, at 12:50 PM, Aleksandar Donev wrote: > Craig Rasmussen wrote: >> struct foo { >> char a; >> char b; >> int i; >> char c; >> char d; >> } >> >> Don't you have to worry about the C compiler reordering the elements >> of foo? > No, the C standard mandates that the order be preserved (with possible > padding). I don't know the C99 standard well-enough to search it > efficiently, but I am pretty sure it is so. Fortran does not guarantee > anything unless it is an interoperable type. > Aleks > _______________________________________________ > mpi3-fortran mailing list > mpi3-fortran(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-fortran -- Jeff Squyres Cisco Systems

1 0

Re: [MPI3 Fortran] Proposing changes to Fortran 2008
by Aleksandar Donev 20 Mar '08

20 Mar '08

Craig Rasmussen wrote: > struct foo { > char a; > char b; > int i; > char c; > char d; > } > > Don't you have to worry about the C compiler reordering the elements > of foo? No, the C standard mandates that the order be preserved (with possible padding). I don't know the C99 standard well-enough to search it efficiently, but I am pretty sure it is so. Fortran does not guarantee anything unless it is an interoperable type. Aleks

1 0

Re: [MPI3 Fortran] Proposing changes to Fortran 2008
by Jeff Squyres 20 Mar '08

20 Mar '08

On Mar 20, 2008, at 11:35 AM, Craig Rasmussen wrote: > > Hum, is there someplace you can direct me to that states that > elements of C structs may not be reordered? I had thought otherwise. No, only anecdotal knowledge. I've been told that C can't re-order structs (by default). Perhaps one of the compiler guys here could confirm/deny? -- Jeff Squyres Cisco Systems

1 0

Re: [MPI3 Fortran] Proposing changes to Fortran 2008
by Craig Rasmussen 20 Mar '08

20 Mar '08

On Mar 20, 2008, at 9:19 AM, Jeff Squyres wrote: > On Mar 20, 2008, at 11:13 AM, Craig Rasmussen wrote: >> Yes, but the same size hole would exist for the "companion" C struct >> as well. So however you calculate stride information for C would >> work for an interoperable derived type in Fortran as well. Suppose >> foo were, >> >> struct foo { >> char a; >> char b; >> int i; >> char c; >> char d; >> } > > Ok, but you said everything in Fortran had to be contiguous. With a > hole, by definition, it's not contiguous (and, per my other mails, you > can be much more arbitrary, such as skipping the "b" field in the > struct above). Does Fortran have a different definition of > "contiguous" than C? :-) Doesn't Fortran have a different definition of everything? :-) An object is contiguous if it is one of several things, including "an array allocated by an ALLOCATE statement." Thus an array of foos (that sounds cool) will be contiguous. That is what I have been meaning. Obviously since foo has two or more elements, successive elements foo.a in the array are not contiguous. > >> Don't you have to worry about the C compiler reordering the elements >> of foo? > > With C++, perhaps. With C, no. Hum, is there someplace you can direct me to that states that elements of C structs may not be reordered? I had thought otherwise. Craig

1 0