March 2011 - mpiwg-fortran - mailman3.cels.anl.gov

Re: [MPI3 Fortran] TYPE(MPI_Status) with SEQUENCE attribute
by Bill Long 31 Mar '11

31 Mar '11

Hi Rolf, On 3/31/11 2:31 PM, Rolf Rabenseifner wrote: > Dear all, > > as discussed, I had removed the SEQUENCE attribute from form the > new TYPE(MPI_Status) derived type within the new mpi_f08 module. > > I checked now with a test program whether the new status can be used > in a user program in the same way as the old status (i.e. an INTEGER, > DIMENSION(MPI_STATUS_SIZE) array) and detected, that the compiler > will issue a lot of incompatibilities if the newly defined status > does not use internally the sequence attribute. > The messages are valid. However, the use of SEQUENCE seems curious. I would have written the module more like this: MODULE mpi_f08 use,intrinsic :: iso_c_binding IMPLICIT NONE TYPE,bind(c) :: MPI_Comm INTEGER(C_int) :: MPI_VAL END TYPE MPI_Comm TYPE,bind(c) :: MPI_Status INTEGER(C_int) :: MPI_SOURCE INTEGER(C_int) :: MPI_TAG INTEGER(C_int) :: MPI_ERROR INTEGER(C_int) :: MPI_intern_highsize INTEGER(C_int) :: MPI_intern_lowsize END TYPE MPI_Status INTEGER(C_int),parameter :: MPI_STATUS_SIZE=5 END MODULE mpi_f08 The use of a bind(c) type ensures that the memory layout of the type is the same as for the corresponding C struct. It would seem like this is the goal. The SEQUENCE type does not necessarily have this characteristic. Cheers, Bill > It is not expected that this SEQUENCE attribute has any negative > impact in scalable applications. > > Therefore, I had to reinvent the SEQUENCE into the TYPE(MPI_Status). > > The test program and compiler output is attached. > The NON_.... variables are declared with types where > the SEQUENCE is missing. > All these NON_.... variables cause errors and the abort of the compilation. > > Same usage patterns with MPI handle and status types internally using > SEQUENCE do not cause any errors. > > Best regards > Rolf > -- Bill Long longb(a)cray.com Fortran Technical Support & voice: 651-605-9024 Bioinformatics Software Development fax: 651-605-9142 Cray Inc./Cray Plaza, Suite 210/380 Jackson St./St. Paul, MN 55101

1 0

[MPI3 Fortran] TYPE(MPI_Status) with SEQUENCE attribute
by Rolf Rabenseifner 31 Mar '11

31 Mar '11

Dear all, as discussed, I had removed the SEQUENCE attribute from form the new TYPE(MPI_Status) derived type within the new mpi_f08 module. I checked now with a test program whether the new status can be used in a user program in the same way as the old status (i.e. an INTEGER, DIMENSION(MPI_STATUS_SIZE) array) and detected, that the compiler will issue a lot of incompatibilities if the newly defined status does not use internally the sequence attribute. It is not expected that this SEQUENCE attribute has any negative impact in scalable applications. Therefore, I had to reinvent the SEQUENCE into the TYPE(MPI_Status). The test program and compiler output is attached. The NON_.... variables are declared with types where the SEQUENCE is missing. All these NON_.... variables cause errors and the abort of the compilation. Same usage patterns with MPI handle and status types internally using SEQUENCE do not cause any errors. Best regards Rolf -- Dr. Rolf Rabenseifner . . . . . . . . . .. email rabenseifner(a)hlrs.de High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner Nobelstr. 19, D-70550 Stuttgart, Germany . (Office: Allmandring 30)

1 0

Re: [MPI3 Fortran] Nonblocking MPI and Fortran temporary memory modifications
by Rolf Rabenseifner 29 Mar '11

29 Mar '11

A few minutes ago, we talked in the MPI Forum plenary about this issue and decided that we as the MPI standardization body will send to the Fortran Standardization body an official request to implement this extension of the ASYNCHRONOUS attribute as it was planned in this J3 vote > SV: extend ASYNCHRONOUS - invent new attribute - undefined: 8-2-3 because we very seriously need this extension. Without it, the combination of any nonblocking MPI routine with Fortran *will not work*. Rolf ----- Original Message ----- > From: "Rolf Rabenseifner" <rabenseifner(a)hlrs.de> > To: "Craig E Rasmussen" <rasmussn(a)lanl.gov> > Cc: "MPI-3 Fortran working group" <mpi3-fortran(a)lists.mpi-forum.org>, "N.M. Maclaren" <nmm1(a)cam.ac.uk>, "Bill Long" > <longb(a)cray.com>, "Reinhold Bader" <reinhold.bader(a)lrz.de>, "Brian E. Smith" <smithbr(a)us.ibm.com> > Sent: Tuesday, March 29, 2011 3:34:50 PM > Subject: Re: Nonblocking MPI and Fortran temporary memory modifications > Hi all, > > this is an excellent basis for the necessary review (deadline March > 31, 2011!) of > TR 29113 (Version N1845) > ftp://ftp.nag.co.uk/sc22wg5/N1801-N1850/N1845.pdf > from > http://www.nag.co.uk/sc22wg5/ > > I put Brain Smith from IBM also on the CC list. > > The cited papers can be found at > http://www.j3-fortran.org/doc/year/09/09-235r2.txt > and > http://www.j3-fortran.org/doc/year/09/09-231.txt > > Goal of this "extend ASYNCHRONOUS" > > SV: extend ASYNCHRONOUS - invent new attribute - undefined: 8-2-3 > > is that the following code is a correct code > and allows that the MPI library internally used DMA, the NIC, > or other methods to communicate parts of the array while > the user application works on another part of the same array: > > USE mpi_f08 > REAL, ASYNCHRONOUS :: buf(100,100) > CALL MPI_Irecv(buf(1,1:100),...req,...) > DO j=1,100 > DO i=2,100 > buf(i,j)=.... > END DO > END DO > CALL MPI_Wait(req,...) > > It is important that the compiler is not allowed to translate this > program (by using temporary memory modifications) into > > USE mpi_f08 > REAL, ASYNCHRONOUS :: buf(100,100), buf_1dim(10000) > EQUIVALENCE (buf(1,1), buf_1dim(1)) > CALL MPI_Irecv(buf(1,1:100),...req,...) > tmp(1:100)=buf(1,1:100) > DO j=1,10000 > buf_1dim(j)=... > END DO > buf(1,1:100)=tmp(1:100) > CALL MPI_Wait(req,...) > > While the MPI library receives buf(1,1:100), > the numerical part overwrites this part of the array > as part of a numerical optimization to achieve one long > loop instead of a 2-loop-nesting. > > The current status of Fortran 2008 + TR 29113 (Version N1845) > is that a compiler can ignore the ASYNCHRONOUS attribute, > for example, because it does not implement Fortran asynchronous > I/O in an asynchronous way, i.e., there is no need > for the compiler to make any restrictions on optimization. > > The goal of the vote > > SV: extend ASYNCHRONOUS - invent new attribute - undefined: 8-2-3 > > was, to extend the meaning of the ASYNCHRONOUS attribute > that it can be used together with MPI or other asynchronous > software. > > As one can see in > https://svn.mpi-forum.org/trac/mpi-forum-web/attachment/ticket/229/mpi-repo… > Page 25 lines 5-14 in the pdf (page 550 in the original document), > the ASYNCHRONOUS attribute is the only one that can solve the problem. > > The word "SOLVED" is already based on the "extend ASYNCHRONOUS", > (i.e., the extended meaning of the Fortran ASYNCHRONOUS attribute) > and can be used for the whole row. > > With the current state of the TR 29113, the whole line is > "NOT solved". > > VOLATILE is not an option, because it would eliminate > any numerical optimization on any code that is used > in overlapping communication and computation. > > Final implication: > To solve this problem is absolute necessary for any > Fortran implementation of the MPI library. > > Additionally to the implementation of the "extend ASYNCHRONOUS" > in the TR 29113 of Fortran 2008, we plan to write into the MPI-3.0 > standard a sentence like this: > > "A valid implementation of the MPI-3.0 standard requires that > the Fortran compiler in use has to guarantee that the > ASYNCHRONOUS attribute guarantees that any MPI nonblocking > operation can be used on data that is marked as ASYNCHRONOUS. > Other parts of the same data can be used in overlapping computation > as long as the application uses the ASYNCHRONOUS attribute > also in this part of the application software." > > Best regards > Rolf > > ----- Original Message ----- > > From: "Craig E Rasmussen" <rasmussn(a)lanl.gov> > > To: "Craig E Rasmussen" <rasmussn(a)lanl.gov> > > Cc: "Rolf Rabenseifner" <rabenseifner(a)hlrs.de>, "MPI-3 Fortran > > working group" <mpi3-fortran(a)lists.mpi-forum.org>, > > "N.M. Maclaren" <nmm1(a)cam.ac.uk>, "Bill Long" <longb(a)cray.com>, > > "Reinhold Bader" <reinhold.bader(a)lrz.de> > > Sent: Tuesday, March 29, 2011 2:50:47 AM > > Subject: Re: Nonblocking MPI and Fortran temporary memory > > modifications > > I've found the relevant paper in J3 meeting minutes (J3 document > > 09-235r2). The paper is 09-231 "Answer to MPI Forum regarding MPI > > asynchronous operations" from J3 meeting 188. The minutes state the > > following regarding this paper: > > > > There was a straw vote taken on what syntax to use to inhibit > > optimizations for MPI asynchronous operations: > > > > Paper 09-231 "Answer to MPI Forum regarding MPI asynchronous > > operations" [Rasmussen] discussed how to prevent code-motion and > > copy-in/out: > > > > SV: extend ASYNCHRONOUS - invent new attribute - undefined: 8-2-3 > > > > > > The vote was 8 for ASYNCHRONOUS and 2 for new syntax with 3 > > abstentions. > > > > There was also a report from the INTEROP subcommittee: > > > > /INTEROP: no further action will be taken on 09-185r1, 09-189r1, > > 09-191, 09-231 - they are all input for the Interop TR > > N1761 > > > > So according to the minutes N1761 was supposed to take this vote and > > paper 09-231 into account. However, to my knowledge no text was > > added > > to the TR in response to this. > > > > This was my bad. I should have realized that just because > > ASYNCHRONOUS > > is currently part of the Fortran standard, that this keyword can be > > used outside of Fortran I/O without further words to that affect in > > the standard. > > > > We can discuss what to do in response to this in the telecon this > > evening. > > > > -craig > > > > > > On Mar 28, 2011, at 4:18 PM, Rasmussen, Craig E wrote: > > > > I'll look for the vote in the minutes. > > > > -craig > > > > On Mar 28, 2011, at 4:12 PM, Rolf Rabenseifner wrote: > > > > Craig, is there a protocol, file, ... on > > http://www.nag.co.uk/sc22wg5/ > > where this vote is protocolled? > > Rolf > > > > ----- Original Message ----- > > From: "Craig E Rasmussen" > > <rasmussn(a)lanl.gov<mailto:[email protected]>> > > To: "Rolf Rabenseifner" > > <rabenseifner(a)hlrs.de<mailto:[email protected]>> > > Cc: "MPI-3 Fortran working group" > > <mpi3-fortran(a)lists.mpi-forum.org<mailto:[email protected]>>, > > "N.M. Maclaren" <nmm1(a)cam.ac.uk<mailto:[email protected]>>, "Bill Long" > > <longb(a)cray.com<mailto:[email protected]>>, "Reinhold Bader" > > <reinhold.bader(a)lrz.de<mailto:[email protected]>> > > Sent: Monday, March 28, 2011 6:14:00 PM > > Subject: Re: Nonblocking MPI and Fortran temporary memory > > modifications > > We've had several discussions regarding the use of ASYNCHRONOUS for > > the usage required by MPI on the J3 committee. Some members thought > > that ASYNCHRONOUS shouldn't be used apart from I/O. However, a vote > > was taken and a large majority voted that the ASYNCHRONOUS attribute > > should be used for this purpose. > > > > So the question is not about whether a compiler implements > > ASYNCHRONOUS I/O but whether it respects the ASYNCHRONOUS semantics. > > Perhaps we need an interp request to formalize this beyond a simple > > J3 > > member vote. > > > > -craig > > > > > > On Mar 27, 2011, at 3:34 AM, Rolf Rabenseifner wrote: > > > > Dear all, > > > > Reinhold showed me that the ASYNCHRONOUS attribute may not help > > because a compiler may implement asynchronous Fortran I/O > > with blocking I/O and therefore may ignore all ASYNCHRONOUS > > attributes. > > > > Please have a look at > > https://svn.mpi-forum.org/trac/mpi-forum-web/attachment/ticket/229/mpi-repo… > > > > Pages 24-31 in the pdf (pages 549-556 in the original document) > > show our knowledge about the nonblocking and datatype (MPI_BOTTOM) > > problems > > in the combination of Fortran and MPI. > > > > Unfortunately in the taböle on page 550, we have to modify all > > entries > > about ASYNCHRONOUS into "NOT solved". > > > > Do you see further solutions? > > > > Do you see further problems with nonblocking and "MPI_BOTTOM", > > that are not mentioned in this section and that should be. > > > > Is the rest of this section correct? > > If something is incorrect, it would be important to understand why. > > > > I've reworked this section based on many discussions and on my best > > knowledge - but I'm not sure whether this was enough. > > > > Best regards > > Rolf > > > > > > -- > > Dr. Rolf Rabenseifner . . . . . . . . . .. email > > rabenseifner(a)hlrs.de<mailto:[email protected]> > > High Performance Computing Center (HLRS) . phone > > ++49(0)711/685-65530 > > University of Stuttgart . . . . . . . . .. fax ++49(0)711 / > > 685-65832 > > Head of Dpmt Parallel Computing . . . > > www.hlrs.de/people/rabenseifner<http://www.hlrs.de/people/rabenseifner> > > Nobelstr. 19, D-70550 Stuttgart, Germany . (Office: Allmandring 30) > > > > -- > > Dr. Rolf Rabenseifner . . . . . . . . . .. email > > rabenseifner(a)hlrs.de<mailto:[email protected]> > > High Performance Computing Center (HLRS) . phone > > ++49(0)711/685-65530 > > University of Stuttgart . . . . . . . . .. fax ++49(0)711 / > > 685-65832 > > Head of Dpmt Parallel Computing . . . > > www.hlrs.de/people/rabenseifner<http://www.hlrs.de/people/rabenseifner> > > Nobelstr. 19, D-70550 Stuttgart, Germany . (Office: Allmandring 30) > > -- > Dr. Rolf Rabenseifner . . . . . . . . . .. email rabenseifner(a)hlrs.de > High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 > University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 > Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner > Nobelstr. 19, D-70550 Stuttgart, Germany . (Office: Allmandring 30) -- Dr. Rolf Rabenseifner . . . . . . . . . .. email rabenseifner(a)hlrs.de High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner Nobelstr. 19, D-70550 Stuttgart, Germany . (Office: Allmandring 30)

1 0

Re: [MPI3 Fortran] Nonblocking MPI and Fortran temporary memory modifications
by Rolf Rabenseifner 29 Mar '11

29 Mar '11

Hi all, this is an excellent basis for the necessary review (deadline March 31, 2011!) of TR 29113 (Version N1845) ftp://ftp.nag.co.uk/sc22wg5/N1801-N1850/N1845.pdf from http://www.nag.co.uk/sc22wg5/ I put Brain Smith from IBM also on the CC list. The cited papers can be found at http://www.j3-fortran.org/doc/year/09/09-235r2.txt and http://www.j3-fortran.org/doc/year/09/09-231.txt Goal of this "extend ASYNCHRONOUS" SV: extend ASYNCHRONOUS - invent new attribute - undefined: 8-2-3 is that the following code is a correct code and allows that the MPI library internally used DMA, the NIC, or other methods to communicate parts of the array while the user application works on another part of the same array: USE mpi_f08 REAL, ASYNCHRONOUS :: buf(100,100) CALL MPI_Irecv(buf(1,1:100),...req,...) DO j=1,100 DO i=2,100 buf(i,j)=.... END DO END DO CALL MPI_Wait(req,...) It is important that the compiler is not allowed to translate this program (by using temporary memory modifications) into USE mpi_f08 REAL, ASYNCHRONOUS :: buf(100,100), buf_1dim(10000) EQUIVALENCE (buf(1,1), buf_1dim(1)) CALL MPI_Irecv(buf(1,1:100),...req,...) tmp(1:100)=buf(1,1:100) DO j=1,10000 buf_1dim(j)=... END DO buf(1,1:100)=tmp(1:100) CALL MPI_Wait(req,...) While the MPI library receives buf(1,1:100), the numerical part overwrites this part of the array as part of a numerical optimization to achieve one long loop instead of a 2-loop-nesting. The current status of Fortran 2008 + TR 29113 (Version N1845) is that a compiler can ignore the ASYNCHRONOUS attribute, for example, because it does not implement Fortran asynchronous I/O in an asynchronous way, i.e., there is no need for the compiler to make any restrictions on optimization. The goal of the vote SV: extend ASYNCHRONOUS - invent new attribute - undefined: 8-2-3 was, to extend the meaning of the ASYNCHRONOUS attribute that it can be used together with MPI or other asynchronous software. As one can see in https://svn.mpi-forum.org/trac/mpi-forum-web/attachment/ticket/229/mpi-repo… Page 25 lines 5-14 in the pdf (page 550 in the original document), the ASYNCHRONOUS attribute is the only one that can solve the problem. The word "SOLVED" is already based on the "extend ASYNCHRONOUS", (i.e., the extended meaning of the Fortran ASYNCHRONOUS attribute) and can be used for the whole row. With the current state of the TR 29113, the whole line is "NOT solved". VOLATILE is not an option, because it would eliminate any numerical optimization on any code that is used in overlapping communication and computation. Final implication: To solve this problem is absolute necessary for any Fortran implementation of the MPI library. Additionally to the implementation of the "extend ASYNCHRONOUS" in the TR 29113 of Fortran 2008, we plan to write into the MPI-3.0 standard a sentence like this: "A valid implementation of the MPI-3.0 standard requires that the Fortran compiler in use has to guarantee that the ASYNCHRONOUS attribute guarantees that any MPI nonblocking operation can be used on data that is marked as ASYNCHRONOUS. Other parts of the same data can be used in overlapping computation as long as the application uses the ASYNCHRONOUS attribute also in this part of the application software." Best regards Rolf ----- Original Message ----- > From: "Craig E Rasmussen" <rasmussn(a)lanl.gov> > To: "Craig E Rasmussen" <rasmussn(a)lanl.gov> > Cc: "Rolf Rabenseifner" <rabenseifner(a)hlrs.de>, "MPI-3 Fortran working group" <mpi3-fortran(a)lists.mpi-forum.org>, > "N.M. Maclaren" <nmm1(a)cam.ac.uk>, "Bill Long" <longb(a)cray.com>, "Reinhold Bader" <reinhold.bader(a)lrz.de> > Sent: Tuesday, March 29, 2011 2:50:47 AM > Subject: Re: Nonblocking MPI and Fortran temporary memory modifications > I've found the relevant paper in J3 meeting minutes (J3 document > 09-235r2). The paper is 09-231 "Answer to MPI Forum regarding MPI > asynchronous operations" from J3 meeting 188. The minutes state the > following regarding this paper: > > There was a straw vote taken on what syntax to use to inhibit > optimizations for MPI asynchronous operations: > > Paper 09-231 "Answer to MPI Forum regarding MPI asynchronous > operations" [Rasmussen] discussed how to prevent code-motion and > copy-in/out: > > SV: extend ASYNCHRONOUS - invent new attribute - undefined: 8-2-3 > > > The vote was 8 for ASYNCHRONOUS and 2 for new syntax with 3 > abstentions. > > There was also a report from the INTEROP subcommittee: > > /INTEROP: no further action will be taken on 09-185r1, 09-189r1, > 09-191, 09-231 - they are all input for the Interop TR > N1761 > > So according to the minutes N1761 was supposed to take this vote and > paper 09-231 into account. However, to my knowledge no text was added > to the TR in response to this. > > This was my bad. I should have realized that just because ASYNCHRONOUS > is currently part of the Fortran standard, that this keyword can be > used outside of Fortran I/O without further words to that affect in > the standard. > > We can discuss what to do in response to this in the telecon this > evening. > > -craig > > > On Mar 28, 2011, at 4:18 PM, Rasmussen, Craig E wrote: > > I'll look for the vote in the minutes. > > -craig > > On Mar 28, 2011, at 4:12 PM, Rolf Rabenseifner wrote: > > Craig, is there a protocol, file, ... on > http://www.nag.co.uk/sc22wg5/ > where this vote is protocolled? > Rolf > > ----- Original Message ----- > From: "Craig E Rasmussen" > <rasmussn(a)lanl.gov<mailto:[email protected]>> > To: "Rolf Rabenseifner" > <rabenseifner(a)hlrs.de<mailto:[email protected]>> > Cc: "MPI-3 Fortran working group" > <mpi3-fortran(a)lists.mpi-forum.org<mailto:[email protected]>>, > "N.M. Maclaren" <nmm1(a)cam.ac.uk<mailto:[email protected]>>, "Bill Long" > <longb(a)cray.com<mailto:[email protected]>>, "Reinhold Bader" > <reinhold.bader(a)lrz.de<mailto:[email protected]>> > Sent: Monday, March 28, 2011 6:14:00 PM > Subject: Re: Nonblocking MPI and Fortran temporary memory > modifications > We've had several discussions regarding the use of ASYNCHRONOUS for > the usage required by MPI on the J3 committee. Some members thought > that ASYNCHRONOUS shouldn't be used apart from I/O. However, a vote > was taken and a large majority voted that the ASYNCHRONOUS attribute > should be used for this purpose. > > So the question is not about whether a compiler implements > ASYNCHRONOUS I/O but whether it respects the ASYNCHRONOUS semantics. > Perhaps we need an interp request to formalize this beyond a simple J3 > member vote. > > -craig > > > On Mar 27, 2011, at 3:34 AM, Rolf Rabenseifner wrote: > > Dear all, > > Reinhold showed me that the ASYNCHRONOUS attribute may not help > because a compiler may implement asynchronous Fortran I/O > with blocking I/O and therefore may ignore all ASYNCHRONOUS > attributes. > > Please have a look at > https://svn.mpi-forum.org/trac/mpi-forum-web/attachment/ticket/229/mpi-repo… > > Pages 24-31 in the pdf (pages 549-556 in the original document) > show our knowledge about the nonblocking and datatype (MPI_BOTTOM) > problems > in the combination of Fortran and MPI. > > Unfortunately in the taböle on page 550, we have to modify all > entries > about ASYNCHRONOUS into "NOT solved". > > Do you see further solutions? > > Do you see further problems with nonblocking and "MPI_BOTTOM", > that are not mentioned in this section and that should be. > > Is the rest of this section correct? > If something is incorrect, it would be important to understand why. > > I've reworked this section based on many discussions and on my best > knowledge - but I'm not sure whether this was enough. > > Best regards > Rolf > > > -- > Dr. Rolf Rabenseifner . . . . . . . . . .. email > rabenseifner(a)hlrs.de<mailto:[email protected]> > High Performance Computing Center (HLRS) . phone > ++49(0)711/685-65530 > University of Stuttgart . . . . . . . . .. fax ++49(0)711 / > 685-65832 > Head of Dpmt Parallel Computing . . . > www.hlrs.de/people/rabenseifner<http://www.hlrs.de/people/rabenseifner> > Nobelstr. 19, D-70550 Stuttgart, Germany . (Office: Allmandring 30) > > -- > Dr. Rolf Rabenseifner . . . . . . . . . .. email > rabenseifner(a)hlrs.de<mailto:[email protected]> > High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 > University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 > Head of Dpmt Parallel Computing . . . > www.hlrs.de/people/rabenseifner<http://www.hlrs.de/people/rabenseifner> > Nobelstr. 19, D-70550 Stuttgart, Germany . (Office: Allmandring 30) -- Dr. Rolf Rabenseifner . . . . . . . . . .. email rabenseifner(a)hlrs.de High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner Nobelstr. 19, D-70550 Stuttgart, Germany . (Office: Allmandring 30)

1 0

Re: [MPI3 Fortran] Nonblocking MPI and Fortran temporary memory modifications
by Rasmussen, Craig E 28 Mar '11

28 Mar '11

I've found the relevant paper in J3 meeting minutes (J3 document 09-235r2). The paper is 09-231 "Answer to MPI Forum regarding MPI asynchronous operations" from J3 meeting 188. The minutes state the following regarding this paper: There was a straw vote taken on what syntax to use to inhibit optimizations for MPI asynchronous operations: Paper 09-231 "Answer to MPI Forum regarding MPI asynchronous operations" [Rasmussen] discussed how to prevent code-motion and copy-in/out: SV: extend ASYNCHRONOUS - invent new attribute - undefined: 8-2-3 The vote was 8 for ASYNCHRONOUS and 2 for new syntax with 3 abstentions. There was also a report from the INTEROP subcommittee: /INTEROP: no further action will be taken on 09-185r1, 09-189r1, 09-191, 09-231 - they are all input for the Interop TR N1761 So according to the minutes N1761 was supposed to take this vote and paper 09-231 into account. However, to my knowledge no text was added to the TR in response to this. This was my bad. I should have realized that just because ASYNCHRONOUS is currently part of the Fortran standard, that this keyword can be used outside of Fortran I/O without further words to that affect in the standard. We can discuss what to do in response to this in the telecon this evening. -craig On Mar 28, 2011, at 4:18 PM, Rasmussen, Craig E wrote: I'll look for the vote in the minutes. -craig On Mar 28, 2011, at 4:12 PM, Rolf Rabenseifner wrote: Craig, is there a protocol, file, ... on http://www.nag.co.uk/sc22wg5/ where this vote is protocolled? Rolf ----- Original Message ----- From: "Craig E Rasmussen" <rasmussn(a)lanl.gov<mailto:[email protected]>> To: "Rolf Rabenseifner" <rabenseifner(a)hlrs.de<mailto:[email protected]>> Cc: "MPI-3 Fortran working group" <mpi3-fortran(a)lists.mpi-forum.org<mailto:[email protected]>>, "N.M. Maclaren" <nmm1(a)cam.ac.uk<mailto:[email protected]>>, "Bill Long" <longb(a)cray.com<mailto:[email protected]>>, "Reinhold Bader" <reinhold.bader(a)lrz.de<mailto:[email protected]>> Sent: Monday, March 28, 2011 6:14:00 PM Subject: Re: Nonblocking MPI and Fortran temporary memory modifications We've had several discussions regarding the use of ASYNCHRONOUS for the usage required by MPI on the J3 committee. Some members thought that ASYNCHRONOUS shouldn't be used apart from I/O. However, a vote was taken and a large majority voted that the ASYNCHRONOUS attribute should be used for this purpose. So the question is not about whether a compiler implements ASYNCHRONOUS I/O but whether it respects the ASYNCHRONOUS semantics. Perhaps we need an interp request to formalize this beyond a simple J3 member vote. -craig On Mar 27, 2011, at 3:34 AM, Rolf Rabenseifner wrote: Dear all, Reinhold showed me that the ASYNCHRONOUS attribute may not help because a compiler may implement asynchronous Fortran I/O with blocking I/O and therefore may ignore all ASYNCHRONOUS attributes. Please have a look at https://svn.mpi-forum.org/trac/mpi-forum-web/attachment/ticket/229/mpi-repo… Pages 24-31 in the pdf (pages 549-556 in the original document) show our knowledge about the nonblocking and datatype (MPI_BOTTOM) problems in the combination of Fortran and MPI. Unfortunately in the tabï¿½le on page 550, we have to modify all entries about ASYNCHRONOUS into "NOT solved". Do you see further solutions? Do you see further problems with nonblocking and "MPI_BOTTOM", that are not mentioned in this section and that should be. Is the rest of this section correct? If something is incorrect, it would be important to understand why. I've reworked this section based on many discussions and on my best knowledge - but I'm not sure whether this was enough. Best regards Rolf -- Dr. Rolf Rabenseifner . . . . . . . . . .. email rabenseifner(a)hlrs.de<mailto:[email protected]> High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner<http://www.hlrs.de/people/rabenseifner> Nobelstr. 19, D-70550 Stuttgart, Germany . (Office: Allmandring 30) -- Dr. Rolf Rabenseifner . . . . . . . . . .. email rabenseifner(a)hlrs.de<mailto:[email protected]> High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner<http://www.hlrs.de/people/rabenseifner> Nobelstr. 19, D-70550 Stuttgart, Germany . (Office: Allmandring 30)

1 0

Re: [MPI3 Fortran] Results of recent J3 Fortran meeting
by Rolf Rabenseifner 28 Mar '11

28 Mar '11

Craig, Because the warning is issued only at the compilation of the MPI library, and not at the caller of such BIND(C) routines, I agree that the problem is nut an issue for the MPI Forum. Thank you pointing this out Rolf ----- Original Message ----- > From: "Craig E Rasmussen" <rasmussn(a)lanl.gov> > To: "Rolf Rabenseifner" <rabenseifner(a)hlrs.de> > Cc: longb(a)cray.com, "MPI-3 Fortran working group" <mpi3-fortran(a)lists.mpi-forum.org>, "Craig E Rasmussen" > <rasmussn(a)lanl.gov>, "N.M. Maclaren" <nmm1(a)cam.ac.uk>, "Reinhold Bader" <reinhold.bader(a)lrz.de> > Sent: Monday, March 28, 2011 6:05:45 PM > Subject: Re: [MPI3 Fortran] Results of recent J3 Fortran meeting > About a comment from Craig: > gfortran issues a warning when having a > INTEGER dummy argument in combination with BIND(C). > This means that the warning of gfortran is not necessary. > Compiling the interface, it would be okay to have an information > message. > On the caller site, a warning would not be helpful > because it would cause a warning with each MPI call. > I do not know, where gfortran currently produces its warning. > > > So that everyone is clear on the warning message. Here is an example > of an interface but it could also be in a derived type definition with > the BIND(C) attribute: > > interface > subroutine foo(i) BIND(C) > integer :: i > end subroutine > end interface > > The gfortran warning is that the integer i MAY not be interoperable. > There is an error if the types don't match up at the calling site. > This is all as it should be. We just need to ask gfortran to turn the > warning off or to provide a compiler option so that the user can turn > it off. > > > I'm not convinced there is a problem here. > If I understood all correctly, then you convinced me > that there is not a problem, because it is part of an MPI library > compilation to check which C integer type is equivalent > to Fortran INTEGER. Because MPI is doing all by call by reference > (i.e. not by VALUE), I expect that the problem is really solved. > > > Whether the call is by reference or by value is not the issue. In > Fortran the type and kind of the actual argument must agree with that > of the dummy argument in the interface. > > -craig -- Dr. Rolf Rabenseifner . . . . . . . . . .. email rabenseifner(a)hlrs.de High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner Nobelstr. 19, D-70550 Stuttgart, Germany . (Office: Allmandring 30)

1 0

Re: [MPI3 Fortran] Nonblocking MPI and Fortran temporary memory modifications
by Rasmussen, Craig E 28 Mar '11

28 Mar '11

I'll look for the vote in the minutes. -craig On Mar 28, 2011, at 4:12 PM, Rolf Rabenseifner wrote: > Craig, is there a protocol, file, ... on > http://www.nag.co.uk/sc22wg5/ > where this vote is protocolled? > Rolf > > ----- Original Message ----- >> From: "Craig E Rasmussen" <rasmussn(a)lanl.gov> >> To: "Rolf Rabenseifner" <rabenseifner(a)hlrs.de> >> Cc: "MPI-3 Fortran working group" <mpi3-fortran(a)lists.mpi-forum.org>, "N.M. Maclaren" <nmm1(a)cam.ac.uk>, "Bill Long" >> <longb(a)cray.com>, "Reinhold Bader" <reinhold.bader(a)lrz.de> >> Sent: Monday, March 28, 2011 6:14:00 PM >> Subject: Re: Nonblocking MPI and Fortran temporary memory modifications >> We've had several discussions regarding the use of ASYNCHRONOUS for >> the usage required by MPI on the J3 committee. Some members thought >> that ASYNCHRONOUS shouldn't be used apart from I/O. However, a vote >> was taken and a large majority voted that the ASYNCHRONOUS attribute >> should be used for this purpose. >> >> So the question is not about whether a compiler implements >> ASYNCHRONOUS I/O but whether it respects the ASYNCHRONOUS semantics. >> Perhaps we need an interp request to formalize this beyond a simple J3 >> member vote. >> >> -craig >> >> >> On Mar 27, 2011, at 3:34 AM, Rolf Rabenseifner wrote: >> >>> Dear all, >>> >>> Reinhold showed me that the ASYNCHRONOUS attribute may not help >>> because a compiler may implement asynchronous Fortran I/O >>> with blocking I/O and therefore may ignore all ASYNCHRONOUS >>> attributes. >>> >>> Please have a look at >>> https://svn.mpi-forum.org/trac/mpi-forum-web/attachment/ticket/229/mpi-repo… >>> >>> Pages 24-31 in the pdf (pages 549-556 in the original document) >>> show our knowledge about the nonblocking and datatype (MPI_BOTTOM) >>> problems >>> in the combination of Fortran and MPI. >>> >>> Unfortunately in the tabï¿½le on page 550, we have to modify all >>> entries >>> about ASYNCHRONOUS into "NOT solved". >>> >>> Do you see further solutions? >>> >>> Do you see further problems with nonblocking and "MPI_BOTTOM", >>> that are not mentioned in this section and that should be. >>> >>> Is the rest of this section correct? >>> If something is incorrect, it would be important to understand why. >>> >>> I've reworked this section based on many discussions and on my best >>> knowledge - but I'm not sure whether this was enough. >>> >>> Best regards >>> Rolf >>> >>> >>> -- >>> Dr. Rolf Rabenseifner . . . . . . . . . .. email >>> rabenseifner(a)hlrs.de >>> High Performance Computing Center (HLRS) . phone >>> ++49(0)711/685-65530 >>> University of Stuttgart . . . . . . . . .. fax ++49(0)711 / >>> 685-65832 >>> Head of Dpmt Parallel Computing . . . >>> www.hlrs.de/people/rabenseifner >>> Nobelstr. 19, D-70550 Stuttgart, Germany . (Office: Allmandring 30) > > -- > Dr. Rolf Rabenseifner . . . . . . . . . .. email rabenseifner(a)hlrs.de > High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 > University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 > Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner > Nobelstr. 19, D-70550 Stuttgart, Germany . (Office: Allmandring 30)

1 0

Re: [MPI3 Fortran] Nonblocking MPI and Fortran temporary memory modifications
by Rolf Rabenseifner 28 Mar '11

28 Mar '11

Craig, is there a protocol, file, ... on http://www.nag.co.uk/sc22wg5/ where this vote is protocolled? Rolf ----- Original Message ----- > From: "Craig E Rasmussen" <rasmussn(a)lanl.gov> > To: "Rolf Rabenseifner" <rabenseifner(a)hlrs.de> > Cc: "MPI-3 Fortran working group" <mpi3-fortran(a)lists.mpi-forum.org>, "N.M. Maclaren" <nmm1(a)cam.ac.uk>, "Bill Long" > <longb(a)cray.com>, "Reinhold Bader" <reinhold.bader(a)lrz.de> > Sent: Monday, March 28, 2011 6:14:00 PM > Subject: Re: Nonblocking MPI and Fortran temporary memory modifications > We've had several discussions regarding the use of ASYNCHRONOUS for > the usage required by MPI on the J3 committee. Some members thought > that ASYNCHRONOUS shouldn't be used apart from I/O. However, a vote > was taken and a large majority voted that the ASYNCHRONOUS attribute > should be used for this purpose. > > So the question is not about whether a compiler implements > ASYNCHRONOUS I/O but whether it respects the ASYNCHRONOUS semantics. > Perhaps we need an interp request to formalize this beyond a simple J3 > member vote. > > -craig > > > On Mar 27, 2011, at 3:34 AM, Rolf Rabenseifner wrote: > > > Dear all, > > > > Reinhold showed me that the ASYNCHRONOUS attribute may not help > > because a compiler may implement asynchronous Fortran I/O > > with blocking I/O and therefore may ignore all ASYNCHRONOUS > > attributes. > > > > Please have a look at > > https://svn.mpi-forum.org/trac/mpi-forum-web/attachment/ticket/229/mpi-repo… > > > > Pages 24-31 in the pdf (pages 549-556 in the original document) > > show our knowledge about the nonblocking and datatype (MPI_BOTTOM) > > problems > > in the combination of Fortran and MPI. > > > > Unfortunately in the taböle on page 550, we have to modify all > > entries > > about ASYNCHRONOUS into "NOT solved". > > > > Do you see further solutions? > > > > Do you see further problems with nonblocking and "MPI_BOTTOM", > > that are not mentioned in this section and that should be. > > > > Is the rest of this section correct? > > If something is incorrect, it would be important to understand why. > > > > I've reworked this section based on many discussions and on my best > > knowledge - but I'm not sure whether this was enough. > > > > Best regards > > Rolf > > > > > > -- > > Dr. Rolf Rabenseifner . . . . . . . . . .. email > > rabenseifner(a)hlrs.de > > High Performance Computing Center (HLRS) . phone > > ++49(0)711/685-65530 > > University of Stuttgart . . . . . . . . .. fax ++49(0)711 / > > 685-65832 > > Head of Dpmt Parallel Computing . . . > > www.hlrs.de/people/rabenseifner > > Nobelstr. 19, D-70550 Stuttgart, Germany . (Office: Allmandring 30) -- Dr. Rolf Rabenseifner . . . . . . . . . .. email rabenseifner(a)hlrs.de High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner Nobelstr. 19, D-70550 Stuttgart, Germany . (Office: Allmandring 30)

1 0

Re: [MPI3 Fortran] Nonblocking MPI and Fortran temporary memory modifications
by Rasmussen, Craig E 28 Mar '11

28 Mar '11

I don't want to debate this further as a vote on the issue was already taken at J3 on the use of the ASYNCHRONOUS attribute in this context. -craig On Mar 28, 2011, at 11:13 AM, Dan Nagle wrote: > Hi, > > I think I understand the problem. > > I don't think you understand the objection to your solution. > > Expecting one processor to generate code based on what another processor *might* do > is a non-starter. > > The whole "MPI is just like I/O" view is wrong. The devil is in the details. > > If you want to debate this further, I suggest we take it offline. > > On Mar 28, 2011, at 13:05 , Rasmussen, Craig E wrote: > >> Dan, >> >> Perhaps I haven't sufficiently explained what we need to do in MPI3, as I'm not sure what you are trying to convey with restrictions on programs versus restrictions on processors. >> >> The ASYNCHRONOUS attribute has a certain semantic meaning in the context of a valid program. For MPI3 we want the semantics of ASYNCHRONOUS to include that a variable with the ASYCHRONOUS attribute may be subject to asynchronous activity, not just asynchronous input/output (as currently stated in 5.3.4). Rolf's worry (at least it should be) is that the MPI3 standard can't point to a place in the Fortran standard that says a compiler/processor in not conforming if it does not also take into account that a variable with the ASYNCHRONOUS attribute may be subject to asynchronous activity and thus restrict code motion and other optimizations for the variable within the scope that the ASYNCHRONOUS attribute is active. >> >> -craig >> >> On Mar 28, 2011, at 10:46 AM, Dan Nagle wrote: >> >>> Hello, >>> >>> On Mar 28, 2011, at 12:14 , Rasmussen, Craig E wrote: >>>> >>>> So the question is not about whether a compiler implements ASYNCHRONOUS I/O but whether it respects the ASYNCHRONOUS semantics. >>> >>> I think the question is whether a processor respects >>> its own asynchronous semantics. >>> >>> I would be perverse to require processor A to respect >>> what processor B (C, D, E, ...) _might_ do in some, >>> perhaps very rare, circumstances. >>> >>> Annex A is four pages. The Fortran standard is a set >>> of restrictions on programs, not processors, unless otherwise >>> explicitly stated. >>> >>> -- >>> Cheers! >>> >>> Dan Nagle >>> >>> >>> >>> >>> _______________________________________________ >>> mpi3-fortran mailing list >>> mpi3-fortran(a)lists.mpi-forum.org >>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-fortran >> >> >> _______________________________________________ >> mpi3-fortran mailing list >> mpi3-fortran(a)lists.mpi-forum.org >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-fortran > > -- > Cheers! > > Dan Nagle > > > > > _______________________________________________ > mpi3-fortran mailing list > mpi3-fortran(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-fortran

1 0

Re: [MPI3 Fortran] Nonblocking MPI and Fortran temporary memory modifications
by Dan Nagle 28 Mar '11

28 Mar '11

Hi, I think I understand the problem. I don't think you understand the objection to your solution. Expecting one processor to generate code based on what another processor *might* do is a non-starter. The whole "MPI is just like I/O" view is wrong. The devil is in the details. If you want to debate this further, I suggest we take it offline. On Mar 28, 2011, at 13:05 , Rasmussen, Craig E wrote: > Dan, > > Perhaps I haven't sufficiently explained what we need to do in MPI3, as I'm not sure what you are trying to convey with restrictions on programs versus restrictions on processors. > > The ASYNCHRONOUS attribute has a certain semantic meaning in the context of a valid program. For MPI3 we want the semantics of ASYNCHRONOUS to include that a variable with the ASYCHRONOUS attribute may be subject to asynchronous activity, not just asynchronous input/output (as currently stated in 5.3.4). Rolf's worry (at least it should be) is that the MPI3 standard can't point to a place in the Fortran standard that says a compiler/processor in not conforming if it does not also take into account that a variable with the ASYNCHRONOUS attribute may be subject to asynchronous activity and thus restrict code motion and other optimizations for the variable within the scope that the ASYNCHRONOUS attribute is active. > > -craig > > On Mar 28, 2011, at 10:46 AM, Dan Nagle wrote: > >> Hello, >> >> On Mar 28, 2011, at 12:14 , Rasmussen, Craig E wrote: >>> >>> So the question is not about whether a compiler implements ASYNCHRONOUS I/O but whether it respects the ASYNCHRONOUS semantics. >> >> I think the question is whether a processor respects >> its own asynchronous semantics. >> >> I would be perverse to require processor A to respect >> what processor B (C, D, E, ...) _might_ do in some, >> perhaps very rare, circumstances. >> >> Annex A is four pages. The Fortran standard is a set >> of restrictions on programs, not processors, unless otherwise >> explicitly stated. >> >> -- >> Cheers! >> >> Dan Nagle >> >> >> >> >> _______________________________________________ >> mpi3-fortran mailing list >> mpi3-fortran(a)lists.mpi-forum.org >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-fortran > > > _______________________________________________ > mpi3-fortran mailing list > mpi3-fortran(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-fortran -- Cheers! Dan Nagle

1 0