February 2009 - mpiwg-fortran - mailman3.cels.anl.gov

Re: [MPI3 Fortran] MPI non-blocking transfer
by Rolf Rabenseifner 09 Feb '09

09 Feb '09

I do not understand, why the DD trick should not work together in the cases of MPI_Isend + MPI_Wait + DD (to prohibit overwriting before MPI_Wait) and MPI_Irecv + MPI_Wait + DD (to prohibit access to buf before MPI_Wait) The trick is part of the MPI standard. Your email may mean that the MPI standard is broken. The advice to users is always to use all other methods mentioned in the standard (e.g. also COMMON blocks), before using VOLATILE. Best regards Rolf On Mon, 9 Feb 2009 15:06:51 -0800 Aleksandar Donev <donev1(a)llnl.gov> wrote: > On Monday 09 February 2009 14:57, Rolf Rabenseifner wrote: >> ï¿½ ï¿½ ï¿½ 2.3 The existing solution must continue to work, i.e., >> ï¿½ ï¿½ ï¿½ ï¿½ ï¿½ all existing and correct MPI applications must continue >> ï¿½ ï¿½ ï¿½ ï¿½ ï¿½ to work. > There is no "existing solution", unless you count VOLATILE, which is > only standard as of Fortran 2003. As you say, >> ï¿½ ï¿½ ï¿½ 2.4 It is not my goal to automatically correct existing wrong >> ï¿½ ï¿½ ï¿½ ï¿½ ï¿½ MPI applications, i.e., applications without the DD trick > the DD thing is a trick (I call them hacks), and actually does not >solve > the full problem, e.g., Nick has pointed out that problems can occur >at > the site of the call to MPI_Isend, *not* just the site of the wait >(but > the cases when it fails are marginal and likely do not happen often > enough to notice). > Best, > Aleks > -- > Aleksandar Donev, Ph.D. > Lawrence Postdoctoral Fellow @ Lawrence Livermore National >Laboratory > High Performance Computational Materials Science and Chemistry > E-mail: donev1(a)llnl.gov > Phone: (925) 424-6816 Fax: (925) 423-0785 > Address: P.O.Box 808, L-367, Livermore, CA 94551-9900 > Web: http://cherrypit.princeton.edu/donev Dr. Rolf Rabenseifner . . . . . . . . . .. email rabenseifner(a)hlrs.de High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner Nobelstr. 19, D-70550 Stuttgart, Germany . (Office: Allmandring 30)

1 0

Re: [MPI3 Fortran] MPI buffer arguments - void and call-by-reference
by Rolf Rabenseifner 09 Feb '09

09 Feb '09

Aleksandar, Yes, the call-by-reference problem is only a problem for non-blocking routines. With the blocking routines, there isn't a problem with copying. Especially with strided subarrays, copying is a feature, i.e., existing MPI code send such arrays with the MPI datatpye description "contiguous array". What you need also to know: I'm not a Fortran specialist. Therefore, it can be that one of the existing features already solve the MPI problems, and I do not know about. I expect, that TARGET, VOLATILE, and ASYNCHRONOUS have more implications on the optimization than an explicit requiring of call-by-reference on a routine call. My major concern: The current F77 behavior together with the Call-by-reference compiler option must be conserved. Are you also at the Fortran meeting together with Craig? Best regards Rolf On Mon, 9 Feb 2009 15:12:17 -0800 Aleksandar Donev <donev1(a)llnl.gov> wrote: > On Monday 09 February 2009 14:59, Rolf Rabenseifner wrote: > >> ï¿½ ï¿½ Fortran compilers are allowed to choose >> call-by-incopy-and-outcopy instead of call-by-reference. > There are already at least two cases where Fortran compilers are not > allowed to choose. These are when both the actual and the dummy have > the attribute: TARGET, VOLATILE, and ASYNCHRONOUS. Perhaps adding >yet > another attribute where the compiler does not have freedom is >useful, > but I am yet to be convinced. > > So first I want to ask: Other than with non-blocking transfer (which >was > the subject of a separate message), what is the problem with copy > in/out? If it is a performance issue, than that is within the realm >of > compiler optimization: As you said, the compiler has the choice and >it > will usually choose the faster (often compilers include a run time > check to see if the actual is contiguous). > >> ï¿½ ï¿½ Meaning: Same as what currently done, i.e., call-by-reference >> ï¿½ ï¿½ on the caller side, if call-by-reference is possible (i.e., >> ï¿½ ï¿½ not for non-contiguous subarrays) > What if call-by-reference is not possible? > How is this different from what is currently there and why is it >useful > enough over what is already there to warrant inclusion in the >Fortran > standard? > > Thanks, > Aleks > > -- > Aleksandar Donev, Ph.D. > Lawrence Postdoctoral Fellow @ Lawrence Livermore National >Laboratory > High Performance Computational Materials Science and Chemistry > E-mail: donev1(a)llnl.gov > Phone: (925) 424-6816 Fax: (925) 423-0785 > Address: P.O.Box 808, L-367, Livermore, CA 94551-9900 > Web: http://cherrypit.princeton.edu/donev Dr. Rolf Rabenseifner . . . . . . . . . .. email rabenseifner(a)hlrs.de High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner Nobelstr. 19, D-70550 Stuttgart, Germany . (Office: Allmandring 30)

1 0

Re: [MPI3 Fortran] MPI buffer arguments - void and call-by-reference
by N.M. Maclaren 09 Feb '09

09 Feb '09

On Feb 9 2009, Aleksandar Donev wrote: >On Monday 09 February 2009 14:59, Rolf Rabenseifner wrote: > >> Meaning: Same as what currently done, i.e., call-by-reference >> on the caller side, if call-by-reference is possible (i.e., >> not for non-contiguous subarrays) > >What if call-by-reference is not possible? >How is this different from what is currently there and why is it useful >enough over what is already there to warrant inclusion in the Fortran >standard? As I teach in my Fortran course, those are not the only two possible options. Fortran permits Algol call-by-name (i.e. the mechanism used for Jensen's device), for example. Regards, Nick Maclaren, University of Cambridge Computing Service, New Museums Site, Pembroke Street, Cambridge CB2 3QH, England. Email: nmm1(a)cam.ac.uk Tel.: +44 1223 334761 Fax: +44 1223 334679

1 0

Re: [MPI3 Fortran] MPI buffer arguments - void and call-by-reference
by Aleksandar Donev 09 Feb '09

09 Feb '09

On Monday 09 February 2009 14:59, Rolf Rabenseifner wrote: > ï¿½ ï¿½ Fortran compilers are allowed to choose > call-by-incopy-and-outcopy instead of call-by-reference. There are already at least two cases where Fortran compilers are not allowed to choose. These are when both the actual and the dummy have the attribute: TARGET, VOLATILE, and ASYNCHRONOUS. Perhaps adding yet another attribute where the compiler does not have freedom is useful, but I am yet to be convinced. So first I want to ask: Other than with non-blocking transfer (which was the subject of a separate message), what is the problem with copy in/out? If it is a performance issue, than that is within the realm of compiler optimization: As you said, the compiler has the choice and it will usually choose the faster (often compilers include a run time check to see if the actual is contiguous). > ï¿½ ï¿½ Meaning: Same as what currently done, i.e., call-by-reference > ï¿½ ï¿½ on the caller side, if call-by-reference is possible (i.e., > ï¿½ ï¿½ not for non-contiguous subarrays) What if call-by-reference is not possible? How is this different from what is currently there and why is it useful enough over what is already there to warrant inclusion in the Fortran standard? Thanks, Aleks -- Aleksandar Donev, Ph.D. Lawrence Postdoctoral Fellow @ Lawrence Livermore National Laboratory High Performance Computational Materials Science and Chemistry E-mail: donev1(a)llnl.gov Phone: (925) 424-6816 Fax: (925) 423-0785 Address: P.O.Box 808, L-367, Livermore, CA 94551-9900 Web: http://cherrypit.princeton.edu/donev

1 0

Re: [MPI3 Fortran] MPI non-blocking transfer
by Aleksandar Donev 09 Feb '09

09 Feb '09

On Monday 09 February 2009 14:57, Rolf Rabenseifner wrote: > ï¿½ ï¿½ ï¿½ 2.3 The existing solution must continue to work, i.e., > ï¿½ ï¿½ ï¿½ ï¿½ ï¿½ all existing and correct MPI applications must continue > ï¿½ ï¿½ ï¿½ ï¿½ ï¿½ to work. There is no "existing solution", unless you count VOLATILE, which is only standard as of Fortran 2003. As you say, > ï¿½ ï¿½ ï¿½ 2.4 It is not my goal to automatically correct existing wrong > ï¿½ ï¿½ ï¿½ ï¿½ ï¿½ MPI applications, i.e., applications without the DD trick the DD thing is a trick (I call them hacks), and actually does not solve the full problem, e.g., Nick has pointed out that problems can occur at the site of the call to MPI_Isend, *not* just the site of the wait (but the cases when it fails are marginal and likely do not happen often enough to notice). Best, Aleks -- Aleksandar Donev, Ph.D. Lawrence Postdoctoral Fellow @ Lawrence Livermore National Laboratory High Performance Computational Materials Science and Chemistry E-mail: donev1(a)llnl.gov Phone: (925) 424-6816 Fax: (925) 423-0785 Address: P.O.Box 808, L-367, Livermore, CA 94551-9900 Web: http://cherrypit.princeton.edu/donev

1 0

[MPI3 Fortran] MPI buffer arguments - void and call-by-reference
by Rolf Rabenseifner 09 Feb '09

09 Feb '09

This is the second part of my two mails: B) VOID buffer declaration -------------------------- B.1 Current status INTEGER :: ibuf CALL MPI_SEND(ibuf,...) REAL, DIMENSION(100) :: rarr CALL MPI_SEND(rarr,...) CALL MPI_SEND(rarr(5:30:3),...) COMPLEX, DIMENSION(10,20,30) :: carr3dim CALL MPI_SEND(carr3dim,...) TYPE buff_type SEQUENCE INTEGER, DIMENSION(3):: i_val DOUBLE PRECISION, DIMENSION(5):: d_val END TYPE buff_type TYPE (buff_type) :: struct CALL MPI_SEND(struct,...) TYPE (buff_type), DIMENSION(10,20) :: struct2dim CALL MPI_SEND(struct2dim,...) ... ... ... ... All this works and the C-writte MPI_SEND receives the address of the first data element of each buffer. Currently, there isn't a F90 interface definition (i.e. only F77 style is used) MPI has its own interface to handle the data structure behind the starting pointer, the so called MPI datatypes. Such an MPI datatype need not to describe the whole structure, i.e., there may be differences between the type information that was told the compiler and the MPI datatype information. B.2 Goals B.2.1 To allow F90 interfaces that do not break eisting codes B.2.2 To minimize the burden for MPI libraries implementers and Fortran compiler implementers. B.3 Maybe a simple but efficient solution SUBROUTINE MPI_SEND(buf,... VOID :: buf END This can be used only in a interface definition for routines that are implemented in C. There is no meaning if one wants to write the routine MPI_SEND in Fortran. Meaning of VOID buf: Exactly the same as if implicit (F77) interfaces are used. (Not needed for MPI, but a possible extension: Instead of VOID, one is using IMPLARG, meaning the same as VOID on the caller side and is also allowed on the soubroutine side. Meaning: Same argument handlich as with implicit interfaces, i.e., handing over the pointer to the data or in cases of noncontiguos subarrays only a pointer to a contiguous copy of the data. ) C) Call by reference and not by in-copy-out-copy ------------------------------------------------ It is clear, that F77 interfaces cannot do strided arrays directly with call-by-reference. For contiguous arrays and non-arrays, it isn't a problem. Fortran compilers are allowed to choose call-by-incopy-and-outcopy instead of call-by-reference. Maybe the following additional option may solve the problem: SUBROUTINE MPI_SEND(buf,... VOID, CALLBYREFERENCE :: buf Meaning: Same as what currently done, i.e., call-by-reference on the caller side, if call-by-reference is possible (i.e., not for non-contiguous subarrays) Dr. Rolf Rabenseifner . . . . . . . . . .. email rabenseifner(a)hlrs.de High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner Nobelstr. 19, D-70550 Stuttgart, Germany . (Office: Allmandring 30)

1 0

[MPI3 Fortran] MPI non-blocking transfer
by Rolf Rabenseifner 09 Feb '09

09 Feb '09

I hope this mail may help to bring the Fortran and MPI sides on this topic more together. With Fortran, I see 3 major problems: A) The hidden buffer access in non-blocking routines B) VOID buffer declaration C) Call by reference and not by in-copy-out-copy I split it off into two mails. Here A) The hidden buffer access in non-blocking routines ---------------------------------------------------- When I uderestand correctly, then the discussion on mpi3-fortran(a)lists.mpi-forum.org, sc22wg5(a)open-std.org, and j3(a)j3-fortran.org could not solve the problems. First I want to summarize the problem with 3 examples that fully fit to the current MPI-2.1 standard rules: A1) buf(1)=0 CALL MPI_IRECV(buf, 1, MPI_REAL , imsg) ! doing something with without accessing the related part of buf, e.g., buf(2) = 2 CALL MPI_WAIT(imsg) CALL DD(buf) or CALL DD(buf(1)) PRINT *, buf(1),buf(2) with separately compiled routine DD, i.e., no chance for the compiler, to see that DD is doing nothing. SUBROUTINE DD(buf) RETURN END A2) SUBROUTINE xxx REAL, DIMENSION(2) :: buf buf(1)=0 CALL myIRECV(buf, 1, MPI_REAL , imsg) ! doing something with without accessing the related part of buf, e.g., buf(2) = 2 CALL myWAIT(imsg) CALL DD(buf) or CALL DD(buf(1)) PRINT *, buf(1),buf(2) END SOUBROUTINE myIRECV(buf, cnt, dt , imsg) CALL MPI_IRECV(buf, cnt, dt , imsg) END SOUBROUTINE myWAIT(imsg) CALL MPI_WAIT(imsg) END with separately compiled routine DD, i.e., no chance for the compiler, to see that DD is doing nothing. SUBROUTINE DD(buf) RETURN END A3) SUBROUTINE xxx REAL, DIMENSION(2) :: buf buf(1)=0 CALL myIRECV(buf, 1, MPI_REAL , imsg) ! doing something with without accessing the related part of buf, e.g., buf(2) = 2 CALL myDDWAIT(imsg, buf) PRINT *, buf(1),buf(2) END SOUBROUTINE myIRECV(buf, cnt, dt , imsg) CALL MPI_IRECV(buf, cnt, dt , imsg) END with separately compiled routine DD, i.e., no chance for the compiler, to see that DD is doing nothing. SOUBROUTINE myDDWAIT(imsg, buf) CALL MPI_WAIT(imsg) END 1. Questions are: 1.1 Are the solutions with soubroutine DD and myDDWAIT correct? 1.2 Can we do it better, especially without the performance-lost through the additional subroutine call to DD or through handling the additional argument buf in the call to myDDWAIT? 2. My goals in this discussion: 2.1 Find a solution that answers 1.2 with YEs. 2.2 For me, it is still okay, that the user must do something. The new solution need not to be easier, it should have better performance. 2.3 The existing solution must continue to work, i.e., all existing and correct MPI applications must continue to work. 2.4 It is not my goal to automatically correct existing wrong MPI applications, i.e., applications without the DD trick or VOLATILE buf. 2.5 No loss of performance in the rest of the application, especially any numerics with buf should be optimized in exactly the same way as today. 3. The idea with VOLATILE SUBROUTINE MPI_WAIT 3.1 After all the discussion, it seems that it is hard to meet Goal 2.5 3.2 In Example A1 and A2, the user can remove the CALL DD, But in Example A2, the user has to declare myWAIT as VOLATILE SUBR. 3.3 Implication: The user has still to understand the problem and to act in a different (VOLATILE SUBROUTINE) way as in the past (CALL DD). 3.4 How can we restrict the VOLATILE SUBROUTINE to only some variables, here "buf"? If there is a solution, then the user seems to be still involved, e.g., telling "buf". 4. My idea to this problem: Allow additional arguments that are "DUMMIES". DUMMY means, that in the call to the routine, it is used, but it never arrives in the called routine. 4.1 SUBROUTINE MPI_WAIT_B(imsg, buf) VOID, DUMMY :: buf or 4.2 SUBROUTINE MPI_WAIT_B(imsg, DUMMY) 4.3 SUBROUTINE MPI_WAITALL_B(cnt,imsg, DUMMYLIST) DUMMY and DUMMYLIST are new keywords (which should be carefully chosen) DUMMYLIST is only allowed at the end of an argument list. With DUMMYLIST, no argument checking will occor on the additional arguments. Any number (including zero) of additional arguments at the end of the list is allowed. DUMMY on the argumentlist is identical with one "VOID,DUMMY" argument. If an argument checking on a dummy argument is wished, then, e.g., REAL,DUMMY::buf can be used. With 4.1 (but not with 4.2 or 4.3), an INTENT IN or OUT may be defined in the interface definition. Rules when calling a routine that has DUMMY arguments in the interface definition: 4.a. DUMMY arguments are not handed over to the called routine 4.b. The used DUMMY arguments are handled in the calling routine as if they were handed over to the called routine and as if the called routine has modified the content of the argument (This may be restricted through the use of INTENT). Rules for the body of the called routine: 4.c The DUMMY arguments are not accessible. 4.d The argument list consists only of the arguments that are not marked as DUMMY. Especially, when the called routine is written in C, this is important. Rules, if no interface definition is made (i.e. old F77 style is used): 4.e DUMMY arguments are handed over. This is necessary because the calling routine does know anything about the clled interface. Dr. Rolf Rabenseifner . . . . . . . . . .. email rabenseifner(a)hlrs.de High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner Nobelstr. 19, D-70550 Stuttgart, Germany . (Office: Allmandring 30)

1 0

Re: [MPI3 Fortran] MPI Fortran meeting next week?
by N.M. Maclaren 08 Feb '09

08 Feb '09

On Feb 6 2009, Craig Rasmussen wrote: >I didn't ask for one but I just sent email to Rolf to see if we >couldn't have a conference call somehow with between the members at >the J3 Fortran standards committee and those interested in Fortran at >the MPI meeting. I'll be at the J3 meeting and we can all share a >phone there. Is there any activity going on by Email? I have seen some organisational messages, but the only other ones are threads copied from the SC22WG5 list. For reasons unconnected with the current financial depression (but enhanced by that), travel funds are damn-near impossible for me to get. Regards, Nick Maclaren, University of Cambridge Computing Service, New Museums Site, Pembroke Street, Cambridge CB2 3QH, England. Email: nmm1(a)cam.ac.uk Tel.: +44 1223 334761 Fax: +44 1223 334679

1 0

Re: [MPI3 Fortran] MPI Fortran meeting next week?
by Jeff Squyres 06 Feb '09

06 Feb '09

Do the times match up? On Feb 6, 2009, at 6:20 PM, Craig Rasmussen wrote: > I didn't ask for one but I just sent email to Rolf to see if we > couldn't have a conference call somehow with between the members at > the J3 Fortran standards committee and those interested in Fortran > at the MPI meeting. I'll be at the J3 meeting and we can all share > a phone there. > > Cheers, > Craig > > On Feb 6, 2009, at 3:29 AM, Jeff Squyres wrote: > >> Craig -- >> >> I see the Fortran WG listed on the agenda for next week at the San >> Jose meeting (http://meetings.mpi-forum.org/secretary/2009/02/agenda.php >> ). Will there actually be a Fortran WG meeting next week? >> >> -- >> Jeff Squyres >> Cisco Systems >> >> _______________________________________________ >> mpi3-fortran mailing list >> mpi3-fortran(a)lists.mpi-forum.org >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-fortran > > _______________________________________________ > mpi3-fortran mailing list > mpi3-fortran(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-fortran -- Jeff Squyres Cisco Systems

1 0

Re: [MPI3 Fortran] MPI Fortran meeting next week?
by Craig Rasmussen 06 Feb '09

06 Feb '09

I didn't ask for one but I just sent email to Rolf to see if we couldn't have a conference call somehow with between the members at the J3 Fortran standards committee and those interested in Fortran at the MPI meeting. I'll be at the J3 meeting and we can all share a phone there. Cheers, Craig On Feb 6, 2009, at 3:29 AM, Jeff Squyres wrote: > Craig -- > > I see the Fortran WG listed on the agenda for next week at the San > Jose meeting (http://meetings.mpi-forum.org/secretary/2009/02/agenda.php > ). Will there actually be a Fortran WG meeting next week? > > -- > Jeff Squyres > Cisco Systems > > _______________________________________________ > mpi3-fortran mailing list > mpi3-fortran(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-fortran

1 0