January 2009 - mpiwg-fortran - mailman3.cels.anl.gov

Re: [MPI3 Fortran] (j3.2006) (SC22WG5.3896) [ukfortran] [MPI3 Fortran] MPI non-blocking transfers
by N.M. Maclaren 26 Jan '09

26 Jan '09

On Jan 26 2009, Bill Long wrote: >> >Yes, but this is a concept from the days when MPI-like transfers >actually used the system's I/O hardware. Today that is increasing >archaic and misleading. Times change, and we should be willing to >change with them. A good place to start is getting rid of outdated >terminology. It helps people think more clearly about what is going >on. [BTW, when I started computing the only I/O was cards, line >printers, paper tape, magnetic tape, and disks.] MPI still does use the system I/O hardware on most systems. I/O over a TCP socket normally uses the same hardware as MPI. I/O is data transfer to or from an external location or agent, and a separate MPI process runs a separate Fortran program. And terminal connexions to computers date from 1955 - I don't know when computer-computer connexions started, but I believe that RJE was no later than about 1960. Regards, Nick Maclaren, University of Cambridge Computing Service, New Museums Site, Pembroke Street, Cambridge CB2 3QH, England. Email: nmm1(a)cam.ac.uk Tel.: +44 1223 334761 Fax: +44 1223 334679

1 0

Re: [MPI3 Fortran] (j3.2006) (SC22WG5.3896) [ukfortran] [MPI3 Fortran] MPI non-blocking transfers
by Bill Long 26 Jan '09

26 Jan '09

N.M. Maclaren wrote: > >> (The other is that it incorrectly suggests that MPI calls >> have something to do with I/O which is not the case (except for the >> actual MPI I/O calls).) >> > > As Aleks has told you, that is wrong. Stream I/O to a FIFO is a form > of I/O, and MPI point-to-point transfers and simply a form of that. > That has been accepted, and all forms of such message passing referred > to as I/O, since before any of us started computing. > > Yes, but this is a concept from the days when MPI-like transfers actually used the system's I/O hardware. Today that is increasing archaic and misleading. Times change, and we should be willing to change with them. A good place to start is getting rid of outdated terminology. It helps people think more clearly about what is going on. [BTW, when I started computing the only I/O was cards, line printers, paper tape, magnetic tape, and disks.] Cheers, Bill -- Bill Long longb(a)cray.com Fortran Technical Support & voice: 651-605-9024 Bioinformatics Software Development fax: 651-605-9142 Cray Inc., 1340 Mendota Heights Rd., Mendota Heights, MN, 55120

1 0

Re: [MPI3 Fortran] (j3.2006) (SC22WG5.3921) (SC22WG5.3917) (SC22WG5.3909) [ukfortran] [MPI3 Fortran] MPI non-blocking transfers
by N.M. Maclaren 24 Jan '09

24 Jan '09

On Jan 24 2009, Aleksandar Donev wrote: > >> The real question is whether "keep rewriting your code until the >> compiler accepts it" is a "solution" to the MPI group. Realistically, I >> think it is the where we'll end up. > >As Nick said, no one can fix broken code. If code passes a non-contigous >array to an MPI routine that cannot handle that, and copy happens, >nothing can fix that but changes to either the code or making a run-time >error in an MPI wrapper (since the routine cannot handle it---and a >compile-time error is much better than a runtime one). Likely the code >would never work correctly to begin with, so it does not sound like a >good reason to change MPI to me. Agreed. All competent MPI/Fortran courses teach their attendees not to do that, already, because it doesn't work and can't be made to. >Remember that ANY of the actually technically-feasible solutions >includes adding some attribute on the buffer array. These are not there >in existing codes. The codes will need to be changed. Bill has mentioned >some sort of implicit acquisition of said attribute but that still >smells fishy to me. Nobody has proposed any way that any of the other 'solutions' would percolate up multiple levels of call; a 'solution' that works only for one or two levels is no solution at all. If you look at real MPI codes, a large proportion have multiple levels between the MPI_Isend and the MPI_Wait. >The only actual problem is that existing constraints for asynchronous, >for a good reason, require that the actual be "simply contiguous". This >is more restrictive than "contiguous". There may be codes that rely on >"oh it will work cause it is contiguous", but to make it simply >contiguous that may require code changes (e.g., adding the CONTIGUOUS >attribute here and there). All of these are nontrivial especially for >people that do not really understand the meaning of these attributes. Yes :-( However, if there are improvements possible, they could and should also be applied to Fortran asynchronous I/O. Regards, Nick Maclaren, University of Cambridge Computing Service, New Museums Site, Pembroke Street, Cambridge CB2 3QH, England. Email: nmm1(a)cam.ac.uk Tel.: +44 1223 334761 Fax: +44 1223 334679

1 0

Re: [MPI3 Fortran] [ukfortran] (SC22WG5.3920) (j3.2006) (SC22WG5.3917) (SC22WG5.3909) [MPI3 Fortran] MPI non-blocking transfers
by N.M. Maclaren 24 Jan '09

24 Jan '09

On Jan 23 2009, Van Snyder wrote: > >> The copy-in/copy-out problem does not affect MPI_wait. There's no >> argument to copy anyway. Copy-in/copy-out is a serious problem for the >> earlier call to MPI_Isend, etc. where you do not want the copy temp >> being the buffer MPI sees, since it will get deleted when the call >> returns (which is before MPI is done with it.) This continues with any >> subsequent call to other routines that have the buffer as the argument, >> because a copy-out runs the risk of overwriting the buffer with the >> wrong values. > >The copy-in/copy-out problem appears to be pretty simple: Define the >buffer dummy argument of MPI_Isend to be ASYNCHRONOUS and assumed size. >Then if somebody hands it an assumed-shape or pointer buffer that's not >contiguous, the code falls afoul of C1239 or C1240. If the user's data >isn't contiguous, it's his responsibility to do the copying at the right >time. I agree. >I don't see the need for changes in the standard, either for the >copy-in/copy-out problem, or the code motion problem. Just define the >MPI interface properly. The only change that I think is necessary is some minor wording changes to extend the description to cover asynchronous I/O performed by a companion processor as well as Fortran asynchronous I/O. Technically, that's trivial. Regards, Nick Maclaren, University of Cambridge Computing Service, New Museums Site, Pembroke Street, Cambridge CB2 3QH, England. Email: nmm1(a)cam.ac.uk Tel.: +44 1223 334761 Fax: +44 1223 334679

1 0

Re: [MPI3 Fortran] (j3.2006) (SC22WG5.3921) (SC22WG5.3917) (SC22WG5.3909) [ukfortran] [MPI3 Fortran] MPI non-blocking transfers
by Aleksandar Donev 23 Jan '09

23 Jan '09

Bill Long wrote: > The real question is whether "keep rewriting your code until the > compiler accepts it" is a "solution" to the MPI group. Realistically, I > think it is the where we'll end up. As Nick said, no one can fix broken code. If code passes a non-contigous array to an MPI routine that cannot handle that, and copy happens, nothing can fix that but changes to either the code or making a run-time error in an MPI wrapper (since the routine cannot handle it---and a compile-time error is much better than a runtime one). Likely the code would never work correctly to begin with, so it does not sound like a good reason to change MPI to me. Remember that ANY of the actually technically-feasible solutions includes adding some attribute on the buffer array. These are not there in existing codes. The codes will need to be changed. Bill has mentioned some sort of implicit acquisition of said attribute but that still smells fishy to me. The only actual problem is that existing constraints for asynchronous, for a good reason, require that the actual be "simply contiguous". This is more restrictive than "contiguous". There may be codes that rely on "oh it will work cause it is contiguous", but to make it simply contiguous that may require code changes (e.g., adding the CONTIGUOUS attribute here and there). All of these are nontrivial especially for people that do not really understand the meaning of these attributes. > I just want to make sure the MPI > group fully understands what they are getting. Well, no one will if you subject them to this ongoing circular discussion which even the discussants can really follow! Aleks

1 0

Re: [MPI3 Fortran] (j3.2006) (SC22WG5.3917) (SC22WG5.3909) [ukfortran] [MPI3 Fortran] MPI non-blocking transfers
by Aleksandar Donev 23 Jan '09

23 Jan '09

Van Snyder wrote: > Just write an extra layer to get at MPI_wait, having an assumed-shape > buffer dummy argument with the ASYNCHRONOUS attribute and without the > CONTIGUOUS attribute. Since the buffer dummy won't be referenced, make > the procedure external, or better yet, write it in C, so a clever > optimizer won't inline it, notice the buffer isn't used, and then move > accesses to the buffer across the remaining reference to MPI_wait, which > is the present problem. > > Make this layer generic with MPI_wait. Tell users to put the buffer or > something associated with it, if one is in scope, as the associated > actual argument, and tell them why. If there were an emoticon for throwing up in disgust I would put it here! This is horrid. Why don't you just tell the user "don't use asynchronous data transfers"??? > Since we already have a solution, I don't see the need for another one, Well I do, since your solution is such a mess it is not really a solution. Yes, you can do HACKs. That is NOT a "solution". Best, Aleks

1 0

Re: [MPI3 Fortran] (j3.2006) (SC22WG5.3917) (SC22WG5.3909) [ukfortran] [MPI3 Fortran] MPI non-blocking transfers
by Bill Long 23 Jan '09

23 Jan '09

Van Snyder wrote: > > The copy-in/copy-out problem appears to be pretty simple: Define the > buffer dummy argument of MPI_Isend to be ASYNCHRONOUS and assumed size. > Then if somebody hands it an assumed-shape or pointer buffer that's not > contiguous, the code falls afoul of C1239 or C1240. If the user's data > isn't contiguous, it's his responsibility to do the copying at the right > time. > > > Yes, exactly. That was my solution #1. (The alternative, #2, was to have MPI routines with assumed-shape dummies, which requires a rewrite of the MPI library, at least at the top layer for certain routines.) The real question is whether "keep rewriting your code until the compiler accepts it" is a "solution" to the MPI group. Realistically, I think it is the where we'll end up. I just want to make sure the MPI group fully understands what they are getting. Cheers, Bill -- Bill Long longb(a)cray.com Fortran Technical Support & voice: 651-605-9024 Bioinformatics Software Development fax: 651-605-9142 Cray Inc., 1340 Mendota Heights Rd., Mendota Heights, MN, 55120

1 0

Re: [MPI3 Fortran] (j3.2006) (SC22WG5.3917) (SC22WG5.3909) [ukfortran] [MPI3 Fortran] MPI non-blocking transfers
by Van Snyder 23 Jan '09

23 Jan '09

On Fri, 2009-01-23 at 15:24 -0800, Bill Long wrote: > > Van Snyder wrote: > > On Fri, 2009-01-23 at 12:44 -0800, Bill Long wrote: > > > > > >> The problem is solved for asynchronous I/O in Fortran by exactly the > >> same means. The solution relies on C1239 and C1240 [299:12-19] in > >> 09-007. If you violate either of these constraints, your program will > >> be rejected by the compiler and you have to fix it.... If we plan to > >> base a solution on asynchronous (or volatile) for MPI, then the > >> solution is exactly the same as for asynchronous I/O in Fortran. > >> > > > > Since solutions exist for asynchronous Fortran I/O, which is > > indistinguishable from MPI hiding in an external procedure, why are we > > having this conversation? > > > > Just write an extra layer to get at MPI_wait, having an assumed-shape > > buffer dummy argument with the ASYNCHRONOUS attribute and without the > > CONTIGUOUS attribute. Since the buffer dummy won't be referenced, make > > the procedure external, or better yet, write it in C, so a clever > > optimizer won't inline it, notice the buffer isn't used, and then move > > accesses to the buffer across the remaining reference to MPI_wait, which > > is the present problem. > > > > > > The copy-in/copy-out problem does not affect MPI_wait. There's no > argument to copy anyway. Copy-in/copy-out is a serious problem for the > earlier call to MPI_Isend, etc. where you do not want the copy temp > being the buffer MPI sees, since it will get deleted when the call > returns (which is before MPI is done with it.) This continues with any > subsequent call to other routines that have the buffer as the argument, > because a copy-out runs the risk of overwriting the buffer with the > wrong values. The copy-in/copy-out problem appears to be pretty simple: Define the buffer dummy argument of MPI_Isend to be ASYNCHRONOUS and assumed size. Then if somebody hands it an assumed-shape or pointer buffer that's not contiguous, the code falls afoul of C1239 or C1240. If the user's data isn't contiguous, it's his responsibility to do the copying at the right time. I don't see the need for changes in the standard, either for the copy-in/copy-out problem, or the code motion problem. Just define the MPI interface properly.

1 0

Re: [MPI3 Fortran] (j3.2006) (SC22WG5.3917) (SC22WG5.3909) [ukfortran] [MPI3 Fortran] MPI non-blocking transfers
by Bill Long 23 Jan '09

23 Jan '09

Van Snyder wrote: > On Fri, 2009-01-23 at 12:44 -0800, Bill Long wrote: > > >> The problem is solved for asynchronous I/O in Fortran by exactly the >> same means. The solution relies on C1239 and C1240 [299:12-19] in >> 09-007. If you violate either of these constraints, your program will >> be rejected by the compiler and you have to fix it.... If we plan to >> base a solution on asynchronous (or volatile) for MPI, then the >> solution is exactly the same as for asynchronous I/O in Fortran. >> > > Since solutions exist for asynchronous Fortran I/O, which is > indistinguishable from MPI hiding in an external procedure, why are we > having this conversation? > > Just write an extra layer to get at MPI_wait, having an assumed-shape > buffer dummy argument with the ASYNCHRONOUS attribute and without the > CONTIGUOUS attribute. Since the buffer dummy won't be referenced, make > the procedure external, or better yet, write it in C, so a clever > optimizer won't inline it, notice the buffer isn't used, and then move > accesses to the buffer across the remaining reference to MPI_wait, which > is the present problem. > > The copy-in/copy-out problem does not affect MPI_wait. There's no argument to copy anyway. Copy-in/copy-out is a serious problem for the earlier call to MPI_Isend, etc. where you do not want the copy temp being the buffer MPI sees, since it will get deleted when the call returns (which is before MPI is done with it.) This continues with any subsequent call to other routines that have the buffer as the argument, because a copy-out runs the risk of overwriting the buffer with the wrong values. Cheers, Bill -- Bill Long longb(a)cray.com Fortran Technical Support & voice: 651-605-9024 Bioinformatics Software Development fax: 651-605-9142 Cray Inc., 1340 Mendota Heights Rd., Mendota Heights, MN, 55120

1 0

Re: [MPI3 Fortran] (j3.2006) (SC22WG5.3917) (SC22WG5.3909) [ukfortran] [MPI3 Fortran] MPI non-blocking transfers
by Van Snyder 23 Jan '09

23 Jan '09

On Fri, 2009-01-23 at 12:44 -0800, Bill Long wrote: > The problem is solved for asynchronous I/O in Fortran by exactly the > same means. The solution relies on C1239 and C1240 [299:12-19] in > 09-007. If you violate either of these constraints, your program will > be rejected by the compiler and you have to fix it.... If we plan to > base a solution on asynchronous (or volatile) for MPI, then the > solution is exactly the same as for asynchronous I/O in Fortran. Since solutions exist for asynchronous Fortran I/O, which is indistinguishable from MPI hiding in an external procedure, why are we having this conversation? Just write an extra layer to get at MPI_wait, having an assumed-shape buffer dummy argument with the ASYNCHRONOUS attribute and without the CONTIGUOUS attribute. Since the buffer dummy won't be referenced, make the procedure external, or better yet, write it in C, so a clever optimizer won't inline it, notice the buffer isn't used, and then move accesses to the buffer across the remaining reference to MPI_wait, which is the present problem. Make this layer generic with MPI_wait. Tell users to put the buffer or something associated with it, if one is in scope, as the associated actual argument, and tell them why. If no buffer with which the "real" buffer might be associated is accessible in the scope of the call to MPI_wait, the processor can't possibly affect the buffer by moving code that accesses it across the call to MPI_wait. This is actually *more precise* than a Fortran WAIT statement, because processors can't move acceses to ANY asynchronous variable across a WAIT statement. Since we already have a solution, I don't see the need for another one, especially not a means to say "this procedure effectively contains a WAIT statement, so don't move accesses to anything with the ASYNCHRONOUS attribute across references to it." If you want to accomplish that effect, just put a WAIT statement, maybe for a nonexistent unit (see 09-007:233:12-14), before and after the call to MPI_wait. Maybe add Note 9.52a "An optimizer cannot move accesses to any variables with the ASYNCHRONOUS attribute across a WAIT statement, even one for a nonexistent unit." Besides, Resolution T7 says "don't invent new stuff."

1 0