October 2013 - mpiwg-fortran - mailman3.cels.anl.gov

Re: [MPIWG Fortran] ticket 351
by Jeff Hammond 22 Oct '13

22 Oct '13

>>> And, lastly, MPI 3.0 already defines a way of getting this that is >>> the same in both Fortran and C - using MPI_INFO_ENV. >> >> >> If only that were true. > > > Point taken. I think that's a ghastly specification, but I can see why > you want it. > > But I still don't think that adding yet another hack on top of all of > the existing ones is going to help. My remarks that the current mechanism > isn't reliable for C weren't theoretical, incidentally, though how many > of the implementations that caused trouble are still extant and doing so > I can't say. > > Realistically, the simplest solution is to say that implementations > should take steps not to pollute the programmer's argument list, which > would force those without mpiexec equivalents to use environment variables > or equivalent. I don't think that would fly because of opposition, but > it's the only simple one I can see that would actually clean up the > current specification. I would not oppose an attempt to specify that MPI cannot mess with argc+argv but I'm not sure that will pass. We'd need input from a historian like Bill Gropp to determine the odds. It would seem that the WG is far from universally approving of the ticket and is unlikely to reach a consensus (other than a negative one) any time soon. Craig, JeffS, Rolf: Do you have any thoughts? If you don't have any counters to Bill and Nick, I think we should withdraw the ticket. Jeff -- Jeff Hammond jeff.science(a)gmail.com

1 0

Re: [MPIWG Fortran] ticket 351
by N.M. Maclaren 22 Oct '13

22 Oct '13

On Oct 22 2013, Jeff Hammond wrote: >> >> And, lastly, MPI 3.0 already defines a way of getting this that is >> the same in both Fortran and C - using MPI_INFO_ENV. > >If only that were true. Point taken. I think that's a ghastly specification, but I can see why you want it. But I still don't think that adding yet another hack on top of all of the existing ones is going to help. My remarks that the current mechanism isn't reliable for C weren't theoretical, incidentally, though how many of the implementations that caused trouble are still extant and doing so I can't say. Realistically, the simplest solution is to say that implementations should take steps not to pollute the programmer's argument list, which would force those without mpiexec equivalents to use environment variables or equivalent. I don't think that would fly because of opposition, but it's the only simple one I can see that would actually clean up the current specification. Regards, Nick Maclaren.

1 0

Re: [MPIWG Fortran] ticket 351
by Jeff Hammond 22 Oct '13

22 Oct '13

> And, lastly, MPI 3.0 already defines a way of getting this that is > the same in both Fortran and C - using MPI_INFO_ENV. If only that were true. >From https://www.open-mpi.org/doc/v1.7/man3/MPI_Info_env.3.php: argv - "The argv given for the application. If no arguments are passed to the application, then this value will not be set. It will also not be set in the case of a singleton that calls MPI_Init with NULL parameters, or a Fortran program." Furthermore, from page 359 of MPI 3.0: "The info object MPI_INFO_ENV need not contain a (key,value) pair for each of these predefined keys; the set of (key,value) pairs provided is implementation-dependent." Jeff -- Jeff Hammond jeff.science(a)gmail.com

1 0

Re: [MPIWG Fortran] ticket 351
by N.M. Maclaren 22 Oct '13

22 Oct '13

On Oct 22 2013, Jeff Hammond wrote: > >> However, I have always felt that the original C approach was a horrible >> hack, just as having C's argument list (and arguments) writable is. >> There were good reasons at the time but, given that Fortran has done >> without it with little to no trouble for so long, I agree with Bill. > >I do not see how Fortran has any power to fix this any more than C >does. The perceived superiority of Fortran over C is not a valid >argument within the MPI Forum. The MPI Forum only addressed the C >case because Fortran did not have a standard way to access argc/argv >until the 2003 specification. That's not my point. It didn't have to (ab)use it. MPI_Init could have returned an integer saying how many arguments it had used, and not updated the list at all. I know that the C standard permits the argument list and arguments to be written to, but that was because adding const would have broken K&R codes. Some people on WG14 wanted that to be explicitly undefined, and it was (is?) a seriously bad idea in some implementations. Inter alia, it is on any system where the run-time system uses related tricks. Implementing it on any system where the arguments are provided as read-only (as read-only data should be), or when any library call needs to access them, or on any system which passes references to the same arguments to all processes started by mpiexec/mpirun, is a real pain. All can be kludged up (I have), but it's totally unnecessary and horrible hackery. >I do not understand your apparent argument that argc/argv are not >useful. Do you assume that Fortran applications always use the exact >same input filename? I know many Fortran codes that do this and it's >really sad to see how they contort to provide a general workflow. I never said that, nor did I imply it. It's the hackery that I dislike. In addition to the above, MPI_Init(int *argc, char ***argv) was NOT compatible with the normative wording of C90, nor does it say that MPI_Init MUST change argc and argv. And, lastly, MPI 3.0 already defines a way of getting this that is the same in both Fortran and C - using MPI_INFO_ENV. Regards, Nick Maclaren.

1 0

Re: [MPIWG Fortran] ticket 351
by Jeff Hammond 22 Oct '13

22 Oct '13

The UChicagoan in me is compelled to say, "That's all well and good in practice but how does it work in theory?" I would expect this program to work every time the following does. #include <stdio.h> int main(int argc, char * argv[]) { for (int i=0; i<argc; i++) printf("argv[%d] = %s \n", i, argv[i]); return 0; } That this function could potentially not work as expected when launched as an MPI program is why the MPI Forum defined MPI_INIT as they did. Jeff On Tue, Oct 22, 2013 at 10:50 AM, Bill Long <longb(a)cray.com> wrote: > Here's the program: > > program test_command > > character(50) :: value > > call get_command_argument (0, value) > print *, trim(value) > > end program test_command > > Cheers, > Bill > > > > > On 10/22/13 10:44 AM, Jeff Hammond wrote: >> >> I am hesitant to assume this problem is resolved just because all >> Linux-based clusters are fine with this*. I don't think that this >> will be compiler sensitive either, so testing 5 compilers doesn't >> prove much. Testing 5 process launchers might. >> >> Best, >> >> Jeff >> >> * This is not meant to disparage Cray by suggesting their machines >> behave like Linux clusters in terms of the process environment; >> rather, this is a good thing. >> >> On Tue, Oct 22, 2013 at 10:30 AM, Bill Long <longb(a)cray.com> wrote: >>> >>> Hi Jeff, >>> >>> I admit that the Fortran standard is (intentionally) vague about what >>> constitutes a "command line". However, I tried the 5 compilers I have >>> available (Cray, Intel, PGI, gfortran, Pathscale) with a simple program >>> that >>> prints argument 0 == the "command name by which the program was invoked", >>> and in all cases the launcher text (aprun -n1 ) was ignored and the >>> program >>> printed "./a.out". Not conclusive, but suggests that the vendors have >>> figured out what the right "quality of implementation" action is here. >>> If >>> there are examples to the contrary, I'd be interested in knowing about >>> them. >>> >>> Cheers, >>> Bill >>> >>> >>> >>> On 10/22/13 10:12 AM, N.M. Maclaren wrote: >>>> >>>> >>>> On Oct 22 2013, Bill Long wrote: >>>>> >>>>> >>>>> >>>>> 2) I don't really see the point of this at all. The user can just >>>>> call the native Fortran intrinsics directly. Why should there be a >>>>> duplicate version that starts with MPI_ ? I would suggest, instead, >>>>> that you just add a note in the spec (advice to users) point out that >>>>> Fortran programmers can access the command line arguments using the >>>>> language intrinsics. Maybe someone can convince me this is not >>>>> adequate, but I've been using those intrinsics with parallel programs >>>>> for some time and they work just fine. >>>> >>>> >>>> >>>> I assume that the desire for this is to support the systems which do not >>>> use a mpiexec/mpirun command, but fire up MPI directly, as IBM poe does. >>>> >>>> However, I have always felt that the original C approach was a horrible >>>> hack, just as having C's argument list (and arguments) writable is. >>>> There were good reasons at the time but, given that Fortran has done >>>> without it with little to no trouble for so long, I agree with Bill. >>>> >>>> >>>> Regards, >>>> Nick Maclaren. >>>> >>>> _______________________________________________ >>>> mpiwg-fortran mailing list >>>> mpiwg-fortran(a)lists.mpi-forum.org >>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-fortran >>> >>> >>> >>> -- >>> Bill Long longb(a)cray.com >>> Fortran Technical Support & voice: 651-605-9024 >>> Bioinformatics Software Development fax: 651-605-9142 >>> Cray Inc./Cray Plaza, Suite 210/380 Jackson St./St. Paul, MN 55101 >>> >>> >>> _______________________________________________ >>> mpiwg-fortran mailing list >>> mpiwg-fortran(a)lists.mpi-forum.org >>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-fortran >> >> >> >> > > -- > Bill Long longb(a)cray.com > Fortran Technical Support & voice: 651-605-9024 > Bioinformatics Software Development fax: 651-605-9142 > Cray Inc./Cray Plaza, Suite 210/380 Jackson St./St. Paul, MN 55101 > > -- Jeff Hammond jeff.science(a)gmail.com

1 0

Re: [MPIWG Fortran] ticket 351
by Bill Long 22 Oct '13

22 Oct '13

Here's the program: program test_command character(50) :: value call get_command_argument (0, value) print *, trim(value) end program test_command Cheers, Bill On 10/22/13 10:44 AM, Jeff Hammond wrote: > I am hesitant to assume this problem is resolved just because all > Linux-based clusters are fine with this*. I don't think that this > will be compiler sensitive either, so testing 5 compilers doesn't > prove much. Testing 5 process launchers might. > > Best, > > Jeff > > * This is not meant to disparage Cray by suggesting their machines > behave like Linux clusters in terms of the process environment; > rather, this is a good thing. > > On Tue, Oct 22, 2013 at 10:30 AM, Bill Long <longb(a)cray.com> wrote: >> Hi Jeff, >> >> I admit that the Fortran standard is (intentionally) vague about what >> constitutes a "command line". However, I tried the 5 compilers I have >> available (Cray, Intel, PGI, gfortran, Pathscale) with a simple program that >> prints argument 0 == the "command name by which the program was invoked", >> and in all cases the launcher text (aprun -n1 ) was ignored and the program >> printed "./a.out". Not conclusive, but suggests that the vendors have >> figured out what the right "quality of implementation" action is here. If >> there are examples to the contrary, I'd be interested in knowing about them. >> >> Cheers, >> Bill >> >> >> >> On 10/22/13 10:12 AM, N.M. Maclaren wrote: >>> >>> On Oct 22 2013, Bill Long wrote: >>>> >>>> >>>> 2) I don't really see the point of this at all. The user can just >>>> call the native Fortran intrinsics directly. Why should there be a >>>> duplicate version that starts with MPI_ ? I would suggest, instead, >>>> that you just add a note in the spec (advice to users) point out that >>>> Fortran programmers can access the command line arguments using the >>>> language intrinsics. Maybe someone can convince me this is not >>>> adequate, but I've been using those intrinsics with parallel programs >>>> for some time and they work just fine. >>> >>> >>> I assume that the desire for this is to support the systems which do not >>> use a mpiexec/mpirun command, but fire up MPI directly, as IBM poe does. >>> >>> However, I have always felt that the original C approach was a horrible >>> hack, just as having C's argument list (and arguments) writable is. >>> There were good reasons at the time but, given that Fortran has done >>> without it with little to no trouble for so long, I agree with Bill. >>> >>> >>> Regards, >>> Nick Maclaren. >>> >>> _______________________________________________ >>> mpiwg-fortran mailing list >>> mpiwg-fortran(a)lists.mpi-forum.org >>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-fortran >> >> >> -- >> Bill Long longb(a)cray.com >> Fortran Technical Support & voice: 651-605-9024 >> Bioinformatics Software Development fax: 651-605-9142 >> Cray Inc./Cray Plaza, Suite 210/380 Jackson St./St. Paul, MN 55101 >> >> >> _______________________________________________ >> mpiwg-fortran mailing list >> mpiwg-fortran(a)lists.mpi-forum.org >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-fortran > > > -- Bill Long longb(a)cray.com Fortran Technical Support & voice: 651-605-9024 Bioinformatics Software Development fax: 651-605-9142 Cray Inc./Cray Plaza, Suite 210/380 Jackson St./St. Paul, MN 55101

1 0

Re: [MPIWG Fortran] ticket 351
by Jeff Hammond 22 Oct '13

22 Oct '13

I am hesitant to assume this problem is resolved just because all Linux-based clusters are fine with this*. I don't think that this will be compiler sensitive either, so testing 5 compilers doesn't prove much. Testing 5 process launchers might. Best, Jeff * This is not meant to disparage Cray by suggesting their machines behave like Linux clusters in terms of the process environment; rather, this is a good thing. On Tue, Oct 22, 2013 at 10:30 AM, Bill Long <longb(a)cray.com> wrote: > Hi Jeff, > > I admit that the Fortran standard is (intentionally) vague about what > constitutes a "command line". However, I tried the 5 compilers I have > available (Cray, Intel, PGI, gfortran, Pathscale) with a simple program that > prints argument 0 == the "command name by which the program was invoked", > and in all cases the launcher text (aprun -n1 ) was ignored and the program > printed "./a.out". Not conclusive, but suggests that the vendors have > figured out what the right "quality of implementation" action is here. If > there are examples to the contrary, I'd be interested in knowing about them. > > Cheers, > Bill > > > > On 10/22/13 10:12 AM, N.M. Maclaren wrote: >> >> On Oct 22 2013, Bill Long wrote: >>> >>> >>> 2) I don't really see the point of this at all. The user can just >>> call the native Fortran intrinsics directly. Why should there be a >>> duplicate version that starts with MPI_ ? I would suggest, instead, >>> that you just add a note in the spec (advice to users) point out that >>> Fortran programmers can access the command line arguments using the >>> language intrinsics. Maybe someone can convince me this is not >>> adequate, but I've been using those intrinsics with parallel programs >>> for some time and they work just fine. >> >> >> I assume that the desire for this is to support the systems which do not >> use a mpiexec/mpirun command, but fire up MPI directly, as IBM poe does. >> >> However, I have always felt that the original C approach was a horrible >> hack, just as having C's argument list (and arguments) writable is. >> There were good reasons at the time but, given that Fortran has done >> without it with little to no trouble for so long, I agree with Bill. >> >> >> Regards, >> Nick Maclaren. >> >> _______________________________________________ >> mpiwg-fortran mailing list >> mpiwg-fortran(a)lists.mpi-forum.org >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-fortran > > > -- > Bill Long longb(a)cray.com > Fortran Technical Support & voice: 651-605-9024 > Bioinformatics Software Development fax: 651-605-9142 > Cray Inc./Cray Plaza, Suite 210/380 Jackson St./St. Paul, MN 55101 > > > _______________________________________________ > mpiwg-fortran mailing list > mpiwg-fortran(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-fortran -- Jeff Hammond jeff.science(a)gmail.com

1 0

Re: [MPIWG Fortran] ticket 351
by Jeff Hammond 22 Oct '13

22 Oct '13

>> 2) I don't really see the point of this at all. The user can just call >> the native Fortran intrinsics directly. Why should there be a duplicate >> version that starts with MPI_ ? I would suggest, instead, that you just add >> a note in the spec (advice to users) point out that Fortran programmers can >> access the command line arguments using the language intrinsics. Maybe >> someone can convince me this is not adequate, but I've been using those >> intrinsics with parallel programs for some time and they work just fine. > > I assume that the desire for this is to support the systems which do not > use a mpiexec/mpirun command, but fire up MPI directly, as IBM poe does. I merely want to provide symmetry in the standard. This is an important aspect and why we provide all a number of features that do not necessarily make sense in isolation. > However, I have always felt that the original C approach was a horrible > hack, just as having C's argument list (and arguments) writable is. > There were good reasons at the time but, given that Fortran has done > without it with little to no trouble for so long, I agree with Bill. I do not see how Fortran has any power to fix this any more than C does. The perceived superiority of Fortran over C is not a valid argument within the MPI Forum. The MPI Forum only addressed the C case because Fortran did not have a standard way to access argc/argv until the 2003 specification. I do not understand your apparent argument that argc/argv are not useful. Do you assume that Fortran applications always use the exact same input filename? I know many Fortran codes that do this and it's really sad to see how they contort to provide a general workflow. Jeff -- Jeff Hammond jeff.science(a)gmail.com

1 0

Re: [MPIWG Fortran] ticket 351
by N.M. Maclaren 22 Oct '13

22 Oct '13

On Oct 22 2013, N.M. Maclaren wrote: > >However, I have always felt that the original C approach was a horrible >hack, just as having C's argument list (and arguments) writable is. >There were good reasons at the time but, given that Fortran has done >without it with little to no trouble for so long, I agree with Bill. Sorry about following up to myself, but I need to clarify. My position is not that the functionality is inappropriate, but that the proposal is compounding other people's mistakes. It might be the best possible solution, given Fortran's constraints on pointers, but I am not sure that it is. I have never liked Fortran's interface, which was partly forced by the restrictions in Fortran 90 that were relaxed in Fortran 95 and 2003. Regards, Nick Maclaren.

1 0

Re: [MPIWG Fortran] ticket 351
by Bill Long 22 Oct '13

22 Oct '13

Hi Jeff, I admit that the Fortran standard is (intentionally) vague about what constitutes a "command line". However, I tried the 5 compilers I have available (Cray, Intel, PGI, gfortran, Pathscale) with a simple program that prints argument 0 == the "command name by which the program was invoked", and in all cases the launcher text (aprun -n1 ) was ignored and the program printed "./a.out". Not conclusive, but suggests that the vendors have figured out what the right "quality of implementation" action is here. If there are examples to the contrary, I'd be interested in knowing about them. Cheers, Bill On 10/22/13 10:12 AM, N.M. Maclaren wrote: > On Oct 22 2013, Bill Long wrote: >> >> 2) I don't really see the point of this at all. The user can just >> call the native Fortran intrinsics directly. Why should there be a >> duplicate version that starts with MPI_ ? I would suggest, instead, >> that you just add a note in the spec (advice to users) point out that >> Fortran programmers can access the command line arguments using the >> language intrinsics. Maybe someone can convince me this is not >> adequate, but I've been using those intrinsics with parallel programs >> for some time and they work just fine. > > I assume that the desire for this is to support the systems which do not > use a mpiexec/mpirun command, but fire up MPI directly, as IBM poe does. > > However, I have always felt that the original C approach was a horrible > hack, just as having C's argument list (and arguments) writable is. > There were good reasons at the time but, given that Fortran has done > without it with little to no trouble for so long, I agree with Bill. > > > Regards, > Nick Maclaren. > > _______________________________________________ > mpiwg-fortran mailing list > mpiwg-fortran(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-fortran -- Bill Long longb(a)cray.com Fortran Technical Support & voice: 651-605-9024 Bioinformatics Software Development fax: 651-605-9142 Cray Inc./Cray Plaza, Suite 210/380 Jackson St./St. Paul, MN 55101

1 0