January 2010 - mpiwg-fortran - mailman3.cels.anl.gov

Re: [MPI3 Fortran] What to do with mpif.h in MPI-3?
by Jeff Squyres 25 Jan '10

25 Jan '10

On Jan 25, 2010, at 11:08 PM, Torsten Hoefler wrote: > D'oh, I didn't take my Fortran book on my travel but > http://www.fortran.com/F77_std/f77_std.html doesn't list any mechanism > to influence the size of an INTEGER. What am I missing? I think the key issue here is that there is no compiler in use anymore that *only* supports F77 syntax (even those that have "77" in their executable name). There hasn't been a only-supports-fortran-77-syntax compiler for many, many years (since the early-/mid-90s? That's a guess). So using (KIND=foo) syntax, even though that didn't debut until F90 or F95 (or whatever; I'm no Fortran expert) is perfectly acceptable here. -- Jeff Squyres jsquyres(a)cisco.com

1 0

Re: [MPI3 Fortran] What to do with mpif.h in MPI-3?
by Torsten Hoefler 25 Jan '10

25 Jan '10

On Mon, Jan 25, 2010 at 10:51:38PM -0500, Jeff Squyres wrote: > On Jan 25, 2010, at 10:46 PM, Torsten Hoefler wrote: > > > > There has been some discussion on the mailing list (by 1 person) since > > > ATL about *not* using an "MPI_Count" datatype (rather, use uint64_t or > > > somesuch), but my $0.02 is that we should use MPI_Count. > > Sure, the naming is an orthogonal discussion. I tend to agree to use the > > standard-conforming approach for F77 (i.e., *no* MPI_COUNT but rather > > INTERGER*8). > > Heh -- INTEGER*8 isn't standard. :-) D'oh, I didn't take my Fortran book on my travel but http://www.fortran.com/F77_std/f77_std.html doesn't list any mechanism to influence the size of an INTEGER. What am I missing? > I would think that a standards-based approach would be something like > INTEGER(KIND=MPI_COUNT_KIND), right? Ok, I thought both was allowed, but I'm voting for whatever the F77 standard supports. Best, Torsten -- bash$ :(){ :|:&};: --------------------- http://www.unixer.de/ ----- Torsten Hoefler | Postdoctoral Fellow Open Systems Lab | Indiana University 150 S. Woodlawn Ave. | Bloomington, IN, 474045, USA Lindley Hall Room 135 | +01 (812) 856-0501

1 0

Re: [MPI3 Fortran] What to do with mpif.h in MPI-3?
by Jeff Squyres 25 Jan '10

25 Jan '10

On Jan 25, 2010, at 10:46 PM, Torsten Hoefler wrote: > > There has been some discussion on the mailing list (by 1 person) since > > ATL about *not* using an "MPI_Count" datatype (rather, use uint64_t or > > somesuch), but my $0.02 is that we should use MPI_Count. > Sure, the naming is an orthogonal discussion. I tend to agree to use the > standard-conforming approach for F77 (i.e., *no* MPI_COUNT but rather > INTERGER*8). Heh -- INTEGER*8 isn't standard. :-) I would think that a standards-based approach would be something like INTEGER(KIND=MPI_COUNT_KIND), right? -- Jeff Squyres jsquyres(a)cisco.com

1 0

Re: [MPI3 Fortran] What to do with mpif.h in MPI-3?
by Torsten Hoefler 25 Jan '10

25 Jan '10

On Mon, Jan 25, 2010 at 12:29:03PM -0500, Jeff Squyres wrote: > On Jan 24, 2010, at 6:38 PM, Torsten Hoefler wrote: > > > > What is everyone's opinion on which (1-4) the Forum should go with? > > It seems like the route to go with the C-bindings (and also the general > > bindings) is not clear yet and this would decide the outcome. Because if > > we decide to take the "64-bit" (tm) approach and double the number of > > function symbols, then the issue would be much simpler. However, if we > > decide to take the int -> MPI_Count approach, then I don't see how we > > can "extend" F77 in a standard-conforming and portable manner (but I > > might miss something). > > Hmm -- maybe I missed something in ATL, but I thought we agreed that > we would do *both* things: > > - Add new versions of various MPI functions with a <suffix> appended (e.g., MPI_File_write<suffix>) and take the large count parameter > - The large count parameter in these new functions would have (in C) a parameter type of MPI_Count This is exactly what I meant with "new symbols" (such symbols would of course have a different count datatype, otherwise it wouldn't make sense). I must have missed the consensus there; all I remember is that there were strong reservations about the number of new functions. I do not remember a straw vote? > There has been some discussion on the mailing list (by 1 person) since > ATL about *not* using an "MPI_Count" datatype (rather, use uint64_t or > somesuch), but my $0.02 is that we should use MPI_Count. Sure, the naming is an orthogonal discussion. I tend to agree to use the standard-conforming approach for F77 (i.e., *no* MPI_COUNT but rather INTERGER*8). > > So depending on the outcome, I would vote: > > > > - if we choose the extra function symbol for long types: option 4 > > - if we choose the MPI_Count approach: option 3 > > Based on my understanding, I'm not sure how to parse your answer > because the current direction of the Forum is to do *both* things: new > functions *and* use MPI_Count. So *if* we decided for the extra symbols: option 4 :-). Best, Torsten -- bash$ :(){ :|:&};: --------------------- http://www.unixer.de/ ----- Torsten Hoefler | Postdoctoral Fellow Open Systems Lab | Indiana University 150 S. Woodlawn Ave. | Bloomington, IN, 474045, USA Lindley Hall Room 135 | +01 (812) 856-0501

1 0

Re: [MPI3 Fortran] What to do with mpif.h in MPI-3?
by N.M. Maclaren 25 Jan '10

25 Jan '10

On Jan 25 2010, Jeff Squyres wrote: >On Jan 24, 2010, at 6:38 PM, Torsten Hoefler wrote: > >> > What is everyone's opinion on which (1-4) the Forum should go with? >> It seems like the route to go with the C-bindings (and also the general >> bindings) is not clear yet and this would decide the outcome. Because if >> we decide to take the "64-bit" (tm) approach and double the number of >> function symbols, then the issue would be much simpler. However, if we >> decide to take the int -> MPI_Count approach, then I don't see how we >> can "extend" F77 in a standard-conforming and portable manner (but I >> might miss something). > > Hmm -- maybe I missed something in ATL, but I thought we agreed that we > would do *both* things: > > - Add new versions of various MPI functions with a <suffix> appended > (e.g., MPI_File_write<suffix>) and take the large count parameter - The > large count parameter in these new functions would have (in C) a > parameter type of MPI_Count That's upwards compatible in both C and Fortran. > There has been some discussion on the mailing list (by 1 person) since > ATL about *not* using an "MPI_Count" datatype (rather, use uint64_t or > somesuch), but my $0.02 is that we should use MPI_Count. Unquestionably! Please do not touch any of the <stdint.h> stuff with a bargepole! Stick to the basic types and MPI type names. There are many different ways where <stdint.h> integers cause portability, efficiency and other problems. Inter alia, the current revision of C is thinking of changing them in ways that could be incompatible with old code, and they are semantically incompatible with Fortran and many other languages in several respects. MPI_count gives the opportunity to evade any such chaos, whether introduced by the standard or an implementation. Regards, Nick maclaren.

1 0

Re: [MPI3 Fortran] What to do with mpif.h in MPI-3?
by Jeff Squyres 25 Jan '10

25 Jan '10

On Jan 24, 2010, at 6:38 PM, Torsten Hoefler wrote: > > What is everyone's opinion on which (1-4) the Forum should go with? > It seems like the route to go with the C-bindings (and also the general > bindings) is not clear yet and this would decide the outcome. Because if > we decide to take the "64-bit" (tm) approach and double the number of > function symbols, then the issue would be much simpler. However, if we > decide to take the int -> MPI_Count approach, then I don't see how we > can "extend" F77 in a standard-conforming and portable manner (but I > might miss something). Hmm -- maybe I missed something in ATL, but I thought we agreed that we would do *both* things: - Add new versions of various MPI functions with a <suffix> appended (e.g., MPI_File_write<suffix>) and take the large count parameter - The large count parameter in these new functions would have (in C) a parameter type of MPI_Count There has been some discussion on the mailing list (by 1 person) since ATL about *not* using an "MPI_Count" datatype (rather, use uint64_t or somesuch), but my $0.02 is that we should use MPI_Count. > So depending on the outcome, I would vote: > > - if we choose the extra function symbol for long types: option 4 > - if we choose the MPI_Count approach: option 3 Based on my understanding, I'm not sure how to parse your answer because the current direction of the Forum is to do *both* things: new functions *and* use MPI_Count. -- Jeff Squyres jsquyres(a)cisco.com

1 0

Re: [MPI3 Fortran] What to do with mpif.h in MPI-3?
by Jeff Squyres 25 Jan '10

25 Jan '10

Just a short note -- I agree with just about everything you and Bill said. -jms On Jan 22, 2010, at 6:32 PM, N.M. Maclaren wrote: > On Jan 22 2010, Bill Long wrote: > > > >INTEGER*8 is not part of any Fortran standard - though the * notation is > >popular in old codes and supported by many vendors. > > With a variety of different meanings! I have seen all of word count, > byte count, and a simple sequence from 1 upwards. > > >I'm sure there is some difference of opinion on what is meant by > >"Fortran 77 style interfaces". My opinion is that such interfaces are > >free to use more modern syntax and parameterized intrinsic types. The > >restriction is that only argument passing mechanisms, data types, and > >kinds of arrays that were in f77 should be allowed. Basically, derived > >types and dummy argument declarations that would force the caller to > >have a visible interface are not allowed. > > Yes, I agree. > > >> 2. Don't deprecate mpif.h, but do not include any new MPI-3 > >> functionality in it. This is really only a minor distinction from > >> deprecating; it's just a slightly less-strong statement meant to assure > >> Fortran programmers that we are NOT abandoning mpif.h. However, the road > >> to all new MPI-3 functionality is via the proposed MPI-3 explicit > >> bindings (i.e., "use mpi3"). > > > >I like this better than (1) - it is more honest. It makes it clear that > >mpif.h will continue to work in legacy codes, which I think is an > >important message. > > Again, I agree. The reason to preserve the old interfaces is for old code. > There's no need to be dogmatic about this, and trivial extensions could be > added where necessary, but the general rule would be that they wouldn't be. > > Also, I think that this will be by far the least hassle. > > >> 3. Include all new MPI-3 functionality in mpif.h *except* for the > >> "large count" functionality. This allows mpif.h-using applications to > >> get most of the MPI-3 goodness, with the exception that only INTEGER > >> counts will be supported. If your app needs larger counts than that, you > >> must "use mpi3". > > > >I suspect there could be more "exceptions" uncovered as work progressed. > > I think it is simpler to be all one way or the other. > > Non-blocking buffer attributes, to name just one. -- Jeff Squyres jsquyres(a)cisco.com

1 0

Re: [MPI3 Fortran] One more MPI Fortran thing...
by Jeff Squyres 25 Jan '10

25 Jan '10

Puri -- are you involved with the MPI-3 hybrid working group? I have not been following it closely -- are they already talking about such issues? On Jan 22, 2010, at 2:47 PM, Bill Long wrote: > > > Craig Rasmussen wrote: > > This is a good question that Bill Long should respond to. > > > > Bill has long claimed that Cray's coarray implementation works fine with their MPI library. I'm not sure what this means precisely. > Certainly there is no problem with using the MPI API with a coarray > program that I can think of because MPI would just do send/recv using > local memory > (but I can imagine some interesting race conditions if users aren't > careful). > I would guess that mpirun would have to know it is running a coarray > program so that the two runtimes don't interfere with each other. > > > > It is not necessarily the case that there would be "two runtimes". > Vendors have done the work for MPI to launch and execute a parallel > program consisting of replicated instances of the program running > concurrently and communicating with each other. Fortran 2008 coarrays > use the same basic model. There is some additional infrastructure > needed for Fortran (and UPC), but this is incremental, not conflicting. > It is quite reasonable, though not required by the Fortran standard, > that the vendor would just enhance their current runtime and use it for > all of MPI, Fortran 2008, and UPC. Cray calls the program launcher > 'aprun' rather than 'mpirun', promoting the idea that the same > infrastructure is used for all. Such reuse saves development costs, > which the bean counters like, and has the desirable side effect of > facilitating execution of mixed codes that include any combination of > MPI, Fortran 2008, or UPC (or SHMEM, for that matter). I don't think > any of the relevant standards specify this sort of commonality, but I > expect customers to effectively require this capability. > > > > I think running a hybrid MPI and coarray program will be common as MPI has more functionality at the moment (and probably always as it is library based). > > > > I agree, but I think 'hybrid' needs some clarification. I see two, > rather distinct cases: > > 1) Multi-level parallelism. Today this is characterized by MPI > programs that use OpenMP within each rank to implement an additional > layer of local, shared-memory parallelism. This has been around for > years. It is quite reasonable to expect the use of OpenMP within a > Fortran 2008 image in just the same way. Several years ago I saw a > comparison of running N MPI ranks with M-way OpenMP parallelism within > each rank and the same code with OpenMP turned off and run only with > MPI using N*M ranks (same total number of processors). In the past, the > all-MPI version tended to have better performance. However, with the > more recent trends in core counts within a socket (4, 6, 8, 12, ... > hundreds for a GPU) the balance is turning the other way. I would > expect to see more local, shared-memory parallelism within an MPI > rank/Fortran image. It might be from OpenMP (well established, mostly > portable), automatic detection by the compiler (my preference), the new > do concurrent construct in Fortran 2008 (also fine), or even CUDA/OpenCL > (which I don't consider reasonable options for 'normal' programmers, but > OK for BLAS-type library writers). > > 2) Co-existing parallelism at the same level. This would entail using > Fortran 2008 parallelism with coarrays, or UPC, as an alternate > mechanism to implement parallelism at the same level as MPI. There is a > one-to-one correspondence between the MPI rank and Fortran image / UPC > process. This could be going on without the programmer's knowledge (for > example, parts of the Scalapack library might be written using > coarrays), or by the programmer explicitly mixing the methods in the > same program. > > Assuming that Craig's comment was really about option (2), I would > expect that this will be very common. MPI has been around for many > years, it has proven to be the most successful scheme so far for > parallelism that does not reply on shared memory, and hundreds > (thousands?) of programmer-centuries have been invested in creating huge > MPI-based codes. With its combination of success and inertia, MPI is > not going away for a very very long time, if ever. And it is not likely > that a large MPI code will suddenly be scrapped and rewritten entirely > with Fortran 2008. Instead, initially some section of a code might be > rewritten using coarrays, leaving the rest of the code alone. This has > already happened for users with early access to coarrays. This is a > quite reasonable scheme for gradually evolving a code. (An potential > candidate that comes to mind is in WRF, where the transfer of a very > large data structure is coded as hundreds of calls to 'pack' routines > followed by a bulk MPI Send call for the resulting buffer, and on the > receive side a mirror set of 'unpack' routines. The coarray alternative > is a single assignment statement.) > > As suggested by the end of Craig's comment, the reverse evolution is > also possible. A new code might be developed using coarrays, but might > call a highly optimized MPI routine for a particular collective > communication pattern. This could be explicit, or hidden in a set of > wrapper library routines that internally take advantage of the work that > has gone into MPI optimization. The Fortran committee has discussed > the inclusion of collective intrinsics in the standard (the topic is > currently deferred to a TR). One school of though is that at least the > bulk of these should be provided by a LaPack-style library outside the > language spec. If we go that route (currently a minority view), a > vendor might very well leverage the existing MPI work inside the library. > > > Cheers, > Bill > > > > > -craig > > > > On Jan 21, 2010, at 6:42 PM, Purushotham Bangalore wrote: > > > >> The other issue that I remember after the meeting: Since co-arrays will be part of the F08 standard, are they any potential issues that we need to be aware of? Or will the solutions from the Hybrid programming WG solve any potential problems that might arise? > >> > >> Thanks > >> Puri > >> ----- Original Message ----- > >> From: "Jeff Squyres" <jsquyres(a)cisco.com> > >> To: "Craig Rasmussen" <crasmussen(a)newmexicoconsortium.org> > >> Cc: "Craig Rasmussen" <crasmussen(a)lanl.gov>, "Purushotham Bangalore" <puri(a)cis.uab.edu> > >> Sent: Thursday, January 21, 2010 7:04:05 PM > >> Subject: One more MPI Fortran thing... > >> > >> Craig -- I forgot to mention this on the phone today: Puri asked in the Atlanta meeting today if we need to say anything about F08 in MPI-3. I.e., do we need a paragraph/page/section/whatever about any issues that we expect might occur with using the explicit interface with F08 compilers? > >> > >> -- > >> Jeff Squyres > >> jsquyres(a)cisco.com > >> > > > > > > _______________________________________________ > > mpi3-fortran mailing list > > mpi3-fortran(a)lists.mpi-forum.org > > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-fortran > > -- > Bill Long longb(a)cray.com > Fortran Technical Support & voice: 651-605-9024 > Bioinformatics Software Development fax: 651-605-9142 > Cray Inc./Cray Plaza, Suite 210/380 Jackson St./St. Paul, MN 55101 > > > _______________________________________________ > mpi3-fortran mailing list > mpi3-fortran(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-fortran > -- Jeff Squyres jsquyres(a)cisco.com

1 0

Re: [MPI3 Fortran] What to do with mpif.h in MPI-3?
by Torsten Hoefler 24 Jan '10

24 Jan '10

Jeff, > It seems that there is a spectrum of 4 options: > > 1. Deprecate mpif.h and MPI's implicit Fortran interfaces. This is NOT removing mpif.h support from the spec or any implementation -- deprecating just states that the standard *MAY* remove them sometime in the future. If we went this route, I'm sure that it would be a LOOOOONG time before mpif.h is removed from the spec because of the millions of lines of existing MPI codes out there that use it. > > 2. Don't deprecate mpif.h, but do not include any new MPI-3 functionality in it. This is really only a minor distinction from deprecating; it's just a slightly less-strong statement meant to assure Fortran programmers that we are NOT abandoning mpif.h. However, the road to all new MPI-3 functionality is via the proposed MPI-3 explicit bindings (i.e., "use mpi3"). > > 3. Include all new MPI-3 functionality in mpif.h *except* for the "large count" functionality. This allows mpif.h-using applications to get most of the MPI-3 goodness, with the exception that only INTEGER counts will be supported. If your app needs larger counts than that, you must "use mpi3". > > 4. Include all new MPI-3 functionality in mpif.h, including "large count" functionality. > > What is everyone's opinion on which (1-4) the Forum should go with? It seems like the route to go with the C-bindings (and also the general bindings) is not clear yet and this would decide the outcome. Because if we decide to take the "64-bit" (tm) approach and double the number of function symbols, then the issue would be much simpler. However, if we decide to take the int -> MPI_Count approach, then I don't see how we can "extend" F77 in a standard-conforming and portable manner (but I might miss something). So depending on the outcome, I would vote: - if we choose the extra function symbol for long types: option 4 - if we choose the MPI_Count approach: option 3 All the Best, Torsten -- bash$ :(){ :|:&};: --------------------- http://www.unixer.de/ ----- Torsten Hoefler | Postdoctoral Fellow Open Systems Lab | Indiana University 150 S. Woodlawn Ave. | Bloomington, IN, 474045, USA Lindley Hall Room 135 | +01 (812) 856-0501

1 0

Re: [MPI3 Fortran] What to do with mpif.h in MPI-3?
by N.M. Maclaren 22 Jan '10

22 Jan '10

On Jan 22 2010, Bill Long wrote: > >INTEGER*8 is not part of any Fortran standard - though the * notation is >popular in old codes and supported by many vendors. With a variety of different meanings! I have seen all of word count, byte count, and a simple sequence from 1 upwards. >I'm sure there is some difference of opinion on what is meant by >"Fortran 77 style interfaces". My opinion is that such interfaces are >free to use more modern syntax and parameterized intrinsic types. The >restriction is that only argument passing mechanisms, data types, and >kinds of arrays that were in f77 should be allowed. Basically, derived >types and dummy argument declarations that would force the caller to >have a visible interface are not allowed. Yes, I agree. >> 2. Don't deprecate mpif.h, but do not include any new MPI-3 >> functionality in it. This is really only a minor distinction from >> deprecating; it's just a slightly less-strong statement meant to assure >> Fortran programmers that we are NOT abandoning mpif.h. However, the road >> to all new MPI-3 functionality is via the proposed MPI-3 explicit >> bindings (i.e., "use mpi3"). > >I like this better than (1) - it is more honest. It makes it clear that >mpif.h will continue to work in legacy codes, which I think is an >important message. Again, I agree. The reason to preserve the old interfaces is for old code. There's no need to be dogmatic about this, and trivial extensions could be added where necessary, but the general rule would be that they wouldn't be. Also, I think that this will be by far the least hassle. >> 3. Include all new MPI-3 functionality in mpif.h *except* for the >> "large count" functionality. This allows mpif.h-using applications to >> get most of the MPI-3 goodness, with the exception that only INTEGER >> counts will be supported. If your app needs larger counts than that, you >> must "use mpi3". > >I suspect there could be more "exceptions" uncovered as work progressed. > I think it is simpler to be all one way or the other. Non-blocking buffer attributes, to name just one. Regards, Nick Maclaren.

1 0