September 2011 - mpiwg-fortran - mailman3.cels.anl.gov

Re: [MPI3 Fortran] Questions on the F2008 profiling interface issues
by Supalov, Alexander 29 Sep '11

29 Sep '11

Thanks. Should probably go to the Fortran list as well (CC). Don't you guys think this is getting out of hand? E.g., prescribing (rather than recommending) a certain implementation path is certainly a novum. -----Original Message----- From: mpi3-tools-bounces(a)lists.mpi-forum.org [mailto:[email protected]] On Behalf Of Schulz, Martin Sent: Friday, September 30, 2011 4:36 AM To: Rasmussen, Craig E; Rolf Rabenseifner; Jeff Squyres Cc: Andreas Knï¿½pfer; Todd Gamblin; Tobias Hilbrich; MPI3 Tools Subject: [Mpi3-tools] Questions on the F2008 profiling interface issues Hi Rolf, Craig, and Jeff, I was in Dresden the last few days (visiting the Vampir group, in particular Andreas Knuepfer and Tobias Hilbrich who are responsible for their MPI wrappers) and we sat down to talk about the issues around the profiling interface in ticket #229. Here is a quick summary with a few questions that came up: * With the new Fortran symbol naming variants, there are 6 or 8 or 10 versions for every MPI subroutine. Tools could intercept all versions by providing own symbols for all versions. Now the problem: Within the wrapper function a tool needs to call the correct PMPI call which usually is the same prefix (either '_f08', '_f', '__' etc.) as the MPI call it is in. A mapping to matching C functions (as most tools do it now) is not possible due to problems with calling conventions for callback routines. * This problem is only there for Fortran, since in C we only have one interface and we are guaranteed that this is always there. * Would the following solution work? All wrappers for Fortran are written in Fortran instead of C. For every MPI function, there are 6 or 8 variants and all of them need to be provided by a tool. This would probably mean that we have to compile wrappers with all three calling conventions (mpif.h, use mpi_f and use mpi_f08) and then link them together? Then within the Fortran wrappers, one could safely call the plain Fortran name of the PMPI function. Based on the specification of the standard, the MPI implementation would then provide a matching Fortran "P" call that the Fortran compiler then needs to transform into the correct symbol name with one of the prefixes, the same way it works for the regular "MPI" routines. Does this sound right or are we missing something? Does this separate compilation of tool wrappers using all three methods and then linking them together really work? * I think you mentioned this before, but I forgot - why is the BINDC option in Chapter 16 coupled to the SUBARRAYS flag? Why are those not separate issue? * As discussed at the forum, if we want to keep our tools in C (which is probably still the easier and better option), we need to know the symbols on the "p" side and we need a portable way to figure this out. We talked about the solution of two groups of routines - one for which we have _f08 symbols for one for which we don't. Rolf, how will you add this to the standard, in particular how will you form those groups. * Once we have the groups, is the following correct? ** If the symbol pmpi_foo exists, then pmpi_bar exists for any bar ** If the symbol pmpi_foo_ exists, then pmpi_bar_ exists for any bar ** If the symbol pmpi_foo__ exists, then pmpi_bar__ exists for any bar ** If the symbol PMPI_Foo_f exists, then PMPI_Bar_f exists for any bar ** If the symbol pmpi_foo_f08 exists, then pmpi_bar_f08 exists for any bar in the group of _f08 routines defined in the standard, but does not exist for any other bar ** If the symbol pmpi_foo_f08_ exists, then pmpi_bar_f08_ exists for any bar in the group of _f08 routines defined in the standard, but does not exist for any other bar ** If the symbol pmpi_foo_f08__ exists, then pmpi_bar_f08__ exists for any bar in the group of _f08 routines defined in the standard, but does not exist for any other bar ** If the symbol PMPI_Foo_f08 exists, then PMPI_Bar_f08 exists for any bar in the group of _f08 routines defined in the standard, but does not exist for any other bar Is this correct and complete or am I missing any symbols. Does this all sound correct? We just wanted to make sure we understand all the nuances and have things right before the discussion on Monday. Thanks! Martin -- ________________________________________________________________________ Martin Schulz, schulzm(a)llnl.gov, http://people.llnl.gov/schulzm CASC @ Lawrence Livermore National Laboratory, Livermore, USA _______________________________________________ Mpi3-tools mailing list Mpi3-tools(a)lists.mpi-forum.org http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-tools -------------------------------------------------------------------------------------- Intel GmbH Dornacher Strasse 1 85622 Feldkirchen/Muenchen, Deutschland Sitz der Gesellschaft: Feldkirchen bei Muenchen Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer Registergericht: Muenchen HRB 47456 Ust.-IdNr./VAT Registration No.: DE129385895 Citibank Frankfurt a.M. (BLZ 502 109 00) 600119052

1 0

Re: [MPI3 Fortran] Fwd: Serious problem/bug in MPI libraries with the alignment of MPI_DOUBLE_PRECISION
by Supalov, Alexander 29 Sep '11

29 Sep '11

Hi, Thanks. I hope Nick and others have adequately addressed your question: the issue does not seem to totally belong into the scope of the MPI standard, and so the MPI standard may not be the right place to address it. By the way, could you please consider saying "Intel 64" instead of "IA64" in the table? Otherwise some people might think it was Itanium. Best regards. Alexander -----Original Message----- From: mpi3-fortran-bounces(a)lists.mpi-forum.org [mailto:[email protected]] On Behalf Of Rolf Rabenseifner Sent: Wednesday, September 28, 2011 7:38 PM To: MPI-3 Fortran working group Subject: Re: [MPI3 Fortran] Fwd: Serious problem/bug in MPI libraries with the alignment of MPI_DOUBLE_PRECISION Alexander and Charles, thank you for your lists. I updated the summary_out.txt . It has now three bad categories: A) 4 / 8 / 4 and B) 4 / 8 / 8 and C) 4 / 4 / 8 which is new and very strange because MPI's alignment has nothing to do with the compilers alignment. Alexander, > Why don't we leave this to the implementation and the compiler to > decide? The reason is simple: It is not possible to write portable code according to the rule in MPI-2.2 p79:7-9 "The definition of extent is motivated by the assumption that the amount of padding added at the end of each structure in an array of structures is the least needed to fulfill alignment constraints." The goal of this standardization is to be able to write portable application code. And the goal of this MPI sentence is, to be able to write such portable code without use of MPI_TYPE_CREATE_RESIZED. In C, this should work, in Fortran there are incompatibilities that should be resolved with MPI-3.0. Best regards Rolf ----- Original Message ----- > From: "Alexander Supalov" <alexander.supalov(a)intel.com> > To: "MPI-3 Fortran working group" <mpi3-fortran(a)lists.mpi-forum.org> > Sent: Wednesday, September 28, 2011 5:48:09 PM > Subject: Re: [MPI3 Fortran] Fwd: Serious problem/bug in MPI libraries > with the alignment of MPI_DOUBLE_PRECISION Hi, > > Why don't we leave this to the implementation and the compiler to > decide? Our testing results (see attachment) are all over the map on > the platforms we currently support. I'd not like to deal w/ any of > this specifically in the future product. What happens is platform > specific and looks acceptable to the users. And the standard example > may just show one possible resolution. Why bother? > > Best regards. > > Alexander > > -----Original Message----- > From: mpi3-fortran-bounces(a)lists.mpi-forum.org > [mailto:[email protected]] On Behalf Of Rolf > Rabenseifner > Sent: Wednesday, September 28, 2011 9:14 AM > To: MPI-3 Fortran working group > Subject: Re: [MPI3 Fortran] Fwd: Serious problem/bug in MPI libraries > with the alignment of MPI_DOUBLE_PRECISION > > Fab and all, > > your idea seems optimal, but ... (see later)!!!! > > In detail: > We have the problem that the Intel ***Fortran*** compiler answers with > two different values a=4 and b=8. > Both are Fortran values, a=4 for Fortran SEQUENCE derived types and > b=8 for Fortran BIND(C) derived types. > (Fortran non-sequence and non-bind(C) derived types are not really > supported by MPI). > The MPI Forum has to decide which value should be used in the example > on page 581 of > https://svn.mpi-forum.org/trac/mpi-forum-web/attachment/ticket/229/mpi > -report-F2008-2011-09-08-changeonlyplustickets.pdf > i.e., whether Alternative 1 or 2 is the correct one. > > Your proposal (I name it Alternative 3) is that the user has to choose > the mapped C basic type. > This is a non-trivial mapping. > For MPI_INTEGER it is MPI_INT or MPI_LONG? > For MPI_INTEGER8 it is MPI_LONG_LONG? > What are we doing with the unnamed predefined datatypes, produced by > MPI_TYPE_CREATE_F90_COMPLEX / _REAL / _INTEGER? > > But your proposal inspires me to propose an Alternative 4.1 and 4.2: > > We need a routine that converts a given Fortran basic named or unnamed > datatype with > > 4.1) by default "SEQUENCE alignment" into an unnamed predefined > datatype with "BIND(C) alignment"; this works together with > Alternative 1 for page 581. > MPI_F2003_CONVERT_SEQ_TO_BIND_C(IN seq_datatype, OUT bind_c_datatype) > > 4.2) by default "BIND(C) alignment" into an unnamed predefined > datatype with "SEQUENCE alignment"; this works together with > Alternative 2 for page 581. > MPI_F2003_CONVERT_BIND_C_TO_SEQ(IN bind_c_datatype, OUT seq_datatype) > > Best regards > Rolf > > ----- Original Message ----- > > From: "Fab Tillier" <ftillier(a)microsoft.com> > > To: "Rolf Rabenseifner" <rabenseifner(a)hlrs.de>, "Jeff Squyres" > > <jsquyres(a)cisco.com>, "Shinji Sumimoto" > > <s-sumi(a)labs.fujitsu.com>, "Hubert Ritzdorf" > > <hubert.ritzdorf(a)emea.nec.com>, "Howard Pritchard" > > <howardp(a)cray.com>, > > "Brian Smith" <smithbr(a)us.ibm.com>, "Charles J Archer" > > <archerc(a)us.ibm.com>, "Rajeev Thakur" <thakur(a)mcs.anl.gov>, "Bill > > Long" <longb(a)cray.com>, "Bill Gropp" <wgropp(a)uiuc.edu>, "Richard > > Graham" <rlgraham(a)ornl.gov> > > Cc: "N.M. Maclaren" <nmm1(a)cam.ac.uk>, "Iain Bason" > > <iain.bason(a)oracle.com> > > Sent: Wednesday, September 28, 2011 12:35:53 AM > > Subject: RE: Serious problem/bug in MPI libraries with the alignment > > of MPI_DOUBLE_PRECISION Hi Rolf, > > > > It seems that all the a == 4 results come from using the Intel > > Fortran compiler suite. Given that Microsoft doesn't ship a Fortran > > compiler, mandating a == 8 doesn't put us in a position to succeed. > > Therefore, I cannot support such a proposal. > > > > however, shouldn't a DOUBLE PRECISION BIND(C) derived type map to > > MPI_DOUBLE, not MPI_DOUBLE_PRECISION? > > > Alternative 3: > > Can't we simply document that MPI_DOUBLE_PRECISION (and any Fortran > > types that need similar treatment) apply only to SEQUENCE derived > > types, while BIND(C) derived types should use the C types? > > > > -Fab > > > > Rolf Rabenseifner wrote on Tue, 27 Sep 2011 at 14:38:45 > > > > > Dear all, > > > > > > thank you very much for the fast answers. > > > The result is maximal bad, > > > i.e., about 50% of the MPIs use SEQUENCE alignments and other > > > about 50% use BIND(C) based calculation of the alignment. > > > > > > Here the results (also attached as summary_out.txt). > > > Please read carefully my proposal after this summary sheet. > > > > > > In all protocols: > > > ----------------- > > > Size of one Fortran REAL = 4 > > > Size of one Fortran DOUBLE PRECISION = 8 > > > Results: > > > -------- > > > > > > a / b / c = Alignments of: > > > - DOUBLE PRECISION within a SEQUENCE derived type = a > > > - DOUBLE PRECISION within a BIND(C) derived type = b > > > - MPI_DOUBLE_PRECISION (= k_i in MPI-2.2 p.78:45) = c > > > > > > A) The 4 / 8 / 4 group = alignment of MPI_DOUBLE_PRECISION is > > > correct for (old-style) SEQUENCE derived types > > > > > > JeffSquyres_OpenMP_linux-x86-64-ifort.txt: Intel + OpenMPI on > > > x86-64 > > > Results: 4 / 8 / 4 > > > FabTillier_Intel+MicrosoftMPI_out.txt: Intel + MS-MPI on ? > > > Results: 4 / 8 / 4 > > > > > > B) The 4 / 8 / 8 group = alignment of MPI_DOUBLE_PRECISION is > > > correct for (modern) BIND(C) derived types > > > > > > Rajeev_mpich_out.ifort.x86-64.txt: Intel + mpich2 on x86-64 > > > Results: 4 / 8 / 8 > > > Rolf_asama+cray_output.txt: Intel + NEC-MPI? on asama > > > Results: 4 / 8 / 8 > > > BillLong_Cray_5compiler_out.txt: Intel + CrayMPI on Cray XE6 > > > Results: 4 / 8 / 8 > > > BrianSmith-IBM_outlog-bgp-xl.txt: Intel + IBM-XL on BGP > > > Results: 4 / 8 / 8 > > > BrianSmith-IBM_outlog-xlf.txt: xlf + IBM-XL on ? > > > Results: 4 / 8 / 8 > > > > > > C) No problems because all DOUBLE PRECISION alignments > > > are identical = 8 > > > > > > BillLong_Cray_5compiler_out.txt: PGI + CrayMPI on Cray XE6 > > > Results: 8 / 8 / 8 > > > BillLong_Cray_5compiler_out.txt: Cray + CrayMPI on Cray XE6 > > > Results: 8 / 8 / 8 > > > BillLong_Cray_5compiler_out.txt: gfortran + CrayMPI on Cray XE6 > > > Results: 8 / 8 / 8 > > > BillLong_Cray_5compiler_out.txt: pathscale + CrayMPI on Cray XE6 > > > Results: 8 / 8 / 8 > > > BrianSmith-IBM_outlog-gcc.txt: gfortran + IBM-XL on BGQ > > > Results: 8 / 8 / 8 > > > JeffSquyres_OpenMP_linux-x86-64-gfortran.txt: gfortran + OpenMPI > > > on x86-64 > > > Results: 8 / 8 / 8 > > > JeffSquyres_OpenMP_osx-x86-84-gfortran.txt: gfortran + OpenMPI on > > > osx-x86-64 > > > Results: 8 / 8 / 8 > > > Rajeev_gnu_laptop_out.txt: gfortran + mpich2 on laptop > > > Results: 8 / 8 / 8 > > > Rajeev_mpich_out.gfortran.x86-64.txt: gfortran + mpich2 on x86-64 > > > Results: 8 / 8 / 8 > > > > > > D) No problems because all DOUBLE PRECISION alignments are > > > identical = 4 > > > > > > Rajeev_mpich_out.gfortran.ia32.txt: gfortran + mpich2 on ia32 > > > Results: 4 / 4 / 4 > > > Rajeev_mpich_out.ifort.ia32.txt: Intel + mpich2 on ia32 > > > Results: 4 / 4 / 4 > > > > > > Proposal: > > > --------- > > > > > > The goal of a standard is the possibility to write portable code. > > > Therefore the answer should be standardized and not > > > implementation- > > > specific, or in other words, not defining it is the worst > > > solution. > > > > > > I expect only a few Fortran codes are using arrays of derived > > > types in Fortran. > > > The oldest library is mpich. > > > Therefore I would propose to standardize Fortran alignments are > > > based on BIND(C) which should be the same as in C. > > > > > > This would imply that IBM-XL and mpich2 and derivatives need no > > > change. > > > But it also implies that the younger OpenMPI and Microsoft-MPI > > > need a change. These implementation may add an environment switch > > > to keep the old behavior if this switch is set, but I would expect > > > that there is nearly no user who needs or want to use this switch. > > > > > > Fab and Jeff, could you live with this proposal? > > > > > > Best regards > > > Rolf > > > > > > PS: Additional remarks: > > > Yes, some compiler detect that DOUBLE PR. on 4 byte alignment > > > is > > > inefficient, but this is only an information. Other compilers > > > detect, that we use DOUBLE PRECISION in BIND(C) derived types > > > instead of using the C double through these new intrinsic C > > > types in > > > Fortran. This is also okay and all compiler work as expected, > > > they > > > use the C alignment. > > > > > > > > >> Rolf Rabenseifner wrote on Tue, 27 Sep 2011 at 09:14:33 > > >> > > >>> In my output from Cray, the compilers > > >>> - gnu > > >>> - pgi > > >>> - cray > > >>> all use 8 byte double alignment in C and Fortran. > > >>> ==> no problem with those compilers on the tested platform. > > >>> Maybe we have differences between IA64 and 32 architectures for > > >>> the same compiler. > > >>> > > >>> Important is to check MPI with those compilers that have Fortran > > >>> 4 byte double precision alignment (e.g. Intel). > > >>> > > >>> Best regards > > >>> Rolf > > >>> > > > > > >>>> On Sep 27, 2011, at 10:21 AM, Rolf Rabenseifner wrote: > > >>>> > > >>>>> Dear all, (now with attachment) > > >>>>> > > >>>>> as far as I know, you represent some MPI libraries that are > > >>>>> not directly based on another one in the list: > > >>>>> - mpich2: Rajeev > > >>>>> - OpenMPI: Jeff > > >>>>> - IBM: Rich Treumann > > >>>>> - NEC: Hubert > > >>>>> - Fujitsu: Shinji Sumimoto > > >>>>> - Microsoft: Fab > > >>>>> (Which independent library is missing on this list?) (Who of > > >>>>> the list uses already mich2 or OpenMPI as bases or does not > > >>>>> provide a Fortran binding, and can be therefore removed from > > >>>>> this list?) > > >>>>> > > >>>>> The problem is simple and has at the end only three numbers: > > >>>>> > > >>>>> Alignments of: > > >>>>> - DOUBLE PRECISION within a SEQUENCE derived type = 4 (e.g. > > >>>>> with > > >>>>> Intel) > > >>>>> - DOUBLE PRECISION within a BIND(C) derived type = 8 > > >>>>> - MPI_DOUBLE_PRECISION (= k_i in MPI-2.2 p.78:45) = 4 or 8 > > >>>>> The details: > > >>>>> Which is the alignment of Fortran DOUBLE PRECISION according > > >>>>> to k_i on MPI-2.2, page 78 line 45 and page 96 line 42. > > >>>>> > > >>>>> Since 1977 (Fortran 77) in Fortran COMMON blocks and 1990 > > >>>>> (Fortran > > >>>>> 90) > > >>>>> in Fortran SEQUENCE derived types (which have the same memory > > >>>>> layout according to the SEQUENCE-rules), the MPI alignment and > > >>>>> the Fortran alignment in such constructs should be the same, > > >>>>> e.g., the Intel compiler has alignment 4 !!! > > >>>>> > > >>>>> This may be significantly different to C, where Intel uses > > >>>>> alignment 8! > > >>>>> > > >>>>> With Fortran 2003, we got the Fortran BIND(C) derived types. > > >>>>> Here, a Fortran DOUBLE PRECISION has with Intel's compiler the > > >>>>> alignment 8. > > >>>>> > > >>>>> As far as I understand, mpich2 and Cray produce following > > >>>>> results with an Intel compiler: > > >>>>> > > >>>>> Alignments of: > > >>>>> - DOUBLE PRECISION within a SEQUENCE derived type = 4 > > >>>>> - DOUBLE PRECISION within a BIND(C) derived type = 8 > > >>>>> - MPI_DOUBLE_PRECISION (= k_i in MPI-2.2 p.78:45) = 8 > > >>>>> > > >>>>> This is definitely wrong compared to MPI-1.1 and MPI-2.0 but > > >>>>> may be helpful for MPI-3.0 when we like to switch to a > > >>>>> definition that is based on BIND(C). > > >>>>> > > >>>>> Question: > > >>>>> What is the output of the attached test program when running > > >>>>> with > > >>>>> - your MPI library > > >>>>> - and your compiler > > >>>>> - and with Intel's compiler > > >>>>> > > >>>>> Please can you send me the output of such mpiruns? > > >>>>> > > >>>>> Thanks in advance and best regards Rolf > > >>>>> > > >>>>> PS: We need not to care about 8/8/8. > > >>>>> Only 4/8/x causes problems: > > >>>>> If all those have x=8 then I'll propose to > > >>>>> adopt the MPI-3.0 to the reality. > > >>>>> If we have some implementations with 4/8/8 and > > >>>>> others with 4/8/4 then we have a serious problem > > >>>>> because only one answer can be correct, > > >>>>> and the question would be whether the historical > > >>>>> answer (x=4) or the modern answer (x=8) > > >>>>> should be chosen. > > > > > > > > -- > Dr. Rolf Rabenseifner . . . . . . . . . .. email rabenseifner(a)hlrs.de > High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 > University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 > Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner > Nobelstr. 19, D-70550 Stuttgart, Germany . (Office: Allmandring 30) > _______________________________________________ > mpi3-fortran mailing list > mpi3-fortran(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-fortran > ---------------------------------------------------------------------- > ---------------- > Intel GmbH > Dornacher Strasse 1 > 85622 Feldkirchen/Muenchen, Deutschland Sitz der Gesellschaft: > Feldkirchen bei Muenchen > Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer > Registergericht: Muenchen HRB 47456 > Ust.-IdNr./VAT Registration No.: DE129385895 Citibank Frankfurt a.M. > (BLZ 502 109 00) 600119052 > > _______________________________________________ > mpi3-fortran mailing list > mpi3-fortran(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-fortran -- Dr. Rolf Rabenseifner . . . . . . . . . .. email rabenseifner(a)hlrs.de High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner Nobelstr. 19, D-70550 Stuttgart, Germany . (Office: Allmandring 30) -------------------------------------------------------------------------------------- Intel GmbH Dornacher Strasse 1 85622 Feldkirchen/Muenchen, Deutschland Sitz der Gesellschaft: Feldkirchen bei Muenchen Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer Registergericht: Muenchen HRB 47456 Ust.-IdNr./VAT Registration No.: DE129385895 Citibank Frankfurt a.M. (BLZ 502 109 00) 600119052

1 0

Re: [MPI3 Fortran] Serious problem/bug in MPI libraries with the alignment of MPI_DOUBLE_PRECISION
by Rolf Rabenseifner 29 Sep '11

29 Sep '11

Dear all who contributed to this discussion, for MPI-3.0, I got the strong conclusion that we should not try to define anything about alignment in the combination of Fortran and MPI. I'll try to choose careful wording and advices. Thanks and best regards Rolf ----- Original Message ----- > From: "Bill Long" <longb(a)cray.com> > To: mpi3-fortran(a)lists.mpi-forum.org > Sent: Thursday, September 29, 2011 2:37:13 PM > Subject: Re: [MPI3 Fortran] Serious problem/bug in MPI libraries with the alignment of MPI_DOUBLE_PRECISION > On 9/29/11 3:22 AM, Rolf Rabenseifner wrote: > > I never talked about DOUBLE PRECISION size. > > All platforms used a size of 8 bytes. > > > > I hope that assumption is not embedded in the MPI design, since I know > of examples where default REAL is 8 bytes and DOUBLE PRECISION is 16 > bytes. > > > > The background of the problem is: > > In the Intel compiler on IA64 and IBM's xlf on Power, > > the alignment of (8 byte) DOUBLE PRECISION is > > - 4 byte within a SEQUENCE derived type > > (always the first number in my summary sheet) > > - 8 byte within a BIND(C) derived type > > (always the second number in my summary sheet) > > The rules for BIND(C) layouts are determined by the particular > compiler > that is identified as the "companion processor" for the Fortran > compiler. Usually it is the C compiler from the same vendor. These > rules do not need to match those for a SEQUENCE type. > > > > > And MPI has strong alignment rules on MPI-2.2 page 78 and 79. > > > > The problem is: > > This alignment is in some MPI libraries 4 and others 8 byte > > with those compilers (always the third number in my summary sheet). > > And a standard is not a standard if it is not standardizing > > such things. It would be only a problem-description. > > > > It depends on what kind of standard it is. If it is a standard for a > higher level language, like Fortran, that is intended to be > (relatively) > machine hardware independent, then I would expect alignment > specifications to be largely omitted as inappropriate and harmful to > efficient implementations. If the standard is for calling sequence > conventions, or for internal memory representations, the the > expectations are different. > > Cheers, > Bill > > > > > > Rolf > > > > ----- Original Message ----- > >> From: "Craig E Rasmussen"<rasmussn(a)lanl.gov> > >> To: "MPI-3 Fortran working group"<mpi3-fortran(a)lists.mpi-forum.org> > >> Sent: Wednesday, September 28, 2011 7:58:00 PM > >> Subject: Re: [MPI3 Fortran] Serious problem/bug in MPI libraries > >> with the alignment of MPI_DOUBLE_PRECISION > >> Rolf, > >> > >> > >> I don't know where you get the idea that a DOUBLE PRECISION > >> variable > >> should occupy 4 bytes. The standard says: > >> > >> > >> > >> A nonpointer scalar object of type double precision real or default > >> complex occupies two contiguous numeric storage units; > >> > >> > >> > >> The derived types we worked out in Santorini were especially > >> constructed so that double precisions numbers would be misaligned > >> if > >> the compiler was forced to do bad things via the SEQUENCE > >> statement. > >> Can you give me an idea which part of the tests we wrote indicate a > >> double precision number is 4 bytes? > >> > >> > >> Also, I checked the F2003 and F2008 standards and this wording (see > >> above) has not been changed. > >> > >> > >> -craig > >> > >> > >> > >> > >> > >> > >> On Sep 28, 2011, at 11:00 AM, Rolf Rabenseifner wrote: > >> > >> > >> > >> > >> > >> A very good question. Instead of referring to SEQUENCE, it should > >> say > >> > >> > >> that MPI_DOUBLE_PRECISION would use the numeric storage sequence > >> rules > >> > >> > >> (16.4.3.1 in Fortran 2003) for alignment. > >> > >> Nick, > >> please can you explain, what "16.4.3.1 in Fortran 2003" tells > >> about alignments. > >> In SEQUENCE derived types and COMMON blocks, > >> when I understand this correctly, then DOUBLE PRECISION > >> must have also one numerical storage unit as alignment, > >> i.e., with REAL=4 bytes, DOUBLE PRECISION alignment must be > >> also 4 byte, i.e., exactly what the Intel compiler is doing > >> in a SEQUENCE derived type or COMMON block. > >> > >> The implication is that the compiler issues nice "warnings" about > >> misaligned data. > >> > >> Do I understand correctly, that the other compilers with > >> REAL-size=4 byte and DOUBLE PRECISION size and alignment = 8 byte, > >> do it wrong, i.e.not according to Fortran 2003 or 2008? > >> > >> Best regards > >> Rolf > >> > >> ----- Original Message ----- > >> > >> > >> From: "N.M. Maclaren"< nmm1(a)cam.ac.uk> > >> > >> > >> To: "Fab Tillier"< ftillier(a)microsoft.com> > >> > >> > >> Cc: "Rolf Rabenseifner"< rabenseifner(a)hlrs.de>, "Jeff Squyres"< > >> jsquyres(a)cisco.com>, "Shinji Sumimoto" > >> > >> > >> < s-sumi(a)labs.fujitsu.com>, "Hubert Ritzdorf"< > >> hubert.ritzdorf(a)emea.nec.com>, "Howard Pritchard"< howardp(a)cray.com > >>> , > >> > >> > >> "Brian Smith"< smithbr(a)us.ibm.com>, "Charles J Archer"< > >> archerc(a)us.ibm.com>, "Rajeev Thakur"< thakur(a)mcs.anl.gov>, > >> > >> > >> "Bill Long"< longb(a)cray.com>, "Bill Gropp"< wgropp(a)uiuc.edu>, > >> "Richard Graham"< rlgraham(a)ornl.gov>, "Iain Bason" > >> > >> > >> < iain.bason(a)oracle.com> > >> > >> > >> Sent: Wednesday, September 28, 2011 10:26:37 AM > >> > >> > >> Subject: RE: Serious problem/bug in MPI libraries with the > >> alignment > >> of MPI_DOUBLE_PRECISION > >> > >> > >> On Sep 27 2011, Fab Tillier wrote: > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> It seems that all the a == 4 results come from using the Intel > >> > >> > >> > >> > >> Fortran > >> > >> > >> > >> > >> compiler suite. Given that Microsoft doesn't ship a Fortran > >> > >> > >> > >> > >> compiler, > >> > >> > >> > >> > >> mandating a == 8 doesn't put us in a position to succeed. > >> Therefore, > >> > >> > >> > >> > >> I > >> > >> > >> > >> > >> cannot support such a proposal. > >> > >> > >> > >> > >> > >> I agree that Microsoft should not be forgotten, but it is not a > >> major > >> > >> > >> platform for MPI use, and the tail should not wag the dog. We have > >> the > >> > >> > >> ghastly example of C99 long long where some people claimed > >> Microsoft > >> > >> > >> compatibility required an incompatible change to the standard; > >> largely > >> > >> > >> as a result, the official C standard is no longer used as a > >> portable > >> > >> > >> programming base by a good half of the C-using community. > >> > >> > >> > >> > >> > >> > >> > >> however, shouldn't a DOUBLE PRECISION BIND(C) derived type map to > >> > >> > >> > >> > >> MPI_DOUBLE, not MPI_DOUBLE_PRECISION? > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> Can't we simply document that MPI_DOUBLE_PRECISION (and any Fortran > >> > >> > >> > >> > >> types > >> > >> > >> > >> > >> that need similar treatment) apply only to SEQUENCE derived types, > >> > >> > >> > >> > >> while > >> > >> > >> > >> > >> BIND(C) derived types should use the C types? > >> > >> > >> > >> > >> > >> A very good question. Instead of referring to SEQUENCE, it should > >> say > >> > >> > >> that MPI_DOUBLE_PRECISION would use the numeric storage sequence > >> rules > >> > >> > >> (16.4.3.1 in Fortran 2003) for alignment. > >> > >> > >> > >> > >> > >> > >> > >> > >> Slightly related to this, can't we PLEASE deprecate those horrible > >> > >> > >> MPI_INTEGER8 etc. types? They have been a portability disaster for > >> > >> > >> half a century, even among Fortran IV compilers (where they > >> started)! > >> > >> > >> MPI has provided some perfectly good mechanisms to use Fortran's > >> > >> > >> selectable KINDs. The reason that it is related is that there is no > >> > >> > >> guarantee that a compiler will give them the same alignment as > >> another > >> > >> > >> type even if they use the same storage, because they are > >> non-standard > >> > >> > >> and therefore any specification is the compiler's choice. > >> > >> > >> > >> > >> > >> > >> > >> > >> Regards, > >> > >> > >> Nick Maclaren. > >> > >> -- > >> Dr. Rolf Rabenseifner . . . . . . . . . .. email > >> rabenseifner(a)hlrs.de > >> High Performance Computing Center (HLRS) . phone > >> ++49(0)711/685-65530 > >> University of Stuttgart . . . . . . . . .. fax ++49(0)711 / > >> 685-65832 > >> Head of Dpmt Parallel Computing . . . > >> www.hlrs.de/people/rabenseifner > >> Nobelstr. 19, D-70550 Stuttgart, Germany . (Office: Allmandring 30) > >> _______________________________________________ > >> mpi3-fortran mailing list > >> mpi3-fortran(a)lists.mpi-forum.org > >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-fortran > >> > >> > >> _______________________________________________ > >> mpi3-fortran mailing list > >> mpi3-fortran(a)lists.mpi-forum.org > >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-fortran > > > > -- > Bill Long longb(a)cray.com > Fortran Technical Support & voice: 651-605-9024 > Bioinformatics Software Development fax: 651-605-9142 > Cray Inc./Cray Plaza, Suite 210/380 Jackson St./St. Paul, MN 55101 > > > _______________________________________________ > mpi3-fortran mailing list > mpi3-fortran(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-fortran -- Dr. Rolf Rabenseifner . . . . . . . . . .. email rabenseifner(a)hlrs.de High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner Nobelstr. 19, D-70550 Stuttgart, Germany . (Office: Allmandring 30)

1 0

Re: [MPI3 Fortran] Serious problem/bug in MPI libraries with the alignment of MPI_DOUBLE_PRECISION
by Bill Long 29 Sep '11

29 Sep '11

On 9/29/11 3:22 AM, Rolf Rabenseifner wrote: > I never talked about DOUBLE PRECISION size. > All platforms used a size of 8 bytes. > I hope that assumption is not embedded in the MPI design, since I know of examples where default REAL is 8 bytes and DOUBLE PRECISION is 16 bytes. > The background of the problem is: > In the Intel compiler on IA64 and IBM's xlf on Power, > the alignment of (8 byte) DOUBLE PRECISION is > - 4 byte within a SEQUENCE derived type > (always the first number in my summary sheet) > - 8 byte within a BIND(C) derived type > (always the second number in my summary sheet) The rules for BIND(C) layouts are determined by the particular compiler that is identified as the "companion processor" for the Fortran compiler. Usually it is the C compiler from the same vendor. These rules do not need to match those for a SEQUENCE type. > > And MPI has strong alignment rules on MPI-2.2 page 78 and 79. > > The problem is: > This alignment is in some MPI libraries 4 and others 8 byte > with those compilers (always the third number in my summary sheet). > And a standard is not a standard if it is not standardizing > such things. It would be only a problem-description. > It depends on what kind of standard it is. If it is a standard for a higher level language, like Fortran, that is intended to be (relatively) machine hardware independent, then I would expect alignment specifications to be largely omitted as inappropriate and harmful to efficient implementations. If the standard is for calling sequence conventions, or for internal memory representations, the the expectations are different. Cheers, Bill > Rolf > > ----- Original Message ----- >> From: "Craig E Rasmussen"<rasmussn(a)lanl.gov> >> To: "MPI-3 Fortran working group"<mpi3-fortran(a)lists.mpi-forum.org> >> Sent: Wednesday, September 28, 2011 7:58:00 PM >> Subject: Re: [MPI3 Fortran] Serious problem/bug in MPI libraries with the alignment of MPI_DOUBLE_PRECISION >> Rolf, >> >> >> I don't know where you get the idea that a DOUBLE PRECISION variable >> should occupy 4 bytes. The standard says: >> >> >> >> A nonpointer scalar object of type double precision real or default >> complex occupies two contiguous numeric storage units; >> >> >> >> The derived types we worked out in Santorini were especially >> constructed so that double precisions numbers would be misaligned if >> the compiler was forced to do bad things via the SEQUENCE statement. >> Can you give me an idea which part of the tests we wrote indicate a >> double precision number is 4 bytes? >> >> >> Also, I checked the F2003 and F2008 standards and this wording (see >> above) has not been changed. >> >> >> -craig >> >> >> >> >> >> >> On Sep 28, 2011, at 11:00 AM, Rolf Rabenseifner wrote: >> >> >> >> >> >> A very good question. Instead of referring to SEQUENCE, it should say >> >> >> that MPI_DOUBLE_PRECISION would use the numeric storage sequence rules >> >> >> (16.4.3.1 in Fortran 2003) for alignment. >> >> Nick, >> please can you explain, what "16.4.3.1 in Fortran 2003" tells >> about alignments. >> In SEQUENCE derived types and COMMON blocks, >> when I understand this correctly, then DOUBLE PRECISION >> must have also one numerical storage unit as alignment, >> i.e., with REAL=4 bytes, DOUBLE PRECISION alignment must be >> also 4 byte, i.e., exactly what the Intel compiler is doing >> in a SEQUENCE derived type or COMMON block. >> >> The implication is that the compiler issues nice "warnings" about >> misaligned data. >> >> Do I understand correctly, that the other compilers with >> REAL-size=4 byte and DOUBLE PRECISION size and alignment = 8 byte, >> do it wrong, i.e.not according to Fortran 2003 or 2008? >> >> Best regards >> Rolf >> >> ----- Original Message ----- >> >> >> From: "N.M. Maclaren"< nmm1(a)cam.ac.uk> >> >> >> To: "Fab Tillier"< ftillier(a)microsoft.com> >> >> >> Cc: "Rolf Rabenseifner"< rabenseifner(a)hlrs.de>, "Jeff Squyres"< >> jsquyres(a)cisco.com>, "Shinji Sumimoto" >> >> >> < s-sumi(a)labs.fujitsu.com>, "Hubert Ritzdorf"< >> hubert.ritzdorf(a)emea.nec.com>, "Howard Pritchard"< howardp(a)cray.com >>> , >> >> >> "Brian Smith"< smithbr(a)us.ibm.com>, "Charles J Archer"< >> archerc(a)us.ibm.com>, "Rajeev Thakur"< thakur(a)mcs.anl.gov>, >> >> >> "Bill Long"< longb(a)cray.com>, "Bill Gropp"< wgropp(a)uiuc.edu>, >> "Richard Graham"< rlgraham(a)ornl.gov>, "Iain Bason" >> >> >> < iain.bason(a)oracle.com> >> >> >> Sent: Wednesday, September 28, 2011 10:26:37 AM >> >> >> Subject: RE: Serious problem/bug in MPI libraries with the alignment >> of MPI_DOUBLE_PRECISION >> >> >> On Sep 27 2011, Fab Tillier wrote: >> >> >> >> >> >> >> >> >> >> It seems that all the a == 4 results come from using the Intel >> >> >> >> >> Fortran >> >> >> >> >> compiler suite. Given that Microsoft doesn't ship a Fortran >> >> >> >> >> compiler, >> >> >> >> >> mandating a == 8 doesn't put us in a position to succeed. Therefore, >> >> >> >> >> I >> >> >> >> >> cannot support such a proposal. >> >> >> >> >> >> I agree that Microsoft should not be forgotten, but it is not a major >> >> >> platform for MPI use, and the tail should not wag the dog. We have the >> >> >> ghastly example of C99 long long where some people claimed Microsoft >> >> >> compatibility required an incompatible change to the standard; largely >> >> >> as a result, the official C standard is no longer used as a portable >> >> >> programming base by a good half of the C-using community. >> >> >> >> >> >> >> >> however, shouldn't a DOUBLE PRECISION BIND(C) derived type map to >> >> >> >> >> MPI_DOUBLE, not MPI_DOUBLE_PRECISION? >> >> >> >> >> >> >> >> >> >> Can't we simply document that MPI_DOUBLE_PRECISION (and any Fortran >> >> >> >> >> types >> >> >> >> >> that need similar treatment) apply only to SEQUENCE derived types, >> >> >> >> >> while >> >> >> >> >> BIND(C) derived types should use the C types? >> >> >> >> >> >> A very good question. Instead of referring to SEQUENCE, it should say >> >> >> that MPI_DOUBLE_PRECISION would use the numeric storage sequence rules >> >> >> (16.4.3.1 in Fortran 2003) for alignment. >> >> >> >> >> >> >> >> >> Slightly related to this, can't we PLEASE deprecate those horrible >> >> >> MPI_INTEGER8 etc. types? They have been a portability disaster for >> >> >> half a century, even among Fortran IV compilers (where they started)! >> >> >> MPI has provided some perfectly good mechanisms to use Fortran's >> >> >> selectable KINDs. The reason that it is related is that there is no >> >> >> guarantee that a compiler will give them the same alignment as another >> >> >> type even if they use the same storage, because they are non-standard >> >> >> and therefore any specification is the compiler's choice. >> >> >> >> >> >> >> >> >> Regards, >> >> >> Nick Maclaren. >> >> -- >> Dr. Rolf Rabenseifner . . . . . . . . . .. email rabenseifner(a)hlrs.de >> High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 >> University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 >> Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner >> Nobelstr. 19, D-70550 Stuttgart, Germany . (Office: Allmandring 30) >> _______________________________________________ >> mpi3-fortran mailing list >> mpi3-fortran(a)lists.mpi-forum.org >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-fortran >> >> >> _______________________________________________ >> mpi3-fortran mailing list >> mpi3-fortran(a)lists.mpi-forum.org >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-fortran > -- Bill Long longb(a)cray.com Fortran Technical Support & voice: 651-605-9024 Bioinformatics Software Development fax: 651-605-9142 Cray Inc./Cray Plaza, Suite 210/380 Jackson St./St. Paul, MN 55101

1 0

Re: [MPI3 Fortran] The alignment kerfluffle
by N.M. Maclaren 29 Sep '11

29 Sep '11

On Sep 28 2011, Jeff Squyres wrote: < Points omitted. > Yes. I agree with all of those. The ONLY way that these can be fixed properly by by the language itself. At best, MPI can describe practice that works on most systems that support MPI, today. > 5. Different environments have different sizes for un-sized types (e.g., > DOUBLE PRECISION and others). As Fab cited, one choice that a developer > of portable software can make is to use explicitly-sized types (e.g., > int32_t). Actually, no. All that does is to degrade the meaning of the word 'portability' to mean that it applies to only an extremely small subset of systems and requirements. If one wants to write code that is portable over time, over scale of problem or to all potential systems, then it is precisely the wrong thing to do. Also int32_t doesn't mean what most people think that it does. We have seen that practice harm portability at least a dozen times, to my personal recollection! > With these points, I conclude that nothing should be done to the > standard, except maybe an ATU advising them of the potential portability > issue, and suggestions on how to code around it. I fully agree. Most of the claims that things are required by the various standards should be changed to saying that this will work on almost all systems that support MPI, today, but may not work on absolutely everything and may not be reliable a decade hence. > We'll fix the alignment test in Open MPI and I suspect that everyone will > continue to not care. :-) Yup :-) Regards, Nick Maclaren.

1 0

Re: [MPI3 Fortran] Serious problem/bug in MPI libraries with the alignment of MPI_DOUBLE_PRECISION
by Rolf Rabenseifner 29 Sep '11

29 Sep '11

I never talked about DOUBLE PRECISION size. All platforms used a size of 8 bytes. The background of the problem is: In the Intel compiler on IA64 and IBM's xlf on Power, the alignment of (8 byte) DOUBLE PRECISION is - 4 byte within a SEQUENCE derived type (always the first number in my summary sheet) - 8 byte within a BIND(C) derived type (always the second number in my summary sheet) And MPI has strong alignment rules on MPI-2.2 page 78 and 79. The problem is: This alignment is in some MPI libraries 4 and others 8 byte with those compilers (always the third number in my summary sheet). And a standard is not a standard if it is not standardizing such things. It would be only a problem-description. Rolf ----- Original Message ----- > From: "Craig E Rasmussen" <rasmussn(a)lanl.gov> > To: "MPI-3 Fortran working group" <mpi3-fortran(a)lists.mpi-forum.org> > Sent: Wednesday, September 28, 2011 7:58:00 PM > Subject: Re: [MPI3 Fortran] Serious problem/bug in MPI libraries with the alignment of MPI_DOUBLE_PRECISION > Rolf, > > > I don't know where you get the idea that a DOUBLE PRECISION variable > should occupy 4 bytes. The standard says: > > > > A nonpointer scalar object of type double precision real or default > complex occupies two contiguous numeric storage units; > > > > The derived types we worked out in Santorini were especially > constructed so that double precisions numbers would be misaligned if > the compiler was forced to do bad things via the SEQUENCE statement. > Can you give me an idea which part of the tests we wrote indicate a > double precision number is 4 bytes? > > > Also, I checked the F2003 and F2008 standards and this wording (see > above) has not been changed. > > > -craig > > > > > > > On Sep 28, 2011, at 11:00 AM, Rolf Rabenseifner wrote: > > > > > > A very good question. Instead of referring to SEQUENCE, it should say > > > that MPI_DOUBLE_PRECISION would use the numeric storage sequence rules > > > (16.4.3.1 in Fortran 2003) for alignment. > > Nick, > please can you explain, what "16.4.3.1 in Fortran 2003" tells > about alignments. > In SEQUENCE derived types and COMMON blocks, > when I understand this correctly, then DOUBLE PRECISION > must have also one numerical storage unit as alignment, > i.e., with REAL=4 bytes, DOUBLE PRECISION alignment must be > also 4 byte, i.e., exactly what the Intel compiler is doing > in a SEQUENCE derived type or COMMON block. > > The implication is that the compiler issues nice "warnings" about > misaligned data. > > Do I understand correctly, that the other compilers with > REAL-size=4 byte and DOUBLE PRECISION size and alignment = 8 byte, > do it wrong, i.e.not according to Fortran 2003 or 2008? > > Best regards > Rolf > > ----- Original Message ----- > > > From: "N.M. Maclaren" < nmm1(a)cam.ac.uk > > > > To: "Fab Tillier" < ftillier(a)microsoft.com > > > > Cc: "Rolf Rabenseifner" < rabenseifner(a)hlrs.de >, "Jeff Squyres" < > jsquyres(a)cisco.com >, "Shinji Sumimoto" > > > < s-sumi(a)labs.fujitsu.com >, "Hubert Ritzdorf" < > hubert.ritzdorf(a)emea.nec.com >, "Howard Pritchard" < howardp(a)cray.com > >, > > > "Brian Smith" < smithbr(a)us.ibm.com >, "Charles J Archer" < > archerc(a)us.ibm.com >, "Rajeev Thakur" < thakur(a)mcs.anl.gov >, > > > "Bill Long" < longb(a)cray.com >, "Bill Gropp" < wgropp(a)uiuc.edu >, > "Richard Graham" < rlgraham(a)ornl.gov >, "Iain Bason" > > > < iain.bason(a)oracle.com > > > > Sent: Wednesday, September 28, 2011 10:26:37 AM > > > Subject: RE: Serious problem/bug in MPI libraries with the alignment > of MPI_DOUBLE_PRECISION > > > On Sep 27 2011, Fab Tillier wrote: > > > > > > > > > > It seems that all the a == 4 results come from using the Intel > > > > > Fortran > > > > > compiler suite. Given that Microsoft doesn't ship a Fortran > > > > > compiler, > > > > > mandating a == 8 doesn't put us in a position to succeed. Therefore, > > > > > I > > > > > cannot support such a proposal. > > > > > > I agree that Microsoft should not be forgotten, but it is not a major > > > platform for MPI use, and the tail should not wag the dog. We have the > > > ghastly example of C99 long long where some people claimed Microsoft > > > compatibility required an incompatible change to the standard; largely > > > as a result, the official C standard is no longer used as a portable > > > programming base by a good half of the C-using community. > > > > > > > > however, shouldn't a DOUBLE PRECISION BIND(C) derived type map to > > > > > MPI_DOUBLE, not MPI_DOUBLE_PRECISION? > > > > > > > > > > Can't we simply document that MPI_DOUBLE_PRECISION (and any Fortran > > > > > types > > > > > that need similar treatment) apply only to SEQUENCE derived types, > > > > > while > > > > > BIND(C) derived types should use the C types? > > > > > > A very good question. Instead of referring to SEQUENCE, it should say > > > that MPI_DOUBLE_PRECISION would use the numeric storage sequence rules > > > (16.4.3.1 in Fortran 2003) for alignment. > > > > > > > > > Slightly related to this, can't we PLEASE deprecate those horrible > > > MPI_INTEGER8 etc. types? They have been a portability disaster for > > > half a century, even among Fortran IV compilers (where they started)! > > > MPI has provided some perfectly good mechanisms to use Fortran's > > > selectable KINDs. The reason that it is related is that there is no > > > guarantee that a compiler will give them the same alignment as another > > > type even if they use the same storage, because they are non-standard > > > and therefore any specification is the compiler's choice. > > > > > > > > > Regards, > > > Nick Maclaren. > > -- > Dr. Rolf Rabenseifner . . . . . . . . . .. email rabenseifner(a)hlrs.de > High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 > University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 > Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner > Nobelstr. 19, D-70550 Stuttgart, Germany . (Office: Allmandring 30) > _______________________________________________ > mpi3-fortran mailing list > mpi3-fortran(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-fortran > > > _______________________________________________ > mpi3-fortran mailing list > mpi3-fortran(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-fortran -- Dr. Rolf Rabenseifner . . . . . . . . . .. email rabenseifner(a)hlrs.de High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner Nobelstr. 19, D-70550 Stuttgart, Germany . (Office: Allmandring 30)

1 0

[MPI3 Fortran] The alignment kerfluffle
by Jeff Squyres 28 Sep '11

28 Sep '11

I talked with some people about this today, re-read the entire mail threads, and summarize the takeaways from these discussions, in no particular order: 1. If the compiler can change its alignment of a given type depending on the specific situation (per NM's mail), then this is beyond the scope of MPI to "fix". 2. Open MPI's alignment test is probably wrong, but it has been wrong for a long time. Note that we used the exact same alignment test as LAM/MPI, which goes all the way back to the mid/late 90's. So it's been "wrong" for a very, very long time. Apparently, no one cares. :-) 3. Multiple people have cited on the threads that if the language itself doesn't enforce portability (in the way that Rolf is defining it), then it is madness to try to fix it in MPI. 4. As rationale for trying to fix this issue, MPI-2.2 p79 7-9 was quoted: The definition of extent is motivated by the assumption that the amount of padding added at the end of each structure in an array of structures is the least needed to fulfill alignment constraints. But the *very next sentence* says: More explicit control of the extent is provided in Section 4.1.6. Section 4.1.6 defined MPI_UB and MPI_LB and friends. This gives credence to Bill Gropp's assertion that MPI already has the tools necessary for this issue. 5. Different environments have different sizes for un-sized types (e.g., DOUBLE PRECISION and others). As Fab cited, one choice that a developer of portable software can make is to use explicitly-sized types (e.g., int32_t). ----- With these points, I conclude that nothing should be done to the standard, except maybe an ATU advising them of the potential portability issue, and suggestions on how to code around it. We'll fix the alignment test in Open MPI and I suspect that everyone will continue to not care. :-) -- Jeff Squyres jsquyres(a)cisco.com For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/

1 0

Re: [MPI3 Fortran] Fwd: Serious problem/bug in MPI libraries with the alignment of MPI_DOUBLE_PRECISION
by N.M. Maclaren 28 Sep '11

28 Sep '11

On Sep 28 2011, Rolf Rabenseifner wrote: >Alexander, > >> Why don't we leave this to the implementation and the compiler to >> decide? > >The reason is simple: It is not possible to write portable code >according to the rule in MPI-2.2 p79:7-9 > >"The definition of extent is motivated by the assumption >that the amount of padding added at the end of each structure >in an array of structures is the least needed to fulfill >alignment constraints." > >... > >In C, this should work, in Fortran there are incompatibilities >that should be resolved with MPI-3.0. You're kidding? That assumption is nowhere supported by the C standard, and is demonstrably false for certain architectures. In particular, the representation of all pointers to structures is required to be the same (6.2.5 #27), which means that structures must have the same alignment on any system where pointer values are multiples of the alignment. I have no idea whether there are any still extant, but I can tell you that there were deliberately allowed for by WG14 during the standardisation of C90. Furthermore, there have definitely been some compilers where all structures had 8-byte alignment even on byte-addressed systems, and probably still are some. Regards, Nick Maclaren.

1 0

Re: [MPI3 Fortran] Serious problem/bug in MPI libraries with the alignment of MPI_DOUBLE_PRECISION
by N.M. Maclaren 28 Sep '11

28 Sep '11

On Sep 28 2011, Rasmussen, Craig E wrote: > > I don't know where you get the idea that a DOUBLE PRECISION variable > should occupy 4 bytes. The standard says: No, he meant aligned at 4-byte multiples. Regards, Nick Maclaren.

1 0

Re: [MPI3 Fortran] Serious problem/bug in MPI libraries with the alignment of MPI_DOUBLE_PRECISION
by Rasmussen, Craig E 28 Sep '11

28 Sep '11

Rolf, I don't know where you get the idea that a DOUBLE PRECISION variable should occupy 4 bytes. The standard says: A nonpointer scalar object of type double precision real or default complex occupies two contiguous numeric storage units; The derived types we worked out in Santorini were especially constructed so that double precisions numbers would be misaligned if the compiler was forced to do bad things via the SEQUENCE statement. Can you give me an idea which part of the tests we wrote indicate a double precision number is 4 bytes? Also, I checked the F2003 and F2008 standards and this wording (see above) has not been changed. -craig On Sep 28, 2011, at 11:00 AM, Rolf Rabenseifner wrote: A very good question. Instead of referring to SEQUENCE, it should say that MPI_DOUBLE_PRECISION would use the numeric storage sequence rules (16.4.3.1 in Fortran 2003) for alignment. Nick, please can you explain, what "16.4.3.1 in Fortran 2003" tells about alignments. In SEQUENCE derived types and COMMON blocks, when I understand this correctly, then DOUBLE PRECISION must have also one numerical storage unit as alignment, i.e., with REAL=4 bytes, DOUBLE PRECISION alignment must be also 4 byte, i.e., exactly what the Intel compiler is doing in a SEQUENCE derived type or COMMON block. The implication is that the compiler issues nice "warnings" about misaligned data. Do I understand correctly, that the other compilers with REAL-size=4 byte and DOUBLE PRECISION size and alignment = 8 byte, do it wrong, i.e.not according to Fortran 2003 or 2008? Best regards Rolf ----- Original Message ----- From: "N.M. Maclaren" <nmm1(a)cam.ac.uk<mailto:[email protected]>> To: "Fab Tillier" <ftillier(a)microsoft.com<mailto:[email protected]>> Cc: "Rolf Rabenseifner" <rabenseifner(a)hlrs.de<mailto:[email protected]>>, "Jeff Squyres" <jsquyres(a)cisco.com<mailto:[email protected]>>, "Shinji Sumimoto" <s-sumi(a)labs.fujitsu.com<mailto:[email protected]>>, "Hubert Ritzdorf" <hubert.ritzdorf(a)emea.nec.com<mailto:[email protected]>>, "Howard Pritchard" <howardp(a)cray.com<mailto:[email protected]>>, "Brian Smith" <smithbr(a)us.ibm.com<mailto:[email protected]>>, "Charles J Archer" <archerc(a)us.ibm.com<mailto:[email protected]>>, "Rajeev Thakur" <thakur(a)mcs.anl.gov<mailto:[email protected]>>, "Bill Long" <longb(a)cray.com<mailto:[email protected]>>, "Bill Gropp" <wgropp(a)uiuc.edu<mailto:[email protected]>>, "Richard Graham" <rlgraham(a)ornl.gov<mailto:[email protected]>>, "Iain Bason" <iain.bason(a)oracle.com<mailto:[email protected]>> Sent: Wednesday, September 28, 2011 10:26:37 AM Subject: RE: Serious problem/bug in MPI libraries with the alignment of MPI_DOUBLE_PRECISION On Sep 27 2011, Fab Tillier wrote: It seems that all the a == 4 results come from using the Intel Fortran compiler suite. Given that Microsoft doesn't ship a Fortran compiler, mandating a == 8 doesn't put us in a position to succeed. Therefore, I cannot support such a proposal. I agree that Microsoft should not be forgotten, but it is not a major platform for MPI use, and the tail should not wag the dog. We have the ghastly example of C99 long long where some people claimed Microsoft compatibility required an incompatible change to the standard; largely as a result, the official C standard is no longer used as a portable programming base by a good half of the C-using community. however, shouldn't a DOUBLE PRECISION BIND(C) derived type map to MPI_DOUBLE, not MPI_DOUBLE_PRECISION? Can't we simply document that MPI_DOUBLE_PRECISION (and any Fortran types that need similar treatment) apply only to SEQUENCE derived types, while BIND(C) derived types should use the C types? A very good question. Instead of referring to SEQUENCE, it should say that MPI_DOUBLE_PRECISION would use the numeric storage sequence rules (16.4.3.1 in Fortran 2003) for alignment. Slightly related to this, can't we PLEASE deprecate those horrible MPI_INTEGER8 etc. types? They have been a portability disaster for half a century, even among Fortran IV compilers (where they started)! MPI has provided some perfectly good mechanisms to use Fortran's selectable KINDs. The reason that it is related is that there is no guarantee that a compiler will give them the same alignment as another type even if they use the same storage, because they are non-standard and therefore any specification is the compiler's choice. Regards, Nick Maclaren. -- Dr. Rolf Rabenseifner . . . . . . . . . .. email rabenseifner(a)hlrs.de<mailto:[email protected]> High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner<http://www.hlrs.de/people/rabenseifner> Nobelstr. 19, D-70550 Stuttgart, Germany . (Office: Allmandring 30) _______________________________________________ mpi3-fortran mailing list mpi3-fortran(a)lists.mpi-forum.org<mailto:[email protected]> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-fortran

1 0