June 2017 - mpiwg-p2p - mailman3.cels.anl.gov

FP16 Support
by Atsushi HORI 21 Jun '17

21 Jun '17

Hello, Since I am a newcomer to this working group (and any of other WGs), I would like to start this e-mail with self introduction. I am Atsushi Hori (just call me 'Atsushi') at Riken and in charge of Flagship-2020 project to develop the post-K computer. Note that the project name do NOT suggest the year when the post-K computer will be installed. For some reason, I am responsible to push FP16 into the MPI standard and the straw vote in the MPI Forum meeting in Aachen ended up to have a new ticket for FP16. I was very amazed that hot discussions were carried out immediately after the Forum meeting was over. I appreciate Rolf, Kawashima-san, Jeff and others who gave me very important information. Let me clarify several, non-technical, points before start making FP16 draft; * WG It is the Martin's decision to discuss this ticket in this PT2PT WG. This is just because there is no appropriate WG to discuss this issue. Rolf discussed this with the Fortran WG too. Shall we discuss this issue with the Fortran WG too ? * Timing Rolf suggested that this issue must be "voted in only after the C standardization gets confirmed." In my opinion, all standardization effort of C (and Fortran), IEEE 754 (binary representation), and MPI are "independent." The compiler implementations are also independent. So, in theory, MPI standard may have FP16 independent from the others. In practice and my opinion, (final?) voting must take place when GCC supports FP16. * FP128 Jeff mentioned about FP128. Unlike FP16, FP128 is already supported by GCC on many CPU architectures. Were there any discussions on FP128 ever ? I think it might be a good idea to have both FP16 and FP128 in the future release of MPI standard. Or should it be discussed in a separate ticket ? Any comments are welcome. ----- Atsushi HORI ahori(a)riken.jp http://aics-sys.riken.jp

1 0

Re: [mpiwg-p2p] [MPIWG Fortran] FP16 16-bit-floating-point
by Jeff Hammond 16 Jun '17

16 Jun '17

> (we never did the MAX/MINLOC reductions also for fixed-size types) Yes, because it leads to an absurd explosion of enumerations, which is why I proposed a utility routine to construct pair types that behave like built-ins ( https://github.com/mpi-forum/mpi-issues/issues/18#issuecomment-155661199) While it's embarrassing that MPI has failed to keep up with ISO language built-in types, I console myself with the fact that I can implement all of this in BigMPI or MPICH in a day or two and Open-MPI with more effort. I don't know where float128_t sits w.r.t. ISO standardization in C or C++ but users have been asking for __float128 support for a long time to no avail. Jeff On Fri, Jun 16, 2017 at 11:04 AM, Rolf Rabenseifner <rabenseifner(a)hlrs.de> wrote: > Atsushi-san, (now only in the P2P-WG) > > > http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2016/p0192r0.pdf > > http://www.open-std.org/jtc1/sc22/wg14/www/docs/n2016.pdf > > Therefore for the C-side, I would propose for > (oc course voted in only after the C standardization gets confirmed) > > MPI-3.1 table on page 673 > > Named Predefined Datatypes | C types > ---------------------------+------------ > MPI_SHORT_FLOAT | short float > MPI_FLOAT16_T | float16_t > MPI_FLOAT32_T | float32_t > MPI_FLOAT64_T | float64_t > > (We must check, whether they also include float128_t into the C Standard, > or whether they only standardize short float without all ..._T) > > MPI-3.1 table on page 675 lines 1-9 > > Datatypes for reduction functions (C) > ------------------------------------- > MPI_SHORT_FLOAT_INT > > (we never did the MAX/MINLOC reductions also for fixed-size types) > > MPI-3.1 Table 13.2 on page 540 ***and*** > MPI-3.1 Table 3.2 on page 26 > > Type | Length > ----------------+------- > MPI_SHORT_FLOAT | 2 > MPI_FLOAT16_T | 2 > MPI_FLOAT32_T | 4 > MPI_FLOAT64_T | 8 > > And adding the following to "Floating point:" > on page 176, line 45: > > - before MPI_FLOAT: > MPI_SHORT_FLOAT, > - after MPI_DOUBLE: > MPI_FLOAT16_T, MPI_FLOAT32_T, MPI_FLOAT64_T, > > By the way, the table on page 540 should be done > with normal Latex and the correct macros for constants > to guarantee that this Location is also visible in the > constants-index page 823ff. > > You may have detected further Locations that are relevant. > > And of course, the appropriate change-log index is also needed. > > Best regards > Rolf > > ----- Original Message ----- > > From: "Kawashima, Takahiro" <t-kawashima(a)jp.fujitsu.com> > > To: "MPI WG Point To Point Communications working group" < > mpiwg-p2p(a)lists.mpi-forum.org> > > Cc: "Rolf Rabenseifner" <rabenseifner(a)hlrs.de>, "MPI-WG Fortran working > group" <mpiwg-fortran(a)lists.mpi-forum.org>, > > "Anthony Skjellum" <tony(a)runtimecomputing.com>, "Reinhold Bader" < > Reinhold.Bader(a)lrz-muenchen.de>, "Atsushi Hori" > > <ahori(a)riken.jp>, "fortran standards email list for J3" < > j3(a)j3-fortran.org> > > Sent: Friday, June 16, 2017 3:16:57 PM > > Subject: Re: [mpiwg-p2p] [MPIWG Fortran] FP16 16-bit-floating-point > > > Hi, > > > > I don't know the status of C standardization nor contact person > > but the suggestion of 'short float' was seem to made in a WG. > > > > http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2016/p0192r0.pdf > > http://www.open-std.org/jtc1/sc22/wg14/www/docs/n2016.pdf > > > > Regards, > > Takahiro Kawashima, > > Fujitsu > > > >> Tony, > >> > >> we try to find the type names used in C and Fortran before we define > >> the new predefined MPI datatype handles for representing those > >> language types. > >> > >> Who knows, whom we can contact on the C standardization side? > >> > >> Best regards > >> Rolf > >> > >> ----- Original Message ----- > >> > From: "Anthony Skjellum" <tony(a)runtimecomputing.com> > >> > To: "MPI WG Point To Point Communications working group" > >> > <mpiwg-p2p(a)lists.mpi-forum.org> > >> > Cc: "Reinhold Bader" <Reinhold.Bader(a)lrz-muenchen.de>, "Atsushi Hori" > >> > <ahori(a)riken.jp>, "fortran standards email list > >> > for J3" <j3(a)j3-fortran.org>, "MPI-WG Fortran working group" > >> > <mpiwg-fortran(a)lists.mpi-forum.org> > >> > Sent: Friday, June 16, 2017 2:50:00 PM > >> > Subject: Re: [MPIWG Fortran] [mpiwg-p2p] FP16 16-bit-floating-point > >> > >> > Rolf, apart from the Fortan angle, may I strongly encourage that we > adopt this > >> > datatype for MPI-4. It is part of real MPI codes already, but now we > have to > >> > fake transfer datatypes as 16-bit integers or bytes... which means > not really > >> > right if heterogeneous interoperability were to be maintained. And > this is a > >> > canonical IEEE type now, as you note. There is also the "2 > half-float" datatype > >> > (32-bits) at least in CUDA to be looked at; maybe ignored or maybe > supported, > >> > but it is there too. Key to this MPI users and applications use half > floats and > >> > 2 half-floats :-) in practice. > >> > > >> > May I also suggest a forward look at the types in AVX-512, in case > any of these > >> > become particularly relevant in future. > >> > At least 8-bit integers are conceptually new, apart from BYTE. [ > >> > https://en.wikipedia.org/wiki/AVX-512 | > https://en.wikipedia.org/wiki/AVX-512 ] > >> > > >> > Regards, > >> > Tony > >> > > >> > > >> > On Fri, Jun 16, 2017 at 6:52 AM, Rolf Rabenseifner < [ > >> > mailto:[email protected] | rabenseifner(a)hlrs.de ] > wrote: > >> > > >> > > >> > Hi all together, > >> > > >> > I'm sitting in the MPI Forum Meeting and we discuss the new IEEE FP16 > >> > datatype. > >> > > >> > Questions to the Fortran atandardization body: > >> > > >> > - Does the Fortran Standardization Body has already a proposal for > >> > the new name in Fortran for this IEEE FP16? > >> > > >> > - Will be REAL*2 identical to IEEE FP16? > >> > > >> > Best regards > > > > Rolf > > -- > Dr. Rolf Rabenseifner . . . . . . . . . .. email rabenseifner(a)hlrs.de . > High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 . > University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 . > Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner . > Nobelstr. 19, D-70550 Stuttgart, Germany . . . . (Office: Room 1.307) . > _______________________________________________ > mpiwg-p2p mailing list > mpiwg-p2p(a)lists.mpi-forum.org > https://lists.mpi-forum.org/mailman/listinfo/mpiwg-p2p > -- Jeff Hammond jeff.science(a)gmail.com http://jeffhammond.github.io/

1 0

Re: [mpiwg-p2p] [MPIWG Fortran] FP16 16-bit-floating-point
by Rolf Rabenseifner 16 Jun '17

16 Jun '17

Atsushi-san, (now only in the P2P-WG) > http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2016/p0192r0.pdf > http://www.open-std.org/jtc1/sc22/wg14/www/docs/n2016.pdf Therefore for the C-side, I would propose for (oc course voted in only after the C standardization gets confirmed) MPI-3.1 table on page 673 Named Predefined Datatypes | C types ---------------------------+------------ MPI_SHORT_FLOAT | short float MPI_FLOAT16_T | float16_t MPI_FLOAT32_T | float32_t MPI_FLOAT64_T | float64_t (We must check, whether they also include float128_t into the C Standard, or whether they only standardize short float without all ..._T) MPI-3.1 table on page 675 lines 1-9 Datatypes for reduction functions (C) ------------------------------------- MPI_SHORT_FLOAT_INT (we never did the MAX/MINLOC reductions also for fixed-size types) MPI-3.1 Table 13.2 on page 540 ***and*** MPI-3.1 Table 3.2 on page 26 Type | Length ----------------+------- MPI_SHORT_FLOAT | 2 MPI_FLOAT16_T | 2 MPI_FLOAT32_T | 4 MPI_FLOAT64_T | 8 And adding the following to "Floating point:" on page 176, line 45: - before MPI_FLOAT: MPI_SHORT_FLOAT, - after MPI_DOUBLE: MPI_FLOAT16_T, MPI_FLOAT32_T, MPI_FLOAT64_T, By the way, the table on page 540 should be done with normal Latex and the correct macros for constants to guarantee that this Location is also visible in the constants-index page 823ff. You may have detected further Locations that are relevant. And of course, the appropriate change-log index is also needed. Best regards Rolf ----- Original Message ----- > From: "Kawashima, Takahiro" <t-kawashima(a)jp.fujitsu.com> > To: "MPI WG Point To Point Communications working group" <mpiwg-p2p(a)lists.mpi-forum.org> > Cc: "Rolf Rabenseifner" <rabenseifner(a)hlrs.de>, "MPI-WG Fortran working group" <mpiwg-fortran(a)lists.mpi-forum.org>, > "Anthony Skjellum" <tony(a)runtimecomputing.com>, "Reinhold Bader" <Reinhold.Bader(a)lrz-muenchen.de>, "Atsushi Hori" > <ahori(a)riken.jp>, "fortran standards email list for J3" <j3(a)j3-fortran.org> > Sent: Friday, June 16, 2017 3:16:57 PM > Subject: Re: [mpiwg-p2p] [MPIWG Fortran] FP16 16-bit-floating-point > Hi, > > I don't know the status of C standardization nor contact person > but the suggestion of 'short float' was seem to made in a WG. > > http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2016/p0192r0.pdf > http://www.open-std.org/jtc1/sc22/wg14/www/docs/n2016.pdf > > Regards, > Takahiro Kawashima, > Fujitsu > >> Tony, >> >> we try to find the type names used in C and Fortran before we define >> the new predefined MPI datatype handles for representing those >> language types. >> >> Who knows, whom we can contact on the C standardization side? >> >> Best regards >> Rolf >> >> ----- Original Message ----- >> > From: "Anthony Skjellum" <tony(a)runtimecomputing.com> >> > To: "MPI WG Point To Point Communications working group" >> > <mpiwg-p2p(a)lists.mpi-forum.org> >> > Cc: "Reinhold Bader" <Reinhold.Bader(a)lrz-muenchen.de>, "Atsushi Hori" >> > <ahori(a)riken.jp>, "fortran standards email list >> > for J3" <j3(a)j3-fortran.org>, "MPI-WG Fortran working group" >> > <mpiwg-fortran(a)lists.mpi-forum.org> >> > Sent: Friday, June 16, 2017 2:50:00 PM >> > Subject: Re: [MPIWG Fortran] [mpiwg-p2p] FP16 16-bit-floating-point >> >> > Rolf, apart from the Fortan angle, may I strongly encourage that we adopt this >> > datatype for MPI-4. It is part of real MPI codes already, but now we have to >> > fake transfer datatypes as 16-bit integers or bytes... which means not really >> > right if heterogeneous interoperability were to be maintained. And this is a >> > canonical IEEE type now, as you note. There is also the "2 half-float" datatype >> > (32-bits) at least in CUDA to be looked at; maybe ignored or maybe supported, >> > but it is there too. Key to this MPI users and applications use half floats and >> > 2 half-floats :-) in practice. >> > >> > May I also suggest a forward look at the types in AVX-512, in case any of these >> > become particularly relevant in future. >> > At least 8-bit integers are conceptually new, apart from BYTE. [ >> > https://en.wikipedia.org/wiki/AVX-512 | https://en.wikipedia.org/wiki/AVX-512 ] >> > >> > Regards, >> > Tony >> > >> > >> > On Fri, Jun 16, 2017 at 6:52 AM, Rolf Rabenseifner < [ >> > mailto:[email protected] | rabenseifner(a)hlrs.de ] > wrote: >> > >> > >> > Hi all together, >> > >> > I'm sitting in the MPI Forum Meeting and we discuss the new IEEE FP16 >> > datatype. >> > >> > Questions to the Fortran atandardization body: >> > >> > - Does the Fortran Standardization Body has already a proposal for >> > the new name in Fortran for this IEEE FP16? >> > >> > - Will be REAL*2 identical to IEEE FP16? >> > >> > Best regards > > > Rolf -- Dr. Rolf Rabenseifner . . . . . . . . . .. email rabenseifner(a)hlrs.de . High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 . University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 . Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner . Nobelstr. 19, D-70550 Stuttgart, Germany . . . . (Office: Room 1.307) .

1 0

Re: [mpiwg-p2p] [MPIWG Fortran] FP16 16-bit-floating-point
by Rolf Rabenseifner 16 Jun '17

16 Jun '17

Sorry, I Forget to add the P2P-WG First, I want to remeber to Takahiro's answer aboout C. > http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2016/p0192r0.pdf > http://www.open-std.org/jtc1/sc22/wg14/www/docs/n2016.pdf They will probably add - short float, and - float16_t, float32_t, float64_t to the C Standard. For Fortran, the MPI standard currently supports floating points through - MPI_REAL for REAL - MPI_DOUBLE_PRECISION for DOUBLE PRECISION and only when supported by the Compiler (i.e. optional) - MPI_REAL2 for REAL*2 - MPI_REAL4 for REAL*4 - MPI_REAL8 for REAL*8 - MPI_REAL16 for REAL*16 The question is, whether Fortran will add something similar as "short float". Another question is, whether we never detected that Fortran has officially byte- or bit-sized floats, like the REAL*8. Rolf ----- Original Message ----- > From: "Kawashima, Takahiro" <t-kawashima(a)jp.fujitsu.com> > To: "MPI WG Point To Point Communications working group" <mpiwg-p2p(a)lists.mpi-forum.org> > Cc: "Rolf Rabenseifner" <rabenseifner(a)hlrs.de>, "MPI-WG Fortran working group" <mpiwg-fortran(a)lists.mpi-forum.org>, > "Anthony Skjellum" <tony(a)runtimecomputing.com>, "Reinhold Bader" <Reinhold.Bader(a)lrz-muenchen.de>, "Atsushi Hori" > <ahori(a)riken.jp>, "fortran standards email list for J3" <j3(a)j3-fortran.org> > Sent: Friday, June 16, 2017 3:16:57 PM > Subject: Re: [mpiwg-p2p] [MPIWG Fortran] FP16 16-bit-floating-point > Hi, > > I don't know the status of C standardization nor contact person > but the suggestion of 'short float' was seem to made in a WG. > > http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2016/p0192r0.pdf > http://www.open-std.org/jtc1/sc22/wg14/www/docs/n2016.pdf > > Regards, > Takahiro Kawashima, > Fujitsu ----- Original Message ----- > From: "longb" <longb(a)cray.com> > To: "MPI-WG Fortran working group" <mpiwg-fortran(a)lists.mpi-forum.org> > Cc: "Atsushi Hori" <ahori(a)riken.jp>, "Reinhold Bader" <Reinhold.Bader(a)lrz-muenchen.de>, "fortran standards email list > for J3" <j3(a)j3-fortran.org> > Sent: Friday, June 16, 2017 7:22:42 PM > Subject: Re: [MPIWG Fortran] [mpiwg-p2p] FP16 16-bit-floating-point >> On Jun 16, 2017, at 7:19 PM, Rolf Rabenseifner <rabenseifner(a)hlrs.de> wrote: >> >> Jeff is partially right: >> If the Fortran user uses selected_real_kind(p, r), then all is done. >> >> On the other hand, >> I expect that the Fortran standardization body will do the same >> as the C standardization body, namely to add a new language Basic type. > > No. The basic type is REAL. This is just another KIND of REAL. This is not > much of an issue for the standard. It is a HUGE deal for implementations. > > Cheers, > Bill > > >> >> For this of course, we have to provide the corresponding MPI predefined >> datatype. >> Rolf >> >> >> ----- Original Message ----- >>> From: "Jeff Hammond" <jeff.science(a)gmail.com> >>> To: "MPI WG Point To Point Communications working group" >>> <mpiwg-p2p(a)lists.mpi-forum.org> >>> Cc: "Atsushi Hori" <ahori(a)riken.jp>, "Reinhold Bader" >>> <Reinhold.Bader(a)lrz-muenchen.de>, "fortran standards email list >>> for J3" <j3(a)j3-fortran.org>, "MPI-WG Fortran working group" >>> <mpiwg-fortran(a)lists.mpi-forum.org> >>> Sent: Friday, June 16, 2017 5:11:24 PM >>> Subject: Re: [MPIWG Fortran] [mpiwg-p2p] FP16 16-bit-floating-point >> >>> No change to the MPI standard is required here. MPI_Type_create_f90_real is >>> already sufficient. >>> >>> "This function returns a predefined MPI datatype that matches a REAL variable of >>> KIND selected_real_kind(p, r)." - I do not see why we need to enumerate a >>> predefined datatype name associated with float16 in Fortran for this statement >>> to be true. An implementation can return a datatype with the appropriate >>> properties without being enumerated like MPI_REAL. >>> >>> "It is erroneous to supply values for p and r not supported by the compiler." - >>> Conversely, the user should be allowed to supply values that are supported by >>> the compiler (although the standard doesn't guarentee it today). MPI >>> implementations might not support every type the compiler does, as is already >>> the case for Fortran float128 (because MPI libraries are written in C, and do >>> not call back into Fortran to do math on these types, which may not be >>> identical to C extensions like __float128), but I consider that an >>> implementation defect. >>> >>> Thus, I propose to merely add the text that allows users to rely on >>> MPI_Type_create_f90_real for float16 in Fortran, provided that the compiler >>> supports it. It will then be up to implementations to figure out how to do math >>> on these types internally. >>> >>> While we are on this subject, we should be considering the more general >>> inadequacy of the current state of the MPI type system from the perspective of >>> reductions. See [ https://github.com/mpi-forum/mpi-issues/issues/18 | >>> https://github.com/mpi-forum/mpi-issues/issues/18 ] for details. >>> >>> As for ISO C/C++, I can talk to Clark Nelson if a better candidate is not >>> identified, but Lawrence Crowl is leading the C++ effort and I suspect the C >>> folks will follow, although I will confirm this with Clark. >>> >>> Jeff >>> >>> On Fri, Jun 16, 2017 at 6:16 AM, Kawashima, Takahiro < [ >>> mailto:[email protected] | t-kawashima(a)jp.fujitsu.com ] > wrote: >>> >>> >>> Hi, >>> >>> I don't know the status of C standardization nor contact person >>> but the suggestion of 'short float' was seem to made in a WG. >>> >>> [ http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2016/p0192r0.pdf | >>> http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2016/p0192r0.pdf ] >>> [ http://www.open-std.org/jtc1/sc22/wg14/www/docs/n2016.pdf | >>> http://www.open-std.org/jtc1/sc22/wg14/www/docs/n2016.pdf ] >>> >>> Regards, >>> Takahiro Kawashima, >>> Fujitsu >>> >>>> Tony, >>>> >>>> we try to find the type names used in C and Fortran before we define >>>> the new predefined MPI datatype handles for representing those >>>> language types. >>>> >>>> Who knows, whom we can contact on the C standardization side? >>>> >>>> Best regards >>>> Rolf >>>> >>>> ----- Original Message ----- >>>>> From: "Anthony Skjellum" < [ mailto:[email protected] | >>>>> tony(a)runtimecomputing.com ] > >>>>> To: "MPI WG Point To Point Communications working group" < [ >>>>> mailto:[email protected] | mpiwg-p2p(a)lists.mpi-forum.org ] > >>>>> Cc: "Reinhold Bader" < [ mailto:[email protected] | >>>>> Reinhold.Bader(a)lrz-muenchen.de ] >, "Atsushi Hori" < [ mailto:[email protected] | >>>>> ahori(a)riken.jp ] >, "fortran standards email list >>>>> for J3" < [ mailto:[email protected] | j3(a)j3-fortran.org ] >, "MPI-WG Fortran >>>>> working group" < [ mailto:[email protected] | >>>>> mpiwg-fortran(a)lists.mpi-forum.org ] > >>>>> Sent: Friday, June 16, 2017 2:50:00 PM >>>>> Subject: Re: [MPIWG Fortran] [mpiwg-p2p] FP16 16-bit-floating-point >>>> >>>>> Rolf, apart from the Fortan angle, may I strongly encourage that we adopt this >>>>> datatype for MPI-4. It is part of real MPI codes already, but now we have to >>>>> fake transfer datatypes as 16-bit integers or bytes... which means not really >>>>> right if heterogeneous interoperability were to be maintained. And this is a >>>>> canonical IEEE type now, as you note. There is also the "2 half-float" datatype >>>>> (32-bits) at least in CUDA to be looked at; maybe ignored or maybe supported, >>>>> but it is there too. Key to this MPI users and applications use half floats and >>>>> 2 half-floats :-) in practice. >>>>> >>>>> May I also suggest a forward look at the types in AVX-512, in case any of these >>>>> become particularly relevant in future. >>>>> At least 8-bit integers are conceptually new, apart from BYTE. [ >>>>> [ https://en.wikipedia.org/wiki/AVX-512 | https://en.wikipedia.org/wiki/AVX-512 >>>>> ] | [ https://en.wikipedia.org/wiki/AVX-512 | >>>>> https://en.wikipedia.org/wiki/AVX-512 ] ] >>>>> >>>>> Regards, >>>>> Tony >>>>> >>>>> >>>>> On Fri, Jun 16, 2017 at 6:52 AM, Rolf Rabenseifner < [ >>>>> mailto: [ mailto:[email protected] | rabenseifner(a)hlrs.de ] | [ >>>>> mailto:[email protected] | rabenseifner(a)hlrs.de ] ] > wrote: >>>>> >>>>> >>>>> Hi all together, >>>>> >>>>> I'm sitting in the MPI Forum Meeting and we discuss the new IEEE FP16 >>>>> datatype. >>>>> >>>>> Questions to the Fortran atandardization body: >>>>> >>>>> - Does the Fortran Standardization Body has already a proposal for >>>>> the new name in Fortran for this IEEE FP16? >>>>> >>>>> - Will be REAL*2 identical to IEEE FP16? >>>>> >>>>> Best regards >>>>> Rolf >>> _______________________________________________ >>> mpiwg-p2p mailing list >>> [ mailto:[email protected] | mpiwg-p2p(a)lists.mpi-forum.org ] >>> [ https://lists.mpi-forum.org/mailman/listinfo/mpiwg-p2p | >>> https://lists.mpi-forum.org/mailman/listinfo/mpiwg-p2p ] >>> >>> >>> >>> -- >>> Jeff Hammond >>> [ mailto:[email protected] | jeff.science(a)gmail.com ] >>> [ http://jeffhammond.github.io/ | http://jeffhammond.github.io/ ] >>> >>> _______________________________________________ >>> mpiwg-fortran mailing list >>> mpiwg-fortran(a)lists.mpi-forum.org >>> https://lists.mpi-forum.org/mailman/listinfo/mpiwg-fortran >> >> -- >> Dr. Rolf Rabenseifner . . . . . . . . . .. email rabenseifner(a)hlrs.de . >> High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 . >> University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 . >> Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner . >> Nobelstr. 19, D-70550 Stuttgart, Germany . . . . (Office: Room 1.307) . >> _______________________________________________ >> mpiwg-fortran mailing list >> mpiwg-fortran(a)lists.mpi-forum.org >> https://lists.mpi-forum.org/mailman/listinfo/mpiwg-fortran > > Bill Long > longb(a)cray.com > Principal Engineer, Fortran Technical Support & voice: 651-605-9024 > Bioinformatics Software Development fax: 651-605-9143 > Cray Inc./ 2131 Lindau Lane/ Suite 1000/ Bloomington, MN 55425 > > > _______________________________________________ > mpiwg-fortran mailing list > mpiwg-fortran(a)lists.mpi-forum.org > https://lists.mpi-forum.org/mailman/listinfo/mpiwg-fortran -- Dr. Rolf Rabenseifner . . . . . . . . . .. email rabenseifner(a)hlrs.de . High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 . University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 . Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner . Nobelstr. 19, D-70550 Stuttgart, Germany . . . . (Office: Room 1.307) . _______________________________________________ mpiwg-fortran mailing list mpiwg-fortran(a)lists.mpi-forum.org https://lists.mpi-forum.org/mailman/listinfo/mpiwg-fortran

1 0

Re: [mpiwg-p2p] [MPIWG Fortran] FP16 16-bit-floating-point
by Rolf Rabenseifner 16 Jun '17

16 Jun '17

Jeff is partially right: If the Fortran user uses selected_real_kind(p, r), then all is done. On the other hand, I expect that the Fortran standardization body will do the same as the C standardization body, namely to add a new language Basic type. For this of course, we have to provide the corresponding MPI predefined datatype. Rolf ----- Original Message ----- > From: "Jeff Hammond" <jeff.science(a)gmail.com> > To: "MPI WG Point To Point Communications working group" <mpiwg-p2p(a)lists.mpi-forum.org> > Cc: "Atsushi Hori" <ahori(a)riken.jp>, "Reinhold Bader" <Reinhold.Bader(a)lrz-muenchen.de>, "fortran standards email list > for J3" <j3(a)j3-fortran.org>, "MPI-WG Fortran working group" <mpiwg-fortran(a)lists.mpi-forum.org> > Sent: Friday, June 16, 2017 5:11:24 PM > Subject: Re: [MPIWG Fortran] [mpiwg-p2p] FP16 16-bit-floating-point > No change to the MPI standard is required here. MPI_Type_create_f90_real is > already sufficient. > > "This function returns a predefined MPI datatype that matches a REAL variable of > KIND selected_real_kind(p, r)." - I do not see why we need to enumerate a > predefined datatype name associated with float16 in Fortran for this statement > to be true. An implementation can return a datatype with the appropriate > properties without being enumerated like MPI_REAL. > > "It is erroneous to supply values for p and r not supported by the compiler." - > Conversely, the user should be allowed to supply values that are supported by > the compiler (although the standard doesn't guarentee it today). MPI > implementations might not support every type the compiler does, as is already > the case for Fortran float128 (because MPI libraries are written in C, and do > not call back into Fortran to do math on these types, which may not be > identical to C extensions like __float128), but I consider that an > implementation defect. > > Thus, I propose to merely add the text that allows users to rely on > MPI_Type_create_f90_real for float16 in Fortran, provided that the compiler > supports it. It will then be up to implementations to figure out how to do math > on these types internally. > > While we are on this subject, we should be considering the more general > inadequacy of the current state of the MPI type system from the perspective of > reductions. See [ https://github.com/mpi-forum/mpi-issues/issues/18 | > https://github.com/mpi-forum/mpi-issues/issues/18 ] for details. > > As for ISO C/C++, I can talk to Clark Nelson if a better candidate is not > identified, but Lawrence Crowl is leading the C++ effort and I suspect the C > folks will follow, although I will confirm this with Clark. > > Jeff > > On Fri, Jun 16, 2017 at 6:16 AM, Kawashima, Takahiro < [ > mailto:[email protected] | t-kawashima(a)jp.fujitsu.com ] > wrote: > > > Hi, > > I don't know the status of C standardization nor contact person > but the suggestion of 'short float' was seem to made in a WG. > > [ http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2016/p0192r0.pdf | > http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2016/p0192r0.pdf ] > [ http://www.open-std.org/jtc1/sc22/wg14/www/docs/n2016.pdf | > http://www.open-std.org/jtc1/sc22/wg14/www/docs/n2016.pdf ] > > Regards, > Takahiro Kawashima, > Fujitsu > >> Tony, >> >> we try to find the type names used in C and Fortran before we define >> the new predefined MPI datatype handles for representing those >> language types. >> >> Who knows, whom we can contact on the C standardization side? >> >> Best regards >> Rolf >> >> ----- Original Message ----- >> > From: "Anthony Skjellum" < [ mailto:[email protected] | >> > tony(a)runtimecomputing.com ] > >> > To: "MPI WG Point To Point Communications working group" < [ >> > mailto:[email protected] | mpiwg-p2p(a)lists.mpi-forum.org ] > >> > Cc: "Reinhold Bader" < [ mailto:[email protected] | >> > Reinhold.Bader(a)lrz-muenchen.de ] >, "Atsushi Hori" < [ mailto:[email protected] | >> > ahori(a)riken.jp ] >, "fortran standards email list >> > for J3" < [ mailto:[email protected] | j3(a)j3-fortran.org ] >, "MPI-WG Fortran >> > working group" < [ mailto:[email protected] | >> > mpiwg-fortran(a)lists.mpi-forum.org ] > >> > Sent: Friday, June 16, 2017 2:50:00 PM >> > Subject: Re: [MPIWG Fortran] [mpiwg-p2p] FP16 16-bit-floating-point >> >> > Rolf, apart from the Fortan angle, may I strongly encourage that we adopt this >> > datatype for MPI-4. It is part of real MPI codes already, but now we have to >> > fake transfer datatypes as 16-bit integers or bytes... which means not really >> > right if heterogeneous interoperability were to be maintained. And this is a >> > canonical IEEE type now, as you note. There is also the "2 half-float" datatype >> > (32-bits) at least in CUDA to be looked at; maybe ignored or maybe supported, >> > but it is there too. Key to this MPI users and applications use half floats and >> > 2 half-floats :-) in practice. >> > >> > May I also suggest a forward look at the types in AVX-512, in case any of these >> > become particularly relevant in future. >> > At least 8-bit integers are conceptually new, apart from BYTE. [ >> > [ https://en.wikipedia.org/wiki/AVX-512 | https://en.wikipedia.org/wiki/AVX-512 >> > ] | [ https://en.wikipedia.org/wiki/AVX-512 | >> > https://en.wikipedia.org/wiki/AVX-512 ] ] >> > >> > Regards, >> > Tony >> > >> > >> > On Fri, Jun 16, 2017 at 6:52 AM, Rolf Rabenseifner < [ >> > mailto: [ mailto:[email protected] | rabenseifner(a)hlrs.de ] | [ >> > mailto:[email protected] | rabenseifner(a)hlrs.de ] ] > wrote: >> > >> > >> > Hi all together, >> > >> > I'm sitting in the MPI Forum Meeting and we discuss the new IEEE FP16 >> > datatype. >> > >> > Questions to the Fortran atandardization body: >> > >> > - Does the Fortran Standardization Body has already a proposal for >> > the new name in Fortran for this IEEE FP16? >> > >> > - Will be REAL*2 identical to IEEE FP16? >> > >> > Best regards >> > Rolf > _______________________________________________ > mpiwg-p2p mailing list > [ mailto:[email protected] | mpiwg-p2p(a)lists.mpi-forum.org ] > [ https://lists.mpi-forum.org/mailman/listinfo/mpiwg-p2p | > https://lists.mpi-forum.org/mailman/listinfo/mpiwg-p2p ] > > > > -- > Jeff Hammond > [ mailto:[email protected] | jeff.science(a)gmail.com ] > [ http://jeffhammond.github.io/ | http://jeffhammond.github.io/ ] > > _______________________________________________ > mpiwg-fortran mailing list > mpiwg-fortran(a)lists.mpi-forum.org > https://lists.mpi-forum.org/mailman/listinfo/mpiwg-fortran -- Dr. Rolf Rabenseifner . . . . . . . . . .. email rabenseifner(a)hlrs.de . High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 . University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 . Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner . Nobelstr. 19, D-70550 Stuttgart, Germany . . . . (Office: Room 1.307) .

1 0

Re: [mpiwg-p2p] [MPIWG Fortran] FP16 16-bit-floating-point
by Bill Long 16 Jun '17

16 Jun '17

Hi Rolf, The Fortran 2015 draft is close to finished. Its IEEE-isms are based on ISO/IEC/IEEE 60559:2011, and includes a lot of changes based on that standard. If this FP16 feature is part of that standard, then we should look into making a change now. Otherwise, it would likely wait for the next revision. As Reinhold pointed out, the parameterized KIND scheme Fortran uses already permits identifying a KIND value for 16-bit floats, assuming the processor (i.e. compiler) knows about such a type. Changes to the standard would be minimal - add REAL16 to the iso_fortran_env module, with the value of -1 if that kind of real is not supported, and the value of 2 for most compilers if it is. Changes to the compiler would be significant. So implementation would likely wait for customer demand. Cheers, Bill > On Jun 16, 2017, at 2:44 PM, Rolf Rabenseifner <rabenseifner(a)hlrs.de> wrote: > > (Because I'm not member of j3, only Reinhold's or Rich's answers will be visible for j3) > > Based on Reinhold's answer, I make my questions more precise: > > - Are there already some activities in J3 on this FP16 topic? > > - When will the Fortran standard include REAL16 to support the new IEEE FP16 standard? > > Best regards > Rolf > > > ----- Original Message ----- >> From: "Reinhold Bader" <Reinhold.Bader(a)lrz.de> >> To: "Rolf Rabenseifner" <rabenseifner(a)hlrs.de>, "fortran standards email list for J3" <j3(a)j3-fortran.org> >> Cc: "MPI-WG Fortran working group" <mpiwg-fortran(a)lists.mpi-forum.org>, "Atsushi Hori" <ahori(a)riken.jp>, "Martin Schulz" >> <schulzm(a)llnl.gov>, "MPI WG Point To Point Communications working group" <mpiwg-p2p(a)lists.mpi-forum.org> >> Sent: Friday, June 16, 2017 2:31:03 PM >> Subject: AW: FP16 16-bit-floating-point > >> Hello Rolf, >> >> to the best of my knowledge, the answers are as follows: >> >> Q1: Not in the sense of having an explicit name like the REAL32, REAL64, REAL128 >> names for specific integer kinds that were added to Fortran 2008. >> However, it is left free to implementors to create a new kind number in the >> implementation that could be returned by SELECTED_REAL_KIND() and, >> if IEEE conforming, by IEEE_SELECTED_REAL_KIND() and consequently all relevant >> IEEE support routines. >> >> Q2: REAL*2 is (like REAL*8) a non-conforming notation (REAL*8 is supported as an >> extension by many implementations), so the answer is no. >> A proper declaration might look like >> >> integer, parameter :: rk = selected_real_kind(2,6) ! at least 2 decimal >> digits and 6 maximal decimal exponent - hope I have this right ... >> real(KIND=rk) :: x >> >> with the following caveats: >> * the KIND that fits the requirement most closely will be used, so it might be >> necessary to use the STORAGE_SIZE intrinsic to check what one gets. >> * KIND numbers are non-portable, so might vary for one and the same kind between >> implementations (at least one compiler I know deviates from all others I know). >> >> I checked three of the compilers I know: they currently do not support FP16. >> >> Regards >> Reinhold >> >> >>> -----Ursprüngliche Nachricht----- >>> Von: Rolf Rabenseifner [mailto:[email protected]] >>> Gesendet: Freitag, 16. Juni 2017 13:53 >>> An: fortran standards email list for J3 <j3(a)j3-fortran.org>; Bader, Reinhold >>> <Reinhold.Bader(a)lrz.de> >>> Cc: MPI-WG Fortran working group <mpiwg-fortran(a)lists.mpi-forum.org>; >>> Atsushi Hori <ahori(a)riken.jp>; Martin Schulz <schulzm(a)llnl.gov>; MPI WG >>> Point To Point Communications working group <mpiwg-p2p(a)lists.mpi- >>> forum.org> >>> Betreff: FP16 16-bit-floating-point >>> >>> Hi all together, >>> >>> I'm sitting in the MPI Forum Meeting and we discuss the new IEEE FP16 >>> datatype. >>> >>> Questions to the Fortran atandardization body: >>> >>> - Does the Fortran Standardization Body has already a proposal for >>> the new name in Fortran for this IEEE FP16? >>> >>> - Will be REAL*2 identical to IEEE FP16? >>> >>> Best regards >>> Rolf >>> >>> -- >>> Dr. Rolf Rabenseifner . . . . . . . . . .. email rabenseifner(a)hlrs.de . >>> High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 . >>> University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 . >>> Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner . >>> Nobelstr. 19, D-70550 Stuttgart, Germany . . . . (Office: Room 1.307) . > > -- > Dr. Rolf Rabenseifner . . . . . . . . . .. email rabenseifner(a)hlrs.de . > High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 . > University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 . > Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner . > Nobelstr. 19, D-70550 Stuttgart, Germany . . . . (Office: Room 1.307) . > _______________________________________________ > mpiwg-fortran mailing list > mpiwg-fortran(a)lists.mpi-forum.org > https://lists.mpi-forum.org/mailman/listinfo/mpiwg-fortran Bill Long longb(a)cray.com Principal Engineer, Fortran Technical Support & voice: 651-605-9024 Bioinformatics Software Development fax: 651-605-9143 Cray Inc./ 2131 Lindau Lane/ Suite 1000/ Bloomington, MN 55425

1 0

Re: [mpiwg-p2p] [MPIWG Fortran] FP16 16-bit-floating-point
by Jeff Hammond 16 Jun '17

16 Jun '17

No change to the MPI standard is required here. MPI_Type_create_f90_real is already sufficient. "This function returns a *predefined* MPI datatype that matches a REAL variable of KIND selected_real_kind(p, r)." - I do not see why we need to enumerate a predefined datatype name associated with float16 in Fortran for this statement to be true. An implementation can return a datatype with the appropriate properties without being enumerated like MPI_REAL. "It is erroneous to supply values for p and r not supported by the compiler." - Conversely, the user should be allowed to supply values that are supported by the compiler (although the standard doesn't guarentee it today). MPI implementations might not support every type the compiler does, as is already the case for Fortran float128 (because MPI libraries are written in C, and do not call back into Fortran to do math on these types, which may not be identical to C extensions like __float128), but I consider that an implementation defect. Thus, I propose to merely add the text that allows users to rely on MPI_Type_create_f90_real for float16 in Fortran, provided that the compiler supports it. It will then be up to implementations to figure out how to do math on these types internally. While we are on this subject, we should be considering the more general inadequacy of the current state of the MPI type system from the perspective of reductions. See https://github.com/mpi-forum/mpi-issues/issues/18 for details. As for ISO C/C++, I can talk to Clark Nelson if a better candidate is not identified, but Lawrence Crowl is leading the C++ effort and I suspect the C folks will follow, although I will confirm this with Clark. Jeff On Fri, Jun 16, 2017 at 6:16 AM, Kawashima, Takahiro < t-kawashima(a)jp.fujitsu.com> wrote: > Hi, > > I don't know the status of C standardization nor contact person > but the suggestion of 'short float' was seem to made in a WG. > > http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2016/p0192r0.pdf > http://www.open-std.org/jtc1/sc22/wg14/www/docs/n2016.pdf > > Regards, > Takahiro Kawashima, > Fujitsu > > > Tony, > > > > we try to find the type names used in C and Fortran before we define > > the new predefined MPI datatype handles for representing those > > language types. > > > > Who knows, whom we can contact on the C standardization side? > > > > Best regards > > Rolf > > > > ----- Original Message ----- > > > From: "Anthony Skjellum" <tony(a)runtimecomputing.com> > > > To: "MPI WG Point To Point Communications working group" < > mpiwg-p2p(a)lists.mpi-forum.org> > > > Cc: "Reinhold Bader" <Reinhold.Bader(a)lrz-muenchen.de>, "Atsushi Hori" > <ahori(a)riken.jp>, "fortran standards email list > > > for J3" <j3(a)j3-fortran.org>, "MPI-WG Fortran working group" < > mpiwg-fortran(a)lists.mpi-forum.org> > > > Sent: Friday, June 16, 2017 2:50:00 PM > > > Subject: Re: [MPIWG Fortran] [mpiwg-p2p] FP16 16-bit-floating-point > > > > > Rolf, apart from the Fortan angle, may I strongly encourage that we > adopt this > > > datatype for MPI-4. It is part of real MPI codes already, but now we > have to > > > fake transfer datatypes as 16-bit integers or bytes... which means not > really > > > right if heterogeneous interoperability were to be maintained. And > this is a > > > canonical IEEE type now, as you note. There is also the "2 half-float" > datatype > > > (32-bits) at least in CUDA to be looked at; maybe ignored or maybe > supported, > > > but it is there too. Key to this MPI users and applications use half > floats and > > > 2 half-floats :-) in practice. > > > > > > May I also suggest a forward look at the types in AVX-512, in case any > of these > > > become particularly relevant in future. > > > At least 8-bit integers are conceptually new, apart from BYTE. [ > > > https://en.wikipedia.org/wiki/AVX-512 | https://en.wikipedia.org/wiki/ > AVX-512 ] > > > > > > Regards, > > > Tony > > > > > > > > > On Fri, Jun 16, 2017 at 6:52 AM, Rolf Rabenseifner < [ > > > mailto:[email protected] | rabenseifner(a)hlrs.de ] > wrote: > > > > > > > > > Hi all together, > > > > > > I'm sitting in the MPI Forum Meeting and we discuss the new IEEE FP16 > > > datatype. > > > > > > Questions to the Fortran atandardization body: > > > > > > - Does the Fortran Standardization Body has already a proposal for > > > the new name in Fortran for this IEEE FP16? > > > > > > - Will be REAL*2 identical to IEEE FP16? > > > > > > Best regards > > > Rolf > _______________________________________________ > mpiwg-p2p mailing list > mpiwg-p2p(a)lists.mpi-forum.org > https://lists.mpi-forum.org/mailman/listinfo/mpiwg-p2p > -- Jeff Hammond jeff.science(a)gmail.com http://jeffhammond.github.io/

1 0

Re: [mpiwg-p2p] [MPIWG Fortran] FP16 16-bit-floating-point
by Kawashima, Takahiro 16 Jun '17

16 Jun '17

Hi, I don't know the status of C standardization nor contact person but the suggestion of 'short float' was seem to made in a WG. http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2016/p0192r0.pdf http://www.open-std.org/jtc1/sc22/wg14/www/docs/n2016.pdf Regards, Takahiro Kawashima, Fujitsu > Tony, > > we try to find the type names used in C and Fortran before we define > the new predefined MPI datatype handles for representing those > language types. > > Who knows, whom we can contact on the C standardization side? > > Best regards > Rolf > > ----- Original Message ----- > > From: "Anthony Skjellum" <tony(a)runtimecomputing.com> > > To: "MPI WG Point To Point Communications working group" <mpiwg-p2p(a)lists.mpi-forum.org> > > Cc: "Reinhold Bader" <Reinhold.Bader(a)lrz-muenchen.de>, "Atsushi Hori" <ahori(a)riken.jp>, "fortran standards email list > > for J3" <j3(a)j3-fortran.org>, "MPI-WG Fortran working group" <mpiwg-fortran(a)lists.mpi-forum.org> > > Sent: Friday, June 16, 2017 2:50:00 PM > > Subject: Re: [MPIWG Fortran] [mpiwg-p2p] FP16 16-bit-floating-point > > > Rolf, apart from the Fortan angle, may I strongly encourage that we adopt this > > datatype for MPI-4. It is part of real MPI codes already, but now we have to > > fake transfer datatypes as 16-bit integers or bytes... which means not really > > right if heterogeneous interoperability were to be maintained. And this is a > > canonical IEEE type now, as you note. There is also the "2 half-float" datatype > > (32-bits) at least in CUDA to be looked at; maybe ignored or maybe supported, > > but it is there too. Key to this MPI users and applications use half floats and > > 2 half-floats :-) in practice. > > > > May I also suggest a forward look at the types in AVX-512, in case any of these > > become particularly relevant in future. > > At least 8-bit integers are conceptually new, apart from BYTE. [ > > https://en.wikipedia.org/wiki/AVX-512 | https://en.wikipedia.org/wiki/AVX-512 ] > > > > Regards, > > Tony > > > > > > On Fri, Jun 16, 2017 at 6:52 AM, Rolf Rabenseifner < [ > > mailto:[email protected] | rabenseifner(a)hlrs.de ] > wrote: > > > > > > Hi all together, > > > > I'm sitting in the MPI Forum Meeting and we discuss the new IEEE FP16 > > datatype. > > > > Questions to the Fortran atandardization body: > > > > - Does the Fortran Standardization Body has already a proposal for > > the new name in Fortran for this IEEE FP16? > > > > - Will be REAL*2 identical to IEEE FP16? > > > > Best regards > > Rolf

1 0

Re: [mpiwg-p2p] [MPIWG Fortran] FP16 16-bit-floating-point
by Rolf Rabenseifner 16 Jun '17

16 Jun '17

Tony, we try to find the type names used in C and Fortran before we define the new predefined MPI datatype handles for representing those language types. Who knows, whom we can contact on the C standardization side? Best regards Rolf ----- Original Message ----- > From: "Anthony Skjellum" <tony(a)runtimecomputing.com> > To: "MPI WG Point To Point Communications working group" <mpiwg-p2p(a)lists.mpi-forum.org> > Cc: "Reinhold Bader" <Reinhold.Bader(a)lrz-muenchen.de>, "Atsushi Hori" <ahori(a)riken.jp>, "fortran standards email list > for J3" <j3(a)j3-fortran.org>, "MPI-WG Fortran working group" <mpiwg-fortran(a)lists.mpi-forum.org> > Sent: Friday, June 16, 2017 2:50:00 PM > Subject: Re: [MPIWG Fortran] [mpiwg-p2p] FP16 16-bit-floating-point > Rolf, apart from the Fortan angle, may I strongly encourage that we adopt this > datatype for MPI-4. It is part of real MPI codes already, but now we have to > fake transfer datatypes as 16-bit integers or bytes... which means not really > right if heterogeneous interoperability were to be maintained. And this is a > canonical IEEE type now, as you note. There is also the "2 half-float" datatype > (32-bits) at least in CUDA to be looked at; maybe ignored or maybe supported, > but it is there too. Key to this MPI users and applications use half floats and > 2 half-floats :-) in practice. > > May I also suggest a forward look at the types in AVX-512, in case any of these > become particularly relevant in future. > At least 8-bit integers are conceptually new, apart from BYTE. [ > https://en.wikipedia.org/wiki/AVX-512 | https://en.wikipedia.org/wiki/AVX-512 ] > > Regards, > Tony > > > On Fri, Jun 16, 2017 at 6:52 AM, Rolf Rabenseifner < [ > mailto:[email protected] | rabenseifner(a)hlrs.de ] > wrote: > > > Hi all together, > > I'm sitting in the MPI Forum Meeting and we discuss the new IEEE FP16 > datatype. > > Questions to the Fortran atandardization body: > > - Does the Fortran Standardization Body has already a proposal for > the new name in Fortran for this IEEE FP16? > > - Will be REAL*2 identical to IEEE FP16? > > Best regards > Rolf > > -- > Dr. Rolf Rabenseifner . . . . . . . . . .. email [ mailto:[email protected] | > rabenseifner(a)hlrs.de ] . > High Performance Computing Center (HLRS) . phone [ > tel:%2B%2B49%280%29711%2F685-65530 | ++49(0)711/685-65530 ] . > University of Stuttgart . . . . . . . . .. fax [ > tel:%2B%2B49%280%29711%20%2F%20685-65832 | ++49(0)711 / 685-65832 ] . > Head of Dpmt Parallel Computing . . . [ http://www.hlrs.de/people/rabenseifner | > www.hlrs.de/people/rabenseifner ] . > Nobelstr. 19, D-70550 Stuttgart, Germany . . . . (Office: Room 1.307) . > _______________________________________________ > mpiwg-p2p mailing list > [ mailto:[email protected] | mpiwg-p2p(a)lists.mpi-forum.org ] > [ https://lists.mpi-forum.org/mailman/listinfo/mpiwg-p2p | > https://lists.mpi-forum.org/mailman/listinfo/mpiwg-p2p ] > > > > -- > Tony Skjellum, PhD > RunTime Computing Solutions, LLC > [ mailto:[email protected] | tony(a)runtimecomputing.com ] > direct: +1-205-918-7514 > cell: +1-205-807-4968 > > _______________________________________________ > mpiwg-fortran mailing list > mpiwg-fortran(a)lists.mpi-forum.org > https://lists.mpi-forum.org/mailman/listinfo/mpiwg-fortran -- Dr. Rolf Rabenseifner . . . . . . . . . .. email rabenseifner(a)hlrs.de . High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 . University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 . Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner . Nobelstr. 19, D-70550 Stuttgart, Germany . . . . (Office: Room 1.307) .

1 0

Re: [mpiwg-p2p] FP16 16-bit-floating-point
by Anthony Skjellum 16 Jun '17

16 Jun '17

Rolf, apart from the Fortan angle, may I strongly encourage that we adopt this datatype for MPI-4. It is part of real MPI codes already, but now we have to fake transfer datatypes as 16-bit integers or bytes... which means not really right if heterogeneous interoperability were to be maintained. And this is a canonical IEEE type now, as you note. There is also the "2 half-float" datatype (32-bits) at least in CUDA to be looked at; maybe ignored or maybe supported, but it is there too. Key to this MPI users and applications use half floats and 2 half-floats :-) in practice. May I also suggest a forward look at the types in AVX-512, in case any of these become particularly relevant in future. At least 8-bit integers are conceptually new, apart from BYTE. https://en.wikipedia.org/wiki/AVX-512 Regards, Tony On Fri, Jun 16, 2017 at 6:52 AM, Rolf Rabenseifner <rabenseifner(a)hlrs.de> wrote: > Hi all together, > > I'm sitting in the MPI Forum Meeting and we discuss the new IEEE FP16 > datatype. > > Questions to the Fortran atandardization body: > > - Does the Fortran Standardization Body has already a proposal for > the new name in Fortran for this IEEE FP16? > > - Will be REAL*2 identical to IEEE FP16? > > Best regards > Rolf > > -- > Dr. Rolf Rabenseifner . . . . . . . . . .. email rabenseifner(a)hlrs.de . > High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 . > University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 . > Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner . > Nobelstr. 19, D-70550 Stuttgart, Germany . . . . (Office: Room 1.307) . > _______________________________________________ > mpiwg-p2p mailing list > mpiwg-p2p(a)lists.mpi-forum.org > https://lists.mpi-forum.org/mailman/listinfo/mpiwg-p2p > -- Tony Skjellum, PhD RunTime Computing Solutions, LLC tony(a)runtimecomputing.com direct: +1-205-918-7514 cell: +1-205-807-4968

1 0