June 2017 - mpiwg-fortran - mailman3.cels.anl.gov

Re: [MPIWG Fortran] [mpiwg-p2p] FP16 16-bit-floating-point
by Rolf Rabenseifner 16 Jun '17

16 Jun '17

Sorry, I Forget to add the P2P-WG First, I want to remeber to Takahiro's answer aboout C. > http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2016/p0192r0.pdf > http://www.open-std.org/jtc1/sc22/wg14/www/docs/n2016.pdf They will probably add - short float, and - float16_t, float32_t, float64_t to the C Standard. For Fortran, the MPI standard currently supports floating points through - MPI_REAL for REAL - MPI_DOUBLE_PRECISION for DOUBLE PRECISION and only when supported by the Compiler (i.e. optional) - MPI_REAL2 for REAL*2 - MPI_REAL4 for REAL*4 - MPI_REAL8 for REAL*8 - MPI_REAL16 for REAL*16 The question is, whether Fortran will add something similar as "short float". Another question is, whether we never detected that Fortran has officially byte- or bit-sized floats, like the REAL*8. Rolf ----- Original Message ----- > From: "Kawashima, Takahiro" <t-kawashima(a)jp.fujitsu.com> > To: "MPI WG Point To Point Communications working group" <mpiwg-p2p(a)lists.mpi-forum.org> > Cc: "Rolf Rabenseifner" <rabenseifner(a)hlrs.de>, "MPI-WG Fortran working group" <mpiwg-fortran(a)lists.mpi-forum.org>, > "Anthony Skjellum" <tony(a)runtimecomputing.com>, "Reinhold Bader" <Reinhold.Bader(a)lrz-muenchen.de>, "Atsushi Hori" > <ahori(a)riken.jp>, "fortran standards email list for J3" <j3(a)j3-fortran.org> > Sent: Friday, June 16, 2017 3:16:57 PM > Subject: Re: [mpiwg-p2p] [MPIWG Fortran] FP16 16-bit-floating-point > Hi, > > I don't know the status of C standardization nor contact person > but the suggestion of 'short float' was seem to made in a WG. > > http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2016/p0192r0.pdf > http://www.open-std.org/jtc1/sc22/wg14/www/docs/n2016.pdf > > Regards, > Takahiro Kawashima, > Fujitsu ----- Original Message ----- > From: "longb" <longb(a)cray.com> > To: "MPI-WG Fortran working group" <mpiwg-fortran(a)lists.mpi-forum.org> > Cc: "Atsushi Hori" <ahori(a)riken.jp>, "Reinhold Bader" <Reinhold.Bader(a)lrz-muenchen.de>, "fortran standards email list > for J3" <j3(a)j3-fortran.org> > Sent: Friday, June 16, 2017 7:22:42 PM > Subject: Re: [MPIWG Fortran] [mpiwg-p2p] FP16 16-bit-floating-point >> On Jun 16, 2017, at 7:19 PM, Rolf Rabenseifner <rabenseifner(a)hlrs.de> wrote: >> >> Jeff is partially right: >> If the Fortran user uses selected_real_kind(p, r), then all is done. >> >> On the other hand, >> I expect that the Fortran standardization body will do the same >> as the C standardization body, namely to add a new language Basic type. > > No. The basic type is REAL. This is just another KIND of REAL. This is not > much of an issue for the standard. It is a HUGE deal for implementations. > > Cheers, > Bill > > >> >> For this of course, we have to provide the corresponding MPI predefined >> datatype. >> Rolf >> >> >> ----- Original Message ----- >>> From: "Jeff Hammond" <jeff.science(a)gmail.com> >>> To: "MPI WG Point To Point Communications working group" >>> <mpiwg-p2p(a)lists.mpi-forum.org> >>> Cc: "Atsushi Hori" <ahori(a)riken.jp>, "Reinhold Bader" >>> <Reinhold.Bader(a)lrz-muenchen.de>, "fortran standards email list >>> for J3" <j3(a)j3-fortran.org>, "MPI-WG Fortran working group" >>> <mpiwg-fortran(a)lists.mpi-forum.org> >>> Sent: Friday, June 16, 2017 5:11:24 PM >>> Subject: Re: [MPIWG Fortran] [mpiwg-p2p] FP16 16-bit-floating-point >> >>> No change to the MPI standard is required here. MPI_Type_create_f90_real is >>> already sufficient. >>> >>> "This function returns a predefined MPI datatype that matches a REAL variable of >>> KIND selected_real_kind(p, r)." - I do not see why we need to enumerate a >>> predefined datatype name associated with float16 in Fortran for this statement >>> to be true. An implementation can return a datatype with the appropriate >>> properties without being enumerated like MPI_REAL. >>> >>> "It is erroneous to supply values for p and r not supported by the compiler." - >>> Conversely, the user should be allowed to supply values that are supported by >>> the compiler (although the standard doesn't guarentee it today). MPI >>> implementations might not support every type the compiler does, as is already >>> the case for Fortran float128 (because MPI libraries are written in C, and do >>> not call back into Fortran to do math on these types, which may not be >>> identical to C extensions like __float128), but I consider that an >>> implementation defect. >>> >>> Thus, I propose to merely add the text that allows users to rely on >>> MPI_Type_create_f90_real for float16 in Fortran, provided that the compiler >>> supports it. It will then be up to implementations to figure out how to do math >>> on these types internally. >>> >>> While we are on this subject, we should be considering the more general >>> inadequacy of the current state of the MPI type system from the perspective of >>> reductions. See [ https://github.com/mpi-forum/mpi-issues/issues/18 | >>> https://github.com/mpi-forum/mpi-issues/issues/18 ] for details. >>> >>> As for ISO C/C++, I can talk to Clark Nelson if a better candidate is not >>> identified, but Lawrence Crowl is leading the C++ effort and I suspect the C >>> folks will follow, although I will confirm this with Clark. >>> >>> Jeff >>> >>> On Fri, Jun 16, 2017 at 6:16 AM, Kawashima, Takahiro < [ >>> mailto:[email protected] | t-kawashima(a)jp.fujitsu.com ] > wrote: >>> >>> >>> Hi, >>> >>> I don't know the status of C standardization nor contact person >>> but the suggestion of 'short float' was seem to made in a WG. >>> >>> [ http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2016/p0192r0.pdf | >>> http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2016/p0192r0.pdf ] >>> [ http://www.open-std.org/jtc1/sc22/wg14/www/docs/n2016.pdf | >>> http://www.open-std.org/jtc1/sc22/wg14/www/docs/n2016.pdf ] >>> >>> Regards, >>> Takahiro Kawashima, >>> Fujitsu >>> >>>> Tony, >>>> >>>> we try to find the type names used in C and Fortran before we define >>>> the new predefined MPI datatype handles for representing those >>>> language types. >>>> >>>> Who knows, whom we can contact on the C standardization side? >>>> >>>> Best regards >>>> Rolf >>>> >>>> ----- Original Message ----- >>>>> From: "Anthony Skjellum" < [ mailto:[email protected] | >>>>> tony(a)runtimecomputing.com ] > >>>>> To: "MPI WG Point To Point Communications working group" < [ >>>>> mailto:[email protected] | mpiwg-p2p(a)lists.mpi-forum.org ] > >>>>> Cc: "Reinhold Bader" < [ mailto:[email protected] | >>>>> Reinhold.Bader(a)lrz-muenchen.de ] >, "Atsushi Hori" < [ mailto:[email protected] | >>>>> ahori(a)riken.jp ] >, "fortran standards email list >>>>> for J3" < [ mailto:[email protected] | j3(a)j3-fortran.org ] >, "MPI-WG Fortran >>>>> working group" < [ mailto:[email protected] | >>>>> mpiwg-fortran(a)lists.mpi-forum.org ] > >>>>> Sent: Friday, June 16, 2017 2:50:00 PM >>>>> Subject: Re: [MPIWG Fortran] [mpiwg-p2p] FP16 16-bit-floating-point >>>> >>>>> Rolf, apart from the Fortan angle, may I strongly encourage that we adopt this >>>>> datatype for MPI-4. It is part of real MPI codes already, but now we have to >>>>> fake transfer datatypes as 16-bit integers or bytes... which means not really >>>>> right if heterogeneous interoperability were to be maintained. And this is a >>>>> canonical IEEE type now, as you note. There is also the "2 half-float" datatype >>>>> (32-bits) at least in CUDA to be looked at; maybe ignored or maybe supported, >>>>> but it is there too. Key to this MPI users and applications use half floats and >>>>> 2 half-floats :-) in practice. >>>>> >>>>> May I also suggest a forward look at the types in AVX-512, in case any of these >>>>> become particularly relevant in future. >>>>> At least 8-bit integers are conceptually new, apart from BYTE. [ >>>>> [ https://en.wikipedia.org/wiki/AVX-512 | https://en.wikipedia.org/wiki/AVX-512 >>>>> ] | [ https://en.wikipedia.org/wiki/AVX-512 | >>>>> https://en.wikipedia.org/wiki/AVX-512 ] ] >>>>> >>>>> Regards, >>>>> Tony >>>>> >>>>> >>>>> On Fri, Jun 16, 2017 at 6:52 AM, Rolf Rabenseifner < [ >>>>> mailto: [ mailto:[email protected] | rabenseifner(a)hlrs.de ] | [ >>>>> mailto:[email protected] | rabenseifner(a)hlrs.de ] ] > wrote: >>>>> >>>>> >>>>> Hi all together, >>>>> >>>>> I'm sitting in the MPI Forum Meeting and we discuss the new IEEE FP16 >>>>> datatype. >>>>> >>>>> Questions to the Fortran atandardization body: >>>>> >>>>> - Does the Fortran Standardization Body has already a proposal for >>>>> the new name in Fortran for this IEEE FP16? >>>>> >>>>> - Will be REAL*2 identical to IEEE FP16? >>>>> >>>>> Best regards >>>>> Rolf >>> _______________________________________________ >>> mpiwg-p2p mailing list >>> [ mailto:[email protected] | mpiwg-p2p(a)lists.mpi-forum.org ] >>> [ https://lists.mpi-forum.org/mailman/listinfo/mpiwg-p2p | >>> https://lists.mpi-forum.org/mailman/listinfo/mpiwg-p2p ] >>> >>> >>> >>> -- >>> Jeff Hammond >>> [ mailto:[email protected] | jeff.science(a)gmail.com ] >>> [ http://jeffhammond.github.io/ | http://jeffhammond.github.io/ ] >>> >>> _______________________________________________ >>> mpiwg-fortran mailing list >>> mpiwg-fortran(a)lists.mpi-forum.org >>> https://lists.mpi-forum.org/mailman/listinfo/mpiwg-fortran >> >> -- >> Dr. Rolf Rabenseifner . . . . . . . . . .. email rabenseifner(a)hlrs.de . >> High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 . >> University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 . >> Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner . >> Nobelstr. 19, D-70550 Stuttgart, Germany . . . . (Office: Room 1.307) . >> _______________________________________________ >> mpiwg-fortran mailing list >> mpiwg-fortran(a)lists.mpi-forum.org >> https://lists.mpi-forum.org/mailman/listinfo/mpiwg-fortran > > Bill Long > longb(a)cray.com > Principal Engineer, Fortran Technical Support & voice: 651-605-9024 > Bioinformatics Software Development fax: 651-605-9143 > Cray Inc./ 2131 Lindau Lane/ Suite 1000/ Bloomington, MN 55425 > > > _______________________________________________ > mpiwg-fortran mailing list > mpiwg-fortran(a)lists.mpi-forum.org > https://lists.mpi-forum.org/mailman/listinfo/mpiwg-fortran -- Dr. Rolf Rabenseifner . . . . . . . . . .. email rabenseifner(a)hlrs.de . High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 . University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 . Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner . Nobelstr. 19, D-70550 Stuttgart, Germany . . . . (Office: Room 1.307) . _______________________________________________ mpiwg-fortran mailing list mpiwg-fortran(a)lists.mpi-forum.org https://lists.mpi-forum.org/mailman/listinfo/mpiwg-fortran

1 0

Re: [MPIWG Fortran] [mpiwg-p2p] FP16 16-bit-floating-point
by Rolf Rabenseifner 16 Jun '17

16 Jun '17

First, I want to remeber to Takahiro's answer aboout C. > http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2016/p0192r0.pdf > http://www.open-std.org/jtc1/sc22/wg14/www/docs/n2016.pdf They will probably add - short float, and - float16_t, float32_t, float64_t to the C Standard. For Fortran, the MPI standard currently supports floating points through - MPI_REAL for REAL - MPI_DOUBLE_PRECISION for DOUBLE PRECISION and only when supported by the Compiler (i.e. optional) - MPI_REAL2 for REAL*2 - MPI_REAL4 for REAL*4 - MPI_REAL8 for REAL*8 - MPI_REAL16 for REAL*16 The question is, whether Fortran will add something similar as "short float". Another question is, whether we never detected that Fortran has officially byte- or bit-sized floats, like the REAL*8. Rolf ----- Original Message ----- > From: "Kawashima, Takahiro" <t-kawashima(a)jp.fujitsu.com> > To: "MPI WG Point To Point Communications working group" <mpiwg-p2p(a)lists.mpi-forum.org> > Cc: "Rolf Rabenseifner" <rabenseifner(a)hlrs.de>, "MPI-WG Fortran working group" <mpiwg-fortran(a)lists.mpi-forum.org>, > "Anthony Skjellum" <tony(a)runtimecomputing.com>, "Reinhold Bader" <Reinhold.Bader(a)lrz-muenchen.de>, "Atsushi Hori" > <ahori(a)riken.jp>, "fortran standards email list for J3" <j3(a)j3-fortran.org> > Sent: Friday, June 16, 2017 3:16:57 PM > Subject: Re: [mpiwg-p2p] [MPIWG Fortran] FP16 16-bit-floating-point > Hi, > > I don't know the status of C standardization nor contact person > but the suggestion of 'short float' was seem to made in a WG. > > http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2016/p0192r0.pdf > http://www.open-std.org/jtc1/sc22/wg14/www/docs/n2016.pdf > > Regards, > Takahiro Kawashima, > Fujitsu ----- Original Message ----- > From: "longb" <longb(a)cray.com> > To: "MPI-WG Fortran working group" <mpiwg-fortran(a)lists.mpi-forum.org> > Cc: "Atsushi Hori" <ahori(a)riken.jp>, "Reinhold Bader" <Reinhold.Bader(a)lrz-muenchen.de>, "fortran standards email list > for J3" <j3(a)j3-fortran.org> > Sent: Friday, June 16, 2017 7:22:42 PM > Subject: Re: [MPIWG Fortran] [mpiwg-p2p] FP16 16-bit-floating-point >> On Jun 16, 2017, at 7:19 PM, Rolf Rabenseifner <rabenseifner(a)hlrs.de> wrote: >> >> Jeff is partially right: >> If the Fortran user uses selected_real_kind(p, r), then all is done. >> >> On the other hand, >> I expect that the Fortran standardization body will do the same >> as the C standardization body, namely to add a new language Basic type. > > No. The basic type is REAL. This is just another KIND of REAL. This is not > much of an issue for the standard. It is a HUGE deal for implementations. > > Cheers, > Bill > > >> >> For this of course, we have to provide the corresponding MPI predefined >> datatype. >> Rolf >> >> >> ----- Original Message ----- >>> From: "Jeff Hammond" <jeff.science(a)gmail.com> >>> To: "MPI WG Point To Point Communications working group" >>> <mpiwg-p2p(a)lists.mpi-forum.org> >>> Cc: "Atsushi Hori" <ahori(a)riken.jp>, "Reinhold Bader" >>> <Reinhold.Bader(a)lrz-muenchen.de>, "fortran standards email list >>> for J3" <j3(a)j3-fortran.org>, "MPI-WG Fortran working group" >>> <mpiwg-fortran(a)lists.mpi-forum.org> >>> Sent: Friday, June 16, 2017 5:11:24 PM >>> Subject: Re: [MPIWG Fortran] [mpiwg-p2p] FP16 16-bit-floating-point >> >>> No change to the MPI standard is required here. MPI_Type_create_f90_real is >>> already sufficient. >>> >>> "This function returns a predefined MPI datatype that matches a REAL variable of >>> KIND selected_real_kind(p, r)." - I do not see why we need to enumerate a >>> predefined datatype name associated with float16 in Fortran for this statement >>> to be true. An implementation can return a datatype with the appropriate >>> properties without being enumerated like MPI_REAL. >>> >>> "It is erroneous to supply values for p and r not supported by the compiler." - >>> Conversely, the user should be allowed to supply values that are supported by >>> the compiler (although the standard doesn't guarentee it today). MPI >>> implementations might not support every type the compiler does, as is already >>> the case for Fortran float128 (because MPI libraries are written in C, and do >>> not call back into Fortran to do math on these types, which may not be >>> identical to C extensions like __float128), but I consider that an >>> implementation defect. >>> >>> Thus, I propose to merely add the text that allows users to rely on >>> MPI_Type_create_f90_real for float16 in Fortran, provided that the compiler >>> supports it. It will then be up to implementations to figure out how to do math >>> on these types internally. >>> >>> While we are on this subject, we should be considering the more general >>> inadequacy of the current state of the MPI type system from the perspective of >>> reductions. See [ https://github.com/mpi-forum/mpi-issues/issues/18 | >>> https://github.com/mpi-forum/mpi-issues/issues/18 ] for details. >>> >>> As for ISO C/C++, I can talk to Clark Nelson if a better candidate is not >>> identified, but Lawrence Crowl is leading the C++ effort and I suspect the C >>> folks will follow, although I will confirm this with Clark. >>> >>> Jeff >>> >>> On Fri, Jun 16, 2017 at 6:16 AM, Kawashima, Takahiro < [ >>> mailto:[email protected] | t-kawashima(a)jp.fujitsu.com ] > wrote: >>> >>> >>> Hi, >>> >>> I don't know the status of C standardization nor contact person >>> but the suggestion of 'short float' was seem to made in a WG. >>> >>> [ http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2016/p0192r0.pdf | >>> http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2016/p0192r0.pdf ] >>> [ http://www.open-std.org/jtc1/sc22/wg14/www/docs/n2016.pdf | >>> http://www.open-std.org/jtc1/sc22/wg14/www/docs/n2016.pdf ] >>> >>> Regards, >>> Takahiro Kawashima, >>> Fujitsu >>> >>>> Tony, >>>> >>>> we try to find the type names used in C and Fortran before we define >>>> the new predefined MPI datatype handles for representing those >>>> language types. >>>> >>>> Who knows, whom we can contact on the C standardization side? >>>> >>>> Best regards >>>> Rolf >>>> >>>> ----- Original Message ----- >>>>> From: "Anthony Skjellum" < [ mailto:[email protected] | >>>>> tony(a)runtimecomputing.com ] > >>>>> To: "MPI WG Point To Point Communications working group" < [ >>>>> mailto:[email protected] | mpiwg-p2p(a)lists.mpi-forum.org ] > >>>>> Cc: "Reinhold Bader" < [ mailto:[email protected] | >>>>> Reinhold.Bader(a)lrz-muenchen.de ] >, "Atsushi Hori" < [ mailto:[email protected] | >>>>> ahori(a)riken.jp ] >, "fortran standards email list >>>>> for J3" < [ mailto:[email protected] | j3(a)j3-fortran.org ] >, "MPI-WG Fortran >>>>> working group" < [ mailto:[email protected] | >>>>> mpiwg-fortran(a)lists.mpi-forum.org ] > >>>>> Sent: Friday, June 16, 2017 2:50:00 PM >>>>> Subject: Re: [MPIWG Fortran] [mpiwg-p2p] FP16 16-bit-floating-point >>>> >>>>> Rolf, apart from the Fortan angle, may I strongly encourage that we adopt this >>>>> datatype for MPI-4. It is part of real MPI codes already, but now we have to >>>>> fake transfer datatypes as 16-bit integers or bytes... which means not really >>>>> right if heterogeneous interoperability were to be maintained. And this is a >>>>> canonical IEEE type now, as you note. There is also the "2 half-float" datatype >>>>> (32-bits) at least in CUDA to be looked at; maybe ignored or maybe supported, >>>>> but it is there too. Key to this MPI users and applications use half floats and >>>>> 2 half-floats :-) in practice. >>>>> >>>>> May I also suggest a forward look at the types in AVX-512, in case any of these >>>>> become particularly relevant in future. >>>>> At least 8-bit integers are conceptually new, apart from BYTE. [ >>>>> [ https://en.wikipedia.org/wiki/AVX-512 | https://en.wikipedia.org/wiki/AVX-512 >>>>> ] | [ https://en.wikipedia.org/wiki/AVX-512 | >>>>> https://en.wikipedia.org/wiki/AVX-512 ] ] >>>>> >>>>> Regards, >>>>> Tony >>>>> >>>>> >>>>> On Fri, Jun 16, 2017 at 6:52 AM, Rolf Rabenseifner < [ >>>>> mailto: [ mailto:[email protected] | rabenseifner(a)hlrs.de ] | [ >>>>> mailto:[email protected] | rabenseifner(a)hlrs.de ] ] > wrote: >>>>> >>>>> >>>>> Hi all together, >>>>> >>>>> I'm sitting in the MPI Forum Meeting and we discuss the new IEEE FP16 >>>>> datatype. >>>>> >>>>> Questions to the Fortran atandardization body: >>>>> >>>>> - Does the Fortran Standardization Body has already a proposal for >>>>> the new name in Fortran for this IEEE FP16? >>>>> >>>>> - Will be REAL*2 identical to IEEE FP16? >>>>> >>>>> Best regards >>>>> Rolf >>> _______________________________________________ >>> mpiwg-p2p mailing list >>> [ mailto:[email protected] | mpiwg-p2p(a)lists.mpi-forum.org ] >>> [ https://lists.mpi-forum.org/mailman/listinfo/mpiwg-p2p | >>> https://lists.mpi-forum.org/mailman/listinfo/mpiwg-p2p ] >>> >>> >>> >>> -- >>> Jeff Hammond >>> [ mailto:[email protected] | jeff.science(a)gmail.com ] >>> [ http://jeffhammond.github.io/ | http://jeffhammond.github.io/ ] >>> >>> _______________________________________________ >>> mpiwg-fortran mailing list >>> mpiwg-fortran(a)lists.mpi-forum.org >>> https://lists.mpi-forum.org/mailman/listinfo/mpiwg-fortran >> >> -- >> Dr. Rolf Rabenseifner . . . . . . . . . .. email rabenseifner(a)hlrs.de . >> High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 . >> University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 . >> Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner . >> Nobelstr. 19, D-70550 Stuttgart, Germany . . . . (Office: Room 1.307) . >> _______________________________________________ >> mpiwg-fortran mailing list >> mpiwg-fortran(a)lists.mpi-forum.org >> https://lists.mpi-forum.org/mailman/listinfo/mpiwg-fortran > > Bill Long > longb(a)cray.com > Principal Engineer, Fortran Technical Support & voice: 651-605-9024 > Bioinformatics Software Development fax: 651-605-9143 > Cray Inc./ 2131 Lindau Lane/ Suite 1000/ Bloomington, MN 55425 > > > _______________________________________________ > mpiwg-fortran mailing list > mpiwg-fortran(a)lists.mpi-forum.org > https://lists.mpi-forum.org/mailman/listinfo/mpiwg-fortran -- Dr. Rolf Rabenseifner . . . . . . . . . .. email rabenseifner(a)hlrs.de . High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 . University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 . Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner . Nobelstr. 19, D-70550 Stuttgart, Germany . . . . (Office: Room 1.307) .

1 0

Re: [MPIWG Fortran] [mpiwg-p2p] FP16 16-bit-floating-point
by Bill Long 16 Jun '17

16 Jun '17

> On Jun 16, 2017, at 7:19 PM, Rolf Rabenseifner <rabenseifner(a)hlrs.de> wrote: > > Jeff is partially right: > If the Fortran user uses selected_real_kind(p, r), then all is done. > > On the other hand, > I expect that the Fortran standardization body will do the same > as the C standardization body, namely to add a new language Basic type. No. The basic type is REAL. This is just another KIND of REAL. This is not much of an issue for the standard. It is a HUGE deal for implementations. Cheers, Bill > > For this of course, we have to provide the corresponding MPI predefined datatype. > Rolf > > > ----- Original Message ----- >> From: "Jeff Hammond" <jeff.science(a)gmail.com> >> To: "MPI WG Point To Point Communications working group" <mpiwg-p2p(a)lists.mpi-forum.org> >> Cc: "Atsushi Hori" <ahori(a)riken.jp>, "Reinhold Bader" <Reinhold.Bader(a)lrz-muenchen.de>, "fortran standards email list >> for J3" <j3(a)j3-fortran.org>, "MPI-WG Fortran working group" <mpiwg-fortran(a)lists.mpi-forum.org> >> Sent: Friday, June 16, 2017 5:11:24 PM >> Subject: Re: [MPIWG Fortran] [mpiwg-p2p] FP16 16-bit-floating-point > >> No change to the MPI standard is required here. MPI_Type_create_f90_real is >> already sufficient. >> >> "This function returns a predefined MPI datatype that matches a REAL variable of >> KIND selected_real_kind(p, r)." - I do not see why we need to enumerate a >> predefined datatype name associated with float16 in Fortran for this statement >> to be true. An implementation can return a datatype with the appropriate >> properties without being enumerated like MPI_REAL. >> >> "It is erroneous to supply values for p and r not supported by the compiler." - >> Conversely, the user should be allowed to supply values that are supported by >> the compiler (although the standard doesn't guarentee it today). MPI >> implementations might not support every type the compiler does, as is already >> the case for Fortran float128 (because MPI libraries are written in C, and do >> not call back into Fortran to do math on these types, which may not be >> identical to C extensions like __float128), but I consider that an >> implementation defect. >> >> Thus, I propose to merely add the text that allows users to rely on >> MPI_Type_create_f90_real for float16 in Fortran, provided that the compiler >> supports it. It will then be up to implementations to figure out how to do math >> on these types internally. >> >> While we are on this subject, we should be considering the more general >> inadequacy of the current state of the MPI type system from the perspective of >> reductions. See [ https://github.com/mpi-forum/mpi-issues/issues/18 | >> https://github.com/mpi-forum/mpi-issues/issues/18 ] for details. >> >> As for ISO C/C++, I can talk to Clark Nelson if a better candidate is not >> identified, but Lawrence Crowl is leading the C++ effort and I suspect the C >> folks will follow, although I will confirm this with Clark. >> >> Jeff >> >> On Fri, Jun 16, 2017 at 6:16 AM, Kawashima, Takahiro < [ >> mailto:[email protected] | t-kawashima(a)jp.fujitsu.com ] > wrote: >> >> >> Hi, >> >> I don't know the status of C standardization nor contact person >> but the suggestion of 'short float' was seem to made in a WG. >> >> [ http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2016/p0192r0.pdf | >> http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2016/p0192r0.pdf ] >> [ http://www.open-std.org/jtc1/sc22/wg14/www/docs/n2016.pdf | >> http://www.open-std.org/jtc1/sc22/wg14/www/docs/n2016.pdf ] >> >> Regards, >> Takahiro Kawashima, >> Fujitsu >> >>> Tony, >>> >>> we try to find the type names used in C and Fortran before we define >>> the new predefined MPI datatype handles for representing those >>> language types. >>> >>> Who knows, whom we can contact on the C standardization side? >>> >>> Best regards >>> Rolf >>> >>> ----- Original Message ----- >>>> From: "Anthony Skjellum" < [ mailto:[email protected] | >>>> tony(a)runtimecomputing.com ] > >>>> To: "MPI WG Point To Point Communications working group" < [ >>>> mailto:[email protected] | mpiwg-p2p(a)lists.mpi-forum.org ] > >>>> Cc: "Reinhold Bader" < [ mailto:[email protected] | >>>> Reinhold.Bader(a)lrz-muenchen.de ] >, "Atsushi Hori" < [ mailto:[email protected] | >>>> ahori(a)riken.jp ] >, "fortran standards email list >>>> for J3" < [ mailto:[email protected] | j3(a)j3-fortran.org ] >, "MPI-WG Fortran >>>> working group" < [ mailto:[email protected] | >>>> mpiwg-fortran(a)lists.mpi-forum.org ] > >>>> Sent: Friday, June 16, 2017 2:50:00 PM >>>> Subject: Re: [MPIWG Fortran] [mpiwg-p2p] FP16 16-bit-floating-point >>> >>>> Rolf, apart from the Fortan angle, may I strongly encourage that we adopt this >>>> datatype for MPI-4. It is part of real MPI codes already, but now we have to >>>> fake transfer datatypes as 16-bit integers or bytes... which means not really >>>> right if heterogeneous interoperability were to be maintained. And this is a >>>> canonical IEEE type now, as you note. There is also the "2 half-float" datatype >>>> (32-bits) at least in CUDA to be looked at; maybe ignored or maybe supported, >>>> but it is there too. Key to this MPI users and applications use half floats and >>>> 2 half-floats :-) in practice. >>>> >>>> May I also suggest a forward look at the types in AVX-512, in case any of these >>>> become particularly relevant in future. >>>> At least 8-bit integers are conceptually new, apart from BYTE. [ >>>> [ https://en.wikipedia.org/wiki/AVX-512 | https://en.wikipedia.org/wiki/AVX-512 >>>> ] | [ https://en.wikipedia.org/wiki/AVX-512 | >>>> https://en.wikipedia.org/wiki/AVX-512 ] ] >>>> >>>> Regards, >>>> Tony >>>> >>>> >>>> On Fri, Jun 16, 2017 at 6:52 AM, Rolf Rabenseifner < [ >>>> mailto: [ mailto:[email protected] | rabenseifner(a)hlrs.de ] | [ >>>> mailto:[email protected] | rabenseifner(a)hlrs.de ] ] > wrote: >>>> >>>> >>>> Hi all together, >>>> >>>> I'm sitting in the MPI Forum Meeting and we discuss the new IEEE FP16 >>>> datatype. >>>> >>>> Questions to the Fortran atandardization body: >>>> >>>> - Does the Fortran Standardization Body has already a proposal for >>>> the new name in Fortran for this IEEE FP16? >>>> >>>> - Will be REAL*2 identical to IEEE FP16? >>>> >>>> Best regards >>>> Rolf >> _______________________________________________ >> mpiwg-p2p mailing list >> [ mailto:[email protected] | mpiwg-p2p(a)lists.mpi-forum.org ] >> [ https://lists.mpi-forum.org/mailman/listinfo/mpiwg-p2p | >> https://lists.mpi-forum.org/mailman/listinfo/mpiwg-p2p ] >> >> >> >> -- >> Jeff Hammond >> [ mailto:[email protected] | jeff.science(a)gmail.com ] >> [ http://jeffhammond.github.io/ | http://jeffhammond.github.io/ ] >> >> _______________________________________________ >> mpiwg-fortran mailing list >> mpiwg-fortran(a)lists.mpi-forum.org >> https://lists.mpi-forum.org/mailman/listinfo/mpiwg-fortran > > -- > Dr. Rolf Rabenseifner . . . . . . . . . .. email rabenseifner(a)hlrs.de . > High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 . > University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 . > Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner . > Nobelstr. 19, D-70550 Stuttgart, Germany . . . . (Office: Room 1.307) . > _______________________________________________ > mpiwg-fortran mailing list > mpiwg-fortran(a)lists.mpi-forum.org > https://lists.mpi-forum.org/mailman/listinfo/mpiwg-fortran Bill Long longb(a)cray.com Principal Engineer, Fortran Technical Support & voice: 651-605-9024 Bioinformatics Software Development fax: 651-605-9143 Cray Inc./ 2131 Lindau Lane/ Suite 1000/ Bloomington, MN 55425

1 0

Re: [MPIWG Fortran] [mpiwg-p2p] FP16 16-bit-floating-point
by Rolf Rabenseifner 16 Jun '17

16 Jun '17

Jeff is partially right: If the Fortran user uses selected_real_kind(p, r), then all is done. On the other hand, I expect that the Fortran standardization body will do the same as the C standardization body, namely to add a new language Basic type. For this of course, we have to provide the corresponding MPI predefined datatype. Rolf ----- Original Message ----- > From: "Jeff Hammond" <jeff.science(a)gmail.com> > To: "MPI WG Point To Point Communications working group" <mpiwg-p2p(a)lists.mpi-forum.org> > Cc: "Atsushi Hori" <ahori(a)riken.jp>, "Reinhold Bader" <Reinhold.Bader(a)lrz-muenchen.de>, "fortran standards email list > for J3" <j3(a)j3-fortran.org>, "MPI-WG Fortran working group" <mpiwg-fortran(a)lists.mpi-forum.org> > Sent: Friday, June 16, 2017 5:11:24 PM > Subject: Re: [MPIWG Fortran] [mpiwg-p2p] FP16 16-bit-floating-point > No change to the MPI standard is required here. MPI_Type_create_f90_real is > already sufficient. > > "This function returns a predefined MPI datatype that matches a REAL variable of > KIND selected_real_kind(p, r)." - I do not see why we need to enumerate a > predefined datatype name associated with float16 in Fortran for this statement > to be true. An implementation can return a datatype with the appropriate > properties without being enumerated like MPI_REAL. > > "It is erroneous to supply values for p and r not supported by the compiler." - > Conversely, the user should be allowed to supply values that are supported by > the compiler (although the standard doesn't guarentee it today). MPI > implementations might not support every type the compiler does, as is already > the case for Fortran float128 (because MPI libraries are written in C, and do > not call back into Fortran to do math on these types, which may not be > identical to C extensions like __float128), but I consider that an > implementation defect. > > Thus, I propose to merely add the text that allows users to rely on > MPI_Type_create_f90_real for float16 in Fortran, provided that the compiler > supports it. It will then be up to implementations to figure out how to do math > on these types internally. > > While we are on this subject, we should be considering the more general > inadequacy of the current state of the MPI type system from the perspective of > reductions. See [ https://github.com/mpi-forum/mpi-issues/issues/18 | > https://github.com/mpi-forum/mpi-issues/issues/18 ] for details. > > As for ISO C/C++, I can talk to Clark Nelson if a better candidate is not > identified, but Lawrence Crowl is leading the C++ effort and I suspect the C > folks will follow, although I will confirm this with Clark. > > Jeff > > On Fri, Jun 16, 2017 at 6:16 AM, Kawashima, Takahiro < [ > mailto:[email protected] | t-kawashima(a)jp.fujitsu.com ] > wrote: > > > Hi, > > I don't know the status of C standardization nor contact person > but the suggestion of 'short float' was seem to made in a WG. > > [ http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2016/p0192r0.pdf | > http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2016/p0192r0.pdf ] > [ http://www.open-std.org/jtc1/sc22/wg14/www/docs/n2016.pdf | > http://www.open-std.org/jtc1/sc22/wg14/www/docs/n2016.pdf ] > > Regards, > Takahiro Kawashima, > Fujitsu > >> Tony, >> >> we try to find the type names used in C and Fortran before we define >> the new predefined MPI datatype handles for representing those >> language types. >> >> Who knows, whom we can contact on the C standardization side? >> >> Best regards >> Rolf >> >> ----- Original Message ----- >> > From: "Anthony Skjellum" < [ mailto:[email protected] | >> > tony(a)runtimecomputing.com ] > >> > To: "MPI WG Point To Point Communications working group" < [ >> > mailto:[email protected] | mpiwg-p2p(a)lists.mpi-forum.org ] > >> > Cc: "Reinhold Bader" < [ mailto:[email protected] | >> > Reinhold.Bader(a)lrz-muenchen.de ] >, "Atsushi Hori" < [ mailto:[email protected] | >> > ahori(a)riken.jp ] >, "fortran standards email list >> > for J3" < [ mailto:[email protected] | j3(a)j3-fortran.org ] >, "MPI-WG Fortran >> > working group" < [ mailto:[email protected] | >> > mpiwg-fortran(a)lists.mpi-forum.org ] > >> > Sent: Friday, June 16, 2017 2:50:00 PM >> > Subject: Re: [MPIWG Fortran] [mpiwg-p2p] FP16 16-bit-floating-point >> >> > Rolf, apart from the Fortan angle, may I strongly encourage that we adopt this >> > datatype for MPI-4. It is part of real MPI codes already, but now we have to >> > fake transfer datatypes as 16-bit integers or bytes... which means not really >> > right if heterogeneous interoperability were to be maintained. And this is a >> > canonical IEEE type now, as you note. There is also the "2 half-float" datatype >> > (32-bits) at least in CUDA to be looked at; maybe ignored or maybe supported, >> > but it is there too. Key to this MPI users and applications use half floats and >> > 2 half-floats :-) in practice. >> > >> > May I also suggest a forward look at the types in AVX-512, in case any of these >> > become particularly relevant in future. >> > At least 8-bit integers are conceptually new, apart from BYTE. [ >> > [ https://en.wikipedia.org/wiki/AVX-512 | https://en.wikipedia.org/wiki/AVX-512 >> > ] | [ https://en.wikipedia.org/wiki/AVX-512 | >> > https://en.wikipedia.org/wiki/AVX-512 ] ] >> > >> > Regards, >> > Tony >> > >> > >> > On Fri, Jun 16, 2017 at 6:52 AM, Rolf Rabenseifner < [ >> > mailto: [ mailto:[email protected] | rabenseifner(a)hlrs.de ] | [ >> > mailto:[email protected] | rabenseifner(a)hlrs.de ] ] > wrote: >> > >> > >> > Hi all together, >> > >> > I'm sitting in the MPI Forum Meeting and we discuss the new IEEE FP16 >> > datatype. >> > >> > Questions to the Fortran atandardization body: >> > >> > - Does the Fortran Standardization Body has already a proposal for >> > the new name in Fortran for this IEEE FP16? >> > >> > - Will be REAL*2 identical to IEEE FP16? >> > >> > Best regards >> > Rolf > _______________________________________________ > mpiwg-p2p mailing list > [ mailto:[email protected] | mpiwg-p2p(a)lists.mpi-forum.org ] > [ https://lists.mpi-forum.org/mailman/listinfo/mpiwg-p2p | > https://lists.mpi-forum.org/mailman/listinfo/mpiwg-p2p ] > > > > -- > Jeff Hammond > [ mailto:[email protected] | jeff.science(a)gmail.com ] > [ http://jeffhammond.github.io/ | http://jeffhammond.github.io/ ] > > _______________________________________________ > mpiwg-fortran mailing list > mpiwg-fortran(a)lists.mpi-forum.org > https://lists.mpi-forum.org/mailman/listinfo/mpiwg-fortran -- Dr. Rolf Rabenseifner . . . . . . . . . .. email rabenseifner(a)hlrs.de . High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 . University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 . Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner . Nobelstr. 19, D-70550 Stuttgart, Germany . . . . (Office: Room 1.307) .

1 0

Re: [MPIWG Fortran] FP16 16-bit-floating-point
by Bill Long 16 Jun '17

16 Jun '17

Hi Rolf, The Fortran 2015 draft is close to finished. Its IEEE-isms are based on ISO/IEC/IEEE 60559:2011, and includes a lot of changes based on that standard. If this FP16 feature is part of that standard, then we should look into making a change now. Otherwise, it would likely wait for the next revision. As Reinhold pointed out, the parameterized KIND scheme Fortran uses already permits identifying a KIND value for 16-bit floats, assuming the processor (i.e. compiler) knows about such a type. Changes to the standard would be minimal - add REAL16 to the iso_fortran_env module, with the value of -1 if that kind of real is not supported, and the value of 2 for most compilers if it is. Changes to the compiler would be significant. So implementation would likely wait for customer demand. Cheers, Bill > On Jun 16, 2017, at 2:44 PM, Rolf Rabenseifner <rabenseifner(a)hlrs.de> wrote: > > (Because I'm not member of j3, only Reinhold's or Rich's answers will be visible for j3) > > Based on Reinhold's answer, I make my questions more precise: > > - Are there already some activities in J3 on this FP16 topic? > > - When will the Fortran standard include REAL16 to support the new IEEE FP16 standard? > > Best regards > Rolf > > > ----- Original Message ----- >> From: "Reinhold Bader" <Reinhold.Bader(a)lrz.de> >> To: "Rolf Rabenseifner" <rabenseifner(a)hlrs.de>, "fortran standards email list for J3" <j3(a)j3-fortran.org> >> Cc: "MPI-WG Fortran working group" <mpiwg-fortran(a)lists.mpi-forum.org>, "Atsushi Hori" <ahori(a)riken.jp>, "Martin Schulz" >> <schulzm(a)llnl.gov>, "MPI WG Point To Point Communications working group" <mpiwg-p2p(a)lists.mpi-forum.org> >> Sent: Friday, June 16, 2017 2:31:03 PM >> Subject: AW: FP16 16-bit-floating-point > >> Hello Rolf, >> >> to the best of my knowledge, the answers are as follows: >> >> Q1: Not in the sense of having an explicit name like the REAL32, REAL64, REAL128 >> names for specific integer kinds that were added to Fortran 2008. >> However, it is left free to implementors to create a new kind number in the >> implementation that could be returned by SELECTED_REAL_KIND() and, >> if IEEE conforming, by IEEE_SELECTED_REAL_KIND() and consequently all relevant >> IEEE support routines. >> >> Q2: REAL*2 is (like REAL*8) a non-conforming notation (REAL*8 is supported as an >> extension by many implementations), so the answer is no. >> A proper declaration might look like >> >> integer, parameter :: rk = selected_real_kind(2,6) ! at least 2 decimal >> digits and 6 maximal decimal exponent - hope I have this right ... >> real(KIND=rk) :: x >> >> with the following caveats: >> * the KIND that fits the requirement most closely will be used, so it might be >> necessary to use the STORAGE_SIZE intrinsic to check what one gets. >> * KIND numbers are non-portable, so might vary for one and the same kind between >> implementations (at least one compiler I know deviates from all others I know). >> >> I checked three of the compilers I know: they currently do not support FP16. >> >> Regards >> Reinhold >> >> >>> -----Ursprüngliche Nachricht----- >>> Von: Rolf Rabenseifner [mailto:[email protected]] >>> Gesendet: Freitag, 16. Juni 2017 13:53 >>> An: fortran standards email list for J3 <j3(a)j3-fortran.org>; Bader, Reinhold >>> <Reinhold.Bader(a)lrz.de> >>> Cc: MPI-WG Fortran working group <mpiwg-fortran(a)lists.mpi-forum.org>; >>> Atsushi Hori <ahori(a)riken.jp>; Martin Schulz <schulzm(a)llnl.gov>; MPI WG >>> Point To Point Communications working group <mpiwg-p2p(a)lists.mpi- >>> forum.org> >>> Betreff: FP16 16-bit-floating-point >>> >>> Hi all together, >>> >>> I'm sitting in the MPI Forum Meeting and we discuss the new IEEE FP16 >>> datatype. >>> >>> Questions to the Fortran atandardization body: >>> >>> - Does the Fortran Standardization Body has already a proposal for >>> the new name in Fortran for this IEEE FP16? >>> >>> - Will be REAL*2 identical to IEEE FP16? >>> >>> Best regards >>> Rolf >>> >>> -- >>> Dr. Rolf Rabenseifner . . . . . . . . . .. email rabenseifner(a)hlrs.de . >>> High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 . >>> University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 . >>> Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner . >>> Nobelstr. 19, D-70550 Stuttgart, Germany . . . . (Office: Room 1.307) . > > -- > Dr. Rolf Rabenseifner . . . . . . . . . .. email rabenseifner(a)hlrs.de . > High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 . > University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 . > Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner . > Nobelstr. 19, D-70550 Stuttgart, Germany . . . . (Office: Room 1.307) . > _______________________________________________ > mpiwg-fortran mailing list > mpiwg-fortran(a)lists.mpi-forum.org > https://lists.mpi-forum.org/mailman/listinfo/mpiwg-fortran Bill Long longb(a)cray.com Principal Engineer, Fortran Technical Support & voice: 651-605-9024 Bioinformatics Software Development fax: 651-605-9143 Cray Inc./ 2131 Lindau Lane/ Suite 1000/ Bloomington, MN 55425

1 0

Re: [MPIWG Fortran] [mpiwg-p2p] FP16 16-bit-floating-point
by Jeff Hammond 16 Jun '17

16 Jun '17

No change to the MPI standard is required here. MPI_Type_create_f90_real is already sufficient. "This function returns a *predefined* MPI datatype that matches a REAL variable of KIND selected_real_kind(p, r)." - I do not see why we need to enumerate a predefined datatype name associated with float16 in Fortran for this statement to be true. An implementation can return a datatype with the appropriate properties without being enumerated like MPI_REAL. "It is erroneous to supply values for p and r not supported by the compiler." - Conversely, the user should be allowed to supply values that are supported by the compiler (although the standard doesn't guarentee it today). MPI implementations might not support every type the compiler does, as is already the case for Fortran float128 (because MPI libraries are written in C, and do not call back into Fortran to do math on these types, which may not be identical to C extensions like __float128), but I consider that an implementation defect. Thus, I propose to merely add the text that allows users to rely on MPI_Type_create_f90_real for float16 in Fortran, provided that the compiler supports it. It will then be up to implementations to figure out how to do math on these types internally. While we are on this subject, we should be considering the more general inadequacy of the current state of the MPI type system from the perspective of reductions. See https://github.com/mpi-forum/mpi-issues/issues/18 for details. As for ISO C/C++, I can talk to Clark Nelson if a better candidate is not identified, but Lawrence Crowl is leading the C++ effort and I suspect the C folks will follow, although I will confirm this with Clark. Jeff On Fri, Jun 16, 2017 at 6:16 AM, Kawashima, Takahiro < t-kawashima(a)jp.fujitsu.com> wrote: > Hi, > > I don't know the status of C standardization nor contact person > but the suggestion of 'short float' was seem to made in a WG. > > http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2016/p0192r0.pdf > http://www.open-std.org/jtc1/sc22/wg14/www/docs/n2016.pdf > > Regards, > Takahiro Kawashima, > Fujitsu > > > Tony, > > > > we try to find the type names used in C and Fortran before we define > > the new predefined MPI datatype handles for representing those > > language types. > > > > Who knows, whom we can contact on the C standardization side? > > > > Best regards > > Rolf > > > > ----- Original Message ----- > > > From: "Anthony Skjellum" <tony(a)runtimecomputing.com> > > > To: "MPI WG Point To Point Communications working group" < > mpiwg-p2p(a)lists.mpi-forum.org> > > > Cc: "Reinhold Bader" <Reinhold.Bader(a)lrz-muenchen.de>, "Atsushi Hori" > <ahori(a)riken.jp>, "fortran standards email list > > > for J3" <j3(a)j3-fortran.org>, "MPI-WG Fortran working group" < > mpiwg-fortran(a)lists.mpi-forum.org> > > > Sent: Friday, June 16, 2017 2:50:00 PM > > > Subject: Re: [MPIWG Fortran] [mpiwg-p2p] FP16 16-bit-floating-point > > > > > Rolf, apart from the Fortan angle, may I strongly encourage that we > adopt this > > > datatype for MPI-4. It is part of real MPI codes already, but now we > have to > > > fake transfer datatypes as 16-bit integers or bytes... which means not > really > > > right if heterogeneous interoperability were to be maintained. And > this is a > > > canonical IEEE type now, as you note. There is also the "2 half-float" > datatype > > > (32-bits) at least in CUDA to be looked at; maybe ignored or maybe > supported, > > > but it is there too. Key to this MPI users and applications use half > floats and > > > 2 half-floats :-) in practice. > > > > > > May I also suggest a forward look at the types in AVX-512, in case any > of these > > > become particularly relevant in future. > > > At least 8-bit integers are conceptually new, apart from BYTE. [ > > > https://en.wikipedia.org/wiki/AVX-512 | https://en.wikipedia.org/wiki/ > AVX-512 ] > > > > > > Regards, > > > Tony > > > > > > > > > On Fri, Jun 16, 2017 at 6:52 AM, Rolf Rabenseifner < [ > > > mailto:[email protected] | rabenseifner(a)hlrs.de ] > wrote: > > > > > > > > > Hi all together, > > > > > > I'm sitting in the MPI Forum Meeting and we discuss the new IEEE FP16 > > > datatype. > > > > > > Questions to the Fortran atandardization body: > > > > > > - Does the Fortran Standardization Body has already a proposal for > > > the new name in Fortran for this IEEE FP16? > > > > > > - Will be REAL*2 identical to IEEE FP16? > > > > > > Best regards > > > Rolf > _______________________________________________ > mpiwg-p2p mailing list > mpiwg-p2p(a)lists.mpi-forum.org > https://lists.mpi-forum.org/mailman/listinfo/mpiwg-p2p > -- Jeff Hammond jeff.science(a)gmail.com http://jeffhammond.github.io/

1 0

Re: [MPIWG Fortran] [mpiwg-p2p] FP16 16-bit-floating-point
by Rolf Rabenseifner 16 Jun '17

16 Jun '17

Tony, we try to find the type names used in C and Fortran before we define the new predefined MPI datatype handles for representing those language types. Who knows, whom we can contact on the C standardization side? Best regards Rolf ----- Original Message ----- > From: "Anthony Skjellum" <tony(a)runtimecomputing.com> > To: "MPI WG Point To Point Communications working group" <mpiwg-p2p(a)lists.mpi-forum.org> > Cc: "Reinhold Bader" <Reinhold.Bader(a)lrz-muenchen.de>, "Atsushi Hori" <ahori(a)riken.jp>, "fortran standards email list > for J3" <j3(a)j3-fortran.org>, "MPI-WG Fortran working group" <mpiwg-fortran(a)lists.mpi-forum.org> > Sent: Friday, June 16, 2017 2:50:00 PM > Subject: Re: [MPIWG Fortran] [mpiwg-p2p] FP16 16-bit-floating-point > Rolf, apart from the Fortan angle, may I strongly encourage that we adopt this > datatype for MPI-4. It is part of real MPI codes already, but now we have to > fake transfer datatypes as 16-bit integers or bytes... which means not really > right if heterogeneous interoperability were to be maintained. And this is a > canonical IEEE type now, as you note. There is also the "2 half-float" datatype > (32-bits) at least in CUDA to be looked at; maybe ignored or maybe supported, > but it is there too. Key to this MPI users and applications use half floats and > 2 half-floats :-) in practice. > > May I also suggest a forward look at the types in AVX-512, in case any of these > become particularly relevant in future. > At least 8-bit integers are conceptually new, apart from BYTE. [ > https://en.wikipedia.org/wiki/AVX-512 | https://en.wikipedia.org/wiki/AVX-512 ] > > Regards, > Tony > > > On Fri, Jun 16, 2017 at 6:52 AM, Rolf Rabenseifner < [ > mailto:[email protected] | rabenseifner(a)hlrs.de ] > wrote: > > > Hi all together, > > I'm sitting in the MPI Forum Meeting and we discuss the new IEEE FP16 > datatype. > > Questions to the Fortran atandardization body: > > - Does the Fortran Standardization Body has already a proposal for > the new name in Fortran for this IEEE FP16? > > - Will be REAL*2 identical to IEEE FP16? > > Best regards > Rolf > > -- > Dr. Rolf Rabenseifner . . . . . . . . . .. email [ mailto:[email protected] | > rabenseifner(a)hlrs.de ] . > High Performance Computing Center (HLRS) . phone [ > tel:%2B%2B49%280%29711%2F685-65530 | ++49(0)711/685-65530 ] . > University of Stuttgart . . . . . . . . .. fax [ > tel:%2B%2B49%280%29711%20%2F%20685-65832 | ++49(0)711 / 685-65832 ] . > Head of Dpmt Parallel Computing . . . [ http://www.hlrs.de/people/rabenseifner | > www.hlrs.de/people/rabenseifner ] . > Nobelstr. 19, D-70550 Stuttgart, Germany . . . . (Office: Room 1.307) . > _______________________________________________ > mpiwg-p2p mailing list > [ mailto:[email protected] | mpiwg-p2p(a)lists.mpi-forum.org ] > [ https://lists.mpi-forum.org/mailman/listinfo/mpiwg-p2p | > https://lists.mpi-forum.org/mailman/listinfo/mpiwg-p2p ] > > > > -- > Tony Skjellum, PhD > RunTime Computing Solutions, LLC > [ mailto:[email protected] | tony(a)runtimecomputing.com ] > direct: +1-205-918-7514 > cell: +1-205-807-4968 > > _______________________________________________ > mpiwg-fortran mailing list > mpiwg-fortran(a)lists.mpi-forum.org > https://lists.mpi-forum.org/mailman/listinfo/mpiwg-fortran -- Dr. Rolf Rabenseifner . . . . . . . . . .. email rabenseifner(a)hlrs.de . High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 . University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 . Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner . Nobelstr. 19, D-70550 Stuttgart, Germany . . . . (Office: Room 1.307) .

1 0

Re: [MPIWG Fortran] [mpiwg-p2p] FP16 16-bit-floating-point
by Anthony Skjellum 16 Jun '17

16 Jun '17

Rolf, apart from the Fortan angle, may I strongly encourage that we adopt this datatype for MPI-4. It is part of real MPI codes already, but now we have to fake transfer datatypes as 16-bit integers or bytes... which means not really right if heterogeneous interoperability were to be maintained. And this is a canonical IEEE type now, as you note. There is also the "2 half-float" datatype (32-bits) at least in CUDA to be looked at; maybe ignored or maybe supported, but it is there too. Key to this MPI users and applications use half floats and 2 half-floats :-) in practice. May I also suggest a forward look at the types in AVX-512, in case any of these become particularly relevant in future. At least 8-bit integers are conceptually new, apart from BYTE. https://en.wikipedia.org/wiki/AVX-512 Regards, Tony On Fri, Jun 16, 2017 at 6:52 AM, Rolf Rabenseifner <rabenseifner(a)hlrs.de> wrote: > Hi all together, > > I'm sitting in the MPI Forum Meeting and we discuss the new IEEE FP16 > datatype. > > Questions to the Fortran atandardization body: > > - Does the Fortran Standardization Body has already a proposal for > the new name in Fortran for this IEEE FP16? > > - Will be REAL*2 identical to IEEE FP16? > > Best regards > Rolf > > -- > Dr. Rolf Rabenseifner . . . . . . . . . .. email rabenseifner(a)hlrs.de . > High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 . > University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 . > Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner . > Nobelstr. 19, D-70550 Stuttgart, Germany . . . . (Office: Room 1.307) . > _______________________________________________ > mpiwg-p2p mailing list > mpiwg-p2p(a)lists.mpi-forum.org > https://lists.mpi-forum.org/mailman/listinfo/mpiwg-p2p > -- Tony Skjellum, PhD RunTime Computing Solutions, LLC tony(a)runtimecomputing.com direct: +1-205-918-7514 cell: +1-205-807-4968

1 0

Re: [MPIWG Fortran] FP16 16-bit-floating-point
by Rolf Rabenseifner 16 Jun '17

16 Jun '17

(Because I'm not member of j3, only Reinhold's or Rich's answers will be visible for j3) Based on Reinhold's answer, I make my questions more precise: - Are there already some activities in J3 on this FP16 topic? - When will the Fortran standard include REAL16 to support the new IEEE FP16 standard? Best regards Rolf ----- Original Message ----- > From: "Reinhold Bader" <Reinhold.Bader(a)lrz.de> > To: "Rolf Rabenseifner" <rabenseifner(a)hlrs.de>, "fortran standards email list for J3" <j3(a)j3-fortran.org> > Cc: "MPI-WG Fortran working group" <mpiwg-fortran(a)lists.mpi-forum.org>, "Atsushi Hori" <ahori(a)riken.jp>, "Martin Schulz" > <schulzm(a)llnl.gov>, "MPI WG Point To Point Communications working group" <mpiwg-p2p(a)lists.mpi-forum.org> > Sent: Friday, June 16, 2017 2:31:03 PM > Subject: AW: FP16 16-bit-floating-point > Hello Rolf, > > to the best of my knowledge, the answers are as follows: > > Q1: Not in the sense of having an explicit name like the REAL32, REAL64, REAL128 > names for specific integer kinds that were added to Fortran 2008. > However, it is left free to implementors to create a new kind number in the > implementation that could be returned by SELECTED_REAL_KIND() and, > if IEEE conforming, by IEEE_SELECTED_REAL_KIND() and consequently all relevant > IEEE support routines. > > Q2: REAL*2 is (like REAL*8) a non-conforming notation (REAL*8 is supported as an > extension by many implementations), so the answer is no. > A proper declaration might look like > > integer, parameter :: rk = selected_real_kind(2,6) ! at least 2 decimal > digits and 6 maximal decimal exponent - hope I have this right ... > real(KIND=rk) :: x > > with the following caveats: > * the KIND that fits the requirement most closely will be used, so it might be > necessary to use the STORAGE_SIZE intrinsic to check what one gets. > * KIND numbers are non-portable, so might vary for one and the same kind between > implementations (at least one compiler I know deviates from all others I know). > > I checked three of the compilers I know: they currently do not support FP16. > > Regards > Reinhold > > >> -----Ursprüngliche Nachricht----- >> Von: Rolf Rabenseifner [mailto:[email protected]] >> Gesendet: Freitag, 16. Juni 2017 13:53 >> An: fortran standards email list for J3 <j3(a)j3-fortran.org>; Bader, Reinhold >> <Reinhold.Bader(a)lrz.de> >> Cc: MPI-WG Fortran working group <mpiwg-fortran(a)lists.mpi-forum.org>; >> Atsushi Hori <ahori(a)riken.jp>; Martin Schulz <schulzm(a)llnl.gov>; MPI WG >> Point To Point Communications working group <mpiwg-p2p(a)lists.mpi- >> forum.org> >> Betreff: FP16 16-bit-floating-point >> >> Hi all together, >> >> I'm sitting in the MPI Forum Meeting and we discuss the new IEEE FP16 >> datatype. >> >> Questions to the Fortran atandardization body: >> >> - Does the Fortran Standardization Body has already a proposal for >> the new name in Fortran for this IEEE FP16? >> >> - Will be REAL*2 identical to IEEE FP16? >> >> Best regards >> Rolf >> >> -- >> Dr. Rolf Rabenseifner . . . . . . . . . .. email rabenseifner(a)hlrs.de . >> High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 . >> University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 . >> Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner . > > Nobelstr. 19, D-70550 Stuttgart, Germany . . . . (Office: Room 1.307) . -- Dr. Rolf Rabenseifner . . . . . . . . . .. email rabenseifner(a)hlrs.de . High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 . University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 . Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner . Nobelstr. 19, D-70550 Stuttgart, Germany . . . . (Office: Room 1.307) .

1 0

Re: [MPIWG Fortran] FP16 16-bit-floating-point
by Bader, Reinhold 16 Jun '17

16 Jun '17

Hello Rolf, to the best of my knowledge, the answers are as follows: Q1: Not in the sense of having an explicit name like the REAL32, REAL64, REAL128 names for specific integer kinds that were added to Fortran 2008. However, it is left free to implementors to create a new kind number in the implementation that could be returned by SELECTED_REAL_KIND() and, if IEEE conforming, by IEEE_SELECTED_REAL_KIND() and consequently all relevant IEEE support routines. Q2: REAL*2 is (like REAL*8) a non-conforming notation (REAL*8 is supported as an extension by many implementations), so the answer is no. A proper declaration might look like integer, parameter :: rk = selected_real_kind(2,6) ! at least 2 decimal digits and 6 maximal decimal exponent - hope I have this right ... real(KIND=rk) :: x with the following caveats: * the KIND that fits the requirement most closely will be used, so it might be necessary to use the STORAGE_SIZE intrinsic to check what one gets. * KIND numbers are non-portable, so might vary for one and the same kind between implementations (at least one compiler I know deviates from all others I know). I checked three of the compilers I know: they currently do not support FP16. Regards Reinhold > -----Ursprüngliche Nachricht----- > Von: Rolf Rabenseifner [mailto:[email protected]] > Gesendet: Freitag, 16. Juni 2017 13:53 > An: fortran standards email list for J3 <j3(a)j3-fortran.org>; Bader, Reinhold > <Reinhold.Bader(a)lrz.de> > Cc: MPI-WG Fortran working group <mpiwg-fortran(a)lists.mpi-forum.org>; > Atsushi Hori <ahori(a)riken.jp>; Martin Schulz <schulzm(a)llnl.gov>; MPI WG > Point To Point Communications working group <mpiwg-p2p(a)lists.mpi- > forum.org> > Betreff: FP16 16-bit-floating-point > > Hi all together, > > I'm sitting in the MPI Forum Meeting and we discuss the new IEEE FP16 > datatype. > > Questions to the Fortran atandardization body: > > - Does the Fortran Standardization Body has already a proposal for > the new name in Fortran for this IEEE FP16? > > - Will be REAL*2 identical to IEEE FP16? > > Best regards > Rolf > > -- > Dr. Rolf Rabenseifner . . . . . . . . . .. email rabenseifner(a)hlrs.de . > High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 . > University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 . > Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner . > Nobelstr. 19, D-70550 Stuttgart, Germany . . . . (Office: Room 1.307) .

1 0