August 2023 - mpiwg-abi - mailman3.cels.anl.gov

Re: [mpiwg-abi] MPI Fortran ABI discussion scheduling and agenda
by Christoph Niethammer 30 Aug '23

30 Aug '23

Hi Jeff Thanks for the writeup. Just a small note on the attendance in the group: Friends call me Chris but full name is Chrsitoph not Christian. :) Best Christoph Niethammer -- Dr.-Ing. Christoph Niethammer High Performance Computing Center Stuttgart (HLRS) Nobelstrasse 19 70569 Stuttgart Tel: ++49(0)711-685-87203 email: christoph.niethammer(a)hlrs.de https://www.hlrs.de/people/christoph-niethammer ----- Original Message ----- From: "Jeff Hammond" <jeff.science(a)gmail.com> To: "mpiwg-abi" <mpiwg-abi(a)lists.mpi-forum.org>, "MPI Forum" <mpiwg-fortran(a)lists.mpi-forum.org> Sent: Tuesday, 29 August, 2023 17:17:40 Subject: Re: [mpiwg-abi] MPI Fortran ABI discussion scheduling and agenda Terse minutes from the meeting follow. Please correct the record if I have made an error. Thanks to everyone who attended (I didn't transcript it, but at least Jeff H, Quincy K, Bill W, Bill L, Ken R, Julien J, Joseph S, Christian N, M Klemm, and Abraham N were there). Decision 1: The sizes of Fortran types is part of the MPI ABI and each choice/combination of these requires a separate implementation. The stipulation that the user must use exactly one of these throughout the application extends to the Fortran sizes. Rationale: Every other option is extremely difficult to implement and unpleasant to reason about. If we figure out a more compact way to solve this problem, we can standardize it in the future. The O(n) solution is practical because today no implementation implements n>2. Corollary 1: There is one mpi.h, one libmpi_abi.so and one of each for the Fortran modules (and header, if supported) for every MPI C+Fortran ABI. If an implementation provides more than one MPI C+Fortran ABI, the implementation should distinguish them using paths. Decision 2: The sizes prescribed by the external32 specification are required to be supported in the ABI. All other sets of sizes are optional. Corollary 2: User code must do one of the following: 1. Assume the external32 ABI. If wrong SO is linked, that's a user error and bad things will happen. 2. Query the Fortran size ABI to ensure that the MPI SO matches the Fortran object code, and abort if they don't match. 2. Query the Fortran size ABI to ensure that the MPI SO matches the Fortran object code, and handle any differences manually, e.g. using MPI_INTEGER8 instead of MPI_INTEGER. Corollary 3: MPI_Type_size must be callable before MPI_Init for all predefined datatypes (and only predefined datatypes), because this is how a user can detect potential ABI problems. If people don't like this, then I'll add MPI_Abi_type_sizes that meets the need, without supporting the generality of MPI_Type_size. Jeff On Fri, Aug 25, 2023 at 11:35 AM Jeff Hammond <jeff.science(a)gmail.com> wrote: > We need to meet to discuss some of the open problems with the MPI Fortran > ABI. > > https://www.when2meet.com/?21032099-f9qjp > > The agenda is > https://docs.google.com/document/d/1URUIbHhGQfUcoNg_KTX1lhWS-Z80LwJA9LH2QL7…. > You'll need to request access to comment/edit because this email list is > public and I do not want to deal with vandalism. > > Jeff > > -- > Jeff Hammond > jeff.science(a)gmail.com > http://jeffhammond.github.io/ > -- Jeff Hammond jeff.science(a)gmail.com http://jeffhammond.github.io/ -- mpiwg-abi mailing list mpiwg-abi(a)lists.mpi-forum.org https://lists.mpi-forum.org/mailman/listinfo/mpiwg-abi

1 0

Re: [mpiwg-abi] MPI Fortran ABI discussion scheduling and agenda
by Jeff Hammond 29 Aug '23

29 Aug '23

1 0

Re: [mpiwg-abi] MPI Fortran ABI discussion scheduling and agenda
by Holmes, Daniel John 29 Aug '23

29 Aug '23

Hi Jeff, Do the calls to MPI_TYPE_SIZE really need to happen before MPI_INIT? The use-case for “before” is that the user could refuse to call MPI_INIT if they get an answer they don’t like, but The MPI Way ™ for that is to initialise first, query the environment/situation, and then abort if desired. The “after” approach works better with sessions as well. #if USE_SESSIONS MPI_Info_set(my_info, …); // set a new key “mpi_assert_abi_sizes” to “ILP64” (request/assert) MPI_Session_init(my_info, &my_session); MPI_Info_get(my_info, …); // ask about the ABI of the library that was loaded (query) MPI_TYPE_SIZE(MPI_INTEGER, &size_of_fortran_integer); If (BAD) MPI_Abort(); #else // out-of-band, e.g. env var like mpi_mem_alloc_kinds (request/assert) MPI_Init(); MPI_Info_get(MPI_INFO_ENV, …); // ask about the ABI of the library that was loaded (query) MPI_TYPE_SIZE(MPI_INTEGER, &size_of_fortran_integer); If (BAD) MPI_Abort(); #endif Best wishes, Dan. From: mpiwg-abi <mpiwg-abi-bounces(a)lists.mpi-forum.org> On Behalf Of Jeff Hammond Sent: Tuesday, August 29, 2023 4:18 PM To: mpiwg-abi <mpiwg-abi(a)lists.mpi-forum.org>; MPI Forum <mpiwg-fortran(a)lists.mpi-forum.org> Subject: Re: [mpiwg-abi] MPI Fortran ABI discussion scheduling and agenda Terse minutes from the meeting follow. Please correct the record if I have made an error. Thanks to everyone who attended (I didn't transcript it, but at least Jeff H, Quincy K, Bill W, Bill L, Ken R, Julien J, Joseph S, Christian N, M Klemm, and Abraham N were there). Decision 1: The sizes of Fortran types is part of the MPI ABI and each choice/combination of these requires a separate implementation. The stipulation that the user must use exactly one of these throughout the application extends to the Fortran sizes. Rationale: Every other option is extremely difficult to implement and unpleasant to reason about. If we figure out a more compact way to solve this problem, we can standardize it in the future. The O(n) solution is practical because today no implementation implements n>2. Corollary 1: There is one mpi.h, one libmpi_abi.so and one of each for the Fortran modules (and header, if supported) for every MPI C+Fortran ABI. If an implementation provides more than one MPI C+Fortran ABI, the implementation should distinguish them using paths. Decision 2: The sizes prescribed by the external32 specification are required to be supported in the ABI. All other sets of sizes are optional. Corollary 2: User code must do one of the following: 1. Assume the external32 ABI. If wrong SO is linked, that's a user error and bad things will happen. 2. Query the Fortran size ABI to ensure that the MPI SO matches the Fortran object code, and abort if they don't match. 2. Query the Fortran size ABI to ensure that the MPI SO matches the Fortran object code, and handle any differences manually, e.g. using MPI_INTEGER8 instead of MPI_INTEGER. Corollary 3: MPI_Type_size must be callable before MPI_Init for all predefined datatypes (and only predefined datatypes), because this is how a user can detect potential ABI problems. If people don't like this, then I'll add MPI_Abi_type_sizes that meets the need, without supporting the generality of MPI_Type_size. Jeff On Fri, Aug 25, 2023 at 11:35 AM Jeff Hammond <jeff.science(a)gmail.com<mailto:[email protected]>> wrote: We need to meet to discuss some of the open problems with the MPI Fortran ABI. https://www.when2meet.com/?21032099-f9qjp The agenda is https://docs.google.com/document/d/1URUIbHhGQfUcoNg_KTX1lhWS-Z80LwJA9LH2QL7…. You'll need to request access to comment/edit because this email list is public and I do not want to deal with vandalism. Jeff -- Jeff Hammond jeff.science(a)gmail.com<mailto:[email protected]> http://jeffhammond.github.io/ -- Jeff Hammond jeff.science(a)gmail.com<mailto:[email protected]> http://jeffhammond.github.io/

1 0

Re: [mpiwg-abi] MPI Fortran ABI discussion scheduling and agenda
by Jeff Hammond 29 Aug '23

29 Aug '23

Terse minutes from the meeting follow. Please correct the record if I have made an error. Thanks to everyone who attended (I didn't transcript it, but at least Jeff H, Quincy K, Bill W, Bill L, Ken R, Julien J, Joseph S, Christian N, M Klemm, and Abraham N were there). Decision 1: The sizes of Fortran types is part of the MPI ABI and each choice/combination of these requires a separate implementation. The stipulation that the user must use exactly one of these throughout the application extends to the Fortran sizes. Rationale: Every other option is extremely difficult to implement and unpleasant to reason about. If we figure out a more compact way to solve this problem, we can standardize it in the future. The O(n) solution is practical because today no implementation implements n>2. Corollary 1: There is one mpi.h, one libmpi_abi.so and one of each for the Fortran modules (and header, if supported) for every MPI C+Fortran ABI. If an implementation provides more than one MPI C+Fortran ABI, the implementation should distinguish them using paths. Decision 2: The sizes prescribed by the external32 specification are required to be supported in the ABI. All other sets of sizes are optional. Corollary 2: User code must do one of the following: 1. Assume the external32 ABI. If wrong SO is linked, that's a user error and bad things will happen. 2. Query the Fortran size ABI to ensure that the MPI SO matches the Fortran object code, and abort if they don't match. 2. Query the Fortran size ABI to ensure that the MPI SO matches the Fortran object code, and handle any differences manually, e.g. using MPI_INTEGER8 instead of MPI_INTEGER. Corollary 3: MPI_Type_size must be callable before MPI_Init for all predefined datatypes (and only predefined datatypes), because this is how a user can detect potential ABI problems. If people don't like this, then I'll add MPI_Abi_type_sizes that meets the need, without supporting the generality of MPI_Type_size. Jeff On Fri, Aug 25, 2023 at 11:35 AM Jeff Hammond <jeff.science(a)gmail.com> wrote: > We need to meet to discuss some of the open problems with the MPI Fortran > ABI. > > https://www.when2meet.com/?21032099-f9qjp > > The agenda is > https://docs.google.com/document/d/1URUIbHhGQfUcoNg_KTX1lhWS-Z80LwJA9LH2QL7…. > You'll need to request access to comment/edit because this email list is > public and I do not want to deal with vandalism. > > Jeff > > -- > Jeff Hammond > jeff.science(a)gmail.com > http://jeffhammond.github.io/ > -- Jeff Hammond jeff.science(a)gmail.com http://jeffhammond.github.io/

1 0

MPI Fortran ABI discussion - Tuesday at 5 PM EET
by Jeff Hammond 28 Aug '23

28 Aug '23

Based on availability, we will meet tomorrow at 5 PM Helsinki time, which is 10 AM New York time and 7 AM California time. ________________________________________________________________________________ Microsoft Teams meeting Join on your computer, mobile app or room device Click here to join the meeting <https://teams.microsoft.com/l/meetup-join/19%3ameeting_N2FjZDVkOWItMDc4OS00…> Meeting ID: 286 245 066 993 Passcode: SUGbfQ Download Teams <https://www.microsoft.com/en-us/microsoft-teams/download-app> | Join on the web <https://www.microsoft.com/microsoft-teams/join-a-meeting> Join with a video conferencing device teams(a)vc.nvidia.com Video Conference ID: 116 411 016 9 Alternate VTC instructions <https://pexip.me/teams/vc.nvidia.com/1164110169> Or call in (audio only) +1 949-570-1120,,518282305# <+19495701120,,518282305#> United States, Irvine Phone Conference ID: 518 282 305# Find a local number <https://dialin.teams.microsoft.com/c58d41ec-314f-4761-9381-98dfc4066d1b?id=…> | Reset PIN <https://dialin.teams.microsoft.com/usp/pstnconferencing> Learn More <https://aka.ms/JoinTeamsMeeting> | Meeting options <https://teams.microsoft.com/meetingOptions/?organizerId=64532211-6004-4136-…> ________________________________________________________________________________

1 0

Re: [mpiwg-abi] [MPIWG Fortran] MPI Fortran ABI discussion scheduling and agenda
by Jeff Hammond 25 Aug '23

25 Aug '23

Yes, you are right, but MPI_DOUBLE_PRECISION is a unique predefined datatype, and we need to understand what size it is, regardless of how it relates to MPI_REAL, MPI_REAL4, MPI_REAL8 or MPI_REAL16. DOUBLE PRECISION and MPI_DOUBLE_PRECISION are the least of my concerns right now. Jeff On Fri, Aug 25, 2023 at 7:16 PM Long, Bill F <william.long(a)hpe.com> wrote: > Just a note that DOUBLR PRECISION is not a separate type in Fortran, but > just one of the REAL types: > > 1. > > "The keyword DOUBLE PRECISION is an alternative specifier for one kind > of real type." > > I think that in C float and double are different. > > CHeers, > Bill > > ------------------------------ > *From:* mpiwg-fortran <mpiwg-fortran-bounces(a)lists.mpi-forum.org> on > behalf of Jeff Hammond via mpiwg-fortran < > mpiwg-fortran(a)lists.mpi-forum.org> > *Sent:* Friday, August 25, 2023 3:35 AM > *To:* mpiwg-abi <mpiwg-abi(a)lists.mpi-forum.org>; MPI Forum < > mpiwg-fortran(a)lists.mpi-forum.org> > *Cc:* Jeff Hammond <jeff.science(a)gmail.com> > *Subject:* [MPIWG Fortran] MPI Fortran ABI discussion scheduling and > agenda > > We need to meet to discuss some of the open problems with the MPI Fortran > ABI. > > https://www.when2meet.com/?21032099-f9qjp > > The agenda is > https://docs.google.com/document/d/1URUIbHhGQfUcoNg_KTX1lhWS-Z80LwJA9LH2QL7…. > You'll need to request access to comment/edit because this email list is > public and I do not want to deal with vandalism. > > Jeff > > -- > Jeff Hammond > jeff.science(a)gmail.com > http://jeffhammond.github.io/ > -- Jeff Hammond jeff.science(a)gmail.com http://jeffhammond.github.io/

1 0

Re: [mpiwg-abi] [MPIWG Fortran] MPI Fortran ABI discussion scheduling and agenda
by Long, Bill F 25 Aug '23

25 Aug '23

Just a note that DOUBLR PRECISION is not a separate type in Fortran, but just one of the REAL types: 1. "The keyword DOUBLE PRECISION is an alternative specifier for one kind of real type." I think that in C float and double are different. CHeers, Bill ________________________________ From: mpiwg-fortran <mpiwg-fortran-bounces(a)lists.mpi-forum.org> on behalf of Jeff Hammond via mpiwg-fortran <mpiwg-fortran(a)lists.mpi-forum.org> Sent: Friday, August 25, 2023 3:35 AM To: mpiwg-abi <mpiwg-abi(a)lists.mpi-forum.org>; MPI Forum <mpiwg-fortran(a)lists.mpi-forum.org> Cc: Jeff Hammond <jeff.science(a)gmail.com> Subject: [MPIWG Fortran] MPI Fortran ABI discussion scheduling and agenda We need to meet to discuss some of the open problems with the MPI Fortran ABI. https://www.when2meet.com/?21032099-f9qjp<https://www.when2meet.com/?21032099-f9qjp> The agenda is https://docs.google.com/document/d/1URUIbHhGQfUcoNg_KTX1lhWS-Z80LwJA9LH2QL7…<https://docs.google.com/document/d/1URUIbHhGQfUcoNg_KTX1lhWS-Z80LwJA9LH2QL7…>. You'll need to request access to comment/edit because this email list is public and I do not want to deal with vandalism. Jeff -- Jeff Hammond jeff.science(a)gmail.com<mailto:[email protected]> http://jeffhammond.github.io/<http://jeffhammond.github.io/>

1 0

MPI Fortran ABI discussion scheduling and agenda
by Jeff Hammond 25 Aug '23

25 Aug '23

We need to meet to discuss some of the open problems with the MPI Fortran ABI. https://www.when2meet.com/?21032099-f9qjp The agenda is https://docs.google.com/document/d/1URUIbHhGQfUcoNg_KTX1lhWS-Z80LwJA9LH2QL7…. You'll need to request access to comment/edit because this email list is public and I do not want to deal with vandalism. Jeff -- Jeff Hammond jeff.science(a)gmail.com http://jeffhammond.github.io/

1 0

Re: [mpiwg-abi] MPI_Fint in the ABI
by Jeff Hammond 21 Aug '23

21 Aug '23

selected_int_kind(R) behaves differently than you are thinking. "The result has a value equal to the value of the kind type parameter of an integer type that represents all values n in the range −(10^R) < n < (10^R), or if no such kind type parameter is available on the processor, the result is −1. If more than one kind type parameter meets the criterion, the value returned is the one with the smallest decimal exponent range, unless there are several such values, in which case the smallest of these kind values is returned.” selected_int_kind(0) doesn’t return the default, but the smallest supported integer kind that can represent zero, and indeed with gfortran, it is an 8-bit integer. In any case, changing from INTEGER to something else for the handle arguments is an API change and that will only happen in new modules (and a new mpif.h if someone un-deprecates it). I intend to pursue this for MPI 5.0, only after I have prototyped to determine that there is any benefit. I am skeptical of the performance benefit of eliminating the conversion of non-predefined handles (pre-defined handles have a fast conversion path with the ABI). I don’t think the usability argument is going to be sufficient to drive adoption. We already have a hard time getting people to use MPI_F08 and it overwhelmingly superior to anything else. Jeff On 19. Aug 2023, at 14.03, Lisandro Dalcin <dalcinl(a)gmail.com> wrote: On Sat, 19 Aug 2023 at 13:43, Jeff Hammond <jehammond(a)nvidia.com<mailto:[email protected]>> wrote: Fortran doesn’t have an ABI across compilers but the MPI Fortran ABI is defined by the use of constants that match the C ABI, as is required. It is involves similarly required compatibility with Aint, Offset and Count. Oh, OK, so your definition of Fortran ABI is the one that, for a given Fortran compiler, matches the C ABI. Intel likely achieves their ILP64 support by ignoring MPI_Fint in the header and dlopen-ing the right symbol on launch. I’ll verify that later. Well, my point is, the ILP64 support is Fortran-only. There are no provisions for C and mpi.h to interoperate with the ILP64 library. A more expansive Fortran ABI definition that does what you propose has been considered. It represents a breaking API change and thus requires new module (and deprecated header) names. That's why I asked for a Fortran lawyer. My limited knowledge of Fortran tells me that "integer" and "integer(kind=selected_int_kind(0))" are the same type. I fail to see the extent of the breakage, or why new module names are needed. If the new module uses "integer(kind=MPI_HANDLE_KIND)" with MPI_HANDLE_KIND equal to selected_int_kind(0) for a standard invocation of the compiler, old code that uses plain "integer :: comm" declarations should continue to work, as long as users do not compile with -i8. For new code to work seamlessly with an ILP64 Fortran library, users have to update their Fortran code to use integer(kind=MPI_HANDLE_KIND), but in this case MPI_HANDLE_KIND is still a 32bits integer. -- Lisandro Dalcin ============ Senior Research Scientist Extreme Computing Research Center (ECRC) King Abdullah University of Science and Technology (KAUST) http://ecrc.kaust.edu.sa/ -- mpiwg-abi mailing list mpiwg-abi(a)lists.mpi-forum.org https://lists.mpi-forum.org/mailman/listinfo/mpiwg-abi

1 0

Re: [mpiwg-abi] MPI_Fint in the ABI
by Lisandro Dalcin 19 Aug '23

19 Aug '23

On Sat, 19 Aug 2023 at 13:43, Jeff Hammond <jehammond(a)nvidia.com> wrote: > Fortran doesn’t have an ABI across compilers but the MPI Fortran ABI is > defined by the use of constants that match the C ABI, as is required. It is > involves similarly required compatibility with Aint, Offset and Count. > > Oh, OK, so your definition of Fortran ABI is the one that, for a given Fortran compiler, matches the C ABI. > Intel likely achieves their ILP64 support by ignoring MPI_Fint in the > header and dlopen-ing the right symbol on launch. I’ll verify that later. > Well, my point is, the ILP64 support is Fortran-only. There are no provisions for C and mpi.h to interoperate with the ILP64 library. > A more expansive Fortran ABI definition that does what you propose has > been considered. It represents a breaking API change and thus requires new > module (and deprecated header) names. > That's why I asked for a Fortran lawyer. My limited knowledge of Fortran tells me that "integer" and "integer(kind=selected_int_kind(0))" are the same type. I fail to see the extent of the breakage, or why new module names are needed. If the new module uses "integer(kind=MPI_HANDLE_KIND)" with MPI_HANDLE_KIND equal to selected_int_kind(0) for a standard invocation of the compiler, old code that uses plain "integer :: comm" declarations should continue to work, as long as users do not compile with -i8. For new code to work seamlessly with an ILP64 Fortran library, users have to update their Fortran code to use integer(kind=MPI_HANDLE_KIND), but in this case MPI_HANDLE_KIND is still a 32bits integer. -- Lisandro Dalcin ============ Senior Research Scientist Extreme Computing Research Center (ECRC) King Abdullah University of Science and Technology (KAUST) http://ecrc.kaust.edu.sa/

1 0