- mpiwg-abi - mailman3.cels.anl.gov

Re: [mpiwg-abi] Tuesday (20 February 2023) meeting agenda
by Jeff Hammond 21 Feb '23

21 Feb '23

Thanks to everyone who joined our nearly 2 hour meeting today. There were 12-15 people in attendance (I can produce the details if compelled). We made the following decisions, which are of course not final in any sense, but will be reflected in the latest proposal once I have time to update it, and is the basis of the implementation work that Hui and I are doing. 0. MPI handles should be incomplete struct pointers. This guarantees type safety. Compilers will notice if users pass MPI_Comm instead of MPI_Group, for example. Some users and the Open-MPI developers have made this a high priority. A corollary to this is that the name of the struct becomes part of the ABI. For example, we will specify: typedef struct MPI_ABI_Comm * MPI_Comm; and *all* of this will become part of the ABI. Rust, for example, needs to know the struct name. 1. MPI predefined handle constants should be constant integer expressions, not addresses to a named location in the implementation. While the debate was heated, the poll was unanimous in favor of this choice. For example, this means that we have: #define MPI_INT (MPI_Datatype)0x3; This is legal in C and C++, and relies on implementation defined behavior that is already assumed in MPI (e.g. MPI_IN_PLACE) and known to work everywhere. The reasons for this decision are: a. Using link-time constant addresses as constant initializers is not completely portable. For example, it does not work on Windows, and Open-MPI does not support Windows because of this. b. We can do efficient encoding of predefined handles if they are integer values. More on this below... c. Compile-time constants will make it easier to standardize a Fortran ABI that uses the exact same handle values as C so there is zero overhead from Fortran (F08/F18). Fortran does not allow link-time resolution of initializer expressions, which implies overhead for implementations that do not have compile-time constants for the C handle constants (e.g. Open-MPI). d. It makes it easier to define/use predefined handle constants from binary, which was raised as a use case from a tool perspective (e.g. Dynist and Papi). The arguments against this decision were: e. We will need to state and reserve the integer values used for this. In practice, this is portable because of the zero page, but is otherwise implementable because implementations can just verify that allocated handles do not conflict with the predefined ones. 2. MPI_<HANDLE>_NULL should not be 0 (i.e. NULL). The arguments in favor of this were: f. It makes it clear when uninitialized handles are used, as this will always be erroneous. g. MPI_COMM_NULL has a name and is thus not the same thing as NULL would imply. The arguments in against this decision were: h. Zero initialization is common and some consider it convenient to have this cause handles to be automatically initialized to MPI_<HANDLE>_NULL. 3. We will attempt to Huffman encode predefined handles into a unique set of values, rather than have each set of handles be a sequence of integers. Arguments in favor: i. Unique numbers for every handle help debugging. j. We can encode useful information in handles, especially for datatypes, which allows optimizations similar to what MPICH does today. Arguments against: k. A sequence of integers means that a translation layer can use an array to translate from application handles to implementation handles 4. Non-decision: we discussed but made no decision about symbol visibility. We will address this later. In the next meeting, to be scheduled later, I will propose values for a bunch of constants and we can argue about them. In cases where MPICH and Open-MPI today agree, I will propose that we use those values. Most of this activity will happen on GitHub and the meeting will be to argue about any controversies. A copy of these minutes will be uploaded to GitHub shortly. Jeff > > > > On 2/20/23 03:55, Jeff Hammond wrote: > > > For the meeting tomorrow, I would like you all to consider the > > following: > > > > > > There is consensus regarding the following to get type safety: > > > typedef struct MPI_ABI_Handle * MPI_Handle; > > > > > > We have two choices for predefined handles: > > > > > > Option 1: > > > #define MPI_INT (MPI_Datatype)0x3 > > > > > > Option 2: > > > #define MPI_INT (MPI_Datatype)&MPI_ABI_INT > > > > > > Option 1 relies on implementation defined behavior regarding the > > > casting of integers to pointers, but it is safe. Gonzalo and I > > had a > > > long discussion of this, and the only issue is that > > (intptr_t)MPI_INT > > > == 0x3 is not guaranteed to be true, but (intptr_t)MPI_INT == > > > (intptr_t)(void*)0x3 is. Both MPICH and Open-MPI already rely > > on this > > > technique for MPI_IN_PLACE, among others. > > > > > > The advantage of Option 1 - assuming we use a sensible set of > > > integer values - is that ABI compatibility layers can do > > translation > > > via a table. This is most useful for datatypes, where there are > > ~100 > > > predefined handles, and to a lesser extent ops. For all other > > > handles, there are no more than 3 predefined handles (IIRC). > > > > > > Option 2 allows run-time resolution of predefined handles, which I > > > thought was good until I implemented it in > > > https://github.com/jeffhammond/mukautuva. While it works and > > means I > > > can avoid predefined handle translations in the forward > > direction, I > > > still have it in the backwards direction, and it's not exactly > > simple > > > to resolve all the symbols. I think Hui also implemented this > > in his > > > compatibility layer, although I don't know if he hates it as > > much as I do. > > > > > > The second question is whether to #define MPI_HANDLE_NULL > > > (MPI_Handle*)0. Lisandro argues for this, and it would make a > > lot of > > > things easy. > > > > > > I hope that we can decide these two questions today. For the > > second > > > meeting, we will address predefined integer constants. > > > > > > Jeff > > > > > > -- > > > Jeff Hammond > > > jeff.science(a)gmail.com > > > http://jeffhammond.github.io/ > > > > > > > -- > > mpiwg-abi mailing list > > mpiwg-abi(a)lists.mpi-forum.org > > https://lists.mpi-forum.org/mailman/listinfo/mpiwg-abi > > -- > > mpiwg-abi mailing list > > mpiwg-abi(a)lists.mpi-forum.org > > https://lists.mpi-forum.org/mailman/listinfo/mpiwg-abi > > > > > > > > -- > > Jeff Hammond > > jeff.science(a)gmail.com > > http://jeffhammond.github.io/ > > -- Jeff Hammond jeff.science(a)gmail.com http://jeffhammond.github.io/

1 0

Re: [mpiwg-abi] Tuesday (20 February 2023) meeting agenda
by Joseph Schuchart 20 Feb '23

20 Feb '23

Jeff, Hui, all, On 2/20/23 12:07, Jeff Hammond wrote: > > > On Mon, Feb 20, 2023 at 6:53 PM Zhou, Hui <zhouh(a)anl.gov> wrote: > > >> While it works and means I can avoid predefined handle > translations > in the forward direction, I still have it in the backwards direction, > and it's not exactly simple to resolve all the symbols. > > > > Can you elaborate a bit on the backward translation? I'm not sure I > understand what you mean by that. > > I think Jeff meant for output parameters such as in > MPI_Comm_create. I believe all handle constants are input only > except for NULL handles. > > > Yeah, NULL handle outputs Still not sure I understand how NULL would make a difference for output handles. > but also MPI_Type_get_contents, because "If these were predefined > datatypes, then the returned datatype is equal to that (constant) > predefined datatype and cannot be freed." I agree that this is difficult. How are applications dealing with it today? I wonder whether we could allow calling MPI_Type_free on predefined datatypes and make it a no-op to avoid going through the troubles of checking for it. > >> The second question is whether to #define MPI_HANDLE_NULL > (MPI_Handle*)0. Lisandro argues for this, and it would make a lot of > things easy. > > > I believe OMPI uses special null objects for nearly all its null > handles > to avoid checking for NULL on every call. > > From the user's point of view, using 0 as NULL is convenient. > Uninitialized static variables are zero-filled. It is also > convenient to zero-fill an array. > > > Indeed. It would be nice to have zero initialization be equivalent to > null handle initialization. Relying on default initialization to get a valid MPI handle is a tough sell for me. Proper initialization is not hard (typing 16 characters shouldn't be too much to ask) and in many cases the pointer will first be set by MPI (group/comm/file/win/session) anyway before it's legitimately used, so its initial state is irrelevant. In fact, accidentally passing `MPI_XXX_NULL` into MPI because the application did not create a proper xxx first potentially hides program errors that would otherwise be detected if NULL was passed (either through a Segfault, a tool, or the MPI library itself). The only somewhat legit argument for zero initialization I can see is MPI_REQUEST_NULL in an array of requests passed to test/wait functions. However, there are multiple ways around this (the count argument, explicit initialization). I don't think this warrants using NULL pointers, esp since MPI_REQUEST_NULL is a valid state (yielding an empty status). Backtracking a bit, there are two categories of null handles afaics: - Valid as input to MPI: MPI_INFO_NULL, MPI_REQUEST_NULL, null copy and delete functions, MPI_PROC_NULL (integer anyway, and never 0), MPI_FILE_NULL (to register default error file error handler); starting with 4.1 we will allow querying names for some null handles (MPI_WIN_NULL/MPI_DATATYPE_NULL/MPI_COMM_NULL) [1] - Invalid as input to MPI but returned from MPI: MPI_OP_NULL, MPI_MESSAGE_NULL, MPI_GROUP_NULL, MPI_ERRHANDLER_NULL, MPI_SESSION_NULL I hope I didn't miss any. For some in the second category, it is not actually specified that are not allowed as input: Can I pass MPI_DATATYPE_NULL to MPI_TYPE_FREE? Could be a no-op, could be an error. At least for the null handles that are valid inputs to MPI, there is a disconnect between the semantics of NULL and MPI_XXX_NULL: an MPI null handle is a valid handle that may legitimately be passed into MPI and used there in with well-defined semantics. NULL, on the other hand, is a pointer that represents an invalid state (like malloc returning NULL if OOM; yes you can pass NULL to realloc but that means that there was no valid state to begin with). In the interest of consistency I argue that all MPI null handles should have a valid representation: a) to catch application errors, and b) to avoid the intentional valid use of pointers to invalid objects in a public API. [1] https://github.com/mpi-forum/mpi-issues/issues/544 Cheers Joseph > > Jeff > > -- > Hui > ------------------------------------------------------------------------ > *From:* mpiwg-abi <mpiwg-abi-bounces(a)lists.mpi-forum.org> on > behalf of Joseph Schuchart <schuchart(a)icl.utk.edu> > *Sent:* Monday, February 20, 2023 8:53 AM > *To:* mpiwg-abi(a)lists.mpi-forum.org <mpiwg-abi(a)lists.mpi-forum.org> > *Subject:* Re: [mpiwg-abi] Tuesday (20 February 2023) meeting agenda > Jeff, > > You said: > > > While it works and means I can avoid predefined handle > translations > in the forward direction, I still have it in the backwards direction, > and it's not exactly simple to resolve all the symbols. > > Can you elaborate a bit on the backward translation? I'm not sure I > understand what you mean by that. > > > The second question is whether to #define MPI_HANDLE_NULL > (MPI_Handle*)0. Lisandro argues for this, and it would make a lot of > things easy. > > I believe OMPI uses special null objects for nearly all its null > handles > to avoid checking for NULL on every call. A null request handle for > example is a valid input into `MPI_WAIT` and we can get the empty > status > from the null request just like we do from any other request. NULL > function pointers (MPI_TYPE_NULL_DELETE_FN) can point to an empty > function that we can simply invoke. NULL pointers require special > treatment, which is tedious and error prone and has UB lurking around > the corner. With null objects, we will never receive (or hand out) an > invalid pointer, which is safe engineering. > > I'm curious what the benefits are from an application's (or > middleware) > standpoint? > > Cheers > Joseph > > On 2/20/23 03:55, Jeff Hammond wrote: > > For the meeting tomorrow, I would like you all to consider the > following: > > > > There is consensus regarding the following to get type safety: > > typedef struct MPI_ABI_Handle * MPI_Handle; > > > > We have two choices for predefined handles: > > > > Option 1: > > #define MPI_INT (MPI_Datatype)0x3 > > > > Option 2: > > #define MPI_INT (MPI_Datatype)&MPI_ABI_INT > > > > Option 1 relies on implementation defined behavior regarding the > > casting of integers to pointers, but it is safe. Gonzalo and I > had a > > long discussion of this, and the only issue is that > (intptr_t)MPI_INT > > == 0x3 is not guaranteed to be true, but (intptr_t)MPI_INT == > > (intptr_t)(void*)0x3 is. Both MPICH and Open-MPI already rely > on this > > technique for MPI_IN_PLACE, among others. > > > > The advantage of Option 1 - assuming we use a sensible set of > > integer values - is that ABI compatibility layers can do > translation > > via a table. This is most useful for datatypes, where there are > ~100 > > predefined handles, and to a lesser extent ops. For all other > > handles, there are no more than 3 predefined handles (IIRC). > > > > Option 2 allows run-time resolution of predefined handles, which I > > thought was good until I implemented it in > > https://github.com/jeffhammond/mukautuva. While it works and > means I > > can avoid predefined handle translations in the forward > direction, I > > still have it in the backwards direction, and it's not exactly > simple > > to resolve all the symbols. I think Hui also implemented this > in his > > compatibility layer, although I don't know if he hates it as > much as I do. > > > > The second question is whether to #define MPI_HANDLE_NULL > > (MPI_Handle*)0. Lisandro argues for this, and it would make a > lot of > > things easy. > > > > I hope that we can decide these two questions today. For the > second > > meeting, we will address predefined integer constants. > > > > Jeff > > > > -- > > Jeff Hammond > > jeff.science(a)gmail.com > > http://jeffhammond.github.io/ > > > > -- > mpiwg-abi mailing list > mpiwg-abi(a)lists.mpi-forum.org > https://lists.mpi-forum.org/mailman/listinfo/mpiwg-abi > -- > mpiwg-abi mailing list > mpiwg-abi(a)lists.mpi-forum.org > https://lists.mpi-forum.org/mailman/listinfo/mpiwg-abi > > > > -- > Jeff Hammond > jeff.science(a)gmail.com > http://jeffhammond.github.io/

1 0

Re: [mpiwg-abi] Tuesday (20 February 2023) meeting agenda
by Jeff Hammond 20 Feb '23

20 Feb '23

>>>> On Mon, Feb 20, 2023 at 6:53 PM Zhou, Hui <zhouh(a)anl.gov> wrote: >>>> >> While it works and means I can avoid predefined handle translations >>>> in the forward direction, I still have it in the backwards direction, >>>> and it's not exactly simple to resolve all the symbols. >>>> > >>>> > Can you elaborate a bit on the backward translation? I'm not sure I >>>> understand what you mean by that. >>>> >>>> I think Jeff meant for output parameters such as in MPI_Comm_create. I believe all handle constants are input only except for NULL handles. >> >> MPI_GROUP_EMPTY is the valid output for all group manipulation functions (they do not return MPI_GROUP_NULL). >> >> >> Yeah, NULL handle outputs, but also MPI_Type_get_contents, because "If these were predefined datatypes, then the returned datatype is equal to that (constant) predefined datatype and cannot be freed." >> >>> >> The second question is whether to #define MPI_HANDLE_NULL >>> (MPI_Handle*)0. Lisandro argues for this, and it would make a lot of >>> things easy. >>> >>> > I believe OMPI uses special null objects for nearly all its null handles >>> to avoid checking for NULL on every call. >>> >>> From the user's point of view, using 0 as NULL is convenient. Uninitialized static variables are zero-filled. It is also convenient to zero-fill an array. > > Convenient but error prone. If users want a NULL object they need to set it clearly, not rely on assumptions about zero-filled memory regions. OMPI has been doing this for years and no users complained, because it is good practice. > Can you give an example of how it’s error prone? What errors is OMPI catching because it’s not null? Jeff > George. > > >> >> Indeed. It would be nice to have zero initialization be equivalent to null handle initialization. >> >> Jeff >> >> >>> -- >>> Hui >>> From: mpiwg-abi <mpiwg-abi-bounces(a)lists.mpi-forum.org> on behalf of Joseph Schuchart <schuchart(a)icl.utk.edu> >>> Sent: Monday, February 20, 2023 8:53 AM >>> To: mpiwg-abi(a)lists.mpi-forum.org <mpiwg-abi(a)lists.mpi-forum.org> >>> Subject: Re: [mpiwg-abi] Tuesday (20 February 2023) meeting agenda >>> >>> Jeff, >>> >>> You said: >>> >>> > While it works and means I can avoid predefined handle translations >>> in the forward direction, I still have it in the backwards direction, >>> and it's not exactly simple to resolve all the symbols. >>> >>> Can you elaborate a bit on the backward translation? I'm not sure I >>> understand what you mean by that. >>> >>> > The second question is whether to #define MPI_HANDLE_NULL >>> (MPI_Handle*)0. Lisandro argues for this, and it would make a lot of >>> things easy. >>> >>> I believe OMPI uses special null objects for nearly all its null handles >>> to avoid checking for NULL on every call. A null request handle for >>> example is a valid input into `MPI_WAIT` and we can get the empty status >>> from the null request just like we do from any other request. NULL >>> function pointers (MPI_TYPE_NULL_DELETE_FN) can point to an empty >>> function that we can simply invoke. NULL pointers require special >>> treatment, which is tedious and error prone and has UB lurking around >>> the corner. With null objects, we will never receive (or hand out) an >>> invalid pointer, which is safe engineering. >>> >>> I'm curious what the benefits are from an application's (or middleware) >>> standpoint? >>> >>> Cheers >>> Joseph >>> >>> On 2/20/23 03:55, Jeff Hammond wrote: >>> > For the meeting tomorrow, I would like you all to consider the following: >>> > >>> > There is consensus regarding the following to get type safety: >>> > typedef struct MPI_ABI_Handle * MPI_Handle; >>> > >>> > We have two choices for predefined handles: >>> > >>> > Option 1: >>> > #define MPI_INT (MPI_Datatype)0x3 >>> > >>> > Option 2: >>> > #define MPI_INT (MPI_Datatype)&MPI_ABI_INT >>> > >>> > Option 1 relies on implementation defined behavior regarding the >>> > casting of integers to pointers, but it is safe. Gonzalo and I had a >>> > long discussion of this, and the only issue is that (intptr_t)MPI_INT >>> > == 0x3 is not guaranteed to be true, but (intptr_t)MPI_INT == >>> > (intptr_t)(void*)0x3 is. Both MPICH and Open-MPI already rely on this >>> > technique for MPI_IN_PLACE, among others. >>> > >>> > The advantage of Option 1 - assuming we use a sensible set of >>> > integer values - is that ABI compatibility layers can do translation >>> > via a table. This is most useful for datatypes, where there are ~100 >>> > predefined handles, and to a lesser extent ops. For all other >>> > handles, there are no more than 3 predefined handles (IIRC). >>> > >>> > Option 2 allows run-time resolution of predefined handles, which I >>> > thought was good until I implemented it in >>> > https://github.com/jeffhammond/mukautuva. While it works and means I >>> > can avoid predefined handle translations in the forward direction, I >>> > still have it in the backwards direction, and it's not exactly simple >>> > to resolve all the symbols. I think Hui also implemented this in his >>> > compatibility layer, although I don't know if he hates it as much as I do. >>> > >>> > The second question is whether to #define MPI_HANDLE_NULL >>> > (MPI_Handle*)0. Lisandro argues for this, and it would make a lot of >>> > things easy. >>> > >>> > I hope that we can decide these two questions today. For the second >>> > meeting, we will address predefined integer constants. >>> > >>> > Jeff >>> > >>> > -- >>> > Jeff Hammond >>> > jeff.science(a)gmail.com >>> > http://jeffhammond.github.io/ >>> > >>> >>> -- >>> mpiwg-abi mailing list >>> mpiwg-abi(a)lists.mpi-forum.org >>> https://lists.mpi-forum.org/mailman/listinfo/mpiwg-abi >>> -- >>> mpiwg-abi mailing list >>> mpiwg-abi(a)lists.mpi-forum.org >>> https://lists.mpi-forum.org/mailman/listinfo/mpiwg-abi >> >> >> -- >> Jeff Hammond >> jeff.science(a)gmail.com >> http://jeffhammond.github.io/ >> -- >> mpiwg-abi mailing list >> mpiwg-abi(a)lists.mpi-forum.org >> https://lists.mpi-forum.org/mailman/listinfo/mpiwg-abi

1 0

Re: [mpiwg-abi] Tuesday (20 February 2023) meeting agenda
by Zhou, Hui 20 Feb '23

20 Feb '23

> OMPI has been doing this for years and no users complained, because it is good practice. Well, Lisandro just wished 🙂. MPICH also doesn't use NULL for NULL handles, so I am not standing one-side or another. But I question this kind of reasoning. I am pretty sure that if MPI standardized to use NULL for NULL handles, no one would've complained either and maybe less wishes from users. 🙂 -- Hui ________________________________ From: George Bosilca <bosilca(a)icl.utk.edu> Sent: Monday, February 20, 2023 12:13 PM To: Jeff Hammond <jeff.science(a)gmail.com> Cc: Zhou, Hui <zhouh(a)anl.gov>; mpiwg-abi(a)lists.mpi-forum.org <mpiwg-abi(a)lists.mpi-forum.org> Subject: Re: [mpiwg-abi] Tuesday (20 February 2023) meeting agenda On Mon, Feb 20, 2023 at 12:08 PM Jeff Hammond <jeff.science(a)gmail.com<mailto:[email protected]>> wrote: On Mon, Feb 20, 2023 at 6:53 PM Zhou, Hui <zhouh(a)anl.gov<mailto:[email protected]>> wrote: >> While it works and means I can avoid predefined handle translations in the forward direction, I still have it in the backwards direction, and it's not exactly simple to resolve all the symbols. > > Can you elaborate a bit on the backward translation? I'm not sure I understand what you mean by that. I think Jeff meant for output parameters such as in MPI_Comm_create. I believe all handle constants are input only except for NULL handles. MPI_GROUP_EMPTY is the valid output for all group manipulation functions (they do not return MPI_GROUP_NULL). Yeah, NULL handle outputs, but also MPI_Type_get_contents, because "If these were predefined datatypes, then the returned datatype is equal to that (constant) predefined datatype and cannot be freed." >> The second question is whether to #define MPI_HANDLE_NULL (MPI_Handle*)0. Lisandro argues for this, and it would make a lot of things easy. > I believe OMPI uses special null objects for nearly all its null handles to avoid checking for NULL on every call. From the user's point of view, using 0 as NULL is convenient. Uninitialized static variables are zero-filled. It is also convenient to zero-fill an array. Convenient but error prone. If users want a NULL object they need to set it clearly, not rely on assumptions about zero-filled memory regions. OMPI has been doing this for years and no users complained, because it is good practice. George. Indeed. It would be nice to have zero initialization be equivalent to null handle initialization. Jeff -- Hui ________________________________ From: mpiwg-abi <mpiwg-abi-bounces(a)lists.mpi-forum.org<mailto:[email protected]>> on behalf of Joseph Schuchart <schuchart(a)icl.utk.edu<mailto:[email protected]>> Sent: Monday, February 20, 2023 8:53 AM To: mpiwg-abi(a)lists.mpi-forum.org<mailto:[email protected]> <mpiwg-abi(a)lists.mpi-forum.org<mailto:[email protected]>> Subject: Re: [mpiwg-abi] Tuesday (20 February 2023) meeting agenda Jeff, You said: > While it works and means I can avoid predefined handle translations in the forward direction, I still have it in the backwards direction, and it's not exactly simple to resolve all the symbols. Can you elaborate a bit on the backward translation? I'm not sure I understand what you mean by that. > The second question is whether to #define MPI_HANDLE_NULL (MPI_Handle*)0. Lisandro argues for this, and it would make a lot of things easy. I believe OMPI uses special null objects for nearly all its null handles to avoid checking for NULL on every call. A null request handle for example is a valid input into `MPI_WAIT` and we can get the empty status from the null request just like we do from any other request. NULL function pointers (MPI_TYPE_NULL_DELETE_FN) can point to an empty function that we can simply invoke. NULL pointers require special treatment, which is tedious and error prone and has UB lurking around the corner. With null objects, we will never receive (or hand out) an invalid pointer, which is safe engineering. I'm curious what the benefits are from an application's (or middleware) standpoint? Cheers Joseph On 2/20/23 03:55, Jeff Hammond wrote: > For the meeting tomorrow, I would like you all to consider the following: > > There is consensus regarding the following to get type safety: > typedef struct MPI_ABI_Handle * MPI_Handle; > > We have two choices for predefined handles: > > Option 1: > #define MPI_INT (MPI_Datatype)0x3 > > Option 2: > #define MPI_INT (MPI_Datatype)&MPI_ABI_INT > > Option 1 relies on implementation defined behavior regarding the > casting of integers to pointers, but it is safe. Gonzalo and I had a > long discussion of this, and the only issue is that (intptr_t)MPI_INT > == 0x3 is not guaranteed to be true, but (intptr_t)MPI_INT == > (intptr_t)(void*)0x3 is. Both MPICH and Open-MPI already rely on this > technique for MPI_IN_PLACE, among others. > > The advantage of Option 1 - assuming we use a sensible set of > integer values - is that ABI compatibility layers can do translation > via a table. This is most useful for datatypes, where there are ~100 > predefined handles, and to a lesser extent ops. For all other > handles, there are no more than 3 predefined handles (IIRC). > > Option 2 allows run-time resolution of predefined handles, which I > thought was good until I implemented it in > https://github.com/jeffhammond/mukautuva. While it works and means I > can avoid predefined handle translations in the forward direction, I > still have it in the backwards direction, and it's not exactly simple > to resolve all the symbols. I think Hui also implemented this in his > compatibility layer, although I don't know if he hates it as much as I do. > > The second question is whether to #define MPI_HANDLE_NULL > (MPI_Handle*)0. Lisandro argues for this, and it would make a lot of > things easy. > > I hope that we can decide these two questions today. For the second > meeting, we will address predefined integer constants. > > Jeff > > -- > Jeff Hammond > jeff.science(a)gmail.com<mailto:[email protected]> > http://jeffhammond.github.io/ > -- mpiwg-abi mailing list mpiwg-abi(a)lists.mpi-forum.org<mailto:[email protected]> https://lists.mpi-forum.org/mailman/listinfo/mpiwg-abi -- mpiwg-abi mailing list mpiwg-abi(a)lists.mpi-forum.org<mailto:[email protected]> https://lists.mpi-forum.org/mailman/listinfo/mpiwg-abi -- Jeff Hammond jeff.science(a)gmail.com<mailto:[email protected]> http://jeffhammond.github.io/ -- mpiwg-abi mailing list mpiwg-abi(a)lists.mpi-forum.org<mailto:[email protected]> https://lists.mpi-forum.org/mailman/listinfo/mpiwg-abi

1 0

Re: [mpiwg-abi] Tuesday (20 February 2023) meeting agenda
by George Bosilca 20 Feb '23

20 Feb '23

On Mon, Feb 20, 2023 at 12:08 PM Jeff Hammond <jeff.science(a)gmail.com> wrote: > > > On Mon, Feb 20, 2023 at 6:53 PM Zhou, Hui <zhouh(a)anl.gov> wrote: > >> >> While it works and means I can avoid predefined handle translations >> in the forward direction, I still have it in the backwards direction, >> and it's not exactly simple to resolve all the symbols. >> > >> > Can you elaborate a bit on the backward translation? I'm not sure I >> understand what you mean by that. >> >> I think Jeff meant for output parameters such as in MPI_Comm_create. I >> believe all handle constants are input only except for NULL handles. >> > MPI_GROUP_EMPTY is the valid output for all group manipulation functions (they do not return MPI_GROUP_NULL). > > Yeah, NULL handle outputs, but also MPI_Type_get_contents, because "If > these were predefined datatypes, then the returned datatype is equal to > that (constant) predefined datatype and cannot be freed." > > >> >> The second question is whether to #define MPI_HANDLE_NULL >> (MPI_Handle*)0. Lisandro argues for this, and it would make a lot of >> things easy. >> >> > I believe OMPI uses special null objects for nearly all its null >> handles >> to avoid checking for NULL on every call. >> >> From the user's point of view, using 0 as NULL is convenient. >> Uninitialized static variables are zero-filled. It is also convenient to >> zero-fill an array. >> > Convenient but error prone. If users want a NULL object they need to set it clearly, not rely on assumptions about zero-filled memory regions. OMPI has been doing this for years and no users complained, because it is good practice. George. > > Indeed. It would be nice to have zero initialization be equivalent to > null handle initialization. > > Jeff > > > >> -- >> Hui >> ------------------------------ >> *From:* mpiwg-abi <mpiwg-abi-bounces(a)lists.mpi-forum.org> on behalf of >> Joseph Schuchart <schuchart(a)icl.utk.edu> >> *Sent:* Monday, February 20, 2023 8:53 AM >> *To:* mpiwg-abi(a)lists.mpi-forum.org <mpiwg-abi(a)lists.mpi-forum.org> >> *Subject:* Re: [mpiwg-abi] Tuesday (20 February 2023) meeting agenda >> >> Jeff, >> >> You said: >> >> > While it works and means I can avoid predefined handle translations >> in the forward direction, I still have it in the backwards direction, >> and it's not exactly simple to resolve all the symbols. >> >> Can you elaborate a bit on the backward translation? I'm not sure I >> understand what you mean by that. >> >> > The second question is whether to #define MPI_HANDLE_NULL >> (MPI_Handle*)0. Lisandro argues for this, and it would make a lot of >> things easy. >> >> I believe OMPI uses special null objects for nearly all its null handles >> to avoid checking for NULL on every call. A null request handle for >> example is a valid input into `MPI_WAIT` and we can get the empty status >> from the null request just like we do from any other request. NULL >> function pointers (MPI_TYPE_NULL_DELETE_FN) can point to an empty >> function that we can simply invoke. NULL pointers require special >> treatment, which is tedious and error prone and has UB lurking around >> the corner. With null objects, we will never receive (or hand out) an >> invalid pointer, which is safe engineering. >> >> I'm curious what the benefits are from an application's (or middleware) >> standpoint? >> >> Cheers >> Joseph >> >> On 2/20/23 03:55, Jeff Hammond wrote: >> > For the meeting tomorrow, I would like you all to consider the >> following: >> > >> > There is consensus regarding the following to get type safety: >> > typedef struct MPI_ABI_Handle * MPI_Handle; >> > >> > We have two choices for predefined handles: >> > >> > Option 1: >> > #define MPI_INT (MPI_Datatype)0x3 >> > >> > Option 2: >> > #define MPI_INT (MPI_Datatype)&MPI_ABI_INT >> > >> > Option 1 relies on implementation defined behavior regarding the >> > casting of integers to pointers, but it is safe. Gonzalo and I had a >> > long discussion of this, and the only issue is that (intptr_t)MPI_INT >> > == 0x3 is not guaranteed to be true, but (intptr_t)MPI_INT == >> > (intptr_t)(void*)0x3 is. Both MPICH and Open-MPI already rely on this >> > technique for MPI_IN_PLACE, among others. >> > >> > The advantage of Option 1 - assuming we use a sensible set of >> > integer values - is that ABI compatibility layers can do translation >> > via a table. This is most useful for datatypes, where there are ~100 >> > predefined handles, and to a lesser extent ops. For all other >> > handles, there are no more than 3 predefined handles (IIRC). >> > >> > Option 2 allows run-time resolution of predefined handles, which I >> > thought was good until I implemented it in >> > https://github.com/jeffhammond/mukautuva. While it works and means I >> > can avoid predefined handle translations in the forward direction, I >> > still have it in the backwards direction, and it's not exactly simple >> > to resolve all the symbols. I think Hui also implemented this in his >> > compatibility layer, although I don't know if he hates it as much as I >> do. >> > >> > The second question is whether to #define MPI_HANDLE_NULL >> > (MPI_Handle*)0. Lisandro argues for this, and it would make a lot of >> > things easy. >> > >> > I hope that we can decide these two questions today. For the second >> > meeting, we will address predefined integer constants. >> > >> > Jeff >> > >> > -- >> > Jeff Hammond >> > jeff.science(a)gmail.com >> > http://jeffhammond.github.io/ >> > >> >> -- >> mpiwg-abi mailing list >> mpiwg-abi(a)lists.mpi-forum.org >> https://lists.mpi-forum.org/mailman/listinfo/mpiwg-abi >> -- >> mpiwg-abi mailing list >> mpiwg-abi(a)lists.mpi-forum.org >> https://lists.mpi-forum.org/mailman/listinfo/mpiwg-abi >> > > > -- > Jeff Hammond > jeff.science(a)gmail.com > http://jeffhammond.github.io/ > -- > mpiwg-abi mailing list > mpiwg-abi(a)lists.mpi-forum.org > https://lists.mpi-forum.org/mailman/listinfo/mpiwg-abi >

1 0

Re: [mpiwg-abi] Tuesday (20 February 2023) meeting agenda
by Jeff Hammond 20 Feb '23

20 Feb '23

On Mon, Feb 20, 2023 at 6:53 PM Zhou, Hui <zhouh(a)anl.gov> wrote: > >> While it works and means I can avoid predefined handle translations > in the forward direction, I still have it in the backwards direction, > and it's not exactly simple to resolve all the symbols. > > > > Can you elaborate a bit on the backward translation? I'm not sure I > understand what you mean by that. > > I think Jeff meant for output parameters such as in MPI_Comm_create. I > believe all handle constants are input only except for NULL handles. > Yeah, NULL handle outputs, but also MPI_Type_get_contents, because "If these were predefined datatypes, then the returned datatype is equal to that (constant) predefined datatype and cannot be freed." > >> The second question is whether to #define MPI_HANDLE_NULL > (MPI_Handle*)0. Lisandro argues for this, and it would make a lot of > things easy. > > > I believe OMPI uses special null objects for nearly all its null > handles > to avoid checking for NULL on every call. > > From the user's point of view, using 0 as NULL is convenient. > Uninitialized static variables are zero-filled. It is also convenient to > zero-fill an array. > Indeed. It would be nice to have zero initialization be equivalent to null handle initialization. Jeff > -- > Hui > ------------------------------ > *From:* mpiwg-abi <mpiwg-abi-bounces(a)lists.mpi-forum.org> on behalf of > Joseph Schuchart <schuchart(a)icl.utk.edu> > *Sent:* Monday, February 20, 2023 8:53 AM > *To:* mpiwg-abi(a)lists.mpi-forum.org <mpiwg-abi(a)lists.mpi-forum.org> > *Subject:* Re: [mpiwg-abi] Tuesday (20 February 2023) meeting agenda > > Jeff, > > You said: > > > While it works and means I can avoid predefined handle translations > in the forward direction, I still have it in the backwards direction, > and it's not exactly simple to resolve all the symbols. > > Can you elaborate a bit on the backward translation? I'm not sure I > understand what you mean by that. > > > The second question is whether to #define MPI_HANDLE_NULL > (MPI_Handle*)0. Lisandro argues for this, and it would make a lot of > things easy. > > I believe OMPI uses special null objects for nearly all its null handles > to avoid checking for NULL on every call. A null request handle for > example is a valid input into `MPI_WAIT` and we can get the empty status > from the null request just like we do from any other request. NULL > function pointers (MPI_TYPE_NULL_DELETE_FN) can point to an empty > function that we can simply invoke. NULL pointers require special > treatment, which is tedious and error prone and has UB lurking around > the corner. With null objects, we will never receive (or hand out) an > invalid pointer, which is safe engineering. > > I'm curious what the benefits are from an application's (or middleware) > standpoint? > > Cheers > Joseph > > On 2/20/23 03:55, Jeff Hammond wrote: > > For the meeting tomorrow, I would like you all to consider the following: > > > > There is consensus regarding the following to get type safety: > > typedef struct MPI_ABI_Handle * MPI_Handle; > > > > We have two choices for predefined handles: > > > > Option 1: > > #define MPI_INT (MPI_Datatype)0x3 > > > > Option 2: > > #define MPI_INT (MPI_Datatype)&MPI_ABI_INT > > > > Option 1 relies on implementation defined behavior regarding the > > casting of integers to pointers, but it is safe. Gonzalo and I had a > > long discussion of this, and the only issue is that (intptr_t)MPI_INT > > == 0x3 is not guaranteed to be true, but (intptr_t)MPI_INT == > > (intptr_t)(void*)0x3 is. Both MPICH and Open-MPI already rely on this > > technique for MPI_IN_PLACE, among others. > > > > The advantage of Option 1 - assuming we use a sensible set of > > integer values - is that ABI compatibility layers can do translation > > via a table. This is most useful for datatypes, where there are ~100 > > predefined handles, and to a lesser extent ops. For all other > > handles, there are no more than 3 predefined handles (IIRC). > > > > Option 2 allows run-time resolution of predefined handles, which I > > thought was good until I implemented it in > > https://github.com/jeffhammond/mukautuva. While it works and means I > > can avoid predefined handle translations in the forward direction, I > > still have it in the backwards direction, and it's not exactly simple > > to resolve all the symbols. I think Hui also implemented this in his > > compatibility layer, although I don't know if he hates it as much as I > do. > > > > The second question is whether to #define MPI_HANDLE_NULL > > (MPI_Handle*)0. Lisandro argues for this, and it would make a lot of > > things easy. > > > > I hope that we can decide these two questions today. For the second > > meeting, we will address predefined integer constants. > > > > Jeff > > > > -- > > Jeff Hammond > > jeff.science(a)gmail.com > > http://jeffhammond.github.io/ > > > > -- > mpiwg-abi mailing list > mpiwg-abi(a)lists.mpi-forum.org > https://lists.mpi-forum.org/mailman/listinfo/mpiwg-abi > -- > mpiwg-abi mailing list > mpiwg-abi(a)lists.mpi-forum.org > https://lists.mpi-forum.org/mailman/listinfo/mpiwg-abi > -- Jeff Hammond jeff.science(a)gmail.com http://jeffhammond.github.io/

1 0

Re: [mpiwg-abi] Tuesday (20 February 2023) meeting agenda
by Zhou, Hui 20 Feb '23

20 Feb '23

>> While it works and means I can avoid predefined handle translations in the forward direction, I still have it in the backwards direction, and it's not exactly simple to resolve all the symbols. > > Can you elaborate a bit on the backward translation? I'm not sure I understand what you mean by that. I think Jeff meant for output parameters such as in MPI_Comm_create. I believe all handle constants are input only except for NULL handles. >> The second question is whether to #define MPI_HANDLE_NULL (MPI_Handle*)0. Lisandro argues for this, and it would make a lot of things easy. > I believe OMPI uses special null objects for nearly all its null handles to avoid checking for NULL on every call. >From the user's point of view, using 0 as NULL is convenient. Uninitialized static variables are zero-filled. It is also convenient to zero-fill an array. -- Hui ________________________________ From: mpiwg-abi <mpiwg-abi-bounces(a)lists.mpi-forum.org> on behalf of Joseph Schuchart <schuchart(a)icl.utk.edu> Sent: Monday, February 20, 2023 8:53 AM To: mpiwg-abi(a)lists.mpi-forum.org <mpiwg-abi(a)lists.mpi-forum.org> Subject: Re: [mpiwg-abi] Tuesday (20 February 2023) meeting agenda Jeff, You said: > While it works and means I can avoid predefined handle translations in the forward direction, I still have it in the backwards direction, and it's not exactly simple to resolve all the symbols. Can you elaborate a bit on the backward translation? I'm not sure I understand what you mean by that. > The second question is whether to #define MPI_HANDLE_NULL (MPI_Handle*)0. Lisandro argues for this, and it would make a lot of things easy. I believe OMPI uses special null objects for nearly all its null handles to avoid checking for NULL on every call. A null request handle for example is a valid input into `MPI_WAIT` and we can get the empty status from the null request just like we do from any other request. NULL function pointers (MPI_TYPE_NULL_DELETE_FN) can point to an empty function that we can simply invoke. NULL pointers require special treatment, which is tedious and error prone and has UB lurking around the corner. With null objects, we will never receive (or hand out) an invalid pointer, which is safe engineering. I'm curious what the benefits are from an application's (or middleware) standpoint? Cheers Joseph On 2/20/23 03:55, Jeff Hammond wrote: > For the meeting tomorrow, I would like you all to consider the following: > > There is consensus regarding the following to get type safety: > typedef struct MPI_ABI_Handle * MPI_Handle; > > We have two choices for predefined handles: > > Option 1: > #define MPI_INT (MPI_Datatype)0x3 > > Option 2: > #define MPI_INT (MPI_Datatype)&MPI_ABI_INT > > Option 1 relies on implementation defined behavior regarding the > casting of integers to pointers, but it is safe. Gonzalo and I had a > long discussion of this, and the only issue is that (intptr_t)MPI_INT > == 0x3 is not guaranteed to be true, but (intptr_t)MPI_INT == > (intptr_t)(void*)0x3 is. Both MPICH and Open-MPI already rely on this > technique for MPI_IN_PLACE, among others. > > The advantage of Option 1 - assuming we use a sensible set of > integer values - is that ABI compatibility layers can do translation > via a table. This is most useful for datatypes, where there are ~100 > predefined handles, and to a lesser extent ops. For all other > handles, there are no more than 3 predefined handles (IIRC). > > Option 2 allows run-time resolution of predefined handles, which I > thought was good until I implemented it in > https://github.com/jeffhammond/mukautuva. While it works and means I > can avoid predefined handle translations in the forward direction, I > still have it in the backwards direction, and it's not exactly simple > to resolve all the symbols. I think Hui also implemented this in his > compatibility layer, although I don't know if he hates it as much as I do. > > The second question is whether to #define MPI_HANDLE_NULL > (MPI_Handle*)0. Lisandro argues for this, and it would make a lot of > things easy. > > I hope that we can decide these two questions today. For the second > meeting, we will address predefined integer constants. > > Jeff > > -- > Jeff Hammond > jeff.science(a)gmail.com > http://jeffhammond.github.io/ > -- mpiwg-abi mailing list mpiwg-abi(a)lists.mpi-forum.org https://lists.mpi-forum.org/mailman/listinfo/mpiwg-abi

1 0

Re: [mpiwg-abi] Tuesday (20 February 2023) meeting agenda
by Joseph Schuchart 20 Feb '23

20 Feb '23

Jeff, You said: > While it works and means I can avoid predefined handle translations in the forward direction, I still have it in the backwards direction, and it's not exactly simple to resolve all the symbols. Can you elaborate a bit on the backward translation? I'm not sure I understand what you mean by that. > The second question is whether to #define MPI_HANDLE_NULL (MPI_Handle*)0. Lisandro argues for this, and it would make a lot of things easy. I believe OMPI uses special null objects for nearly all its null handles to avoid checking for NULL on every call. A null request handle for example is a valid input into `MPI_WAIT` and we can get the empty status from the null request just like we do from any other request. NULL function pointers (MPI_TYPE_NULL_DELETE_FN) can point to an empty function that we can simply invoke. NULL pointers require special treatment, which is tedious and error prone and has UB lurking around the corner. With null objects, we will never receive (or hand out) an invalid pointer, which is safe engineering. I'm curious what the benefits are from an application's (or middleware) standpoint? Cheers Joseph On 2/20/23 03:55, Jeff Hammond wrote: > For the meeting tomorrow, I would like you all to consider the following: > > There is consensus regarding the following to get type safety: > typedef struct MPI_ABI_Handle * MPI_Handle; > > We have two choices for predefined handles: > > Option 1: > #define MPI_INT (MPI_Datatype)0x3 > > Option 2: > #define MPI_INT (MPI_Datatype)&MPI_ABI_INT > > Option 1 relies on implementation defined behavior regarding the > casting of integers to pointers, but it is safe. Gonzalo and I had a > long discussion of this, and the only issue is that (intptr_t)MPI_INT > == 0x3 is not guaranteed to be true, but (intptr_t)MPI_INT == > (intptr_t)(void*)0x3 is. Both MPICH and Open-MPI already rely on this > technique for MPI_IN_PLACE, among others. > > The advantage of Option 1 - assuming we use a sensible set of > integer values - is that ABI compatibility layers can do translation > via a table. This is most useful for datatypes, where there are ~100 > predefined handles, and to a lesser extent ops. For all other > handles, there are no more than 3 predefined handles (IIRC). > > Option 2 allows run-time resolution of predefined handles, which I > thought was good until I implemented it in > https://github.com/jeffhammond/mukautuva. While it works and means I > can avoid predefined handle translations in the forward direction, I > still have it in the backwards direction, and it's not exactly simple > to resolve all the symbols. I think Hui also implemented this in his > compatibility layer, although I don't know if he hates it as much as I do. > > The second question is whether to #define MPI_HANDLE_NULL > (MPI_Handle*)0. Lisandro argues for this, and it would make a lot of > things easy. > > I hope that we can decide these two questions today. For the second > meeting, we will address predefined integer constants. > > Jeff > > -- > Jeff Hammond > jeff.science(a)gmail.com > http://jeffhammond.github.io/ >

1 0

Tuesday (20 February 2023) meeting agenda
by Jeff Hammond 20 Feb '23

20 Feb '23

For the meeting tomorrow, I would like you all to consider the following: There is consensus regarding the following to get type safety: typedef struct MPI_ABI_Handle * MPI_Handle; We have two choices for predefined handles: Option 1: #define MPI_INT (MPI_Datatype)0x3 Option 2: #define MPI_INT (MPI_Datatype)&MPI_ABI_INT Option 1 relies on implementation defined behavior regarding the casting of integers to pointers, but it is safe. Gonzalo and I had a long discussion of this, and the only issue is that (intptr_t)MPI_INT == 0x3 is not guaranteed to be true, but (intptr_t)MPI_INT == (intptr_t)(void*)0x3 is. Both MPICH and Open-MPI already rely on this technique for MPI_IN_PLACE, among others. The advantage of Option 1 - assuming we use a sensible set of integer values - is that ABI compatibility layers can do translation via a table. This is most useful for datatypes, where there are ~100 predefined handles, and to a lesser extent ops. For all other handles, there are no more than 3 predefined handles (IIRC). Option 2 allows run-time resolution of predefined handles, which I thought was good until I implemented it in https://github.com/jeffhammond/mukautuva. While it works and means I can avoid predefined handle translations in the forward direction, I still have it in the backwards direction, and it's not exactly simple to resolve all the symbols. I think Hui also implemented this in his compatibility layer, although I don't know if he hates it as much as I do. The second question is whether to #define MPI_HANDLE_NULL (MPI_Handle*)0. Lisandro argues for this, and it would make a lot of things easy. I hope that we can decide these two questions today. For the second meeting, we will address predefined integer constants. Jeff -- Jeff Hammond jeff.science(a)gmail.com http://jeffhammond.github.io/

1 0

Invitation: MPI ABI WG discussion @ Tue 2023-02-21 5pm - 6pm (EET) ([email protected])
by jeff.science＠gmail.com 20 Feb '23

20 Feb '23

MPI ABI WG discussion Tuesday 2023-02-21 ⋅ 5pm – 6pm Eastern European Time - Helsinki Location See Slack for the Teams meeting information. https://www.google.com/maps/search/See+Slack+for+the+Teams+meeting+informat… See Slack for the Teams meeting information. Not posting publicly. Organizer jeff.science(a)gmail.com jeff.science(a)gmail.com Reply for mpiwg-abi(a)lists.mpi-forum.org and view more details https://calendar.google.com/calendar/event?action=VIEW&eid=NGh0bWU4bWo2MHBi… Your attendance is optional. ~~//~~ Invitation from Google Calendar: https://calendar.google.com/calendar/ You are receiving this email because you are an attendee on the event. To stop receiving future updates for this event, decline this event. Forwarding this invitation could allow any recipient to send a response to the organizer, be added to the guest list, invite others regardless of their own invitation status, or modify your RSVP. Learn more https://support.google.com/calendar/answer/37135#forwarding

1 0