April 2023 - mpiwg-abi - mailman3.cels.anl.gov

Re: [mpiwg-abi] asynchronous voting to determine what requires more discussion (ie a meeting)
by Raffenetti, Ken 24 Apr '23

24 Apr '23

We have an item on our ABI wishlist to extend MPI_MAX_INFO_VAL once we have the chance. I believe the motivator was some RMA info extensions Min was working on years ago. So something > 1024 sounds good to me. Nothing else on our wishlist has any impact at this stage, I don't think. For reference: https://github.com/pmodels/mpich/wiki/ABI-Change-Wishlist Ken -----Original Message----- From: mpiwg-abi <mpiwg-abi-bounces(a)lists.mpi-forum.org <mailto:[email protected]>> on behalf of "Holmes, Daniel John" <daniel.john.holmes(a)intel.com <mailto:[email protected]>> Date: Monday, April 24, 2023 at 9:47 AM To: Christoph Niethammer <niethammer(a)hlrs.de <mailto:[email protected]>>, Jeff Hammond <jeff.science(a)gmail.com <mailto:[email protected]>> Cc: mpiwg-abi <mpiwg-abi(a)lists.mpi-forum.org <mailto:[email protected]>> Subject: Re: [mpiwg-abi] asynchronous voting to determine what requires more discussion (ie a meeting) Or a new set of INFO keys of the form "io_node_list_part_N" where N is 1,2,3,... for as many parts as are needed to express the whole value -- the user is invited to concatenate the INFO string values together to obtain the whole node list. PRO: arbitrarily extendable; CON: faff. Best wishes, Dan. -----Original Message----- From: mpiwg-abi <mpiwg-abi-bounces(a)lists.mpi-forum.org <mailto:[email protected]>> On Behalf Of Christoph Niethammer Sent: 24 April 2023 15:42 To: Jeff Hammond <jeff.science(a)gmail.com <mailto:[email protected]>> Cc: mpiwg-abi <mpiwg-abi(a)lists.mpi-forum.org <mailto:[email protected]>> Subject: Re: [mpiwg-abi] asynchronous voting to determine what requires more discussion (ie a meeting) For the io_node_list I could think of a solution by providing a new info key to give a path to a file containing a node list like the PBS/SLURM node list. However this might be tricky again if we have no shared file system on hand. Looking forward to Quincy's expertise in the field. Best Christoph ----- Original Message ----- From: "Jeff Hammond" <jeff.science(a)gmail.com <mailto:[email protected]>> To: "Christoph Niethammer" <niethammer(a)hlrs.de <mailto:[email protected]>> Cc: "mpiwg-abi" <mpiwg-abi(a)lists.mpi-forum.org <mailto:[email protected]>> Sent: Monday, 24 April, 2023 16:28:03 Subject: Re: [mpiwg-abi] asynchronous voting to determine what requires more discussion (ie a meeting) > > 4. Looking at the string constants I am a bit concerned about the > limit of MPI_MAX_INFO_VAL choice of only 1024. > We have, e.g., the Info keys "io_node_list" or "filename" where I could imagine that we could run into problems. > The "io_node_list" key could be a problem on lager systems that come with more than 100 cabinets - assuming for hostnames typically around 9 characters/15 byte string IP + 1 comma for separation. > The "filename" key could be a problem if we interpret the description in the standard - "This hint specifies the file name used when the file was opened." - to include the path. > This is unlimited in length for nearly all common filesystems (thinking ext4, lustre, ...). However, Linux will limit the combined filenames/path length to 4096 characters for us. Interesting. I have merely worked on the assumption that MPICH and OMPI already thought about this enough and that the larger of their choices was sufficient. It is possible that is not the case. For filename, it seems that the problem appears only in MPI_FILE_GET_INFO, because I don't see a limit for the input to MPI_FILE_OPEN. I'm not sure that making the max info val larger is the right solution. I'd be more inclined to ask the IO WG to add MPI_FILE_GET_NAME that is designed for accessing this, and design it to not have a hard-coded limit. I wonder what Quincy thinks about io_node_list, since he may work for the company with the largest ensemble of distributed storage on the planet. Jeff -- mpiwg-abi mailing list mpiwg-abi(a)lists.mpi-forum.org <mailto:[email protected]> https://lists.mpi-forum.org/mailman/listinfo/mpiwg-abi <https://lists.mpi-forum.org/mailman/listinfo/mpiwg-abi> -- mpiwg-abi mailing list mpiwg-abi(a)lists.mpi-forum.org <mailto:[email protected]> https://lists.mpi-forum.org/mailman/listinfo/mpiwg-abi <https://lists.mpi-forum.org/mailman/listinfo/mpiwg-abi>

1 0

Re: [mpiwg-abi] asynchronous voting to determine what requires more discussion (ie a meeting)
by Jeff Hammond 24 Apr '23

24 Apr '23

Yeah, I’m definitely in favor of letting implementations innovate here when the time comes, and to live within the current understanding of what and how to implement the thing we have now. Jeff > On 24. Apr 2023, at 17.47, Holmes, Daniel John <daniel.john.holmes(a)intel.com> wrote: > > Or a new set of INFO keys of the form "io_node_list_part_N" where N is 1,2,3,... for as many parts as are needed to express the whole value -- the user is invited to concatenate the INFO string values together to obtain the whole node list. PRO: arbitrarily extendable; CON: faff. > > Best wishes, > Dan. > > -----Original Message----- > From: mpiwg-abi <mpiwg-abi-bounces(a)lists.mpi-forum.org> On Behalf Of Christoph Niethammer > Sent: 24 April 2023 15:42 > To: Jeff Hammond <jeff.science(a)gmail.com> > Cc: mpiwg-abi <mpiwg-abi(a)lists.mpi-forum.org> > Subject: Re: [mpiwg-abi] asynchronous voting to determine what requires more discussion (ie a meeting) > > For the io_node_list I could think of a solution by providing a new info key to give a path to a file containing a node list like the PBS/SLURM node list. However this might be tricky again if we have no shared file system on hand. > > Looking forward to Quincy's expertise in the field. > > Best > Christoph > > > > ----- Original Message ----- > From: "Jeff Hammond" <jeff.science(a)gmail.com> > To: "Christoph Niethammer" <niethammer(a)hlrs.de> > Cc: "mpiwg-abi" <mpiwg-abi(a)lists.mpi-forum.org> > Sent: Monday, 24 April, 2023 16:28:03 > Subject: Re: [mpiwg-abi] asynchronous voting to determine what requires more discussion (ie a meeting) > >> >> 4. Looking at the string constants I am a bit concerned about the >> limit of MPI_MAX_INFO_VAL choice of only 1024. >> We have, e.g., the Info keys "io_node_list" or "filename" where I could imagine that we could run into problems. >> The "io_node_list" key could be a problem on lager systems that come with more than 100 cabinets - assuming for hostnames typically around 9 characters/15 byte string IP + 1 comma for separation. >> The "filename" key could be a problem if we interpret the description in the standard - "This hint specifies the file name used when the file was opened." - to include the path. >> This is unlimited in length for nearly all common filesystems (thinking ext4, lustre, ...). However, Linux will limit the combined filenames/path length to 4096 characters for us. > > Interesting. I have merely worked on the assumption that MPICH and OMPI already thought about this enough and that the larger of their choices was sufficient. It is possible that is not the case. > > For filename, it seems that the problem appears only in MPI_FILE_GET_INFO, because I don't see a limit for the input to MPI_FILE_OPEN. I'm not sure that making the max info val larger is the right solution. I'd be more inclined to ask the IO WG to add MPI_FILE_GET_NAME that is designed for accessing this, and design it to not have a hard-coded limit. > > I wonder what Quincy thinks about io_node_list, since he may work for the company with the largest ensemble of distributed storage on the planet. > > Jeff > -- > mpiwg-abi mailing list > mpiwg-abi(a)lists.mpi-forum.org > https://lists.mpi-forum.org/mailman/listinfo/mpiwg-abi

1 0

Re: [mpiwg-abi] asynchronous voting to determine what requires more discussion (ie a meeting)
by Holmes, Daniel John 24 Apr '23

24 Apr '23

Or a new set of INFO keys of the form "io_node_list_part_N" where N is 1,2,3,... for as many parts as are needed to express the whole value -- the user is invited to concatenate the INFO string values together to obtain the whole node list. PRO: arbitrarily extendable; CON: faff. Best wishes, Dan. -----Original Message----- From: mpiwg-abi <mpiwg-abi-bounces(a)lists.mpi-forum.org> On Behalf Of Christoph Niethammer Sent: 24 April 2023 15:42 To: Jeff Hammond <jeff.science(a)gmail.com> Cc: mpiwg-abi <mpiwg-abi(a)lists.mpi-forum.org> Subject: Re: [mpiwg-abi] asynchronous voting to determine what requires more discussion (ie a meeting) For the io_node_list I could think of a solution by providing a new info key to give a path to a file containing a node list like the PBS/SLURM node list. However this might be tricky again if we have no shared file system on hand. Looking forward to Quincy's expertise in the field. Best Christoph ----- Original Message ----- From: "Jeff Hammond" <jeff.science(a)gmail.com> To: "Christoph Niethammer" <niethammer(a)hlrs.de> Cc: "mpiwg-abi" <mpiwg-abi(a)lists.mpi-forum.org> Sent: Monday, 24 April, 2023 16:28:03 Subject: Re: [mpiwg-abi] asynchronous voting to determine what requires more discussion (ie a meeting) > > 4. Looking at the string constants I am a bit concerned about the > limit of MPI_MAX_INFO_VAL choice of only 1024. > We have, e.g., the Info keys "io_node_list" or "filename" where I could imagine that we could run into problems. > The "io_node_list" key could be a problem on lager systems that come with more than 100 cabinets - assuming for hostnames typically around 9 characters/15 byte string IP + 1 comma for separation. > The "filename" key could be a problem if we interpret the description in the standard - "This hint specifies the file name used when the file was opened." - to include the path. > This is unlimited in length for nearly all common filesystems (thinking ext4, lustre, ...). However, Linux will limit the combined filenames/path length to 4096 characters for us. Interesting. I have merely worked on the assumption that MPICH and OMPI already thought about this enough and that the larger of their choices was sufficient. It is possible that is not the case. For filename, it seems that the problem appears only in MPI_FILE_GET_INFO, because I don't see a limit for the input to MPI_FILE_OPEN. I'm not sure that making the max info val larger is the right solution. I'd be more inclined to ask the IO WG to add MPI_FILE_GET_NAME that is designed for accessing this, and design it to not have a hard-coded limit. I wonder what Quincy thinks about io_node_list, since he may work for the company with the largest ensemble of distributed storage on the planet. Jeff -- mpiwg-abi mailing list mpiwg-abi(a)lists.mpi-forum.org https://lists.mpi-forum.org/mailman/listinfo/mpiwg-abi

1 0

Re: [mpiwg-abi] asynchronous voting to determine what requires more discussion (ie a meeting)
by Christoph Niethammer 24 Apr '23

24 Apr '23

For the io_node_list I could think of a solution by providing a new info key to give a path to a file containing a node list like the PBS/SLURM node list. However this might be tricky again if we have no shared file system on hand. Looking forward to Quincy's expertise in the field. Best Christoph ----- Original Message ----- From: "Jeff Hammond" <jeff.science(a)gmail.com> To: "Christoph Niethammer" <niethammer(a)hlrs.de> Cc: "mpiwg-abi" <mpiwg-abi(a)lists.mpi-forum.org> Sent: Monday, 24 April, 2023 16:28:03 Subject: Re: [mpiwg-abi] asynchronous voting to determine what requires more discussion (ie a meeting) > > 4. Looking at the string constants I am a bit concerned about the limit of > MPI_MAX_INFO_VAL choice of only 1024. > We have, e.g., the Info keys "io_node_list" or "filename" where I could imagine that we could run into problems. > The "io_node_list" key could be a problem on lager systems that come with more than 100 cabinets - assuming for hostnames typically around 9 characters/15 byte string IP + 1 comma for separation. > The "filename" key could be a problem if we interpret the description in the standard - "This hint specifies the file name used when the file was opened." - to include the path. > This is unlimited in length for nearly all common filesystems (thinking ext4, lustre, ...). However, Linux will limit the combined filenames/path length to 4096 characters for us. Interesting. I have merely worked on the assumption that MPICH and OMPI already thought about this enough and that the larger of their choices was sufficient. It is possible that is not the case. For filename, it seems that the problem appears only in MPI_FILE_GET_INFO, because I don’t see a limit for the input to MPI_FILE_OPEN. I’m not sure that making the max info val larger is the right solution. I’d be more inclined to ask the IO WG to add MPI_FILE_GET_NAME that is designed for accessing this, and design it to not have a hard-coded limit. I wonder what Quincy thinks about io_node_list, since he may work for the company with the largest ensemble of distributed storage on the planet. Jeff

1 0

Re: [mpiwg-abi] asynchronous voting to determine what requires more discussion (ie a meeting)
by Jeff Hammond 24 Apr '23

24 Apr '23

> > 4. Looking at the string constants I am a bit concerned about the limit of > MPI_MAX_INFO_VAL choice of only 1024. > We have, e.g., the Info keys "io_node_list" or "filename" where I could imagine that we could run into problems. > The "io_node_list" key could be a problem on lager systems that come with more than 100 cabinets - assuming for hostnames typically around 9 characters/15 byte string IP + 1 comma for separation. > The "filename" key could be a problem if we interpret the description in the standard - "This hint specifies the file name used when the file was opened." - to include the path. > This is unlimited in length for nearly all common filesystems (thinking ext4, lustre, ...). However, Linux will limit the combined filenames/path length to 4096 characters for us. Interesting. I have merely worked on the assumption that MPICH and OMPI already thought about this enough and that the larger of their choices was sufficient. It is possible that is not the case. For filename, it seems that the problem appears only in MPI_FILE_GET_INFO, because I don’t see a limit for the input to MPI_FILE_OPEN. I’m not sure that making the max info val larger is the right solution. I’d be more inclined to ask the IO WG to add MPI_FILE_GET_NAME that is designed for accessing this, and design it to not have a hard-coded limit. I wonder what Quincy thinks about io_node_list, since he may work for the company with the largest ensemble of distributed storage on the planet. Jeff

1 0

Re: [mpiwg-abi] asynchronous voting to determine what requires more discussion (ie a meeting)
by Christoph Niethammer 24 Apr '23

24 Apr '23

Hello Jeff, Here my answers maybe repeating some of my comments from the last telco: 1. MPI Status: The current proposal looks good to me 2. Buffer addresses - I do not have any objections 3. Bsend overhead - I don't feel competent enough for a found comment here 4. Looking at the string constants I am a bit concerned about the limit of MPI_MAX_INFO_VAL choice of only 1024. We have, e.g., the Info keys "io_node_list" or "filename" where I could imagine that we could run into problems. The "io_node_list" key could be a problem on lager systems that come with more than 100 cabinets - assuming for hostnames typically around 9 characters/15 byte string IP + 1 comma for separation. The "filename" key could be a problem if we interpret the description in the standard - "This hint specifies the file name used when the file was opened." - to include the path. This is unlimited in length for nearly all common filesystems (thinking ext4, lustre, ...). However, Linux will limit the combined filenames/path length to 4096 characters for us. Best Christoph ----- Original Message ----- From: "Jeff Hammond" <jeff.science(a)gmail.com> To: "mpiwg-abi" <mpiwg-abi(a)lists.mpi-forum.org> Sent: Monday, 24 April, 2023 09:10:33 Subject: [mpiwg-abi] asynchronous voting to determine what requires more discussion (ie a meeting) I do not think we need a meeting unless things are complicated or controversial. I posted a Google Sheet link on Slack so we can vote on a bunch of things asynchronously. If we don't have a consensus, I'll schedule a meeting. 1. MPI_Status object definition. 2. Buffer address constants. 3. BSEND overhead 4. String length constants We discussed string constants a bunch already but many people didn't comment, so it is not clear what the overall group attitude is. I can share the Google Sheet link if necessary, but I won't post it on a public mailing list. Thanks, Jeff -- Jeff Hammond jeff.science(a)gmail.com http://jeffhammond.github.io/ -- mpiwg-abi mailing list mpiwg-abi(a)lists.mpi-forum.org https://lists.mpi-forum.org/mailman/listinfo/mpiwg-abi

1 0

asynchronous voting to determine what requires more discussion (ie a meeting)
by Jeff Hammond 24 Apr '23

24 Apr '23

I do not think we need a meeting unless things are complicated or controversial. I posted a Google Sheet link on Slack so we can vote on a bunch of things asynchronously. If we don't have a consensus, I'll schedule a meeting. 1. MPI_Status object definition. 2. Buffer address constants. 3. BSEND overhead 4. String length constants We discussed string constants a bunch already but many people didn't comment, so it is not clear what the overall group attitude is. I can share the Google Sheet link if necessary, but I won't post it on a public mailing list. Thanks, Jeff -- Jeff Hammond jeff.science(a)gmail.com http://jeffhammond.github.io/

1 0

Re: [mpiwg-abi] MPI ABI WG meeting to discuss constants
by Koziol, Quincey 15 Apr '23

15 Apr '23

Jeff - All your changes make reasonable sense to me. Quincey On Apr 12, 2023, at 2:30 AM, Jeff Hammond <jeff.science(a)gmail.com> wrote: CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe. Here is a summary of the meeting and related Slack discussion. During the meeting, I presented my proposal for integer and handle constants. However, in response to feedback, I made significant changes to it, which I will describe below. There was some debate about using the Huffman code directly versus a table. Joseph wants a table, and one that is smaller than 1024 entries. I have changed the Huffman code so that the maximum table size is not large, and is furthermore amenable to a two-level table that is even smaller, if someone wants to trade time for space. We also discussed integer constants, particularly the ones that have interesting positive values, e.g. MPI_BSEND_OVERHEAD. Dan and I agree that we should have a way to query the exact value, which is less than the ABI value, which needs to be an upper bound on all implementations. Dan and I disagree on how to do this. I am going to propose attributes similar to MPI_TAG_UB. Dan or others may propose info or something else. I encourage everyone to look at https://github.com/mpiwg-abi/specification-text-draft/blob/main/IntegerCons… and complain about things they don't like. After some pondering, I concluded that we should not encode the size of types into handles if the types are not strictly fixed-size by the language. Consider MPI_LONG, for example. Once we fix the platform ABI, e.g. LP64, we know sizeof(long) and thus can encode it in MPI_LONG. However, this would cause two problems. First, the value of MPI_LONG would not be a strict constant and second, third-party languages would need to know the behavior of C to compute the value of MPI_LONG. Lisandro made the point that we create problems for heterogeneous MPI (i.e. ILP32 on one end and LP64 on another) and related activities, and I do not want to break that. Thus, implementations will need to figure out the size of MPI_LONG and related. Fortunately, message rate benchmarks almost always use MPI_CHAR, which is a fixed-size type. The latest version of the Huffman code is in https://github.com/mpiwg-abi/specification-text-draft/pull/6/files, and it has the following features: - I made minor changes to the MPI_Op path, because I want to be consistent that MPI_*_NULL is always the zero bit pattern other than the handle type part. - Fixed-size datatypes are slightly changed, and exist in the range [576,747]. - Other datatypes changed a lot, because many C/C++ types are not actually fixed-size. These types are also more table-friendly, as they only span the range [512,572]. There are a few different implementations that make sense here: 1. All handles in a single table of 1024 entries (where values is whatever the implementation uses internally: MPICH -> int; OMPI -> pointer), which is ~90% sparse. The worst case storage requirement is 8KiB, which is not onerous. 2. Datatypes fit into a table of 236 entries, which can be compressed by using an 8-bit hash (internal = table[indirection[ handle & 0b11111111]]) into a table of ~71 entries. 3. Datatypes use a mostly dense table of 60 entries for non-fixed-size types and something else for the fixed-size types. 4. The Huffman code is used directly, which requires a handful of register bit mask/shift ops to lookup internal handle values. There is an open question about whether we should make the buffer address constants equivalent to zero when they have the same effect as C NULL, or whether we should give them distinct values for debugging purposes. I am inclined to make all of the following zero except IN_PLACE, UNWEIGHTED and WEIGHTS_EMPTY, but I am happy to hear feedback to the contrary. MPI_STATUS_IGNORE = 0 MPI_STATUSES_IGNORE = 0 MPI_ERRCODES_IGNORE = 0 MPI_ARGV_NULL = 0 MPI_ARGVS_NULL = 0 MPI_IN_PLACE != MPI_BOTTOM = 0 MPI_WEIGHTS_EMPTY != MPI_UNWEIGHTED != 0 Jeff On Tue, Apr 4, 2023 at 6:05 PM Jeff Hammond <jehammond(a)nvidia.com<mailto:[email protected]>> wrote: I apologize for my incompetence with meeting scheduling and attendance this week. Hopefully this time I am able to get the timing correct and also attend. The topics / decisions of interest are: A. Integer constants We have constants that must be powers of two (mode constants), constants that must have a < relationship (error codes and thread levels), constants that should be negative to avoid conflicting with e.g. ranks, and constants that can be anything. I have a proposal for constants here: https://github.com/mpiwg-abi/specification-text-draft/blob/main/IntegerCons…. I have not yet implemented Dan's suggestion to make constants unique (presumably, for the aforementioned categories that allow it). I would like feedback on the following: 1. Are the values of MPI_MAX_*** acceptable? In all cases, I chose the greater of MPICH and OMPI. For MPI_BSEND_OVERHEAD, I used 512 to be safe. Is that too large? Is 128 better? 2. Are there any opponents to Dan's suggestion that all the constants are unique, within reason? If the consensus favors this, I'll redo all the constants accordingly. 3. Are there any other integer constant values that bother people? I have very little attachment to any of them, and it is trivial to change them now. B. Handle constants I implemented a Huffman code for these (https://github.com/mpiwg-abi/specification-text-draft/blob/main/HandleConst…) I made a Python code<https://github.com/mpiwg-abi/specification-text-draft/blob/main/print-handl…> that implements it, and the program dumps all the values, and can be modified easily to generate mpi_abi.h. Does anyone oppose the idea of a Huffman code? I know Hui is indifferent, which is obviously fine. One can ignore the Huffman code and just view the results as some random values I chose :-) If you like Huffman codes, but dislike mine, then please complain soon. There are some parts that I do not love. For example, fixed size types are handled consistently and encode their size (as the log_2 of bytes) in bits 3:5, while language default types are on a different branch and encode their size in bits 8:10. I can makes those consistent, but it means the code branches aren't sequential in the bit indices. I think that's fine, but I am new to Huffman codes. An alternative to the above is to say that in mpi_abi.h, that MPI_INT, MPI_LONG and MPI_LONG_LONG are aliased to MPI_INTn_T according to the ABI definition, and do not exist on their own. This has some appeal, but will change the results of MPI_Type_get_name. What do people think about this? The good news is there is lots of free space in the Huffman code for new handle types and new constants. I am not worried about running out of space. Already, I have reserved space for a bunch of types that are likely to exist in C and C++ in the near future, so those will be trivial to add later. Finally, as noted on Slack, we have to figure out whether we reserve space for or standardize some types that OMPI defines today in the MPI_ namespace. This is the lowest priority for me right now, so if we don't address it this week, that is fine. Thanks, Jeff -- Jeff Hammond jeff.science(a)gmail.com<mailto:[email protected]> http://jeffhammond.github.io/ -- mpiwg-abi mailing list mpiwg-abi(a)lists.mpi-forum.org https://lists.mpi-forum.org/mailman/listinfo/mpiwg-abi

1 0

Re: [mpiwg-abi] MPI ABI WG meeting to discuss constants
by Jeff Hammond 12 Apr '23

12 Apr '23

Here is a summary of the meeting and related Slack discussion. During the meeting, I presented my proposal for integer and handle constants. However, in response to feedback, I made significant changes to it, which I will describe below. There was some debate about using the Huffman code directly versus a table. Joseph wants a table, and one that is smaller than 1024 entries. I have changed the Huffman code so that the maximum table size is not large, and is furthermore amenable to a two-level table that is even smaller, if someone wants to trade time for space. We also discussed integer constants, particularly the ones that have interesting positive values, e.g. MPI_BSEND_OVERHEAD. Dan and I agree that we should have a way to query the exact value, which is less than the ABI value, which needs to be an upper bound on all implementations. Dan and I disagree on how to do this. I am going to propose attributes similar to MPI_TAG_UB. Dan or others may propose info or something else. I encourage everyone to look at https://github.com/mpiwg-abi/specification-text-draft/blob/main/IntegerCons… and complain about things they don't like. After some pondering, I concluded that we should not encode the size of types into handles if the types are not strictly fixed-size by the language. Consider MPI_LONG, for example. Once we fix the platform ABI, e.g. LP64, we know sizeof(long) and thus can encode it in MPI_LONG. However, this would cause two problems. First, the value of MPI_LONG would not be a strict constant and second, third-party languages would need to know the behavior of C to compute the value of MPI_LONG. Lisandro made the point that we create problems for heterogeneous MPI (i.e. ILP32 on one end and LP64 on another) and related activities, and I do not want to break that. Thus, implementations will need to figure out the size of MPI_LONG and related. Fortunately, message rate benchmarks almost always use MPI_CHAR, which is a fixed-size type. The latest version of the Huffman code is in https://github.com/mpiwg-abi/specification-text-draft/pull/6/files, and it has the following features: - I made minor changes to the MPI_Op path, because I want to be consistent that MPI_*_NULL is always the zero bit pattern other than the handle type part. - Fixed-size datatypes are slightly changed, and exist in the range [576,747]. - Other datatypes changed a lot, because many C/C++ types are not actually fixed-size. These types are also more table-friendly, as they only span the range [512,572]. There are a few different implementations that make sense here: 1. All handles in a single table of 1024 entries (where values is whatever the implementation uses internally: MPICH -> int; OMPI -> pointer), which is ~90% sparse. The worst case storage requirement is 8KiB, which is not onerous. 2. Datatypes fit into a table of 236 entries, which can be compressed by using an 8-bit hash (internal = table[indirection[ handle & 0b11111111]]) into a table of ~71 entries. 3. Datatypes use a mostly dense table of 60 entries for non-fixed-size types and something else for the fixed-size types. 4. The Huffman code is used directly, which requires a handful of register bit mask/shift ops to lookup internal handle values. There is an open question about whether we should make the buffer address constants equivalent to zero when they have the same effect as C NULL, or whether we should give them distinct values for debugging purposes. I am inclined to make all of the following zero except IN_PLACE, UNWEIGHTED and WEIGHTS_EMPTY, but I am happy to hear feedback to the contrary. MPI_STATUS_IGNORE = 0 MPI_STATUSES_IGNORE = 0 MPI_ERRCODES_IGNORE = 0 MPI_ARGV_NULL = 0 MPI_ARGVS_NULL = 0 MPI_IN_PLACE != MPI_BOTTOM = 0 MPI_WEIGHTS_EMPTY != MPI_UNWEIGHTED != 0 Jeff On Tue, Apr 4, 2023 at 6:05 PM Jeff Hammond <jehammond(a)nvidia.com> wrote: > I apologize for my incompetence with meeting scheduling and attendance > this week. Hopefully this time I am able to get the timing correct and > also attend. > > > > The topics / decisions of interest are: > > > > A. Integer constants > > > > We have constants that must be powers of two (mode constants), > constants that must have a < relationship (error codes and thread levels), > constants that should be negative to avoid conflicting with e.g. ranks, and > constants that can be anything. > > > > I have a proposal for constants here: > https://github.com/mpiwg-abi/specification-text-draft/blob/main/IntegerCons…. > I have not yet implemented Dan's suggestion to make constants unique > (presumably, for the aforementioned categories that allow it). > > > > I would like feedback on the following: > > 1. Are the values of MPI_MAX_*** acceptable? In all cases, I chose the > greater of MPICH and OMPI. For MPI_BSEND_OVERHEAD, I used 512 to be safe. > Is that too large? Is 128 better? > > 2. Are there any opponents to Dan's suggestion that all the constants are > unique, within reason? If the consensus favors this, I'll redo all the > constants accordingly. > > 3. Are there any other integer constant values that bother people? I have > very little attachment to any of them, and it is trivial to change them now. > > > > B. Handle constants > > > > I implemented a Huffman code for these ( > https://github.com/mpiwg-abi/specification-text-draft/blob/main/HandleConst…) > I made a Python code > <https://github.com/mpiwg-abi/specification-text-draft/blob/main/print-handl…> that > implements it, and the program dumps all the values, and can be modified > easily to generate mpi_abi.h. > > > > Does anyone oppose the idea of a Huffman code? I know Hui is indifferent, > which is obviously fine. One can ignore the Huffman code and just view the > results as some random values I chose :-) > > > > If you like Huffman codes, but dislike mine, then please complain soon. > There are some parts that I do not love. For example, fixed size types are > handled consistently and encode their size (as the log_2 of bytes) in bits > 3:5, while language default types are on a different branch and encode > their size in bits 8:10. I can makes those consistent, but it means the > code branches aren't sequential in the bit indices. I think that's fine, > but I am new to Huffman codes. > > > > An alternative to the above is to say that in mpi_abi.h, that MPI_INT, > MPI_LONG and MPI_LONG_LONG are aliased to MPI_INTn_T according to > the ABI definition, and do not exist on their own. This has some appeal, > but will change the results of MPI_Type_get_name. What do people think > about this? > > > > The good news is there is lots of free space in the Huffman code for new > handle types and new constants. I am not worried about running out of > space. Already, I have reserved space for a bunch of types that are likely > to exist in C and C++ in the near future, so those will be trivial to add > later. > > > > Finally, as noted on Slack, we have to figure out whether we reserve space > for or standardize some types that OMPI defines today in the MPI_ > namespace. This is the lowest priority for me right now, so if we don't > address it this week, that is fine. > > > > Thanks, > > > > Jeff > -- Jeff Hammond jeff.science(a)gmail.com http://jeffhammond.github.io/

1 0

MPI ABI WG meeting to discuss constants
by Jeff Hammond 04 Apr '23

04 Apr '23

I apologize for my incompetence with meeting scheduling and attendance this week. Hopefully this time I am able to get the timing correct and also attend. The topics / decisions of interest are: A. Integer constants We have constants that must be powers of two (mode constants), constants that must have a < relationship (error codes and thread levels), constants that should be negative to avoid conflicting with e.g. ranks, and constants that can be anything. I have a proposal for constants here: https://github.com/mpiwg-abi/specification-text-draft/blob/main/IntegerCons…. I have not yet implemented Dan's suggestion to make constants unique (presumably, for the aforementioned categories that allow it). I would like feedback on the following: 1. Are the values of MPI_MAX_*** acceptable? In all cases, I chose the greater of MPICH and OMPI. For MPI_BSEND_OVERHEAD, I used 512 to be safe. Is that too large? Is 128 better? 2. Are there any opponents to Dan's suggestion that all the constants are unique, within reason? If the consensus favors this, I'll redo all the constants accordingly. 3. Are there any other integer constant values that bother people? I have very little attachment to any of them, and it is trivial to change them now. B. Handle constants I implemented a Huffman code for these (https://github.com/mpiwg-abi/specification-text-draft/blob/main/HandleConst…) I made a Python code<https://github.com/mpiwg-abi/specification-text-draft/blob/main/print-handl…> that implements it, and the program dumps all the values, and can be modified easily to generate mpi_abi.h. Does anyone oppose the idea of a Huffman code? I know Hui is indifferent, which is obviously fine. One can ignore the Huffman code and just view the results as some random values I chose :-) If you like Huffman codes, but dislike mine, then please complain soon. There are some parts that I do not love. For example, fixed size types are handled consistently and encode their size (as the log_2 of bytes) in bits 3:5, while language default types are on a different branch and encode their size in bits 8:10. I can makes those consistent, but it means the code branches aren't sequential in the bit indices. I think that's fine, but I am new to Huffman codes. An alternative to the above is to say that in mpi_abi.h, that MPI_INT, MPI_LONG and MPI_LONG_LONG are aliased to MPI_INTn_T according to the ABI definition, and do not exist on their own. This has some appeal, but will change the results of MPI_Type_get_name. What do people think about this? The good news is there is lots of free space in the Huffman code for new handle types and new constants. I am not worried about running out of space. Already, I have reserved space for a bunch of types that are likely to exist in C and C++ in the near future, so those will be trivial to add later. Finally, as noted on Slack, we have to figure out whether we reserve space for or standardize some types that OMPI defines today in the MPI_ namespace. This is the lowest priority for me right now, so if we don't address it this week, that is fine. Thanks, Jeff ________________________________________________________________________________ Microsoft Teams meeting Join on your computer, mobile app or room device Click here to join the meeting<https://teams.microsoft.com/l/meetup-join/19%3ameeting_NmYyZGVjZTEtMGY4Yi00…> Meeting ID: 252 426 152 185 Passcode: s23ETu Download Teams<https://www.microsoft.com/en-us/microsoft-teams/download-app> | Join on the web<https://www.microsoft.com/microsoft-teams/join-a-meeting> Join with a video conferencing device teams(a)vc.nvidia.com Video Conference ID: 114 591 421 4 Alternate VTC instructions<https://pexip.me/teams/vc.nvidia.com/1145914214> Or call in (audio only) +1 949-570-1120,,525452510#<tel:+19495701120,,525452510#> United States, Irvine Phone Conference ID: 525 452 510# Find a local number<https://dialin.teams.microsoft.com/c58d41ec-314f-4761-9381-98dfc4066d1b?id=…> | Reset PIN<https://dialin.teams.microsoft.com/usp/pstnconferencing> Learn More<https://aka.ms/JoinTeamsMeeting> | Meeting options<https://teams.microsoft.com/meetingOptions/?organizerId=64532211-6004-4136-…> ________________________________________________________________________________

1 0