Thanks. I know it works in practice on Linux. I just can’t figure out if the C standard’s notion of an array object includes Linux’s flat virtual address space or if it means specific C arrays.
Subtraction seems like an unnecessary way to cast to ptrdiff_t, but obviously it works.
My primary goal here is understanding what, if any, impact it will have to remove all the segmented addressing text from the standard.
Jeff
> On 23. Jan 2024, at 4.55, Zhou, Hui <zhouh(a)anl.gov> wrote:
>
> At least for Linux, Windows, and osX, we are using non-segmented virtual address, thus all address points to (at least in principle) the same array object. Thus, it is not undefined.
>
> The #ifdef CHAR_PTR_IS_ADDRESS branch casts pointer to MPI_Aint.
>
> The alternative branch casts ptrdiff_t to MPI_Aint.
>
> --
> Hui
> From: Jeff Hammond via devel <devel(a)mpich.org>
> Sent: Monday, January 22, 2024 1:46 AM
> To: MPICH <devel(a)mpich.org>
> Cc: Jeff Hammond <jeff.science(a)gmail.com>
> Subject: [mpich-devel] MPIR_Get_address_impl
>
> I am trying to understand MPI_Get_address and its interactions with C better. I've included the relevant text from C89 and C23 on pointer subtraction at the bottom.
>
> As best I can tell, MPICH's "portable" way to generate an address is undefined behavior unless pointers behave as if they point to the same array object, which is always true only when MPI_BOTTOM is zero. Thus, subtraction of MPI_BOTTOM is pointless: it either has no effect or is undefined behavior.
>
> The SX-4 workaround from the late 1990s is caused by a compiler bug, unless the SX-4 C compiler defines ptrdiff_t to be int, which would be nonsensical on a 64-bit system. However, the workaround is actually the correct implementation, based on the previous paragraph.
>
> Does this make sense?
>
> Jeff
>
> int MPIR_Get_address_impl(const void *location, MPI_Aint * address)
> {
> /* SX_4 needs to set CHAR_PTR_IS_ADDRESS
> * The reason is that it computes the different in two pointers in
> * an "int", and addresses typically have the high (bit 31) bit set;
> * thus the difference, when cast as MPI_Aint (long), is sign-extended,
> * making the absolute address negative. Without a copy of the C
> * standard, I can't tell if this is a compiler bug or a language bug.
> */
> #ifdef CHAR_PTR_IS_ADDRESS
> *address = (MPI_Aint) location;
> #else
> /* Note that this is the "portable" way to generate an address.
> * The difference of two pointers is the number of elements
> * between them, so this gives the number of chars between location
> * and ptr. As long as sizeof(char) represents one byte,
> * of bytes from 0 to location */
> *address = (MPI_Aint) ((char *) location - (char *) MPI_BOTTOM);
> #endif
> /* The same code is used in MPI_Address */
> return MPI_SUCCESS;
> }
>
> ANSI/ISO 9899:1990
>
> 6.3.6 Additive operators
> "When two pointers to elements of the same array object are subtracted, the result is the difference of the subscripts of the two array elements. The size of the result is implementation-defined. and its type (a signed integral type) is ptrdiff_t defined in the <stddef. h> header. As with any other arithmetic overflow, if the result does not fit in the space provided, the behavior is undefined. In other words, if the expressions P and Q point to, respectively, the i-th and j-th elements of an array object, the expression (P)-(Q) has the value i-j provided the value fits in an object of type ptrdiff_t. Moreover, if the expression P points either to an element of an array object or one past the last element of an array object, and the expression Q points IO the last element of the same array object. the expression ((Q)+1) - (P) has the same value as ((Q) - (P))+1 and as -((P)- ((Q)+1)), and has the value zero if the expression P points one past the last element of the array object, even though the expression (Q)+1 does not point to an element of the array object. Unless both pointers point to elements of the same array object, or one past the last element of the array object, the behavior is undefined."
>
> G.2 Undefined behavior
> "Pointers that do not behave as if they point to the same array object are subtracted (6.3.6)."
>
> ISO/IEC 9899:2023
> §6.5.6 Additive operators
> 10 When two pointers are subtracted, both shall point to elements of the same array object, or one past the last element of the array object; the result is the difference of the subscripts of the two array elements. The size of the result is implementation-defined, and its type (a signed integer type) is ptrdiff_t defined in the <stddef.h> header. If the result is not representable in an object of that type, the behavior is undefined. In other words, if the expressions P and Q point to, respectively, the i-th and j-th elements of an array object, the expression (P)-(Q) has the value i − j provided the value fits in an object of type ptrdiff_t. Moreover, if the expression P points either to an element of an array object or one past the last element of an array object, and the expression Q points to the last element of the same array object, the expression ((Q)+1)-(P) has the same value as ((Q)-(P))+1 and as -((P)-((Q)+1)), and has the value zero if the expression P points one past the last element of the array object, even though the expression (Q)+1 does not point to an element of the array object.119)
>
> J.2 Undefined behavior
> 1 The behavior is undefined in the following circumstances:
> (43) Addition or subtraction of a pointer into, or just beyond, an array object and an integer type produces a result that does not point into, or just beyond, the same array object (6.5.6).
>
> --
> Jeff Hammond
> jeff.science(a)gmail.com <mailto:[email protected]>
> http://jeffhammond.github.io/
At least for Linux, Windows, and osX, we are using non-segmented virtual address, thus all address points to (at least in principle) the same array object. Thus, it is not undefined.
The #ifdef CHAR_PTR_IS_ADDRESS branch casts pointer to MPI_Aint.
The alternative branch casts ptrdiff_t to MPI_Aint.
--
Hui
________________________________
From: Jeff Hammond via devel <devel(a)mpich.org>
Sent: Monday, January 22, 2024 1:46 AM
To: MPICH <devel(a)mpich.org>
Cc: Jeff Hammond <jeff.science(a)gmail.com>
Subject: [mpich-devel] MPIR_Get_address_impl
I am trying to understand MPI_Get_address and its interactions with C better. I've included the relevant text from C89 and C23 on pointer subtraction at the bottom.
As best I can tell, MPICH's "portable" way to generate an address is undefined behavior unless pointers behave as if they point to the same array object, which is always true only when MPI_BOTTOM is zero. Thus, subtraction of MPI_BOTTOM is pointless: it either has no effect or is undefined behavior.
The SX-4 workaround from the late 1990s is caused by a compiler bug, unless the SX-4 C compiler defines ptrdiff_t to be int, which would be nonsensical on a 64-bit system. However, the workaround is actually the correct implementation, based on the previous paragraph.
Does this make sense?
Jeff
int MPIR_Get_address_impl(const void *location, MPI_Aint * address)
{
/* SX_4 needs to set CHAR_PTR_IS_ADDRESS
* The reason is that it computes the different in two pointers in
* an "int", and addresses typically have the high (bit 31) bit set;
* thus the difference, when cast as MPI_Aint (long), is sign-extended,
* making the absolute address negative. Without a copy of the C
* standard, I can't tell if this is a compiler bug or a language bug.
*/
#ifdef CHAR_PTR_IS_ADDRESS
*address = (MPI_Aint) location;
#else
/* Note that this is the "portable" way to generate an address.
* The difference of two pointers is the number of elements
* between them, so this gives the number of chars between location
* and ptr. As long as sizeof(char) represents one byte,
* of bytes from 0 to location */
*address = (MPI_Aint) ((char *) location - (char *) MPI_BOTTOM);
#endif
/* The same code is used in MPI_Address */
return MPI_SUCCESS;
}
ANSI/ISO 9899:1990
6.3.6 Additive operators
"When two pointers to elements of the same array object are subtracted, the result is the difference of the subscripts of the two array elements. The size of the result is implementation-defined. and its type (a signed integral type) is ptrdiff_t defined in the <stddef. h> header. As with any other arithmetic overflow, if the result does not fit in the space provided, the behavior is undefined. In other words, if the expressions P and Q point to, respectively, the i-th and j-th elements of an array object, the expression (P)-(Q) has the value i-j provided the value fits in an object of type ptrdiff_t. Moreover, if the expression P points either to an element of an array object or one past the last element of an array object, and the expression Q points IO the last element of the same array object. the expression ((Q)+1) - (P) has the same value as ((Q) - (P))+1 and as -((P)- ((Q)+1)), and has the value zero if the expression P points one past the last element of the array object, even though the expression (Q)+1 does not point to an element of the array object. Unless both pointers point to elements of the same array object, or one past the last element of the array object, the behavior is undefined."
G.2 Undefined behavior
"Pointers that do not behave as if they point to the same array object are subtracted (6.3.6)."
ISO/IEC 9899:2023
§6.5.6 Additive operators
1.
2. 10 When two pointers are subtracted, both shall point to elements of the same array object, or one past the last element of the array object; the result is the difference of the subscripts of the two array elements. The size of the result is implementation-defined, and its type (a signed integer type) is ptrdiff_t defined in the <stddef.h> header. If the result is not representable in an object of that type, the behavior is undefined. In other words, if the expressions P and Q point to, respectively, the i-th and j-th elements of an array object, the expression (P)-(Q) has the value i − j provided the value fits in an object of type ptrdiff_t. Moreover, if the expression P points either to an element of an array object or one past the last element of an array object, and the expression Q points to the last element of the same array object, the expression ((Q)+1)-(P) has the same value as ((Q)-(P))+1 and as -((P)-((Q)+1)), and has the value zero if the expression P points one past the last element of the array object, even though the expression (Q)+1 does not point to an element of the array object.119)
J.2 Undefined behavior
1 The behavior is undefined in the following circumstances:
(43) Addition or subtraction of a pointer into, or just beyond, an array object and an integer type produces a result that does not point into, or just beyond, the same array object (6.5.6).
--
Jeff Hammond
jeff.science(a)gmail.com<mailto:[email protected]>
http://jeffhammond.github.io/
I am trying to understand MPI_Get_address and its interactions with C
better. I've included the relevant text from C89 and C23 on pointer
subtraction at the bottom.
As best I can tell, MPICH's "portable" way to generate an address is
undefined behavior unless pointers behave as if they point to the same
array object, which is always true only when MPI_BOTTOM is zero. Thus,
subtraction of MPI_BOTTOM is pointless: it either has no effect or is
undefined behavior.
The SX-4 workaround from the late 1990s is caused by a compiler bug, unless
the SX-4 C compiler defines ptrdiff_t to be int, which would be nonsensical
on a 64-bit system. However, the workaround is actually the correct
implementation, based on the previous paragraph.
Does this make sense?
Jeff
int MPIR_Get_address_impl(const void *location, MPI_Aint * address)
{
/* SX_4 needs to set CHAR_PTR_IS_ADDRESS
* The reason is that it computes the different in two pointers in
* an "int", and addresses typically have the high (bit 31) bit set;
* thus the difference, when cast as MPI_Aint (long), is sign-extended,
* making the absolute address negative. Without a copy of the C
* standard, I can't tell if this is a compiler bug or a language bug.
*/
#ifdef CHAR_PTR_IS_ADDRESS
*address = (MPI_Aint) location;
#else
/* Note that this is the "portable" way to generate an address.
* The difference of two pointers is the number of elements
* between them, so this gives the number of chars between location
* and ptr. As long as sizeof(char) represents one byte,
* of bytes from 0 to location */
*address = (MPI_Aint) ((char *) location - (char *) MPI_BOTTOM);
#endif
/* The same code is used in MPI_Address */
return MPI_SUCCESS;
}
ANSI/ISO 9899:1990
6.3.6 Additive operators
"When two pointers to elements of the same array object are subtracted, the
result is the difference of the subscripts of the two array elements. The
size of the result is implementation-defined. and its type (a signed
integral type) is ptrdiff_t defined in the <stddef. h> header. As with any
other arithmetic overflow, if the result does not fit in the space
provided, the behavior is undefined. In other words, if the expressions P
and Q point to, respectively, the i-th and j-th elements of an array
object, the expression (P)-(Q) has the value i-j provided the value fits in
an object of type ptrdiff_t. Moreover, if the expression P points either to
an element of an array object or one past the last element of an array
object, and the expression Q points IO the last element of the same array
object. the expression ((Q)+1) - (P) has the same value as ((Q) - (P))+1
and as -((P)- ((Q)+1)), and has the value zero if the expression P points
one past the last element of the array object, even though the expression
(Q)+1 does not point to an element of the array object. Unless both
pointers point to elements of the same array object, or one past the last
element of the array object, the behavior is undefined."
G.2 Undefined behavior
"Pointers that do not behave as if they point to the same array object are
subtracted (6.3.6)."
ISO/IEC 9899:2023
§6.5.6 Additive operators
1.
2. 10 When two pointers are subtracted, both shall point to elements of
the same array object, or one past the last element of the array object;
the result is the difference of the subscripts of the two array elements.
The size of the result is implementation-defined, and its type (a signed
integer type) is ptrdiff_t defined in the <stddef.h> header. If the
result is not representable in an object of that type, the behavior is
undefined. In other words, if the expressions P and Q point to,
respectively, the i-th and j-th elements of an array object, the
expression (P)-(Q) has the value i − j provided the value fits in an
object of type ptrdiff_t. Moreover, if the expression P points either to
an element of an array object or one past the last element of an array
object, and the expression Q points to the last element of the same
array object, the expression ((Q)+1)-(P) has the same value as ((Q)-(P))+1
and as -((P)-((Q)+1)), and has the value zero if the expression P points
one past the last element of the array object, even though the expression
(Q)+1 does not point to an element of the array object.119)
J.2 Undefined behavior
1 The behavior is undefined in the following circumstances:
(43) Addition or subtraction of a pointer into, or just beyond, an array
object and an integer type produces a result that does not point into, or
just beyond, the same array object (6.5.6).
--
Jeff Hammond
jeff.science(a)gmail.com
http://jeffhammond.github.io/
