October 2020 - mpi-forum - mailman3.cels.anl.gov

Errata Announcement for Dec. Meeting
by Aurelien Bouteiller 28 Oct '20

28 Oct '20

Hi all, I’d like to announce the following issue/PR for inclusion in the MPI 4.0 RC and voting at the December meeting: Ticket - https://github.com/mpi-forum/mpi-issues/issues/313 PR - https://github.com/mpi-forum/mpi-standard/pull/281 This is the issue I read last week in the virtual meeting which adds producing wrong outputs to the list of behaviors that can limit the ability of MPI to produce meaningful error codes. Best Regards, Aurelien -- Aurelien Bouteiller, Ph.D. Innovative Computing Laboratory; The University of Tennessee; 1122 Volunteer Blvd.; Claxton suite 203; 37996, Knoxville, TN, USA +1 865 974 9308 (p); +1 865 974 8296 (f); Claxton 316 (f2f)

1 0

Virtual MPI Forum Meeting Tomorrow/Wednesday 10/28 10am CT (NOTE CHANGED TIME IN EUROPE)
by Martin Schulz 27 Oct '20

27 Oct '20

Hi all, Just the usual reminder - we have our next virtual MPI forum meeting tomorrow (Wednesday) at the usual time at 10am CT. The link is - as always - available from the forum website. For the European folks, note that we have shifted away from daylight savings, while the US has not, hence the meeting is an hour earlier this week. My apologies, I do not know the impact on folks in Asia, but I assume that the impact is delayed by a week there (please chime in with details!). As in the previous weeks, the plan is to go over all items on the MPI 4.0 project board (https://github.com/mpi-forum/mpi-issues/projects/2) If you have any items, please make sure to update them following Wesley's Wiki instructions. Thanks, Martin -- Prof. Dr. Martin Schulz, Chair of Computer Architecture and Parallel Systems Department of Informatics, TU-Munich, Boltzmannstraße 3, D-85748 Garching Member of the Board of Directors at the Leibniz Supercomputing Centre (LRZ) Email: schulzm(a)in.tum.de

1 0

Errata Announcement for December Meeting / MPI 4.0 RC
by Wesley Bland 27 Oct '20

27 Oct '20

Hi all, I’d like to announce the following issue/PR for inclusion in the MPI 4.0 RC and voting at the December meeting: Error: Section 12.4.3 - p. 515 - MPI_INIT argc/argv values Issue - https://github.com/mpi-forum/mpi-issues/issues/168 <https://github.com/mpi-forum/mpi-issues/issues/168> PR - https://github.com/mpi-forum/mpi-standard/pull/182 <https://github.com/mpi-forum/mpi-standard/pull/182> This is the issue I read last week in the virtual meeting which copies the text about how to handle argc/argv in MPI_INIT_THREAD. Thanks, Wes

1 0

MPI Forum Virtual Meeting Tomorrow
by Wesley Bland 20 Oct '20

20 Oct '20

Hi all, I just wanted to remind everyone to be updating the status of any issues/pull requests being worked on for MPI 4.0 on the project board (https://github.com/mpi-forum/mpi-issues/projects/2 <https://github.com/mpi-forum/mpi-issues/projects/2>). Tomorrow we have a virtual meeting to get updates on anything that needs to be discussed there. A few folks have told me that they would like to go ahead with some unofficial readings for those items so we can move them forward for the release candidate coming out in a few weeks. Please remember that we need to have everything that has already been voted into MPI 4.0 merged into the mpi-4.x branch by the end of the month so we can start merging the remaining errata and other fixes into the release candidate branch immediately after that to be able to generate the release candidate for us to review and release at SC20. See you tomorrow. Thanks, Wes

1 0

Re: [Mpi-forum] Question about MPI_Alloc_mem
by Jeff Hammond 15 Oct '20

15 Oct '20

Just implement MPI allocation sanity once and forget about it :-) #include <mpi.h> void * MPIX_Malloc(MPI_Aint size, MPI_Info info, int * status) { void * ptr = NULL; *status = MPI_Alloc_mem(size, info, &ptr); return ptr; } int MPIX_Free(void * ptr) { return MPI_Free_mem(ptr); } Jeff On Wed, Oct 14, 2020 at 12:53 PM Skjellum, Anthony via mpi-forum < mpi-forum(a)lists.mpi-forum.org> wrote: > Bill, I thank you much for your input too. > > POSIX has no problem making you do the cast in C 🙂 > > *int posix_memalign(void ****memptr**, size_t* *alignment**, size_t* *size**);* > > I get why its that way now in MPI, I still don't like it 🙂 > > Tony > > cf, > https://linux.die.net/man/3/posix_memalign > posix_memalign(3): allocate aligned memory - Linux man page > <https://linux.die.net/man/3/posix_memalign> > The function posix_memalign() allocates size bytes and places the address > of the allocated memory in *memptr. The address of the allocated memory > will be a ... > linux.die.net > > > > > > Anthony Skjellum, PhD > > Professor of Computer Science and Chair of Excellence > > Director, SimCenter > > University of Tennessee at Chattanooga (UTC) > > tony-skjellum(a)utc.edu [or skjellum(a)gmail.com] > > cell: 205-807-4968 > > > ------------------------------ > *From:* William Gropp <wgropp(a)illinois.edu> > *Sent:* Wednesday, October 14, 2020 3:48 PM > *To:* Main MPI Forum mailing list <mpi-forum(a)lists.mpi-forum.org> > *Cc:* Rolf Rabenseifner <rabenseifner(a)hlrs.de>; Skjellum, Anthony < > Tony-Skjellum(a)utc.edu> > *Subject:* Re: [Mpi-forum] Question about MPI_Alloc_mem > > My previous email (probably crossing paths with Tony’s reply) addressed > most of this. The one thing that I want to add is that the API choice made > in MPI for the C binding, both in the MPI-1 attribute get functions and > here, has nothing to do with the Fortran binding. These were made to match > C’s limitations on anonymous pointers, and to avoid requiring pointer > casts. > > Bill > > William Gropp > Director and Chief Scientist, NCSA > Thomas M. Siebel Chair in Computer Science > University of Illinois Urbana-Champaign > > > > > > > On Oct 14, 2020, at 2:42 PM, Skjellum, Anthony via mpi-forum < > mpi-forum(a)lists.mpi-forum.org> wrote: > > Rolf, the rationale is not clear at all to me: "to facilitate type > casting" is self-understood or a canonical term of art. I've been > programming in C for 40 years... and I never would write the API as we have > done it there. > > For pointers, out arguments are literally truthful: void **ptr is an out > argument of a void pointer, and void *ptr is an in argument of a void > pointer. > > {Years ago, before there was void *, we would have written char **ptr for > a char out argument (and cast it to whatever type we wanted), and char *ptr > for an in point argument. } > > The fact that a void ** can stand in the place of a void * is clearly a > weakened typing in the language, because void * can point at anything > (unknown). Since a void * is a pointer, it can also hold a pointer to a > pointer to void, sure. But the intent of the second * is to remind you > that you have an out argument. And you *must* pass a pointer to a pointer > for the API to work in C. > > Somehow this is helping the Fortran and C_PTR API of Fortran, as it > states. So the C API is made this way for convenience of Fortran. > > The rationale shows the argument with the & to pass the pointer to the API > by reference. But, the rationale is not necessarily normative. A > reasonable person reads the standard, sees void * and passes the baseptr > without the &. > > Seems like a bad API to me. > > Tony > > > Anthony Skjellum, PhD > Professor of Computer Science and Chair of Excellence > Director, SimCenter > University of Tennessee at Chattanooga (UTC) > tony-skjellum(a)utc.edu [or skjellum(a)gmail.com] > cell: 205-807-4968 > > ------------------------------ > *From:* mpi-forum <mpi-forum-bounces(a)lists.mpi-forum.org> on behalf of > Rolf Rabenseifner via mpi-forum <mpi-forum(a)lists.mpi-forum.org> > *Sent:* Wednesday, October 14, 2020 1:32 PM > *To:* Main MPI Forum mailing list <mpi-forum(a)lists.mpi-forum.org> > *Cc:* Rolf Rabenseifner <rabenseifner(a)hlrs.de> > *Subject:* Re: [Mpi-forum] Question about MPI_Alloc_mem > > Dear Tony, > > please read MPI-3.1 page 338, lines 36-41. > Do these lines resolve your question? > > Best regards > Rolf > > ----- Original Message ----- > > From: "Main MPI Forum mailing list" <mpi-forum(a)lists.mpi-forum.org> > > To: "Main MPI Forum mailing list" <mpi-forum(a)lists.mpi-forum.org> > > Cc: "Anthony Skjellum" <skjellum(a)gmail.com> > > Sent: Wednesday, October 14, 2020 7:01:54 PM > > Subject: [Mpi-forum] Question about MPI_Alloc_mem > > > Folks, I know we have had this function for a long time, and I've > implemented > > ports of MPI that actually use it (e.g., with pre-pinned memory). But, I > am > > trying to understand the logic for why baseptr is passed by value, > instead of > > by reference. In C, everything is by value, so the last argument in > normal C > > programs would be void **baseptr. > > > > The standard has: > > int MPI_Alloc_mem(MPI_Aint size, MPI_Info info, void *baseptr); > > Now, MPICC/GCC takes this with > > void *memory = (void *)0; > > int error = MPI_Alloc_mem(1024, MPI_INFO_NULL, &memory); > > and you get the memory allocated properly. > > What is more, this is incorrect: > > int error = MPI_Alloc_mem(1024, MPI_INFO_NULL, memory); > > although it compiles fine because that is, indeed, the API. > > Why would we have the pointer going in by value, when it is coming out > > as an OUT argument? > > Isn't this plain wrong? void **baseptr means ---> I am passing you the > > address of a void pointer. > > Every viable implementation must do *(void **)baseptr = ... > > when providing the "malloc'd/special" memory. So... why did we fudge > > the C API. Is there some tie-in with the Fortran API? > > Thanks in advance, > > Tony Skjellum > > > > > > -- > > Anthony Skjellum, PhD > > [ mailto:[email protected] <skjellum(a)gmail.com> | skjellum(a)gmail.com ] > > Cell: +1-205-807-4968 > > > > > > > > _______________________________________________ > > mpi-forum mailing list > > mpi-forum(a)lists.mpi-forum.org > > https://lists.mpi-forum.org/mailman/listinfo/mpi-forum > > -- > Dr. Rolf Rabenseifner . . . . . . . . . .. email rabenseifner(a)hlrs.de . > High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 . > University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 . > Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner . > Nobelstr. 19, D-70550 Stuttgart, Germany . . . . (Office: Room 1.307) . > _______________________________________________ > mpi-forum mailing list > mpi-forum(a)lists.mpi-forum.org > https://lists.mpi-forum.org/mailman/listinfo/mpi-forum > _______________________________________________ > mpi-forum mailing list > mpi-forum(a)lists.mpi-forum.org > https://lists.mpi-forum.org/mailman/listinfo/mpi-forum > > > _______________________________________________ > mpi-forum mailing list > mpi-forum(a)lists.mpi-forum.org > https://lists.mpi-forum.org/mailman/listinfo/mpi-forum > -- Jeff Hammond jeff.science(a)gmail.com http://jeffhammond.github.io/

1 0

Re: [Mpi-forum] Question about MPI_Alloc_mem
by Skjellum, Anthony 14 Oct '20

14 Oct '20

Bill, I thank you much for your input too. POSIX has no problem making you do the cast in C ?? int posix_memalign(void **memptr, size_t alignment, size_t size); I get why its that way now in MPI, I still don't like it ?? Tony cf, https://linux.die.net/man/3/posix_memalign posix_memalign(3): allocate aligned memory - Linux man page<https://linux.die.net/man/3/posix_memalign> The function posix_memalign() allocates size bytes and places the address of the allocated memory in *memptr. The address of the allocated memory will be a ... linux.die.net Anthony Skjellum, PhD Professor of Computer Science and Chair of Excellence Director, SimCenter University of Tennessee at Chattanooga (UTC) tony-skjellum(a)utc.edu [or skjellum(a)gmail.com] cell: 205-807-4968 ________________________________ From: William Gropp <wgropp(a)illinois.edu> Sent: Wednesday, October 14, 2020 3:48 PM To: Main MPI Forum mailing list <mpi-forum(a)lists.mpi-forum.org> Cc: Rolf Rabenseifner <rabenseifner(a)hlrs.de>; Skjellum, Anthony <Tony-Skjellum(a)utc.edu> Subject: Re: [Mpi-forum] Question about MPI_Alloc_mem My previous email (probably crossing paths with Tony’s reply) addressed most of this. The one thing that I want to add is that the API choice made in MPI for the C binding, both in the MPI-1 attribute get functions and here, has nothing to do with the Fortran binding. These were made to match C’s limitations on anonymous pointers, and to avoid requiring pointer casts. Bill William Gropp Director and Chief Scientist, NCSA Thomas M. Siebel Chair in Computer Science University of Illinois Urbana-Champaign On Oct 14, 2020, at 2:42 PM, Skjellum, Anthony via mpi-forum <mpi-forum(a)lists.mpi-forum.org<mailto:[email protected]>> wrote: Rolf, the rationale is not clear at all to me: "to facilitate type casting" is self-understood or a canonical term of art. I've been programming in C for 40 years... and I never would write the API as we have done it there. For pointers, out arguments are literally truthful: void **ptr is an out argument of a void pointer, and void *ptr is an in argument of a void pointer. {Years ago, before there was void *, we would have written char **ptr for a char out argument (and cast it to whatever type we wanted), and char *ptr for an in point argument. } The fact that a void ** can stand in the place of a void * is clearly a weakened typing in the language, because void * can point at anything (unknown). Since a void * is a pointer, it can also hold a pointer to a pointer to void, sure. But the intent of the second * is to remind you that you have an out argument. And you *must* pass a pointer to a pointer for the API to work in C. Somehow this is helping the Fortran and C_PTR API of Fortran, as it states. So the C API is made this way for convenience of Fortran. The rationale shows the argument with the & to pass the pointer to the API by reference. But, the rationale is not necessarily normative. A reasonable person reads the standard, sees void * and passes the baseptr without the &. Seems like a bad API to me. Tony Anthony Skjellum, PhD Professor of Computer Science and Chair of Excellence Director, SimCenter University of Tennessee at Chattanooga (UTC) tony-skjellum(a)utc.edu<mailto:[email protected]> [or skjellum(a)gmail.com<mailto:[email protected]>] cell: 205-807-4968 ________________________________ From: mpi-forum <mpi-forum-bounces(a)lists.mpi-forum.org<mailto:[email protected]>> on behalf of Rolf Rabenseifner via mpi-forum <mpi-forum(a)lists.mpi-forum.org<mailto:[email protected]>> Sent: Wednesday, October 14, 2020 1:32 PM To: Main MPI Forum mailing list <mpi-forum(a)lists.mpi-forum.org<mailto:[email protected]>> Cc: Rolf Rabenseifner <rabenseifner(a)hlrs.de<mailto:[email protected]>> Subject: Re: [Mpi-forum] Question about MPI_Alloc_mem Dear Tony, please read MPI-3.1 page 338, lines 36-41. Do these lines resolve your question? Best regards Rolf ----- Original Message ----- > From: "Main MPI Forum mailing list" <mpi-forum(a)lists.mpi-forum.org<mailto:[email protected]>> > To: "Main MPI Forum mailing list" <mpi-forum(a)lists.mpi-forum.org<mailto:[email protected]>> > Cc: "Anthony Skjellum" <skjellum(a)gmail.com<mailto:[email protected]>> > Sent: Wednesday, October 14, 2020 7:01:54 PM > Subject: [Mpi-forum] Question about MPI_Alloc_mem > Folks, I know we have had this function for a long time, and I've implemented > ports of MPI that actually use it (e.g., with pre-pinned memory). But, I am > trying to understand the logic for why baseptr is passed by value, instead of > by reference. In C, everything is by value, so the last argument in normal C > programs would be void **baseptr. > > The standard has: > int MPI_Alloc_mem(MPI_Aint size, MPI_Info info, void *baseptr); > Now, MPICC/GCC takes this with > void *memory = (void *)0; > int error = MPI_Alloc_mem(1024, MPI_INFO_NULL, &memory); > and you get the memory allocated properly. > What is more, this is incorrect: > int error = MPI_Alloc_mem(1024, MPI_INFO_NULL, memory); > although it compiles fine because that is, indeed, the API. > Why would we have the pointer going in by value, when it is coming out > as an OUT argument? > Isn't this plain wrong? void **baseptr means ---> I am passing you the > address of a void pointer. > Every viable implementation must do *(void **)baseptr = ... > when providing the "malloc'd/special" memory. So... why did we fudge > the C API. Is there some tie-in with the Fortran API? > Thanks in advance, > Tony Skjellum > > > -- > Anthony Skjellum, PhD > [ mailto:[email protected] | skjellum(a)gmail.com<mailto:[email protected]> ] > Cell: +1-205-807-4968 > > > > _______________________________________________ > mpi-forum mailing list > mpi-forum(a)lists.mpi-forum.org<mailto:[email protected]> > https://lists.mpi-forum.org/mailman/listinfo/mpi-forum -- Dr. Rolf Rabenseifner . . . . . . . . . .. email rabenseifner(a)hlrs.de<mailto:[email protected]> . High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 . University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 . Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner<http://www.hlrs.de/people/rabenseifner> . Nobelstr. 19, D-70550 Stuttgart, Germany . . . . (Office: Room 1.307) . _______________________________________________ mpi-forum mailing list mpi-forum(a)lists.mpi-forum.org<mailto:[email protected]> https://lists.mpi-forum.org/mailman/listinfo/mpi-forum _______________________________________________ mpi-forum mailing list mpi-forum(a)lists.mpi-forum.org<mailto:[email protected]> https://lists.mpi-forum.org/mailman/listinfo/mpi-forum

1 0

Re: [Mpi-forum] Question about MPI_Alloc_mem
by William Gropp 14 Oct '20

14 Oct '20

My previous email (probably crossing paths with Tony’s reply) addressed most of this. The one thing that I want to add is that the API choice made in MPI for the C binding, both in the MPI-1 attribute get functions and here, has nothing to do with the Fortran binding. These were made to match C’s limitations on anonymous pointers, and to avoid requiring pointer casts. Bill William Gropp Director and Chief Scientist, NCSA Thomas M. Siebel Chair in Computer Science University of Illinois Urbana-Champaign > On Oct 14, 2020, at 2:42 PM, Skjellum, Anthony via mpi-forum <mpi-forum(a)lists.mpi-forum.org> wrote: > > Rolf, the rationale is not clear at all to me: "to facilitate type casting" is self-understood or a canonical term of art. I've been programming in C for 40 years... and I never would write the API as we have done it there. > > For pointers, out arguments are literally truthful: void **ptr is an out argument of a void pointer, and void *ptr is an in argument of a void pointer. > > {Years ago, before there was void *, we would have written char **ptr for a char out argument (and cast it to whatever type we wanted), and char *ptr for an in point argument. } > > The fact that a void ** can stand in the place of a void * is clearly a weakened typing in the language, because void * can point at anything (unknown). Since a void * is a pointer, it can also hold a pointer to a pointer to void, sure. But the intent of the second * is to remind you that you have an out argument. And you *must* pass a pointer to a pointer for the API to work in C. > > Somehow this is helping the Fortran and C_PTR API of Fortran, as it states. So the C API is made this way for convenience of Fortran. > > The rationale shows the argument with the & to pass the pointer to the API by reference. But, the rationale is not necessarily normative. A reasonable person reads the standard, sees void * and passes the baseptr without the &. > > Seems like a bad API to me. > > Tony > > > Anthony Skjellum, PhD > Professor of Computer Science and Chair of Excellence > Director, SimCenter > University of Tennessee at Chattanooga (UTC) > tony-skjellum(a)utc.edu <mailto:[email protected]> [or skjellum(a)gmail.com <mailto:[email protected]>] > cell: 205-807-4968 > > From: mpi-forum <mpi-forum-bounces(a)lists.mpi-forum.org <mailto:[email protected]>> on behalf of Rolf Rabenseifner via mpi-forum <mpi-forum(a)lists.mpi-forum.org <mailto:[email protected]>> > Sent: Wednesday, October 14, 2020 1:32 PM > To: Main MPI Forum mailing list <mpi-forum(a)lists.mpi-forum.org <mailto:[email protected]>> > Cc: Rolf Rabenseifner <rabenseifner(a)hlrs.de <mailto:[email protected]>> > Subject: Re: [Mpi-forum] Question about MPI_Alloc_mem > > Dear Tony, > > please read MPI-3.1 page 338, lines 36-41. > Do these lines resolve your question? > > Best regards > Rolf > > ----- Original Message ----- > > From: "Main MPI Forum mailing list" <mpi-forum(a)lists.mpi-forum.org <mailto:[email protected]>> > > To: "Main MPI Forum mailing list" <mpi-forum(a)lists.mpi-forum.org <mailto:[email protected]>> > > Cc: "Anthony Skjellum" <skjellum(a)gmail.com <mailto:[email protected]>> > > Sent: Wednesday, October 14, 2020 7:01:54 PM > > Subject: [Mpi-forum] Question about MPI_Alloc_mem > > > Folks, I know we have had this function for a long time, and I've implemented > > ports of MPI that actually use it (e.g., with pre-pinned memory). But, I am > > trying to understand the logic for why baseptr is passed by value, instead of > > by reference. In C, everything is by value, so the last argument in normal C > > programs would be void **baseptr. > > > > The standard has: > > int MPI_Alloc_mem(MPI_Aint size, MPI_Info info, void *baseptr); > > Now, MPICC/GCC takes this with > > void *memory = (void *)0; > > int error = MPI_Alloc_mem(1024, MPI_INFO_NULL, &memory); > > and you get the memory allocated properly. > > What is more, this is incorrect: > > int error = MPI_Alloc_mem(1024, MPI_INFO_NULL, memory); > > although it compiles fine because that is, indeed, the API. > > Why would we have the pointer going in by value, when it is coming out > > as an OUT argument? > > Isn't this plain wrong? void **baseptr means ---> I am passing you the > > address of a void pointer. > > Every viable implementation must do *(void **)baseptr = ... > > when providing the "malloc'd/special" memory. So... why did we fudge > > the C API. Is there some tie-in with the Fortran API? > > Thanks in advance, > > Tony Skjellum > > > > > > -- > > Anthony Skjellum, PhD > > [ mailto:[email protected] <mailto:[email protected]> | skjellum(a)gmail.com <mailto:[email protected]> ] > > Cell: +1-205-807-4968 > > > > > > > > _______________________________________________ > > mpi-forum mailing list > > mpi-forum(a)lists.mpi-forum.org <mailto:[email protected]> > > https://lists.mpi-forum.org/mailman/listinfo/mpi-forum <https://lists.mpi-forum.org/mailman/listinfo/mpi-forum> > > -- > Dr. Rolf Rabenseifner . . . . . . . . . .. email rabenseifner(a)hlrs.de <mailto:[email protected]> . > High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 . > University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 . > Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner <http://www.hlrs.de/people/rabenseifner> . > Nobelstr. 19, D-70550 Stuttgart, Germany . . . . (Office: Room 1.307) . > _______________________________________________ > mpi-forum mailing list > mpi-forum(a)lists.mpi-forum.org <mailto:[email protected]> > https://lists.mpi-forum.org/mailman/listinfo/mpi-forum <https://lists.mpi-forum.org/mailman/listinfo/mpi-forum> > _______________________________________________ > mpi-forum mailing list > mpi-forum(a)lists.mpi-forum.org <mailto:[email protected]> > https://lists.mpi-forum.org/mailman/listinfo/mpi-forum <https://lists.mpi-forum.org/mailman/listinfo/mpi-forum>

1 0

Re: [Mpi-forum] Question about MPI_Alloc_mem
by Skjellum, Anthony 14 Oct '20

14 Oct '20

Rolf, the rationale is not clear at all to me: "to facilitate type casting" is self-understood or a canonical term of art. I've been programming in C for 40 years... and I never would write the API as we have done it there. For pointers, out arguments are literally truthful: void **ptr is an out argument of a void pointer, and void *ptr is an in argument of a void pointer. {Years ago, before there was void *, we would have written char **ptr for a char out argument (and cast it to whatever type we wanted), and char *ptr for an in point argument. } The fact that a void ** can stand in the place of a void * is clearly a weakened typing in the language, because void * can point at anything (unknown). Since a void * is a pointer, it can also hold a pointer to a pointer to void, sure. But the intent of the second * is to remind you that you have an out argument. And you *must* pass a pointer to a pointer for the API to work in C. Somehow this is helping the Fortran and C_PTR API of Fortran, as it states. So the C API is made this way for convenience of Fortran. The rationale shows the argument with the & to pass the pointer to the API by reference. But, the rationale is not necessarily normative. A reasonable person reads the standard, sees void * and passes the baseptr without the &. Seems like a bad API to me. Tony Anthony Skjellum, PhD Professor of Computer Science and Chair of Excellence Director, SimCenter University of Tennessee at Chattanooga (UTC) tony-skjellum(a)utc.edu [or skjellum(a)gmail.com] cell: 205-807-4968 ________________________________ From: mpi-forum <mpi-forum-bounces(a)lists.mpi-forum.org> on behalf of Rolf Rabenseifner via mpi-forum <mpi-forum(a)lists.mpi-forum.org> Sent: Wednesday, October 14, 2020 1:32 PM To: Main MPI Forum mailing list <mpi-forum(a)lists.mpi-forum.org> Cc: Rolf Rabenseifner <rabenseifner(a)hlrs.de> Subject: Re: [Mpi-forum] Question about MPI_Alloc_mem Dear Tony, please read MPI-3.1 page 338, lines 36-41. Do these lines resolve your question? Best regards Rolf ----- Original Message ----- > From: "Main MPI Forum mailing list" <mpi-forum(a)lists.mpi-forum.org> > To: "Main MPI Forum mailing list" <mpi-forum(a)lists.mpi-forum.org> > Cc: "Anthony Skjellum" <skjellum(a)gmail.com> > Sent: Wednesday, October 14, 2020 7:01:54 PM > Subject: [Mpi-forum] Question about MPI_Alloc_mem > Folks, I know we have had this function for a long time, and I've implemented > ports of MPI that actually use it (e.g., with pre-pinned memory). But, I am > trying to understand the logic for why baseptr is passed by value, instead of > by reference. In C, everything is by value, so the last argument in normal C > programs would be void **baseptr. > > The standard has: > int MPI_Alloc_mem(MPI_Aint size, MPI_Info info, void *baseptr); > Now, MPICC/GCC takes this with > void *memory = (void *)0; > int error = MPI_Alloc_mem(1024, MPI_INFO_NULL, &memory); > and you get the memory allocated properly. > What is more, this is incorrect: > int error = MPI_Alloc_mem(1024, MPI_INFO_NULL, memory); > although it compiles fine because that is, indeed, the API. > Why would we have the pointer going in by value, when it is coming out > as an OUT argument? > Isn't this plain wrong? void **baseptr means ---> I am passing you the > address of a void pointer. > Every viable implementation must do *(void **)baseptr = ... > when providing the "malloc'd/special" memory. So... why did we fudge > the C API. Is there some tie-in with the Fortran API? > Thanks in advance, > Tony Skjellum > > > -- > Anthony Skjellum, PhD > [ mailto:[email protected] | skjellum(a)gmail.com ] > Cell: +1-205-807-4968 > > > > _______________________________________________ > mpi-forum mailing list > mpi-forum(a)lists.mpi-forum.org > https://lists.mpi-forum.org/mailman/listinfo/mpi-forum -- Dr. Rolf Rabenseifner . . . . . . . . . .. email rabenseifner(a)hlrs.de . High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 . University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 . Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner<http://www.hlrs.de/people/rabenseifner> . Nobelstr. 19, D-70550 Stuttgart, Germany . . . . (Office: Room 1.307) . _______________________________________________ mpi-forum mailing list mpi-forum(a)lists.mpi-forum.org https://lists.mpi-forum.org/mailman/listinfo/mpi-forum

1 0

Re: [Mpi-forum] [EXT]: Progress Question
by Jim Dinan 14 Oct '20

14 Oct '20

The question does essentially boil down to whether a full fan-out of nonblocking send/recv pairs followed by wait-all is a valid implementation of MPI_Barrier. Reviewing the text that Dan cited for MPI 4.0: > (§5.14, p234 MPI-2019-draft): “A correct, portable program must invoke collective communications so that deadlock will not occur” There isn't any convenient way the user can find out about remote completion of a barrier (short of building their own barrier with synchronous send). So, we can either interpret the above statement to place a strong completion requirement on collectives (bad for performance). Or, we can interpret it to mean that there's really no safe time when a user can call into a blocking external interface. The RMA progress passage that Martin referenced seems to support this latter interpretation with the sockets example given in the rationale. ~Jim. On Mon, Oct 12, 2020 at 11:17 AM HOLMES Daniel <d.holmes(a)epcc.ed.ac.uk> wrote: > Hi Jim, et al, > > Unless the point-to-point pseudo-code given is proven to be a valid > implementation of MPI Barrier, then reasoning about MPI Barrier using it as > a basis is unlikely to be edifying. > I also have a (possibly flawed) implementation of MPI Barrier that > exhibits some odd semantics/behaviours and I could use that to assert > (likely incorrectly) that MPI Barrier is defined in a way that exhibits > those semantics/behaviours. However, that serves no purpose, so I won’t > dwell on it any further. > > I’m glad that someone responded with a reference to the MPI Standard, > thanks Martin. In that vein, here’s my tuppence: > > The definition of MPI Barrier in the MPI Standard states (§5.3, p149 in > MPI-2019-draft): > “If comm is an intracommunicator, MPI_BARRIER blocks the caller until all > group members have called it. The call returns at any process only after > all group members have entered the call.” > > There is a happens-before between “all MPI processes have entered the MPI > Barrier call” and “MPI processes are permitted to leave the call”. That’s > it; that’s all MPI Barrier does/is required to ensure. > > There is no indication or requirement for alacrity. This appears to be a > valid (although stupid) implementation: > int MPI_Barrier(MPI_comm comm) { > int ret = PMPI_Barrier(comm); > sleep(100days); > return ret; > } > > There is no indication or suggestion for how or when MPI processes become > aware that the necessary pre-condition for returning control to the user > has been satisfied. Some may become aware of this situation a significant > amount of time before/after others. Local completion does not guarantee > remote completion in MPI (except for passive-target RMA, e.g. > MPI_Win_unlock). > > There is no indication or requirement that the necessary pre-condition is > also a sufficient pre-condition, although we may wish to assume that and we > may wish to clarify the wording of the MPI Standard to specify that > explicitly. If the MPI Standard text were changed to “The call returns at > any process <strike>only</strike>immediately after all group members have > entered the call.” then (given the other usage of immediately in the MPI > Standard) we could assume that the procedure becomes strong local > (immediate) once the necessary pre-condition is met. Without the word > “immediate” in the sentence, the return of the MPI procedure is permitted > to require remote progress, i.e. after the necessary pre-condition is met, > it becomes weak local (called local in the MPI Standard). Some MPI > libraries (can, if configured in a particular way) provide strong progress; > however, MPI only requires weak progress. Weak progress means it is > permitted for remote progress to happen only during remote MPI procedure > calls. > > So, > > If MPI required “returns immediately after...” (which it does not) then > every MPI process would be required to ensure the remote completion of its > “send” (as well as local completion of the “recv”) before it returns > control to the user. This would mean that our intuitive feel for what > MPI_Barrier should do would be correct and the suggested point-to-point > code would be an incorrect implementation of MPI_Barrier. > If MPI required strong progress (which it does not) then every MPI process > would eventually become aware that it is permitted to return control to the > user, without additional remote MPI procedure calls. This would mean that > our intuitive feel for what MPI_Barrier should do would be correct and the > suggested point-to-point code would be a correct implementation of > MPI_Barrier. > > As it is, our intuitive feel for what MPI_Barrier should do is probably > wrong (i.e. not what MPI actually specifies), or at least too optimistic > because it depends on a high quality implementation that exceeds what is > minimally specified by the MPI Standard as required. > As it is, the MPI_Barrier in the original question does not guard against > problems with the non-MPI file operations - indeed, adding it introduces a > new possibility of a deadlock, which would not be present in the code > without the MPI_Barrier operation. > > I would argue that the original code is therefore erroneous > (incorrect/non-portable) because (§5.14, p234 MPI-2019-draft): > “A correct, portable program must invoke collective communications so that > deadlock will not occur” > > One correct program that achieves what the original looks like it might be > trying to achieve (IHMO) is as follows: > if (rank == 1) > create_file("test”); > MPI_Barrier(); > if (rank == 0) > while not_exists("test") > sleep(1); > This program still assumes that the file creation actually creates the > file and flushes it to a filesystem that makes it visible to the existence > check but that must be true if the code-without-MPI is correct, i.e. adding > MPI has not introduced a new problem to the code. > > Taking this reasoning about the minimal requirements of MPI Barrier (at > least) one step too far, the only restriction on implementation of > MPI_Barrier seems to be “do not return until <something happens>”, which > suggests this is a valid (although very unhelpful) implementation: > int MPI_Barrier(MPI_comm comm) { > while (1); // do not return, ever > } > > To guard against low-quality/malicious implementations of the MPI > Standard, we could either clarify the wording of the text about MPI_Barrier > (and probably the text about every other MPI procedure) to include the > concept of becoming an “immediate” procedure once certain criteria are met > (likely to be a lot of effort/angst for some), or mandate strong progress > for all MPI libraries (likely to be very unpopular for some). > > Cheers, > Dan. > — > Dr Daniel Holmes PhD > Architect (HPC Research) > d.holmes(a)epcc.ed.ac.uk > Phone: +44 (0) 131 651 3465 > Mobile: +44 (0) 7940 524 088 > Address: Room 2.09, Bayes Centre, 47 Potterrow, Central Area, Edinburgh, > EH8 9BT > — > The University of Edinburgh is a charitable body, registered in Scotland, > with registration number SC005336. > — > > On 12 Oct 2020, at 10:04, Martin Schulz via mpi-forum < > mpi-forum(a)lists.mpi-forum.org> wrote: > > Hi Jim, all, > > We had a similar discussion (in a smaller circle) during the terms > discussions – at least to my understanding, all bets are off as soon as you > add dependencies and wait conditions outside of MPI, like here with the > file. A note to this point is in a rational (Section 11.7, page 491 in the > 2019 draft) – based on that an MPI implementation is allowed to deadlock > (or cause a deadlock) – if all dependencies would be in MPI calls, then > “eventual” progress should be guaranteed – even if it is after the 100 days > in Rajeev’s example: that would – as far as I understand – still be correct > behavior, as no MPI call is guaranteed to return in a fixed finite time > (all calls are at best “weak local”). > > Martin > > > > -- > Prof. Dr. Martin Schulz, Chair of Computer Architecture and Parallel > Systems > Department of Informatics, TU-Munich, Boltzmannstraße 3, D-85748 Garching > Member of the Board of Directors at the Leibniz Supercomputing Centre (LRZ) > Email: schulzm(a)in.tum.de > > > > *From: *mpi-forum <mpi-forum-bounces(a)lists.mpi-forum.org> on behalf of > Jim Dinan via mpi-forum <mpi-forum(a)lists.mpi-forum.org> > *Reply-To: *Main MPI Forum mailing list <mpi-forum(a)lists.mpi-forum.org> > *Date: *Sunday, 11. October 2020 at 23:41 > *To: *"Skjellum, Anthony" <Tony-Skjellum(a)utc.edu> > *Cc: *Jim Dinan <james.dinan(a)gmail.com>, Main MPI Forum mailing list < > mpi-forum(a)lists.mpi-forum.org> > *Subject: *Re: [Mpi-forum] [EXT]: Progress Question > > You can have a situation where the isend/irecv pair completes at process 0 > before process 1 has called irecv or waitall. Since process 0 is now busy > waiting on the file, it will not make progress on MPI calls and can result > in deadlock. > > ~Jim. > > On Sat, Oct 10, 2020 at 2:17 PM Skjellum, Anthony <Tony-Skjellum(a)utc.edu> > wrote: > > Jim, OK, my attempt at answering below. > > See if you agree with my annotations. > > -Tony > > > Anthony Skjellum, PhD > Professor of Computer Science and Chair of Excellence > Director, SimCenter > University of Tennessee at Chattanooga (UTC) > tony-skjellum(a)utc.edu [or skjellum(a)gmail.com] > cell: 205-807-4968 > > > ------------------------------ > *From:* mpi-forum <mpi-forum-bounces(a)lists.mpi-forum.org> on behalf of > Jim Dinan via mpi-forum <mpi-forum(a)lists.mpi-forum.org> > *Sent:* Saturday, October 10, 2020 1:31 PM > *To:* Main MPI Forum mailing list <mpi-forum(a)lists.mpi-forum.org> > *Cc:* Jim Dinan <james.dinan(a)gmail.com> > *Subject:* [EXT]: [Mpi-forum] Progress Question > > *External Email* > Hi All, > > A colleague recently asked a question that I wasn't able to answer > definitively. Is the following code guaranteed to make progress? > > > MPI_Barrier(); > -- everything is uncertain to within one message, if layered on pt2pt; > --- let's assume a power of 2, and recursive doubling (RD). > --- At each stage, it posts an irecv and isend to its corresponding > element in RD > --- All stages must complete to get to the last stage. > --- At the last stage, it appears like your example below for N/2 > independent process pairs, which appears always to complete. > Oif rank == 1 > create_file("test") > if rank == 0 > while not_exists("test") > sleep(1); > > > That is, can rank 1 require rank 0 to make MPI calls after its return from > the barrier, in order for rank 1 to complete the barrier? If the code were > written as follows: > > > isend(..., other_rank, &req[0]) > irecv(..., other_rank, &req[1]) > waitall(2, req) > --- Assume both isends buffer on the send-side and return > immediately--valid. > --- Both irecvs are posted, but unmatched as yet. Nothing has transferred > on network. > --- Waitall would mark the isends done at once, and work to complete the > irecvs; in > that process, each would have to progress the isends across the > network. On this comm > and all comms, incidentally. > --- When waitall returns, the data has transferred to the receiver, > otherwise the irecvs > aren't done. > if rank == 1 > create_file("test") > if rank == 0 > while not_exists("test") > sleep(1); > > > I think it would clearly not guarantee progress since the send data can be > buffered. Is the same true for barrier? > > Cheers, > ~Jim. > *This message is not from a UTC.EDU <http://utc.edu/> address. Caution > should be used in clicking links and downloading attachments from unknown > senders or unexpected email. * > > > _______________________________________________ > mpi-forum mailing list > mpi-forum(a)lists.mpi-forum.org > https://lists.mpi-forum.org/mailman/listinfo/mpi-forum > > >

1 0

Re: [Mpi-forum] Question about MPI_Alloc_mem
by William Gropp 14 Oct '20

14 Oct '20

To be more specific as to the reason, this is the C idiom - it you pass it as a pointer to a pointer, you have to cast it as a void pointer to a pointer. If it is just a void *, C views this as an anonymous pointer, and you can use the address - as in int *intp; MPI_Alloc_mem(1000, MPI_INFO_NULL, &intp); If baseptr is a void**, you need to do this: MPI_Alloc_mem(1000, MPI_INFO_NULL, (void **)&intp); I don’t like it either, but this is the choice in C. Bill William Gropp Director and Chief Scientist, NCSA Thomas M. Siebel Chair in Computer Science University of Illinois Urbana-Champaign > On Oct 14, 2020, at 12:01 PM, Anthony Skjellum via mpi-forum <mpi-forum(a)lists.mpi-forum.org> wrote: > > Folks, I know we have had this function for a long time, and I've implemented ports of MPI that actually use it (e.g., with pre-pinned memory). But, I am trying to understand the logic for why baseptr is passed by value, instead of by reference. In C, everything is by value, so the last argument in normal C programs would be void **baseptr. > > The standard has: > int MPI_Alloc_mem(MPI_Aint size, MPI_Info info, void *baseptr); > Now, MPICC/GCC takes this with > void *memory = (void *)0; > int error = MPI_Alloc_mem(1024, MPI_INFO_NULL, &memory); > and you get the memory allocated properly. > What is more, this is incorrect: > int error = MPI_Alloc_mem(1024, MPI_INFO_NULL, memory); > although it compiles fine because that is, indeed, the API. > Why would we have the pointer going in by value, when it is coming out > as an OUT argument? > Isn't this plain wrong? void **baseptr means ---> I am passing you the > address of a void pointer. > Every viable implementation must do *(void **)baseptr = ... > when providing the "malloc'd/special" memory. So... why did we fudge > the C API. Is there some tie-in with the Fortran API? > Thanks in advance, > Tony Skjellum > > > -- > Anthony Skjellum, PhD > skjellum(a)gmail.com <mailto:[email protected]> > Cell: +1-205-807-4968 > > > _______________________________________________ > mpi-forum mailing list > mpi-forum(a)lists.mpi-forum.org > https://lists.mpi-forum.org/mailman/listinfo/mpi-forum

1 0