January 2020 - mpiwg-p2p - mailman3.cels.anl.gov

Re: [mpiwg-p2p] Question on MPI_PACKED
by Gropp, William D 26 Jan '20

26 Jan '20

Yes, you can send packed data and then receive it with a datatype. Example 4.21 (all references to MPI 3.1) does exactly that. An implementation may include metadata in the packed buffer; that’s the reason for MPI_PACK_SIZE and the constraint that you can’t concatenate pack buffers. This does mean that an implementation that includes metadata must handle this case; it can do so by noting that the data was sent with MPI_PACK. There isn’t a truncation problem because the metadata is not delivered; it would only be used to handle and necessary data transformations (e.g., big to little endian). Bill William Gropp Director and Chief Scientist, NCSA Thomas M. Siebel Chair in Computer Science University of Illinois Urbana-Champaign On Jan 24, 2020, at 4:16 AM, Joachim Protze via mpiwg-p2p <mpiwg-p2p(a)lists.mpi-forum.org<mailto:[email protected]>> wrote: Hi all, the exception on MPI_PACKED in the type matching rules (3.3.1) suggests, that one side of the communication can pack/unpack the message, while the other side reads/writes natively. I.e., is the following code valid MPI code? #include <mpi.h> #include <stdio.h> #include <stdlib.h> int main() { int A[100], B[10], psize; for (int i = 0; i < 100; i++) A[i] = i; MPI_Init(NULL, NULL); MPI_Pack_size(1, MPI_INT, MPI_COMM_SELF, &psize); void *parr = malloc(psize * 10); int pos = 0; for (int i = 0; i < 10; i++) MPI_Pack(&(A[i * 10]), 1, MPI_INT, parr, psize * 10, &pos, MPI_COMM_SELF); // Send the packed data and receive a contiguous buffer MPI_Sendrecv(parr, pos, MPI_PACKED, 0, 42, B, 10, MPI_INT, 0, 42, MPI_COMM_SELF, MPI_STATUS_IGNORE); for (int i = 0; i < 10; i++) printf("%i\n", B[i]); MPI_Finalize(); } Can the packed buffer include meta data? This would be reflected by a larger value in /pos/, right? Can this lead to unexpected message truncation? Thanks Joachim -- Dipl.-Inf. Joachim Protze IT Center Group: High Performance Computing Division: Computational Science and Engineering RWTH Aachen University Seffenter Weg 23 D 52074 Aachen (Germany) Tel: +49 241 80- 24765 Fax: +49 241 80-624765 protze(a)itc.rwth-aachen.de<mailto:[email protected]> www.itc.rwth-aachen.de _______________________________________________ mpiwg-p2p mailing list mpiwg-p2p(a)lists.mpi-forum.org https://lists.mpi-forum.org/mailman/listinfo/mpiwg-p2p

1 0

Re: [mpiwg-p2p] Review of Pythonization changes for Chapter 3
by Rolf Rabenseifner 25 Jan '20

25 Jan '20

Dear Dan and Bill, Yes, Bill, and more than that: Fortran INTENT(OUT) means that the Compiler may through the Input away, i.e., if a field should not be changed, e.g. because there is no such Output of Status (e.g. WAIT for a send request), then the status my be changed into some undefined values. Therefore, we ware very careful for the mpi_f08 module never to add a risky INTENT(OUT). Best regards Rolf ----- Original Message ----- > From: "MPI WG Point To Point Communications working group" <mpiwg-p2p(a)lists.mpi-forum.org> > To: "MPI WG Point To Point Communications working group" <mpiwg-p2p(a)lists.mpi-forum.org> > Cc: "wgropp" <wgropp(a)illinois.edu> > Sent: Saturday, January 25, 2020 6:04:12 PM > Subject: Re: [mpiwg-p2p] Review of Pythonization changes for Chapter 3 > I believe this choice is because of MPI_STATUS_IGNORE - there is, in fact, both > an IN value (don’t fill in the status) and an OUT value (place the status value > here). We do note in the intro that the MPI IN and OUT do not exactly match > Fortran, for these reasons. > > Bill > > William Gropp > Director and Chief Scientist, NCSA > Thomas M. Siebel Chair in Computer Science > University of Illinois Urbana-Champaign > > > On Jan 22, 2020, at 5:07 AM, HOLMES Daniel via mpiwg-p2p < [ > mailto:[email protected] | mpiwg-p2p(a)lists.mpi-forum.org ] > wrote: > > 3) The F08 bindings for all variants of the MPI_TEST, MPI_WAIT, and MPI_PROBE > functions and the MPI_REQUEST_GET_STATUS function do not specify an INTENT for > the status(es) parameter. We could fix this by including “INTENT(OUT)” in all > cases (by choosing the correct Python setting). > > > > _______________________________________________ > mpiwg-p2p mailing list > mpiwg-p2p(a)lists.mpi-forum.org > https://lists.mpi-forum.org/mailman/listinfo/mpiwg-p2p -- Dr. Rolf Rabenseifner . . . . . . . . . .. email rabenseifner(a)hlrs.de . High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 . University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 . Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner . Nobelstr. 19, D-70550 Stuttgart, Germany . . . . (Office: Room 1.307) .

1 0

Re: [mpiwg-p2p] Review of Pythonization changes for Chapter 3
by Gropp, William D 25 Jan '20

25 Jan '20

I believe this choice is because of MPI_STATUS_IGNORE - there is, in fact, both an IN value (don’t fill in the status) and an OUT value (place the status value here). We do note in the intro that the MPI IN and OUT do not exactly match Fortran, for these reasons. Bill William Gropp Director and Chief Scientist, NCSA Thomas M. Siebel Chair in Computer Science University of Illinois Urbana-Champaign On Jan 22, 2020, at 5:07 AM, HOLMES Daniel via mpiwg-p2p <mpiwg-p2p(a)lists.mpi-forum.org<mailto:[email protected]>> wrote: 3) The F08 bindings for all variants of the MPI_TEST, MPI_WAIT, and MPI_PROBE functions and the MPI_REQUEST_GET_STATUS function do not specify an INTENT for the status(es) parameter. We could fix this by including “INTENT(OUT)” in all cases (by choosing the correct Python setting).

1 0

Question on MPI_PACKED
by Joachim Protze 24 Jan '20

24 Jan '20

Hi all, the exception on MPI_PACKED in the type matching rules (3.3.1) suggests, that one side of the communication can pack/unpack the message, while the other side reads/writes natively. I.e., is the following code valid MPI code? #include <mpi.h> #include <stdio.h> #include <stdlib.h> int main() { int A[100], B[10], psize; for (int i = 0; i < 100; i++) A[i] = i; MPI_Init(NULL, NULL); MPI_Pack_size(1, MPI_INT, MPI_COMM_SELF, &psize); void *parr = malloc(psize * 10); int pos = 0; for (int i = 0; i < 10; i++) MPI_Pack(&(A[i * 10]), 1, MPI_INT, parr, psize * 10, &pos, MPI_COMM_SELF); // Send the packed data and receive a contiguous buffer MPI_Sendrecv(parr, pos, MPI_PACKED, 0, 42, B, 10, MPI_INT, 0, 42, MPI_COMM_SELF, MPI_STATUS_IGNORE); for (int i = 0; i < 10; i++) printf("%i\n", B[i]); MPI_Finalize(); } Can the packed buffer include meta data? This would be reflected by a larger value in /pos/, right? Can this lead to unexpected message truncation? Thanks Joachim -- Dipl.-Inf. Joachim Protze IT Center Group: High Performance Computing Division: Computational Science and Engineering RWTH Aachen University Seffenter Weg 23 D 52074 Aachen (Germany) Tel: +49 241 80- 24765 Fax: +49 241 80-624765 protze(a)itc.rwth-aachen.de www.itc.rwth-aachen.de

1 0

Review of Pythonization changes for Chapter 3
by HOLMES Daniel 22 Jan '20

22 Jan '20

Hi all, I have done a pass through Chapter 3 looking at the Pythonization changes (a little late). I have found only one issue with the changes being made/reviewed. 0) In MPI_WAITANY and MPI_TESTANY the index parameter is described as "(integer)" in both versions but should be "(non-negative integer)" in the new version. This makes it consistent with tag, source, etc. It is non-negative or MPI_UNDEFINED but never negative. I reported this to the BigCount WG already but please indicate (asap) if you disagree with this alteration. I have marked the Google spreadsheet with an ‘X’ to indicate that our review of this chapter for Pythonization is complete. Let me know ASAP if you have Pythonization-related issues. — I have also found a few things that we should deal with as separate errata. 1) The LIS binding for MPI_TESTANY includes a mention of MPI_UNDEFINED in its description of the index parameter but MPI_WAITANY does not. We could fix that inconsistency either by removing the mention from the MPI_TESTANY binding or by adding it to the MPI_WAITANY binding. The normative text describes the use of MPI_UNDEFINED in both cases, so we would only be changing one of the LIS bindings. 2) The MPI_WAITALL and MPI_TESTALL bindings for F08 give the exact size for the array_of_requests but not for the array_of_statuses, even though they must be identically sized. We could fix this by changing "(*)" to "(count)” in the rendered document (by including the size of the array in the Python). 3) The F08 bindings for all variants of the MPI_TEST, MPI_WAIT, and MPI_PROBE functions and the MPI_REQUEST_GET_STATUS function do not specify an INTENT for the status(es) parameter. We could fix this by including “INTENT(OUT)” in all cases (by choosing the correct Python setting). 4) The LIS binding for MPI_CANCEL lists the request parameter as IN but the function “marks the request for cancellation”, i.e. could modify it. This intent is implied by the C binding (the absence of const). Compare with MPI_START as well. We could fix this by changing “IN” to “INOUT” in the LIS binding and changing the F08 binding from “INTENT(IN)” to “INTENT(INOUT)”. I’d like to discuss these errata over email this week - if we can decide to proceed on any of them soon enough, we can create errata tickets in time for the next face-to-face, i.e. in time to be included in the MPI-4.0 document. Cheers, Dan. — Dr Daniel Holmes PhD Architect (HPC Research) d.holmes(a)epcc.ed.ac.uk<mailto:[email protected]> Phone: +44 (0) 131 651 3465 Mobile: +44 (0) 7940 524 088 Address: Room 2.09, Bayes Centre, 47 Potterrow, Central Area, Edinburgh, EH8 9BT — The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. —

1 0