- mpi-comments - mailman3.cels.anl.gov

Re: [Mpi-comments] Comments on MPI 3.0 draft 2 [part 5]
by Rajeev Thakur 03 Sep '12

03 Sep '12

I made only obvious ticket 0 changes. Anything else will have to be looked at by a group for the next version. > Here are my responses to those, skipping the ones that I don't have any comments on: > >>> Page 495 lines 11-12: Are there any guarantees about the behavior of >>> MPI_MODE_DELETE_ON_CLOSE if the program exits (normally or abnormally) >>> without explicitly closing the file? >> No > > Is that stated explicitly anywhere? I don't know. For next version. > >>> Page 499 lines 44-45: Wouldn't using log on the bit to test and then >>> mod and integer division on the amode be simpler than the loop? >>> Floating-point errors might cause trouble there, however, or if one >>> of the masks was not a single bit. >> Maybe, but it is more than a ticket 0 change > > Even for an example? If it is too big of a change, please put it on the > list for the next version. It's more than a typo fix and may introduce a bug of its own unless someone else verifies it. For next version. > >>> Page 503 lines 22-23: Is this hint intended to say how many devices >>> to use, not how many are in the system? >> No, it is correct. The next hint "striping_factor" says how many >> devices to use. > > What is the point of putting in a hint about the system configuration > (which the MPI implementation should be able to get on its own)? I don't remember the history behind it (1996). >>> Page 506 lines 30-33: Is there an easy way to tell that a datatype is >>> derived? Is the datatype returned, if derived, guaranteed to be new >>> (and not, for example, one passed into MPI_FILE_SET_VIEW by the >>> user)? >> Yes, using MPI_Type_get_envelope. If derived, the text says, the >> function returns a datatype that the user is responsible for freeing. >> It doesn't talk in terms of new or not. > > That seems like it needs advice to users (I didn't know before your > comment how to determine whether a datatype is derived, and the test > might be subtle for people who are not experts in datatype handling in > MPI). For next version. > >>> Page 508 lines 9-11: Does that mean a buffer being written to a file >>> can't be concurrently sent? It seems like that restriction may be >>> too strong, especially when you can use a datatype to write the same >>> data to a file more than once in a single operation. >> It would be more than a ticket 0 change. > > OK, but it should probably be on the list to look at for the next > version. Yes, particularly since the restriction has been removed for nonblocking sends. >>> Page 536 lines 34-35: Unicode has several encoding formats; which one >>> are you intending (UTF-7, UTF-8, UTF-16, UCS-16, UCS-32, or something >>> else)? >> Don't know. > > It would be a problem for interoperability of external32 to not have > a normative answer to that. Also, different Unicode formats change > which code points can be represented (many UCS-2 implementations, for > example, cannot encode code points greather than 0xFFFF, for example; > see http://www.unicode.org/faq/basic_q.html#14) Next version. >>> Page 537 lines 9-12: Some of this information (what happens on >>> overflows from down-converting number formats, for example) is >>> normative and should be in the main text. Also, taking the >>> least-significant bytes of a larger number does not always preserve >>> the sign bit in case of overflows (converting the MPI_INT 32768 to an >>> MPI_INTEGER2 leads to -32768, for example). Is making sure to >>> preserve the sign bit always the intended behavior? >> Not changed. > > Why? Next version. >>> Page 538 line 9: Defining wchar as two bytes prevents it from >>> representing all valid Unicode code points (some of them are above >>> 2^16). If such code points are forbidden (even on platforms where >>> wchar_t is 32 bits), be sure to say that explicitly. >> Not changed. > > Why? Again, more than a typo fix. >>> Does the MPI standard say anywhere that a byte is assumed to be 8 bits? >> Don't know. > > It turns out that the answer is "yes" (page 25 line 28). Glad! >>> Page 541 lines 47-48: Does that mean that point-to-point operations >>> are OK? Non-collective file I/O? >> Sounds like it. > > That seems odd -- I would expect that I/O callbacks would be completely > local and not do I/O themselves. That might be worth considering for > the next version. Yes. >>> Page 542 lines 17-25: The first item in this list appears to be a >>> superset of the third one, and it seems like all of them might be >>> required to be satisfied when applicable (not just "at least one"). >> The first one refers to a program using only basic datatypes. The 3rd >> one refers to a program using derived datatypes. > > The third bullet point there is still about "predefined datatypes", or > does "data item" refer to an element of a derived datatype? The data item could be part of a derived datatype. This bullet doesn't restrict the use to basic datatypes as item 1 does. >>> Page 543 lines 4-5: How do you get multiple file handles out of the >>> same collective open of a file? Accesses from different processes >>> only? Or can file descriptors be sent between processes? >> Multiple means the return values on different processes. > > Does that mean that file handles can be sent between processes? If not, > why would the behavior of multiple processes accessing the same file > change based on whether the collective open operation returned different > file handles to the different processes rather than returning the same > one everywhere? The first case describes the (obvious) behavior when there is a total of one process opening one file. The second case is when two processes call a collective open (using COMM_WORLD, for example) to open the same file. The third case is when two processes independently open the same file (using COMM_SELF, for example). > >>> Page 547 line 10: Is MPI_Offset required to be signed? >> Yes. you can seek backwards from the current location. > > That's what I would expect, but that might be worthwhile to say > ("MPI_Offset is a <ins>signed</ins> integer type ..."). Maybe, but even MPI_Aint doesn't say it should be signed. (In any case, I am not making that change on my own.) Rajeev

1 0

Re: [Mpi-comments] Comments on MPI 3.0 draft 2 [part 5]
by Jeremiah Willcock 03 Sep '12

03 Sep '12

Here are my responses to those, skipping the ones that I don't have any comments on: >> Page 495 lines 11-12: Are there any guarantees about the behavior of >> MPI_MODE_DELETE_ON_CLOSE if the program exits (normally or abnormally) >> without explicitly closing the file? > No Is that stated explicitly anywhere? >> Page 499 lines 44-45: Wouldn't using log on the bit to test and then >> mod and integer division on the amode be simpler than the loop? >> Floating-point errors might cause trouble there, however, or if one >> of the masks was not a single bit. > Maybe, but it is more than a ticket 0 change Even for an example? If it is too big of a change, please put it on the list for the next version. >> Page 503 lines 22-23: Is this hint intended to say how many devices >> to use, not how many are in the system? > No, it is correct. The next hint "striping_factor" says how many > devices to use. What is the point of putting in a hint about the system configuration (which the MPI implementation should be able to get on its own)? >> Page 506 lines 30-33: Is there an easy way to tell that a datatype is >> derived? Is the datatype returned, if derived, guaranteed to be new >> (and not, for example, one passed into MPI_FILE_SET_VIEW by the >> user)? > Yes, using MPI_Type_get_envelope. If derived, the text says, the > function returns a datatype that the user is responsible for freeing. > It doesn't talk in terms of new or not. That seems like it needs advice to users (I didn't know before your comment how to determine whether a datatype is derived, and the test might be subtle for people who are not experts in datatype handling in MPI). >> Page 508 lines 9-11: Does that mean a buffer being written to a file >> can't be concurrently sent? It seems like that restriction may be >> too strong, especially when you can use a datatype to write the same >> data to a file more than once in a single operation. > It would be more than a ticket 0 change. OK, but it should probably be on the list to look at for the next version. >> Page 536 lines 34-35: Unicode has several encoding formats; which one >> are you intending (UTF-7, UTF-8, UTF-16, UCS-16, UCS-32, or something >> else)? > Don't know. It would be a problem for interoperability of external32 to not have a normative answer to that. Also, different Unicode formats change which code points can be represented (many UCS-2 implementations, for example, cannot encode code points greather than 0xFFFF, for example; see http://www.unicode.org/faq/basic_q.html#14) >> Page 537 lines 9-12: Some of this information (what happens on >> overflows from down-converting number formats, for example) is >> normative and should be in the main text. Also, taking the >> least-significant bytes of a larger number does not always preserve >> the sign bit in case of overflows (converting the MPI_INT 32768 to an >> MPI_INTEGER2 leads to -32768, for example). Is making sure to >> preserve the sign bit always the intended behavior? > Not changed. Why? >> Page 538 line 9: Defining wchar as two bytes prevents it from >> representing all valid Unicode code points (some of them are above >> 2^16). If such code points are forbidden (even on platforms where >> wchar_t is 32 bits), be sure to say that explicitly. > Not changed. Why? >> Does the MPI standard say anywhere that a byte is assumed to be 8 bits? > Don't know. It turns out that the answer is "yes" (page 25 line 28). >> Page 541 lines 47-48: Does that mean that point-to-point operations >> are OK? Non-collective file I/O? > Sounds like it. That seems odd -- I would expect that I/O callbacks would be completely local and not do I/O themselves. That might be worth considering for the next version. >> Page 542 lines 17-25: The first item in this list appears to be a >> superset of the third one, and it seems like all of them might be >> required to be satisfied when applicable (not just "at least one"). > The first one refers to a program using only basic datatypes. The 3rd > one refers to a program using derived datatypes. The third bullet point there is still about "predefined datatypes", or does "data item" refer to an element of a derived datatype? >> Page 543 lines 4-5: How do you get multiple file handles out of the >> same collective open of a file? Accesses from different processes >> only? Or can file descriptors be sent between processes? > Multiple means the return values on different processes. Does that mean that file handles can be sent between processes? If not, why would the behavior of multiple processes accessing the same file change based on whether the collective open operation returned different file handles to the different processes rather than returning the same one everywhere? >> Page 547 line 10: Is MPI_Offset required to be signed? > Yes. you can seek backwards from the current location. That's what I would expect, but that might be worthwhile to say ("MPI_Offset is a <ins>signed</ins> integer type ..."). -- Jeremiah Willcock On Mon, 3 Sep 2012, Rajeev Thakur wrote: > Jeremiah, > I appreciate your careful reading of the MPI standard (over 800 pages) from front to back. It must have taken many days to read everything and type in all the comments. > > I have incorporated your comments on the I/O chapter and sent the revised version to Bill/Jeff/Rich to check in. Attached is my response to your comments. I have incorporated almost all of them. > > Thanks again, > Rajeev > > >

1 0

Re: [Mpi-comments] Comments on the MPI-3 draft
by Rajeev Thakur 03 Sep '12

03 Sep '12

Reinhold, Thanks to you and your colleagues at LRZ for your careful reading of the draft MPI-3 standard. I have fixed the typos you pointed out in the I/O chapter. [p535:23] Replace "implictly" by "implicitly" (typo) [p542:27-28] Replace "compatiblty" by "compatibility" (typo) Rajeev On Aug 31, 2012, at 9:47 AM, Bader, Reinhold wrote: > Please consult the attached text file. > > Best Regards > > -- > Dr. Reinhold Bader > > Leibniz Supercomputing Centre (http://www.lrz.de) / HPC Support > Tel. +49 89 35831 8825 - Fax +49 89 35831 8625 > > <MPI_3.0_comments.txt>_______________________________________________ > mpi-comments mailing list > mpi-comments(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-comments

1 0

Re: [Mpi-comments] Comments on MPI 3.0 draft 2 [part 5]
by Rajeev Thakur 03 Sep '12

03 Sep '12

Jeremiah, I appreciate your careful reading of the MPI standard (over 800 pages) from front to back. It must have taken many days to read everything and type in all the comments. I have incorporated your comments on the I/O chapter and sent the revised version to Bill/Jeff/Rich to check in. Attached is my response to your comments. I have incorporated almost all of them. Thanks again, Rajeev

1 0

Comments on the MPI-3 draft
by Bader, Reinhold 31 Aug '12

31 Aug '12

Please consult the attached text file. Best Regards -- Dr. Reinhold Bader Leibniz Supercomputing Centre (http://www.lrz.de) / HPC Support Tel. +49 89 35831 8825 - Fax +49 89 35831 8625

1 0

Re: [Mpi-comments] MPI_Get_count_x
by Hubert Ritzdorf 31 Aug '12

31 Aug '12

Hi George, I know MPI_Get_elements_x (). But Page 483 claims: These functions modify the opaque part of status so that a call to MPI_GET_ELEMENTS or MPI_GET_ELEMENTS_X will return count. MPI_GET_COUNT will return a compatible value. This will not work for large MPI_Count values provided by MPI_SET_ELEMENTS_X (i.e. MPI_UNDEFINED is returned by MPI_Get_count() in significant use of MPI_SET_ELEMENT_X). And Page 114 says that function MPI_Get_count has a different behavior than MPI_GET_ELEMENTS/MPI_GET_ELEMENTS_X. Thus, MPI_GET_COUNT is not obsolete and application has to develop a separate version of MPI_Get_count_x() if MPI_SET_ELEMENT_X might be used in the application or a library. Hubert ________________________________ From: mpi-comments-bounces(a)lists.mpi-forum.org [mpi-comments-bounces(a)lists.mpi-forum.org] on behalf of George Bosilca [bosilca(a)eecs.utk.edu] Sent: Friday, August 31, 2012 2:26 PM To: Comments on the MPI Standard Subject: Re: [Mpi-comments] MPI_Get_count_x MPI_Get_elements_x, chap Datatypes page 114 (current make cleandoc). george. On Aug 31, 2012, at 13:45 , Hubert Ritzdorf <Hubert.Ritzdorf(a)EMEA.NEC.COM<mailto:[email protected]>> wrote: Hi, I think, that function MPI_Get_count_x (const MPI_Status *status, MPI_Datatype datatype, MPI_Count *count) is missing. After use of function MPI_Status_set_elements_x, MPI_Get_count is possibly not able to return a compatible value. This is contradictory to Page 483, Lines 22-33. Hubert _______________________________________________ mpi-comments mailing list mpi-comments(a)lists.mpi-forum.org<mailto:[email protected]> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-comments Click here<https://www.mailcontrol.com/sr/lGzi4c6egezTndxI!oX7UkkR2vxuiI13vs0Z85bSifOX…> to report this email as spam.

1 0

Re: [Mpi-comments] MPI_Get_count_x
by George Bosilca 31 Aug '12

31 Aug '12

MPI_Get_elements_x, chap Datatypes page 114 (current make cleandoc). george. On Aug 31, 2012, at 13:45 , Hubert Ritzdorf <Hubert.Ritzdorf(a)EMEA.NEC.COM> wrote: > Hi, > > I think, that function > > MPI_Get_count_x (const MPI_Status *status, MPI_Datatype datatype, MPI_Count *count) > > is missing. After use of function MPI_Status_set_elements_x, MPI_Get_count is possibly not able > to return a compatible value. This is contradictory to Page 483, Lines 22-33. > > Hubert > > > > _______________________________________________ > mpi-comments mailing list > mpi-comments(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-comments

1 0

MPI_Get_count_x
by Hubert Ritzdorf 31 Aug '12

31 Aug '12

Hi, I think, that function MPI_Get_count_x (const MPI_Status *status, MPI_Datatype datatype, MPI_Count *count) is missing. After use of function MPI_Status_set_elements_x, MPI_Get_count is possibly not able to return a compatible value. This is contradictory to Page 483, Lines 22-33. Hubert

1 0

Re: [Mpi-comments] MPI-3.0 public draft, small request for clarification
by William Gropp 29 Aug '12

29 Aug '12

I believe that your program is correct. The "e.g." is a "for example", and does not mandate the use of the particular completion functions mentioned. Thus MPI_WIN_TEST as you have used it is sufficient, and I don't believe any additional text is needed. I believe that your vendor has misread the standard. Bill William Gropp Director, Parallel Computing Institute Deputy Director for Research Institute for Advanced Computing Applications and Technologies Paul and Cynthia Saylor Professor of Computer Science University of Illinois Urbana-Champaign On Aug 29, 2012, at 9:34 AM, Florian Prill wrote: > Hello, > > I have a small, very technical comment on the formulation of MPI_WIN_FREE and MPI_WIN_TEST which might be of interest. I already came across this rather subtle point in the MPI standard v2.2 and it seems to me that the issue has not been clarified in the new MPI 3.0 draft. > > The problem is as follows: For a small test program I encountered a crash on some platforms (NEC-SX9) while everything runs smoothly elsewhere (Linux-gfortran). > > PROGRAM main > USE MPI > IMPLICIT NONE > > INTEGER, PARAMETER :: nfieldsize = 10 > REAL :: send_buf(nfieldsize), recv_buf(nfieldsize) > INTEGER :: win01, ierr, rank, sizeofreal, msg_size > INTEGER :: group, togroup, fromgroup > INTEGER(KIND=MPI_ADDRESS_KIND) :: bytesize > INTEGER :: ranks(0:1) > LOGICAL :: l_complete > > CALL MPI_INIT(ierr) > CALL MPI_COMM_RANK(MPI_COMM_WORLD, rank, ierr) ! get own rank > CALL MPI_TYPE_SIZE(MPI_REAL, sizeofreal, ierr) > WRITE (*,*) "proc ", rank, ": creating window" > bytesize = nfieldsize*sizeofreal > CALL MPI_WIN_CREATE(recv_buf, bytesize, sizeofreal, MPI_INFO_NULL, MPI_COMM_WORLD, win01, ierr) > send_buf = 1 > recv_buf = 99 > WRITE (*,*) "proc ", rank, ": Before: ", recv_buf > ! build two new groups: one contains only the sender, the other > ! group is the receiver > CALL MPI_COMM_GROUP(MPI_COMM_WORLD, group, ierr) > ranks = (/ 0, 1 /) > CALL MPI_GROUP_INCL(group, 1, ranks(1:1), togroup, ierr) > CALL MPI_GROUP_INCL(group, 1, ranks(0:0), fromgroup, ierr) > IF (rank == 0) THEN > CALL MPI_WIN_START(togroup, 0, win01, ierr) > ELSE > CALL MPI_WIN_POST(fromgroup, 0, win01, ierr) > END IF > IF (rank == 0) THEN > WRITE (*,*) "proc ", rank, ": one-sided put" > msg_size = 1 > CALL MPI_PUT(send_buf, msg_size, MPI_REAL, 1, 0_MPI_ADDRESS_KIND, & > & msg_size, MPI_REAL, win01, ierr) > END IF > WRITE (*,*) "proc ", rank, ": closing fence" > IF (rank == 0) THEN > CALL MPI_WIN_COMPLETE(win01, ierr) > ELSE > DO > WRITE (*,*) "test..." > ! check if RMA access completed: > CALL MPI_WIN_TEST(win01, l_complete, ierr) > IF (l_complete) EXIT > END DO > ! CALL MPI_WIN_WAIT(win01, ierr) ! <<< needed on some machines !!! > END IF > WRITE (*,*) "proc ", rank, ": After: ", recv_buf > CALL MPI_WIN_FREE(win01, ierr) > CALL MPI_BARRIER(MPI_COMM_WORLD, ierr) > CALL MPI_FINALIZE(ierr) > END PROGRAM main > > Our system vendor insisted that the MPI implementation is standard conforming and that the program crash was caused by the use of MPI_WIN_FREE. True enough, the standard says (11.2.5 Window Destruction): > > "MPI_WIN_FREE(win) can be invoked by a process only after it has completed its involvement in RMA communications on window win: e.g., the process has called MPI_WIN_FENCE, or called MPI_WIN_WAIT to match a previous call to MPI_WIN_POST or called MPI_WIN_COMPLETE to match a previous call to MPI_WIN_START or called MPI_WIN_UNLOCK to match a previous call to MPI_WIN_LOCK. [...]" > > I knew about this definition, however, the standard also contains the following passage (11.5.2, p.445): > > "The effect of return of MPI_WIN_TEST with flag = true is the same as the effect of a return of MPI_WIN_WAIT." > > Doesn't this mean that the above test program is correct? Perhaps one of these two passages in the standard needs clarification? > > Regards > Florian > > _______________________________________________ > mpi-comments mailing list > mpi-comments(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-comments

1 0

MPI-3.0 public draft, small request for clarification
by Florian Prill 29 Aug '12

29 Aug '12

Hello, I have a small, very technical comment on the formulation of MPI_WIN_FREE and MPI_WIN_TEST which might be of interest. I already came across this rather subtle point in the MPI standard v2.2 and it seems to me that the issue has not been clarified in the new MPI 3.0 draft. The problem is as follows: For a small test program I encountered a crash on some platforms (NEC-SX9) while everything runs smoothly elsewhere (Linux-gfortran). PROGRAM main USE MPI IMPLICIT NONE INTEGER, PARAMETER :: nfieldsize = 10 REAL :: send_buf(nfieldsize), recv_buf(nfieldsize) INTEGER :: win01, ierr, rank, sizeofreal, msg_size INTEGER :: group, togroup, fromgroup INTEGER(KIND=MPI_ADDRESS_KIND) :: bytesize INTEGER :: ranks(0:1) LOGICAL :: l_complete CALL MPI_INIT(ierr) CALL MPI_COMM_RANK(MPI_COMM_WORLD, rank, ierr) ! get own rank CALL MPI_TYPE_SIZE(MPI_REAL, sizeofreal, ierr) WRITE (*,*) "proc ", rank, ": creating window" bytesize = nfieldsize*sizeofreal CALL MPI_WIN_CREATE(recv_buf, bytesize, sizeofreal, MPI_INFO_NULL, MPI_COMM_WORLD, win01, ierr) send_buf = 1 recv_buf = 99 WRITE (*,*) "proc ", rank, ": Before: ", recv_buf ! build two new groups: one contains only the sender, the other ! group is the receiver CALL MPI_COMM_GROUP(MPI_COMM_WORLD, group, ierr) ranks = (/ 0, 1 /) CALL MPI_GROUP_INCL(group, 1, ranks(1:1), togroup, ierr) CALL MPI_GROUP_INCL(group, 1, ranks(0:0), fromgroup, ierr) IF (rank == 0) THEN CALL MPI_WIN_START(togroup, 0, win01, ierr) ELSE CALL MPI_WIN_POST(fromgroup, 0, win01, ierr) END IF IF (rank == 0) THEN WRITE (*,*) "proc ", rank, ": one-sided put" msg_size = 1 CALL MPI_PUT(send_buf, msg_size, MPI_REAL, 1, 0_MPI_ADDRESS_KIND, & & msg_size, MPI_REAL, win01, ierr) END IF WRITE (*,*) "proc ", rank, ": closing fence" IF (rank == 0) THEN CALL MPI_WIN_COMPLETE(win01, ierr) ELSE DO WRITE (*,*) "test..." ! check if RMA access completed: CALL MPI_WIN_TEST(win01, l_complete, ierr) IF (l_complete) EXIT END DO ! CALL MPI_WIN_WAIT(win01, ierr) ! <<< needed on some machines !!! END IF WRITE (*,*) "proc ", rank, ": After: ", recv_buf CALL MPI_WIN_FREE(win01, ierr) CALL MPI_BARRIER(MPI_COMM_WORLD, ierr) CALL MPI_FINALIZE(ierr) END PROGRAM main Our system vendor insisted that the MPI implementation is standard conforming and that the program crash was caused by the use of MPI_WIN_FREE. True enough, the standard says (11.2.5 Window Destruction): "MPI_WIN_FREE(win) can be invoked by a process only after it has completed its involvement in RMA communications on window win: e.g., the process has called MPI_WIN_FENCE, or called MPI_WIN_WAIT to match a previous call to MPI_WIN_POST or called MPI_WIN_COMPLETE to match a previous call to MPI_WIN_START or called MPI_WIN_UNLOCK to match a previous call to MPI_WIN_LOCK. [...]" I knew about this definition, however, the standard also contains the following passage (11.5.2, p.445): "The effect of return of MPI_WIN_TEST with flag = true is the same as the effect of a return of MPI_WIN_WAIT." Doesn't this mean that the above test program is correct? Perhaps one of these two passages in the standard needs clarification? Regards Florian

1 0