August 2012 - mpi-comments - mailman3.cels.anl.gov

Comments on the MPI-3 draft
by Bader, Reinhold 31 Aug '12

31 Aug '12

Please consult the attached text file. Best Regards -- Dr. Reinhold Bader Leibniz Supercomputing Centre (http://www.lrz.de) / HPC Support Tel. +49 89 35831 8825 - Fax +49 89 35831 8625

1 0

Re: [Mpi-comments] MPI_Get_count_x
by Hubert Ritzdorf 31 Aug '12

31 Aug '12

Hi George, I know MPI_Get_elements_x (). But Page 483 claims: These functions modify the opaque part of status so that a call to MPI_GET_ELEMENTS or MPI_GET_ELEMENTS_X will return count. MPI_GET_COUNT will return a compatible value. This will not work for large MPI_Count values provided by MPI_SET_ELEMENTS_X (i.e. MPI_UNDEFINED is returned by MPI_Get_count() in significant use of MPI_SET_ELEMENT_X). And Page 114 says that function MPI_Get_count has a different behavior than MPI_GET_ELEMENTS/MPI_GET_ELEMENTS_X. Thus, MPI_GET_COUNT is not obsolete and application has to develop a separate version of MPI_Get_count_x() if MPI_SET_ELEMENT_X might be used in the application or a library. Hubert ________________________________ From: mpi-comments-bounces(a)lists.mpi-forum.org [mpi-comments-bounces(a)lists.mpi-forum.org] on behalf of George Bosilca [bosilca(a)eecs.utk.edu] Sent: Friday, August 31, 2012 2:26 PM To: Comments on the MPI Standard Subject: Re: [Mpi-comments] MPI_Get_count_x MPI_Get_elements_x, chap Datatypes page 114 (current make cleandoc). george. On Aug 31, 2012, at 13:45 , Hubert Ritzdorf <Hubert.Ritzdorf(a)EMEA.NEC.COM<mailto:[email protected]>> wrote: Hi, I think, that function MPI_Get_count_x (const MPI_Status *status, MPI_Datatype datatype, MPI_Count *count) is missing. After use of function MPI_Status_set_elements_x, MPI_Get_count is possibly not able to return a compatible value. This is contradictory to Page 483, Lines 22-33. Hubert _______________________________________________ mpi-comments mailing list mpi-comments(a)lists.mpi-forum.org<mailto:[email protected]> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-comments Click here<https://www.mailcontrol.com/sr/lGzi4c6egezTndxI!oX7UkkR2vxuiI13vs0Z85bSifOX…> to report this email as spam.

1 0

Re: [Mpi-comments] MPI_Get_count_x
by George Bosilca 31 Aug '12

31 Aug '12

MPI_Get_elements_x, chap Datatypes page 114 (current make cleandoc). george. On Aug 31, 2012, at 13:45 , Hubert Ritzdorf <Hubert.Ritzdorf(a)EMEA.NEC.COM> wrote: > Hi, > > I think, that function > > MPI_Get_count_x (const MPI_Status *status, MPI_Datatype datatype, MPI_Count *count) > > is missing. After use of function MPI_Status_set_elements_x, MPI_Get_count is possibly not able > to return a compatible value. This is contradictory to Page 483, Lines 22-33. > > Hubert > > > > _______________________________________________ > mpi-comments mailing list > mpi-comments(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-comments

1 0

MPI_Get_count_x
by Hubert Ritzdorf 31 Aug '12

31 Aug '12

Hi, I think, that function MPI_Get_count_x (const MPI_Status *status, MPI_Datatype datatype, MPI_Count *count) is missing. After use of function MPI_Status_set_elements_x, MPI_Get_count is possibly not able to return a compatible value. This is contradictory to Page 483, Lines 22-33. Hubert

1 0

Re: [Mpi-comments] MPI-3.0 public draft, small request for clarification
by William Gropp 29 Aug '12

29 Aug '12

I believe that your program is correct. The "e.g." is a "for example", and does not mandate the use of the particular completion functions mentioned. Thus MPI_WIN_TEST as you have used it is sufficient, and I don't believe any additional text is needed. I believe that your vendor has misread the standard. Bill William Gropp Director, Parallel Computing Institute Deputy Director for Research Institute for Advanced Computing Applications and Technologies Paul and Cynthia Saylor Professor of Computer Science University of Illinois Urbana-Champaign On Aug 29, 2012, at 9:34 AM, Florian Prill wrote: > Hello, > > I have a small, very technical comment on the formulation of MPI_WIN_FREE and MPI_WIN_TEST which might be of interest. I already came across this rather subtle point in the MPI standard v2.2 and it seems to me that the issue has not been clarified in the new MPI 3.0 draft. > > The problem is as follows: For a small test program I encountered a crash on some platforms (NEC-SX9) while everything runs smoothly elsewhere (Linux-gfortran). > > PROGRAM main > USE MPI > IMPLICIT NONE > > INTEGER, PARAMETER :: nfieldsize = 10 > REAL :: send_buf(nfieldsize), recv_buf(nfieldsize) > INTEGER :: win01, ierr, rank, sizeofreal, msg_size > INTEGER :: group, togroup, fromgroup > INTEGER(KIND=MPI_ADDRESS_KIND) :: bytesize > INTEGER :: ranks(0:1) > LOGICAL :: l_complete > > CALL MPI_INIT(ierr) > CALL MPI_COMM_RANK(MPI_COMM_WORLD, rank, ierr) ! get own rank > CALL MPI_TYPE_SIZE(MPI_REAL, sizeofreal, ierr) > WRITE (*,*) "proc ", rank, ": creating window" > bytesize = nfieldsize*sizeofreal > CALL MPI_WIN_CREATE(recv_buf, bytesize, sizeofreal, MPI_INFO_NULL, MPI_COMM_WORLD, win01, ierr) > send_buf = 1 > recv_buf = 99 > WRITE (*,*) "proc ", rank, ": Before: ", recv_buf > ! build two new groups: one contains only the sender, the other > ! group is the receiver > CALL MPI_COMM_GROUP(MPI_COMM_WORLD, group, ierr) > ranks = (/ 0, 1 /) > CALL MPI_GROUP_INCL(group, 1, ranks(1:1), togroup, ierr) > CALL MPI_GROUP_INCL(group, 1, ranks(0:0), fromgroup, ierr) > IF (rank == 0) THEN > CALL MPI_WIN_START(togroup, 0, win01, ierr) > ELSE > CALL MPI_WIN_POST(fromgroup, 0, win01, ierr) > END IF > IF (rank == 0) THEN > WRITE (*,*) "proc ", rank, ": one-sided put" > msg_size = 1 > CALL MPI_PUT(send_buf, msg_size, MPI_REAL, 1, 0_MPI_ADDRESS_KIND, & > & msg_size, MPI_REAL, win01, ierr) > END IF > WRITE (*,*) "proc ", rank, ": closing fence" > IF (rank == 0) THEN > CALL MPI_WIN_COMPLETE(win01, ierr) > ELSE > DO > WRITE (*,*) "test..." > ! check if RMA access completed: > CALL MPI_WIN_TEST(win01, l_complete, ierr) > IF (l_complete) EXIT > END DO > ! CALL MPI_WIN_WAIT(win01, ierr) ! <<< needed on some machines !!! > END IF > WRITE (*,*) "proc ", rank, ": After: ", recv_buf > CALL MPI_WIN_FREE(win01, ierr) > CALL MPI_BARRIER(MPI_COMM_WORLD, ierr) > CALL MPI_FINALIZE(ierr) > END PROGRAM main > > Our system vendor insisted that the MPI implementation is standard conforming and that the program crash was caused by the use of MPI_WIN_FREE. True enough, the standard says (11.2.5 Window Destruction): > > "MPI_WIN_FREE(win) can be invoked by a process only after it has completed its involvement in RMA communications on window win: e.g., the process has called MPI_WIN_FENCE, or called MPI_WIN_WAIT to match a previous call to MPI_WIN_POST or called MPI_WIN_COMPLETE to match a previous call to MPI_WIN_START or called MPI_WIN_UNLOCK to match a previous call to MPI_WIN_LOCK. [...]" > > I knew about this definition, however, the standard also contains the following passage (11.5.2, p.445): > > "The effect of return of MPI_WIN_TEST with flag = true is the same as the effect of a return of MPI_WIN_WAIT." > > Doesn't this mean that the above test program is correct? Perhaps one of these two passages in the standard needs clarification? > > Regards > Florian > > _______________________________________________ > mpi-comments mailing list > mpi-comments(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-comments

1 0

MPI-3.0 public draft, small request for clarification
by Florian Prill 29 Aug '12

29 Aug '12

Hello, I have a small, very technical comment on the formulation of MPI_WIN_FREE and MPI_WIN_TEST which might be of interest. I already came across this rather subtle point in the MPI standard v2.2 and it seems to me that the issue has not been clarified in the new MPI 3.0 draft. The problem is as follows: For a small test program I encountered a crash on some platforms (NEC-SX9) while everything runs smoothly elsewhere (Linux-gfortran). PROGRAM main USE MPI IMPLICIT NONE INTEGER, PARAMETER :: nfieldsize = 10 REAL :: send_buf(nfieldsize), recv_buf(nfieldsize) INTEGER :: win01, ierr, rank, sizeofreal, msg_size INTEGER :: group, togroup, fromgroup INTEGER(KIND=MPI_ADDRESS_KIND) :: bytesize INTEGER :: ranks(0:1) LOGICAL :: l_complete CALL MPI_INIT(ierr) CALL MPI_COMM_RANK(MPI_COMM_WORLD, rank, ierr) ! get own rank CALL MPI_TYPE_SIZE(MPI_REAL, sizeofreal, ierr) WRITE (*,*) "proc ", rank, ": creating window" bytesize = nfieldsize*sizeofreal CALL MPI_WIN_CREATE(recv_buf, bytesize, sizeofreal, MPI_INFO_NULL, MPI_COMM_WORLD, win01, ierr) send_buf = 1 recv_buf = 99 WRITE (*,*) "proc ", rank, ": Before: ", recv_buf ! build two new groups: one contains only the sender, the other ! group is the receiver CALL MPI_COMM_GROUP(MPI_COMM_WORLD, group, ierr) ranks = (/ 0, 1 /) CALL MPI_GROUP_INCL(group, 1, ranks(1:1), togroup, ierr) CALL MPI_GROUP_INCL(group, 1, ranks(0:0), fromgroup, ierr) IF (rank == 0) THEN CALL MPI_WIN_START(togroup, 0, win01, ierr) ELSE CALL MPI_WIN_POST(fromgroup, 0, win01, ierr) END IF IF (rank == 0) THEN WRITE (*,*) "proc ", rank, ": one-sided put" msg_size = 1 CALL MPI_PUT(send_buf, msg_size, MPI_REAL, 1, 0_MPI_ADDRESS_KIND, & & msg_size, MPI_REAL, win01, ierr) END IF WRITE (*,*) "proc ", rank, ": closing fence" IF (rank == 0) THEN CALL MPI_WIN_COMPLETE(win01, ierr) ELSE DO WRITE (*,*) "test..." ! check if RMA access completed: CALL MPI_WIN_TEST(win01, l_complete, ierr) IF (l_complete) EXIT END DO ! CALL MPI_WIN_WAIT(win01, ierr) ! <<< needed on some machines !!! END IF WRITE (*,*) "proc ", rank, ": After: ", recv_buf CALL MPI_WIN_FREE(win01, ierr) CALL MPI_BARRIER(MPI_COMM_WORLD, ierr) CALL MPI_FINALIZE(ierr) END PROGRAM main Our system vendor insisted that the MPI implementation is standard conforming and that the program crash was caused by the use of MPI_WIN_FREE. True enough, the standard says (11.2.5 Window Destruction): "MPI_WIN_FREE(win) can be invoked by a process only after it has completed its involvement in RMA communications on window win: e.g., the process has called MPI_WIN_FENCE, or called MPI_WIN_WAIT to match a previous call to MPI_WIN_POST or called MPI_WIN_COMPLETE to match a previous call to MPI_WIN_START or called MPI_WIN_UNLOCK to match a previous call to MPI_WIN_LOCK. [...]" I knew about this definition, however, the standard also contains the following passage (11.5.2, p.445): "The effect of return of MPI_WIN_TEST with flag = true is the same as the effect of a return of MPI_WIN_WAIT." Doesn't this mean that the above test program is correct? Perhaps one of these two passages in the standard needs clarification? Regards Florian

1 0

Re: [Mpi-comments] RMA llist bug
by William Gropp 27 Aug '12

27 Aug '12

I'll fix this. Bill William Gropp Director, Parallel Computing Institute Deputy Director for Research Institute for Advanced Computing Applications and Technologies Paul and Cynthia Saylor Professor of Computer Science University of Illinois Urbana-Champaign On Aug 27, 2012, at 5:34 PM, Jim Dinan wrote: > There is a bug on pg. 485, line 38: > > my_elems = realloc(my_elems, my_elems_size); > > Should be: > > my_elems = realloc(my_elems, my_elems_size*sizeof(void*)); > > ~Jim. > _______________________________________________ > mpi-comments mailing list > mpi-comments(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-comments

1 0

RMA llist bug
by Jim Dinan 27 Aug '12

27 Aug '12

There is a bug on pg. 485, line 38: my_elems = realloc(my_elems, my_elems_size); Should be: my_elems = realloc(my_elems, my_elems_size*sizeof(void*)); ~Jim.

1 0

Comments on MPI 3.0 draft 2 [part 5]
by Jeremiah Willcock 26 Aug '12

26 Aug '12

These are in reference to the draft at <URL:http://meetings.mpi-forum.org/mpi3.0_draft_2.pdf>. Thank you for considering them. -- Jeremiah Willcock Chapter 13: Page 492 line 47: "8th" should be "eighth". Page 492 line 48: Although "formal parameter" is the correct term normally, MPI typically uses "argument" to mean that. Page 494 lines 11-21: Some of the example text in here might be worth putting in code font. Page 494 line 31: Is MPI_MODE_RDWR intended to be equal to MPI_MODE_RDONLY | MPI_MODE_WRONLY? If so, that will require changes in some of the nearby text. Page 495 lines 11-12: Are there any guarantees about the behavior of MPI_MODE_DELETE_ON_CLOSE if the program exits (normally or abnormally) without explicitly closing the file? Page 499 lines 44-45: Wouldn't using log on the bit to test and then mod and integer division on the amode be simpler than the loop? Floating-point errors might cause trouble there, however, or if one of the masks was not a single bit. Page 500 line 36: "Section" should be "Chapter". Page 501 line 20: There is an unnecessary line break at the end of the line. Page 501 line 47: "pair" should be "pairs". Page 502 line 3: The "and" might need to be changed to "or". Page 502 line 12: "Section" should be "Chapter". Page 502 line 34: Is there a reason that "CYCLIC" is all-uppercase? Page 503 lines 22-23: Is this hint intended to say how many devices to use, not how many are in the system? Page 505 lines 15, 25-28, and 32: "etype" and "filetype" should be in text font when they are not referring to specific argument names. Page 505 line 30: There should be a comma after "etype". Page 506 lines 30-33: Is there an easy way to tell that a datatype is derived? Is the datatype returned, if derived, guaranteed to be new (and not, for example, one passed into MPI_FILE_SET_VIEW by the user)? Page 506 lines 40-42: There is extra space after "vs." here (probably needs to be 'vs.\ ' in LaTeX). I don't remember whether vs is italicized elsewhere in the standard. Page 506 line 44: The POSIX function names might need to be in code font. Page 507 line 25: The em-dash should have spaces around it to match the rest of the standard. This issue may occur elsewhere in the chapter as well. Page 508 line 3: "in to" should be "into". Page 508 lines 9-11: Does that mean a buffer being written to a file can't be concurrently sent? It seems like that restriction may be too strong, especially when you can use a datatype to write the same data to a file more than once in a single operation. Page 509 lines 8-12: The section titles should be in the normal text font, the commas and final period should be inside the quotes, and there needs to be a comma after "Temporary Data Movements". Page 509 line 18: The fields in MPI_Status are referred to in sans-serif on page 30 where they are introduced. Page 520 lines 30-31: Do all processes need to use equivalent views of the file to use the shared file pointer? If not, what is the file pointer relative to (i.e., if I read using it in a process with one view, then read using it in a process with a different view, what happens)? Pages 524-525: Are MPI_FILE_SEEK_SHARED and/or MPI_FILE_GET_POSITION_SHARED collective? Page 534 line 24: "of the MPI-1 document" should be removed, and "page" should be "pages". Page 536 line 26: There should be a comma after "8". Page 536 line 30: This list should also include the C++ bool type now that there is an MPI datatype for it. Page 536 line 31: There should be a comma after "double _Complex". This list should include the C++ complex types. In what order do the real and imaginary components appear in the file? Page 536 lines 34-35: Unicode has several encoding formats; which one are you intending (UTF-7, UTF-8, UTF-16, UCS-16, UCS-32, or something else)? Page 537 line 5: "size" should be "sizes". Page 537 line 10: "less" should be "least". Page 537 lines 9-12: Some of this information (what happens on overflows from down-converting number formats, for example) is normative and should be in the main text. Also, taking the least-significant bytes of a larger number does not always preserve the sign bit in case of overflows (converting the MPI_INT 32768 to an MPI_INTEGER2 leads to -32768, for example). Is making sure to preserve the sign bit always the intended behavior? Page 538 line 9: Defining wchar as two bytes prevents it from representing all valid Unicode code points (some of them are above 2^16). If such code points are forbidden (even on platforms where wchar_t is 32 bits), be sure to say that explicitly. Page 538 line 40: Why is MPI_LOGICAL defined as 4 bytes instead of 1? Does the MPI standard say anywhere that a byte is assumed to be 8 bits? Page 540 line 17: There is extra whitespace at the end of the line. Page 541 lines 29 and 44: What does "equivalent" mean? That they have the same typemap and displacements, or that they are exactly the handles that the user passed in (allowing caching)? Page 541 lines 47-48: Does that mean that point-to-point operations are OK? Non-collective file I/O? Page 542 line 14: The first quotation mark is backwards. Page 542 lines 17-25: The first item in this list appears to be a superset of the third one, and it seems like all of them might be required to be satisfied when applicable (not just "at least one"). Page 542 lines 47-: "FH", "fh", and "FOO" in math mode might need to be in \mathit (I can't tell for sure). Page 543 lines 4-5: How do you get multiple file handles out of the same collective open of a file? Accesses from different processes only? Or can file descriptors be sent between processes? Page 545 line 32: There need to be spaces around the em dash. Page 546 line 27: "dup" should be "duplicate", there should be spaces around the em dash. Page 547 line 10: Is MPI_Offset required to be signed? Page 547 line 16: "KIND" might need to be in another font and/or lowercase. Page 547 line 26: "info" (and maybe "hints") should be in text font. Page 550 line 31: "myfile" might need to be in code font. Page 552 line 11: The em dash needs spaces around it. Page 552 line 27: The comma should be a period. Page 552 line 34: The line has extra space at the end. Page 555 line 40: The ranges here appear to have hyphens in them, rather than the correct en dashes. Chapter 14: Page 559 line 28: The default definition of MPI_PCONTROL needs to return successfully; that is not stated explicitly. Page 560: Line 20 should be moved above line 17 in the example so that the MPI_Type_size call is not timed. Page 560 line 33: "system" should be capitalized as a proper noun. Page 560 line 47: ", however" should be "; however,". Page 561 line 47: "they're" should be "they are", and there is extra space just before the question mark. Page 562 line 2: "single threaded" should be hyphenated. Page 562 line 18: There should be a comma after "Therefore". Page 562 line 25: "ared out" is not a standard term as far as I know; maybe say something like "copied out of the base library and into the profiling one using a tool such as ar.". Why is the information on enumerations first in the documentation, before any discussion of variables or how to use them? Page 564 line 11: There should be a comma after "tuners". Page 564 line 12: There should be a comma after "detailed". Page 564 line 27, page 565 line 14, page 567 line 22, page 571 line 26, page 591 line 45: Table captions in the standard typically do not end with periods. Page 567 line 30: Remove the comma before "which". Page 568 line 18: Add a comma and space between "type" and "as". Page 568 line 20: "enumerations" should be singular. Page 568 lines 44-45: "of at least length one" should be "of length at least one". Page 569 line 10: The first comma should be before the closing quote. Page 569 line 21: "its value" should be "their values". Page 570 line 30: Can the number and/or names of enumerators change during the course of a run? Page 570 line 42: "mpiargenumtype" is a typo. Page 571 line 33: Example titles have normally been subsections (with numbers) before the bold "Example n.n" text. Page 573 line 7: There is extra space at the end of this line. Page 573 line 18: "variables" should be singular. Page 574 line 21: There should be a comma after the right parenthesis. Page 575 lines 17-18: "implementation specific" should be hyphenated when used as an adjective (this applies to other parts of the standard as well). Page 575 line 34: "reseting" should be "resetting". Page 576-577: The datatype names in here should be in sans-serif. Page 577 line 21: "MPI_T_PVAR_CLASS_AGGREGATE" should be in sans-serif. Page 577 lines 33-34: "variable specific" and "implementation defined" both need to be hyphenated. Page 577 line 40: ", otherwise" should be "; otherwise,". Page 577 line 46: "its value" should be "their values". Page 579 line 31: Remove the comma after "one". Why are control variables and performance variables separate, given that they have so much in common and relatively few differences (in operations, attributes, etc.)? It does look like performance variables need to be relative to a session, but there can be a flag for "session-specific" or similar to handle that. Does MPI_INIT need to be called before running performance experiments? Page 580 line 44: "such object handles" should be "such an object handle". Page 581 line 1: There is extra space at the end of the line. Page 583 line 45, page 584 line 4: "sampling based" should be hyphenated. Page 584 line 6: "guaranteeing" should be "requiring". Page 584 line 11: There should be a comma after "write". Page 584 line 21: The opening quotation mark is backwards. Page 584 line 26: "clean up" should be "cleanup" or hyphenated. Page 584 line 46, page 586 line 20: These lines are indented more than those around them. Page 589 line 43: "have been" should be "may have been". Page 590 line 2: There should be a comma after "MPI_T_CATEGORY_GET_PVARS". Page 590 line 3: There should be a comma after "MPI_T_PVAR_GET_INFO". Page 590 line 20: "user provided" should be hyphenated. Page 591 lines 11-12: "or has been deleted" seems to conflict with enumeration indices not being allowed to be deleted (as far as I can tell). By the way, can they be added during the run? Page 591 line 26: The function names should be in sans-serif font. Page 591 line 31: "READREST" should be "READRESET".

1 0

Re: [Mpi-comments] Comments on MPI 3.0 draft 2 [part 2]
by George Bosilca 23 Aug '12

23 Aug '12

Dear Jeremiah, Thank you very much for your detailed comments on the chapters I maintain. I agree with most of your suggested changes, and I have applied them. A patch with the required changes have been sent to the gatekeepers. Thanks, george. On Aug 19, 2012, at 18:06 , Jeremiah Willcock <jewillco(a)osl.iu.edu> wrote: > Page 335 lines 28-29: Are these constants required to be macros (rather than const global variables), and do they need to be defined exactly as shown? Otherwise, someone could write "#define MPI_VERSION 1+2" or (in C++) "static const int MPI_VERSION = 3;". We want them to be used in #if, so they should be #define > Page 336 line 46: "value" should be "values". > > Page 337 line 8: What is "myrank"? > > Page 337 line 12: "implementation specific" should be "implementation-specific". > > Page 337 line 23: There should be a comma before "inclusive". > > Page 337 line 41: There should not be a comma before "I/O". > > Page 341 line 26: "pointer" should be "pointers". > > Page 341 line 48: This text should end with a period, and be on the same page as the code. Done. > Page 342: What is the return value of an error handler used for? Is it meaningful at all? Is it guaranteed that, if an MPI function returns at all after an error, its return value is the same (implementation-defined) error code that was passed to the handler? The prototype of error handlers is defined on page 344 at line 11, and there is no return value. > Page 342 lines 35-36: "Thus, if the user chooses not to control error handling, every error that MPI handles is treated as fatal." is not true for I/O. The text implies MPI errors. What is the I/O specific error that is treated differently? > Page 342 lines 40-41: "non trivial" should be "non-trivial". > > Page 343 line 29: "implementation" should be "implementations". > > Page 344 line 16: "stdargs" is not a standard term. I would use varargs instead. Not very standard either, but it is a minimal change improving readability. > Page 349 lines 44-45: Line 45 appears to be redundant with the last part of line 44. Repetition is the mother of all learnings syndrome ï¿½ > Page 357 line 6: "high-resolution" should be "high resolution". Also, this text might want to repeat the wording from Annex A that MPI_{Wtime,Wtick} can be macros and/or not go through the profiling interface. There is a reference to the section that describe them as macros. > Page 357 line 18: The discussion of MPI_WTIME_IS_GLOBAL should refer to the section where it is defined. OK. > Page 357 lines 34-: MPI_INIT_THREAD is not mentioned here; this seems like the logical place to describe it. We can't just reference it, we will have to describe the function and the prototype. It is a larger change, more than I would accept at this stage. > Page 359 line 38: There is extra space in the middle of the line. Tis seems to be an issue with one of our environments leaving trash behind. We will investigate more. > Page 359 line 45: There should be a comma after "MPI_REQUEST_FREE". > > Page 359 line 48: "operations" should be "operation". > > Page 360 lines 1-2: Are there any requirements that particular objects be freed before MPI_FINALIZE? Not imposed by the standard itself. > Page 361 line 31: Who/what does "they" refer to? > > Page 362: Both functions on this page have extra whitespace in their abstract signatures. > > Page 362 line 41: There should be a comma after "accepted". > > Page 363 lines 1 and 7: "error code" should be in sans-serif. OK. Thanks, george.

1 0