March 2011 - mpi-forum - mailman3.cels.anl.gov

Re: [Mpi-forum] Comment on Fortran WG5 ballot N1846
by Graham, Richard L. 31 Mar '11

31 Mar '11

Bill, The concern that was raised to me (I have not followed the standard, but rely on other folks in the forum for this) is that there is potential for incorrect behavior if a single data object is being used both for computation and for asynchronous communication (disjoint portions of the object are involved in each). The specific concern, if I understood things correctly - others, please correct me if I am wrong - that compilers may respect the ASYNCHRONOUS only with respect to other fortran libraries, such as fortran async I/O, and feel free to modify portions of the buffer being used for communications, only later to restore the parameters. If this is indeed what could happen, this would clearly be a correctness problem for MPI. Seems to me that if this is indeed an issue, MPI will not be the only thing impacted, but also code using other languages, async libraries, etc. It is possible that there is a misunderstanding here, but with the concern about potential correctness issues, want to raise this earlier rather than later. Is there something to be concerned about here ? Thanks, Rich On 3/30/11 11:24 PM, "Bill Long" <longb(a)cray.com> wrote: >Hi Rich and others, > > >On 3/30/11 11:51 AM, Graham, Richard L. wrote: >> Dear member of the WG5 and J3 Fortran standardization committee, > >As I understand the core of this thread, the problem is that some >vendors don't actually implement anything for ASYNCHRONOUS because their >libraries do not support actual asynchronous I/O operations, which makes >the ASYNCHRONOUS semantics moot. If all of the vendors supported the >semantics as expected, there would be no need for any changes. > >> >> We recommend the following change of the first two paragraphs in this >> section of the Fortran 2008 standard: >> >> 5.3.4 ASYNCHRONOUS attribute >> >> 1 An entity with the ASYNCHRONOUS attribute is a variable that may be >> subject to asynchronous input/output or other asynchronous access >> by means other than Fortran. >> > >The "or other asynchronous access be means other than Fortran" seems >semantically identical to "an object may be referenced, defined, or >become undefined, by means not specified by the program.". The second >phrase is, of course, the definition of the VOLATILE attribute. The >proposed change, then, seems to have the effect of making ASYNCHRONOUS >the same as VOLATILE. Some comments on that possibility: > >1) It is probably true that some vendors implement ASYNCHRONOUS that way >already. > >2) The vendors that are more precise in their ASYNCHRONOUS >implementation would not be in favor of this change. > >3) If this is the only solution, then using VOLATILE directly, and >forgetting about ASYNCHRONOUS, would be a solution for the MPI >non-blocking transfer routines. The implementation of VOLATILE is >likely more uniform and widespread. > > >> This should be treated only as a wording proposal. We provide the >> above text only as a suggestion; J3/WG5 are certainly free to adjust >> it as they feel necessary. The key idea is that we need ASYNCHRONOUS >> to also include potential memory modifications from agents outside the >> scope of the Fortran standard. >> > >Right. Sounds like VOLATILE to me. > >> The following snipit is a correct code that allows the MPI library to >> internally use DMA, communications offload (e.g., to a NIC), or other >> method to asynchronously communicate parts of the array while the user >> application/main CPU simultaneously reads and writes to ***another*** >> part of the same array. Specifically, this code snipit is is free of >> race conditions. >> >> USE mpi_f08 >> REAL, ASYNCHRONOUS :: buf(100,100) >> ! communicating a boundary of the array: >> CALL MPI_Irecv(buf(1,1:100),...req,...) >> DO j=1,100 >> DO i=2,100 >> buf(i,j)=.... ! work only on the inner area >> END DO >> END DO >> CALL MPI_Wait(req,...) >> > >It is not clear to me how this program example would show any difference >in performance or semantics if ASYNCHRONOUS were replaced by VOLATILE. >Even with the current definitions of the attributes, it is extremely >unlikely that a compiler applies the asynchronous attribute on an >element by element basis for the array buf. > > >> >> With the current status of Fortran 2008 + TR 29113 (Version N1845), a >> compiler can ignore the ASYNCHRONOUS attribute. For example, a >> compiler may choose to not implement Fortran asynchronous I/O in an >> asynchronous way, meaning that there is no need for the compiler to >> make any restrictions on optimization. >> >> To be clear: with the current Fortran 2008, this optimization is still >> allowed. With our proposal, this optimization will be prohibited. >> >> We want to mention that the use of VOLATILE would solve the problem, >> but at a high performance price because it would also prohibit any >> optimization of the "work" part (e.g., automatic cache optimization, >> instruction reordering, etc.). Especially since Fortran is used with >> MPI in high-performance, numerically intensive applications, we feel >> that VOLATILE cannot be recommended as a solution. >> > >So, it appears that you are saying that the compiler cannot make the >element-by-element distinction for the VOLATILE attribute, but it could >for ASYNCHRONOUS (in order for your example to optimize the way you >hope). Seems like a weak point in the argument. > >I agree that VOLATILE has the issues you state. But I don't see how the >new definition of ASYNCHRONOUS avoids the same issues. > >The MPI standard can state what is required of compiler implementations >for them to be compatible with MPI. I think a more direct solution >would be to require that the compiler vendor supports the ASYNCHRONOUS >semantics if it is to be used with MPI. > >Cheers, >Bill > > > >-- >Bill Long longb(a)cray.com >Fortran Technical Support & voice: 651-605-9024 >Bioinformatics Software Development fax: 651-605-9142 >Cray Inc./Cray Plaza, Suite 210/380 Jackson St./St. Paul, MN 55101 > >

1 0

slides from the Forum
by Jeff Squyres 31 Mar '11

31 Mar '11

Please send me any slides that were presented this week. Thanks. -- Jeff Squyres jsquyres(a)cisco.com For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/

1 0

MPI_Count PDF
by Jeff Squyres 30 Mar '11

30 Mar '11

Short version: 1. Bill pointed out the latex error in the MPI_Count latex in about .3 microseconds 2. Bill said he'll think about how to make PDFs that only include markup for a single ticket More details: No names mentioned to protect the guilty who made the 2nd half of the MPI_Count document entirely red, but the culprit was someone whose name rhymes with "Kreff." If anyone cares, the https://svn.mpi-forum.org/svn/mpi-forum-docs/trunk/working-groups/mpi-3/bw-… tree now compiles into a proper document with (mostly) correct markup for #265 (aside from some weirdness around page breaks sometimes). I can supply a proper PDF if anyone wants it. As noted in the meeting today, we have enough comments from the Forum to reset the clock. We'll be working on those items over the next few weeks and will have another document soon. -- Jeff Squyres jsquyres(a)cisco.com For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/

1 0

Comment on Fortran WG5 ballot N1846
by Graham, Richard L. 30 Mar '11

30 Mar '11

Dear member of the WG5 and J3 Fortran standardization committee, First, we want to thank you for your effort in defining the TR 29113 as an important key to solve the problems between MPI and Fortran. This is a comment from the MPI Forum on your WG5 letter ballot on N1845 in ftp://ftp.nag.co.uk/sc22wg5/N1801-N1850/N1846.txt and http://www.nag.co.uk/sc22wg5/. After reading your N1845 working draft from March 3, 2011, we detected one major gap: there is no standardized possibility of using nonblocking features of MPI in Fortran programs to overlap computation with communication. Specifically, Fortran MPI applications will be prohibited from initiating any form of asynchronous communication and then continuing to perform local computations while MPI advances the communication "in the background." This is a key issue for us because nonblocking communication is a critical technique for deadlock-free communication and high performance performance. An easy use case to describe is one that is popular in current high performance computing environments: when the MPI library utilizes hardware offload technologies (such as RDMA) with a communication co-processor (such as a NIC). The problem is, in principle, independent of MPI. It also occurs with libc/POSIX asynchronous IO (aio), and is therefore a topic that should be solved by the "TR on further interoperability of Fortran with C." We noticed that there is already a decision about this issue: to extend the meaning of the ASYNCHRONOUS attribute rather than to define a new attribute. http://www.j3-fortran.org/doc/year/09/09-235r2.txt contains the minutes from the J3 meeting, May 4-8, 2009. On this topic, the following vote was recorded: Paper 09-231 "Answer to MPI Forum regarding MPI asynchronous operations" [Rasmussen] discussed how to prevent code-motion and copy-in/out: SV: extend ASYNCHRONOUS - invent new attribute - undefined: 8-2-3 First, we would like to propose how to solve the problem, then we will present a short example to illustrate the specific problem. We recommend the following change of the first two paragraphs in this section of the Fortran 2008 standard: 5.3.4 ASYNCHRONOUS attribute 1 An entity with the ASYNCHRONOUS attribute is a variable that may be subject to asynchronous input/output or other asynchronous access by means other than Fortran. Asynchronous input/output can be any asynchronous access to the variable or parts of it, potentially within the scope of this standard by Fortran asynchronous I/O or possibly via methods outside the scope of this standard, such as the libc/POSIX asynchronous IO (aio) or nonblocking message passing, one-sided communication, or nonblocking parallel I/O as part of the Message Passing Interface (MPI) standard. 2 The base object of a variable shall have the ASYNCHRONOUS attribute in a scoping unit if - the variable appears in an executable statement or specification expression in that scoping unit and - any statement of the scoping unit is executed while the variable is a pending I/O storage sequence affector (9.6.2.5) or any other pending asynchronous access by means other than Fortran. This should be treated only as a wording proposal. We provide the above text only as a suggestion; J3/WG5 are certainly free to adjust it as they feel necessary. The key idea is that we need ASYNCHRONOUS to also include potential memory modifications from agents outside the scope of the Fortran standard. The following snipit is a correct code that allows the MPI library to internally use DMA, communications offload (e.g., to a NIC), or other method to asynchronously communicate parts of the array while the user application/main CPU simultaneously reads and writes to ***another*** part of the same array. Specifically, this code snipit is is free of race conditions. USE mpi_f08 REAL, ASYNCHRONOUS :: buf(100,100) ! communicating a boundary of the array: CALL MPI_Irecv(buf(1,1:100),...req,...) DO j=1,100 DO i=2,100 buf(i,j)=.... ! work only on the inner area END DO END DO CALL MPI_Wait(req,...) It is important that the compiler is ***not*** allowed to translate this program (by using temporary memory modifications) into something like this: USE mpi_f08 REAL, ASYNCHRONOUS :: buf(100,100), buf_1dim(10000) EQUIVALENCE (buf(1,1), buf_1dim(1)) CALL MPI_Irecv(buf(1,1:100),...req,...) tmp(1:100)=buf(1,1:100) ! saving the boundary DO k=1,10000 buf_1dim(k)=... ! work on the whole array END DO buf(1,1:100)=tmp(1:100) ! restoring the boundary CALL MPI_Wait(req,...) While the MPI library receives buf(1,1:100) "in the background", the "work" part overwrites this part of the array as part of a numerical optimization to achieve one long loop instead of a 2-loop-nesting. With the current status of Fortran 2008 + TR 29113 (Version N1845), a compiler can ignore the ASYNCHRONOUS attribute. For example, a compiler may choose to not implement Fortran asynchronous I/O in an asynchronous way, meaning that there is no need for the compiler to make any restrictions on optimization. To be clear: with the current Fortran 2008, this optimization is still allowed. With our proposal, this optimization will be prohibited. We want to mention that the use of VOLATILE would solve the problem, but at a high performance price because it would also prohibit any optimization of the "work" part (e.g., automatic cache optimization, instruction reordering, etc.). Especially since Fortran is used with MPI in high-performance, numerically intensive applications, we feel that VOLATILE cannot be recommended as a solution. In the era of multi-core hardware and vector instructions on each CPU, nonblocking communication must be allowed to overlap with efficient numerical code written in Fortran in order to achieve scalable, highly-performance applications. Your vote in J3 gave us the confidence that you can solve this problem within your TR 29113. In short, if we were members of WG5, we would answer the ballot with "Yes, but I recommend the following changes..." with the proposal described above. We appreciate the fact that you have asked for external comments on N1845. We hope that our comments above are sufficient information, and also hope that this letter conveys the critical importance with which we consider this issue. Many thanks for your time.

1 0

Re: [Mpi-forum] Confusion about new 1 sided
by Rajeev Thakur 28 Mar '11

28 Mar '11

During the reading, let's look at specific cases that we can fix. Rajeev On Mar 28, 2011, at 1:41 PM, Bronis R. de Supinski wrote: > > Resending so Rich does not have to approve my post from my phone... > > On Mon, 28 Mar 2011, de Supinski, Bronis R. wrote: > >> >> I think using black text for moved text is a bad idea. I think it should be shown as an addition in the new location and as a deletion in the old. Perhaps comments could be added to indicate the move action but no indication makes checking difficult. >> >> That said, I am reading the whole chapter as if it is all new. Our users do not use the existing interface so my concern is whether the overall interface will change that. >> >> I will send my new comments that I did on the plane when I am not sending from my phone. >> >> >> ----- Original Message ----- >> From: Jeff Squyres (jsquyres) [mailto:[email protected]] >> Sent: Monday, March 28, 2011 09:38 AM >> To: Main MPI Forum mailing list <mpi-forum(a)lists.mpi-forum.org> >> Cc: Main MPI Forum mailing list <mpi-forum(a)lists.mpi-forum.org> >> Subject: Re: [Mpi-forum] Confusion about new 1 sided >> >> Ok, thanks. >> >> Sent from my phone. No type good. >> >> On Mar 28, 2011, at 11:33 AM, "Rajeev Thakur" <thakur(a)mcs.anl.gov> wrote: >> >>> Jeff, >>> I printed out the old and the new, and as far as I can tell, everything is there, but there are so many changes that things are not exactly in the same spot. >>> >>> Rajeev >>> >>> >>> On Mar 28, 2011, at 9:19 AM, Rajeev Thakur wrote: >>> >>>> Text has been moved around in the chapter, so green text may not appear in exactly the same location in 2.2. >>>> >>>> >>>> On Mar 28, 2011, at 9:13 AM, Jeff Squyres (jsquyres) wrote: >>>> >>>>> ...and green text that is not in 2.2. >>>>> >>>>> Did the wrong document get posted? >>>>> >>>>> Sent from my phone. No type good. >>>>> >>>>> On Mar 28, 2011, at 10:07 AM, "Rajeev Thakur" <thakur(a)mcs.anl.gov> wrote: >>>>> >>>>>> It should be that green=old text to be deleted, red=new text. What specific case are your referring to? >>>>>> >>>>>> Rajeev >>>>>> >>>>>> >>>>>> On Mar 28, 2011, at 8:39 AM, Jeff Squyres (jsquyres) wrote: >>>>>> >>>>>>> I'm tryin to read the new doc that Bill sent around and it does not seem to be comparable to the 2.2 doc. Ie, the changes anddeletions highlighted in red and green do not appear in the actual 2.2 text. >>>>>>> >>>>>>> So what exactly are the red and green in this document? What is being added anddeleted from the original 2.2 doc? >>>>>>> >>>>>>> Sent from my phone. No type good. >>>>>>> _______________________________________________ >>>>>>> mpi-forum mailing list >>>>>>> mpi-forum(a)lists.mpi-forum.org >>>>>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum >>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> mpi-forum mailing list >>>>>> mpi-forum(a)lists.mpi-forum.org >>>>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum >>>>> >>>>> _______________________________________________ >>>>> mpi-forum mailing list >>>>> mpi-forum(a)lists.mpi-forum.org >>>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum >>>> >>> >>> >>> _______________________________________________ >>> mpi-forum mailing list >>> mpi-forum(a)lists.mpi-forum.org >>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum >> >> _______________________________________________ >> mpi-forum mailing list >> mpi-forum(a)lists.mpi-forum.org >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum >> > _______________________________________________ > mpi-forum mailing list > mpi-forum(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum

1 0

Re: [Mpi-forum] Confusion about new 1 sided
by Graham, Richard L. 28 Mar '11

28 Mar '11

:-) On 3/28/11 2:41 PM, "Bronis R. de Supinski" <bronis(a)llnl.gov> wrote: > >Resending so Rich does not have to approve my post from my phone... > >On Mon, 28 Mar 2011, de Supinski, Bronis R. wrote: > >> >> I think using black text for moved text is a bad idea. I think it >>should be shown as an addition in the new location and as a deletion in >>the old. Perhaps comments could be added to indicate the move action but >>no indication makes checking difficult. >> >> That said, I am reading the whole chapter as if it is all new. Our >>users do not use the existing interface so my concern is whether the >>overall interface will change that. >> >> I will send my new comments that I did on the plane when I am not >>sending from my phone. >> >> >> ----- Original Message ----- >> From: Jeff Squyres (jsquyres) [mailto:[email protected]] >> Sent: Monday, March 28, 2011 09:38 AM >> To: Main MPI Forum mailing list <mpi-forum(a)lists.mpi-forum.org> >> Cc: Main MPI Forum mailing list <mpi-forum(a)lists.mpi-forum.org> >> Subject: Re: [Mpi-forum] Confusion about new 1 sided >> >> Ok, thanks. >> >> Sent from my phone. No type good. >> >> On Mar 28, 2011, at 11:33 AM, "Rajeev Thakur" <thakur(a)mcs.anl.gov> >>wrote: >> >>> Jeff, >>> I printed out the old and the new, and as far as I can tell, >>>everything is there, but there are so many changes that things are not >>>exactly in the same spot. >>> >>> Rajeev >>> >>> >>> On Mar 28, 2011, at 9:19 AM, Rajeev Thakur wrote: >>> >>>> Text has been moved around in the chapter, so green text may not >>>>appear in exactly the same location in 2.2. >>>> >>>> >>>> On Mar 28, 2011, at 9:13 AM, Jeff Squyres (jsquyres) wrote: >>>> >>>>> ...and green text that is not in 2.2. >>>>> >>>>> Did the wrong document get posted? >>>>> >>>>> Sent from my phone. No type good. >>>>> >>>>> On Mar 28, 2011, at 10:07 AM, "Rajeev Thakur" <thakur(a)mcs.anl.gov> >>>>>wrote: >>>>> >>>>>> It should be that green=old text to be deleted, red=new text. What >>>>>>specific case are your referring to? >>>>>> >>>>>> Rajeev >>>>>> >>>>>> >>>>>> On Mar 28, 2011, at 8:39 AM, Jeff Squyres (jsquyres) wrote: >>>>>> >>>>>>> I'm tryin to read the new doc that Bill sent around and it does >>>>>>>not seem to be comparable to the 2.2 doc. Ie, the changes >>>>>>>anddeletions highlighted in red and green do not appear in the >>>>>>>actual 2.2 text. >>>>>>> >>>>>>> So what exactly are the red and green in this document? What is >>>>>>>being added anddeleted from the original 2.2 doc? >>>>>>> >>>>>>> Sent from my phone. No type good. >>>>>>> _______________________________________________ >>>>>>> mpi-forum mailing list >>>>>>> mpi-forum(a)lists.mpi-forum.org >>>>>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum >>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> mpi-forum mailing list >>>>>> mpi-forum(a)lists.mpi-forum.org >>>>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum >>>>> >>>>> _______________________________________________ >>>>> mpi-forum mailing list >>>>> mpi-forum(a)lists.mpi-forum.org >>>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum >>>> >>> >>> >>> _______________________________________________ >>> mpi-forum mailing list >>> mpi-forum(a)lists.mpi-forum.org >>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum >> >> _______________________________________________ >> mpi-forum mailing list >> mpi-forum(a)lists.mpi-forum.org >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum >> >_______________________________________________ >mpi-forum mailing list >mpi-forum(a)lists.mpi-forum.org >http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum

1 0

Re: [Mpi-forum] Confusion about new 1 sided
by Bronis R. de Supinski 28 Mar '11

28 Mar '11

Resending so Rich does not have to approve my post from my phone... On Mon, 28 Mar 2011, de Supinski, Bronis R. wrote: > > I think using black text for moved text is a bad idea. I think it should be shown as an addition in the new location and as a deletion in the old. Perhaps comments could be added to indicate the move action but no indication makes checking difficult. > > That said, I am reading the whole chapter as if it is all new. Our users do not use the existing interface so my concern is whether the overall interface will change that. > > I will send my new comments that I did on the plane when I am not sending from my phone. > > > ----- Original Message ----- > From: Jeff Squyres (jsquyres) [mailto:[email protected]] > Sent: Monday, March 28, 2011 09:38 AM > To: Main MPI Forum mailing list <mpi-forum(a)lists.mpi-forum.org> > Cc: Main MPI Forum mailing list <mpi-forum(a)lists.mpi-forum.org> > Subject: Re: [Mpi-forum] Confusion about new 1 sided > > Ok, thanks. > > Sent from my phone. No type good. > > On Mar 28, 2011, at 11:33 AM, "Rajeev Thakur" <thakur(a)mcs.anl.gov> wrote: > >> Jeff, >> I printed out the old and the new, and as far as I can tell, everything is there, but there are so many changes that things are not exactly in the same spot. >> >> Rajeev >> >> >> On Mar 28, 2011, at 9:19 AM, Rajeev Thakur wrote: >> >>> Text has been moved around in the chapter, so green text may not appear in exactly the same location in 2.2. >>> >>> >>> On Mar 28, 2011, at 9:13 AM, Jeff Squyres (jsquyres) wrote: >>> >>>> ...and green text that is not in 2.2. >>>> >>>> Did the wrong document get posted? >>>> >>>> Sent from my phone. No type good. >>>> >>>> On Mar 28, 2011, at 10:07 AM, "Rajeev Thakur" <thakur(a)mcs.anl.gov> wrote: >>>> >>>>> It should be that green=old text to be deleted, red=new text. What specific case are your referring to? >>>>> >>>>> Rajeev >>>>> >>>>> >>>>> On Mar 28, 2011, at 8:39 AM, Jeff Squyres (jsquyres) wrote: >>>>> >>>>>> I'm tryin to read the new doc that Bill sent around and it does not seem to be comparable to the 2.2 doc. Ie, the changes anddeletions highlighted in red and green do not appear in the actual 2.2 text. >>>>>> >>>>>> So what exactly are the red and green in this document? What is being added anddeleted from the original 2.2 doc? >>>>>> >>>>>> Sent from my phone. No type good. >>>>>> _______________________________________________ >>>>>> mpi-forum mailing list >>>>>> mpi-forum(a)lists.mpi-forum.org >>>>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum >>>>> >>>>> >>>>> _______________________________________________ >>>>> mpi-forum mailing list >>>>> mpi-forum(a)lists.mpi-forum.org >>>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum >>>> >>>> _______________________________________________ >>>> mpi-forum mailing list >>>> mpi-forum(a)lists.mpi-forum.org >>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum >>> >> >> >> _______________________________________________ >> mpi-forum mailing list >> mpi-forum(a)lists.mpi-forum.org >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum > > _______________________________________________ > mpi-forum mailing list > mpi-forum(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum >

1 0

Re: [Mpi-forum] Confusion about new 1 sided
by Rajeev Thakur 28 Mar '11

28 Mar '11

Cool. Thanks for the careful read. There will likely be several ticket 0 issues (Bronis has many). Rajeev On Mar 28, 2011, at 11:46 AM, Jeff Squyres (jsquyres) wrote: > FWIW, I have found many ticket 0 quality issues and I have a lot of questions, but the biggest mistake I four so far is that the fortran binding for fetch and op on page 21 is very wrong (prob cut n paste error). > > Sent from my phone. No type good. > > On Mar 28, 2011, at 11:40 AM, "Jeff Squyres (jsquyres)" <jsquyres(a)cisco.com> wrote: > >> Ok, thanks. >> >> Sent from my phone. No type good. >> >> On Mar 28, 2011, at 11:33 AM, "Rajeev Thakur" <thakur(a)mcs.anl.gov> wrote: >> >>> Jeff, >>> I printed out the old and the new, and as far as I can tell, everything is there, but there are so many changes that things are not exactly in the same spot. >>> >>> Rajeev >>> >>> >>> On Mar 28, 2011, at 9:19 AM, Rajeev Thakur wrote: >>> >>>> Text has been moved around in the chapter, so green text may not appear in exactly the same location in 2.2. >>>> >>>> >>>> On Mar 28, 2011, at 9:13 AM, Jeff Squyres (jsquyres) wrote: >>>> >>>>> ...and green text that is not in 2.2. >>>>> >>>>> Did the wrong document get posted? >>>>> >>>>> Sent from my phone. No type good. >>>>> >>>>> On Mar 28, 2011, at 10:07 AM, "Rajeev Thakur" <thakur(a)mcs.anl.gov> wrote: >>>>> >>>>>> It should be that green=old text to be deleted, red=new text. What specific case are your referring to? >>>>>> >>>>>> Rajeev >>>>>> >>>>>> >>>>>> On Mar 28, 2011, at 8:39 AM, Jeff Squyres (jsquyres) wrote: >>>>>> >>>>>>> I'm tryin to read the new doc that Bill sent around and it does not seem to be comparable to the 2.2 doc. Ie, the changes anddeletions highlighted in red and green do not appear in the actual 2.2 text. >>>>>>> >>>>>>> So what exactly are the red and green in this document? What is being added anddeleted from the original 2.2 doc? >>>>>>> >>>>>>> Sent from my phone. No type good. >>>>>>> _______________________________________________ >>>>>>> mpi-forum mailing list >>>>>>> mpi-forum(a)lists.mpi-forum.org >>>>>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum >>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> mpi-forum mailing list >>>>>> mpi-forum(a)lists.mpi-forum.org >>>>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum >>>>> >>>>> _______________________________________________ >>>>> mpi-forum mailing list >>>>> mpi-forum(a)lists.mpi-forum.org >>>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum >>>> >>> >>> >>> _______________________________________________ >>> mpi-forum mailing list >>> mpi-forum(a)lists.mpi-forum.org >>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum >> >> _______________________________________________ >> mpi-forum mailing list >> mpi-forum(a)lists.mpi-forum.org >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum > > _______________________________________________ > mpi-forum mailing list > mpi-forum(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum

1 0

Re: [Mpi-forum] Confusion about new 1 sided
by Jeff Squyres (jsquyres) 28 Mar '11

28 Mar '11

FWIW, I have found many ticket 0 quality issues and I have a lot of questions, but the biggest mistake I four so far is that the fortran binding for fetch and op on page 21 is very wrong (prob cut n paste error). Sent from my phone. No type good. On Mar 28, 2011, at 11:40 AM, "Jeff Squyres (jsquyres)" <jsquyres(a)cisco.com> wrote: > Ok, thanks. > > Sent from my phone. No type good. > > On Mar 28, 2011, at 11:33 AM, "Rajeev Thakur" <thakur(a)mcs.anl.gov> wrote: > >> Jeff, >> I printed out the old and the new, and as far as I can tell, everything is there, but there are so many changes that things are not exactly in the same spot. >> >> Rajeev >> >> >> On Mar 28, 2011, at 9:19 AM, Rajeev Thakur wrote: >> >>> Text has been moved around in the chapter, so green text may not appear in exactly the same location in 2.2. >>> >>> >>> On Mar 28, 2011, at 9:13 AM, Jeff Squyres (jsquyres) wrote: >>> >>>> ...and green text that is not in 2.2. >>>> >>>> Did the wrong document get posted? >>>> >>>> Sent from my phone. No type good. >>>> >>>> On Mar 28, 2011, at 10:07 AM, "Rajeev Thakur" <thakur(a)mcs.anl.gov> wrote: >>>> >>>>> It should be that green=old text to be deleted, red=new text. What specific case are your referring to? >>>>> >>>>> Rajeev >>>>> >>>>> >>>>> On Mar 28, 2011, at 8:39 AM, Jeff Squyres (jsquyres) wrote: >>>>> >>>>>> I'm tryin to read the new doc that Bill sent around and it does not seem to be comparable to the 2.2 doc. Ie, the changes anddeletions highlighted in red and green do not appear in the actual 2.2 text. >>>>>> >>>>>> So what exactly are the red and green in this document? What is being added anddeleted from the original 2.2 doc? >>>>>> >>>>>> Sent from my phone. No type good. >>>>>> _______________________________________________ >>>>>> mpi-forum mailing list >>>>>> mpi-forum(a)lists.mpi-forum.org >>>>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum >>>>> >>>>> >>>>> _______________________________________________ >>>>> mpi-forum mailing list >>>>> mpi-forum(a)lists.mpi-forum.org >>>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum >>>> >>>> _______________________________________________ >>>> mpi-forum mailing list >>>> mpi-forum(a)lists.mpi-forum.org >>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum >>> >> >> >> _______________________________________________ >> mpi-forum mailing list >> mpi-forum(a)lists.mpi-forum.org >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum > > _______________________________________________ > mpi-forum mailing list > mpi-forum(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum

1 0

Re: [Mpi-forum] Confusion about new 1 sided
by Jeff Squyres (jsquyres) 28 Mar '11

28 Mar '11

Ok, thanks. Sent from my phone. No type good. On Mar 28, 2011, at 11:33 AM, "Rajeev Thakur" <thakur(a)mcs.anl.gov> wrote: > Jeff, > I printed out the old and the new, and as far as I can tell, everything is there, but there are so many changes that things are not exactly in the same spot. > > Rajeev > > > On Mar 28, 2011, at 9:19 AM, Rajeev Thakur wrote: > >> Text has been moved around in the chapter, so green text may not appear in exactly the same location in 2.2. >> >> >> On Mar 28, 2011, at 9:13 AM, Jeff Squyres (jsquyres) wrote: >> >>> ...and green text that is not in 2.2. >>> >>> Did the wrong document get posted? >>> >>> Sent from my phone. No type good. >>> >>> On Mar 28, 2011, at 10:07 AM, "Rajeev Thakur" <thakur(a)mcs.anl.gov> wrote: >>> >>>> It should be that green=old text to be deleted, red=new text. What specific case are your referring to? >>>> >>>> Rajeev >>>> >>>> >>>> On Mar 28, 2011, at 8:39 AM, Jeff Squyres (jsquyres) wrote: >>>> >>>>> I'm tryin to read the new doc that Bill sent around and it does not seem to be comparable to the 2.2 doc. Ie, the changes anddeletions highlighted in red and green do not appear in the actual 2.2 text. >>>>> >>>>> So what exactly are the red and green in this document? What is being added anddeleted from the original 2.2 doc? >>>>> >>>>> Sent from my phone. No type good. >>>>> _______________________________________________ >>>>> mpi-forum mailing list >>>>> mpi-forum(a)lists.mpi-forum.org >>>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum >>>> >>>> >>>> _______________________________________________ >>>> mpi-forum mailing list >>>> mpi-forum(a)lists.mpi-forum.org >>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum >>> >>> _______________________________________________ >>> mpi-forum mailing list >>> mpi-forum(a)lists.mpi-forum.org >>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum >> > > > _______________________________________________ > mpi-forum mailing list > mpi-forum(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum

1 0