March 2011 - mpiwg-rma - mailman3.cels.anl.gov

Re: [Mpi3-rma] RMA chapter for quick review
by Bronis R. de Supinski 11 Mar '11

11 Mar '11

Torsten: Re: >> [1, 35-38]: In the list: "MPI_PUT (remote write), MPI_GET (remote read), >> [and] MPI_ACCUMULATE (remote update), MPI_GET_ACCUMULATE, >> MPI_FETCH_AND_OP >> (remote read and update), MPI_COMPARE_AND_SWAP (remote atomic swap >> operations), MPI_RPUT, MPI_RGET, MPI_RACCUMULATE and >> MPI_RGET_ACCUMULATE"; >> it is odd that a subset of the operations have parenthetical expressions >> that indicate the nature of the functionality. Either all operations >> should be followed with a parenthetical explanation or none of them. >> The right solution is probably to reformat into a list with each >> item being the group of operations that provide functionality of the >> same nature (e.g., I am guessing that MPI_GET and MPI_RGET are both >> remote read operations). Initially I thought the ones that are not >> followed by a parenthetical expression were being grouped with the >> following one that has the parenthetical expression (so MPI_GET_ACCUMULATE >> is a remote read and update operation). However, the list ends with >> several operations with no following parenthetical expression. Anyway, >> the implicit grouping is unclear. Making it explicit would be better. >> It would also simplify how the following sentence ("When a reference is >> made to “accumulate” operations in this chapter, it refers to the >> following operations: MPI_ACCUMULATE, MPI_GET_ACCUMULATE, >> MPI_FETCH_AND_OP, >> MPI_COMPARE_AND_SWAP, MPI_RACCUMULATE and MPI_RGET_ACCUMULATE.") simpler. > Ok, I added the descriptions to the R-operations. Hmm. the more I thought about the more I though this would be better as a bulleted list, like the one about the types of collective operations. I think that instead of "The following communication calls are provided:" (etc.) something like this would be best: ---------------------------------------------------------------- One-sided operations can be classified as follows: * Remote read: MPI_GET, MPI_RGET; * Remote write: MPI_PUT, MPI_RPUT; * Remote update: MPI_ACCUMULATE, MPI_RACCUMULATE; * Remote read and update: MPI_GET_ACCUMULATE, MPI_FETCH_AND_OP, MPI_RGET_ACCUMULATE; * Remote atomic swap operations: MPI_COMPARE_AND_SWAP. This chapter refers to operations set that includes all remote update, remote read and update, and remote atomic swap operations as “accumulate” operations. ------------------------------------------------------------------- This version makes it much clearer that grouping is intended and makes the definition of accumulate operations simpler. It also makes it clearer that the ommission of an MPI_RCOMPARE_AND_SWAP operation is intentional. [snip] >> [2, 12] Use active voice: "However, support for asynchronous communication >> agents in software (handlers, threads, etc.) might be needed, for certain >> RMA functions, in a distributed memory environment." => "However, certain >> RMA functions might need support for asynchronous communication agents in >> software (handlers, threads, etc.) in a distributed memory environment." > I'll defer this one for now because it doesn't seem essential and it's a > macro-mess :). I'm not sure that I understand. > Generally, I fear that it will be a lot of work if we're trying to fix > all the little wording things in the OS chapter. Many of them (most of > the ones you found) have been there since MPI-2 and are in the places > that we didn't want to touch. It might be better to not apply such > changes at this point so that reviewers can focus on the semantic > changes (and not have colored sections all over the document). Hmm. I will need to read through the whole chapter in order to vote on the changes. In that process, I will hit these types of corrections. How would you prefer that we handle them? Bronis

1 0

Re: [Mpi3-rma] RMA chapter for quick review
by Rajeev Thakur 11 Mar '11

11 Mar '11

On Mar 11, 2011, at 2:41 PM, James Dinan wrote: > MPI supports two fundamentally dierent memory models: unified and > separate Good point. We should at least change the order in that sentence to "separate and unified" Rajeev

1 0

Re: [Mpi3-rma] RMA chapter for quick review
by James Dinan 11 Mar '11

11 Mar '11

On 03/11/2011 02:21 PM, Barrett, Brian W wrote: > On 3/11/11 12:10 PM, "Torsten Hoefler" <htor(a)illinois.edu> wrote: > >>> >>> Page and line numbers are from the marked-up copy, not the clean copy. >>> >>> Pg 1, ln 45/46 & pg 2, ln 1&2: The discussion of communication >>> operations >>> being delayed for the separate model and immediate for the unified model >>> is a bit confusing. When the implementation *starts* the communication >>> is >>> the same for both models, depending on the synchronization calls. The >>> only difference between the two models is when the private and public >>> copies on the target must be synchronized. As I read the standard, an >>> implantation is free to start a put immediately upon calling MPI_PUT in >>> the separate model and also free to start a put during MPI_UNLOCK in the >>> unified model. >> It's a very high-level view - we don't separate between completion and >> start at this point (in the intro). Both statements are "can" statements >> and not wrong. Do you have a concrete wording proposal that makes it >> less confusing? > > No, I wasn't suggesting that we do. My main concern was that we were > introducing the potential for someone to read a requirement where none > existed. To be honest, I think the best route is to remove the two > clauses: > > "; for efficiency, the implementation can delay communication operations > until the synchronization calls occur" and > "In this model, communication can be independent of synchronization > calls." > > Neither are strictly necessary for the point being made, and there's no > great way to describe the concept without more detail than makes sense for > an introduction. > > Thoughts from others? I don't know if this was already pointed out, but I think in that text on page 1 "first model" (unified) and "second model" (separate) are referring to the wrong things: "MPI supports two fundamentally dierent memory models: unified and separate. The *first model* makes no assumption about memory consistency and is highly portable. This model is similar to that of weakly coherent memory systems: correct ordering of memory accesses has to be imposed by the user, using synchronization calls; /for efficiency, the implementation can delay communication operations until the synchronization calls occur/. The *second model* can exploit cache coherent hardware and hardware-accelerated one-sided operations that are commonly available in high-performance systems. /In this model, communication can be independent of synchronization calls./ The two different models are discussed in detail in Section 11.4. A large number of synchronization calls are provided for both models to support dierent synchronization styles." We should be saying "the unified model" and "the separate model" to be certain we're referring to the correct model. I agree with Brian, the above statements seem out of place in the introduction. They seems more like an advice comment, should we decide we want to keep them. Also, do we want to say cache coherent in this paragraph? I thought we were trying to avoid that. ~Jim.

1 0

Re: [Mpi3-rma] RMA chapter for quick review
by Barrett, Brian W 11 Mar '11

11 Mar '11

On 3/11/11 12:10 PM, "Torsten Hoefler" <htor(a)illinois.edu> wrote: >> >> Page and line numbers are from the marked-up copy, not the clean copy. >> >> Pg 1, ln 45/46 & pg 2, ln 1&2: The discussion of communication >>operations >> being delayed for the separate model and immediate for the unified model >> is a bit confusing. When the implementation *starts* the communication >>is >> the same for both models, depending on the synchronization calls. The >> only difference between the two models is when the private and public >> copies on the target must be synchronized. As I read the standard, an >> implantation is free to start a put immediately upon calling MPI_PUT in >> the separate model and also free to start a put during MPI_UNLOCK in the >> unified model. >It's a very high-level view - we don't separate between completion and >start at this point (in the intro). Both statements are "can" statements >and not wrong. Do you have a concrete wording proposal that makes it >less confusing? No, I wasn't suggesting that we do. My main concern was that we were introducing the potential for someone to read a requirement where none existed. To be honest, I think the best route is to remove the two clauses: "; for efficiency, the implementation can delay communication operations until the synchronization calls occur" and "In this model, communication can be independent of synchronization calls." Neither are strictly necessary for the point being made, and there's no great way to describe the concept without more detail than makes sense for an introduction. Thoughts from others? Brian -- Brian W. Barrett Dept. 1423: Scalable System Software Sandia National Laboratories

1 0

Re: [Mpi3-rma] RMA chapter for quick review
by Torsten Hoefler 11 Mar '11

11 Mar '11

Bronis, Thanks for your review! Comments inline below. >> An updated version of the RMA chapter is attached to this mail and on >> the wiki. We would like to make it available to the Forum by tomorrow >> afternoon in order to meet the two-week window for the first reading at >> the March meeting. If you have a chance, please take a look at it and >> let us know your comments before tomorrow afternoon. > > I went through section 11.1 and found ticket 0 type changes. > I don't know when I will have time to finish so if I hit > something more significant I will just have to apologize. They'll all be rolled into the large ticket for changing the chapter. > Anyway, using [p#, l#] notation: > > [1, 20]: fix split infinitive: to access or update => to access or to update fixed > [1, 21-23]: I would like to remove the personification of the > processes in "However, processes may not know which data in their own > memory need to be accessed or updated by remote processes, and may not > even know the identity of these processes." Processes do not "know"; > however, I have not thought of good alternative wording yet. Perhaps: > "However, a process may not necessarily be able to compute what data > in its local memory that remote processes need to access or even the > identity of those processes." Your proposal still personifies the processes. I don't see a good alternative either, thus, I'll leave it unchanged for now. I'm open to suggestions! > [1, 26]: Clarify antecedent: This may => This distribution may fixed > [1, 27]: Fix split infinitives and dangling preposition: > > to periodically poll for potential communication requests to receive and > act upon. => > > to poll for potential communication requests to receive and upon which > to act periodically. fixed > [1, 35-38]: In the list: "MPI_PUT (remote write), MPI_GET (remote read), > [and] MPI_ACCUMULATE (remote update), MPI_GET_ACCUMULATE, > MPI_FETCH_AND_OP > (remote read and update), MPI_COMPARE_AND_SWAP (remote atomic swap > operations), MPI_RPUT, MPI_RGET, MPI_RACCUMULATE and > MPI_RGET_ACCUMULATE"; > it is odd that a subset of the operations have parenthetical expressions > that indicate the nature of the functionality. Either all operations > should be followed with a parenthetical explanation or none of them. > The right solution is probably to reformat into a list with each > item being the group of operations that provide functionality of the > same nature (e.g., I am guessing that MPI_GET and MPI_RGET are both > remote read operations). Initially I thought the ones that are not > followed by a parenthetical expression were being grouped with the > following one that has the parenthetical expression (so MPI_GET_ACCUMULATE > is a remote read and update operation). However, the list ends with > several operations with no following parenthetical expression. Anyway, > the implicit grouping is unclear. Making it explicit would be better. > It would also simplify how the following sentence ("When a reference is > made to “accumulate” operations in this chapter, it refers to the > following operations: MPI_ACCUMULATE, MPI_GET_ACCUMULATE, > MPI_FETCH_AND_OP, > MPI_COMPARE_AND_SWAP, MPI_RACCUMULATE and MPI_RGET_ACCUMULATE.") simpler. Ok, I added the descriptions to the R-operations. > [1, 44-45] Use active voice: correct ordering of memory accesses has to > be imposed by the user, using synchronization calls; => the user must > impose correct ordering of memory accesses through synchronization calls; fixed > [2, 8] Change: "operations, communication coprocessors, etc." to > "operations, and communication coprocessors." The clause begins > with "such as" so the "etc." is redundant (not to mention caused > the omission of the period that should end the sentence). fixed > [2, 12] Use active voice: "However, support for asynchronous communication > agents in software (handlers, threads, etc.) might be needed, for certain > RMA functions, in a distributed memory environment." => "However, certain > RMA functions might need support for asynchronous communication agents in > software (handlers, threads, etc.) in a distributed memory environment." I'll defer this one for now because it doesn't seem essential and it's a macro-mess :). Generally, I fear that it will be a lot of work if we're trying to fix all the little wording things in the OS chapter. Many of them (most of the ones you found) have been there since MPI-2 and are in the places that we didn't want to touch. It might be better to not apply such changes at this point so that reviewers can focus on the semantic changes (and not have colored sections all over the document). I'll upload shortly! Thanks & All the Best, Torsten -- bash$ :(){ :|:&};: --------------------- http://www.unixer.de/ ----- Torsten Hoefler | Performance Modeling and Simulation Lead Blue Waters Directorate | University of Illinois (UIUC) 1205 W Clark Street | Urbana, IL, 61801 NCSA Building | +01 (217) 244-7736

1 0

Re: [Mpi3-rma] RMA chapter for quick review
by Torsten Hoefler 11 Mar '11

11 Mar '11

Hi Brian, Thanks for you review! > >An updated version of the RMA chapter is attached to this mail and on the > >wiki. We would like to make it available to the Forum by tomorrow > >afternoon in order to meet the two-week window for the first reading at > >the March meeting. If you have a chance, please take a look at it and let > >us know your comments before tomorrow afternoon. > > > >In particular: > > - We fixed some issues with the rules on pg 47 and 48. Please take a > >look at those rules in their entirety. > > - pg 3, ln 43, has been updated to specify that default value for the > >accumulate_ops key is same_op_no_op. > > Page and line numbers are from the marked-up copy, not the clean copy. > > Pg 1, ln 45/46 & pg 2, ln 1&2: The discussion of communication operations > being delayed for the separate model and immediate for the unified model > is a bit confusing. When the implementation *starts* the communication is > the same for both models, depending on the synchronization calls. The > only difference between the two models is when the private and public > copies on the target must be synchronized. As I read the standard, an > implantation is free to start a put immediately upon calling MPI_PUT in > the separate model and also free to start a put during MPI_UNLOCK in the > unified model. It's a very high-level view - we don't separate between completion and start at this point (in the intro). Both statements are "can" statements and not wrong. Do you have a concrete wording proposal that makes it less confusing? > Pg 2, ln 3&4: I think the last sentence might read a bit better as "A > large number of synchronization calls, which support both models, are > provided to support difference synchronization styles." But I'm open to > better wording :). fixed > Pg 26, ln 47: There's a word missing in the final sentence of the > paragraph. Perhaps: "The stronger semantics of the RMA unified model > allow the user to omit some synchronization calls and potentially improve > performance." fixed > Pg 50, ln 2: "synchronization in Process B", the p in process should be > lowercase. > Pg 50, ln 28: p in Process A should be lowercase > Pg 50, ln 29: p in Process B should be lowercase (twice) > Pg 51, ln 15: p in Process A should be lowercase fixed > Pg 51, ln 25: Should be "local store by process B before the lock" (insert > word process) > Pg 52, ln 21: insert word "process" at "allow B to read", "stored by A" > Pg 52, ln 24: reword sentence, should be "in the private copy of process A" > Pg 52, ln 24: "the update to Y", not "the update on Y" > Pg 52, ln 27: insert work process in "the value stored by A" > Pg 52, ln 32: insert "process" at "private copy of X at process B" > Pg 53, ln 13: can remove the comment, someone fixed the latex foo All fixed. > Pg 53, ln 42: The citation for the algorithm is missing (Torsten?) I removed the reference because the algorithm is very well known (google). I will put it up at the wiki soon! All the Best, Torsten -- bash$ :(){ :|:&};: --------------------- http://www.unixer.de/ ----- Torsten Hoefler | Performance Modeling and Simulation Lead Blue Waters Directorate | University of Illinois (UIUC) 1205 W Clark Street | Urbana, IL, 61801 NCSA Building | +01 (217) 244-7736

1 0

Re: [Mpi3-rma] RMA chapter for quick review
by Bronis R. de Supinski 11 Mar '11

11 Mar '11

Rajeev: Re: > An updated version of the RMA chapter is attached to this mail and on > the wiki. We would like to make it available to the Forum by tomorrow > afternoon in order to meet the two-week window for the first reading at > the March meeting. If you have a chance, please take a look at it and > let us know your comments before tomorrow afternoon. I went through section 11.1 and found ticket 0 type changes. I don't know when I will have time to finish so if I hit something more significant I will just have to apologize. Anyway, using [p#, l#] notation: [1, 20]: fix split infinitive: to access or update => to access or to update [1, 21-23]: I would like to remove the personification of the processes in "However, processes may not know which data in their own memory need to be accessed or updated by remote processes, and may not even know the identity of these processes." Processes do not "know"; however, I have not thought of good alternative wording yet. Perhaps: "However, a process may not necessarily be able to compute what data in its local memory that remote processes need to access or even the identity of those processes." [1, 26]: Clarify antecedent: This may => This distribution may [1, 27]: Fix split infinitives and dangling preposition: to periodically poll for potential communication requests to receive and act upon. => to poll for potential communication requests to receive and upon which to act periodically. [1, 35-38]: In the list: "MPI_PUT (remote write), MPI_GET (remote read), [and] MPI_ACCUMULATE (remote update), MPI_GET_ACCUMULATE, MPI_FETCH_AND_OP (remote read and update), MPI_COMPARE_AND_SWAP (remote atomic swap operations), MPI_RPUT, MPI_RGET, MPI_RACCUMULATE and MPI_RGET_ACCUMULATE"; it is odd that a subset of the operations have parenthetical expressions that indicate the nature of the functionality. Either all operations should be followed with a parenthetical explanation or none of them. The right solution is probably to reformat into a list with each item being the group of operations that provide functionality of the same nature (e.g., I am guessing that MPI_GET and MPI_RGET are both remote read operations). Initially I thought the ones that are not followed by a parenthetical expression were being grouped with the following one that has the parenthetical expression (so MPI_GET_ACCUMULATE is a remote read and update operation). However, the list ends with several operations with no following parenthetical expression. Anyway, the implicit grouping is unclear. Making it explicit would be better. It would also simplify how the following sentence ("When a reference is made to “accumulate” operations in this chapter, it refers to the following operations: MPI_ACCUMULATE, MPI_GET_ACCUMULATE, MPI_FETCH_AND_OP, MPI_COMPARE_AND_SWAP, MPI_RACCUMULATE and MPI_RGET_ACCUMULATE.") simpler. [1, 44-45] Use active voice: correct ordering of memory accesses has to be imposed by the user, using synchronization calls; => the user must impose correct ordering of memory accesses through synchronization calls; [2, 8] Change: "operations, communication coprocessors, etc." to "operations, and communication coprocessors." The clause begins with "such as" so the "etc." is redundant (not to mention caused the omission of the period that should end the sentence). [2, 12] Use active voice: "However, support for asynchronous communication agents in software (handlers, threads, etc.) might be needed, for certain RMA functions, in a distributed memory environment." => "However, certain RMA functions might need support for asynchronous communication agents in software (handlers, threads, etc.) in a distributed memory environment." Bronis > In particular: > - We fixed some issues with the rules on pg 47 and 48. Please take a look at those rules in their entirety. > - pg 3, ln 43, has been updated to specify that default value for the accumulate_ops key is same_op_no_op. > > Thanks, > Rajeev > >

1 0

Re: [Mpi3-rma] RMA chapter for quick review
by Barrett, Brian W 11 Mar '11

11 Mar '11

On 3/10/11 6:54 PM, "Rajeev Thakur" <thakur(a)mcs.anl.gov> wrote: >An updated version of the RMA chapter is attached to this mail and on the >wiki. We would like to make it available to the Forum by tomorrow >afternoon in order to meet the two-week window for the first reading at >the March meeting. If you have a chance, please take a look at it and let >us know your comments before tomorrow afternoon. > >In particular: > - We fixed some issues with the rules on pg 47 and 48. Please take a >look at those rules in their entirety. > - pg 3, ln 43, has been updated to specify that default value for the >accumulate_ops key is same_op_no_op. Page and line numbers are from the marked-up copy, not the clean copy. Pg 1, ln 45/46 & pg 2, ln 1&2: The discussion of communication operations being delayed for the separate model and immediate for the unified model is a bit confusing. When the implementation *starts* the communication is the same for both models, depending on the synchronization calls. The only difference between the two models is when the private and public copies on the target must be synchronized. As I read the standard, an implantation is free to start a put immediately upon calling MPI_PUT in the separate model and also free to start a put during MPI_UNLOCK in the unified model. Pg 2, ln 3&4: I think the last sentence might read a bit better as "A large number of synchronization calls, which support both models, are provided to support difference synchronization styles." But I'm open to better wording :). Pg 26, ln 47: There's a word missing in the final sentence of the paragraph. Perhaps: "The stronger semantics of the RMA unified model allow the user to omit some synchronization calls and potentially improve performance." Pg 50, ln 2: "synchronization in Process B", the p in process should be lowercase. Pg 50, ln 28: p in Process A should be lowercase Pg 50, ln 29: p in Process B should be lowercase (twice) Pg 51, ln 15: p in Process A should be lowercase Pg 51, ln 25: Should be "local store by process B before the lock" (insert word process) Pg 52, ln 21: insert word "process" at "allow B to read", "stored by A" Pg 52, ln 24: reword sentence, should be "in the private copy of process A" Pg 52, ln 24: "the update to Y", not "the update on Y" Pg 52, ln 27: insert work process in "the value stored by A" Pg 52, ln 32: insert "process" at "private copy of X at process B" Pg 53, ln 13: can remove the comment, someone fixed the latex foo Pg 53, ln 42: The citation for the algorithm is missing (Torsten?) -- Brian W. Barrett Dept. 1423: Scalable System Software Sandia National Laboratories

1 0

Re: [Mpi3-rma] RMA chapter for quick review
by James Dinan 10 Mar '11

10 Mar '11

Hi Rajeev, I'd like to get a copy of the linked list example in there too before we circulate it. Could we try with the current version that I sent to the list a couple days ago and see how well it fits? The comment at the top should be readily convertible into a caption for the example. Thanks, ~Jim. On 3/10/11 7:54 PM, Rajeev Thakur wrote: > An updated version of the RMA chapter is attached to this mail and on the wiki. We would like to make it available to the Forum by tomorrow afternoon in order to meet the two-week window for the first reading at the March meeting. If you have a chance, please take a look at it and let us know your comments before tomorrow afternoon. > > In particular: > - We fixed some issues with the rules on pg 47 and 48. Please take a look at those rules in their entirety. > - pg 3, ln 43, has been updated to specify that default value for the accumulate_ops key is same_op_no_op. > > Thanks, > Rajeev

1 0

[Mpi3-rma] RMA chapter for quick review
by Rajeev Thakur 10 Mar '11

10 Mar '11

An updated version of the RMA chapter is attached to this mail and on the wiki. We would like to make it available to the Forum by tomorrow afternoon in order to meet the two-week window for the first reading at the March meeting. If you have a chance, please take a look at it and let us know your comments before tomorrow afternoon. In particular: - We fixed some issues with the rules on pg 47 and 48. Please take a look at those rules in their entirety. - pg 3, ln 43, has been updated to specify that default value for the accumulate_ops key is same_op_no_op. Thanks, Rajeev

1 0