January 2009 - mpiwg-coll - mailman3.cels.anl.gov

[mpi3-coll] Comments on nonblocking collectives section
by Jesper Larsson Traeff 27 Jan '09

27 Jan '09

Dear all, sorry for my late input. I've had time to read the document, and have found something. I refer to the version of 21st January. Jesper p. 49, line 26: "the performance of many systems..." -> "the performance of many applications can be improved by overlapping communication and computation, and many systems enable this." p. 49, line 28: "utilize overlap -> "exploit overlap" p. 49, line 32: drop "the use of" (grammar: "the use of" is no mechanism) p.49, line 35: "a collective communication, which..." -> "a collective communication operation, that..." ("that" better than "which" here?) p.49, line 36: "the communication may progress" -> "the collective communication may then progress" p.49, line 38: "mitigate synchronizing effects" -> "mitigate synchronization effects" p. 50, line 9-11: drop sentence "Collective operations...", allready said in line 4-7? p.50, after line 18, add: "Nonblocking collective requests are not persistent, and MPI_REQUEST_NULL is returned after successful completion." p. 50, line 18: I still think that MPI_Request_free should be allowed. The rationale also still reads contrived. p.50, line 20 (or elsewhere), add: "Completion of a particular nonblocking collective operations does \emph{not} imply completion of any other posted nonblocking collective (or send-receive) operations, whether they are posted before or after the operation in case." p.50, line 24: "Canceling a request" -> "Canceling a nonblocking request" p.50, line 24-25: "cannot be defined clearly." -> "is not well-defined." p.50, line 33: "In this sense"?? What does this refer to? Drop, just say "Blocking and nonblocking..." p.50, line 35: "In particular," -> "This means that" p.50, line 44: "could find an equilibrium between" -> "could attempt to balance" p. 50, line 47-48: "than for point-to-point operations." -> "than for point-to-point operations, where for instance a blocking send can be matched by a nonblocking receive." p.50, line 48: "collective operations is required" -> "collective operation semantics is required" p.51, line 6 (or elsewhere), add: "In terms of data movements each nonblocking collective operation upon completion has the same effect as its blocking counterpart, both for intracommunicators and intercommunicators. The use of the "in place" option is allowed exactly as described for the corresponding blocking collective operations. Likewise, the nonblocking collective reduction operations upon completion have the same effect as their blocking counterparts, and the same restrictions and recommendations on reduction orders apply." I think this is quite important for the semantic definition of the nonblocking collectives. The individual remarks for each of the operations could be dropped, although I think it is good to have them there (for people just checking the one operation they are interested in). However, the English is in most of the individual cases not good, see below. p.51, line 26-30: Advice reads a bit awkward, but have no suggestion. p.53, line 37: "are identical as after a call" - bad Engish, should rather be: "are the same as after a call". I would strongly suggest using exactly the same phrase for all of the data moving collectives! This is not a document being read as a whole, so we don't need to fear "repetition". Perhaps add: "The "in place" option is allowed for intracommunicators, with the same effect as for MPI_Gather." p, 54, line 41: Language! p. 55, line 35: Same p. 56, line 34: Same p. 57, line 32: Same p. 58, line 26: Same p. 59, line 32: Same p. 60, line 28: Same p. 61, line 47: Reads sort of ok here, would still be in favor of using the same phrase everywhere. p. 62, line 35-39: Drop advice to users, but add "The same rules and restrictions as for MPI_Reduce apply." (consistency is no hobgoblin here, and I think it's important that the nonblocking collectives "does" the same as their blocking counterparts) p. 63, line 26: Language. p. 64, line 11: Sort of OK. p. 65, line 25: Sort of OK. p.68, line 21: does a race condition "cause" non-determinism, or is it the other way round? p. 68, line 26: move "outstanding" to "even if there is an outstanding nonblocking..." p. 69, after line 34: Would suggest an example 5.31 that shows that e.g. MPI_IAlltoall does not match MPI_Alltoall. "Blocking and nonblocking collective operations do not match. The following is illegal:" p.70, line 6: "the "MPI_Recv." -> "the MPI_Recv call." p. 70, line 42: "lower performance" (sounds too colloqial) -> "impact performance negatively." or "have a negative effect on performance."

1 0

Re: [mpi3-coll] Second January Telecon
by Torsten Hoefler 24 Jan '09

24 Jan '09

Hello Coll-WG, Our second teleconference this month is scheduled for Tuesday noon EST. The access coordinates are: Conf Id: 152891 Date: 1/27/2009 Begin Time: 12:00:00 PM Eastern Time (US & Canada) Phone Number: (812) 856-3600 Participant PIN: 002118# We will be discussing last issues with NBC #109 and if time, some MPI-2.2 things. The tentative agenda is at: https://svn.mpi-forum.org/trac/mpi-forum-web/wiki/Telecon012709 Thanks & Best, Torsten -- bash$ :(){ :|:&};: --------------------- http://www.unixer.de/ ----- Torsten Hoefler | Postdoctoral Researcher Open Systems Lab | Indiana University 150 S. Woodlawn Ave. | Bloomington, IN, 474045, USA Lindley Hall Room 135 | +01 (812) 855-3608

1 0

Re: [mpi3-coll] [Mpi-22] [MPI Forum] #96: MPI_OP_CREATE and associativity
by Torsten Hoefler 19 Jan '09

19 Jan '09

On Tue, Jan 13, 2009 at 09:23:20AM +0100, Jesper Larsson Traeff wrote: > maybe I misunderstand you, but I don't see that allowing operators to > be explicitly non-associative would give you more freedom for optimizations. > Doesn't your arguments above show that there are rather less than for > associative ops? But maybe it is better to discuss this at the next meeting? just for the other readers: we clarified the discussion during the last Collectives Workinggroup Teleconference. And decided to update ticket #96 and merge ticket #95 into it. For details see: https://svn.mpi-forum.org/trac/mpi-forum-web/ticket/96 > If your argument is: "there are natural (user-defined) operators in app > A, B and C that are mathematically not associative, and users want to perform > reductions with these" - then I agree that MPI is missing something. yes, exactly. Such operators are MPI_SUM of MPI_FLOAT and friends. > Maybe an "MPI_Op_create_nonassoc(...)"? (I also don't think an extra function > for setting the associativity of an already defined function fits with MPI; > although a function like MPI_Op_use_some_special_canonical_order(MPI_Op f), > where f could be a user as well as a predefined MPI_Op, > could perhaps make sense, and would allow to control the evaluation order > each time MPI_Reduce/MPI_Allreduce/... is called, but then such a function > would either have to be collective or users would be required to use it > consistently, ... that is, many complications) exactly, this is why the whole discussion was moved from the context of MPI-2.2 to MPI-3 into the collective working group. > Note also that non-associativity does put some non-trivial burden on > implementers: for non-associative (with or without hyphen?) operators, > a special algorithm that does the reduction in the sequential order is needed I don't see a big/non-trivial problem. I copied the collective working group list in order to take further discussion off the MPI-2.2 list. All the Best, Torsten -- bash$ :(){ :|:&};: --------------------- http://www.unixer.de/ ----- Torsten Hoefler | Postdoctoral Researcher Open Systems Lab | Indiana University 150 S. Woodlawn Ave. | Bloomington, IN, 474045, USA Lindley Hall Room 135 | +01 (812) 855-3608

1 0

Re: [mpi3-coll] Second January Telecon
by Richard Graham 16 Jan '09

16 Jan '09

The 28th conflicts with the FT working group call, but the 27th will work. Thanks, Rich On 1/16/09 2:52 AM, "Jesper Larsson Traeff" <traff(a)it.neclab.eu> wrote: > > > On Thu, Jan 15, 2009 at 05:33:51PM -0500, Torsten Hoefler wrote: >> > On Thu, Jan 15, 2009 at 01:49:25PM -0800, Bronis R. de Supinski wrote: >>> > > >>> > > Torsten: >>> > > >>> > > Re: >>>> > > > Jesper proposed a second January telecon for the 29th. However, I >>>> > > > couldn't attent on the 29th and would like to move it to the 28th >>>> > > > (noon). >>>> > > > >>>> > > > Does anybody object? We can also choose another time or day in this >>>> > > > week. >>> > > >>> > > I (probably) have a conflict on the 28th. The 27th is >>> > > more open for me... >> > 27th is fine with me - other opinions? >> > > 27th will probably also be fine with me > > Jesper > > _______________________________________________ > mpi3-coll mailing list > mpi3-coll(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-coll >

1 0

Re: [mpi3-coll] Second January Telecon
by Jesper Larsson Traeff 16 Jan '09

16 Jan '09

On Thu, Jan 15, 2009 at 05:33:51PM -0500, Torsten Hoefler wrote: > On Thu, Jan 15, 2009 at 01:49:25PM -0800, Bronis R. de Supinski wrote: > > > > Torsten: > > > > Re: > > > Jesper proposed a second January telecon for the 29th. However, I > > > couldn't attent on the 29th and would like to move it to the 28th > > > (noon). > > > > > > Does anybody object? We can also choose another time or day in this > > > week. > > > > I (probably) have a conflict on the 28th. The 27th is > > more open for me... > 27th is fine with me - other opinions? > 27th will probably also be fine with me Jesper

1 0

Re: [mpi3-coll] Second January Telecon
by Jesper Larsson Traeff 16 Jan '09

16 Jan '09

28th (same time) is also fine with me jesper On Thu, Jan 15, 2009 at 04:17:29PM -0500, Torsten Hoefler wrote: > Hello Coll-WG, > Jesper proposed a second January telecon for the 29th. However, I > couldn't attent on the 29th and would like to move it to the 28th > (noon). > > Does anybody object? We can also choose another time or day in this > week. > > Thanks & Best, > Torsten > > -- > bash$ :(){ :|:&};: --------------------- http://www.unixer.de/ ----- > Torsten Hoefler | Postdoctoral Researcher > Open Systems Lab | Indiana University > 150 S. Woodlawn Ave. | Bloomington, IN, 474045, USA > Lindley Hall Room 135 | +01 (812) 855-3608 > _______________________________________________ > mpi3-coll mailing list > mpi3-coll(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-coll

1 0

Re: [mpi3-coll] Updates after first reading
by Torsten Hoefler 15 Jan '09

15 Jan '09

Hello, thanks Bin and Bronis! I fixed it in the latest version. It now reads: "The use of tags for collective operations would prevent certain hardware-optimizations and omitting them is consistent with blocking collective operations." Best, Torsten -- bash$ :(){ :|:&};: --------------------- http://www.unixer.de/ ----- Torsten Hoefler | Postdoctoral Researcher Open Systems Lab | Indiana University 150 S. Woodlawn Ave. | Bloomington, IN, 474045, USA Lindley Hall Room 135 | +01 (812) 855-3608

1 0

Re: [mpi3-coll] Second January Telecon
by Torsten Hoefler 15 Jan '09

15 Jan '09

On Thu, Jan 15, 2009 at 01:49:25PM -0800, Bronis R. de Supinski wrote: > > Torsten: > > Re: > > Jesper proposed a second January telecon for the 29th. However, I > > couldn't attent on the 29th and would like to move it to the 28th > > (noon). > > > > Does anybody object? We can also choose another time or day in this > > week. > > I (probably) have a conflict on the 28th. The 27th is > more open for me... 27th is fine with me - other opinions? Best, Torsten -- bash$ :(){ :|:&};: --------------------- http://www.unixer.de/ ----- Torsten Hoefler | Postdoctoral Researcher Open Systems Lab | Indiana University 150 S. Woodlawn Ave. | Bloomington, IN, 474045, USA Lindley Hall Room 135 | +01 (812) 855-3608

1 0

Re: [mpi3-coll] Second January Telecon
by Bronis R. de Supinski 15 Jan '09

15 Jan '09

Torsten: Re: > Jesper proposed a second January telecon for the 29th. However, I > couldn't attent on the 29th and would like to move it to the 28th > (noon). > > Does anybody object? We can also choose another time or day in this > week. I (probably) have a conflict on the 28th. The 27th is more open for me... Bronis

1 0

[mpi3-coll] Second January Telecon
by Torsten Hoefler 15 Jan '09

15 Jan '09

Hello Coll-WG, Jesper proposed a second January telecon for the 29th. However, I couldn't attent on the 29th and would like to move it to the 28th (noon). Does anybody object? We can also choose another time or day in this week. Thanks & Best, Torsten -- bash$ :(){ :|:&};: --------------------- http://www.unixer.de/ ----- Torsten Hoefler | Postdoctoral Researcher Open Systems Lab | Indiana University 150 S. Woodlawn Ave. | Bloomington, IN, 474045, USA Lindley Hall Room 135 | +01 (812) 855-3608

1 0