- mpiwg-coll - mailman3.cels.anl.gov

Re: [mpi3-coll] Non-blocking Collectives Proposal Draft
by Supalov, Alexander 15 Oct '08

15 Oct '08

Hi guys, I wonder why we stopped using Wiki for the bulky stuff. Best regards. Alexander -----Original Message----- From: mpi3-coll-bounces(a)lists.mpi-forum.org [mailto:[email protected]] On Behalf Of Christian Siebert Sent: Wednesday, October 15, 2008 4:03 PM To: MPI-3 Collective Subgroup Discussions Subject: Re: [mpi3-coll] Non-blocking Collectives Proposal Draft @Torsten: Thanks a lot for this first draft! I tried to make some useful comments and attached the results to this mail (sorry, I can't comment PDFs so I did it the "old inconvenient way"). @Working Group (something to discuss): 1) The MPI-2.1 document has 127 occurrences of "nonblocking" and only 8 occurrences of "non-blocking". What would be the correct term? Should we try to be more consistent? 2) Clarification of MPI_Request_free() for requests from non-blocking collective operations. 3) Better definition/description for "matching" (there is nothing like "at the same time" -> logical order?). 4) Define "levels of progression"? To be queried (e.g., for "Synchronous Progress" MPI_Tests are needed for performance, but for "Asynchronous Progress" they would only add unnecessary overhead)? UP >= AP >= SP? 5) "Unexpected Progress" => "buffered" collectives? 6) NBC gives several possible ways for optimizations. With this "General advice to implementers" we stick to only one, and might prevent others. Can we already fix a decision for optimization strategies at this stage? Should we fix it at all? 7) Should there be a concrete code example in the proposal (e.g. an implementation of this double buffering example)? Hope this isn't too much for now. Best regards, Christian Torsten Hoefler wrote: > Hello Working Group, > I finished a first draft of our non-blocking collectives proposal. > Please read it carefully and send comments to me or the list (e.g., > marked-up pdf documents). > > The draft is attached to this mail. > > All the Best, > Torsten -- Christian Siebert, Dipl.-Inf. Research Associate NEC Laboratories Europe, NEC Europe Ltd. Rathausallee 10, D-53757 Sankt Augustin, Germany Phone: +49 (0) 2241 - 92 52 44 Fax: +49 (0) 2241 - 92 52 99 (Registered Office: 1 Victoria Road, London W3 6BL, 2832014) --------------------------------------------------------------------- Intel GmbH Dornacher Strasse 1 85622 Feldkirchen/Muenchen Germany Sitz der Gesellschaft: Feldkirchen bei Muenchen Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer Registergericht: Muenchen HRB 47456 Ust.-IdNr. VAT Registration No.: DE129385895 Citibank Frankfurt (BLZ 502 109 00) 600119052 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.

1 0

Re: [mpi3-coll] Non-blocking Collectives Proposal Draft
by Christian Siebert 15 Oct '08

15 Oct '08

@Torsten: Thanks a lot for this first draft! I tried to make some useful comments and attached the results to this mail (sorry, I can't comment PDFs so I did it the "old inconvenient way"). @Working Group (something to discuss): 1) The MPI-2.1 document has 127 occurrences of "nonblocking" and only 8 occurrences of "non-blocking". What would be the correct term? Should we try to be more consistent? 2) Clarification of MPI_Request_free() for requests from non-blocking collective operations. 3) Better definition/description for "matching" (there is nothing like "at the same time" -> logical order?). 4) Define "levels of progression"? To be queried (e.g., for "Synchronous Progress" MPI_Tests are needed for performance, but for "Asynchronous Progress" they would only add unnecessary overhead)? UP >= AP >= SP? 5) "Unexpected Progress" => "buffered" collectives? 6) NBC gives several possible ways for optimizations. With this "General advice to implementers" we stick to only one, and might prevent others. Can we already fix a decision for optimization strategies at this stage? Should we fix it at all? 7) Should there be a concrete code example in the proposal (e.g. an implementation of this double buffering example)? Hope this isn't too much for now. Best regards, Christian Torsten Hoefler wrote: > Hello Working Group, > I finished a first draft of our non-blocking collectives proposal. > Please read it carefully and send comments to me or the list (e.g., > marked-up pdf documents). > > The draft is attached to this mail. > > All the Best, > Torsten -- Christian Siebert, Dipl.-Inf. Research Associate NEC Laboratories Europe, NEC Europe Ltd. Rathausallee 10, D-53757 Sankt Augustin, Germany Phone: +49 (0) 2241 - 92 52 44 Fax: +49 (0) 2241 - 92 52 99 (Registered Office: 1 Victoria Road, London W3 6BL, 2832014)

1 0

Re: [mpi3-coll] [Mpi-22] Graph Interface Fixes
by Torsten Hoefler 14 Oct '08

14 Oct '08

Hello again, Jesper and me had some offline discussions. He found a bug in my code, added consistent error checking and extended the example. Thanks! The code looks fine to me and I attached the changed code to this mail. Please check it for correctness. All the Best, Torsten -- bash$ :(){ :|:&};: --------------------- http://www.unixer.de/ ----- Torsten Hoefler | Research Assistant Open Systems Lab | Indiana University 150 S. Woodlawn Ave. | Bloomington, IN, 474045, USA Lindley Hall Room 135 | +01 (812) 855-3608

1 0

[mpi3-coll] Non-blocking Collectives Proposal Draft
by Torsten Hoefler 14 Oct '08

14 Oct '08

Hello Working Group, I finished a first draft of our non-blocking collectives proposal. Please read it carefully and send comments to me or the list (e.g., marked-up pdf documents). The draft is attached to this mail. All the Best, Torsten -- bash$ :(){ :|:&};: --------------------- http://www.unixer.de/ ----- Torsten Hoefler | Research Assistant Open Systems Lab | Indiana University 150 S. Woodlawn Ave. | Bloomington, IN, 474045, USA Lindley Hall Room 135 | +01 (812) 855-3608

1 0

[mpi3-coll] Graph Interface Fixes
by Torsten Hoefler 13 Oct '08

13 Oct '08

Hello Group, I finished a first implementation which maps the newly proposed graph interface to the old interface. This implementation is probably suboptimal and of course not scalable (can't be because the old interface isn't ;)). But is should be functional (compiles and works with some toy examples). I also corrected a slight mistake in ticket #33. As usual, I am open for comments. I CC'd the mpi-2.2 mailinglist because I think that might be of interest for the discussions next week. Best, Torsten -- bash$ :(){ :|:&};: --------------------- http://www.unixer.de/ ----- Torsten Hoefler | Research Assistant Open Systems Lab | Indiana University 150 S. Woodlawn Ave. | Bloomington, IN, 474045, USA Lindley Hall Room 135 | +01 (812) 855-3608

1 0

Re: [mpi3-coll] October Telecon
by Torsten Hoefler 10 Oct '08

10 Oct '08

Hello, I just posted my notes from the telecon to the wiki (https://svn.mpi-forum.org/trac/mpi-forum-web/wiki/Telecon101008) I realized that we missed some of our "regulars". I hope we'll have them back when we go back to the old time (Thursday noon). Please speak up if anyone's schedule changed - we can try to decide on another time. See you all in two weeks! All the Best, Torsten -- bash$ :(){ :|:&};: --------------------- http://www.unixer.de/ ----- Torsten Hoefler | Research Assistant Open Systems Lab | Indiana University 150 S. Woodlawn Ave. | Bloomington, IN, 474045, USA Lindley Hall Room 135 | +01 (812) 855-3608

1 0

Re: [mpi3-coll] October Telecon
by Torsten Hoefler 07 Oct '08

07 Oct '08

Hello, there was no big complaint, so I reserved the slot for the telecon. Here's the call-in information for Friday: Conf Id: 107208 Date: 10/10/2008 Begin Time: 12:00:00 PM Eastern Time (US & Canada) Phone Number: (812) 856-3600 PIN: 002118# I hope you can make it! All the best, Torsten Here again the agenda (same as in my last mail): 1) talk about function interfaces (how do they look like) - we decided for the "calls for everything" way - now we should flesh out details how the newly proposed collectives should look like - I will keep track of our decisions and prepare a full proposal document 2) Discuss progress on the subitems a) non-blocking collectives interface will be defined (see straw-votes) * make final decisions about the following items - mix blocking/non-blocking calls - use use MPI_Requests - multiple outstanding colls - nest/mix collectives - prefix (I/A/N/...) - max outstanding colls? - how do we want to support hardware acceleration? - any other details to discuss here? b) discuss research progress on sparse collectives - we have the Octopus EuroPVM paper http://www.unixer.de/publications/#hoefler-europvm08-qm - further directions? - volunteers? c) discuss research progress/evaluation of persistent colls - directions to go? - volunteers? 3) Updates on topological collectives a) figure out which operatiosn make sense and how to call them - neighbor exchange (alltoall?) - neigbor reduce - neighbor allgather - neighbor allreduce - versions for cart comms with distance > 1? - more? b) discuss applications using them (neighbor exchange is clear, but the others?) - need to get in touch with real users to decide anything useful 4) Updates on MPI Plans - keep it going (research-wise) - finish prototype implementation 5) Variable size collectives (does anyone pick this topic up?) - Alexander/Hans-Joachim? - do we have an application/implementation to look at? - any news? 6) MPI-2.2 issues - revisited a) fix non-scalable graph interface - will do implementation - discuss issues (if any) b) local reduction operation (needed by libraries, e.g., LibNBC) - any issues? c) local progress function (caused heavy discussions in Forum) - issues ;)? d) request completion callbacks (for better progression, e.g., LibNBC) - discuss what to allow/disallow during callbacks - do we need to limit anything? e) partial pack/unpack (needed by libraries, e.g., LibNBC) - do we want to continue this one? 7) other items (please post here!) -- bash$ :(){ :|:&};: --------------------- http://www.unixer.de/ ----- Torsten Hoefler | Research Assistant Open Systems Lab | Indiana University 150 S. Woodlawn Ave. | Bloomington, IN, 474045, USA Lindley Hall Room 135 | +01 (812) 855-3608

1 0

Re: [mpi3-coll] October Telecon
by Jesper Larsson Traeff 17 Sep '08

17 Sep '08

On Tue, Sep 16, 2008 at 05:06:18PM -0400, Torsten Hoefler wrote: > On Tue, Sep 16, 2008 at 04:57:01PM -0400, Rich L. Graham wrote: > > Can we move this to the following week (or following day) ? [...] > the following week would be very close to the next Forum. Does anybody > mind of we move the telecon to > > Friday, Oct 10th, 12pm? Should work, but not optimal for me, I may have to leave after an hour (and we have a long agenda). But probably better than 16th Jesper

1 0

Re: [mpi3-coll] October Telecon
by Torsten Hoefler 16 Sep '08

16 Sep '08

On Tue, Sep 16, 2008 at 04:57:01PM -0400, Rich L. Graham wrote: > Can we move this to the following week (or following day) ? [...] the following week would be very close to the next Forum. Does anybody mind of we move the telecon to Friday, Oct 10th, 12pm? If yes, we'll just go with the (suboptimal) 16th. Best, Torsten -- bash$ :(){ :|:&};: --------------------- http://www.unixer.de/ ----- "Software is a place where dreams are planted and nightmares harvested, where terrible demons compete with magical panaceas, a world of werewolves and silver bullets." - Brad J. Cox, Plannning the Software Industrial Revolution", IEEE Software, Nov. 1990

1 0

[mpi3-coll] October Telecon
by Torsten Hoefler 16 Sep '08

16 Sep '08

Hello Collectives-WG, I propose October 9th, 12:00pm EDT as the time/date for our monthly collectives workgroup teleconference. Please let us know if there are any (strong) objections to this time. I will reserve the spot this Thursday if there are no objections. We will mainly be talking about the feedback we got from the whole forum on our proposals. Christian's and my notes are at: https://svn.mpi-forum.org/trac/mpi-forum-web/wiki/forum090408 We will also try to bring everybody up to speed who could not attend the last forum! Im particular we have to decide/discuss (those points also serve as agenda items): 1) talk about function interfaces (how do they look like) - we decided for the "calls for everything" way - now we should flesh out details how the newly proposed collectives should look like - I will keep track of our decisions and prepare a full proposal document 2) Discuss progress on the subitems a) non-blocking collectives interface will be defined (see straw-votes) * make final decisions about the following items - mix blocking/non-blocking calls - use use MPI_Requests - multiple outstanding colls - nest/mix collectives - prefix (I/A/N/...) - max outstanding colls? - how do we want to support hardware acceleration? - any other details to discuss here? b) discuss research progress on sparse collectives - we have the Octopus EuroPVM paper http://www.unixer.de/publications/#hoefler-europvm08-qm - further directions? - volunteers? c) discuss research progress/evaluation of persistent colls - directions to go? - volunteers? 3) Updates on topological collectives a) figure out which operatiosn make sense and how to call them - neighbor exchange (alltoall?) - neigbor reduce - neighbor allgather - neighbor allreduce - versions for cart comms with distance > 1? - more? b) discuss applications using them (neighbor exchange is clear, but the others?) - need to get in touch with real users to decide anything useful 4) Updates on MPI Plans - keep it going (research-wise) - finish prototype implementation 5) Variable size collectives (does anyone pick this topic up?) - Alexander/Hans-Joachim? - do we have an application/implementation to look at? - any news? 6) MPI-2.2 issues - revisited a) fix non-scalable graph interface - will do implementation - discuss issues (if any) b) local reduction operation (needed by libraries, e.g., LibNBC) - any issues? c) local progress function (caused heavy discussions in Forum) - issues ;)? d) request completion callbacks (for better progression, e.g., LibNBC) - discuss what to allow/disallow during callbacks - do we need to limit anything? e) partial pack/unpack (needed by libraries, e.g., LibNBC) - do we want to continue this one? 7) other items (please post here!) Thanks & Best, Torsten Hoefler -- bash$ :(){ :|:&};: --------------------- http://www.unixer.de/ ----- Torsten Hoefler | Research Assistant Open Systems Lab | Indiana University 150 S. Woodlawn Ave. | Bloomington, IN, 474045, USA Lindley Hall Room 135 | +01 (812) 855-3608

1 0