October 2008 - mpiwg-coll - mailman3.cels.anl.gov

Re: [mpi3-coll] Non-blocking Collectives Proposal Draft
by Torsten Hoefler 15 Oct '08

15 Oct '08

On Wed, Oct 15, 2008 at 02:49:03PM -0500, Rajeev Thakur wrote: > > 1) The MPI-2.1 document has 127 occurrences of "nonblocking" > > and only 8 > > occurrences of "non-blocking". What would be the correct > > term? Should we try to be more consistent? > > The technical editor in our division prefers the version without the hyphen. > She will remove the hyphen in our papers. hmm, I would disagree - I had exactly the same discussion with our technical editor :). She explained me for 15 minutes why it is "non-blocking" and not "nonblocking" or "non blocking". Also, if you do the <pick your favorit search enigne> test (I know ... but languages are dynamically changed by their users), you'll see that non-blocking seems to be the most-used term in the I/O and communication context. I'm open for arguments! Best, Torsten -- bash$ :(){ :|:&};: --------------------- http://www.unixer.de/ ----- Torsten Hoefler | Postdoctoral Researcher Open Systems Lab | Indiana University 150 S. Woodlawn Ave. | Bloomington, IN, 474045, USA Lindley Hall Room 135 | +01 (812) 855-3608

1 0

Re: [mpi3-coll] Non-blocking Collectives Proposal Draft
by Rajeev Thakur 15 Oct '08

15 Oct '08

> 1) The MPI-2.1 document has 127 occurrences of "nonblocking" > and only 8 > occurrences of "non-blocking". What would be the correct > term? Should we try to be more consistent? The technical editor in our division prefers the version without the hyphen. She will remove the hyphen in our papers. Rajeev > -----Original Message----- > From: mpi3-coll-bounces(a)lists.mpi-forum.org > [mailto:[email protected]] On Behalf Of > Christian Siebert > Sent: Wednesday, October 15, 2008 9:03 AM > To: MPI-3 Collective Subgroup Discussions > Subject: Re: [mpi3-coll] Non-blocking Collectives Proposal Draft > > @Torsten: > > Thanks a lot for this first draft! I tried to make some > useful comments > and attached the results to this mail (sorry, I can't comment > PDFs so I > did it the "old inconvenient way"). > > @Working Group (something to discuss): > > 1) The MPI-2.1 document has 127 occurrences of "nonblocking" > and only 8 > occurrences of "non-blocking". What would be the correct > term? Should we > try to be more consistent? > > 2) Clarification of MPI_Request_free() for requests from non-blocking > collective operations. > > 3) Better definition/description for "matching" (there is > nothing like > "at the same time" -> logical order?). > > 4) Define "levels of progression"? To be queried (e.g., for > "Synchronous > Progress" MPI_Tests are needed for performance, but for "Asynchronous > Progress" they would only add unnecessary overhead)? UP >= AP >= SP? > > 5) "Unexpected Progress" => "buffered" collectives? > > 6) NBC gives several possible ways for optimizations. With > this "General > advice to implementers" we stick to only one, and might > prevent others. > Can we already fix a decision for optimization strategies at > this stage? > Should we fix it at all? > > 7) Should there be a concrete code example in the proposal (e.g. an > implementation of this double buffering example)? > > Hope this isn't too much for now. > > Best regards, > Christian > > Torsten Hoefler wrote: > > Hello Working Group, > > I finished a first draft of our non-blocking collectives proposal. > > Please read it carefully and send comments to me or the list (e.g., > > marked-up pdf documents). > > > > The draft is attached to this mail. > > > > All the Best, > > Torsten > > -- > Christian Siebert, Dipl.-Inf. Research Associate > > NEC Laboratories Europe, NEC Europe Ltd. > Rathausallee 10, D-53757 Sankt Augustin, Germany > > Phone: +49 (0) 2241 - 92 52 44 Fax: +49 (0) 2241 - 92 52 99 > > (Registered Office: 1 Victoria Road, London W3 6BL, 2832014) >

1 0

Re: [mpi3-coll] Non-blocking Collectives Proposal Draft
by Supalov, Alexander 15 Oct '08

15 Oct '08

Dear Torsten, Thanks for the proposal. A couple of comments in the more old-fashioned way. Generally, some sentences start with lower case letters that should be capitalized. Page 3, top. We probably should not explain here how to ensure progress for nonblocking ops. This matter should be covered by the general progress description elsewhere. Page 3, lower half. Look how progress is defined in this list by referring to the pt2pt progress rules. Ibid. There are already 3 classes of requests: pt2pt, generalized, and file I/O ones. Page 3, bottom. How many nonblocking ops can overlap with a blocking one? I.e., can one have the following situation: Process 0 Process 1 MPI_Ibarrier(req) MPI_Ibarrier(req) MPI_Wait(req) MPI Bcast MPI_Bcast MPI_Wait(req) Page 5, bottom. 32767 simultaneous ops to support is a tall order. I'd say, 1 (one) would be a good lower limit, thus allowing no overlap. Or even 0 (zero), meaning no nonblocking collective support at all. The rest of this passage would probably be better reformulated as an advice to implementors. In the margin: editing a PDF requires creativity (exemplified by the earlier comments) or a pretty expensive Acrobat. Maybe we can find some other way of distributing and commenting on our drafts? Best regards. Alexander -----Original Message----- From: mpi3-coll-bounces(a)lists.mpi-forum.org [mailto:[email protected]] On Behalf Of Torsten Hoefler Sent: Tuesday, October 14, 2008 9:47 PM To: mpi3-coll(a)lists.mpi-forum.org Subject: [mpi3-coll] Non-blocking Collectives Proposal Draft Hello Working Group, I finished a first draft of our non-blocking collectives proposal. Please read it carefully and send comments to me or the list (e.g., marked-up pdf documents). The draft is attached to this mail. All the Best, Torsten -- bash$ :(){ :|:&};: --------------------- http://www.unixer.de/ ----- Torsten Hoefler | Research Assistant Open Systems Lab | Indiana University 150 S. Woodlawn Ave. | Bloomington, IN, 474045, USA Lindley Hall Room 135 | +01 (812) 855-3608 --------------------------------------------------------------------- Intel GmbH Dornacher Strasse 1 85622 Feldkirchen/Muenchen Germany Sitz der Gesellschaft: Feldkirchen bei Muenchen Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer Registergericht: Muenchen HRB 47456 Ust.-IdNr. VAT Registration No.: DE129385895 Citibank Frankfurt (BLZ 502 109 00) 600119052 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.

1 0

Re: [mpi3-coll] Non-blocking Collectives Proposal Draft
by Supalov, Alexander 15 Oct '08

15 Oct '08

Hi guys, I wonder why we stopped using Wiki for the bulky stuff. Best regards. Alexander -----Original Message----- From: mpi3-coll-bounces(a)lists.mpi-forum.org [mailto:[email protected]] On Behalf Of Christian Siebert Sent: Wednesday, October 15, 2008 4:03 PM To: MPI-3 Collective Subgroup Discussions Subject: Re: [mpi3-coll] Non-blocking Collectives Proposal Draft @Torsten: Thanks a lot for this first draft! I tried to make some useful comments and attached the results to this mail (sorry, I can't comment PDFs so I did it the "old inconvenient way"). @Working Group (something to discuss): 1) The MPI-2.1 document has 127 occurrences of "nonblocking" and only 8 occurrences of "non-blocking". What would be the correct term? Should we try to be more consistent? 2) Clarification of MPI_Request_free() for requests from non-blocking collective operations. 3) Better definition/description for "matching" (there is nothing like "at the same time" -> logical order?). 4) Define "levels of progression"? To be queried (e.g., for "Synchronous Progress" MPI_Tests are needed for performance, but for "Asynchronous Progress" they would only add unnecessary overhead)? UP >= AP >= SP? 5) "Unexpected Progress" => "buffered" collectives? 6) NBC gives several possible ways for optimizations. With this "General advice to implementers" we stick to only one, and might prevent others. Can we already fix a decision for optimization strategies at this stage? Should we fix it at all? 7) Should there be a concrete code example in the proposal (e.g. an implementation of this double buffering example)? Hope this isn't too much for now. Best regards, Christian Torsten Hoefler wrote: > Hello Working Group, > I finished a first draft of our non-blocking collectives proposal. > Please read it carefully and send comments to me or the list (e.g., > marked-up pdf documents). > > The draft is attached to this mail. > > All the Best, > Torsten -- Christian Siebert, Dipl.-Inf. Research Associate NEC Laboratories Europe, NEC Europe Ltd. Rathausallee 10, D-53757 Sankt Augustin, Germany Phone: +49 (0) 2241 - 92 52 44 Fax: +49 (0) 2241 - 92 52 99 (Registered Office: 1 Victoria Road, London W3 6BL, 2832014) --------------------------------------------------------------------- Intel GmbH Dornacher Strasse 1 85622 Feldkirchen/Muenchen Germany Sitz der Gesellschaft: Feldkirchen bei Muenchen Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer Registergericht: Muenchen HRB 47456 Ust.-IdNr. VAT Registration No.: DE129385895 Citibank Frankfurt (BLZ 502 109 00) 600119052 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.

1 0

Re: [mpi3-coll] Non-blocking Collectives Proposal Draft
by Christian Siebert 15 Oct '08

15 Oct '08

@Torsten: Thanks a lot for this first draft! I tried to make some useful comments and attached the results to this mail (sorry, I can't comment PDFs so I did it the "old inconvenient way"). @Working Group (something to discuss): 1) The MPI-2.1 document has 127 occurrences of "nonblocking" and only 8 occurrences of "non-blocking". What would be the correct term? Should we try to be more consistent? 2) Clarification of MPI_Request_free() for requests from non-blocking collective operations. 3) Better definition/description for "matching" (there is nothing like "at the same time" -> logical order?). 4) Define "levels of progression"? To be queried (e.g., for "Synchronous Progress" MPI_Tests are needed for performance, but for "Asynchronous Progress" they would only add unnecessary overhead)? UP >= AP >= SP? 5) "Unexpected Progress" => "buffered" collectives? 6) NBC gives several possible ways for optimizations. With this "General advice to implementers" we stick to only one, and might prevent others. Can we already fix a decision for optimization strategies at this stage? Should we fix it at all? 7) Should there be a concrete code example in the proposal (e.g. an implementation of this double buffering example)? Hope this isn't too much for now. Best regards, Christian Torsten Hoefler wrote: > Hello Working Group, > I finished a first draft of our non-blocking collectives proposal. > Please read it carefully and send comments to me or the list (e.g., > marked-up pdf documents). > > The draft is attached to this mail. > > All the Best, > Torsten -- Christian Siebert, Dipl.-Inf. Research Associate NEC Laboratories Europe, NEC Europe Ltd. Rathausallee 10, D-53757 Sankt Augustin, Germany Phone: +49 (0) 2241 - 92 52 44 Fax: +49 (0) 2241 - 92 52 99 (Registered Office: 1 Victoria Road, London W3 6BL, 2832014)

1 0

Re: [mpi3-coll] [Mpi-22] Graph Interface Fixes
by Torsten Hoefler 14 Oct '08

14 Oct '08

Hello again, Jesper and me had some offline discussions. He found a bug in my code, added consistent error checking and extended the example. Thanks! The code looks fine to me and I attached the changed code to this mail. Please check it for correctness. All the Best, Torsten -- bash$ :(){ :|:&};: --------------------- http://www.unixer.de/ ----- Torsten Hoefler | Research Assistant Open Systems Lab | Indiana University 150 S. Woodlawn Ave. | Bloomington, IN, 474045, USA Lindley Hall Room 135 | +01 (812) 855-3608

1 0

[mpi3-coll] Non-blocking Collectives Proposal Draft
by Torsten Hoefler 14 Oct '08

14 Oct '08

Hello Working Group, I finished a first draft of our non-blocking collectives proposal. Please read it carefully and send comments to me or the list (e.g., marked-up pdf documents). The draft is attached to this mail. All the Best, Torsten -- bash$ :(){ :|:&};: --------------------- http://www.unixer.de/ ----- Torsten Hoefler | Research Assistant Open Systems Lab | Indiana University 150 S. Woodlawn Ave. | Bloomington, IN, 474045, USA Lindley Hall Room 135 | +01 (812) 855-3608

1 0

[mpi3-coll] Graph Interface Fixes
by Torsten Hoefler 13 Oct '08

13 Oct '08

Hello Group, I finished a first implementation which maps the newly proposed graph interface to the old interface. This implementation is probably suboptimal and of course not scalable (can't be because the old interface isn't ;)). But is should be functional (compiles and works with some toy examples). I also corrected a slight mistake in ticket #33. As usual, I am open for comments. I CC'd the mpi-2.2 mailinglist because I think that might be of interest for the discussions next week. Best, Torsten -- bash$ :(){ :|:&};: --------------------- http://www.unixer.de/ ----- Torsten Hoefler | Research Assistant Open Systems Lab | Indiana University 150 S. Woodlawn Ave. | Bloomington, IN, 474045, USA Lindley Hall Room 135 | +01 (812) 855-3608

1 0

Re: [mpi3-coll] October Telecon
by Torsten Hoefler 10 Oct '08

10 Oct '08

Hello, I just posted my notes from the telecon to the wiki (https://svn.mpi-forum.org/trac/mpi-forum-web/wiki/Telecon101008) I realized that we missed some of our "regulars". I hope we'll have them back when we go back to the old time (Thursday noon). Please speak up if anyone's schedule changed - we can try to decide on another time. See you all in two weeks! All the Best, Torsten -- bash$ :(){ :|:&};: --------------------- http://www.unixer.de/ ----- Torsten Hoefler | Research Assistant Open Systems Lab | Indiana University 150 S. Woodlawn Ave. | Bloomington, IN, 474045, USA Lindley Hall Room 135 | +01 (812) 855-3608

1 0

Re: [mpi3-coll] October Telecon
by Torsten Hoefler 07 Oct '08

07 Oct '08

Hello, there was no big complaint, so I reserved the slot for the telecon. Here's the call-in information for Friday: Conf Id: 107208 Date: 10/10/2008 Begin Time: 12:00:00 PM Eastern Time (US & Canada) Phone Number: (812) 856-3600 PIN: 002118# I hope you can make it! All the best, Torsten Here again the agenda (same as in my last mail): 1) talk about function interfaces (how do they look like) - we decided for the "calls for everything" way - now we should flesh out details how the newly proposed collectives should look like - I will keep track of our decisions and prepare a full proposal document 2) Discuss progress on the subitems a) non-blocking collectives interface will be defined (see straw-votes) * make final decisions about the following items - mix blocking/non-blocking calls - use use MPI_Requests - multiple outstanding colls - nest/mix collectives - prefix (I/A/N/...) - max outstanding colls? - how do we want to support hardware acceleration? - any other details to discuss here? b) discuss research progress on sparse collectives - we have the Octopus EuroPVM paper http://www.unixer.de/publications/#hoefler-europvm08-qm - further directions? - volunteers? c) discuss research progress/evaluation of persistent colls - directions to go? - volunteers? 3) Updates on topological collectives a) figure out which operatiosn make sense and how to call them - neighbor exchange (alltoall?) - neigbor reduce - neighbor allgather - neighbor allreduce - versions for cart comms with distance > 1? - more? b) discuss applications using them (neighbor exchange is clear, but the others?) - need to get in touch with real users to decide anything useful 4) Updates on MPI Plans - keep it going (research-wise) - finish prototype implementation 5) Variable size collectives (does anyone pick this topic up?) - Alexander/Hans-Joachim? - do we have an application/implementation to look at? - any news? 6) MPI-2.2 issues - revisited a) fix non-scalable graph interface - will do implementation - discuss issues (if any) b) local reduction operation (needed by libraries, e.g., LibNBC) - any issues? c) local progress function (caused heavy discussions in Forum) - issues ;)? d) request completion callbacks (for better progression, e.g., LibNBC) - discuss what to allow/disallow during callbacks - do we need to limit anything? e) partial pack/unpack (needed by libraries, e.g., LibNBC) - do we want to continue this one? 7) other items (please post here!) -- bash$ :(){ :|:&};: --------------------- http://www.unixer.de/ ----- Torsten Hoefler | Research Assistant Open Systems Lab | Indiana University 150 S. Woodlawn Ave. | Bloomington, IN, 474045, USA Lindley Hall Room 135 | +01 (812) 855-3608

1 0