- mpiwg-coll - mailman3.cels.anl.gov

[mpi3-coll] [MPI Forum] Ticket #168 MPI_Icomm_dup
by Torsten Hoefler 13 Nov '11

13 Nov '11

Hi Collective WG members and "Groups, Contexts, and Communicators" Committee, We discussed the functionality of a nonblocking communicator duplication during MPI-2.2 and deferred the ticket (#164) to MPI-3 because we didn't want to add too much at that point. This functionality is crucial for the implementation of nonblocking libraries (and the library concept in general) in MPI. For a detailed discussion see Hoefler, Snir: "Writing Parallel Libraries with MPI - Common Practice, Issues, and Extensions". I beefed up the ticket to make it a full MPI-3 ticket and I would like to have a reading at the January meeting. Everybody who is in the "To" field of this message, please review this ticket in the context of the groups and communicators and collectives chapters and comment (also indicate when it's ok). Everybody else on the ML is invited to review too. The collective operations and topologies WG will have another telecon where this ticket is discussed soon. Thanks & Best, Torsten PS: please check if I got your username in the CC field right, and correct it if it is wrong (I didn't find an index for those). -- bash$ :(){ :|:&};: --------------------- http://www.unixer.de/ ----- Torsten Hoefler | Performance Modeling and Simulation Lead Blue Waters Directorate | University of Illinois (UIUC) 1205 W Clark Street | Urbana, IL, 61801 NCSA Building | +01 (217) 244-7736

1 0

Re: [mpi3-coll] Missing reduce_local and reduce_scatter_block in create_op text
by Torsten Hoefler 12 Nov '11

12 Nov '11

Hi Adam, > There are a few lists that are missing MPI_REDUCE_LOCAL and > MPI_REDUCE_SCATTER_BLOCK. > > 1) Section 5.9.2 > List of predefined ops missing MPI_REDUCE_LOCAL, > MPI_ALLREDUCE_SCATTER_BLOCK, and all non-blocking variants. > > 2) Description for MPI_OP_CREATE missing MPI_REDUCE_LOCAL, > MPI_REDUCE_SCATTER_BLOCK, and all non-blocking variants. > > I guess we may not need to list the non-blocking variants in this list, > but we should definitely add REDUCE_LOCAL and REDUCE_SCATTER_BLOCK. Good catch -- thanks. I fixed both in mpi-3.0-minutiae (clear Ticket 0). I also mentioned the nonblocking variants. All the Best, Torsten

1 0

Re: [mpi3-coll] [Mpi-forum] MPI_Comm_create_group Meeting
by James Dinan 08 Nov '11

08 Nov '11

Hi All, Notes from yesterday's meeting are attached. Please note that we are moving all future discussion of this ticket to the mpi3-coll mailing list. Best, ~Jim. In attendance: Adam Moody, David Goodell, Jim Dinan, Pavan Balaji, Fab Tiller, Bronis de Supinski, Jeff Hammond Item 1: Name of the routine. We decided to stick with MPI_Comm_create group as the best way of matching the MPI_CLASS_ACTION_SUBSET naming convention. Item 2: Tag versus color. We decided to use a tag, which is defined to be in a collective tag space that is distinct from the point-to-point tag space. The collective tag will follow all MPI tag semantics, however it will not conflict with any point-to-point tags (including MPI_ANY_TAG). This essentially exposes the expected implementation in the standard, providing more flexible semantics to the user. In addition, it allows an implementation to make more efficient use of the tag space by defining tag conflicts across tagged collective operations (currently only MPI_Intercomm_create and MPI_Comm_create_group, but we could define others in the future). In a separate ticket, we can update MPI_Intercomm_create to specify that its tag is also in the collective tag space. This will be backward compatible, but will make MPI_Intercomm_create considerably easier to use because its tags will no longer conflict with point-to-point operations. Tasks: * Jim will work on a revision of the text that follows the above semantics. * The group will reconvene after Thanksgiving to review new text. * The group will move discussion to the collectives mailing list. On 11/4/11 11:07 AM, James Dinan wrote: > Based on the doodle poll, it looks like Monday at 3pm CT will be the > best time to meet. We'll use the hybrid WG telecon bridge: > > Domestic dial number: 866-654-6744 > International dial number: 517-308-8226 > Passcode: 6086006 > > See you then, > ~Jim. > > On 11/1/11 2:33 PM, James Dinan wrote: >> Hi All, >> >> I would like to form a group to review the MPI_Comm_create_group() >> proposal (#286) for a formal reading at the next Forum meeting. If you >> are interested in joining the discussion, could you please fill in your >> availability for a one hour conference call next week: >> >> http://www.doodle.com/9tganzq2hqiqy7kt >> >> We have a couple issues to discuss: >> >> 1.) The name of the function: MPI_Comm_create_group, >> MPI_Group_comm_create, MPI_Comm_create_from_group, ... >> >> 2.) The tag argument: Should we use a tag or a "color" to establish a >> safe communication conduit on the parent communicator. As discussed at >> the Forum, using a tag could conflict with the use of MPI_ANY_TAG on the >> parent communicator. A color would avoid this issue, however it could >> complicate tag matching. >> >> 3.) Changes to the text. >> >> Thanks, >> ~Jim. >> >> PS- I've attached a copy of the current text with comments from the >> formal reading at the last Forum meeting (in Acrobat: Comments -> >> Comments View -> Show Comments List for the full list). >> >> >> _______________________________________________ >> mpi-forum mailing list >> mpi-forum(a)lists.mpi-forum.org >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum > _______________________________________________ > mpi-forum mailing list > mpi-forum(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum

1 0

[mpi3-coll] Missing reduce_local and reduce_scatter_block in create_op text
by Adam T. Moody 28 Oct '11

28 Oct '11

Hi Torsten, There are a few lists that are missing MPI_REDUCE_LOCAL and MPI_REDUCE_SCATTER_BLOCK. 1) Section 5.9.2 List of predefined ops missing MPI_REDUCE_LOCAL, MPI_ALLREDUCE_SCATTER_BLOCK, and all non-blocking variants. 2) Description for MPI_OP_CREATE missing MPI_REDUCE_LOCAL, MPI_REDUCE_SCATTER_BLOCK, and all non-blocking variants. I guess we may not need to list the non-blocking variants in this list, but we should definitely add REDUCE_LOCAL and REDUCE_SCATTER_BLOCK. -Adam

1 0

Re: [mpi3-coll] Scalable Vector Collectives & Next Telecon
by Torsten Hoefler 27 Oct '11

27 Oct '11

Sorry for the double email -- just to make this clear: we are planning a reading in the January meeting, so please take the review seriously! Thanks & Best, Torsten -- bash$ :(){ :|:&};: --------------------- http://www.unixer.de/ ----- Torsten Hoefler | Performance Modeling and Simulation Lead Blue Waters Directorate | University of Illinois (UIUC) 1205 W Clark Street | Urbana, IL, 61801 NCSA Building | +01 (217) 244-7736

1 0

[mpi3-coll] Scalable Vector Collectives & Next Telecon
by Torsten Hoefler 27 Oct '11

27 Oct '11

Hello Coll-WG, Christian and I finished the draft of the scalable collectives proposal. We incorporated all changes that we discussed during the last Forum meeting. The two major changes were: 1) added MPI_(I)Alltoalldw 2) removed the displs arguments The newest version of the chapter including the proposal is attached to ticket 264 (https://svn.mpi-forum.org/trac/mpi-forum-web/ticket/264) Everybody in the chapter committee (the signup sheet we did on Tuesday), please review the chapter carefully and sign off at the ticket. We will schedule a telecon in the week of either 11/28, 12/5, or 12/12. If you have a certain preference, please let me know, otherwise, I'll pick a random week for a doodle. Thanks & All the Best, Torsten -- bash$ :(){ :|:&};: --------------------- http://www.unixer.de/ ----- Torsten Hoefler | Performance Modeling and Simulation Lead Blue Waters Directorate | University of Illinois (UIUC) 1205 W Clark Street | Urbana, IL, 61801 NCSA Building | +01 (217) 244-7736

1 0

Re: [mpi3-coll] Telecon to discuss DV-collectives (Alltoalldv)
by Torsten Hoefler 14 Oct '11

14 Oct '11

Just a quick reminder -- the telecon is going on now! Torsten On Thu, Oct 13, 2011 at 07:28:09PM -0500, Torsten Hoefler wrote: > Hello Coll-WG, > > The doodle converged to have the call tomorrow 3:30-4pm Central time. > > We will be discussing Ticket 264 and the addition of alltoalldv to it > (the detailed interface). Please review the email discussions from > today. The ticket is https://svn.mpi-forum.org/trac/mpi-forum-web/ticket/264 . > > The call-in information is: > > Scheduled Start Time: 03:30 PM Central Daylight Time > Dial-in Number: 1-940-287-4000?(Texas) > Participant Access Code: 8966928 > > Talk to you tomorrow & Best, > Torsten Hoefler > > -- > bash$ :(){ :|:&};: --------------------- http://www.unixer.de/ ----- > Torsten Hoefler | Performance Modeling and Simulation Lead > Blue Waters Directorate | University of Illinois (UIUC) > 1205 W Clark Street | Urbana, IL, 61801 > NCSA Building | +01 (217) 244-7736 > _______________________________________________ > mpi3-coll mailing list > mpi3-coll(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-coll -- bash$ :(){ :|:&};: --------------------- http://www.unixer.de/ ----- Niemand k�me auf die Idee, Tinte mit Tinte abzuwaschen; nur Blut soll immer wieder mit Blut abgewaschen werden. [Marie von Ebner-Eschenbach]

1 0

Re: [mpi3-coll] Telecon to discuss DV-collectives (Alltoalldv)
by Torsten Hoefler 13 Oct '11

13 Oct '11

Hello Coll-WG, The doodle converged to have the call tomorrow 3:30-4pm Central time. We will be discussing Ticket 264 and the addition of alltoalldv to it (the detailed interface). Please review the email discussions from today. The ticket is https://svn.mpi-forum.org/trac/mpi-forum-web/ticket/264 . The call-in information is: Scheduled Start Time: 03:30 PM Central Daylight Time Dial-in Number: 1-940-287-4000�(Texas) Participant Access Code: 8966928 Talk to you tomorrow & Best, Torsten Hoefler -- bash$ :(){ :|:&};: --------------------- http://www.unixer.de/ ----- Torsten Hoefler | Performance Modeling and Simulation Lead Blue Waters Directorate | University of Illinois (UIUC) 1205 W Clark Street | Urbana, IL, 61801 NCSA Building | +01 (217) 244-7736

1 0

Re: [mpi3-coll] Telecon to discuss DV-collectives (Alltoalldv)
by Torsten Hoefler 13 Oct '11

13 Oct '11

Hi Alexander, > To the 2nd item: we had a comparable proposal for the conventional V > ops in the past (see > https://svn.mpi-forum.org/trac/mpi-forum-web/ticket/23) It seems like this proposal requires still all P entries in all arrays and thus has the \Omega(P) time and space overhead that we're trying to push down to O(k) (while we assume that k=O(log(P)) at very large scale). All the Best, Torsten -- bash$ :(){ :|:&};: --------------------- http://www.unixer.de/ ----- Torsten Hoefler | Performance Modeling and Simulation Lead Blue Waters Directorate | University of Illinois (UIUC) 1205 W Clark Street | Urbana, IL, 61801 NCSA Building | +01 (217) 244-7736

1 0

Re: [mpi3-coll] Telecon to discuss DV-collectives (Alltoalldv)
by Torsten Hoefler 13 Oct '11

13 Oct '11

On Thu, Oct 13, 2011 at 10:55:47AM -0700, Adam T. Moody wrote: > A couple more things... :-) -- more comments below! > In Santorini, Rich brought up a couple of concerns that should be > considered. For one, he suggested that a slightly more general > interface might be better in which you specify a base count for all > processes, and then provide a list for processes that are different than > that count. This could subsume the interface I listed below if you set > the base count to be 0 and then list each non-zero item. The nice thing > about the base count approach is that it nicely handles the "mostly > regular" case, in which nearly all procs have the same amount of data > but only a few have a little more or a little less. For example, this > interface might look something like the following (added basecount and > removed displacements which need some thought here): > > MPI_Alltoalldv( > sendbuf, sbasecount, nsends, sendranks[], sendcounts[], sendtype, /* > O(sbasecount*P + k) list */ > recvbuf, rbasecount, nrecvs, recvranks[], recvcounts[], rectype, /* > O(rbasecount*P + k) list */ > comm > ); > > In the above, you would send/receive basecount data items from all > procs, except for a few ranks, whose counts are listed explicitly in > O(k) lists. Setting sbasecount/rbasecount=0 essentially reduces this > interface to the one below (ignoring displacements). Yes, that may be interesting for the use-case I have in mind (a 3d FFT on a circular (cutoff) region), however, may be too limited for other use cases (which I don't know). > The other thing Rich was concerned about was whether an interface could > be specified in such a way to reduce communication of the distributed > count values. One extension that might help here is to allow the > application to provide minimum and maximum count values that would apply > globally across all processes. For example, with a maximum count value > in gatherdv, you could set up a tree expecting the maxcount from all > children but just receive less during each step. On the otherhand, if > the min and max values are far apart, the implementation might fall back > to something more dynamic so it doesn't allocate a bunch of temporary > memory that it'll never use. If an application can't specify minimum or > maximum values, it could always pass MPI_UNDEFINED for the min/max > values. Yes, such tricks may simplify some implementation options. I'm not 100% sure how well they integrate with the current MPI syntax and semantics :-). But I guess recv is also only specifying a maximum. Keep in mind that this discussion may bring us then back to the displs[] array. Thanks & Best, Torsten -- bash$ :(){ :|:&};: --------------------- http://www.unixer.de/ ----- Torsten Hoefler | Performance Modeling and Simulation Lead Blue Waters Directorate | University of Illinois (UIUC) 1205 W Clark Street | Urbana, IL, 61801 NCSA Building | +01 (217) 244-7736

1 0