August 2009 - mpiwg-hybridpm - mailman3.cels.anl.gov

[Mpi3-hybridpm] MPI Hybrid proposal
by Snir Marc 30 Aug '09

30 Aug '09

I put on the wiki a new version of my proposal Marc Snir 4323 Siebel Center, 201 N Goodwin, IL 61801 Tel (217) 244 6568 Web http://www.cs.uiuc.edu/homes/snir

1 0

[Mpi3-hybridpm] Next telecon on 09/02
by Pavan Balaji 21 Aug '09

21 Aug '09

Hi all, Our next telecon will be on 09/02 (Wednesday) with the MPI Forum (see the agenda here: http://meetings.mpi-forum.org/secretary/2009/09/agenda.php) This is a face-to-face meeting, but Jeff has kindly setup a webex for this, so others not attending the forum can participate as well. I'll send out more information on that closer to the meeting. Thanks, -- Pavan -- Pavan Balaji http://www.mcs.anl.gov/~balaji

1 0

Re: [Mpi3-hybridpm] Meeting reminder: 08/18
by Pavan Balaji 18 Aug '09

18 Aug '09

Here are the meeting minutes for the call today: https://svn.mpi-forum.org/trac/mpi-forum-web/wiki/MPI3Hybrid/notes-2009-08-… -- Pavan On 08/16/2009 11:53 AM, Pavan Balaji wrote: > > This is a reminder for the meeting on 08/18 (Tuesday) at 11am central time. > > Passcode: 85444 > Domestic dial number: 888-566-1533 > International dial number: 212-547-0383 > > Agenda: > > 1. Rajeev sent out an initial set of goals for the working group. We > should discuss those, especially with respect to compatibility with > non-threading models. > > 2. We should start looking at an overall proposal from the working group > which will consolidate all the issues we have been looking at. Marc's > proposal seems to form a great starting point, but we'll need to discuss > how the other proposals fall in place with this. (or maybe we can have > multiple proposals?). > > 3. We should start thinking about implementations for the proposals. > > -- Pavan > -- Pavan Balaji http://www.mcs.anl.gov/~balaji

1 0

Re: [Mpi3-hybridpm] Draft of hybrid prog models group goals
by Guillaume Mercier 18 Aug '09

18 Aug '09

Pavan Balaji a e'crit : > I'm mostly concerned whether we will have enough resources and/or > experience in the working group to look into the other models at all. > Whil it's a good goal, it might not be realistic. If we don't, we > might need to restrict the goals of the working group to something > smaller. It's a good point. I think it will largely depend on the experience of people participating in the WG. As Pavan suggested during the Telecon previously, we can send emails about the interfaces/paradigm we want to specifically investigate. As for our group in INRIA, we've been working for quite some time on runtime systems tailored for multithreading. So we are willing to look at this. We also have some people working on the design of a kind of CUDA task scheduler so we are also defintively interested by this point too. Guillaume

1 0

Re: [Mpi3-hybridpm] Draft of hybrid prog models group goals
by Supalov, Alexander 18 Aug '09

18 Aug '09

Thanks. I'm OK with any preamble - mine was a little too direct maybe. I'm certain we should include a reasonably wide scope, because resource allocation and interest to the WG topics will depend on this. The time will show how much we can do in MPI-3. -----Original Message----- From: mpi3-hybridpm-bounces(a)lists.mpi-forum.org [mailto:[email protected]] On Behalf Of Pavan Balaji Sent: Tuesday, August 18, 2009 5:01 PM To: mpi3-hybridpm(a)lists.mpi-forum.org Subject: Re: [Mpi3-hybridpm] Draft of hybrid prog models group goals Thanks Alexander. > Add MPI features necessary for efficient hybrid programming. How about: "Ensure that MPI has the features necessary to facilitate efficient hybrid programming". I picked this statement from Rajeev's summary since it seems to say more than just "adding" MPI features. > * Traditional thread interfaces (e.g., Pthreads, OpenMP) > * Emerging interfaces (like TBB, OpenCL, CUDA, and Ct) > * PGAS (UPC, CAF, etc.) I'm mostly concerned whether we will have enough resources and/or experience in the working group to look into the other models at all. Whil it's a good goal, it might not be realistic. If we don't, we might need to restrict the goals of the working group to something smaller. We can discuss this in the telecon today. -- Pavan -- Pavan Balaji http://www.mcs.anl.gov/~balaji _______________________________________________ Mpi3-hybridpm mailing list Mpi3-hybridpm(a)lists.mpi-forum.org http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-hybridpm --------------------------------------------------------------------- Intel GmbH Dornacher Strasse 1 85622 Feldkirchen/Muenchen Germany Sitz der Gesellschaft: Feldkirchen bei Muenchen Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer Registergericht: Muenchen HRB 47456 Ust.-IdNr. VAT Registration No.: DE129385895 Citibank Frankfurt (BLZ 502 109 00) 600119052 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.

1 0

Re: [Mpi3-hybridpm] Draft of hybrid prog models group goals
by Pavan Balaji 18 Aug '09

18 Aug '09

Thanks Alexander. > Add MPI features necessary for efficient hybrid programming. How about: "Ensure that MPI has the features necessary to facilitate efficient hybrid programming". I picked this statement from Rajeev's summary since it seems to say more than just "adding" MPI features. > * Traditional thread interfaces (e.g., Pthreads, OpenMP) > * Emerging interfaces (like TBB, OpenCL, CUDA, and Ct) > * PGAS (UPC, CAF, etc.) I'm mostly concerned whether we will have enough resources and/or experience in the working group to look into the other models at all. Whil it's a good goal, it might not be realistic. If we don't, we might need to restrict the goals of the working group to something smaller. We can discuss this in the telecon today. -- Pavan -- Pavan Balaji http://www.mcs.anl.gov/~balaji

1 0

Re: [Mpi3-hybridpm] Draft of hybrid prog models group goals
by Supalov, Alexander 18 Aug '09

18 Aug '09

Hi, Thank you. Let me try to propose a slight restructuring/extension. Mission Add MPI features necessary for efficient hybrid programming. Goals Investigate what changes are needed in MPI to better support: * Traditional thread interfaces (e.g., Pthreads, OpenMP) * Emerging interfaces (like TBB, OpenCL, CUDA, and Ct) * PGAS (UPC, CAF, etc.) The scope will change to include other hybrid models as appropriate. Summary Parallel computers are increasingly being built with nodes comprising large numbers of cores that include regular CPUs as well as accelerators such as Cell or GPGPUs. To make better use of shared memory and other resources within a node or address space, users may want to use a hybrid programming model that uses MPI for communicating between nodes or address spaces and some other programming model (X) within the node or address space. Various options for X at present include OpenMP, Pthreads, PGAS languages (UPC, CoArray Fortran), Intel TBB, Cilk, CUDA, OpenCL, and Intel Ct. Best regards. Alexander -----Original Message----- From: mpi3-hybridpm-bounces(a)lists.mpi-forum.org [mailto:[email protected]] On Behalf Of Rajeev Thakur Sent: Thursday, August 13, 2009 10:49 PM To: mpi3-hybridpm(a)lists.mpi-forum.org Subject: [Mpi3-hybridpm] Draft of hybrid prog models group goals All working groups are supposed to come up with a set of goals for the group (i.e., what is the group's mission?). Below is a draft of goals for the hybrid programming models group. It is intended as a starting point for discussion. Please suggest changes as appropriate. Rajeev ---------------- Parallel computers are increasingly being built with nodes comprising large numbers of cores that include regular CPUs as well as accelerators such as Cell or GPUs. To make better use of shared memory and other resources within a node or address space, users may want to use a hybrid programming model that uses MPI for communicating between nodes or address spaces and some other programming model (X) within the node or address space. Various options for X at present include OpenMP, Pthreads, PGAS languages (UPC, CoArray Fortran), Intel TBB, Cilk, CUDA, and OpenCL. The goal of this working group is to ensure that MPI has the features necessary to facilitate efficient hybrid programming. Specifically, the group will investigate: * What changes are needed in MPI to better support threads (Pthreads, OpenMP)? * What changes are needed in MPI to better support MPI+OpenCL or MPI+CUDA programs? * What changes are needed in MPI to better support MPI+UPC or MPI+CAF programs? * (Other hybrid models as appropriate.) _______________________________________________ Mpi3-hybridpm mailing list Mpi3-hybridpm(a)lists.mpi-forum.org http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-hybridpm --------------------------------------------------------------------- Intel GmbH Dornacher Strasse 1 85622 Feldkirchen/Muenchen Germany Sitz der Gesellschaft: Feldkirchen bei Muenchen Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer Registergericht: Muenchen HRB 47456 Ust.-IdNr. VAT Registration No.: DE129385895 Citibank Frankfurt (BLZ 502 109 00) 600119052 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.

1 0

[Mpi3-hybridpm] Meeting reminder: 08/18
by Pavan Balaji 16 Aug '09

16 Aug '09

This is a reminder for the meeting on 08/18 (Tuesday) at 11am central time. Passcode: 85444 Domestic dial number: 888-566-1533 International dial number: 212-547-0383 Agenda: 1. Rajeev sent out an initial set of goals for the working group. We should discuss those, especially with respect to compatibility with non-threading models. 2. We should start looking at an overall proposal from the working group which will consolidate all the issues we have been looking at. Marc's proposal seems to form a great starting point, but we'll need to discuss how the other proposals fall in place with this. (or maybe we can have multiple proposals?). 3. We should start thinking about implementations for the proposals. -- Pavan -- Pavan Balaji http://www.mcs.anl.gov/~balaji

1 0

[Mpi3-hybridpm] Rajeev's and Pavan's questions
by Snir Marc 14 Aug '09

14 Aug '09

> --------------------------------- > > Message: 1 > Date: Thu, 13 Aug 2009 12:37:06 -0500 > From: "Rajeev Thakur" <thakur(a)mcs.anl.gov> > Subject: Re: [Mpi3-hybridpm] MPI + threads / MPI +OpenMP > To: <mpi3-hybridpm(a)lists.mpi-forum.org> > Message-ID: <882ADD2B55184BDC81DB70D0CED7D168(a)mcs.anl.gov> > Content-Type: text/plain; charset="us-ascii" > > Some questions just for clarification: > > * What does MPI_Comm_size of MPI_COMM_TWORLD return -- the total > number > of end points on all processes? Yes -- total number of endpoints > > * For collective operations on MPI_COMM_TWORLD, how many > processes/threads must call them -- exactly one for each end point? If > each process has 11 threads and 5 end points say, the user has to > ensure > that exactly 5 threads on each process call the collective, one for > each > end point? > Yes, a collective communication requires one invocation for each endpoint that is member of the communicator being used. Of course, you are free to use a communicator with only one endpoint at each process, if this is what you want. > > > > > > Sorry about the delay in joining in on this discussion. This is a very > good proposal. A couple of questions: > > 1. I assume the motivation for defining MPI_ENDPOINTS is to allow the > implementation to tell the user how many physical connection resources > it can use (sockets or some other network connection endpoints). > However, it is possible for multiple MPI endpoints to share the same > physical network endpoints. This is obviously inefficient since it'll > bring back all the locking overheads for shared resources, but > probably > not impossible. > > My question is -- do we want to explicitly restrict users from > creating > more end points than the available physical resources? Or is it > beneficial to allow users have any number of MPI endpoints, but have > MPI_ENDPOINTS specify how many it can "efficiently" support? Page 6 says "The [MPI_ENDPOINT_CREATE] call is erroneous if num_endpoints > MPI_ENDPOINTS". The implementation has a defined maximum number of endpoints it supports for each process, and it is erroneous to try to create more. If one uses a hybrid model, there is no strong advantage of having more than one endpoint per process, but for the possible performance benefit. Thus, I do not expect vendors to support an arbitrarily large number of endpoints per process, but only such an (platform and implementation dependent) number that is reasonable, performance wise. Vendors could, for example, allow for more endpoints when there is only one process per node, and fewer endpoints per process, when there are multiple processes per node. > > 2. I didn't quite get the motivation for this statement on page 6: "At > each process, the endpoint with rank 0 in MPI_COMM_PROCESS is also an > endpoint in MPI_COMM_WORLD". Obviously, we want *some* endpoint to > be a > part of COMM_WORLD, but can't the user figure this out at runtime? > Or is > this just for simplicity (which is a good reason, but I just wasn't > sure)? For simplicity -- the "ur-endpoint" has rank zero within MPI_COMM_PROCESS > > Thanks, > > -- Pavan > > -- > Pavan Balaji > http://www.mcs.anl.gov/~balaji > > > ------------------------------ > > _______________________________________________ > Mpi3-hybridpm mailing list > Mpi3-hybridpm(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-hybridpm > > > End of Mpi3-hybridpm Digest, Vol 5, Issue 9 > ******************************************* Marc Snir Michael Faiman and Saburo Muroga Professor Department of Computer Science, University of Illinois at Urbana Champaign 4323 Siebel Center, 201 N Goodwin, IL 61801 Tel (217) 244 6568 Web http://www.cs.uiuc.edu/homes/snir

1 0

[Mpi3-hybridpm] MPI + threads / MPI +OpenMP
by Pavan Balaji 13 Aug '09

13 Aug '09

Sorry about the delay in joining in on this discussion. This is a very good proposal. A couple of questions: 1. I assume the motivation for defining MPI_ENDPOINTS is to allow the implementation to tell the user how many physical connection resources it can use (sockets or some other network connection endpoints). However, it is possible for multiple MPI endpoints to share the same physical network endpoints. This is obviously inefficient since it'll bring back all the locking overheads for shared resources, but probably not impossible. My question is -- do we want to explicitly restrict users from creating more end points than the available physical resources? Or is it beneficial to allow users have any number of MPI endpoints, but have MPI_ENDPOINTS specify how many it can "efficiently" support? 2. I didn't quite get the motivation for this statement on page 6: "At each process, the endpoint with rank 0 in MPI_COMM_PROCESS is also an endpoint in MPI_COMM_WORLD". Obviously, we want *some* endpoint to be a part of COMM_WORLD, but can't the user figure this out at runtime? Or is this just for simplicity (which is a good reason, but I just wasn't sure)? Thanks, -- Pavan -- Pavan Balaji http://www.mcs.anl.gov/~balaji

1 0