February 2011 - mpiwg-hybridpm - mailman3.cels.anl.gov

[Mpi3-hybridpm] MPI_HELPER_TEAM_CREATE description of INFO
by Douglas Miller 17 Feb '11

17 Feb '11

Hi Pavan, I'm still looking for a description of the INFO keys "balanced" and "singular" that I like. I guess what seems odd to me about the description in v1.0 is that "balanced" is described in terms of what the implementation does, while "singular" is in terms of what the application does. Wouldn't it be less confusing if both spoke to how the application writer can determine if their code fits a particular model? The way I imagine it, the application writer will have a block of code (in mind or in source files) that does MPI calls and they think it is a candidate for parallelism using a helper team. They would then compare their code to the requirements for "balanced" and "singular" and set them as appropriate. I guess we would then have "advice to implementers" that helps guide the implementer on how they should handle the INFO. would you agree? I can make an attempt to come up with some text for "balanced" from the application developer's point of view. thanks, _______________________________________________ Douglas Miller BlueGene Messaging Development IBM Corp., Rochester, MN USA Bldg 030-2 A410 dougmill(a)us.ibm.com Douglas Miller/Rochester/IBM

1 0

Re: [Mpi3-hybridpm] SHMEM Interface in Draft External Interfaces
by Hubert Ritzdorf 17 Feb '11

17 Feb '11

Yes, this is the problem. I'm working also in a European project for Climate modeling which use coupled applications/models for simulation. They may start with a single process, which dynamically spawns new parallel applications/models depending on the simulation. Coupled application exchange 3d volume data and could put common chemistry data into shared memory. Thus, it makes sense to place processes of different applications/models on the same node (or on another node on systems which provide shared memory over several nodes). In order to create the proposed MPI_COMM_SHM for MPI_COMM_WORLD in MPI_Init(), the mpi implementer would probably use also MPI_Comm_split() with an appropriate color. Thus, we should provide the corresponding functionality to the user and he/she can create the communicator. For dynamic applications, an additional MPI_Comm_shm_attach () function would be useful in order to attach to already existing shared memory regions. Hubert > -----Original Message----- > From: Snir, Marc [mailto:[email protected]] > Sent: Thursday, February 17, 2011 4:21 AM > To: Hubert Ritzdorf > Cc: mpi3-hybridpm(a)lists.mpi-forum.org; Hubert Ritzdorf > Subject: Re: [Mpi3-hybridpm] SHMEM Interface in Draft External Interfaces > > Could explain the problem with dynamic process management? The only > potential problem would be if dynamic process management is used to add > new processes on existing nodes; if the allocation is in full nodes, I see > no problem. > > Marc Snir > 4323 Siebel Center, 201 N Goodwin, IL 61801 > Tel (217) 244 6568 > Web http://www.cs.uiuc.edu/homes/snir > > On 2/16/11 11:32 AM, "Hubert Ritzdorf" > <hritzdorf(a)hpce.nec.com<mailto:[email protected]>> wrote: > > Hi, > > I have 2 notes to chapter MPI and Shared Memory in External Interface > draft "ei-2-v1.0.pdf": > > (*) I am missing a function MPI_Shmem_flush(shm) > which flushes the shared memory on not cache-coherent systems > (corresponding to shmem functions shmem_udcflush() or > shmem_udcflush_line()). > > (*) The concept of > the intracommunicator MPI_COMM_SHM of all processes the local process > can share > doesn't work for dynamic process management. I would propose > > (*) to define a special constant MPI_COLOR_SHM and > (*) to create a communicator of all processes the local process > can share > in communicator comm. with function > > MPI_Comm_split (comm., MPI_COLOR_SHM, key, comm._shm) > > Best regards > > Hubert Ritzdorf > NEC Germany

1 0

Re: [Mpi3-hybridpm] SHMEM Interface in Draft External Interfaces
by Snir, Marc 16 Feb '11

16 Feb '11

Could explain the problem with dynamic process management? The only potential problem would be if dynamic process management is used to add new processes on existing nodes; if the allocation is in full nodes, I see no problem. Marc Snir 4323 Siebel Center, 201 N Goodwin, IL 61801 Tel (217) 244 6568 Web http://www.cs.uiuc.edu/homes/snir On 2/16/11 11:32 AM, "Hubert Ritzdorf" <hritzdorf(a)hpce.nec.com<mailto:[email protected]>> wrote: Hi, I have 2 notes to chapter MPI and Shared Memory in External Interface draft ï¿½ei-2-v1.0.pdfï¿½: (*) I am missing a function MPI_Shmem_flush(shm) which flushes the shared memory on not cache-coherent systems (corresponding to shmem functions shmem_udcflush() or shmem_udcflush_line()). (*) The concept of the intracommunicator MPI_COMM_SHM of all processes the local process can share doesnï¿½t work for dynamic process management. I would propose (*) to define a special constant MPI_COLOR_SHM and (*) to create a communicator of all processes the local process can share in communicator comm. with function MPI_Comm_split (comm., MPI_COLOR_SHM, key, comm._shm) Best regards Hubert Ritzdorf NEC Germany

1 0

Re: [Mpi3-hybridpm] Regarding making changes to Open-MP to eliminate need for MPI_HELPER_TEAM
by Douglas Miller 16 Feb '11

16 Feb '11

OK, I think that is a lot easier to respond to. I have re-phrased my question to the compiler folks and will compose some text to explain why that idea won't work, or at least defeats the purpose. _______________________________________________ Douglas Miller BlueGene Messaging Development IBM Corp., Rochester, MN USA Bldg 030-2 A410 dougmill(a)us.ibm.com Douglas Miller/Rochester/IBM "Bronis R. de Supinski" <bronis(a)llnl.gov> To Sent by: "mpi3-hybridpm(a)lists.mpi-forum.org" mpi3-hybridpm-bou <mpi3-hybridpm(a)lists.mpi-forum.org> nces(a)lists.mpi-fo cc rum.org Subject Re: [Mpi3-hybridpm] Regarding 02/16/2011 11:45 making changes to Open-MP to AM eliminate need for MPI_HELPER_TEAM Please respond to "Bronis R. de Supinski" <bronis(a)llnl.gov> ; Please respond to mpi3-hybridpm@lis ts.mpi-forum.org Yes, as if by magic. On Wed, 16 Feb 2011, Douglas Miller wrote: > Oh, that's different than what I was thinking. I thought there was some extension to Open-MP being proposed. > > OK, so the suggestion was that MPI could just do a "#pragma omp parallel" (or equivalent) internally? And if the omp runtime/compiler was "smart" enough it would work well? > > > _______________________________________________ > Douglas Miller BlueGene Messaging Development > IBM Corp., Rochester, MN USA Bldg 030-2 A410 > dougmill(a)us.ibm.com Douglas Miller/Rochester/IBM > > [cid:[email protected]]"Bronis R. de Supinski" ---02/16/2011 11:28:04 AM---Doug: > > > "Bronis R. de Supinski" <bronis(a)llnl.gov> > Sent by: mpi3-hybridpm-bounces(a)lists.mpi-forum.org > > 02/16/2011 11:26 AM > Please respond to > "Bronis R. de Supinski" <bronis(a)llnl.gov>; Please respond to > mpi3-hybridpm(a)lists.mpi-forum.org > > > > > To > > > "mpi3-hybridpm(a)lists.mpi-forum.org" <mpi3-hybridpm(a)lists.mpi-forum.org> > > > > cc > > > > Subject > > > Re: [Mpi3-hybridpm] Regarding making changes to Open-MP to eliminate need for MPI_HELPER_TEAM > > > > > > > Doug: > > The suggested was that nested parallelism would magically > result in the MPI thread being able to generate more threads. > > Bronis > > > On Wed, 16 Feb 2011, Douglas Miller wrote: > >> I'm am still discussing this with compiler folks here at IBM, but I was reminded of a point that I think makes the question moot. >> >> Even if open-mp allows the user to get control of threads (or tasks) when they go idle, it doesn't solve the problem which is how does a task that (otherwise) makes no MPI calls get involved in other thread's MPI calls? There is currently no interface into MPI that provides a general-purpose "I want to help others" action. At the very least we'd have to modify the interface or semantics to MPI_WAIT/MPI_TEST to allow for calling without a request (essentially creating MPI_HELPER_TEAM_YIELD). >> >> Unless the idea was that MPI itself somehow registered itself with the open-mp runtime such that the application had no idea. Then I suppose the MPI implementation could provide some internal (MPID) function that did what was necessary. But that raises all sorts of questions for things like applications that use both MPI and UPC, who wins the registration - MPI or UPC? Not to mention that if this were a public open-mp feature what's to stop the application from also wanting to register a function? A lot depends on exactly how this were implemented and exposed. >> >> and then there's the issue of applications that do threading some other way - pthreads, fork, whatever - does every potential threading runtime have to be modified? or how does the user accomplish this same thing without Open-MP? It just seems more universally-useful if MPI does this, rather than every threading platform in the world. Of course, things like UPC and GA/ARMCI also need to provide similar features. It just seems more problematic trying to get OSes and Compilers to add this. >> >> I think I'm rambling-on too much. Maybe no one needs to respond to this - it is just input for our next discussion. >> >> Anyway, I'm still exchanging e-mail with compiler folks on this. Right now, they are having a hard time understanding what I'm asking for. I guess the idea is so vague that I can't even give them sample code or a hypothetical interface. >> >> >> _______________________________________________ >> Douglas Miller BlueGene Messaging Development >> IBM Corp., Rochester, MN USA Bldg 030-2 A410 >> dougmill(a)us.ibm.com Douglas Miller/Rochester/IBM >> > _______________________________________________ > Mpi3-hybridpm mailing list > Mpi3-hybridpm(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-hybridpm > > _______________________________________________ Mpi3-hybridpm mailing list Mpi3-hybridpm(a)lists.mpi-forum.org http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-hybridpm

1 0

Re: [Mpi3-hybridpm] Regarding making changes to Open-MP to eliminate need for MPI_HELPER_TEAM
by Bronis R. de Supinski 16 Feb '11

16 Feb '11

Yes, as if by magic. On Wed, 16 Feb 2011, Douglas Miller wrote: > Oh, that's different than what I was thinking. I thought there was some extension to Open-MP being proposed. > > OK, so the suggestion was that MPI could just do a "#pragma omp parallel" (or equivalent) internally? And if the omp runtime/compiler was "smart" enough it would work well? > > > _______________________________________________ > Douglas Miller BlueGene Messaging Development > IBM Corp., Rochester, MN USA Bldg 030-2 A410 > dougmill(a)us.ibm.com Douglas Miller/Rochester/IBM > > [cid:[email protected]]"Bronis R. de Supinski" ---02/16/2011 11:28:04 AM---Doug: > > > "Bronis R. de Supinski" <bronis(a)llnl.gov> > Sent by: mpi3-hybridpm-bounces(a)lists.mpi-forum.org > > 02/16/2011 11:26 AM > Please respond to > "Bronis R. de Supinski" <bronis(a)llnl.gov>; Please respond to > mpi3-hybridpm(a)lists.mpi-forum.org > > > > > To > > > "mpi3-hybridpm(a)lists.mpi-forum.org" <mpi3-hybridpm(a)lists.mpi-forum.org> > > > > cc > > > > Subject > > > Re: [Mpi3-hybridpm] Regarding making changes to Open-MP to eliminate need for MPI_HELPER_TEAM > > > > > > > Doug: > > The suggested was that nested parallelism would magically > result in the MPI thread being able to generate more threads. > > Bronis > > > On Wed, 16 Feb 2011, Douglas Miller wrote: > >> I'm am still discussing this with compiler folks here at IBM, but I was reminded of a point that I think makes the question moot. >> >> Even if open-mp allows the user to get control of threads (or tasks) when they go idle, it doesn't solve the problem which is how does a task that (otherwise) makes no MPI calls get involved in other thread's MPI calls? There is currently no interface into MPI that provides a general-purpose "I want to help others" action. At the very least we'd have to modify the interface or semantics to MPI_WAIT/MPI_TEST to allow for calling without a request (essentially creating MPI_HELPER_TEAM_YIELD). >> >> Unless the idea was that MPI itself somehow registered itself with the open-mp runtime such that the application had no idea. Then I suppose the MPI implementation could provide some internal (MPID) function that did what was necessary. But that raises all sorts of questions for things like applications that use both MPI and UPC, who wins the registration - MPI or UPC? Not to mention that if this were a public open-mp feature what's to stop the application from also wanting to register a function? A lot depends on exactly how this were implemented and exposed. >> >> and then there's the issue of applications that do threading some other way - pthreads, fork, whatever - does every potential threading runtime have to be modified? or how does the user accomplish this same thing without Open-MP? It just seems more universally-useful if MPI does this, rather than every threading platform in the world. Of course, things like UPC and GA/ARMCI also need to provide similar features. It just seems more problematic trying to get OSes and Compilers to add this. >> >> I think I'm rambling-on too much. Maybe no one needs to respond to this - it is just input for our next discussion. >> >> Anyway, I'm still exchanging e-mail with compiler folks on this. Right now, they are having a hard time understanding what I'm asking for. I guess the idea is so vague that I can't even give them sample code or a hypothetical interface. >> >> >> _______________________________________________ >> Douglas Miller BlueGene Messaging Development >> IBM Corp., Rochester, MN USA Bldg 030-2 A410 >> dougmill(a)us.ibm.com Douglas Miller/Rochester/IBM >> > _______________________________________________ > Mpi3-hybridpm mailing list > Mpi3-hybridpm(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-hybridpm > >

1 0

Re: [Mpi3-hybridpm] Regarding making changes to Open-MP to eliminate need for MPI_HELPER_TEAM
by Douglas Miller 16 Feb '11

16 Feb '11

Oh, that's different than what I was thinking. I thought there was some extension to Open-MP being proposed. OK, so the suggestion was that MPI could just do a "#pragma omp parallel" (or equivalent) internally? And if the omp runtime/compiler was "smart" enough it would work well? _______________________________________________ Douglas Miller BlueGene Messaging Development IBM Corp., Rochester, MN USA Bldg 030-2 A410 dougmill(a)us.ibm.com Douglas Miller/Rochester/IBM "Bronis R. de Supinski" <bronis(a)llnl.gov> To Sent by: "mpi3-hybridpm(a)lists.mpi-forum.org" mpi3-hybridpm-bou <mpi3-hybridpm(a)lists.mpi-forum.org> nces(a)lists.mpi-fo cc rum.org Subject Re: [Mpi3-hybridpm] Regarding 02/16/2011 11:26 making changes to Open-MP to AM eliminate need for MPI_HELPER_TEAM Please respond to "Bronis R. de Supinski" <bronis(a)llnl.gov> ; Please respond to mpi3-hybridpm@lis ts.mpi-forum.org Doug: The suggested was that nested parallelism would magically result in the MPI thread being able to generate more threads. Bronis On Wed, 16 Feb 2011, Douglas Miller wrote: > I'm am still discussing this with compiler folks here at IBM, but I was reminded of a point that I think makes the question moot. > > Even if open-mp allows the user to get control of threads (or tasks) when they go idle, it doesn't solve the problem which is how does a task that (otherwise) makes no MPI calls get involved in other thread's MPI calls? There is currently no interface into MPI that provides a general-purpose "I want to help others" action. At the very least we'd have to modify the interface or semantics to MPI_WAIT/MPI_TEST to allow for calling without a request (essentially creating MPI_HELPER_TEAM_YIELD). > > Unless the idea was that MPI itself somehow registered itself with the open-mp runtime such that the application had no idea. Then I suppose the MPI implementation could provide some internal (MPID) function that did what was necessary. But that raises all sorts of questions for things like applications that use both MPI and UPC, who wins the registration - MPI or UPC? Not to mention that if this were a public open-mp feature what's to stop the application from also wanting to register a function? A lot depends on exactly how this were implemented and exposed. > > and then there's the issue of applications that do threading some other way - pthreads, fork, whatever - does every potential threading runtime have to be modified? or how does the user accomplish this same thing without Open-MP? It just seems more universally-useful if MPI does this, rather than every threading platform in the world. Of course, things like UPC and GA/ARMCI also need to provide similar features. It just seems more problematic trying to get OSes and Compilers to add this. > > I think I'm rambling-on too much. Maybe no one needs to respond to this - it is just input for our next discussion. > > Anyway, I'm still exchanging e-mail with compiler folks on this. Right now, they are having a hard time understanding what I'm asking for. I guess the idea is so vague that I can't even give them sample code or a hypothetical interface. > > > _______________________________________________ > Douglas Miller BlueGene Messaging Development > IBM Corp., Rochester, MN USA Bldg 030-2 A410 > dougmill(a)us.ibm.com Douglas Miller/Rochester/IBM > _______________________________________________ Mpi3-hybridpm mailing list Mpi3-hybridpm(a)lists.mpi-forum.org http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-hybridpm

1 0

[Mpi3-hybridpm] SHMEM Interface in Draft External Interfaces
by Hubert Ritzdorf 16 Feb '11

16 Feb '11

Hi, I have 2 notes to chapter MPI and Shared Memory in External Interface draft "ei-2-v1.0.pdf": (*) I am missing a function MPI_Shmem_flush(shm) which flushes the shared memory on not cache-coherent systems (corresponding to shmem functions shmem_udcflush() or shmem_udcflush_line()). (*) The concept of the intracommunicator MPI_COMM_SHM of all processes the local process can share doesn't work for dynamic process management. I would propose (*) to define a special constant MPI_COLOR_SHM and (*) to create a communicator of all processes the local process can share in communicator comm. with function MPI_Comm_split (comm., MPI_COLOR_SHM, key, comm._shm) Best regards Hubert Ritzdorf NEC Germany

1 0

Re: [Mpi3-hybridpm] Regarding making changes to Open-MP to eliminate need for MPI_HELPER_TEAM
by Bronis R. de Supinski 16 Feb '11

16 Feb '11

Doug: The suggested was that nested parallelism would magically result in the MPI thread being able to generate more threads. Bronis On Wed, 16 Feb 2011, Douglas Miller wrote: > I'm am still discussing this with compiler folks here at IBM, but I was reminded of a point that I think makes the question moot. > > Even if open-mp allows the user to get control of threads (or tasks) when they go idle, it doesn't solve the problem which is how does a task that (otherwise) makes no MPI calls get involved in other thread's MPI calls? There is currently no interface into MPI that provides a general-purpose "I want to help others" action. At the very least we'd have to modify the interface or semantics to MPI_WAIT/MPI_TEST to allow for calling without a request (essentially creating MPI_HELPER_TEAM_YIELD). > > Unless the idea was that MPI itself somehow registered itself with the open-mp runtime such that the application had no idea. Then I suppose the MPI implementation could provide some internal (MPID) function that did what was necessary. But that raises all sorts of questions for things like applications that use both MPI and UPC, who wins the registration - MPI or UPC? Not to mention that if this were a public open-mp feature what's to stop the application from also wanting to register a function? A lot depends on exactly how this were implemented and exposed. > > and then there's the issue of applications that do threading some other way - pthreads, fork, whatever - does every potential threading runtime have to be modified? or how does the user accomplish this same thing without Open-MP? It just seems more universally-useful if MPI does this, rather than every threading platform in the world. Of course, things like UPC and GA/ARMCI also need to provide similar features. It just seems more problematic trying to get OSes and Compilers to add this. > > I think I'm rambling-on too much. Maybe no one needs to respond to this - it is just input for our next discussion. > > Anyway, I'm still exchanging e-mail with compiler folks on this. Right now, they are having a hard time understanding what I'm asking for. I guess the idea is so vague that I can't even give them sample code or a hypothetical interface. > > > _______________________________________________ > Douglas Miller BlueGene Messaging Development > IBM Corp., Rochester, MN USA Bldg 030-2 A410 > dougmill(a)us.ibm.com Douglas Miller/Rochester/IBM >

1 0

[Mpi3-hybridpm] Regarding making changes to Open-MP to eliminate need for MPI_HELPER_TEAM
by Douglas Miller 16 Feb '11

16 Feb '11

I'm am still discussing this with compiler folks here at IBM, but I was reminded of a point that I think makes the question moot. Even if open-mp allows the user to get control of threads (or tasks) when they go idle, it doesn't solve the problem which is how does a task that (otherwise) makes no MPI calls get involved in other thread's MPI calls? There is currently no interface into MPI that provides a general-purpose "I want to help others" action. At the very least we'd have to modify the interface or semantics to MPI_WAIT/MPI_TEST to allow for calling without a request (essentially creating MPI_HELPER_TEAM_YIELD). Unless the idea was that MPI itself somehow registered itself with the open-mp runtime such that the application had no idea. Then I suppose the MPI implementation could provide some internal (MPID) function that did what was necessary. But that raises all sorts of questions for things like applications that use both MPI and UPC, who wins the registration - MPI or UPC? Not to mention that if this were a public open-mp feature what's to stop the application from also wanting to register a function? A lot depends on exactly how this were implemented and exposed. and then there's the issue of applications that do threading some other way - pthreads, fork, whatever - does every potential threading runtime have to be modified? or how does the user accomplish this same thing without Open-MP? It just seems more universally-useful if MPI does this, rather than every threading platform in the world. Of course, things like UPC and GA/ARMCI also need to provide similar features. It just seems more problematic trying to get OSes and Compilers to add this. I think I'm rambling-on too much. Maybe no one needs to respond to this - it is just input for our next discussion. Anyway, I'm still exchanging e-mail with compiler folks on this. Right now, they are having a hard time understanding what I'm asking for. I guess the idea is so vague that I can't even give them sample code or a hypothetical interface. _______________________________________________ Douglas Miller BlueGene Messaging Development IBM Corp., Rochester, MN USA Bldg 030-2 A410 dougmill(a)us.ibm.com Douglas Miller/Rochester/IBM

1 0

[Mpi3-hybridpm] Telecon tomorrow
by Pavan Balaji 14 Feb '11

14 Feb '11

All, We'll have our next telecon tomorrow at 11am central time. We'll discuss the feedback from the forum and the next steps. Domestic dial number: 866-654-6744 International dial number: 517-308-8226 Passcode: 6086006 Thanks, -- Pavan -- Pavan Balaji http://www.mcs.anl.gov/~balaji

1 0