March 2011 - mpiwg-rma - mailman3.cels.anl.gov

Re: [Mpi3-rma] Alternative Proposal for Shared Memory Support
by Rajeev Thakur 15 Mar '11

15 Mar '11

Since processes sharing memory can/will access it directly using memory references (not put/get), and this proposal is for the unified memory model, we need to say whether any of the synchronization functions in the RMA chapter are needed at all for such usage. Rajeev On Mar 15, 2011, at 3:25 PM, Torsten Hoefler wrote: > Hello all, > > I apologize for the cross-post but this is of interest to two groups. I > have an alternative proposal for the shared memory support. I believe > that it would naturally integrate into the One Sided chapter as a > special type of window. This would save some additional function calls > and seems to make semantical definitions tighter and the whole standard > more orthogonal. Thanks to Brian, Bill, Marc, and Ron for preliminary > discussions of the idea. > > Here is the proposed new call: > > MPI_Win_allocate_shared(size, info, comm, *baseptr, *win); > > This would create a special window in the "unified" model which is > referencing the same physical memory on all processes in a shared memory > domain. The group of processes that share memory can be queried with > MPI_Win_get_group() and a new MPI_WIN_FLAVOR_SHARED would be added as > window type. > > The whole concept is even mightier because one could now support NCC > shared memory (I remember than Hubert mentioned that in one of the > discussions) if we would allow the "separate" window type for it. > However, this seems to introduce all kinds of complex interactions > (e.g., does everybody logically have it's own public copy of the *same* > memory). I'm not proposing this at this point but it might be worth > a discussion! > > This would make the new proposed calls "MPI_COMM_SHM_FREE(shm)" and > "MPI_SHM_FENCE" superfluous and would enable all synchronizations and > portable (!!) atomic operations that we defined in the RMA chapter > immediately. This would also remove the need to have a special > MPI_COMM_SHM as required in the hybrid proposal. > > Any comments are welcome. I can talk about this at the next meeting > (most likely in the RMA group on Tuesday). > > All the Best, > Torsten > > -- > bash$ :(){ :|:&};: --------------------- http://www.unixer.de/ ----- > Torsten Hoefler | Performance Modeling and Simulation Lead > Blue Waters Directorate | University of Illinois (UIUC) > 1205 W Clark Street | Urbana, IL, 61801 > NCSA Building | +01 (217) 244-7736 > _______________________________________________ > mpi3-rma mailing list > mpi3-rma(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma

1 0

Re: [Mpi3-rma] Alternative Proposal for Shared Memory Support
by Bronis R. de Supinski 15 Mar '11

15 Mar '11

Torsten: I suggested a joint meeting because that will make it clear on the schedule. I was not thinking about adding administrative overhead. I would like to be present. I will likely miss the Tuesday morning sessions due to a long telecon that I need to attend. I think some sort of "best effort" wording will be needed. It is possible that processes are on the same node but that no shared memory is available. In that case distinct physical memory regions could be returned. I am not quite following the group comments. Are you saying that the different windows are returned to different shared memory domains? I thought the window would be available to all processes but physically shared memory is used when possible. How is the group returned? How do you query it? Bronis On Tue, 15 Mar 2011, Torsten Hoefler wrote: > Bronis, > >> What happens if the processes do not have shared physical >> memory? > If the communicator had P processes, then the new window will have P > windows (or groups) with one process in each. > >> Is it just that physical shared memory is used >> when possible? What if multple nodes are included in comm? > Yes, the call is similar to a comm split where all nodes that share a > common memory have the same color and all nodes that don't have > different colors. It will create multiple distinct windows. The user can > query the group to see which processes share a window. > >> Will each node use shared memory? That seems fairly >> reasonable. > Yes, it's basically the same semantics as Ron's original proposal where > a new communicator MPI_COMM_SHM (?) was defined. This functionality will > be assimilated into one call and group query (which is more logical > because one doesn't necessarily need a full communicator). A new use for > groups :-). > >> Overall, I think this proposal is good. I do think a number of details >> have to be worked out before it is ready but I agree that discussing >> it at a Forum meeting is the way to go. > Yes, there will be some discussion, but it could be worth it. > >> Perhaps we could request a joint RMA/Hybrid WG session to discuss it? >> I think it is a mistake to cover it just within one group or the >> other. > Sure, that would be best. We could also just pick one or the other and > make sure all key people from the both working groups are present ;-). > Seems like less administrative overhead. > > Best, > Torsten > > -- > bash$ :(){ :|:&};: --------------------- http://www.unixer.de/ ----- > Torsten Hoefler | Performance Modeling and Simulation Lead > Blue Waters Directorate | University of Illinois (UIUC) > 1205 W Clark Street | Urbana, IL, 61801 > NCSA Building | +01 (217) 244-7736 >

1 0

Re: [Mpi3-rma] Alternative Proposal for Shared Memory Support
by Torsten Hoefler 15 Mar '11

15 Mar '11

Bronis, > What happens if the processes do not have shared physical > memory? If the communicator had P processes, then the new window will have P windows (or groups) with one process in each. > Is it just that physical shared memory is used > when possible? What if multple nodes are included in comm? Yes, the call is similar to a comm split where all nodes that share a common memory have the same color and all nodes that don't have different colors. It will create multiple distinct windows. The user can query the group to see which processes share a window. > Will each node use shared memory? That seems fairly > reasonable. Yes, it's basically the same semantics as Ron's original proposal where a new communicator MPI_COMM_SHM (?) was defined. This functionality will be assimilated into one call and group query (which is more logical because one doesn't necessarily need a full communicator). A new use for groups :-). > Overall, I think this proposal is good. I do think a number of details > have to be worked out before it is ready but I agree that discussing > it at a Forum meeting is the way to go. Yes, there will be some discussion, but it could be worth it. > Perhaps we could request a joint RMA/Hybrid WG session to discuss it? > I think it is a mistake to cover it just within one group or the > other. Sure, that would be best. We could also just pick one or the other and make sure all key people from the both working groups are present ;-). Seems like less administrative overhead. Best, Torsten -- bash$ :(){ :|:&};: --------------------- http://www.unixer.de/ ----- Torsten Hoefler | Performance Modeling and Simulation Lead Blue Waters Directorate | University of Illinois (UIUC) 1205 W Clark Street | Urbana, IL, 61801 NCSA Building | +01 (217) 244-7736

1 0

Re: [Mpi3-rma] Alternative Proposal for Shared Memory Support
by Bronis R. de Supinski 15 Mar '11

15 Mar '11

Torsten: What happens if the processes do not have shared physical memory? Is it just that physical shared memory is used when possible? What if multple nodes are included in comm? Will each node use shared memory? That seems fairly reasonable. Overall, I think this proposal is good. I do think a number of details have to be worked out before it is ready but I agree that discussing it at a Forum meeting is the way to go. Perhaps we could request a joint RMA/Hybrid WG session to discuss it? I think it is a mistake to cover it just within one group or the other. Bronis On Tue, 15 Mar 2011, Torsten Hoefler wrote: > Hello all, > > I apologize for the cross-post but this is of interest to two groups. I > have an alternative proposal for the shared memory support. I believe > that it would naturally integrate into the One Sided chapter as a > special type of window. This would save some additional function calls > and seems to make semantical definitions tighter and the whole standard > more orthogonal. Thanks to Brian, Bill, Marc, and Ron for preliminary > discussions of the idea. > > Here is the proposed new call: > > MPI_Win_allocate_shared(size, info, comm, *baseptr, *win); > > This would create a special window in the "unified" model which is > referencing the same physical memory on all processes in a shared memory > domain. The group of processes that share memory can be queried with > MPI_Win_get_group() and a new MPI_WIN_FLAVOR_SHARED would be added as > window type. > > The whole concept is even mightier because one could now support NCC > shared memory (I remember than Hubert mentioned that in one of the > discussions) if we would allow the "separate" window type for it. > However, this seems to introduce all kinds of complex interactions > (e.g., does everybody logically have it's own public copy of the *same* > memory). I'm not proposing this at this point but it might be worth > a discussion! > > This would make the new proposed calls "MPI_COMM_SHM_FREE(shm)" and > "MPI_SHM_FENCE" superfluous and would enable all synchronizations and > portable (!!) atomic operations that we defined in the RMA chapter > immediately. This would also remove the need to have a special > MPI_COMM_SHM as required in the hybrid proposal. > > Any comments are welcome. I can talk about this at the next meeting > (most likely in the RMA group on Tuesday). > > All the Best, > Torsten > > -- > bash$ :(){ :|:&};: --------------------- http://www.unixer.de/ ----- > Torsten Hoefler | Performance Modeling and Simulation Lead > Blue Waters Directorate | University of Illinois (UIUC) > 1205 W Clark Street | Urbana, IL, 61801 > NCSA Building | +01 (217) 244-7736 > _______________________________________________ > mpi3-rma mailing list > mpi3-rma(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma >

1 0

[Mpi3-rma] Alternative Proposal for Shared Memory Support
by Torsten Hoefler 15 Mar '11

15 Mar '11

Hello all, I apologize for the cross-post but this is of interest to two groups. I have an alternative proposal for the shared memory support. I believe that it would naturally integrate into the One Sided chapter as a special type of window. This would save some additional function calls and seems to make semantical definitions tighter and the whole standard more orthogonal. Thanks to Brian, Bill, Marc, and Ron for preliminary discussions of the idea. Here is the proposed new call: MPI_Win_allocate_shared(size, info, comm, *baseptr, *win); This would create a special window in the "unified" model which is referencing the same physical memory on all processes in a shared memory domain. The group of processes that share memory can be queried with MPI_Win_get_group() and a new MPI_WIN_FLAVOR_SHARED would be added as window type. The whole concept is even mightier because one could now support NCC shared memory (I remember than Hubert mentioned that in one of the discussions) if we would allow the "separate" window type for it. However, this seems to introduce all kinds of complex interactions (e.g., does everybody logically have it's own public copy of the *same* memory). I'm not proposing this at this point but it might be worth a discussion! This would make the new proposed calls "MPI_COMM_SHM_FREE(shm)" and "MPI_SHM_FENCE" superfluous and would enable all synchronizations and portable (!!) atomic operations that we defined in the RMA chapter immediately. This would also remove the need to have a special MPI_COMM_SHM as required in the hybrid proposal. Any comments are welcome. I can talk about this at the next meeting (most likely in the RMA group on Tuesday). All the Best, Torsten -- bash$ :(){ :|:&};: --------------------- http://www.unixer.de/ ----- Torsten Hoefler | Performance Modeling and Simulation Lead Blue Waters Directorate | University of Illinois (UIUC) 1205 W Clark Street | Urbana, IL, 61801 NCSA Building | +01 (217) 244-7736

1 0

Re: [Mpi3-rma] MPI-3 RMA Chapter available for first reading
by William Gropp 13 Mar '11

13 Mar '11

Yes, it should have gone to mpi-forum as well as the RMA working group. Bill On Mar 13, 2011, at 12:48 AM, Graham, Richard L. wrote: > Bill, > Did you intend to post this to the general list ? > > Rich > > ----- Original Message ----- > From: William Gropp [mailto:[email protected]] > Sent: Saturday, March 12, 2011 07:24 PM > To: Main MPI Forum mailing list <mpi-forum(a)lists.mpi-forum.org> > Cc: MPI 3.0 Remote Memory Access working group <mpi3-rma(a)lists.mpi-forum.org > > > Subject: [Mpi3-rma] MPI-3 RMA Chapter available for first reading > > The RMA working group is pleased to announce the MPI-3 RMA chapter is > ready for its first reading. The PDF is available at https://svn.mpi-forum.org/trac/mpi-forum-web/attachment/wiki/mpi3-rma-propo… > ; see one-side-2.pdf . We will read this at the upcoming MPI Forum > meeting. > > William Gropp > Deputy Director for Research > Institute for Advanced Computing Applications and Technologies > Paul and Cynthia Saylor Professor of Computer Science > University of Illinois Urbana-Champaign > > > > > _______________________________________________ > mpi3-rma mailing list > mpi3-rma(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma > > _______________________________________________ > mpi3-rma mailing list > mpi3-rma(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma William Gropp Deputy Director for Research Institute for Advanced Computing Applications and Technologies Paul and Cynthia Saylor Professor of Computer Science University of Illinois Urbana-Champaign

1 0

Re: [Mpi3-rma] MPI-3 RMA Chapter available for first reading
by Graham, Richard L. 13 Mar '11

13 Mar '11

Bill, Did you intend to post this to the general list ? Rich ----- Original Message ----- From: William Gropp [mailto:[email protected]] Sent: Saturday, March 12, 2011 07:24 PM To: Main MPI Forum mailing list <mpi-forum(a)lists.mpi-forum.org> Cc: MPI 3.0 Remote Memory Access working group <mpi3-rma(a)lists.mpi-forum.org> Subject: [Mpi3-rma] MPI-3 RMA Chapter available for first reading The RMA working group is pleased to announce the MPI-3 RMA chapter is ready for its first reading. The PDF is available at https://svn.mpi-forum.org/trac/mpi-forum-web/attachment/wiki/mpi3-rma-propo… ; see one-side-2.pdf . We will read this at the upcoming MPI Forum meeting. William Gropp Deputy Director for Research Institute for Advanced Computing Applications and Technologies Paul and Cynthia Saylor Professor of Computer Science University of Illinois Urbana-Champaign _______________________________________________ mpi3-rma mailing list mpi3-rma(a)lists.mpi-forum.org http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma

1 0

Re: [Mpi3-rma] MPI-3 RMA Chapter available for first reading
by Bronis R. de Supinski 12 Mar '11

12 Mar '11

OK, I will use this version for the rest of my review. On Sat, 12 Mar 2011, William Gropp wrote: > The RMA working group is pleased to announce the MPI-3 RMA chapter is > ready for its first reading. The PDF is available at https://svn.mpi-forum.org/trac/mpi-forum-web/attachment/wiki/mpi3-rma-propo… > ; see one-side-2.pdf . We will read this at the upcoming MPI Forum > meeting. > > William Gropp > Deputy Director for Research > Institute for Advanced Computing Applications and Technologies > Paul and Cynthia Saylor Professor of Computer Science > University of Illinois Urbana-Champaign > > > > > _______________________________________________ > mpi3-rma mailing list > mpi3-rma(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma >

1 0

[Mpi3-rma] MPI-3 RMA Chapter available for first reading
by William Gropp 12 Mar '11

12 Mar '11

The RMA working group is pleased to announce the MPI-3 RMA chapter is ready for its first reading. The PDF is available at https://svn.mpi-forum.org/trac/mpi-forum-web/attachment/wiki/mpi3-rma-propo… ; see one-side-2.pdf . We will read this at the upcoming MPI Forum meeting. William Gropp Deputy Director for Research Institute for Advanced Computing Applications and Technologies Paul and Cynthia Saylor Professor of Computer Science University of Illinois Urbana-Champaign

1 0

[Mpi3-rma] New MPI-3 RMA Chapter available for Reading
by William Gropp 12 Mar '11

12 Mar '11

I have committed and uploaded a new version of the chapter. This includes the changes I noted, as well as a number of formatting fixes. Please have a quick look at this; I plan to announce it to the full forum later today. Bill William Gropp Deputy Director for Research Institute for Advanced Computing Applications and Technologies Paul and Cynthia Saylor Professor of Computer Science University of Illinois Urbana-Champaign

1 0