January 2021 - mpiwg-sessions - mailman3.cels.anl.gov

cancel today's Sessions WG meeting
by Pritchard Jr., Howard 25 Jan '21

25 Jan '21

Hi All, A number of us have conflicts with today’s WG meeting, so canceling. We’ll resume next week. Howard -- Howard Pritchard HPC-ENV Los Alamos National Laboratory

1 0

Re: [mpiwg-sessions] [EXTERNAL] Better handling of dynamic mixed with sessions without world
by Pritchard Jr., Howard 22 Jan '21

22 Jan '21

HI Dan, Ah yes now I recall better. Let’s discuss on Monday. Howard From: HOLMES Daniel <d.holmes(a)epcc.ed.ac.uk> Date: Friday, January 22, 2021 at 8:56 AM To: "Pritchard Jr., Howard" <howardp(a)lanl.gov> Cc: "schulzm(a)in.tum.de" <schulzm(a)in.tum.de>, MPI Sessions working group <mpiwg-sessions(a)lists.mpi-forum.org> Subject: Re: [EXTERNAL] Better handling of dynamic mixed with sessions without world Hi Howard, I (half-)remember the discussions you mention - they mostly focused on the spawning side, rather than the being spawned side, but there was discussion of which process set(s) would need to exist for the parents and children to find each other. The waters were muddier because we were considering a replacement for MPI_Comm_spawn (called MPI_Exec) that did not return/output an inter-communicator but expected the user to use the process set->group->comm method to find the processes that it just created. Thus, we thought about needing multiple standardised names - whereas for this simpler approach we only need one because it is guaranteed to have unique meaning independent of the number of times spawn is called (because only the spawned processes see it, not the processes that executed spawn). This proposal would not exclude or prevent a DPM-2.0 approach that followed up on the ideas in the old slide deck. There is no chance of getting this into MPI-4.0, although I do wish I’d thought of this earlier, i.e. early enough to have included it at the same time as the rest of SPM-1.0 so we should push on this for the MPI-4.1 timeframe. Cheers, Dan. — Dr Daniel Holmes PhD Architect (HPC Research) d.holmes(a)epcc.ed.ac.uk<mailto:[email protected]> Phone: +44 (0) 131 651 3465 Mobile: +44 (0) 7940 524 088 Address: Room 2.09, Bayes Centre, 47 Potterrow, Central Area, Edinburgh, EH8 9BT — The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. — On 22 Jan 2021, at 15:38, Pritchard Jr., Howard <howardp(a)lanl.gov<mailto:[email protected]>> wrote: This email was sent to you by someone outside the University. You should only click on links or attachments if you are certain that the email is genuine and the content is safe. Hi Dan, I think this is a good idea. I’m not sure we entirely missed it though for sessions 1.0. I vaguely recall discussions about something like this in the context of mpi_exec – but it was quite a while ago. Maybe some problems/limitations were found with this process set approach to capturing info to support spawn functionality. We should probably double check the old slide deck. This would be a 4.1 proposal right? Howard From: HOLMES Daniel <d.holmes(a)epcc.ed.ac.uk<mailto:[email protected]>> Date: Friday, January 22, 2021 at 5:06 AM To: "Pritchard Jr., Howard" <howardp(a)lanl.gov<mailto:[email protected]>>, Martin Schulz <schulzm(a)in.tum.de<mailto:[email protected]>> Cc: MPI Sessions working group <mpiwg-sessions(a)lists.mpi-forum.org<mailto:[email protected]>> Subject: [EXTERNAL] Better handling of dynamic mixed with sessions without world Hi all, I just reviewed PR 506 from Rolf. It adds predefined communicator names to Appendix A. One of those is “MPI_COMM_PARENT”. It is a correct change and only indirectly relevant because it is what set off an interesting train of thought, which I summarise below. This suggests that we missed a trick when proposing Sessions v1.0 - we should have mandated the existence and meaning for an additional process set name: "mpi://MPI_COMM_PARENT” (note the absence of “GET” in that name to match the default string name assigned by MPI to the communicator returned by the MPI_COMM_GET_PARENT procedure). This process set shall contain all the processes that would be in the communicator returned by MPI_COMM_GET_PARENT, either zero processes (if you would get MPI_COMM_NULL) or the union of the local and remote groups (if you would get an inter-communicator). With this addition, MPI processes that are spawned can regain all of the functionality of the Dynamic Model without calling MPI_INIT[_THREAD], i.e. they can create the inter-communicator that they would have got from MPI_COMM_GET_PARENT (which requires a prior call to MPI_INIT because World Model [ED: please check]). ```pseudo-code Create a session Create an MPI_Group, groupParent, from the process set named "mpi://MPI_COMM_PARENT” Create an MPI_Group, groupWorld, from the process set named “mpi://MPI_COMM_WORLD” If (groupParent if MPI_GROUP_NULL) then This process was not spawned by other MPI processes Else Create groupLocal from the intersection of groupWorld and groupParent Create groupRemote from the subtraction of groupWorld from groupParent Create commParent using MPI_COMM_CREATE_FROM_GROUPS with groupLocal and groupRemote ``` No usage of the World Model, but we now have (a duplicate of) the inter-communicator that would have been returned by MPI_COMM_GET_PARENT if we had been permitted to call it. For each MPI process, there is a unique “parent” process set by this definition. Multiple components doing this pseudo-code would get duplicate communicators with their own life-cycle. This contrasts with the current way whereby freeing or disconnecting any communicator handle provided by MPI_COMM_GET_PARENT makes all other such handles stale, which is another instance of global state in MPI baked into the standardised interface and another anathema to any attempt at isolation. Of course, there is already no reason why the parent processes could not have used the Sessions Model to call MPI_COMM_SPAWN (passing in a `comm` that was derived from the Sessions Model), so this completes the picture of how the Dynamic Model can be bolted on successfully to either the World Model (refer to MPI-3.1 and prior) or the Sessions Model (refer to MPI-4.1, which will include this addition) or a mixture (refer to MPI-4.0, without this addition). [EDIT: if we decide that MPI_COMM_GET_PARENT can already be called without/before/after the World Model, then we should add it to Table 11.1 and write some text about it in §11.8.2. In that scenario, the above provides an implementation route for that procedure using only the Sessions Model underneath - in a similar way to the observation that it is possible to implement MPI_INIT[_THREAD] using only the Sessions Model underneath.] Cheers, Dan. — Dr Daniel Holmes PhD Architect (HPC Research) d.holmes(a)epcc.ed.ac.uk<mailto:[email protected]> Phone: +44 (0) 131 651 3465 Mobile: +44 (0) 7940 524 088 Address: Room 2.09, Bayes Centre, 47 Potterrow, Central Area, Edinburgh, EH8 9BT — The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. —

1 0

Re: [mpiwg-sessions] [EXTERNAL] Better handling of dynamic mixed with sessions without world
by HOLMES Daniel 22 Jan '21

22 Jan '21

Hi Howard, I (half-)remember the discussions you mention - they mostly focused on the spawning side, rather than the being spawned side, but there was discussion of which process set(s) would need to exist for the parents and children to find each other. The waters were muddier because we were considering a replacement for MPI_Comm_spawn (called MPI_Exec) that did not return/output an inter-communicator but expected the user to use the process set->group->comm method to find the processes that it just created. Thus, we thought about needing multiple standardised names - whereas for this simpler approach we only need one because it is guaranteed to have unique meaning independent of the number of times spawn is called (because only the spawned processes see it, not the processes that executed spawn). This proposal would not exclude or prevent a DPM-2.0 approach that followed up on the ideas in the old slide deck. There is no chance of getting this into MPI-4.0, although I do wish I’d thought of this earlier, i.e. early enough to have included it at the same time as the rest of SPM-1.0 so we should push on this for the MPI-4.1 timeframe. Cheers, Dan. — Dr Daniel Holmes PhD Architect (HPC Research) d.holmes(a)epcc.ed.ac.uk<mailto:[email protected]> Phone: +44 (0) 131 651 3465 Mobile: +44 (0) 7940 524 088 Address: Room 2.09, Bayes Centre, 47 Potterrow, Central Area, Edinburgh, EH8 9BT — The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. — On 22 Jan 2021, at 15:38, Pritchard Jr., Howard <howardp(a)lanl.gov<mailto:[email protected]>> wrote: This email was sent to you by someone outside the University. You should only click on links or attachments if you are certain that the email is genuine and the content is safe. Hi Dan, I think this is a good idea. I’m not sure we entirely missed it though for sessions 1.0. I vaguely recall discussions about something like this in the context of mpi_exec – but it was quite a while ago. Maybe some problems/limitations were found with this process set approach to capturing info to support spawn functionality. We should probably double check the old slide deck. This would be a 4.1 proposal right? Howard From: HOLMES Daniel <d.holmes(a)epcc.ed.ac.uk<mailto:[email protected]>> Date: Friday, January 22, 2021 at 5:06 AM To: "Pritchard Jr., Howard" <howardp(a)lanl.gov<mailto:[email protected]>>, Martin Schulz <schulzm(a)in.tum.de<mailto:[email protected]>> Cc: MPI Sessions working group <mpiwg-sessions(a)lists.mpi-forum.org<mailto:[email protected]>> Subject: [EXTERNAL] Better handling of dynamic mixed with sessions without world Hi all, I just reviewed PR 506 from Rolf. It adds predefined communicator names to Appendix A. One of those is “MPI_COMM_PARENT”. It is a correct change and only indirectly relevant because it is what set off an interesting train of thought, which I summarise below. This suggests that we missed a trick when proposing Sessions v1.0 - we should have mandated the existence and meaning for an additional process set name: "mpi://MPI_COMM_PARENT” (note the absence of “GET” in that name to match the default string name assigned by MPI to the communicator returned by the MPI_COMM_GET_PARENT procedure). This process set shall contain all the processes that would be in the communicator returned by MPI_COMM_GET_PARENT, either zero processes (if you would get MPI_COMM_NULL) or the union of the local and remote groups (if you would get an inter-communicator). With this addition, MPI processes that are spawned can regain all of the functionality of the Dynamic Model without calling MPI_INIT[_THREAD], i.e. they can create the inter-communicator that they would have got from MPI_COMM_GET_PARENT (which requires a prior call to MPI_INIT because World Model [ED: please check]). ```pseudo-code Create a session Create an MPI_Group, groupParent, from the process set named "mpi://MPI_COMM_PARENT” Create an MPI_Group, groupWorld, from the process set named “mpi://MPI_COMM_WORLD” If (groupParent if MPI_GROUP_NULL) then This process was not spawned by other MPI processes Else Create groupLocal from the intersection of groupWorld and groupParent Create groupRemote from the subtraction of groupWorld from groupParent Create commParent using MPI_COMM_CREATE_FROM_GROUPS with groupLocal and groupRemote ``` No usage of the World Model, but we now have (a duplicate of) the inter-communicator that would have been returned by MPI_COMM_GET_PARENT if we had been permitted to call it. For each MPI process, there is a unique “parent” process set by this definition. Multiple components doing this pseudo-code would get duplicate communicators with their own life-cycle. This contrasts with the current way whereby freeing or disconnecting any communicator handle provided by MPI_COMM_GET_PARENT makes all other such handles stale, which is another instance of global state in MPI baked into the standardised interface and another anathema to any attempt at isolation. Of course, there is already no reason why the parent processes could not have used the Sessions Model to call MPI_COMM_SPAWN (passing in a `comm` that was derived from the Sessions Model), so this completes the picture of how the Dynamic Model can be bolted on successfully to either the World Model (refer to MPI-3.1 and prior) or the Sessions Model (refer to MPI-4.1, which will include this addition) or a mixture (refer to MPI-4.0, without this addition). [EDIT: if we decide that MPI_COMM_GET_PARENT can already be called without/before/after the World Model, then we should add it to Table 11.1 and write some text about it in §11.8.2. In that scenario, the above provides an implementation route for that procedure using only the Sessions Model underneath - in a similar way to the observation that it is possible to implement MPI_INIT[_THREAD] using only the Sessions Model underneath.] Cheers, Dan. — Dr Daniel Holmes PhD Architect (HPC Research) d.holmes(a)epcc.ed.ac.uk<mailto:[email protected]> Phone: +44 (0) 131 651 3465 Mobile: +44 (0) 7940 524 088 Address: Room 2.09, Bayes Centre, 47 Potterrow, Central Area, Edinburgh, EH8 9BT — The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. —

1 0

Re: [mpiwg-sessions] [EXTERNAL] Better handling of dynamic mixed with sessions without world
by Pritchard Jr., Howard 22 Jan '21

22 Jan '21

Hi Dan, I think this is a good idea. I’m not sure we entirely missed it though for sessions 1.0. I vaguely recall discussions about something like this in the context of mpi_exec – but it was quite a while ago. Maybe some problems/limitations were found with this process set approach to capturing info to support spawn functionality. We should probably double check the old slide deck. This would be a 4.1 proposal right? Howard From: HOLMES Daniel <d.holmes(a)epcc.ed.ac.uk> Date: Friday, January 22, 2021 at 5:06 AM To: "Pritchard Jr., Howard" <howardp(a)lanl.gov>, Martin Schulz <schulzm(a)in.tum.de> Cc: MPI Sessions working group <mpiwg-sessions(a)lists.mpi-forum.org> Subject: [EXTERNAL] Better handling of dynamic mixed with sessions without world Hi all, I just reviewed PR 506 from Rolf. It adds predefined communicator names to Appendix A. One of those is “MPI_COMM_PARENT”. It is a correct change and only indirectly relevant because it is what set off an interesting train of thought, which I summarise below. This suggests that we missed a trick when proposing Sessions v1.0 - we should have mandated the existence and meaning for an additional process set name: "mpi://MPI_COMM_PARENT” (note the absence of “GET” in that name to match the default string name assigned by MPI to the communicator returned by the MPI_COMM_GET_PARENT procedure). This process set shall contain all the processes that would be in the communicator returned by MPI_COMM_GET_PARENT, either zero processes (if you would get MPI_COMM_NULL) or the union of the local and remote groups (if you would get an inter-communicator). With this addition, MPI processes that are spawned can regain all of the functionality of the Dynamic Model without calling MPI_INIT[_THREAD], i.e. they can create the inter-communicator that they would have got from MPI_COMM_GET_PARENT (which requires a prior call to MPI_INIT because World Model [ED: please check]). ```pseudo-code Create a session Create an MPI_Group, groupParent, from the process set named "mpi://MPI_COMM_PARENT” Create an MPI_Group, groupWorld, from the process set named “mpi://MPI_COMM_WORLD” If (groupParent if MPI_GROUP_NULL) then This process was not spawned by other MPI processes Else Create groupLocal from the intersection of groupWorld and groupParent Create groupRemote from the subtraction of groupWorld from groupParent Create commParent using MPI_COMM_CREATE_FROM_GROUPS with groupLocal and groupRemote ``` No usage of the World Model, but we now have (a duplicate of) the inter-communicator that would have been returned by MPI_COMM_GET_PARENT if we had been permitted to call it. For each MPI process, there is a unique “parent” process set by this definition. Multiple components doing this pseudo-code would get duplicate communicators with their own life-cycle. This contrasts with the current way whereby freeing or disconnecting any communicator handle provided by MPI_COMM_GET_PARENT makes all other such handles stale, which is another instance of global state in MPI baked into the standardised interface and another anathema to any attempt at isolation. Of course, there is already no reason why the parent processes could not have used the Sessions Model to call MPI_COMM_SPAWN (passing in a `comm` that was derived from the Sessions Model), so this completes the picture of how the Dynamic Model can be bolted on successfully to either the World Model (refer to MPI-3.1 and prior) or the Sessions Model (refer to MPI-4.1, which will include this addition) or a mixture (refer to MPI-4.0, without this addition). [EDIT: if we decide that MPI_COMM_GET_PARENT can already be called without/before/after the World Model, then we should add it to Table 11.1 and write some text about it in §11.8.2. In that scenario, the above provides an implementation route for that procedure using only the Sessions Model underneath - in a similar way to the observation that it is possible to implement MPI_INIT[_THREAD] using only the Sessions Model underneath.] Cheers, Dan. — Dr Daniel Holmes PhD Architect (HPC Research) d.holmes(a)epcc.ed.ac.uk<mailto:[email protected]> Phone: +44 (0) 131 651 3465 Mobile: +44 (0) 7940 524 088 Address: Room 2.09, Bayes Centre, 47 Potterrow, Central Area, Edinburgh, EH8 9BT — The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. —

1 0

Better handling of dynamic mixed with sessions without world
by HOLMES Daniel 22 Jan '21

22 Jan '21

Hi all, I just reviewed PR 506 from Rolf. It adds predefined communicator names to Appendix A. One of those is “MPI_COMM_PARENT”. It is a correct change and only indirectly relevant because it is what set off an interesting train of thought, which I summarise below. This suggests that we missed a trick when proposing Sessions v1.0 - we should have mandated the existence and meaning for an additional process set name: "mpi://MPI_COMM_PARENT” (note the absence of “GET” in that name to match the default string name assigned by MPI to the communicator returned by the MPI_COMM_GET_PARENT procedure). This process set shall contain all the processes that would be in the communicator returned by MPI_COMM_GET_PARENT, either zero processes (if you would get MPI_COMM_NULL) or the union of the local and remote groups (if you would get an inter-communicator). With this addition, MPI processes that are spawned can regain all of the functionality of the Dynamic Model without calling MPI_INIT[_THREAD], i.e. they can create the inter-communicator that they would have got from MPI_COMM_GET_PARENT (which requires a prior call to MPI_INIT because World Model [ED: please check]). ```pseudo-code Create a session Create an MPI_Group, groupParent, from the process set named "mpi://MPI_COMM_PARENT” Create an MPI_Group, groupWorld, from the process set named “mpi://MPI_COMM_WORLD” If (groupParent if MPI_GROUP_NULL) then This process was not spawned by other MPI processes Else Create groupLocal from the intersection of groupWorld and groupParent Create groupRemote from the subtraction of groupWorld from groupParent Create commParent using MPI_COMM_CREATE_FROM_GROUPS with groupLocal and groupRemote ``` No usage of the World Model, but we now have (a duplicate of) the inter-communicator that would have been returned by MPI_COMM_GET_PARENT if we had been permitted to call it. For each MPI process, there is a unique “parent” process set by this definition. Multiple components doing this pseudo-code would get duplicate communicators with their own life-cycle. This contrasts with the current way whereby freeing or disconnecting any communicator handle provided by MPI_COMM_GET_PARENT makes all other such handles stale, which is another instance of global state in MPI baked into the standardised interface and another anathema to any attempt at isolation. Of course, there is already no reason why the parent processes could not have used the Sessions Model to call MPI_COMM_SPAWN (passing in a `comm` that was derived from the Sessions Model), so this completes the picture of how the Dynamic Model can be bolted on successfully to either the World Model (refer to MPI-3.1 and prior) or the Sessions Model (refer to MPI-4.1, which will include this addition) or a mixture (refer to MPI-4.0, without this addition). [EDIT: if we decide that MPI_COMM_GET_PARENT can already be called without/before/after the World Model, then we should add it to Table 11.1 and write some text about it in §11.8.2. In that scenario, the above provides an implementation route for that procedure using only the Sessions Model underneath - in a similar way to the observation that it is possible to implement MPI_INIT[_THREAD] using only the Sessions Model underneath.] Cheers, Dan. — Dr Daniel Holmes PhD Architect (HPC Research) d.holmes(a)epcc.ed.ac.uk<mailto:[email protected]> Phone: +44 (0) 131 651 3465 Mobile: +44 (0) 7940 524 088 Address: Room 2.09, Bayes Centre, 47 Potterrow, Central Area, Edinburgh, EH8 9BT — The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. —

1 0

Re: [mpiwg-sessions] Sessions WG meeting on 1/18/21
by Martin Schulz 14 Jan '21

14 Jan '21

Probably makes sense to cancel and then go back to bubbles once we all can make it, unless we still have an urgent topic for 4.0. Martin -- Prof. Dr. Martin Schulz, Chair of Computer Architecture and Parallel Systems Department of Informatics, TU-Munich, Boltzmannstraße 3, D-85748 Garching Member of the Board of Directors at the Leibniz Supercomputing Centre (LRZ) Email: schulzm(a)in.tum.de From: mpiwg-sessions <mpiwg-sessions-bounces(a)lists.mpi-forum.org> on behalf of HOLMES Daniel via mpiwg-sessions <mpiwg-sessions(a)lists.mpi-forum.org> Reply-To: MPI Sessions working group <mpiwg-sessions(a)lists.mpi-forum.org> Date: Thursday, 14. January 2021 at 17:11 To: mpiwg-sessions <mpiwg-sessions(a)lists.mpi-forum.org> Cc: HOLMES Daniel <d.holmes(a)epcc.ed.ac.uk> Subject: Re: [mpiwg-sessions] Sessions WG meeting on 1/18/21 Hi Howard, Thanks for letting us know. Martin (et al) should we proceed or cancel? Cheers, Dan. — Dr Daniel Holmes PhD Architect (HPC Research) d.holmes(a)epcc.ed.ac.uk Phone: +44 (0) 131 651 3465 Mobile: +44 (0) 7940 524 088 Address: Room 2.09, Bayes Centre, 47 Potterrow, Central Area, Edinburgh, EH8 9BT — The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. — On 14 Jan 2021, at 16:09, Pritchard Jr., Howard via mpiwg-sessions <mpiwg-sessions(a)lists.mpi-forum.org> wrote: This email was sent to you by someone outside the University. You should only click on links or attachments if you are certain that the email is genuine and the content is safe. Hi All, Just a heads up that 1/18/21 is a federal holiday here in the US so I’ll not be calling in to the meeting on Monday. Thanks, Howard -- Howard Pritchard HPC-ENV Los Alamos National Laboratory _______________________________________________ mpiwg-sessions mailing list mpiwg-sessions(a)lists.mpi-forum.org https://lists.mpi-forum.org/mailman/listinfo/mpiwg-sessions

1 0

Re: [mpiwg-sessions] Sessions WG meeting on 1/18/21
by HOLMES Daniel 14 Jan '21

14 Jan '21

Hi Howard, Thanks for letting us know. Martin (et al) should we proceed or cancel? Cheers, Dan. — Dr Daniel Holmes PhD Architect (HPC Research) d.holmes(a)epcc.ed.ac.uk<mailto:[email protected]> Phone: +44 (0) 131 651 3465 Mobile: +44 (0) 7940 524 088 Address: Room 2.09, Bayes Centre, 47 Potterrow, Central Area, Edinburgh, EH8 9BT — The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. — On 14 Jan 2021, at 16:09, Pritchard Jr., Howard via mpiwg-sessions <mpiwg-sessions(a)lists.mpi-forum.org<mailto:[email protected]>> wrote: This email was sent to you by someone outside the University. You should only click on links or attachments if you are certain that the email is genuine and the content is safe. Hi All, Just a heads up that 1/18/21 is a federal holiday here in the US so I’ll not be calling in to the meeting on Monday. Thanks, Howard -- Howard Pritchard HPC-ENV Los Alamos National Laboratory _______________________________________________ mpiwg-sessions mailing list mpiwg-sessions(a)lists.mpi-forum.org<mailto:[email protected]> https://lists.mpi-forum.org/mailman/listinfo/mpiwg-sessions

1 0

Sessions WG meeting on 1/18/21
by Pritchard Jr., Howard 14 Jan '21

14 Jan '21

Hi All, Just a heads up that 1/18/21 is a federal holiday here in the US so I’ll not be calling in to the meeting on Monday. Thanks, Howard -- Howard Pritchard HPC-ENV Los Alamos National Laboratory

1 0

meet today
by Pritchard Jr., Howard 11 Jan '21

11 Jan '21

HI Folks, Let’s meet today. Two items I have for discussion are * Issue https://github.com/mpi-forum/mpi-issues/issues/424 - I may have a WIP PR by the time we meet * Continue with our use case investigation Howard -- Howard Pritchard HPC-ENV Los Alamos National Laboratory

1 0

Sessions WG - no meeting today
by Pritchard Jr., Howard 04 Jan '21

04 Jan '21

HI All, A number of our frequent WG meeting attendees are still on holiday so canceling today’s meeting. We will meet again next week. Howard -- Howard Pritchard HPC-ENV Los Alamos National Laboratory

1 0