December 2020 - mpiwg-hybridpm - mailman3.cels.anl.gov

Re: [mpiwg-hybridpm] Questions about MPI CUDA stream integration
by Jim Dinan 24 Dec '20

24 Dec '20

Hi Jeff, We're actively engaged in the C++ standards work, and across several of the other models you mentioned. These are good targets, and I look forward to discussing how we can also support hybrid programming for these models. All the best, ~Jim. On Thu, Dec 17, 2020 at 5:10 PM Jeff Hammond via mpiwg-hybridpm < mpiwg-hybridpm(a)lists.mpi-forum.org> wrote: > I don't think it's reasonable to ask anybody to reason about the > interaction of MPI with a programming model that has no specification. > Furthermore, I don't think it's fair to ask anyone to dig into a language > that may be encumbered by patents and which has a restrictive, > hardware-locked license. > > I would be happy to discuss ISO C++20, C20 and Fortran 2008, Khronos > OpenCL, SYCL and Vulkan, OpenMP or OpenACC here, because those have > specifications and permit implementations on any hardware. > > Jeff > > On Thu, Dec 17, 2020 at 2:34 AM Joseph Schuchart via mpiwg-hybridpm < > mpiwg-hybridpm(a)lists.mpi-forum.org> wrote: > >> Jim, all, >> >> Thanks for your presentation yesterday (and last time). I had a bunch of >> questions but held back in hopes that we could go through the rest of >> the slides. Maybe it's better to have this discussion on the mailing >> list and save the precious hour in the WG meeting. In essence, my points >> boil down to these two: >> >> 1) I wonder what the benefit is of integrating stream support into MPI >> libraries over accelerator vendors providing their specific APIs on top >> of what MPI already offers? My understanding from the CUDA graph API is >> that you can add a host node that is a callback executed on the CPU. >> That is what I imagine the MPI library would use and it is what a >> third-party library could do as well, right? Otherwise, what is missing >> from the MPI API? >> >> 2) The CUDA stream and graph APIs seem very similar to task dependencies >> in OpenMP, with the same complications when combined with MPI. I think >> Martin hinted at this last night: MPI adds dependencies between nodes in >> one or more graphs that are not exposed to the CUDA scheduler, which >> opens the door for deadlocks. I think we should strive to do better. In >> OpenMP, detached tasks (and the events used to complete them) provide a >> user-controlled completion mechanism. This may be a model that could be >> picked up by the accelerator APIs. Joachim and I have shown that this >> model is easily coupled with callback-based completion notification in >> MPI :) So maybe the burden does not have to be all on MPI here... >> >> The discussion last night got held up by many questions, partly because >> two concepts were mixed: device-side communication initiation (which I >> think is very interesting) and stream integration (which I am less >> convinced of). It might be worth splitting the two aspects into separate >> discussions since you can have one without the other and it might make >> it easier for people to follow along. >> >> Since the problem of passing va-args through PMPI came up last night: >> one way to deal with it would be to provide MPI_Wait_enqueuev(reqs, >> type, va_args) to allow PMPI wrappers to inspect the va-args and pass >> them on to MPI. This is the model that printf/vprintf and friends are >> using. I'm not sure whether that works for Fortran though... >> >> I hope this is a good place to discuss these topics so everyone feel >> free to comment. Otherwise just take it as input for the next WG meeting >> :) >> >> Cheers >> Joseph >> -- >> Dr-Ing. Joseph Schuchart >> High Performance Computing Center Stuttgart (HLRS) >> Nobelstr. 19 >> D-70569 Stuttgart >> >> Tel.: +49(0)711-68565890 >> Fax: +49(0)711-6856832 >> E-Mail: schuchart(a)hlrs.de >> _______________________________________________ >> mpiwg-hybridpm mailing list >> mpiwg-hybridpm(a)lists.mpi-forum.org >> https://lists.mpi-forum.org/mailman/listinfo/mpiwg-hybridpm >> > > > -- > Jeff Hammond > jeff.science(a)gmail.com > http://jeffhammond.github.io/ > _______________________________________________ > mpiwg-hybridpm mailing list > mpiwg-hybridpm(a)lists.mpi-forum.org > https://lists.mpi-forum.org/mailman/listinfo/mpiwg-hybridpm >

1 0

Re: [mpiwg-hybridpm] Questions about MPI CUDA stream integration
by Jeff Hammond 17 Dec '20

17 Dec '20

I don't think it's reasonable to ask anybody to reason about the interaction of MPI with a programming model that has no specification. Furthermore, I don't think it's fair to ask anyone to dig into a language that may be encumbered by patents and which has a restrictive, hardware-locked license. I would be happy to discuss ISO C++20, C20 and Fortran 2008, Khronos OpenCL, SYCL and Vulkan, OpenMP or OpenACC here, because those have specifications and permit implementations on any hardware. Jeff On Thu, Dec 17, 2020 at 2:34 AM Joseph Schuchart via mpiwg-hybridpm < mpiwg-hybridpm(a)lists.mpi-forum.org> wrote: > Jim, all, > > Thanks for your presentation yesterday (and last time). I had a bunch of > questions but held back in hopes that we could go through the rest of > the slides. Maybe it's better to have this discussion on the mailing > list and save the precious hour in the WG meeting. In essence, my points > boil down to these two: > > 1) I wonder what the benefit is of integrating stream support into MPI > libraries over accelerator vendors providing their specific APIs on top > of what MPI already offers? My understanding from the CUDA graph API is > that you can add a host node that is a callback executed on the CPU. > That is what I imagine the MPI library would use and it is what a > third-party library could do as well, right? Otherwise, what is missing > from the MPI API? > > 2) The CUDA stream and graph APIs seem very similar to task dependencies > in OpenMP, with the same complications when combined with MPI. I think > Martin hinted at this last night: MPI adds dependencies between nodes in > one or more graphs that are not exposed to the CUDA scheduler, which > opens the door for deadlocks. I think we should strive to do better. In > OpenMP, detached tasks (and the events used to complete them) provide a > user-controlled completion mechanism. This may be a model that could be > picked up by the accelerator APIs. Joachim and I have shown that this > model is easily coupled with callback-based completion notification in > MPI :) So maybe the burden does not have to be all on MPI here... > > The discussion last night got held up by many questions, partly because > two concepts were mixed: device-side communication initiation (which I > think is very interesting) and stream integration (which I am less > convinced of). It might be worth splitting the two aspects into separate > discussions since you can have one without the other and it might make > it easier for people to follow along. > > Since the problem of passing va-args through PMPI came up last night: > one way to deal with it would be to provide MPI_Wait_enqueuev(reqs, > type, va_args) to allow PMPI wrappers to inspect the va-args and pass > them on to MPI. This is the model that printf/vprintf and friends are > using. I'm not sure whether that works for Fortran though... > > I hope this is a good place to discuss these topics so everyone feel > free to comment. Otherwise just take it as input for the next WG meeting :) > > Cheers > Joseph > -- > Dr-Ing. Joseph Schuchart > High Performance Computing Center Stuttgart (HLRS) > Nobelstr. 19 > D-70569 Stuttgart > > Tel.: +49(0)711-68565890 > Fax: +49(0)711-6856832 > E-Mail: schuchart(a)hlrs.de > _______________________________________________ > mpiwg-hybridpm mailing list > mpiwg-hybridpm(a)lists.mpi-forum.org > https://lists.mpi-forum.org/mailman/listinfo/mpiwg-hybridpm > -- Jeff Hammond jeff.science(a)gmail.com http://jeffhammond.github.io/

1 0

Re: [mpiwg-hybridpm] Questions about MPI CUDA stream integration
by Jim Dinan 17 Dec '20

17 Dec '20

Please see below: On Thu, Dec 17, 2020 at 5:34 AM Joseph Schuchart via mpiwg-hybridpm < mpiwg-hybridpm(a)lists.mpi-forum.org> wrote: > 1) I wonder what the benefit is of integrating stream support into MPI > libraries over accelerator vendors providing their specific APIs on top > of what MPI already offers? My understanding from the CUDA graph API is > that you can add a host node that is a callback executed on the CPU. > That is what I imagine the MPI library would use and it is what a > third-party library could do as well, right? Otherwise, what is missing > from the MPI API? > Host callbacks are not able to make CUDA calls. Today, CUDA-Aware MPI functions perform CUDA pointer queries to find out where a buffer is, launch data pack/unpack kernels, perform CUDA memory copies, etc. I've thought quite a bit about the question of which direction makes more sense to enable interoperability. One explanation for why I think interoperability from MPI => CUDA makes more sense is that MPI is *already* using CUDA/HIP/OneAPI internally to interact with the data on the accelerator (which is why you can't make MPI calls in host callbacks). In the CUDA/HIP models, a number of things MPI does to move data like launching kernels, performing memcpys, etc are performed on streams. Today, MPI libraries use their own internal streams, which are disconnected from the streams the user is using to launch kernels. This forces stream synchronization and causes us to lose the pipelining and overhead hiding benefits from streams. > 2) The CUDA stream and graph APIs seem very similar to task dependencies > in OpenMP, with the same complications when combined with MPI. I think > Martin hinted at this last night: MPI adds dependencies between nodes in > one or more graphs that are not exposed to the CUDA scheduler, which > opens the door for deadlocks. I think we should strive to do better. In > OpenMP, detached tasks (and the events used to complete them) provide a > user-controlled completion mechanism. This may be a model that could be > picked up by the accelerator APIs. Joachim and I have shown that this > model is easily coupled with callback-based completion notification in > MPI :) So maybe the burden does not have to be all on MPI here... > The possibility of deadlocking streams is really an artifact of the queueing model that MPI is being used with, and I think of it as being orthogonal to this proposal. As I mentioned, graphs are a way to avoid deadlock entirely. All communication in NCCL is stream-based. One way that NCCL avoids deadlocking is through communication batching, which this proposal provides through the start-all and wait-all on stream operations. > The discussion last night got held up by many questions, partly because > two concepts were mixed: device-side communication initiation (which I > think is very interesting) and stream integration (which I am less > convinced of). It might be worth splitting the two aspects into separate > discussions since you can have one without the other and it might make > it easier for people to follow along. > This first "kickoff" slide deck was really intended to motivate and get our creativity going (seems to have met these goals). Mixing these two concepts is showing an end-state that I think is very appealing to GPU users. But, these really aren't the right slides for us to have a deeper technical discussion (please bear with me, I'll make better slides). I agree we need to separate these topics going forward. The partitioned discussion should stay in the persistence WG where it has been homed. Since the problem of passing va-args through PMPI came up last night: > one way to deal with it would be to provide MPI_Wait_enqueuev(reqs, > type, va_args) to allow PMPI wrappers to inspect the va-args and pass > them on to MPI. This is the model that printf/vprintf and friends are > using. I'm not sure whether that works for Fortran though... > > I hope this is a good place to discuss these topics so everyone feel > free to comment. Otherwise just take it as input for the next WG meeting :) > I've got a mountain of feedback to work with and I'll work on preparing better materials for us to explore this topic. I will also grab one of our stream/graph experts to come and talk to the WG. They can do a much better job of describing the benefits of the programming model than I have done. I also have a topic lined up to discuss how NCCL, NVSHMEM, and possibly MPLib use streams to help set the context. Appreciate the thoughtful discussion, and look forward to much more. :) ~Jim.

1 0

Questions about MPI CUDA stream integration
by Joseph Schuchart 17 Dec '20

17 Dec '20

Jim, all, Thanks for your presentation yesterday (and last time). I had a bunch of questions but held back in hopes that we could go through the rest of the slides. Maybe it's better to have this discussion on the mailing list and save the precious hour in the WG meeting. In essence, my points boil down to these two: 1) I wonder what the benefit is of integrating stream support into MPI libraries over accelerator vendors providing their specific APIs on top of what MPI already offers? My understanding from the CUDA graph API is that you can add a host node that is a callback executed on the CPU. That is what I imagine the MPI library would use and it is what a third-party library could do as well, right? Otherwise, what is missing from the MPI API? 2) The CUDA stream and graph APIs seem very similar to task dependencies in OpenMP, with the same complications when combined with MPI. I think Martin hinted at this last night: MPI adds dependencies between nodes in one or more graphs that are not exposed to the CUDA scheduler, which opens the door for deadlocks. I think we should strive to do better. In OpenMP, detached tasks (and the events used to complete them) provide a user-controlled completion mechanism. This may be a model that could be picked up by the accelerator APIs. Joachim and I have shown that this model is easily coupled with callback-based completion notification in MPI :) So maybe the burden does not have to be all on MPI here... The discussion last night got held up by many questions, partly because two concepts were mixed: device-side communication initiation (which I think is very interesting) and stream integration (which I am less convinced of). It might be worth splitting the two aspects into separate discussions since you can have one without the other and it might make it easier for people to follow along. Since the problem of passing va-args through PMPI came up last night: one way to deal with it would be to provide MPI_Wait_enqueuev(reqs, type, va_args) to allow PMPI wrappers to inspect the va-args and pass them on to MPI. This is the model that printf/vprintf and friends are using. I'm not sure whether that works for Fortran though... I hope this is a good place to discuss these topics so everyone feel free to comment. Otherwise just take it as input for the next WG meeting :) Cheers Joseph -- Dr-Ing. Joseph Schuchart High Performance Computing Center Stuttgart (HLRS) Nobelstr. 19 D-70569 Stuttgart Tel.: +49(0)711-68565890 Fax: +49(0)711-6856832 E-Mail: schuchart(a)hlrs.de

1 0

Re: [mpiwg-hybridpm] HACC WG Meeting Tomorrow
by Jim Dinan 15 Dec '20

15 Dec '20

Whoops, I goofed on the date. We're meeting tomorrow, Wednesday December *16th*. Best, ~Jim. On Tue, Dec 15, 2020 at 11:41 AM Jim Dinan <james.dinan(a)gmail.com> wrote: > Hi All, > > The Hybrid & Accelerator WG will meet Wednesday, Dec. 15 from 10-11am ET. > We will use the Zoom meeting that is shared with the Persistence WG. So if > you have the Persistence WG meeting on your calendar already, it should > lead you to the correct meeting this time. > > Zoom: > https://tennessee.zoom.us/j/91925030900?pwd=UjlXek5pUnpmL3RuWG5xemNuSHdsUT09 > Password: 136803 > > The agenda for tomorrow's meeting will be to continue discussing the > slides I presented last time: > https://github.com/mpiwg-hybrid/hybrid-issues/tree/master/slides > > See you then, > ~Jim. > > PS - Apologies for cross posting to the main list. Future announcements > will be directed only to the hybridpm mailing list. Signup instructions are > on our wiki: https://github.com/mpiwg-hybrid/hybrid-issues/wiki. >

1 0

HACC WG Meeting Tomorrow
by Jim Dinan 15 Dec '20

15 Dec '20

Hi All, The Hybrid & Accelerator WG will meet Wednesday, Dec. 15 from 10-11am ET. We will use the Zoom meeting that is shared with the Persistence WG. So if you have the Persistence WG meeting on your calendar already, it should lead you to the correct meeting this time. Zoom: https://tennessee.zoom.us/j/91925030900?pwd=UjlXek5pUnpmL3RuWG5xemNuSHdsUT09 Password: 136803 The agenda for tomorrow's meeting will be to continue discussing the slides I presented last time: https://github.com/mpiwg-hybrid/hybrid-issues/tree/master/slides See you then, ~Jim. PS - Apologies for cross posting to the main list. Future announcements will be directed only to the hybridpm mailing list. Signup instructions are on our wiki: https://github.com/mpiwg-hybrid/hybrid-issues/wiki.

1 0

Re: [mpiwg-hybridpm] Hybrid & Accelerator WG
by Jim Dinan 02 Dec '20

02 Dec '20

Hi All, Thanks for the great discussion today! I've posted the slides from today's meeting on the Hybrid GitHub page: https://github.com/mpiwg-hybrid/hybrid-issues/tree/master/slides Best, ~Jim. On Tue, Dec 1, 2020 at 10:33 PM Jim Dinan <james.dinan(a)gmail.com> wrote: > Hi All, > > The Hybrid & Accelerator WG will meet tomorrow (Wednesday) 10am - 11am ET. > Meeting connection info is included below. I'll be giving a kickoff > presentation on ideas for how MPI can improve hybrid programming with > accelerators. > > Apologies for cross-posting to the main list. Future messages will be > directed just to the Hybrid mailing list: > https://lists.mpi-forum.org/mailman/listinfo/mpiwg-hybridpm > > Best, > ~Jim. > > ---8<--- > > > ________________________________________________________________________________ > > Microsoft Teams meeting > > *Join on your computer or mobile app* > > Click here to join the meeting > <https://teams.microsoft.com/l/meetup-join/19%3ameeting_NGM3M2EwMTgtMDlkOC00…> > > *Join with a video conferencing device* > > teams(a)vc.nvidia.com > > Video Conference ID: 118 581 149 6 > > Alternate VTC dialing instructions > <https://pexip.me/teams/vc.nvidia.com/1185811496> > > *Or call in (audio only)* > > +1 949-570-1120,,213300332# <+19495701120,,213300332#> United States, > Irvine > > Phone Conference ID: 213 300 332# > > Find a local number > <https://dialin.teams.microsoft.com/c58d41ec-314f-4761-9381-98dfc4066d1b?id=…> > | Reset PIN <https://mysettings.lync.com/pstnconferencing> > > Learn More <https://aka.ms/JoinTeamsMeeting> | Meeting options > <https://teams.microsoft.com/meetingOptions/?organizerId=fb44160b-9957-4f30-…> > > > ________________________________________________________________________________ >

1 0

Hybrid & Accelerator WG
by Jim Dinan 01 Dec '20

01 Dec '20

Hi All, The Hybrid & Accelerator WG will meet tomorrow (Wednesday) 10am - 11am ET. Meeting connection info is included below. I'll be giving a kickoff presentation on ideas for how MPI can improve hybrid programming with accelerators. Apologies for cross-posting to the main list. Future messages will be directed just to the Hybrid mailing list: https://lists.mpi-forum.org/mailman/listinfo/mpiwg-hybridpm Best, ~Jim. ---8<--- ________________________________________________________________________________ Microsoft Teams meeting *Join on your computer or mobile app* Click here to join the meeting <https://teams.microsoft.com/l/meetup-join/19%3ameeting_NGM3M2EwMTgtMDlkOC00…> *Join with a video conferencing device* teams(a)vc.nvidia.com Video Conference ID: 118 581 149 6 Alternate VTC dialing instructions <https://pexip.me/teams/vc.nvidia.com/1185811496> *Or call in (audio only)* +1 949-570-1120,,213300332# <+19495701120,,213300332#> United States, Irvine Phone Conference ID: 213 300 332# Find a local number <https://dialin.teams.microsoft.com/c58d41ec-314f-4761-9381-98dfc4066d1b?id=…> | Reset PIN <https://mysettings.lync.com/pstnconferencing> Learn More <https://aka.ms/JoinTeamsMeeting> | Meeting options <https://teams.microsoft.com/meetingOptions/?organizerId=fb44160b-9957-4f30-…> ________________________________________________________________________________

1 0