October 2020 - Ecp-aurora-sdk-discuss

Re: [Ecp-aurora-sdk-discuss] Coming updates to modulefile location for Aurora SDK components
by Rosales-fernandez, Carlos 28 Oct '20

28 Oct '20

Correction: note I missed the “modulefiles” directory in this part of my previous note: Simply update your MODULEPATH to contain /soft/restricted/CNDA/modulefiles [...] From: Ecp-aurora-sdk-discuss <ecp-aurora-sdk-discuss-bounces(a)lists.jlse.anl.gov> on behalf of ECP Aurora <ecp-aurora-sdk-discuss(a)lists.jlse.anl.gov> Reply-To: "Rosales-fernandez, Carlos" <carlos.rosales-fernandez(a)intel.com> Date: Wednesday, October 28, 2020 at 1:24 PM To: ECP Aurora <ecp-aurora-sdk-discuss(a)lists.jlse.anl.gov> Subject: [Ecp-aurora-sdk-discuss] Coming updates to modulefile location for Aurora SDK components Hi, In an effort to simplify maintenance we will deprecate all duplicated modulefiles inside the intel_dga/modulefiles directory. This will leave only those components with strict RSNDA requirements in that location after November 5th (such as special PTI and Intel Advisor drops). If you are loading modulefiles from /soft/restricted/intel_dga/modulefiles you will start receiving a deprecation warning today. After 11/05/2020 (Thursday next week) those modulefiles will be removed. Simply update your MODULEPATH to contain /soft/restricted/CNDA before 11/05 and you will see no disruption to your work. If you use advisor-rsnda simply keep both locations in your module path. Regards, Carlos

1 0

Coming updates to modulefile location for Aurora SDK components
by Rosales-fernandez, Carlos 28 Oct '20

28 Oct '20

Hi, In an effort to simplify maintenance we will deprecate all duplicated modulefiles inside the intel_dga/modulefiles directory. This will leave only those components with strict RSNDA requirements in that location after November 5th (such as special PTI and Intel Advisor drops). If you are loading modulefiles from /soft/restricted/intel_dga/modulefiles you will start receiving a deprecation warning today. After 11/05/2020 (Thursday next week) those modulefiles will be removed. Simply update your MODULEPATH to contain /soft/restricted/CNDA before 11/05 and you will see no disruption to your work. If you use advisor-rsnda simply keep both locations in your module path. Regards, Carlos

1 0

Re: [Ecp-aurora-sdk-discuss] Translating Streams to Queues
by Hammond, Jeff R 23 Oct '20

23 Oct '20

If it helps to have an example of the same thing with CUDA and USM explicit alloc and copy... https://github.com/ParRes/Kernels/blob/default/Cxx11/nstream-sycl-explicit-… syclx namespace is just sycl w/ DPC++. It’s there for ComputeCpp (which doesn’t implement USM yet, unfortunately). https://github.com/ParRes/Kernels/blob/default/Cxx11/nstream-cuda.cu Jeff Sent from my iPhone On Oct 20, 2020, at 8:37 AM, Luo, Ye via Ecp-aurora-sdk-discuss <ecp-aurora-sdk-discuss(a)lists.jlse.anl.gov> wrote: You can find all the API spec at https://www.khronos.org/registry/SYCL/specs/sycl-2020-provisional.pdf under Explicit USM chapter sycl::malloc_host is the closest to cudaMallocHost There is also sycl::malloc_device and then memcpy for the second part. Ye ________________________________ From: David Poole <davpoole(a)iastate.edu> Sent: Tuesday, October 20, 2020 10:08 AM To: Luo, Ye <yeluo(a)anl.gov> Cc: Harms, Kevin <harms(a)alcf.anl.gov>; ecp-aurora-sdk-discuss(a)lists.jlse.anl.gov <ecp-aurora-sdk-discuss(a)lists.jlse.anl.gov> Subject: Re: [Ecp-aurora-sdk-discuss] Translating Streams to Queues Hi all, Thank you all very much for the responses! I will figure out how to use malloc_host, then. "Please be more specif on "moves". cudaHostRegister?" The code specifically uses cudaMallocHost/cudaFreeHost to allocate/deallocate host pinned memory, then cudaMemcpy to actually copy memory from host to host pinned memory. "zero copy, page migration or copied to a device memory segment?" This, I will admit I am not sure about. The function used for host pinned-to-device data transfer is cudaMemcpyAsync, if that means anything for this particular question. Sincerely, -David Poole ----------------------------------------------- "It is the mark of an educated mind to be able to entertain a thought without accepting it." - Aristotle David Poole Iowa State University 201 Spedding Hall davpoole(a)iastate.edu<mailto:[email protected]> "I would rather have questions that can't be answered than answers that can't be questioned." - Richard Feynman "For me, I am driven by two main philosophies: know more today about the world than I knew yesterday and lessen the suffering of others. You'd be surprised how far that gets you. " - Neil deGrasse Tyson ----------------------------------------------- On Fri, Oct 16, 2020 at 1:01 PM Luo, Ye <yeluo(a)anl.gov<mailto:[email protected]>> wrote: it moves preallocated memory from host to pinned memory. Please be more specif on "moves". cudaHostRegister? then from pinned memory to device memory zero copy, page migration or copied to a device memory segment? Ye ________________________________ From: Ecp-aurora-sdk-discuss <ecp-aurora-sdk-discuss-bounces(a)lists.jlse.anl.gov<mailto:[email protected]>> on behalf of David Poole via Ecp-aurora-sdk-discuss <ecp-aurora-sdk-discuss(a)lists.jlse.anl.gov<mailto:[email protected]>> Sent: Thursday, October 15, 2020 4:19 PM To: Harms, Kevin <harms(a)alcf.anl.gov<mailto:[email protected]>> Cc: ecp-aurora-sdk-discuss(a)lists.jlse.anl.gov<mailto:[email protected]> <ecp-aurora-sdk-discuss(a)lists.jlse.anl.gov<mailto:[email protected]>> Subject: Re: [Ecp-aurora-sdk-discuss] Translating Streams to Queues Hi all, I have another question, if that is all right. One of the big features of the code that I am working on, is that it moves preallocated memory from host to pinned memory, then from pinned memory to device memory. I was wondering if DPC++ currently has support for pinned memory access/allocations. Thanks again! Sincerely, -David Poole ----------------------------------------------- "It is the mark of an educated mind to be able to entertain a thought without accepting it." - Aristotle David Poole Iowa State University 201 Spedding Hall davpoole(a)iastate.edu<mailto:[email protected]> "I would rather have questions that can't be answered than answers that can't be questioned." - Richard Feynman "For me, I am driven by two main philosophies: know more today about the world than I knew yesterday and lessen the suffering of others. You'd be surprised how far that gets you. " - Neil deGrasse Tyson ----------------------------------------------- On Thu, Oct 15, 2020 at 11:23 AM David Poole <davpoole(a)iastate.edu<mailto:[email protected]>> wrote: Hi Kevin, Ah, that's all good to know! Thank you very much! I will try it out and see what happens. Best, -David Poole ----------------------------------------------- "It is the mark of an educated mind to be able to entertain a thought without accepting it." - Aristotle David Poole Iowa State University 201 Spedding Hall davpoole(a)iastate.edu<mailto:[email protected]> "I would rather have questions that can't be answered than answers that can't be questioned." - Richard Feynman "For me, I am driven by two main philosophies: know more today about the world than I knew yesterday and lessen the suffering of others. You'd be surprised how far that gets you. " - Neil deGrasse Tyson ----------------------------------------------- On Tue, Oct 13, 2020 at 11:16 AM Harms, Kevin <harms(a)alcf.anl.gov<mailto:[email protected]>> wrote: David, cudaStreamSynchronize blocks until all stream operations are completed. A queue wait() function should be equivalent. The one caveat is queue.wait() does not wait for all data transfers to be complete. It is not clear from the documentation on cudaStreamSynchronize if it does or doesn't wait for memory transfer completion. As for cudaStreamCreateWithFlags, if you're using cudaStreamNonBlocking, i think the default behavior of SYCL will be the same as operations in different queues may run concurrently, however, if you introduce some dependency between kernels in different queues, then execution will no longer allow concurrency. kevin ________________________________________ From: Ecp-aurora-sdk-discuss <ecp-aurora-sdk-discuss-bounces(a)lists.jlse.anl.gov<mailto:[email protected]>> on behalf of David Poole via Ecp-aurora-sdk-discuss <ecp-aurora-sdk-discuss(a)lists.jlse.anl.gov<mailto:[email protected]>> Sent: Monday, October 12, 2020 5:44 PM To: ecp-aurora-sdk-discuss(a)lists.jlse.anl.gov<mailto:[email protected]> Subject: [Ecp-aurora-sdk-discuss] Translating Streams to Queues Hi all, I wanted to ask about translating CUDA streams to DPC++ queues in the porting of CUDA to DPC++ code. I am currently working on porting a couple of files which are heavy in stream manipulation, using functions such as cudaStreamSynchronize, cudaStreamCreateWithFlags, and cudaStreamDestroy. In trying the DPC++ translator on the code I am working on, these functions did not translate over. If I recall, CUDA streams (kind of) map to DPC++ queues, and I'm guessing cudaStreamDestroy could be handled by object lifetime rules/RAII; but I'm not quite sure about the other two functions could be mapped to DPC++ queues. How would one go about that, then? Thank you very much in advance! Sincerely, -David Poole ----------------------------------------------- "It is the mark of an educated mind to be able to entertain a thought without accepting it." - Aristotle David Poole Iowa State University 201 Spedding Hall davpoole(a)iastate.edu<mailto:[email protected]><mailto:[email protected]<mailto:[email protected]>> "I would rather have questions that can't be answered than answers that can't be questioned." - Richard Feynman "For me, I am driven by two main philosophies: know more today about the world than I knew yesterday and lessen the suffering of others. You'd be surprised how far that gets you. " - Neil deGrasse Tyson ----------------------------------------------- -- Ecp-aurora-sdk-discuss mailing list Ecp-aurora-sdk-discuss(a)lists.jlse.anl.gov https://lists.jlse.anl.gov/mailman/listinfo/ecp-aurora-sdk-discuss

1 0

New SDK available (and default) in JLSE
by Rosales-fernandez, Carlos 20 Oct '20

20 Oct '20

Hi all, We are pleased to announce that a new Aurora SDK has been deployed in JLSE. This version 2020.10.30.001 includes multiple improvements and bug fixes. For additional details please refer to the latest release notes in the Wiki: https://wiki.jlse.anl.gov/display/inteldga/DGA+SDK+Release+Notes THe "oneapi" module will enable the newer SDK by default, but the older versions remain available for regression testing. Important notes =============== Some changes have been introduced in MPI modulefiles, but defaults remain the same. In particular you will now notice the presence of two MPI options: * mpi/intel_mpi (traditional MPI stack from Intel's Parallel Studio XE) * mpi/aurora_mpich (main Aurora MPI stack) MPICH will the main MPI library on Aurora system, early adoption from users will help us identify issues and gather early feedback. We also provide more features in our special builds. When using MPICH, please use correct environment settings (modules listed below). To use the icx and icpx compilers with MPICH simply use the following options: “MPICH_CC=icx mpicc” or “mpicc -cc=icx” “MPICH_CXX=icpx mpicxx” or “mpicxx -cxx=icpx” Example 1: using “MPICH_CXX=icpx” $ MPICH_CXX=icpx mpicxx -o offload_test -fiopenmp -fopenmp-targets=spir64 -lOpenCL offload-test.cpp Example 2: using “mpicc -cc=icx” $ mpicc -cc=icx -o offload_test -fiopenmp -fopenmp-targets=spir64 -lOpenCL offload-test.c Environment info: $ module use /soft/restricted/CNDA/modulefiles $ module load oneapi mpi/aurora_mpich Note that by itself the oneapi module does not set any MPI environment Public Documentation ==================== * ALCF oneAPI overview - https://www.alcf.anl.gov/support-center/aurora/oneapi-overview * Intel GPGPU Software Information - https://dgpu-docs.intel.com/ * Intel oneAPI Beta Documentation - https://software.intel.com/content/www/us/en/develop/tools/oneapi/documenta… Please let us know of any issues with the release or the environment settings. Regards, Carlos

1 0

Re: [Ecp-aurora-sdk-discuss] Translating Streams to Queues
by Luo, Ye 20 Oct '20

20 Oct '20

You can find all the API spec at https://www.khronos.org/registry/SYCL/specs/sycl-2020-provisional.pdf under Explicit USM chapter sycl::malloc_host is the closest to cudaMallocHost There is also sycl::malloc_device and then memcpy for the second part. Ye ________________________________ From: David Poole <davpoole(a)iastate.edu> Sent: Tuesday, October 20, 2020 10:08 AM To: Luo, Ye <yeluo(a)anl.gov> Cc: Harms, Kevin <harms(a)alcf.anl.gov>; ecp-aurora-sdk-discuss(a)lists.jlse.anl.gov <ecp-aurora-sdk-discuss(a)lists.jlse.anl.gov> Subject: Re: [Ecp-aurora-sdk-discuss] Translating Streams to Queues Hi all, Thank you all very much for the responses! I will figure out how to use malloc_host, then. "Please be more specif on "moves". cudaHostRegister?" The code specifically uses cudaMallocHost/cudaFreeHost to allocate/deallocate host pinned memory, then cudaMemcpy to actually copy memory from host to host pinned memory. "zero copy, page migration or copied to a device memory segment?" This, I will admit I am not sure about. The function used for host pinned-to-device data transfer is cudaMemcpyAsync, if that means anything for this particular question. Sincerely, -David Poole ----------------------------------------------- "It is the mark of an educated mind to be able to entertain a thought without accepting it." - Aristotle David Poole Iowa State University 201 Spedding Hall davpoole(a)iastate.edu<mailto:[email protected]> "I would rather have questions that can't be answered than answers that can't be questioned." - Richard Feynman "For me, I am driven by two main philosophies: know more today about the world than I knew yesterday and lessen the suffering of others. You'd be surprised how far that gets you. " - Neil deGrasse Tyson ----------------------------------------------- On Fri, Oct 16, 2020 at 1:01 PM Luo, Ye <yeluo(a)anl.gov<mailto:[email protected]>> wrote: it moves preallocated memory from host to pinned memory. Please be more specif on "moves". cudaHostRegister? then from pinned memory to device memory zero copy, page migration or copied to a device memory segment? Ye ________________________________ From: Ecp-aurora-sdk-discuss <ecp-aurora-sdk-discuss-bounces(a)lists.jlse.anl.gov<mailto:[email protected]>> on behalf of David Poole via Ecp-aurora-sdk-discuss <ecp-aurora-sdk-discuss(a)lists.jlse.anl.gov<mailto:[email protected]>> Sent: Thursday, October 15, 2020 4:19 PM To: Harms, Kevin <harms(a)alcf.anl.gov<mailto:[email protected]>> Cc: ecp-aurora-sdk-discuss(a)lists.jlse.anl.gov<mailto:[email protected]> <ecp-aurora-sdk-discuss(a)lists.jlse.anl.gov<mailto:[email protected]>> Subject: Re: [Ecp-aurora-sdk-discuss] Translating Streams to Queues Hi all, I have another question, if that is all right. One of the big features of the code that I am working on, is that it moves preallocated memory from host to pinned memory, then from pinned memory to device memory. I was wondering if DPC++ currently has support for pinned memory access/allocations. Thanks again! Sincerely, -David Poole ----------------------------------------------- "It is the mark of an educated mind to be able to entertain a thought without accepting it." - Aristotle David Poole Iowa State University 201 Spedding Hall davpoole(a)iastate.edu<mailto:[email protected]> "I would rather have questions that can't be answered than answers that can't be questioned." - Richard Feynman "For me, I am driven by two main philosophies: know more today about the world than I knew yesterday and lessen the suffering of others. You'd be surprised how far that gets you. " - Neil deGrasse Tyson ----------------------------------------------- On Thu, Oct 15, 2020 at 11:23 AM David Poole <davpoole(a)iastate.edu<mailto:[email protected]>> wrote: Hi Kevin, Ah, that's all good to know! Thank you very much! I will try it out and see what happens. Best, -David Poole ----------------------------------------------- "It is the mark of an educated mind to be able to entertain a thought without accepting it." - Aristotle David Poole Iowa State University 201 Spedding Hall davpoole(a)iastate.edu<mailto:[email protected]> "I would rather have questions that can't be answered than answers that can't be questioned." - Richard Feynman "For me, I am driven by two main philosophies: know more today about the world than I knew yesterday and lessen the suffering of others. You'd be surprised how far that gets you. " - Neil deGrasse Tyson ----------------------------------------------- On Tue, Oct 13, 2020 at 11:16 AM Harms, Kevin <harms(a)alcf.anl.gov<mailto:[email protected]>> wrote: David, cudaStreamSynchronize blocks until all stream operations are completed. A queue wait() function should be equivalent. The one caveat is queue.wait() does not wait for all data transfers to be complete. It is not clear from the documentation on cudaStreamSynchronize if it does or doesn't wait for memory transfer completion. As for cudaStreamCreateWithFlags, if you're using cudaStreamNonBlocking, i think the default behavior of SYCL will be the same as operations in different queues may run concurrently, however, if you introduce some dependency between kernels in different queues, then execution will no longer allow concurrency. kevin ________________________________________ From: Ecp-aurora-sdk-discuss <ecp-aurora-sdk-discuss-bounces(a)lists.jlse.anl.gov<mailto:[email protected]>> on behalf of David Poole via Ecp-aurora-sdk-discuss <ecp-aurora-sdk-discuss(a)lists.jlse.anl.gov<mailto:[email protected]>> Sent: Monday, October 12, 2020 5:44 PM To: ecp-aurora-sdk-discuss(a)lists.jlse.anl.gov<mailto:[email protected]> Subject: [Ecp-aurora-sdk-discuss] Translating Streams to Queues Hi all, I wanted to ask about translating CUDA streams to DPC++ queues in the porting of CUDA to DPC++ code. I am currently working on porting a couple of files which are heavy in stream manipulation, using functions such as cudaStreamSynchronize, cudaStreamCreateWithFlags, and cudaStreamDestroy. In trying the DPC++ translator on the code I am working on, these functions did not translate over. If I recall, CUDA streams (kind of) map to DPC++ queues, and I'm guessing cudaStreamDestroy could be handled by object lifetime rules/RAII; but I'm not quite sure about the other two functions could be mapped to DPC++ queues. How would one go about that, then? Thank you very much in advance! Sincerely, -David Poole ----------------------------------------------- "It is the mark of an educated mind to be able to entertain a thought without accepting it." - Aristotle David Poole Iowa State University 201 Spedding Hall davpoole(a)iastate.edu<mailto:[email protected]><mailto:[email protected]<mailto:[email protected]>> "I would rather have questions that can't be answered than answers that can't be questioned." - Richard Feynman "For me, I am driven by two main philosophies: know more today about the world than I knew yesterday and lessen the suffering of others. You'd be surprised how far that gets you. " - Neil deGrasse Tyson -----------------------------------------------

1 0

Re: [Ecp-aurora-sdk-discuss] Translating Streams to Queues
by David Poole 20 Oct '20

20 Oct '20

Hi all, Thank you all very much for the responses! I will figure out how to use malloc_host, then. "Please be more specif on "moves". cudaHostRegister?" The code specifically uses cudaMallocHost/cudaFreeHost to allocate/deallocate host pinned memory, then cudaMemcpy to actually copy memory from host to host pinned memory. "zero copy, page migration or copied to a device memory segment?" This, I will admit I am not sure about. The function used for host pinned-to-device data transfer is cudaMemcpyAsync, if that means anything for this particular question. Sincerely, -David Poole ----------------------------------------------- *"It is the mark of an educated mind tobe able to entertain a thought withoutaccepting it." - Aristotle* David Poole Iowa State University 201 Spedding Hall davpoole(a)iastate.edu *"I would rather have questions that can'tbe answered than answers that can't bequestioned." - Richard Feynman* *"For me, I am driven by two main * *philosophies: know more today about* *the world than I knew yesterday and* *lessen the suffering of others. You'd* *be surprised how far that gets * *you. " - Neil deGrasse Tyson* ----------------------------------------------- On Fri, Oct 16, 2020 at 1:01 PM Luo, Ye <yeluo(a)anl.gov> wrote: > it moves preallocated memory from host to pinned memory. > > Please be more specif on "moves". cudaHostRegister? > > then from pinned memory to device memory > > zero copy, page migration or copied to a device memory segment? > > Ye > ------------------------------ > *From:* Ecp-aurora-sdk-discuss < > ecp-aurora-sdk-discuss-bounces(a)lists.jlse.anl.gov> on behalf of David > Poole via Ecp-aurora-sdk-discuss < > ecp-aurora-sdk-discuss(a)lists.jlse.anl.gov> > *Sent:* Thursday, October 15, 2020 4:19 PM > *To:* Harms, Kevin <harms(a)alcf.anl.gov> > *Cc:* ecp-aurora-sdk-discuss(a)lists.jlse.anl.gov < > ecp-aurora-sdk-discuss(a)lists.jlse.anl.gov> > *Subject:* Re: [Ecp-aurora-sdk-discuss] Translating Streams to Queues > > Hi all, > > I have another question, if that is all right. > > One of the big features of the code that I am working on, is that it moves > preallocated memory from host to pinned memory, then from pinned memory to > device memory. I was wondering if DPC++ currently has support for pinned > memory access/allocations. > > Thanks again! > > Sincerely, > -David Poole > > ----------------------------------------------- > > > *"It is the mark of an educated mind to be able to entertain a thought > without accepting it." - Aristotle* > > David Poole > Iowa State University > 201 Spedding Hall > davpoole(a)iastate.edu > > > > *"I would rather have questions that can't be answered than answers that > can't be questioned." - Richard Feynman* > > *"For me, I am driven by two main * > *philosophies: know more today about* > *the world than I knew yesterday and* > *lessen the suffering of others. You'd* > *be surprised how far that gets * > *you. " - Neil deGrasse Tyson* > ----------------------------------------------- > > > > On Thu, Oct 15, 2020 at 11:23 AM David Poole <davpoole(a)iastate.edu> wrote: > > Hi Kevin, > > Ah, that's all good to know! Thank you very much! I will try it out and > see what happens. > > Best, > -David Poole > > ----------------------------------------------- > > > *"It is the mark of an educated mind to be able to entertain a thought > without accepting it." - Aristotle* > > David Poole > Iowa State University > 201 Spedding Hall > davpoole(a)iastate.edu > > > > *"I would rather have questions that can't be answered than answers that > can't be questioned." - Richard Feynman* > > *"For me, I am driven by two main * > *philosophies: know more today about* > *the world than I knew yesterday and* > *lessen the suffering of others. You'd* > *be surprised how far that gets * > *you. " - Neil deGrasse Tyson* > ----------------------------------------------- > > > > On Tue, Oct 13, 2020 at 11:16 AM Harms, Kevin <harms(a)alcf.anl.gov> wrote: > > David, > > cudaStreamSynchronize blocks until all stream operations are completed. > A queue wait() function should be equivalent. The one caveat is > queue.wait() does not wait for all data transfers to be complete. It is not > clear from the documentation on cudaStreamSynchronize if it does or doesn't > wait for memory transfer completion. > > As for cudaStreamCreateWithFlags, if you're using cudaStreamNonBlocking, > i think the default behavior of SYCL will be the same as operations in > different queues may run concurrently, however, if you introduce some > dependency between kernels in different queues, then execution will no > longer allow concurrency. > > kevin > > ________________________________________ > From: Ecp-aurora-sdk-discuss < > ecp-aurora-sdk-discuss-bounces(a)lists.jlse.anl.gov> on behalf of David > Poole via Ecp-aurora-sdk-discuss < > ecp-aurora-sdk-discuss(a)lists.jlse.anl.gov> > Sent: Monday, October 12, 2020 5:44 PM > To: ecp-aurora-sdk-discuss(a)lists.jlse.anl.gov > Subject: [Ecp-aurora-sdk-discuss] Translating Streams to Queues > > Hi all, > > I wanted to ask about translating CUDA streams to DPC++ queues in the > porting of CUDA to DPC++ code. > > I am currently working on porting a couple of files which are heavy in > stream manipulation, using functions such as cudaStreamSynchronize, > cudaStreamCreateWithFlags, and cudaStreamDestroy. In trying the DPC++ > translator on the code I am working on, these functions did not translate > over. If I recall, CUDA streams (kind of) map to DPC++ queues, and I'm > guessing cudaStreamDestroy could be handled by object lifetime rules/RAII; > but I'm not quite sure about the other two functions could be mapped to > DPC++ queues. How would one go about that, then? > > Thank you very much in advance! > > Sincerely, > -David Poole > > ----------------------------------------------- > "It is the mark of an educated mind to > be able to entertain a thought without > accepting it." - Aristotle > > David Poole > Iowa State University > 201 Spedding Hall > davpoole(a)iastate.edu<mailto:[email protected]> > > "I would rather have questions that can't > be answered than answers that can't be > questioned." - Richard Feynman > > "For me, I am driven by two main > philosophies: know more today about > the world than I knew yesterday and > lessen the suffering of others. You'd > be surprised how far that gets > you. " - Neil deGrasse Tyson > ----------------------------------------------- > >

1 0

New Aurora SDK available in JLSE
by Rosales-fernandez, Carlos 16 Oct '20

16 Oct '20

Hi all, We are pleased to announce that a new Aurora SDK has been deployed in JLSE. This version 2020.10.30.001 includes multiple improvements and bug fixes. We want to make this available ahead of the ALCF power outages early next week for those of you that would like to start early testing, but full release notes and change of default modules version will happen next week. For now, please use "module load oneapi/2020.10.30.001" if you wish to test the new release. Expect an email on Tuesday morning indicating the default modules have changed and the JLSE wiki has been updated with the proper release notes. Public Documentation ==================== * ALCF oneAPI overview - https://www.alcf.anl.gov/support-center/aurora/oneapi-overview * Intel GPGPU Software Information - https://dgpu-docs.intel.com/ * Intel oneAPI Beta Documentation - https://software.intel.com/content/www/us/en/develop/tools/oneapi/documenta… Please let us know of any issues with the release or the environment settings. Regards, Carlos

1 0

Re: [Ecp-aurora-sdk-discuss] Translating Streams to Queues
by Luo, Ye 16 Oct '20

16 Oct '20

it moves preallocated memory from host to pinned memory. Please be more specif on "moves". cudaHostRegister? then from pinned memory to device memory zero copy, page migration or copied to a device memory segment? Ye ________________________________ From: Ecp-aurora-sdk-discuss <ecp-aurora-sdk-discuss-bounces(a)lists.jlse.anl.gov> on behalf of David Poole via Ecp-aurora-sdk-discuss <ecp-aurora-sdk-discuss(a)lists.jlse.anl.gov> Sent: Thursday, October 15, 2020 4:19 PM To: Harms, Kevin <harms(a)alcf.anl.gov> Cc: ecp-aurora-sdk-discuss(a)lists.jlse.anl.gov <ecp-aurora-sdk-discuss(a)lists.jlse.anl.gov> Subject: Re: [Ecp-aurora-sdk-discuss] Translating Streams to Queues Hi all, I have another question, if that is all right. One of the big features of the code that I am working on, is that it moves preallocated memory from host to pinned memory, then from pinned memory to device memory. I was wondering if DPC++ currently has support for pinned memory access/allocations. Thanks again! Sincerely, -David Poole ----------------------------------------------- "It is the mark of an educated mind to be able to entertain a thought without accepting it." - Aristotle David Poole Iowa State University 201 Spedding Hall davpoole(a)iastate.edu<mailto:[email protected]> "I would rather have questions that can't be answered than answers that can't be questioned." - Richard Feynman "For me, I am driven by two main philosophies: know more today about the world than I knew yesterday and lessen the suffering of others. You'd be surprised how far that gets you. " - Neil deGrasse Tyson ----------------------------------------------- On Thu, Oct 15, 2020 at 11:23 AM David Poole <davpoole(a)iastate.edu<mailto:[email protected]>> wrote: Hi Kevin, Ah, that's all good to know! Thank you very much! I will try it out and see what happens. Best, -David Poole ----------------------------------------------- "It is the mark of an educated mind to be able to entertain a thought without accepting it." - Aristotle David Poole Iowa State University 201 Spedding Hall davpoole(a)iastate.edu<mailto:[email protected]> "I would rather have questions that can't be answered than answers that can't be questioned." - Richard Feynman "For me, I am driven by two main philosophies: know more today about the world than I knew yesterday and lessen the suffering of others. You'd be surprised how far that gets you. " - Neil deGrasse Tyson ----------------------------------------------- On Tue, Oct 13, 2020 at 11:16 AM Harms, Kevin <harms(a)alcf.anl.gov<mailto:[email protected]>> wrote: David, cudaStreamSynchronize blocks until all stream operations are completed. A queue wait() function should be equivalent. The one caveat is queue.wait() does not wait for all data transfers to be complete. It is not clear from the documentation on cudaStreamSynchronize if it does or doesn't wait for memory transfer completion. As for cudaStreamCreateWithFlags, if you're using cudaStreamNonBlocking, i think the default behavior of SYCL will be the same as operations in different queues may run concurrently, however, if you introduce some dependency between kernels in different queues, then execution will no longer allow concurrency. kevin ________________________________________ From: Ecp-aurora-sdk-discuss <ecp-aurora-sdk-discuss-bounces(a)lists.jlse.anl.gov<mailto:[email protected]>> on behalf of David Poole via Ecp-aurora-sdk-discuss <ecp-aurora-sdk-discuss(a)lists.jlse.anl.gov<mailto:[email protected]>> Sent: Monday, October 12, 2020 5:44 PM To: ecp-aurora-sdk-discuss(a)lists.jlse.anl.gov<mailto:[email protected]> Subject: [Ecp-aurora-sdk-discuss] Translating Streams to Queues Hi all, I wanted to ask about translating CUDA streams to DPC++ queues in the porting of CUDA to DPC++ code. I am currently working on porting a couple of files which are heavy in stream manipulation, using functions such as cudaStreamSynchronize, cudaStreamCreateWithFlags, and cudaStreamDestroy. In trying the DPC++ translator on the code I am working on, these functions did not translate over. If I recall, CUDA streams (kind of) map to DPC++ queues, and I'm guessing cudaStreamDestroy could be handled by object lifetime rules/RAII; but I'm not quite sure about the other two functions could be mapped to DPC++ queues. How would one go about that, then? Thank you very much in advance! Sincerely, -David Poole ----------------------------------------------- "It is the mark of an educated mind to be able to entertain a thought without accepting it." - Aristotle David Poole Iowa State University 201 Spedding Hall davpoole(a)iastate.edu<mailto:[email protected]><mailto:[email protected]<mailto:[email protected]>> "I would rather have questions that can't be answered than answers that can't be questioned." - Richard Feynman "For me, I am driven by two main philosophies: know more today about the world than I knew yesterday and lessen the suffering of others. You'd be surprised how far that gets you. " - Neil deGrasse Tyson -----------------------------------------------

1 0

Re: [Ecp-aurora-sdk-discuss] Translating Streams to Queues
by Bryce Allen 16 Oct '20

16 Oct '20

The hipSYCL cuda backend and API names seem to support malloc_host implies pinned hypothesis: https://github.com/illuhad/hipSYCL/blob/develop/src/runtime/cuda/cuda_alloc… ________________________________ From: Ecp-aurora-sdk-discuss <ecp-aurora-sdk-discuss-bounces(a)lists.jlse.anl.gov> on behalf of Hammond, Jeff R via Ecp-aurora-sdk-discuss <ecp-aurora-sdk-discuss(a)lists.jlse.anl.gov> Sent: Friday, October 16, 2020, 12:59 To: David Poole; Harms, Kevin Cc: ecp-aurora-sdk-discuss(a)lists.jlse.anl.gov Subject: Re: [Ecp-aurora-sdk-discuss] Translating Streams to Queues USM malloc_host _should_ do this but I’ve not verified. Jeff From: Ecp-aurora-sdk-discuss <ecp-aurora-sdk-discuss-bounces(a)lists.jlse.anl.gov> on behalf of David Poole via Ecp-aurora-sdk-discuss <ecp-aurora-sdk-discuss(a)lists.jlse.anl.gov> Reply-To: David Poole <davpoole(a)iastate.edu> Date: Thursday, October 15, 2020 at 2:20 PM To: "Kevin Harms (Argonne)" <harms(a)alcf.anl.gov> Cc: "ecp-aurora-sdk-discuss(a)lists.jlse.anl.gov" <ecp-aurora-sdk-discuss(a)lists.jlse.anl.gov> Subject: Re: [Ecp-aurora-sdk-discuss] Translating Streams to Queues Hi all, I have another question, if that is all right. One of the big features of the code that I am working on, is that it moves preallocated memory from host to pinned memory, then from pinned memory to device memory. I was wondering if DPC++ currently has support for pinned memory access/allocations. Thanks again! Sincerely, -David Poole ----------------------------------------------- "It is the mark of an educated mind to be able to entertain a thought without accepting it." - Aristotle David Poole Iowa State University 201 Spedding Hall davpoole(a)iastate.edu<mailto:[email protected]> "I would rather have questions that can't be answered than answers that can't be questioned." - Richard Feynman "For me, I am driven by two main philosophies: know more today about the world than I knew yesterday and lessen the suffering of others. You'd be surprised how far that gets you. " - Neil deGrasse Tyson ----------------------------------------------- On Thu, Oct 15, 2020 at 11:23 AM David Poole <davpoole(a)iastate.edu<mailto:[email protected]>> wrote: Hi Kevin, Ah, that's all good to know! Thank you very much! I will try it out and see what happens. Best, -David Poole ----------------------------------------------- "It is the mark of an educated mind to be able to entertain a thought without accepting it." - Aristotle David Poole Iowa State University 201 Spedding Hall davpoole(a)iastate.edu<mailto:[email protected]> "I would rather have questions that can't be answered than answers that can't be questioned." - Richard Feynman "For me, I am driven by two main philosophies: know more today about the world than I knew yesterday and lessen the suffering of others. You'd be surprised how far that gets you. " - Neil deGrasse Tyson ----------------------------------------------- On Tue, Oct 13, 2020 at 11:16 AM Harms, Kevin <harms(a)alcf.anl.gov<mailto:[email protected]>> wrote: David, cudaStreamSynchronize blocks until all stream operations are completed. A queue wait() function should be equivalent. The one caveat is queue.wait() does not wait for all data transfers to be complete. It is not clear from the documentation on cudaStreamSynchronize if it does or doesn't wait for memory transfer completion. As for cudaStreamCreateWithFlags, if you're using cudaStreamNonBlocking, i think the default behavior of SYCL will be the same as operations in different queues may run concurrently, however, if you introduce some dependency between kernels in different queues, then execution will no longer allow concurrency. kevin ________________________________________ From: Ecp-aurora-sdk-discuss <ecp-aurora-sdk-discuss-bounces(a)lists.jlse.anl.gov<mailto:[email protected]>> on behalf of David Poole via Ecp-aurora-sdk-discuss <ecp-aurora-sdk-discuss(a)lists.jlse.anl.gov<mailto:[email protected]>> Sent: Monday, October 12, 2020 5:44 PM To: ecp-aurora-sdk-discuss(a)lists.jlse.anl.gov<mailto:[email protected]> Subject: [Ecp-aurora-sdk-discuss] Translating Streams to Queues Hi all, I wanted to ask about translating CUDA streams to DPC++ queues in the porting of CUDA to DPC++ code. I am currently working on porting a couple of files which are heavy in stream manipulation, using functions such as cudaStreamSynchronize, cudaStreamCreateWithFlags, and cudaStreamDestroy. In trying the DPC++ translator on the code I am working on, these functions did not translate over. If I recall, CUDA streams (kind of) map to DPC++ queues, and I'm guessing cudaStreamDestroy could be handled by object lifetime rules/RAII; but I'm not quite sure about the other two functions could be mapped to DPC++ queues. How would one go about that, then? Thank you very much in advance! Sincerely, -David Poole ----------------------------------------------- "It is the mark of an educated mind to be able to entertain a thought without accepting it." - Aristotle David Poole Iowa State University 201 Spedding Hall davpoole(a)iastate.edu<mailto:[email protected]><mailto:[email protected]<mailto:[email protected]>> "I would rather have questions that can't be answered than answers that can't be questioned." - Richard Feynman "For me, I am driven by two main philosophies: know more today about the world than I knew yesterday and lessen the suffering of others. You'd be surprised how far that gets you. " - Neil deGrasse Tyson -----------------------------------------------

1 0

Re: [Ecp-aurora-sdk-discuss] Translating Streams to Queues
by Hammond, Jeff R 16 Oct '20

16 Oct '20

USM malloc_host _should_ do this but I’ve not verified. Jeff From: Ecp-aurora-sdk-discuss <ecp-aurora-sdk-discuss-bounces(a)lists.jlse.anl.gov> on behalf of David Poole via Ecp-aurora-sdk-discuss <ecp-aurora-sdk-discuss(a)lists.jlse.anl.gov> Reply-To: David Poole <davpoole(a)iastate.edu> Date: Thursday, October 15, 2020 at 2:20 PM To: "Kevin Harms (Argonne)" <harms(a)alcf.anl.gov> Cc: "ecp-aurora-sdk-discuss(a)lists.jlse.anl.gov" <ecp-aurora-sdk-discuss(a)lists.jlse.anl.gov> Subject: Re: [Ecp-aurora-sdk-discuss] Translating Streams to Queues Hi all, I have another question, if that is all right. One of the big features of the code that I am working on, is that it moves preallocated memory from host to pinned memory, then from pinned memory to device memory. I was wondering if DPC++ currently has support for pinned memory access/allocations. Thanks again! Sincerely, -David Poole ----------------------------------------------- "It is the mark of an educated mind to be able to entertain a thought without accepting it." - Aristotle David Poole Iowa State University 201 Spedding Hall davpoole(a)iastate.edu<mailto:[email protected]> "I would rather have questions that can't be answered than answers that can't be questioned." - Richard Feynman "For me, I am driven by two main philosophies: know more today about the world than I knew yesterday and lessen the suffering of others. You'd be surprised how far that gets you. " - Neil deGrasse Tyson ----------------------------------------------- On Thu, Oct 15, 2020 at 11:23 AM David Poole <davpoole(a)iastate.edu<mailto:[email protected]>> wrote: Hi Kevin, Ah, that's all good to know! Thank you very much! I will try it out and see what happens. Best, -David Poole ----------------------------------------------- "It is the mark of an educated mind to be able to entertain a thought without accepting it." - Aristotle David Poole Iowa State University 201 Spedding Hall davpoole(a)iastate.edu<mailto:[email protected]> "I would rather have questions that can't be answered than answers that can't be questioned." - Richard Feynman "For me, I am driven by two main philosophies: know more today about the world than I knew yesterday and lessen the suffering of others. You'd be surprised how far that gets you. " - Neil deGrasse Tyson ----------------------------------------------- On Tue, Oct 13, 2020 at 11:16 AM Harms, Kevin <harms(a)alcf.anl.gov<mailto:[email protected]>> wrote: David, cudaStreamSynchronize blocks until all stream operations are completed. A queue wait() function should be equivalent. The one caveat is queue.wait() does not wait for all data transfers to be complete. It is not clear from the documentation on cudaStreamSynchronize if it does or doesn't wait for memory transfer completion. As for cudaStreamCreateWithFlags, if you're using cudaStreamNonBlocking, i think the default behavior of SYCL will be the same as operations in different queues may run concurrently, however, if you introduce some dependency between kernels in different queues, then execution will no longer allow concurrency. kevin ________________________________________ From: Ecp-aurora-sdk-discuss <ecp-aurora-sdk-discuss-bounces(a)lists.jlse.anl.gov<mailto:[email protected]>> on behalf of David Poole via Ecp-aurora-sdk-discuss <ecp-aurora-sdk-discuss(a)lists.jlse.anl.gov<mailto:[email protected]>> Sent: Monday, October 12, 2020 5:44 PM To: ecp-aurora-sdk-discuss(a)lists.jlse.anl.gov<mailto:[email protected]> Subject: [Ecp-aurora-sdk-discuss] Translating Streams to Queues Hi all, I wanted to ask about translating CUDA streams to DPC++ queues in the porting of CUDA to DPC++ code. I am currently working on porting a couple of files which are heavy in stream manipulation, using functions such as cudaStreamSynchronize, cudaStreamCreateWithFlags, and cudaStreamDestroy. In trying the DPC++ translator on the code I am working on, these functions did not translate over. If I recall, CUDA streams (kind of) map to DPC++ queues, and I'm guessing cudaStreamDestroy could be handled by object lifetime rules/RAII; but I'm not quite sure about the other two functions could be mapped to DPC++ queues. How would one go about that, then? Thank you very much in advance! Sincerely, -David Poole ----------------------------------------------- "It is the mark of an educated mind to be able to entertain a thought without accepting it." - Aristotle David Poole Iowa State University 201 Spedding Hall davpoole(a)iastate.edu<mailto:[email protected]><mailto:[email protected]<mailto:[email protected]>> "I would rather have questions that can't be answered than answers that can't be questioned." - Richard Feynman "For me, I am driven by two main philosophies: know more today about the world than I knew yesterday and lessen the suffering of others. You'd be surprised how far that gets you. " - Neil deGrasse Tyson -----------------------------------------------

1 0