August 2022 - Ecp-aurora-sdk-discuss

Florentia00 up for user jobs
by Muralidharan, Servesh 23 Aug '22

23 Aug '22

Hello Florentia Node users, We are pleased to announce the addition of the second PVC A0 silicon node for user access i.e. Florentia00. Some points to keep in note, 1. This node has many changes done to the hardware and software and is running a much newer stack in comparison to the other node in use i.e. Florentia01. 2. This node will operate under the `florentia_debug` queue. * Florentia01 can be accessed via `florentia` main queue. 3. In our testing we have seen improvements both to the performance and stability of the node. There should be no changes required from the user’s env perspective. Please do let us know if you encounter any issues. On behalf of early hardware team. -- Servesh

1 0

quick two-question survey about MKL use in applications
by Bertoni, Colleen 19 Aug '22

19 Aug '22

Hi everyone, The MKL team is considering deprecating some features in oneMKL. To help provide input into the decisions, could you reply if: 1. Your application or any you know about uses the DPC++ buffer APIs provided in oneMKL or 2. Your application or any you know about primarily uses the OpenCL backend instead of the (default) Level Zero backend with oneMKL? Thanks, Colleen

1 0

Re: [Ecp-aurora-sdk-discuss] Arcticus and florentia theoretical peak performance
by Muralidharan, Servesh 18 Aug '22

18 Aug '22

Hi Asim, The GPUs on JLSE are definitely different from anything that is available publicly and therefore conform to NDA/TOU. If you look at this page, https://wiki.jlse.anl.gov/display/inteldga/Getting+Started And the presentation titled “NDA Presentation on Aurora Testbed from ECP AM'22 for quick reference” in slide 9 you will find the theoretical max and then slides 21,22,23 will show the measured peaks on very simple benchmark. You had mentioned dense linear algebra, so I was wondering if you were looking for specific GEMM or other BLAS3 performance, which we do not have documented as its highly dependent on the config/opt/implementation. As for the meaning of those env, some of them should have documentation if you do module help on the files. Many still needs this to be added so they are WIP. They are simply setting or rather showing the config of the platform, for ex, in the case of Arcticus, we have multi_gpu which means both the GPUs are visible. -- Servesh From: Asim YarKhan <yarkhan(a)icl.utk.edu> Date: Thursday, August 18, 2022 at 5:53 PM To: Muralidharan, Servesh <servesh(a)anl.gov> Cc: ecp-aurora-sdk-discuss(a)jlse.anl.gov <ecp-aurora-sdk-discuss(a)jlse.anl.gov> Subject: Re: [Ecp-aurora-sdk-discuss] Arcticus and florentia theoretical peak performance Hi Servesh, > Is a table on the JLSE "Intel Discrete GPU" Confluence site that talks about the theoretical peaks of the arcticus and florentia GPUs? > For 1, what specific operation peak are you looking for on these systems? I am looking for something that gives the peak theoretical (or practical) performance of a single GPU on arcticus running double precision operations. I found the following documentation online for the Arctic Sound 2T, https://www.techpowerup.com/gpu-specs/arctic-sound-2t.c3503 The double precision performance here is shown as 3.4 TFlops [cid:ii_l6zmlpg10] That does not match up with my expectations, so I was looking on the JLSE confluence site for more information. I was hoping that there was a page on the JLSE confluence that simply listed theoretical peak GFlops for each precision. For the second question re the meaning of " intel_gpu_env/multi_gpu ", I will wait for more details... Thanks, Asim On Wed, Aug 17, 2022 at 12:41 PM Muralidharan, Servesh <servesh(a)anl.gov<mailto:[email protected]>> wrote: Hi Asim, For 1, what specific operation peak are you looking for on these systems? There is a variety of config options that can influence these peaks and given these are early testbeds that go through upgrades that could influence these numbers as well. For 2, i.e. intel_gpu_env/<modulefile> these are under testing at the moment and we would provide an update aligned with the next SDK release. These modules set various environment options that otherwise would need to be added to user’s job script. Currently, they have a config that defaults to the recommend settings for each of the platform that provides the best out of the box performance/WA for the early hardware. -- Servesh From: Ecp-aurora-sdk-discuss <ecp-aurora-sdk-discuss-bounces(a)lists.jlse.anl.gov<mailto:[email protected]>> on behalf of Asim YarKhan via Ecp-aurora-sdk-discuss <ecp-aurora-sdk-discuss(a)lists.jlse.anl.gov<mailto:[email protected]>> Date: Wednesday, August 17, 2022 at 11:12 AM To: ecp-aurora-sdk-discuss(a)jlse.anl.gov<mailto:[email protected]> <ecp-aurora-sdk-discuss(a)jlse.anl.gov<mailto:[email protected]>> Subject: [Ecp-aurora-sdk-discuss] Arcticus and florentia theoretical peak performance The ECP SLATE project is working on distributed dense linear algebra libraries. We are trying to figure out how well our implementations are performing on these platforms. To that end, I was wondering if there is a table on the JLSE "Intel Discrete GPU" Confluence site that talks about the theoretical peaks of the arcticus and florentia GPUs? Also, is there a page that describes best practices in launching jobs and gives explanations for the meaning of these modules intel_gpu_env/multi_gpu

1 0

Re: [Ecp-aurora-sdk-discuss] Arcticus and florentia theoretical peak performance
by Asim YarKhan 18 Aug '22

18 Aug '22

Hi Servesh, > > Is a table on the JLSE "Intel Discrete GPU" Confluence site that talks > about the theoretical peaks of the arcticus and florentia GPUs? > > For 1, what specific operation peak are you looking for on these > systems? I am looking for something that gives the peak theoretical (or practical) performance of a single GPU on arcticus running double precision operations. I found the following documentation online for the Arctic Sound 2T, https://www.techpowerup.com/gpu-specs/arctic-sound-2t.c3503 The double precision performance here is shown as 3.4 TFlops [image: image.png] That does not match up with my expectations, so I was looking on the JLSE confluence site for more information. I was hoping that there was a page on the JLSE confluence that simply listed theoretical peak GFlops for each precision. For the second question re the meaning of " intel_gpu_env/multi_gpu ", I will wait for more details... Thanks, Asim On Wed, Aug 17, 2022 at 12:41 PM Muralidharan, Servesh <servesh(a)anl.gov> wrote: > Hi Asim, > > > > For 1, what specific operation peak are you looking for on these systems? > There is a variety of config options that can influence these peaks and > given these are early testbeds that go through upgrades that could > influence these numbers as well. > > > > For 2, i.e. intel_gpu_env/<modulefile> these are under testing at the > moment and we would provide an update aligned with the next SDK release. > These modules set various environment options that otherwise would need to > be added to user’s job script. > > Currently, they have a config that defaults to the recommend settings for > each of the platform that provides the best out of the box performance/WA > for the early hardware. > > > > -- > > Servesh > > > > *From: *Ecp-aurora-sdk-discuss < > ecp-aurora-sdk-discuss-bounces(a)lists.jlse.anl.gov> on behalf of Asim > YarKhan via Ecp-aurora-sdk-discuss < > ecp-aurora-sdk-discuss(a)lists.jlse.anl.gov> > *Date: *Wednesday, August 17, 2022 at 11:12 AM > *To: *ecp-aurora-sdk-discuss(a)jlse.anl.gov < > ecp-aurora-sdk-discuss(a)jlse.anl.gov> > *Subject: *[Ecp-aurora-sdk-discuss] Arcticus and florentia theoretical > peak performance > > The ECP SLATE project is working on distributed dense linear algebra > libraries. > > We are trying to figure out how well our implementations are performing on > these platforms. > > > > To that end, I was wondering if there is a table on the JLSE "Intel > Discrete GPU" Confluence site that talks about the theoretical peaks of > the arcticus and florentia GPUs? > > > > Also, is there a page that describes best practices in launching jobs and > gives explanations for the meaning of these modules > > intel_gpu_env/multi_gpu >

1 0

Re: [Ecp-aurora-sdk-discuss] Arcticus and florentia theoretical peak performance
by Muralidharan, Servesh 17 Aug '22

17 Aug '22

Hi Asim, For 1, what specific operation peak are you looking for on these systems? There is a variety of config options that can influence these peaks and given these are early testbeds that go through upgrades that could influence these numbers as well. For 2, i.e. intel_gpu_env/<modulefile> these are under testing at the moment and we would provide an update aligned with the next SDK release. These modules set various environment options that otherwise would need to be added to user’s job script. Currently, they have a config that defaults to the recommend settings for each of the platform that provides the best out of the box performance/WA for the early hardware. -- Servesh From: Ecp-aurora-sdk-discuss <ecp-aurora-sdk-discuss-bounces(a)lists.jlse.anl.gov> on behalf of Asim YarKhan via Ecp-aurora-sdk-discuss <ecp-aurora-sdk-discuss(a)lists.jlse.anl.gov> Date: Wednesday, August 17, 2022 at 11:12 AM To: ecp-aurora-sdk-discuss(a)jlse.anl.gov <ecp-aurora-sdk-discuss(a)jlse.anl.gov> Subject: [Ecp-aurora-sdk-discuss] Arcticus and florentia theoretical peak performance The ECP SLATE project is working on distributed dense linear algebra libraries. We are trying to figure out how well our implementations are performing on these platforms. To that end, I was wondering if there is a table on the JLSE "Intel Discrete GPU" Confluence site that talks about the theoretical peaks of the arcticus and florentia GPUs? Also, is there a page that describes best practices in launching jobs and gives explanations for the meaning of these modules intel_gpu_env/multi_gpu

1 0

Arcticus and florentia theoretical peak performance
by Asim YarKhan 17 Aug '22

17 Aug '22

The ECP SLATE project is working on distributed dense linear algebra libraries. We are trying to figure out how well our implementations are performing on these platforms. To that end, I was wondering if there is a table on the JLSE "Intel Discrete GPU" Confluence site that talks about the theoretical peaks of the arcticus and florentia GPUs? Also, is there a page that describes best practices in launching jobs and gives explanations for the meaning of these modules intel_gpu_env/multi_gpu

1 0

Announcing availability of 2022.3.0 release of the DL Frameworks
by Ramer, Rebecca 12 Aug '22

12 Aug '22

Hi, We would like to announce availability of a new frameworks module for pytorch and tensorflow for use in the JLSE. This release is based on the 2022.3.0 versions of the DeepLearning Frameworks. These versions of the frameworks can be run on on ATS-P B0 only, i.e. arcticus nodes. At this time, distributed training workloads are not supported. An additional announcement will be sent when the configuration needed to run distributed training workloads is available. This release is available via: frameworks/2022.08.15.001 which supports implicit scaling frameworks/2022.08.15.002 which supports explicit scaling Validation of the NRE models showed that the tensorflow FFN and the pytorch LQCD models need to use the frameworks/2022.08.15.002 modulefile to run successfully. Information on how to setup and work with this release can be found here for pytorch: https://wiki.jlse.anl.gov/display/inteldga/2022.3.0+-+Pytorch and here for tensorflow: https://wiki.jlse.anl.gov/display/inteldga/2022.3.0+-+Tensorflow Please let us know of any issues encountered. Rebecca Ramer rebecca.ramer(a)intel.com<mailto:[email protected]>

1 0