October 2017 - Anl-intel-lrz-knr - mailman3.cels.anl.gov

Re: [Anl-intel-csa] pragma omp target example
by Vitali A. Morozov 31 Oct '17

31 Oct '17

Hi Tim, I would like to confirm that both CSA_SAVE_TEMPS and CSA_TEMP_PREFIX variables are working as expected. The offloaded section of the code is compiled such that the graph takes the correct name and placed to the working directory. A consecutive execution of the offloaded section dumps statistics to the file with an increment in the name so that each invocation can be examined later for performance. Thanks a lot for the quick update! Vitali On 10/30/2017 01:36 PM, Creech, Timothy M wrote: > Hi Vitali, > > We have just enabled a new release on JLSE: 2017-10-29-release. Please see the new $CSA_TOP/samples/demo/Makefile to learn about the "CSA_TEMP_PREFIX" variable. The offload plugin in this release also numbers dump files according to a counter, allowing you to see statistics for each offload. > > Thanks, > Tim > > -----Original Message----- > From: Vitali A. Morozov [mailto:[email protected]] > Sent: Monday, October 16, 2017 12:38 PM > To: Creech, Timothy M <timothy.m.creech(a)intel.com>; anl-intel-csa(a)lists.jlse.anl.gov > Cc: Glossop, Kent <kent.glossop(a)intel.com>; Kalyanasundaram Kumaran <kumaran(a)alcf.anl.gov>; Vitali A. Morozov <morozov(a)alcf.anl.gov> > Subject: Re: pragma omp target example > > Thanks Tim! I found an example in > $(CSA_TOP)/samples/omp-offload/demo/Makefile very useful. > > CSA_DUMP_STATS=1 works - the stat file is created. If the offload section is called several times, it seems to overwrite the files with the statistics corresponding to the last offload. Is there any way to 1) keep statistics for each offload, and/or 2) choose to aggregate all statistics from an offload section into a single file? > > CSA_SAVE_TEMPS variable is also working, it keeps the graph file in /tmp. I think this is quite a polluting solution, a user would definitely want to see the graph (at least at this stage of the usage) and the number of generated graphs will quickly accumulate polluting the /tmp. Of cause, the user should clean after himself ... Is there a way to specify the name of the generated graph for each offloaded section? > At first, something like a baseline name and a counter that specify the order of the offload section would suffice. Ideally, not in /tmp but in working directory. > > Thanks for you help! > > Vitali > > > On 10/16/2017 10:33 AM, Creech, Timothy M wrote: >> Hi Vitali, >> >> To ask the offload plugin to generate ".stat" files, please try running with the environment variable "CSA_DUMP_STATS" set to "1". >> >> To read about more environment variables which influence offload compilation/execution, you might take a look at this file: >> $CSA_TOP/samples/omp-offload/demo/Makefile >> >> It uses and documents (in comments) virtually all of the features of >> the current CSA offload plugin. The most useful one that we haven't >> discussed in this thread is probably "CSA_COMPILE_OPTIONS", which >> allows you to specify the flags given to Clang for compiling IR when >> the host executable is run. (E.g., >> CSA_COMPILE_OPTIONS="-ffp-contract=fast" .) >> >> Thanks, >> Tim >> >> -----Original Message----- >> From: Vitali A Morozov [mailto:[email protected]] >> Sent: Friday, October 13, 2017 7:45 PM >> To: Creech, Timothy M <timothy.m.creech(a)intel.com> >> Cc: Glossop, Kent <kent.glossop(a)intel.com>; Kalyanasundaram Kumaran >> <kumaran(a)alcf.anl.gov>; Vitali A. Morozov <morozov(a)alcf.anl.gov> >> Subject: Re: pragma omp target example >> >> Thanks Tim, >> >> You are right - this seems to be running on the CSA as it is running approximately as slow as previously with the csa_invoke(). When the run is finished, I am getting the correct answer; now, how can I get the usual CSA statistics? >> >> Thanks again! >> >> Vitali >> >> >> >> >> On 10/13/2017 06:16 PM, Creech, Timothy M wrote: >>> Hi Vitali, >>> >>> I believe you're hitting a bug in the upstream OpenMP offload logic. >>> It's not recognizing that it needs to pass the results of the >>> offloaded region back to the host, so it's "optimizing" it down to an >>> empty offload function. (Try running with CSA_VERBOSE=1 to catch this >>> it in the act.) >>> >>> You can work around the problem by changing your target pragma slightly: >>> #pragma omp target map(tofrom: dx2,dy2,dz2) >>> >>> This should get you correct results running on CSA. Unfortunately, I've been kicked off of our systems for weekend maintenance so I can't say much about the resulting performance right now. There may be some easy improvements we can try next week. >>> >>> Thanks, >>> Tim >>> >>> -----Original Message----- >>> From: Vitali A. Morozov [mailto:[email protected]] >>> Sent: Friday, October 13, 2017 6:04 PM >>> To: Creech, Timothy M <timothy.m.creech(a)intel.com> >>> Cc: Glossop, Kent <kent.glossop(a)intel.com>; Kalyanasundaram Kumaran >>> <kumaran(a)alcf.anl.gov>; Vitali A. Morozov <morozov(a)alcf.anl.gov> >>> Subject: pragma omp target example >>> >>> Hi Tim, >>> >>> I am trying to reproduce the "#pragma omp target" example from your slide on HACCmk example. >>> >>> 1. Compile and run everything with "#pragma omp target" commented out as a sanity check that clang does the right thing on it own. >>> >>> SUCCESS: compiled, ran, and the result is correct: >>> >>> OMP_NUM_THREADS=1 ./HACCmk >>> Count is set 327 >>> 768: result 120062.03561401 expected 120060.7215 >>> >>> 2. Uncommented the statement. Compiling: >>> >>> /soft/restricted/intel_csa/comp/tools/2017-09-06-release/tools/instal >>> l >>> /bin/clang >>> -O2 -fopenmp -fopenmp-targets=csa -ffp-contract=fast -c main.cpp -o >>> main.o >>> /soft/restricted/intel_csa/comp/tools/2017-09-06-release/tools/instal >>> l >>> /bin/clang >>> -O2 -fopenmp -fopenmp-targets=csa -o HACCmk main.o >>> -L/soft/restricted/intel_csa/comp/tools/2017-09-06-release/tools/loca >>> l >>> /lib >>> -lm >>> >>> Seems to be a success, the executable is created, and the library "libomptarget.so" is linked in: >>> >>> ldd ./HACCmk >>> linux-vdso.so.1 => (0x00007ffce43f8000) >>> libm.so.6 => /lib64/libm.so.6 (0x00002ac9cbdc8000) >>> libomp.so => >>> /soft/restricted/intel_csa/comp/tools/2017-09-06-release/tools/local/ >>> l >>> ib/libomp.so >>> (0x00002ac9cc0ca000) >>> libomptarget.so => >>> /soft/restricted/intel_csa/comp/tools/2017-09-06-release/tools/local/ >>> l >>> ib/libomptarget.so >>> (0x00002ac9cc3a8000) >>> libpthread.so.0 => /lib64/libpthread.so.0 (0x00002ac9cc602000) >>> libc.so.6 => /lib64/libc.so.6 (0x00002ac9cc81e000) >>> /lib64/ld-linux-x86-64.so.2 (0x0000563f1f42e000) >>> libdl.so.2 => /lib64/libdl.so.2 (0x00002ac9ccbe2000) >>> >>> 3. Running does not produce the right answer. From the run time, it does not look like an executable is calling a simulator: >>> >>> ./HACCmk >>> Count is set 327 >>> 768: result 0.58757814 expected 120060.7215 >>> >>> OMP_NUM_THREADS=1 ./HACCmk >>> Count is set 327 >>> 768: result 0.58757814 expected 120060.7215 >>> >>> The tarball is attached. Any advice? >>> >>> Thanks, >>> >>> Vitali >>> >>> >>> >>>

1 0

Re: [Anl-intel-csa] pragma omp target example
by Creech, Timothy M 30 Oct '17

30 Oct '17

Hi Vitali, We have just enabled a new release on JLSE: 2017-10-29-release. Please see the new $CSA_TOP/samples/demo/Makefile to learn about the "CSA_TEMP_PREFIX" variable. The offload plugin in this release also numbers dump files according to a counter, allowing you to see statistics for each offload. Thanks, Tim -----Original Message----- From: Vitali A. Morozov [mailto:[email protected]] Sent: Monday, October 16, 2017 12:38 PM To: Creech, Timothy M <timothy.m.creech(a)intel.com>; anl-intel-csa(a)lists.jlse.anl.gov Cc: Glossop, Kent <kent.glossop(a)intel.com>; Kalyanasundaram Kumaran <kumaran(a)alcf.anl.gov>; Vitali A. Morozov <morozov(a)alcf.anl.gov> Subject: Re: pragma omp target example Thanks Tim! I found an example in $(CSA_TOP)/samples/omp-offload/demo/Makefile very useful. CSA_DUMP_STATS=1 works - the stat file is created. If the offload section is called several times, it seems to overwrite the files with the statistics corresponding to the last offload. Is there any way to 1) keep statistics for each offload, and/or 2) choose to aggregate all statistics from an offload section into a single file? CSA_SAVE_TEMPS variable is also working, it keeps the graph file in /tmp. I think this is quite a polluting solution, a user would definitely want to see the graph (at least at this stage of the usage) and the number of generated graphs will quickly accumulate polluting the /tmp. Of cause, the user should clean after himself ... Is there a way to specify the name of the generated graph for each offloaded section? At first, something like a baseline name and a counter that specify the order of the offload section would suffice. Ideally, not in /tmp but in working directory. Thanks for you help! Vitali On 10/16/2017 10:33 AM, Creech, Timothy M wrote: > Hi Vitali, > > To ask the offload plugin to generate ".stat" files, please try running with the environment variable "CSA_DUMP_STATS" set to "1". > > To read about more environment variables which influence offload compilation/execution, you might take a look at this file: > $CSA_TOP/samples/omp-offload/demo/Makefile > > It uses and documents (in comments) virtually all of the features of > the current CSA offload plugin. The most useful one that we haven't > discussed in this thread is probably "CSA_COMPILE_OPTIONS", which > allows you to specify the flags given to Clang for compiling IR when > the host executable is run. (E.g., > CSA_COMPILE_OPTIONS="-ffp-contract=fast" .) > > Thanks, > Tim > > -----Original Message----- > From: Vitali A Morozov [mailto:[email protected]] > Sent: Friday, October 13, 2017 7:45 PM > To: Creech, Timothy M <timothy.m.creech(a)intel.com> > Cc: Glossop, Kent <kent.glossop(a)intel.com>; Kalyanasundaram Kumaran > <kumaran(a)alcf.anl.gov>; Vitali A. Morozov <morozov(a)alcf.anl.gov> > Subject: Re: pragma omp target example > > Thanks Tim, > > You are right - this seems to be running on the CSA as it is running approximately as slow as previously with the csa_invoke(). When the run is finished, I am getting the correct answer; now, how can I get the usual CSA statistics? > > Thanks again! > > Vitali > > > > > On 10/13/2017 06:16 PM, Creech, Timothy M wrote: >> Hi Vitali, >> >> I believe you're hitting a bug in the upstream OpenMP offload logic. >> It's not recognizing that it needs to pass the results of the >> offloaded region back to the host, so it's "optimizing" it down to an >> empty offload function. (Try running with CSA_VERBOSE=1 to catch this >> it in the act.) >> >> You can work around the problem by changing your target pragma slightly: >> #pragma omp target map(tofrom: dx2,dy2,dz2) >> >> This should get you correct results running on CSA. Unfortunately, I've been kicked off of our systems for weekend maintenance so I can't say much about the resulting performance right now. There may be some easy improvements we can try next week. >> >> Thanks, >> Tim >> >> -----Original Message----- >> From: Vitali A. Morozov [mailto:[email protected]] >> Sent: Friday, October 13, 2017 6:04 PM >> To: Creech, Timothy M <timothy.m.creech(a)intel.com> >> Cc: Glossop, Kent <kent.glossop(a)intel.com>; Kalyanasundaram Kumaran >> <kumaran(a)alcf.anl.gov>; Vitali A. Morozov <morozov(a)alcf.anl.gov> >> Subject: pragma omp target example >> >> Hi Tim, >> >> I am trying to reproduce the "#pragma omp target" example from your slide on HACCmk example. >> >> 1. Compile and run everything with "#pragma omp target" commented out as a sanity check that clang does the right thing on it own. >> >> SUCCESS: compiled, ran, and the result is correct: >> >> OMP_NUM_THREADS=1 ./HACCmk >> Count is set 327 >> 768: result 120062.03561401 expected 120060.7215 >> >> 2. Uncommented the statement. Compiling: >> >> /soft/restricted/intel_csa/comp/tools/2017-09-06-release/tools/instal >> l >> /bin/clang >> -O2 -fopenmp -fopenmp-targets=csa -ffp-contract=fast -c main.cpp -o >> main.o >> /soft/restricted/intel_csa/comp/tools/2017-09-06-release/tools/instal >> l >> /bin/clang >> -O2 -fopenmp -fopenmp-targets=csa -o HACCmk main.o >> -L/soft/restricted/intel_csa/comp/tools/2017-09-06-release/tools/loca >> l >> /lib >> -lm >> >> Seems to be a success, the executable is created, and the library "libomptarget.so" is linked in: >> >> ldd ./HACCmk >> linux-vdso.so.1 => (0x00007ffce43f8000) >> libm.so.6 => /lib64/libm.so.6 (0x00002ac9cbdc8000) >> libomp.so => >> /soft/restricted/intel_csa/comp/tools/2017-09-06-release/tools/local/ >> l >> ib/libomp.so >> (0x00002ac9cc0ca000) >> libomptarget.so => >> /soft/restricted/intel_csa/comp/tools/2017-09-06-release/tools/local/ >> l >> ib/libomptarget.so >> (0x00002ac9cc3a8000) >> libpthread.so.0 => /lib64/libpthread.so.0 (0x00002ac9cc602000) >> libc.so.6 => /lib64/libc.so.6 (0x00002ac9cc81e000) >> /lib64/ld-linux-x86-64.so.2 (0x0000563f1f42e000) >> libdl.so.2 => /lib64/libdl.so.2 (0x00002ac9ccbe2000) >> >> 3. Running does not produce the right answer. From the run time, it does not look like an executable is calling a simulator: >> >> ./HACCmk >> Count is set 327 >> 768: result 0.58757814 expected 120060.7215 >> >> OMP_NUM_THREADS=1 ./HACCmk >> Count is set 327 >> 768: result 0.58757814 expected 120060.7215 >> >> The tarball is attached. Any advice? >> >> Thanks, >> >> Vitali >> >> >> >>

1 0

[Anl-intel-csa] Conference call tomorrow - Oct 27th (Friday) at 1:00pm
by Kumaran, Kalyan 26 Oct '17

26 Oct '17

We do have our regularly scheduled conference call this Friday, October 27th, at 1:00pm central. For Argonne folks I have reserved 4313. For folks dialing in you will find the Skype information on the confluence page. Elliott Fleming will be presenting his recent talk at PF. Agenda: Title: “CSA late tools” Abstract: This talk covers the toolchain responsible for mapping the abstract CSA dataflow architecture to the physical CSA execution array. We will provide an overview of research and results in this area. Topics covered include: Buffer insertion Place and route Operator fusion Memory operation mapping -Kumar

1 0

Re: [Anl-intel-csa] ANL/Intel CSA Workload Characterization
by Van Der Wijngaart, Rob F 25 Oct '17

25 Oct '17

Very good, I’ll do that. Rob From: Hal Finkel [mailto:[email protected]] Sent: Wednesday, October 25, 2017 12:26 PM To: Van Der Wijngaart, Rob F <rob.f.van.der.wijngaart(a)intel.com>; Petrini, Fabrizio <fabrizio.petrini(a)intel.com>; Zhang, Chuanjun <chuanjun.zhang(a)intel.com>; Fleming, Kermin <kermin.fleming(a)intel.com>; Huber, Joe <joe.huber(a)intel.com>; Franza, Olivier <olivier.franza(a)intel.com>; Antonellis, Richard <richard.antonellis(a)intel.com>; anl-intel-csa(a)jlse.anl.gov; 'kumaran(a)alcf.anl.gov' <kumaran(a)alcf.anl.gov>; Glossop, Kent <kent.glossop(a)intel.com>; Hawes, FrederickX <frederickx.hawes(a)intel.com> Cc: 'Williams, Timothy J.' <zippy(a)anl.gov>; Moreno, Camilo A <camilo.a.moreno(a)intel.com>; Tannenbaum, Barry M <barry.m.tannenbaum(a)intel.com>; Xiang, Lingxiang <lingxiang.xiang(a)intel.com>; Pamnany, Kiran <kiran.pamnany(a)intel.com>; Matveev, Zakhar A <zakhar.a.matveev(a)intel.com>; 'Jennings, Elise' <ejennings(a)anl.gov>; Creech, Timothy M <timothy.m.creech(a)intel.com>; Gara, Alan <alan.gara(a)intel.com>; 'Jin, Zheming' <zjin(a)anl.gov>; 'Beckman, Pete' <beckman(a)mcs.anl.gov>; 'Yoshii, Kazutomo' <kazutomo(a)mcs.anl.gov>; Khaldi, Dounia <dounia.khaldi(a)intel.com>; Zhang, Yongzhi <yongzhi.zhang(a)intel.com>; 'Bair, Raymond A.' <rbair(a)anl.gov>; Garg, Rajeev K <rajeev.k.garg(a)intel.com>; Raj, Swapna <swapna.raj(a)intel.com>; 'Patel, Pragneshkumar Balvantbhai' <ppatel(a)anl.gov>; Cherukuri, Naveen <naveen.cherukuri(a)intel.com>; Wnuk, Danica <danica.wnuk(a)intel.com>; Narayanasetty, Bhargavi <bhargavi.narayanasetty(a)intel.com>; Hall, Jonathan C <jonathan.c.hall(a)intel.com>; Woodworth, Daniel <daniel.woodworth(a)intel.com>; Mattson, Timothy G <timothy.g.mattson(a)intel.com> Subject: Re: ANL/Intel CSA Workload Characterization On 10/25/2017 01:52 PM, Van Der Wijngaart, Rob F wrote: Certainly, Hal. Zheming was also asking for examples. Let me put together a few slides demonstrating the different variations that I mentioned in my emails. I can present these, or just share with the team. Thanks. I think that just sharing them would be good -- and then we can arrange for a follow-up conversation after we've looked at them. -Hal Rob From: Hal Finkel [mailto:[email protected]] Sent: Wednesday, October 25, 2017 11:01 AM To: Van Der Wijngaart, Rob F <rob.f.van.der.wijngaart(a)intel.com><mailto:[email protected]>; Petrini, Fabrizio <fabrizio.petrini(a)intel.com><mailto:[email protected]>; Zhang, Chuanjun <chuanjun.zhang(a)intel.com><mailto:[email protected]>; Fleming, Kermin <kermin.fleming(a)intel.com><mailto:[email protected]>; Huber, Joe <joe.huber(a)intel.com><mailto:[email protected]>; Franza, Olivier <olivier.franza(a)intel.com><mailto:[email protected]>; Antonellis, Richard <richard.antonellis(a)intel.com><mailto:[email protected]>; anl-intel-csa(a)jlse.anl.gov<mailto:[email protected]>; 'kumaran(a)alcf.anl.gov<mailto:[email protected]>' <kumaran(a)alcf.anl.gov><mailto:[email protected]>; Glossop, Kent <kent.glossop(a)intel.com><mailto:[email protected]>; Hawes, FrederickX <frederickx.hawes(a)intel.com><mailto:[email protected]> Cc: 'Williams, Timothy J.' <zippy(a)anl.gov><mailto:[email protected]>; Moreno, Camilo A <camilo.a.moreno(a)intel.com><mailto:[email protected]>; Tannenbaum, Barry M <barry.m.tannenbaum(a)intel.com><mailto:[email protected]>; Xiang, Lingxiang <lingxiang.xiang(a)intel.com><mailto:[email protected]>; Pamnany, Kiran <kiran.pamnany(a)intel.com><mailto:[email protected]>; Matveev, Zakhar A <zakhar.a.matveev(a)intel.com><mailto:[email protected]>; 'Jennings, Elise' <ejennings(a)anl.gov><mailto:[email protected]>; Creech, Timothy M <timothy.m.creech(a)intel.com><mailto:[email protected]>; Gara, Alan <alan.gara(a)intel.com><mailto:[email protected]>; 'Jin, Zheming' <zjin(a)anl.gov><mailto:[email protected]>; 'Beckman, Pete' <beckman(a)mcs.anl.gov><mailto:[email protected]>; 'Yoshii, Kazutomo' <kazutomo(a)mcs.anl.gov><mailto:[email protected]>; Khaldi, Dounia <dounia.khaldi(a)intel.com><mailto:[email protected]>; Zhang, Yongzhi <yongzhi.zhang(a)intel.com><mailto:[email protected]>; 'Bair, Raymond A.' <rbair(a)anl.gov><mailto:[email protected]>; Garg, Rajeev K <rajeev.k.garg(a)intel.com><mailto:[email protected]>; Raj, Swapna <swapna.raj(a)intel.com><mailto:[email protected]>; 'Patel, Pragneshkumar Balvantbhai' <ppatel(a)anl.gov><mailto:[email protected]>; Cherukuri, Naveen <naveen.cherukuri(a)intel.com><mailto:[email protected]>; Wnuk, Danica <danica.wnuk(a)intel.com><mailto:[email protected]>; Narayanasetty, Bhargavi <bhargavi.narayanasetty(a)intel.com><mailto:[email protected]>; Hall, Jonathan C <jonathan.c.hall(a)intel.com><mailto:[email protected]>; Woodworth, Daniel <daniel.woodworth(a)intel.com><mailto:[email protected]>; Mattson, Timothy G <timothy.g.mattson(a)intel.com><mailto:[email protected]> Subject: Re: ANL/Intel CSA Workload Characterization On 10/19/2017 12:58 PM, Van Der Wijngaart, Rob F wrote: Just a few more notes on the topic: 1. In many application codes stencil operations are broken into multiple steps. It’s rare to have an assignment like: a(j,k) = w1*b(j-1,k) + w2*b(j,k) + w3*b(j+1,k) + w4*b(j,k-1) + w5*b(j,k+1). Usually there will be one or more loop nests in which only offsets in one index occur, followed by nest(s) with offsets in another index. It may not be easy for a compiler to understand that a multi-dimensional stencil is being evaluated. Can you provide an example of what you mean (i.e., what these loop nests look like)? Are there performance reasons that it might be best to have separate loop nests in this way? It sounds inefficient to scan the "b" array above multiple times. When I was in graduate school, and when I was a postdoc, I worked on several applications that used box stencils (etc.), but we always generated them with the help of a computer-algebra system, and so they were generally single loop nests. Maybe writing these "by hand" yields a different way of organizing the code? Are the different loop nest each handling some different aspect of the physics? Thanks again, Hal 1. Box-shaped (i.e. all neighbor points within an L1-distance of RADIUS from the central point are used, as opposed to just those on the legs of a star) stencils are quite common these days. This can, in principle, greatly increase data reuse. However, see item 1 for how these stencils are usually implemented. Rob _____________________________________________ From: Van Der Wijngaart, Rob F Sent: Tuesday, October 17, 2017 12:14 PM To: Petrini, Fabrizio <fabrizio.petrini(a)intel.com><mailto:[email protected]>; Zhang, Chuanjun <chuanjun.zhang(a)intel.com><mailto:[email protected]>; Fleming, Kermin <kermin.fleming(a)intel.com><mailto:[email protected]>; Huber, Joe <joe.huber(a)intel.com><mailto:[email protected]>; Franza, Olivier <olivier.franza(a)intel.com><mailto:[email protected]>; Antonellis, Richard <richard.antonellis(a)intel.com><mailto:[email protected]>; anl-intel-csa(a)jlse.anl.gov<mailto:[email protected]>; 'kumaran(a)alcf.anl.gov<mailto:[email protected]>' <kumaran(a)alcf.anl.gov><mailto:[email protected]>; Glossop, Kent <kent.glossop(a)intel.com><mailto:[email protected]>; Hawes, FrederickX <frederickx.hawes(a)intel.com><mailto:[email protected]> Cc: 'Williams, Timothy J.' <zippy(a)anl.gov><mailto:[email protected]>; 'Finkel, Hal J.' <hfinkel(a)anl.gov><mailto:[email protected]>; Moreno, Camilo A <camilo.a.moreno(a)intel.com><mailto:[email protected]>; Tannenbaum, Barry M <barry.m.tannenbaum(a)intel.com><mailto:[email protected]>; Xiang, Lingxiang <lingxiang.xiang(a)intel.com><mailto:[email protected]>; Pamnany, Kiran <kiran.pamnany(a)intel.com><mailto:[email protected]>; Matveev, Zakhar A <zakhar.a.matveev(a)intel.com><mailto:[email protected]>; 'Jennings, Elise' <ejennings(a)anl.gov><mailto:[email protected]>; Creech, Timothy M <timothy.m.creech(a)intel.com><mailto:[email protected]>; Gara, Alan <alan.gara(a)intel.com><mailto:[email protected]>; 'Jin, Zheming' <zjin(a)anl.gov><mailto:[email protected]>; 'Beckman, Pete' <beckman(a)mcs.anl.gov><mailto:[email protected]>; 'Yoshii, Kazutomo' <kazutomo(a)mcs.anl.gov><mailto:[email protected]>; Khaldi, Dounia <dounia.khaldi(a)intel.com><mailto:[email protected]>; Zhang, Yongzhi <yongzhi.zhang(a)intel.com><mailto:[email protected]>; 'Bair, Raymond A.' <rbair(a)anl.gov><mailto:[email protected]>; Garg, Rajeev K <rajeev.k.garg(a)intel.com><mailto:[email protected]>; Raj, Swapna <swapna.raj(a)intel.com><mailto:[email protected]>; 'Patel, Pragneshkumar Balvantbhai' <ppatel(a)anl.gov><mailto:[email protected]>; Cherukuri, Naveen <naveen.cherukuri(a)intel.com><mailto:[email protected]>; Wnuk, Danica <danica.wnuk(a)intel.com><mailto:[email protected]>; Narayanasetty, Bhargavi <bhargavi.narayanasetty(a)intel.com><mailto:[email protected]>; Hall, Jonathan C <jonathan.c.hall(a)intel.com><mailto:[email protected]>; Woodworth, Daniel <daniel.woodworth(a)intel.com><mailto:[email protected]>; Mattson, Timothy G <timothy.g.mattson(a)intel.com><mailto:[email protected]> Subject: RE: ANL/Intel CSA Workload Characterization Hello All, During last Friday’s meeting I asked a question about the implications of encountering real-life conditions in stencil-based codes on porting to CSA. I wanted to give some explanation here. I apologize in advance to the numericists in this audience, especially those with a background in CFD, for whom most of my remarks will be trivial. 1. In many instances non-trivial boundary conditions give rise to complications. Dirichlet conditions specify that values on the boundary of the grid are known and should not be touched. These are easy and do not destroy the regular structure of the stencil. However, a simple outflow condition such as dvx/dx=0 (where vx is the velocity vector in the x-direction; if the boundary of the grid is not aligned with one of the coordinate directions, even such a simple condition becomes quite complicated to implement) would destroy the regularity of the stencil; the weights and extent of the stencil near the grid boundary are completely different from those in the interior of the grid. 2. Larger stencils that accommodate higher-order methods (these have become more popular over the years), or that implement higher derivatives are often modified near grid boundaries to avoid losing accuracy or consistency. Look for example at the way artificial viscosity is represented near boundaries in the NAS Parallel Benchmarks BT, SP, and LU (computation of array rhs). These modifications again destroy the regularity of the stencil. 3. In many important applications body-fitted grids are used that employ curvilinear coordinates. While the stencils used may be topologically fixed (same local neighborhood of grid points is used to form discretized derivatives), the weights differ from point to point. 4. In non-linear problems (the rule in CFD, rather than the exception), the stencil weights differ from point to point, as well as from one iteration to the next. My question was how much such realistic complicating factors would affect the various optimizations presented during the talk. They certainly affect efficiency of codes mapped to CPUs, since they increase memory bandwidth pressure, may reduce vectorization, may prevent loop collapse, etc. With CSA we have the added complexity that these complications could dramatically increase the size of the graph, which would affect the level to which loops can be unrolled. I cannot oversee all the implications, and am in general upbeat about the potential for success of CSA, but we need to be careful not to translate superior performance of a canonical stencil kernel into application success. If anybody would like to see pictures and/or code illustrating my points, I’ll be happy to put some slides together and distribute. Rob -----Original Appointment----- From: Petrini, Fabrizio Sent: Tuesday, July 25, 2017 7:36 PM To: Petrini, Fabrizio; Van Der Wijngaart, Rob F; Zhang, Chuanjun; Fleming, Kermin; Huber, Joe; Franza, Olivier; Antonellis, Richard; anl-intel-csa(a)jlse.anl.gov<mailto:[email protected]>; 'kumaran(a)alcf.anl.gov<mailto:[email protected]>'; Glossop, Kent; Hawes, FrederickX Cc: 'Williams, Timothy J.'; 'Finkel, Hal J.'; Moreno, Camilo A; Tannenbaum, Barry M; Xiang, Lingxiang; Pamnany, Kiran; Matveev, Zakhar A; 'Jennings, Elise'; Wilson, Amalee; Creech, Timothy M; Gara, Alan; 'Jin, Zheming'; 'Beckman, Pete'; 'Yoshii, Kazutomo'; Khaldi, Dounia; Zhang, Yongzhi; 'Bair, Raymond A.'; Garg, Rajeev K; Raj, Swapna; 'Patel, Pragneshkumar Balvantbhai'; Cherukuri, Naveen; Wnuk, Danica; Narayanasetty, Bhargavi; Hall, Jonathan C; Woodworth, Daniel Subject: FW: ANL/Intel CSA Workload Characterization When: Friday, October 13, 2017 11:00 AM-12:00 PM (UTC-08:00) Pacific Time (US & Canada). Where: Skype Meeting -----Original Appointment----- From: Petrini, Fabrizio Sent: Monday, March 27, 2017 10:29 AM To: Petrini, Fabrizio; Fleming, Kermin; Huber, Joe; Franza, Olivier; Antonellis, Richard; anl-intel-csa(a)jlse.anl.gov<mailto:[email protected]>; 'kumaran(a)alcf.anl.gov<mailto:[email protected]>'; Glossop, Kent; Hawes, FrederickX; Tithi, Jesmin Jahan; Steely Jr, Simon C Cc: 'Williams, Timothy J.'; 'Finkel, Hal J.'; Moreno, Camilo A; Tannenbaum, Barry M; Xiang, Lingxiang; Pamnany, Kiran; Matveev, Zakhar A; 'Jennings, Elise'; Wilson, Amalee; Creech, Timothy M; Gara, Alan; 'Jin, Zheming'; 'Beckman, Pete'; 'Yoshii, Kazutomo'; Khaldi, Dounia; Zhang, Yongzhi; Zhang, Chuanjun; 'Bair, Raymond A.'; Garg, Rajeev K; Raj, Swapna; 'Patel, Pragneshkumar Balvantbhai'; Cherukuri, Naveen; Wnuk, Danica; Narayanasetty, Bhargavi; Hall, Jonathan C; Woodworth, Daniel; 'Stevens, Rick L.'; Khudia, Daya S; Halpern, Pablo G; Hammond, Jeff R; Ebeling, Carl; Liu, Xing1; Dhamankar, Nitin S; 'Romero, Nichols A.'; 'Alvaro'; Lyons, Gail; Magro, Bill Subject: ANL/Intel CSA Workload Characterization When: Occurs every Friday effective 3/31/2017 until 4/6/2018 from 11:00 AM to 12:00 PM (UTC-08:00) Pacific Time (US & Canada). Where: Skype Meeting Placeholder for weekly ANL/Intel CSA Exascale workload characterization meeting ......................................................................................................................................... --> Join Skype Meeting<https://meet.intel.com/fabrizio.petrini/7K7FM32F> Trouble Joining? Try Skype Web App<https://meet.intel.com/fabrizio.petrini/7K7FM32F?sl=1> Join by phone +1(916)356-2663 (or your local bridge access #) Choose bridge 5.<tel:+1%28916%29356-2663%20%28or%20your%20local%20bridge%20access%20#%29%20Choose%20bridge%205.> (Global) English (United States) Find a local number<https://dial.intel.com> Conference ID: 791574018 Forgot your dial-in PIN?<https://dial.intel.com> |Help<https://o15.officeredir.microsoft.com/r/rlidLync15?clid=1033&p1=5&p2=2009> [!OC([1033])!] ......................................................................................................................................... Skype users: Recordings are subject to the Audio/Video Recording Policy<https://iref.intel.com/GetDoc.aspx?RefLibObjectID=0902007c8003c7b5&Latest=T…>. -- Hal Finkel Lead, Compiler Technology and Programming Languages Leadership Computing Facility Argonne National Laboratory -- Hal Finkel Lead, Compiler Technology and Programming Languages Leadership Computing Facility Argonne National Laboratory

1 0

Re: [Anl-intel-csa] ANL/Intel CSA Workload Characterization
by Hal Finkel 25 Oct '17

25 Oct '17

On 10/25/2017 01:52 PM, Van Der Wijngaart, Rob F wrote: > > Certainly, Hal. > > Zheming was also asking for examples. Let me put together a few slides > demonstrating the different variations that I mentioned in my emails. > I can present these, or just share with the team. > Thanks. I think that just sharing them would be good -- and then we can arrange for a follow-up conversation after we've looked at them. -Hal > Rob > > *From:*Hal Finkel [mailto:[email protected]] > *Sent:* Wednesday, October 25, 2017 11:01 AM > *To:* Van Der Wijngaart, Rob F <rob.f.van.der.wijngaart(a)intel.com>; > Petrini, Fabrizio <fabrizio.petrini(a)intel.com>; Zhang, Chuanjun > <chuanjun.zhang(a)intel.com>; Fleming, Kermin > <kermin.fleming(a)intel.com>; Huber, Joe <joe.huber(a)intel.com>; Franza, > Olivier <olivier.franza(a)intel.com>; Antonellis, Richard > <richard.antonellis(a)intel.com>; anl-intel-csa(a)jlse.anl.gov; > 'kumaran(a)alcf.anl.gov' <kumaran(a)alcf.anl.gov>; Glossop, Kent > <kent.glossop(a)intel.com>; Hawes, FrederickX <frederickx.hawes(a)intel.com> > *Cc:* 'Williams, Timothy J.' <zippy(a)anl.gov>; Moreno, Camilo A > <camilo.a.moreno(a)intel.com>; Tannenbaum, Barry M > <barry.m.tannenbaum(a)intel.com>; Xiang, Lingxiang > <lingxiang.xiang(a)intel.com>; Pamnany, Kiran <kiran.pamnany(a)intel.com>; > Matveev, Zakhar A <zakhar.a.matveev(a)intel.com>; 'Jennings, Elise' > <ejennings(a)anl.gov>; Creech, Timothy M <timothy.m.creech(a)intel.com>; > Gara, Alan <alan.gara(a)intel.com>; 'Jin, Zheming' <zjin(a)anl.gov>; > 'Beckman, Pete' <beckman(a)mcs.anl.gov>; 'Yoshii, Kazutomo' > <kazutomo(a)mcs.anl.gov>; Khaldi, Dounia <dounia.khaldi(a)intel.com>; > Zhang, Yongzhi <yongzhi.zhang(a)intel.com>; 'Bair, Raymond A.' > <rbair(a)anl.gov>; Garg, Rajeev K <rajeev.k.garg(a)intel.com>; Raj, Swapna > <swapna.raj(a)intel.com>; 'Patel, Pragneshkumar Balvantbhai' > <ppatel(a)anl.gov>; Cherukuri, Naveen <naveen.cherukuri(a)intel.com>; > Wnuk, Danica <danica.wnuk(a)intel.com>; Narayanasetty, Bhargavi > <bhargavi.narayanasetty(a)intel.com>; Hall, Jonathan C > <jonathan.c.hall(a)intel.com>; Woodworth, Daniel > <daniel.woodworth(a)intel.com>; Mattson, Timothy G > <timothy.g.mattson(a)intel.com> > *Subject:* Re: ANL/Intel CSA Workload Characterization > > On 10/19/2017 12:58 PM, Van Der Wijngaart, Rob F wrote: > > Just a few more notes on the topic: > > 1.In many application codes stencil operations are broken into > multiple steps. It’s rare to have an assignment like: a(j,k) = > w1*b(j-1,k) + w2*b(j,k) + w3*b(j+1,k) + w4*b(j,k-1) + w5*b(j,k+1). > Usually there will be one or more loop nests in which only offsets > in one index occur, followed by nest(s) with offsets in another > index. It may not be easy for a compiler to understand that a > multi-dimensional stencil is being evaluated. > > > Can you provide an example of what you mean (i.e., what these loop > nests look like)? Are there performance reasons that it might be best > to have separate loop nests in this way? It sounds inefficient to scan > the "b" array above multiple times. > > When I was in graduate school, and when I was a postdoc, I worked on > several applications that used box stencils (etc.), but we always > generated them with the help of a computer-algebra system, and so they > were generally single loop nests. Maybe writing these "by hand" yields > a different way of organizing the code? Are the different loop nest > each handling some different aspect of the physics? > > Thanks again, > Hal > > > 1.Box-shaped (i.e. all neighbor points within an L1-distance of > RADIUS from the central point are used, as opposed to just those > on the legs of a star) stencils are quite common these days. This > can, in principle, greatly increase data reuse. However, see item > 1 for how these stencils are usually implemented. > > Rob > > _____________________________________________ > *From:* Van Der Wijngaart, Rob F > *Sent:* Tuesday, October 17, 2017 12:14 PM > *To:* Petrini, Fabrizio <fabrizio.petrini(a)intel.com> > <mailto:[email protected]>; Zhang, Chuanjun > <chuanjun.zhang(a)intel.com> <mailto:[email protected]>; > Fleming, Kermin <kermin.fleming(a)intel.com> > <mailto:[email protected]>; Huber, Joe > <joe.huber(a)intel.com> <mailto:[email protected]>; Franza, > Olivier <olivier.franza(a)intel.com> > <mailto:[email protected]>; Antonellis, Richard > <richard.antonellis(a)intel.com> > <mailto:[email protected]>; anl-intel-csa(a)jlse.anl.gov > <mailto:[email protected]>; 'kumaran(a)alcf.anl.gov > <mailto:[email protected]>' <kumaran(a)alcf.anl.gov> > <mailto:[email protected]>; Glossop, Kent > <kent.glossop(a)intel.com> <mailto:[email protected]>; Hawes, > FrederickX <frederickx.hawes(a)intel.com> > <mailto:[email protected]> > *Cc:* 'Williams, Timothy J.' <zippy(a)anl.gov> > <mailto:[email protected]>; 'Finkel, Hal J.' <hfinkel(a)anl.gov> > <mailto:[email protected]>; Moreno, Camilo A > <camilo.a.moreno(a)intel.com> <mailto:[email protected]>; > Tannenbaum, Barry M <barry.m.tannenbaum(a)intel.com> > <mailto:[email protected]>; Xiang, Lingxiang > <lingxiang.xiang(a)intel.com> <mailto:[email protected]>; > Pamnany, Kiran <kiran.pamnany(a)intel.com> > <mailto:[email protected]>; Matveev, Zakhar A > <zakhar.a.matveev(a)intel.com> <mailto:[email protected]>; > 'Jennings, Elise' <ejennings(a)anl.gov> <mailto:[email protected]>; > Creech, Timothy M <timothy.m.creech(a)intel.com> > <mailto:[email protected]>; Gara, Alan > <alan.gara(a)intel.com> <mailto:[email protected]>; 'Jin, Zheming' > <zjin(a)anl.gov> <mailto:[email protected]>; 'Beckman, Pete' > <beckman(a)mcs.anl.gov> <mailto:[email protected]>; 'Yoshii, > Kazutomo' <kazutomo(a)mcs.anl.gov> <mailto:[email protected]>; > Khaldi, Dounia <dounia.khaldi(a)intel.com> > <mailto:[email protected]>; Zhang, Yongzhi > <yongzhi.zhang(a)intel.com> <mailto:[email protected]>; 'Bair, > Raymond A.' <rbair(a)anl.gov> <mailto:[email protected]>; Garg, Rajeev K > <rajeev.k.garg(a)intel.com> <mailto:[email protected]>; Raj, > Swapna <swapna.raj(a)intel.com> <mailto:[email protected]>; > 'Patel, Pragneshkumar Balvantbhai' <ppatel(a)anl.gov> > <mailto:[email protected]>; Cherukuri, Naveen > <naveen.cherukuri(a)intel.com> <mailto:[email protected]>; > Wnuk, Danica <danica.wnuk(a)intel.com> > <mailto:[email protected]>; Narayanasetty, Bhargavi > <bhargavi.narayanasetty(a)intel.com> > <mailto:[email protected]>; Hall, Jonathan C > <jonathan.c.hall(a)intel.com> <mailto:[email protected]>; > Woodworth, Daniel <daniel.woodworth(a)intel.com> > <mailto:[email protected]>; Mattson, Timothy G > <timothy.g.mattson(a)intel.com> <mailto:[email protected]> > *Subject:* RE: ANL/Intel CSA Workload Characterization > > Hello All, > > During last Friday’s meeting I asked a question about the > implications of encountering real-life conditions in stencil-based > codes on porting to CSA. I wanted to give some explanation here. I > apologize in advance to the numericists in this audience, > especially those with a background in CFD, for whom most of my > remarks will be trivial. > > 1.In many instances non-trivial boundary conditions give rise to > complications. Dirichlet conditions specify that values on the > boundary of the grid are known and should not be touched. These > are easy and do not destroy the regular structure of the stencil. > However, a simple outflow condition such as dv_x /dx=0 (where v_x > is the velocity vector in the x-direction; if the boundary of the > grid is not aligned with one of the coordinate directions, even > such a simple condition becomes quite complicated to implement) > would destroy the regularity of the stencil; the weights and > extent of the stencil near the grid boundary are completely > different from those in the interior of the grid. > > 2.Larger stencils that accommodate higher-order methods (these > have become more popular over the years), or that implement higher > derivatives are often modified near grid boundaries to avoid > losing accuracy or consistency. Look for example at the way > artificial viscosity is represented near boundaries in the NAS > Parallel Benchmarks BT, SP, and LU (computation of array rhs). > These modifications again destroy the regularity of the stencil. > > 3.In many important applications body-fitted grids are used that > employ curvilinear coordinates. While the stencils used may be > topologically fixed (same local neighborhood of grid points is > used to form discretized derivatives), the weights differ from > point to point. > > 4.In non-linear problems (the rule in CFD, rather than the > exception), the stencil weights differ from point to point, as > well as from one iteration to the next. > > My question was how much such realistic complicating factors would > affect the various optimizations presented during the talk. They > certainly affect efficiency of codes mapped to CPUs, since they > increase memory bandwidth pressure, may reduce vectorization, may > prevent loop collapse, etc. With CSA we have the added complexity > that these complications could dramatically increase the size of > the graph, which would affect the level to which loops can be > unrolled. > > I cannot oversee all the implications, and am in general upbeat > about the potential for success of CSA, but we need to be careful > not to translate superior performance of a canonical stencil > kernel into application success. > > If anybody would like to see pictures and/or code illustrating my > points, I’ll be happy to put some slides together and distribute. > > Rob > > -----Original Appointment----- > *From:* Petrini, Fabrizio > *Sent:* Tuesday, July 25, 2017 7:36 PM > *To:* Petrini, Fabrizio; Van Der Wijngaart, Rob F; Zhang, > Chuanjun; Fleming, Kermin; Huber, Joe; Franza, Olivier; > Antonellis, Richard; anl-intel-csa(a)jlse.anl.gov > <mailto:[email protected]>; 'kumaran(a)alcf.anl.gov > <mailto:[email protected]>'; Glossop, Kent; Hawes, FrederickX > *Cc:* 'Williams, Timothy J.'; 'Finkel, Hal J.'; Moreno, Camilo A; > Tannenbaum, Barry M; Xiang, Lingxiang; Pamnany, Kiran; Matveev, > Zakhar A; 'Jennings, Elise'; Wilson, Amalee; Creech, Timothy M; > Gara, Alan; 'Jin, Zheming'; 'Beckman, Pete'; 'Yoshii, Kazutomo'; > Khaldi, Dounia; Zhang, Yongzhi; 'Bair, Raymond A.'; Garg, Rajeev > K; Raj, Swapna; 'Patel, Pragneshkumar Balvantbhai'; Cherukuri, > Naveen; Wnuk, Danica; Narayanasetty, Bhargavi; Hall, Jonathan C; > Woodworth, Daniel > *Subject:* FW: ANL/Intel CSA Workload Characterization > *When:* Friday, October 13, 2017 11:00 AM-12:00 PM (UTC-08:00) > Pacific Time (US & Canada). > *Where:* Skype Meeting > > -----Original Appointment----- > *From:* Petrini, Fabrizio > *Sent:* Monday, March 27, 2017 10:29 AM > *To:* Petrini, Fabrizio; Fleming, Kermin; Huber, Joe; Franza, > Olivier; Antonellis, Richard; anl-intel-csa(a)jlse.anl.gov > <mailto:[email protected]>; 'kumaran(a)alcf.anl.gov > <mailto:[email protected]>'; Glossop, Kent; Hawes, FrederickX; > Tithi, Jesmin Jahan; Steely Jr, Simon C > *Cc:* 'Williams, Timothy J.'; 'Finkel, Hal J.'; Moreno, Camilo A; > Tannenbaum, Barry M; Xiang, Lingxiang; Pamnany, Kiran; Matveev, > Zakhar A; 'Jennings, Elise'; Wilson, Amalee; Creech, Timothy M; > Gara, Alan; 'Jin, Zheming'; 'Beckman, Pete'; 'Yoshii, Kazutomo'; > Khaldi, Dounia; Zhang, Yongzhi; Zhang, Chuanjun; 'Bair, Raymond > A.'; Garg, Rajeev K; Raj, Swapna; 'Patel, Pragneshkumar > Balvantbhai'; Cherukuri, Naveen; Wnuk, Danica; Narayanasetty, > Bhargavi; Hall, Jonathan C; Woodworth, Daniel; 'Stevens, Rick L.'; > Khudia, Daya S; Halpern, Pablo G; Hammond, Jeff R; Ebeling, Carl; > Liu, Xing1; Dhamankar, Nitin S; 'Romero, Nichols A.'; 'Alvaro'; > Lyons, Gail; Magro, Bill > *Subject:* ANL/Intel CSA Workload Characterization > *When:* Occurs every Friday effective 3/31/2017 until 4/6/2018 > from 11:00 AM to 12:00 PM (UTC-08:00) Pacific Time (US & Canada). > *Where:* Skype Meeting > > Placeholder for weekly ANL/Intel CSA Exascale workload > characterization meeting > > ......................................................................................................................................... > > àJoin Skype Meeting <https://meet.intel.com/fabrizio.petrini/7K7FM32F> > > Trouble Joining? Try Skype Web App > <https://meet.intel.com/fabrizio.petrini/7K7FM32F?sl=1>__ > > Join by phone > > +1(916)356-2663 (or your local bridge access #) Choose bridge 5. > <tel:+1%28916%29356-2663%20%28or%20your%20local%20bridge%20access%20#%29%20Choose%20bridge%205.>(Global) > English (United States) > > Find a local number <https://dial.intel.com> > > Conference ID: 791574018 > > Forgot your dial-in PIN? <https://dial.intel.com> |Help > <https://o15.officeredir.microsoft.com/r/rlidLync15?clid=1033&p1=5&p2=2009> > > > _[!OC([1033])!] > > ......................................................................................................................................... > > Skype users: Recordings are subject to the Audio/Video Recording > Policy > <https://iref.intel.com/GetDoc.aspx?RefLibObjectID=0902007c8003c7b5&Latest=T…>. > > > > > -- > Hal Finkel > Lead, Compiler Technology and Programming Languages > Leadership Computing Facility > Argonne National Laboratory -- Hal Finkel Lead, Compiler Technology and Programming Languages Leadership Computing Facility Argonne National Laboratory

1 0

Re: [Anl-intel-csa] ANL/Intel CSA Workload Characterization
by Van Der Wijngaart, Rob F 25 Oct '17

25 Oct '17

Certainly, Hal. Zheming was also asking for examples. Let me put together a few slides demonstrating the different variations that I mentioned in my emails. I can present these, or just share with the team. Rob From: Hal Finkel [mailto:[email protected]] Sent: Wednesday, October 25, 2017 11:01 AM To: Van Der Wijngaart, Rob F <rob.f.van.der.wijngaart(a)intel.com>; Petrini, Fabrizio <fabrizio.petrini(a)intel.com>; Zhang, Chuanjun <chuanjun.zhang(a)intel.com>; Fleming, Kermin <kermin.fleming(a)intel.com>; Huber, Joe <joe.huber(a)intel.com>; Franza, Olivier <olivier.franza(a)intel.com>; Antonellis, Richard <richard.antonellis(a)intel.com>; anl-intel-csa(a)jlse.anl.gov; 'kumaran(a)alcf.anl.gov' <kumaran(a)alcf.anl.gov>; Glossop, Kent <kent.glossop(a)intel.com>; Hawes, FrederickX <frederickx.hawes(a)intel.com> Cc: 'Williams, Timothy J.' <zippy(a)anl.gov>; Moreno, Camilo A <camilo.a.moreno(a)intel.com>; Tannenbaum, Barry M <barry.m.tannenbaum(a)intel.com>; Xiang, Lingxiang <lingxiang.xiang(a)intel.com>; Pamnany, Kiran <kiran.pamnany(a)intel.com>; Matveev, Zakhar A <zakhar.a.matveev(a)intel.com>; 'Jennings, Elise' <ejennings(a)anl.gov>; Creech, Timothy M <timothy.m.creech(a)intel.com>; Gara, Alan <alan.gara(a)intel.com>; 'Jin, Zheming' <zjin(a)anl.gov>; 'Beckman, Pete' <beckman(a)mcs.anl.gov>; 'Yoshii, Kazutomo' <kazutomo(a)mcs.anl.gov>; Khaldi, Dounia <dounia.khaldi(a)intel.com>; Zhang, Yongzhi <yongzhi.zhang(a)intel.com>; 'Bair, Raymond A.' <rbair(a)anl.gov>; Garg, Rajeev K <rajeev.k.garg(a)intel.com>; Raj, Swapna <swapna.raj(a)intel.com>; 'Patel, Pragneshkumar Balvantbhai' <ppatel(a)anl.gov>; Cherukuri, Naveen <naveen.cherukuri(a)intel.com>; Wnuk, Danica <danica.wnuk(a)intel.com>; Narayanasetty, Bhargavi <bhargavi.narayanasetty(a)intel.com>; Hall, Jonathan C <jonathan.c.hall(a)intel.com>; Woodworth, Daniel <daniel.woodworth(a)intel.com>; Mattson, Timothy G <timothy.g.mattson(a)intel.com> Subject: Re: ANL/Intel CSA Workload Characterization On 10/19/2017 12:58 PM, Van Der Wijngaart, Rob F wrote: Just a few more notes on the topic: 1. In many application codes stencil operations are broken into multiple steps. It’s rare to have an assignment like: a(j,k) = w1*b(j-1,k) + w2*b(j,k) + w3*b(j+1,k) + w4*b(j,k-1) + w5*b(j,k+1). Usually there will be one or more loop nests in which only offsets in one index occur, followed by nest(s) with offsets in another index. It may not be easy for a compiler to understand that a multi-dimensional stencil is being evaluated. Can you provide an example of what you mean (i.e., what these loop nests look like)? Are there performance reasons that it might be best to have separate loop nests in this way? It sounds inefficient to scan the "b" array above multiple times. When I was in graduate school, and when I was a postdoc, I worked on several applications that used box stencils (etc.), but we always generated them with the help of a computer-algebra system, and so they were generally single loop nests. Maybe writing these "by hand" yields a different way of organizing the code? Are the different loop nest each handling some different aspect of the physics? Thanks again, Hal 1. Box-shaped (i.e. all neighbor points within an L1-distance of RADIUS from the central point are used, as opposed to just those on the legs of a star) stencils are quite common these days. This can, in principle, greatly increase data reuse. However, see item 1 for how these stencils are usually implemented. Rob _____________________________________________ From: Van Der Wijngaart, Rob F Sent: Tuesday, October 17, 2017 12:14 PM To: Petrini, Fabrizio <fabrizio.petrini(a)intel.com><mailto:[email protected]>; Zhang, Chuanjun <chuanjun.zhang(a)intel.com><mailto:[email protected]>; Fleming, Kermin <kermin.fleming(a)intel.com><mailto:[email protected]>; Huber, Joe <joe.huber(a)intel.com><mailto:[email protected]>; Franza, Olivier <olivier.franza(a)intel.com><mailto:[email protected]>; Antonellis, Richard <richard.antonellis(a)intel.com><mailto:[email protected]>; anl-intel-csa(a)jlse.anl.gov<mailto:[email protected]>; 'kumaran(a)alcf.anl.gov<mailto:[email protected]>' <kumaran(a)alcf.anl.gov><mailto:[email protected]>; Glossop, Kent <kent.glossop(a)intel.com><mailto:[email protected]>; Hawes, FrederickX <frederickx.hawes(a)intel.com><mailto:[email protected]> Cc: 'Williams, Timothy J.' <zippy(a)anl.gov><mailto:[email protected]>; 'Finkel, Hal J.' <hfinkel(a)anl.gov><mailto:[email protected]>; Moreno, Camilo A <camilo.a.moreno(a)intel.com><mailto:[email protected]>; Tannenbaum, Barry M <barry.m.tannenbaum(a)intel.com><mailto:[email protected]>; Xiang, Lingxiang <lingxiang.xiang(a)intel.com><mailto:[email protected]>; Pamnany, Kiran <kiran.pamnany(a)intel.com><mailto:[email protected]>; Matveev, Zakhar A <zakhar.a.matveev(a)intel.com><mailto:[email protected]>; 'Jennings, Elise' <ejennings(a)anl.gov><mailto:[email protected]>; Creech, Timothy M <timothy.m.creech(a)intel.com><mailto:[email protected]>; Gara, Alan <alan.gara(a)intel.com><mailto:[email protected]>; 'Jin, Zheming' <zjin(a)anl.gov><mailto:[email protected]>; 'Beckman, Pete' <beckman(a)mcs.anl.gov><mailto:[email protected]>; 'Yoshii, Kazutomo' <kazutomo(a)mcs.anl.gov><mailto:[email protected]>; Khaldi, Dounia <dounia.khaldi(a)intel.com><mailto:[email protected]>; Zhang, Yongzhi <yongzhi.zhang(a)intel.com><mailto:[email protected]>; 'Bair, Raymond A.' <rbair(a)anl.gov><mailto:[email protected]>; Garg, Rajeev K <rajeev.k.garg(a)intel.com><mailto:[email protected]>; Raj, Swapna <swapna.raj(a)intel.com><mailto:[email protected]>; 'Patel, Pragneshkumar Balvantbhai' <ppatel(a)anl.gov><mailto:[email protected]>; Cherukuri, Naveen <naveen.cherukuri(a)intel.com><mailto:[email protected]>; Wnuk, Danica <danica.wnuk(a)intel.com><mailto:[email protected]>; Narayanasetty, Bhargavi <bhargavi.narayanasetty(a)intel.com><mailto:[email protected]>; Hall, Jonathan C <jonathan.c.hall(a)intel.com><mailto:[email protected]>; Woodworth, Daniel <daniel.woodworth(a)intel.com><mailto:[email protected]>; Mattson, Timothy G <timothy.g.mattson(a)intel.com><mailto:[email protected]> Subject: RE: ANL/Intel CSA Workload Characterization Hello All, During last Friday’s meeting I asked a question about the implications of encountering real-life conditions in stencil-based codes on porting to CSA. I wanted to give some explanation here. I apologize in advance to the numericists in this audience, especially those with a background in CFD, for whom most of my remarks will be trivial. 1. In many instances non-trivial boundary conditions give rise to complications. Dirichlet conditions specify that values on the boundary of the grid are known and should not be touched. These are easy and do not destroy the regular structure of the stencil. However, a simple outflow condition such as dvx/dx=0 (where vx is the velocity vector in the x-direction; if the boundary of the grid is not aligned with one of the coordinate directions, even such a simple condition becomes quite complicated to implement) would destroy the regularity of the stencil; the weights and extent of the stencil near the grid boundary are completely different from those in the interior of the grid. 2. Larger stencils that accommodate higher-order methods (these have become more popular over the years), or that implement higher derivatives are often modified near grid boundaries to avoid losing accuracy or consistency. Look for example at the way artificial viscosity is represented near boundaries in the NAS Parallel Benchmarks BT, SP, and LU (computation of array rhs). These modifications again destroy the regularity of the stencil. 3. In many important applications body-fitted grids are used that employ curvilinear coordinates. While the stencils used may be topologically fixed (same local neighborhood of grid points is used to form discretized derivatives), the weights differ from point to point. 4. In non-linear problems (the rule in CFD, rather than the exception), the stencil weights differ from point to point, as well as from one iteration to the next. My question was how much such realistic complicating factors would affect the various optimizations presented during the talk. They certainly affect efficiency of codes mapped to CPUs, since they increase memory bandwidth pressure, may reduce vectorization, may prevent loop collapse, etc. With CSA we have the added complexity that these complications could dramatically increase the size of the graph, which would affect the level to which loops can be unrolled. I cannot oversee all the implications, and am in general upbeat about the potential for success of CSA, but we need to be careful not to translate superior performance of a canonical stencil kernel into application success. If anybody would like to see pictures and/or code illustrating my points, I’ll be happy to put some slides together and distribute. Rob -----Original Appointment----- From: Petrini, Fabrizio Sent: Tuesday, July 25, 2017 7:36 PM To: Petrini, Fabrizio; Van Der Wijngaart, Rob F; Zhang, Chuanjun; Fleming, Kermin; Huber, Joe; Franza, Olivier; Antonellis, Richard; anl-intel-csa(a)jlse.anl.gov<mailto:[email protected]>; 'kumaran(a)alcf.anl.gov<mailto:[email protected]>'; Glossop, Kent; Hawes, FrederickX Cc: 'Williams, Timothy J.'; 'Finkel, Hal J.'; Moreno, Camilo A; Tannenbaum, Barry M; Xiang, Lingxiang; Pamnany, Kiran; Matveev, Zakhar A; 'Jennings, Elise'; Wilson, Amalee; Creech, Timothy M; Gara, Alan; 'Jin, Zheming'; 'Beckman, Pete'; 'Yoshii, Kazutomo'; Khaldi, Dounia; Zhang, Yongzhi; 'Bair, Raymond A.'; Garg, Rajeev K; Raj, Swapna; 'Patel, Pragneshkumar Balvantbhai'; Cherukuri, Naveen; Wnuk, Danica; Narayanasetty, Bhargavi; Hall, Jonathan C; Woodworth, Daniel Subject: FW: ANL/Intel CSA Workload Characterization When: Friday, October 13, 2017 11:00 AM-12:00 PM (UTC-08:00) Pacific Time (US & Canada). Where: Skype Meeting -----Original Appointment----- From: Petrini, Fabrizio Sent: Monday, March 27, 2017 10:29 AM To: Petrini, Fabrizio; Fleming, Kermin; Huber, Joe; Franza, Olivier; Antonellis, Richard; anl-intel-csa(a)jlse.anl.gov<mailto:[email protected]>; 'kumaran(a)alcf.anl.gov<mailto:[email protected]>'; Glossop, Kent; Hawes, FrederickX; Tithi, Jesmin Jahan; Steely Jr, Simon C Cc: 'Williams, Timothy J.'; 'Finkel, Hal J.'; Moreno, Camilo A; Tannenbaum, Barry M; Xiang, Lingxiang; Pamnany, Kiran; Matveev, Zakhar A; 'Jennings, Elise'; Wilson, Amalee; Creech, Timothy M; Gara, Alan; 'Jin, Zheming'; 'Beckman, Pete'; 'Yoshii, Kazutomo'; Khaldi, Dounia; Zhang, Yongzhi; Zhang, Chuanjun; 'Bair, Raymond A.'; Garg, Rajeev K; Raj, Swapna; 'Patel, Pragneshkumar Balvantbhai'; Cherukuri, Naveen; Wnuk, Danica; Narayanasetty, Bhargavi; Hall, Jonathan C; Woodworth, Daniel; 'Stevens, Rick L.'; Khudia, Daya S; Halpern, Pablo G; Hammond, Jeff R; Ebeling, Carl; Liu, Xing1; Dhamankar, Nitin S; 'Romero, Nichols A.'; 'Alvaro'; Lyons, Gail; Magro, Bill Subject: ANL/Intel CSA Workload Characterization When: Occurs every Friday effective 3/31/2017 until 4/6/2018 from 11:00 AM to 12:00 PM (UTC-08:00) Pacific Time (US & Canada). Where: Skype Meeting Placeholder for weekly ANL/Intel CSA Exascale workload characterization meeting ......................................................................................................................................... --> Join Skype Meeting<https://meet.intel.com/fabrizio.petrini/7K7FM32F> Trouble Joining? Try Skype Web App<https://meet.intel.com/fabrizio.petrini/7K7FM32F?sl=1> Join by phone +1(916)356-2663 (or your local bridge access #) Choose bridge 5.<tel:+1%28916%29356-2663%20%28or%20your%20local%20bridge%20access%20#%29%20Choose%20bridge%205.> (Global) English (United States) Find a local number<https://dial.intel.com> Conference ID: 791574018 Forgot your dial-in PIN?<https://dial.intel.com> |Help<https://o15.officeredir.microsoft.com/r/rlidLync15?clid=1033&p1=5&p2=2009> [!OC([1033])!] ......................................................................................................................................... Skype users: Recordings are subject to the Audio/Video Recording Policy<https://iref.intel.com/GetDoc.aspx?RefLibObjectID=0902007c8003c7b5&Latest=T…>. -- Hal Finkel Lead, Compiler Technology and Programming Languages Leadership Computing Facility Argonne National Laboratory

1 0

Re: [Anl-intel-csa] ANL/Intel CSA Workload Characterization
by Hal Finkel 25 Oct '17

25 Oct '17

On 10/19/2017 12:58 PM, Van Der Wijngaart, Rob F wrote: > Just a few more notes on the topic: > > 1. In many application codes stencil operations are broken into > multiple steps. It’s rare to have an assignment like: a(j,k) = > w1*b(j-1,k) + w2*b(j,k) + w3*b(j+1,k) + w4*b(j,k-1) + w5*b(j,k+1). > Usually there will be one or more loop nests in which only offsets > in one index occur, followed by nest(s) with offsets in another > index. It may not be easy for a compiler to understand that a > multi-dimensional stencil is being evaluated. > Can you provide an example of what you mean (i.e., what these loop nests look like)? Are there performance reasons that it might be best to have separate loop nests in this way? It sounds inefficient to scan the "b" array above multiple times. When I was in graduate school, and when I was a postdoc, I worked on several applications that used box stencils (etc.), but we always generated them with the help of a computer-algebra system, and so they were generally single loop nests. Maybe writing these "by hand" yields a different way of organizing the code? Are the different loop nest each handling some different aspect of the physics? Thanks again, Hal > 1. Box-shaped (i.e. all neighbor points within an L1-distance of > RADIUS from the central point are used, as opposed to just those > on the legs of a star) stencils are quite common these days. This > can, in principle, greatly increase data reuse. However, see item > 1 for how these stencils are usually implemented. > > Rob > _____________________________________________ > *From:* Van Der Wijngaart, Rob F > *Sent:* Tuesday, October 17, 2017 12:14 PM > *To:* Petrini, Fabrizio <fabrizio.petrini(a)intel.com>; Zhang, Chuanjun > <chuanjun.zhang(a)intel.com>; Fleming, Kermin > <kermin.fleming(a)intel.com>; Huber, Joe <joe.huber(a)intel.com>; Franza, > Olivier <olivier.franza(a)intel.com>; Antonellis, Richard > <richard.antonellis(a)intel.com>; anl-intel-csa(a)jlse.anl.gov; > 'kumaran(a)alcf.anl.gov' <kumaran(a)alcf.anl.gov>; Glossop, Kent > <kent.glossop(a)intel.com>; Hawes, FrederickX <frederickx.hawes(a)intel.com> > *Cc:* 'Williams, Timothy J.' <zippy(a)anl.gov>; 'Finkel, Hal J.' > <hfinkel(a)anl.gov>; Moreno, Camilo A <camilo.a.moreno(a)intel.com>; > Tannenbaum, Barry M <barry.m.tannenbaum(a)intel.com>; Xiang, Lingxiang > <lingxiang.xiang(a)intel.com>; Pamnany, Kiran <kiran.pamnany(a)intel.com>; > Matveev, Zakhar A <zakhar.a.matveev(a)intel.com>; 'Jennings, Elise' > <ejennings(a)anl.gov>; Creech, Timothy M <timothy.m.creech(a)intel.com>; > Gara, Alan <alan.gara(a)intel.com>; 'Jin, Zheming' <zjin(a)anl.gov>; > 'Beckman, Pete' <beckman(a)mcs.anl.gov>; 'Yoshii, Kazutomo' > <kazutomo(a)mcs.anl.gov>; Khaldi, Dounia <dounia.khaldi(a)intel.com>; > Zhang, Yongzhi <yongzhi.zhang(a)intel.com>; 'Bair, Raymond A.' > <rbair(a)anl.gov>; Garg, Rajeev K <rajeev.k.garg(a)intel.com>; Raj, Swapna > <swapna.raj(a)intel.com>; 'Patel, Pragneshkumar Balvantbhai' > <ppatel(a)anl.gov>; Cherukuri, Naveen <naveen.cherukuri(a)intel.com>; > Wnuk, Danica <danica.wnuk(a)intel.com>; Narayanasetty, Bhargavi > <bhargavi.narayanasetty(a)intel.com>; Hall, Jonathan C > <jonathan.c.hall(a)intel.com>; Woodworth, Daniel > <daniel.woodworth(a)intel.com>; Mattson, Timothy G > <timothy.g.mattson(a)intel.com> > *Subject:* RE: ANL/Intel CSA Workload Characterization > Hello All, > During last Friday’s meeting I asked a question about the implications > of encountering real-life conditions in stencil-based codes on porting > to CSA. I wanted to give some explanation here. I apologize in advance > to the numericists in this audience, especially those with a > background in CFD, for whom most of my remarks will be trivial. > > 1. In many instances non-trivial boundary conditions give rise to > complications. Dirichlet conditions specify that values on the > boundary of the grid are known and should not be touched. These > are easy and do not destroy the regular structure of the stencil. > However, a simple outflow condition such as dv_x /dx=0 (where v_x > is the velocity vector in the x-direction; if the boundary of the > grid is not aligned with one of the coordinate directions, even > such a simple condition becomes quite complicated to implement) > would destroy the regularity of the stencil; the weights and > extent of the stencil near the grid boundary are completely > different from those in the interior of the grid. > 2. Larger stencils that accommodate higher-order methods (these have > become more popular over the years), or that implement higher > derivatives are often modified near grid boundaries to avoid > losing accuracy or consistency. Look for example at the way > artificial viscosity is represented near boundaries in the NAS > Parallel Benchmarks BT, SP, and LU (computation of array rhs). > These modifications again destroy the regularity of the stencil. > 3. In many important applications body-fitted grids are used that > employ curvilinear coordinates. While the stencils used may be > topologically fixed (same local neighborhood of grid points is > used to form discretized derivatives), the weights differ from > point to point. > 4. In non-linear problems (the rule in CFD, rather than the > exception), the stencil weights differ from point to point, as > well as from one iteration to the next. > > My question was how much such realistic complicating factors would > affect the various optimizations presented during the talk. They > certainly affect efficiency of codes mapped to CPUs, since they > increase memory bandwidth pressure, may reduce vectorization, may > prevent loop collapse, etc. With CSA we have the added complexity that > these complications could dramatically increase the size of the graph, > which would affect the level to which loops can be unrolled. > I cannot oversee all the implications, and am in general upbeat about > the potential for success of CSA, but we need to be careful not to > translate superior performance of a canonical stencil kernel into > application success. > If anybody would like to see pictures and/or code illustrating my > points, I’ll be happy to put some slides together and distribute. > Rob > -----Original Appointment----- > *From:* Petrini, Fabrizio > *Sent:* Tuesday, July 25, 2017 7:36 PM > *To:* Petrini, Fabrizio; Van Der Wijngaart, Rob F; Zhang, Chuanjun; > Fleming, Kermin; Huber, Joe; Franza, Olivier; Antonellis, Richard; > _anl-intel-csa(a)jlse.anl.gov_ <mailto:[email protected]>; > 'kumaran(a)alcf.anl.gov'; Glossop, Kent; Hawes, FrederickX > *Cc:* 'Williams, Timothy J.'; 'Finkel, Hal J.'; Moreno, Camilo A; > Tannenbaum, Barry M; Xiang, Lingxiang; Pamnany, Kiran; Matveev, Zakhar > A; 'Jennings, Elise'; Wilson, Amalee; Creech, Timothy M; Gara, Alan; > 'Jin, Zheming'; 'Beckman, Pete'; 'Yoshii, Kazutomo'; Khaldi, Dounia; > Zhang, Yongzhi; 'Bair, Raymond A.'; Garg, Rajeev K; Raj, Swapna; > 'Patel, Pragneshkumar Balvantbhai'; Cherukuri, Naveen; Wnuk, Danica; > Narayanasetty, Bhargavi; Hall, Jonathan C; Woodworth, Daniel > *Subject:* FW: ANL/Intel CSA Workload Characterization > *When:* Friday, October 13, 2017 11:00 AM-12:00 PM (UTC-08:00) Pacific > Time (US & Canada). > *Where:* Skype Meeting > -----Original Appointment----- > *From:* Petrini, Fabrizio > *Sent:* Monday, March 27, 2017 10:29 AM > *To:* Petrini, Fabrizio; Fleming, Kermin; Huber, Joe; Franza, Olivier; > Antonellis, Richard; _anl-intel-csa(a)jlse.anl.gov_ > <mailto:[email protected]>; 'kumaran(a)alcf.anl.gov'; Glossop, > Kent; Hawes, FrederickX; Tithi, Jesmin Jahan; Steely Jr, Simon C > *Cc:* 'Williams, Timothy J.'; 'Finkel, Hal J.'; Moreno, Camilo A; > Tannenbaum, Barry M; Xiang, Lingxiang; Pamnany, Kiran; Matveev, Zakhar > A; 'Jennings, Elise'; Wilson, Amalee; Creech, Timothy M; Gara, Alan; > 'Jin, Zheming'; 'Beckman, Pete'; 'Yoshii, Kazutomo'; Khaldi, Dounia; > Zhang, Yongzhi; Zhang, Chuanjun; 'Bair, Raymond A.'; Garg, Rajeev K; > Raj, Swapna; 'Patel, Pragneshkumar Balvantbhai'; Cherukuri, Naveen; > Wnuk, Danica; Narayanasetty, Bhargavi; Hall, Jonathan C; Woodworth, > Daniel; 'Stevens, Rick L.'; Khudia, Daya S; Halpern, Pablo G; Hammond, > Jeff R; Ebeling, Carl; Liu, Xing1; Dhamankar, Nitin S; 'Romero, > Nichols A.'; 'Alvaro'; Lyons, Gail; Magro, Bill > *Subject:* ANL/Intel CSA Workload Characterization > *When:* Occurs every Friday effective 3/31/2017 until 4/6/2018 from > 11:00 AM to 12:00 PM (UTC-08:00) Pacific Time (US & Canada). > *Where:* Skype Meeting > Placeholder for weekly ANL/Intel CSA Exascale workload > characterization meeting > ......................................................................................................................................... > à _Join Skype Meeting_ <https://meet.intel.com/fabrizio.petrini/7K7FM32F> > Trouble Joining? _Try Skype Web App_ > <https://meet.intel.com/fabrizio.petrini/7K7FM32F?sl=1>__ > Join by phone > _+1(916)356-2663 (or your local bridge access #) Choose bridge 5._ > <tel:+1%28916%29356-2663%20%28or%20your%20local%20bridge%20access%20#%29%20Choose%20bridge%205.>(Global) > English (United States) > _Find a local number_ <https://dial.intel.com> > Conference ID: 791574018 > _Forgot your dial-in PIN?_ <https://dial.intel.com> |_Help_ > <https://o15.officeredir.microsoft.com/r/rlidLync15?clid=1033&p1=5&p2=2009> > > _[!OC([1033])!] > ......................................................................................................................................... > Skype users: Recordings are subject to the _Audio/Video Recording > Policy_ > <https://iref.intel.com/GetDoc.aspx?RefLibObjectID=0902007c8003c7b5&Latest=T…>. > -- Hal Finkel Lead, Compiler Technology and Programming Languages Leadership Computing Facility Argonne National Laboratory

1 0

[Anl-intel-csa] CSA resources usage information
by Kumaran, Kalyan 23 Oct '17

23 Oct '17

Chris and Vitali have put this excel template together to collect information from our E6 partners on the apps they are running on the simulator. Would it be possible for Intel to use a similar spreadsheet for the PF deep dives and collect this information for the different apps being tested there? We would like a copy of the spreadsheet to with filled in info. Thanks, Kumar

1 0

Re: [Anl-intel-csa] CSA SDK Release Notes for 2017/10/11 version (updated)
by Fleming, Kermin 20 Oct '17

20 Oct '17

Hi, We provided some of this in the PF presentation in Livermore, but here is what is going on. [cid:[email protected]] [cid:[email protected]] In your code, you can specify @xxx to indicate to the tools that you want extra buffer. We honor this request, but without checking to see if the request is legal. We will eventually do that, but we wanted to give some slack during the co-design phase to really get and understanding of what workloads need. We will eventually start releasing a buffer insertion tool as part of the software stack, but we are still working on it. I think there is a rudimentary insertion tool already available. -Elliott -----Original Message----- From: anl-intel-csa [mailto:[email protected]] On Behalf Of Christopher Knight Sent: Friday, October 20, 2017 1:23 PM To: anl-intel-csa(a)lists.jlse.anl.gov Subject: Re: [Anl-intel-csa] CSA SDK Release Notes for 2017/10/11 version (updated) Hello, Expanding on Kaz's question some, is there a description for how all of the new PE categories are accumulated? In the example below, I can back out how some categories are computed (e.g. pe_DP_instances = sqrtf64 + addf64 + subf64 + mulf64 + divf64 + cmpltf64 + cmpgtf64), but some general guidance would be helpful for the different categories. pe_SP_DP_instances 0 pe_DP_instances 96 pe_ILight_instances 147 pe_ILight_instances_eliminated_by_fusion 18 pe_IHeavy_instances 161 pe_IHeavy_instances_eliminated_by_fusion 6 pe_MBox1_instances 42 pe_MBox2_instances 11 pe_Lib_instances 8 pe_Buffer_instances 807 And, again similar to Kaz's question, how should I interpret pe_Buffer_instances of 807 being larger than pe_Buffer_max 256? pe_Buffer_instances 807 pe_Buffer_max 256 chris On Oct 18, 2017, at 2:53 PM, Kazutomo Yoshii <kazutomo(a)mcs.anl.gov<mailto:[email protected]>> wrote: On 10/18/2017 09:48 AM, Glossop, Kent wrote: > * Counts by PE category are now included in the stats > This is a nice feature! I noticed that pe_Buffer_instances of a compiler generated version of the BWA inner loop exceeds its max. pe_Buffer_instances 297 pe_Buffer_max 256 It does not look the simulator complains about this. What are "Buffer" and "MBOX", btw? Is "Buffer" related to LICs? - kaz _______________________________________________ anl-intel-csa mailing list anl-intel-csa(a)lists.jlse.anl.gov<mailto:[email protected]> https://lists.jlse.anl.gov/mailman/listinfo/anl-intel-csa _______________________________________________ anl-intel-csa mailing list anl-intel-csa(a)lists.jlse.anl.gov<mailto:[email protected]> https://lists.jlse.anl.gov/mailman/listinfo/anl-intel-csa

1 0

Re: [Anl-intel-csa] CSA SDK Release Notes for 2017/10/11 version (updated)
by Christopher Knight 20 Oct '17

20 Oct '17

Hello, Expanding on Kaz’s question some, is there a description for how all of the new PE categories are accumulated? In the example below, I can back out how some categories are computed (e.g. pe_DP_instances = sqrtf64 + addf64 + subf64 + mulf64 + divf64 + cmpltf64 + cmpgtf64), but some general guidance would be helpful for the different categories. pe_SP_DP_instances 0 pe_DP_instances 96 pe_ILight_instances 147 pe_ILight_instances_eliminated_by_fusion 18 pe_IHeavy_instances 161 pe_IHeavy_instances_eliminated_by_fusion 6 pe_MBox1_instances 42 pe_MBox2_instances 11 pe_Lib_instances 8 pe_Buffer_instances 807 And, again similar to Kaz’s question, how should I interpret pe_Buffer_instances of 807 being larger than pe_Buffer_max 256? pe_Buffer_instances 807 pe_Buffer_max 256 chris On Oct 18, 2017, at 2:53 PM, Kazutomo Yoshii <kazutomo(a)mcs.anl.gov> wrote: On 10/18/2017 09:48 AM, Glossop, Kent wrote: > · Counts by PE category are now included in the stats > This is a nice feature! I noticed that pe_Buffer_instances of a compiler generated version of the BWA inner loop exceeds its max. pe_Buffer_instances 297 pe_Buffer_max 256 It does not look the simulator complains about this. What are "Buffer" and "MBOX", btw? Is "Buffer" related to LICs? - kaz _______________________________________________ anl-intel-csa mailing list anl-intel-csa(a)lists.jlse.anl.gov https://lists.jlse.anl.gov/mailman/listinfo/anl-intel-csa

1 0