- Intel-nda - mailman3.cels.anl.gov

[JLSE-Intel-NDA] Fwd: A0
by Kumaran, Kalyan 17 Mar '16

17 Mar '16

Begin forwarded message: From: "Fromkin, Russ" <russ.fromkin(a)intel.com<mailto:[email protected]>> Subject: A0 Date: March 17, 2016 at 11:47:42 AM CDT To: "Kumaran, Kalyan" <kumaran(a)alcf.anl.gov<mailto:[email protected]>>, "Allen, Benjamin S." <bsallen(a)alcf.anl.gov<mailto:[email protected]>> Cc: "Kirkendall, Keith G" <keith.g.kirkendall(a)intel.com<mailto:[email protected]>>, "Allison, CM" <cm.allison(a)intel.com<mailto:[email protected]>> Kumar, Ben, If all goes well, the A0 will ship to your attention today or tomorrow. My guess is that you will see it tomorrow or Monday. It should come with the instructions from the Adams Pass team. In addition, Avinash Sodani would like to discuss with you what you are seeing. I will add him to our other email string shortly. Russ Russ Fromkin Intel Corporation Director, Americas HPC Sales Office: (732) 790-0178 Mobile: (201) 532-1858

1 0

[JLSE-Intel-NDA] Fwd: B0 box
by Kumaran, Kalyan 16 Mar '16

16 Mar '16

Begin forwarded message: From: "Fromkin, Russ" <russ.fromkin(a)intel.com<mailto:[email protected]>> Subject: RE: B0 box Date: March 16, 2016 at 10:46:18 AM CDT To: "Allen, Benjamin S." <bsallen(a)alcf.anl.gov<mailto:[email protected]>> Cc: "Allison, CM" <cm.allison(a)intel.com<mailto:[email protected]>>, "Kirkendall, Keith G" <keith.g.kirkendall(a)intel.com<mailto:[email protected]>>, "Kumaran, Kalyan" <kumaran(a)alcf.anl.gov<mailto:[email protected]>> Ben, So there are three options to upgrade the bios. 1) Return the system to be updated? 2) Send a new flash chip? 3) Find an A0 CPU and ship it to ANL? For #1, I don't trust having our system out of our hands. Possession in 9/10th the law. :-) For #2: I am being told that it might be a little wait for new flash chips. For #3: I don't have any A0s, but with the B0s rolling into the Intel internal systems, we think we can get one pretty quickly and send it out. For #3, this is the most work on the ANL side. We would need to pull the B0 and replace with the A0. Then we can update the bios. After it is update, we can pull the A0 and replace the B0. Are you okay with this? Russ

1 0

[JLSE-Intel-NDA] Access for Jeff Hammond
by Kumaran, Kalyan 16 Mar '16

16 Mar '16

Hi Ben, is it possible to provide Jeff access to our KNL box? Jeff might be able to help us with software and also explain some of the performance issues we have. Thanks, Kumar

1 0

[JLSE-Intel-NDA] Fwd: B0 box
by Kumaran, Kalyan 15 Mar '16

15 Mar '16

Begin forwarded message: From: "Allen, Benjamin S." <bsallen(a)alcf.anl.gov<mailto:[email protected]>> Date: March 15, 2016 at 5:43:27 PM CDT To: "Fromkin, Russ" <russ.fromkin(a)intel.com<mailto:[email protected]>> Cc: "Allison, CM" <cm.allison(a)intel.com<mailto:[email protected]>>, "Kirkendall, Keith G" <keith.g.kirkendall(a)intel.com<mailto:[email protected]>>, "Kumaran, Kalyan" <kumaran(a)alcf.anl.gov<mailto:[email protected]>> Subject: Re: B0 box Russ, Both the single 16GB DIMM and 6x 32GB DIMMs are coming up at the same speed 2134Mt/s, and are otherwise spec'ed the same. As such, I don't think thats the issue at hand. If anything with 6 DIMMs we should be getter better memory throughput, since KNL has 6 memory channels. Vitali has benchmarked with STREAM a decrease in memory throughput between having a single 16GB DIMM vs 6 32GB DIMMs. Even though the test should be small enough to stay on the MCDRAM cached, and we have the MCDRAM in cached mode. We're looking to discuss why that is as its unexpected, and ensure we didn't overlook something. Note, the goal with 6 x 32GB DIMMs is to simulate a Theta node. Thanks, Ben On Mar 15, 2016, at 4:34 PM, Kumaran, Kalyan <kumaran(a)alcf.anl.gov<mailto:[email protected]>> wrote: Hi Russ, yes, it is the memory bus frequency issue. -Kumar On Mar 15, 2016, at 4:33 PM, Fromkin, Russ <russ.fromkin(a)intel.com<mailto:[email protected]>> wrote: Kumar, I this the 2133 vs 2400 question or something different? I believe we need a bios upgrade, but i am hearing we may need to send an A0 for it to work. We are questioning why and will have an answer soon. This is our PCSC beta box, so there isnt regular support for it yet. Only a few people have this box. Russ Sent via the Samsung Galaxy S® 6, an AT&T 4G LTE smartphone -------- Original message -------- From: "Kumaran, Kalyan" <kumaran(a)alcf.anl.gov<mailto:[email protected]>> Date: 3/15/2016 4:20 PM (GMT-05:00) To: "Fromkin, Russ" <russ.fromkin(a)intel.com<mailto:[email protected]>>, "Allison, CM" <cm.allison(a)intel.com<mailto:[email protected]>>, "Kirkendall, Keith G" <keith.g.kirkendall(a)intel.com<mailto:[email protected]>> Cc: "Allen, Benjamin S." <bsallen(a)alcf.anl.gov<mailto:[email protected]>> Subject: B0 box Russ, to whom do we send our questions to on the box? We removed the 16GB DIMM and added 192 GB of memory. Our Stream results are not good. We want to make sure we are doing the right thing. Thanks, Kumar

1 0

Re: [JLSE-Intel-NDA] MILCmk results on KNL b0
by Kumaran, Kalyan 14 Mar '16

14 Mar '16

Please use intel-nda list. -Kumar > On Mar 14, 2016, at 3:29 PM, Harms, Kevin <harms(a)alcf.anl.gov> wrote: > > > ok. I will take a look at that one and see what I can find out. > > kevin > > > > >> Hi Kevin, >> >> Probably 65536 is the most relevant. This code is old now so I don't >> care about it too much now anyway. The performance is still slower than >> I would have expected even without any vectorization. It may be nice to >> understand what the bottleneck is for say the single precision, 65536, >> QLA_V_vpeq_M_times_pV case (or for simplicity, even the >> QLA_V_veq_Ma_times_V one since that one has a more regular memory access >> pattern). >> >> Cheers, >> James >> >> >> On 03/11/2016 12:10 PM, Harms, Kevin wrote: >>> James, >>> >>> I ran the MILCmk CORAL benchmarks on the KNL b0. I'm attaching the results. For larger sizes, there's a 2-3x speedup over mira. What sizes do you consider to be the important ones? >>> >>> kevin >>> >>> MILC-v1 benchmark from CORAL site >>> A2 results from running defaults in Makefile >>> KNL B0 results compiled with Intel using basic optimization >>> >>> icc -std=c99 -O3 -xMIC-AVX512 -openmp -Iqla-1.7.1 -DQLA_Precision=\'F\' -DQLA_Colors=3 -DQLA_Nc=3 -c qla-1.7.1/QLA_F3_c1_veq_V_dot_V.c -o qla-1.7.1/QLA_F3_c1_veq_V_dot_V.o >>> >>> Run with OMP_NUM_THREADS=64 (have not tried 128 yet) >>> >>> I could not run the larger sizes on KNL because lack of memory. >>> >>>

1 0

Re: [JLSE-Intel-NDA] Bringing KNL Machine Down for SSD Install
by Allen, Benjamin S. 14 Mar '16

14 Mar '16

This is done. The machine is back open for regular use. Ben > On Mar 14, 2016, at 1:56 PM, Allen, Benjamin S. <bsallen(a)alcf.anl.gov> wrote: > > Adding in a 200GB SATA SSD. The machine will be down for 20-30minutes. > > Ben > > _______________________________________________ > Intel-nda mailing list > Intel-nda(a)lists.jlse.anl.gov > https://lists.jlse.anl.gov/mailman/listinfo/intel-nda

1 0

[JLSE-Intel-NDA] Bringing KNL Machine Down for SSD Install
by Allen, Benjamin S. 14 Mar '16

14 Mar '16

Adding in a 200GB SATA SSD. The machine will be down for 20-30minutes. Ben

1 0

[JLSE-Intel-NDA] STREAM: 1 ch versus 6 ch
by Vitali A. Morozov 14 Mar '16

14 Mar '16

After reboot, I still see the same values of about 22-24 GB/s for triad on 1.6-2.7 GB data set. I tried this benchmark on the host node "ftlogin2", corrected to AVX compilation versus AVX512 compilation: OMP_NUM_THREADS=8 KMP_AFFINITY=scatter ./stream Function Rate (MB/s) Avg time Min time Max time Copy: 41624.1384 0.0464 0.0461 0.0467 Scale: 40246.4026 0.0480 0.0477 0.0485 Add: 39071.5522 0.0741 0.0737 0.0747 Triad: 38904.1905 0.0744 0.0740 0.0751 Store: 42255.5550 0.0229 0.0227 0.0231 This machine has 4 channels of DDR3 800-1600, so the expected bandwidth is somewhere in between 25.6 - 51.2 GB/s. I have measured 38.9 GB/s, seems ok. Vitali For reference: STREAM triad has degraded on small data size about 1.6 GB. I would expect that, in cache mode, the result should be identical, as the entire data set fits into IPM. Threads 1 channel 16 GB 6 channels192 GB 4 22798.1543 12401.8232 8 34495.5022 19618.5959 16 61197.7447 21515.0307 32 90566.432622264.3076 64 172575.511725265.8044

1 0

[JLSE-Intel-NDA] test
by Kumaran, Kalyan 14 Mar '16

14 Mar '16

test

1 0