May 2016 - Intel-nda - mailman3.cels.anl.gov

Re: [JLSE-Intel-NDA] KNL00 BIOS Update
by Delalondre Fabien Jonathan 23 May '16

23 May '16

Ok thanks. Does this sentence on the wiki still applies ? "Please note that currently [..] reboot into SNC mode hangs" Best, Fabien ________________________________________ From: Scott Parker <sparker(a)anl.gov> Sent: Monday, May 23, 2016 11:16 PM To: Delalondre Fabien Jonathan; Allen, Benjamin S.; intel-nda(a)lists.jlse.anl.gov Subject: Re: [JLSE-Intel-NDA] KNL00 BIOS Update Nothing significant - there was a small bump for flat-quad mode from a previously measure peak of 435 to 449 GB/s but this is only 3% and more or less in the noise range of the measurements. SNC-4 mode continues to exhibit problems with one of the NUMA domains being slower and peak bandwidth of 440 GB/s only achievable with 128 threads (vs 64 threads for peak of very close to peak in other modes). Cache mode wasn't tested. -Scott On 5/23/16 4:08 PM, Delalondre Fabien Jonathan wrote: > Did you observe any performance improvements for STREAM ? > > Cheers, > > Fabien > ________________________________________ > From: intel-nda-bounces(a)lists.jlse.anl.gov <intel-nda-bounces(a)lists.jlse.anl.gov> on behalf of Allen, Benjamin S. <bsallen(a)alcf.anl.gov> > Sent: Monday, May 23, 2016 11:05 PM > To: intel-nda(a)lists.jlse.anl.gov > Subject: Re: [JLSE-Intel-NDA] KNL00 BIOS Update > > We're done testing. Sorry for taking the machine away for a bit longer than expected. > > Ben > >> On May 23, 2016, at 3:03 PM, Allen, Benjamin S. <bsallen(a)alcf.anl.gov> wrote: >> >> This work is done. Note, we're still doing a few benchmarks to see if the BIOS has fixed any of our standing issues. >> >> Also, this BIOS update renamed the BIOS parameters for the Memory Model and Mode. They are now "Memory Mode" and "Cluster Mode". The Wiki has been updated to show this. >> >> A separate Intel Vtune 2016 copy has been installed under /soft/restricted/intel-vtune-nda. This install contains the Intel NDA KNL add-on. I've not tested it, so please let me know if it works. I have compiled the sep4_0 kernel module from this VTune install, and set knl00 the load it on boot. >> >> Ben >> >> >>> On May 23, 2016, at 9:20 AM, Allen, Benjamin S. <bsallen(a)alcf.anl.gov> wrote: >>> >>> I'll be installing BIOS 01.01.0132, ME03.01.03.012.0, BMC FW 00.17.9699, and FRUSDR 1.14 later today. Release notes attached. >>> >>> Ben >>> >>> <BIOS Release Notes 01.01.0132.txt><BMC Release Notes 00.17.9699.txt><FRUSDR Release Notes 1.14.txt>_______________________________________________ >>> Intel-nda mailing list >>> Intel-nda(a)lists.jlse.anl.gov >>> https://lists.jlse.anl.gov/mailman/listinfo/intel-nda >> _______________________________________________ >> Intel-nda mailing list >> Intel-nda(a)lists.jlse.anl.gov >> https://lists.jlse.anl.gov/mailman/listinfo/intel-nda > _______________________________________________ > Intel-nda mailing list > Intel-nda(a)lists.jlse.anl.gov > https://lists.jlse.anl.gov/mailman/listinfo/intel-nda > _______________________________________________ > Intel-nda mailing list > Intel-nda(a)lists.jlse.anl.gov > https://lists.jlse.anl.gov/mailman/listinfo/intel-nda

1 0

Re: [JLSE-Intel-NDA] KNL00 BIOS Update
by Scott Parker 23 May '16

23 May '16

Nothing significant - there was a small bump for flat-quad mode from a previously measure peak of 435 to 449 GB/s but this is only 3% and more or less in the noise range of the measurements. SNC-4 mode continues to exhibit problems with one of the NUMA domains being slower and peak bandwidth of 440 GB/s only achievable with 128 threads (vs 64 threads for peak of very close to peak in other modes). Cache mode wasn't tested. -Scott On 5/23/16 4:08 PM, Delalondre Fabien Jonathan wrote: > Did you observe any performance improvements for STREAM ? > > Cheers, > > Fabien > ________________________________________ > From: intel-nda-bounces(a)lists.jlse.anl.gov <intel-nda-bounces(a)lists.jlse.anl.gov> on behalf of Allen, Benjamin S. <bsallen(a)alcf.anl.gov> > Sent: Monday, May 23, 2016 11:05 PM > To: intel-nda(a)lists.jlse.anl.gov > Subject: Re: [JLSE-Intel-NDA] KNL00 BIOS Update > > We're done testing. Sorry for taking the machine away for a bit longer than expected. > > Ben > >> On May 23, 2016, at 3:03 PM, Allen, Benjamin S. <bsallen(a)alcf.anl.gov> wrote: >> >> This work is done. Note, we're still doing a few benchmarks to see if the BIOS has fixed any of our standing issues. >> >> Also, this BIOS update renamed the BIOS parameters for the Memory Model and Mode. They are now "Memory Mode" and "Cluster Mode". The Wiki has been updated to show this. >> >> A separate Intel Vtune 2016 copy has been installed under /soft/restricted/intel-vtune-nda. This install contains the Intel NDA KNL add-on. I've not tested it, so please let me know if it works. I have compiled the sep4_0 kernel module from this VTune install, and set knl00 the load it on boot. >> >> Ben >> >> >>> On May 23, 2016, at 9:20 AM, Allen, Benjamin S. <bsallen(a)alcf.anl.gov> wrote: >>> >>> I'll be installing BIOS 01.01.0132, ME03.01.03.012.0, BMC FW 00.17.9699, and FRUSDR 1.14 later today. Release notes attached. >>> >>> Ben >>> >>> <BIOS Release Notes 01.01.0132.txt><BMC Release Notes 00.17.9699.txt><FRUSDR Release Notes 1.14.txt>_______________________________________________ >>> Intel-nda mailing list >>> Intel-nda(a)lists.jlse.anl.gov >>> https://lists.jlse.anl.gov/mailman/listinfo/intel-nda >> _______________________________________________ >> Intel-nda mailing list >> Intel-nda(a)lists.jlse.anl.gov >> https://lists.jlse.anl.gov/mailman/listinfo/intel-nda > _______________________________________________ > Intel-nda mailing list > Intel-nda(a)lists.jlse.anl.gov > https://lists.jlse.anl.gov/mailman/listinfo/intel-nda > _______________________________________________ > Intel-nda mailing list > Intel-nda(a)lists.jlse.anl.gov > https://lists.jlse.anl.gov/mailman/listinfo/intel-nda

1 0

Re: [JLSE-Intel-NDA] KNL00 BIOS Update
by Delalondre Fabien Jonathan 23 May '16

23 May '16

Did you observe any performance improvements for STREAM ? Cheers, Fabien ________________________________________ From: intel-nda-bounces(a)lists.jlse.anl.gov <intel-nda-bounces(a)lists.jlse.anl.gov> on behalf of Allen, Benjamin S. <bsallen(a)alcf.anl.gov> Sent: Monday, May 23, 2016 11:05 PM To: intel-nda(a)lists.jlse.anl.gov Subject: Re: [JLSE-Intel-NDA] KNL00 BIOS Update We're done testing. Sorry for taking the machine away for a bit longer than expected. Ben > On May 23, 2016, at 3:03 PM, Allen, Benjamin S. <bsallen(a)alcf.anl.gov> wrote: > > This work is done. Note, we're still doing a few benchmarks to see if the BIOS has fixed any of our standing issues. > > Also, this BIOS update renamed the BIOS parameters for the Memory Model and Mode. They are now "Memory Mode" and "Cluster Mode". The Wiki has been updated to show this. > > A separate Intel Vtune 2016 copy has been installed under /soft/restricted/intel-vtune-nda. This install contains the Intel NDA KNL add-on. I've not tested it, so please let me know if it works. I have compiled the sep4_0 kernel module from this VTune install, and set knl00 the load it on boot. > > Ben > > >> On May 23, 2016, at 9:20 AM, Allen, Benjamin S. <bsallen(a)alcf.anl.gov> wrote: >> >> I'll be installing BIOS 01.01.0132, ME03.01.03.012.0, BMC FW 00.17.9699, and FRUSDR 1.14 later today. Release notes attached. >> >> Ben >> >> <BIOS Release Notes 01.01.0132.txt><BMC Release Notes 00.17.9699.txt><FRUSDR Release Notes 1.14.txt>_______________________________________________ >> Intel-nda mailing list >> Intel-nda(a)lists.jlse.anl.gov >> https://lists.jlse.anl.gov/mailman/listinfo/intel-nda > > _______________________________________________ > Intel-nda mailing list > Intel-nda(a)lists.jlse.anl.gov > https://lists.jlse.anl.gov/mailman/listinfo/intel-nda _______________________________________________ Intel-nda mailing list Intel-nda(a)lists.jlse.anl.gov https://lists.jlse.anl.gov/mailman/listinfo/intel-nda

1 0

Re: [JLSE-Intel-NDA] KNL00 BIOS Update
by Allen, Benjamin S. 23 May '16

23 May '16

We're done testing. Sorry for taking the machine away for a bit longer than expected. Ben > On May 23, 2016, at 3:03 PM, Allen, Benjamin S. <bsallen(a)alcf.anl.gov> wrote: > > This work is done. Note, we're still doing a few benchmarks to see if the BIOS has fixed any of our standing issues. > > Also, this BIOS update renamed the BIOS parameters for the Memory Model and Mode. They are now "Memory Mode" and "Cluster Mode". The Wiki has been updated to show this. > > A separate Intel Vtune 2016 copy has been installed under /soft/restricted/intel-vtune-nda. This install contains the Intel NDA KNL add-on. I've not tested it, so please let me know if it works. I have compiled the sep4_0 kernel module from this VTune install, and set knl00 the load it on boot. > > Ben > > >> On May 23, 2016, at 9:20 AM, Allen, Benjamin S. <bsallen(a)alcf.anl.gov> wrote: >> >> I'll be installing BIOS 01.01.0132, ME03.01.03.012.0, BMC FW 00.17.9699, and FRUSDR 1.14 later today. Release notes attached. >> >> Ben >> >> <BIOS Release Notes 01.01.0132.txt><BMC Release Notes 00.17.9699.txt><FRUSDR Release Notes 1.14.txt>_______________________________________________ >> Intel-nda mailing list >> Intel-nda(a)lists.jlse.anl.gov >> https://lists.jlse.anl.gov/mailman/listinfo/intel-nda > > _______________________________________________ > Intel-nda mailing list > Intel-nda(a)lists.jlse.anl.gov > https://lists.jlse.anl.gov/mailman/listinfo/intel-nda

1 0

Re: [JLSE-Intel-NDA] KNL00 BIOS Update
by Allen, Benjamin S. 23 May '16

23 May '16

This work is done. Note, we're still doing a few benchmarks to see if the BIOS has fixed any of our standing issues. Also, this BIOS update renamed the BIOS parameters for the Memory Model and Mode. They are now "Memory Mode" and "Cluster Mode". The Wiki has been updated to show this. A separate Intel Vtune 2016 copy has been installed under /soft/restricted/intel-vtune-nda. This install contains the Intel NDA KNL add-on. I've not tested it, so please let me know if it works. I have compiled the sep4_0 kernel module from this VTune install, and set knl00 the load it on boot. Ben > On May 23, 2016, at 9:20 AM, Allen, Benjamin S. <bsallen(a)alcf.anl.gov> wrote: > > I'll be installing BIOS 01.01.0132, ME03.01.03.012.0, BMC FW 00.17.9699, and FRUSDR 1.14 later today. Release notes attached. > > Ben > > <BIOS Release Notes 01.01.0132.txt><BMC Release Notes 00.17.9699.txt><FRUSDR Release Notes 1.14.txt>_______________________________________________ > Intel-nda mailing list > Intel-nda(a)lists.jlse.anl.gov > https://lists.jlse.anl.gov/mailman/listinfo/intel-nda

1 0

Re: [JLSE-Intel-NDA] KNL00 BIOS Update
by Kumaran, Kalyan 23 May '16

23 May '16

Hi Avinash, Sundaram, Does this fix the STREAM bandwidth difference in one of the clusters in cluster mode? [EPSD100252464] Fixed Cluster mode (SRK) vs Memory Model (ADP) option in BIOS Thanks, Kumar > On May 23, 2016, at 9:20 AM, Allen, Benjamin S. <bsallen(a)alcf.anl.gov> wrote: > > I'll be installing BIOS 01.01.0132, ME03.01.03.012.0, BMC FW 00.17.9699, and FRUSDR 1.14 later today. Release notes attached. > > Ben > > <BIOS Release Notes 01.01.0132.txt><BMC Release Notes 00.17.9699.txt><FRUSDR Release Notes 1.14.txt>_______________________________________________ > Intel-nda mailing list > Intel-nda(a)lists.jlse.anl.gov > https://lists.jlse.anl.gov/mailman/listinfo/intel-nda

1 0

[JLSE-Intel-NDA] KNL00 BIOS Update
by Allen, Benjamin S. 23 May '16

23 May '16

I'll be installing BIOS 01.01.0132, ME03.01.03.012.0, BMC FW 00.17.9699, and FRUSDR 1.14 later today. Release notes attached. Ben

1 0

[JLSE-Intel-NDA] Fwd: KNL DAP Pedestal manual
by Allen, Benjamin S. 20 May '16

20 May '16

Manual for the pedestal (workstation) machines. This is Intel NDA'ed material. Ben > Begin forwarded message: > > From: Gautam Shah <shah(a)colfax-intl.com> > Subject: KNL DAP Pedestal manual > Date: May 20, 2016 at 1:27:09 PM CDT > To: "'Allen, Benjamin S.'" <bsallen(a)alcf.anl.gov> > Reply-To: <shah(a)colfax-intl.com> > > Hi Ben, > > Sorry for the delay in getting this over to you. Attached is the early draft > of the manual. It is provided to you under your NDA with Intel. Please do > not distribute. > > Many thanks, > > Gautam

1 0

[JLSE-Intel-NDA] knl00 sudo privileges
by Delalondre Fabien Jonathan 18 May '16

18 May '16

Hi, ?Following JLSE wiki page, I would like to get sudo privileges to be able to reboot knl00 in different modes to do some application testing. Thanks, Fabien

1 0

Re: [JLSE-Intel-NDA] LAMMPS and QBox on KNL
by Christopher Knight 16 May '16

16 May '16

No, I haven’t tried specifying anything yet with ‘numactl’. chris On May 16, 2016, at 9:54 AM, Harms, Kevin <harms(a)alcf.anl.gov> wrote: Chris, for flat+quadrant mode, did you try running with 'mpirun numactl -m 1 ./app.exe' ? kevin > > > > Hello, > > > I placed some updated slides on the KNL confluence page. The pdf now includes additional comparisons and some timings for the WEST code used in one of the ESP projects. > > > I noticed that “cache+quadrant” mode can in some cases yield ~2x speedups in runtime for the planewave DFT codes compared to “flat+quadrant” with most of the runtime difference coming from the 3D FFT related kernels. Maybe explicitly allocating > memory to HBM will recover performance in “flat+quadrant” mode? > > > chris > > > > > > > > On May 9, 2016, at 10:16 AM, Christopher Knight <knightc(a)anl.gov> wrote: > > Hello, > > Here’s quick summary of what I’ve seen so far with LAMMPS and QBox on KNL. WEST code is work in progress. > > > For the LAMMPS rhodo benchmark, I see core-to-core speedup of 3.39x on KNL and a node-to-node speedup of 5.40x using 64 MPI ranks and 2 OpenMP threads on KNL. Using Intel optimized code on KNL (can’t run on BG/Q), I see core-to-core speedup of 10.49x and node-to-node > speedup of 12.47x. > > For the LAMMPS reaxc benchmark using multi-threaded package, I see core-to-core speedup of 4.05x on KNL and a node-to-node speedup of 5.19x using 16 MPI ranks and 8 OpenMP threads on KNL. > > > For QBox, I’m using inputs related to an ESP project. I see core-to-core speedups in the range of 2.5 – 3.5x on KNL and a node-to-node speedups in range of 3.8 – 4.5x depending on model, system, and compilation flags. Runtimes sensitive to FFT interface selected > for MKL. > > > chris > > > > > > > > > > _______________________________________________ > Intel-nda mailing list > Intel-nda(a)lists.jlse.anl.gov > https://lists.jlse.anl.gov/mailman/listinfo/intel-nda > > > > <knl_LAMMPS_QBOX_May2016.pdf> > >

1 0