- Intel-nda - mailman3.cels.anl.gov

Re: [JLSE-Intel-NDA] KNL User Guide
by Delalondre Fabien Jonathan 10 May '16

10 May '16

I could access the page. Best, Fabien ________________________________________ From: intel-nda-bounces(a)lists.jlse.anl.gov <intel-nda-bounces(a)lists.jlse.anl.gov> on behalf of Christopher Knight <knightc(a)anl.gov> Sent: Tuesday, May 10, 2016 4:21 PM To: Harms, Kevin Cc: intel-nda(a)lists.jlse.anl.gov Subject: Re: [JLSE-Intel-NDA] KNL User Guide Hi Kevin, Tried viewing the page with jlse user/passwd and it returned an “operation not permitted” message. Does anyone else see similar message or has succeeded in viewing page? chris On May 9, 2016, at 4:27 PM, Harms, Kevin <harms(a)alcf.anl.gov> wrote: Here's a user guide for the KNL node. https://wiki.jlse.anl.gov/display/knl/Intel+KNL I believe anyone with Confluence access and in the 'intel' group should be able to edit it. Please make any corrections as needed. kevin _______________________________________________ Intel-nda mailing list Intel-nda(a)lists.jlse.anl.gov https://lists.jlse.anl.gov/mailman/listinfo/intel-nda _______________________________________________ Intel-nda mailing list Intel-nda(a)lists.jlse.anl.gov https://lists.jlse.anl.gov/mailman/listinfo/intel-nda

1 0

Re: [JLSE-Intel-NDA] KNL User Guide
by Christopher Knight 10 May '16

10 May '16

Hi Kevin, Tried viewing the page with jlse user/passwd and it returned an “operation not permitted” message. Does anyone else see similar message or has succeeded in viewing page? chris On May 9, 2016, at 4:27 PM, Harms, Kevin <harms(a)alcf.anl.gov> wrote: Here's a user guide for the KNL node. https://wiki.jlse.anl.gov/display/knl/Intel+KNL I believe anyone with Confluence access and in the 'intel' group should be able to edit it. Please make any corrections as needed. kevin _______________________________________________ Intel-nda mailing list Intel-nda(a)lists.jlse.anl.gov https://lists.jlse.anl.gov/mailman/listinfo/intel-nda

1 0

[JLSE-Intel-NDA] KNL User Guide
by Harms, Kevin 09 May '16

09 May '16

Here's a user guide for the KNL node. https://wiki.jlse.anl.gov/display/knl/Intel+KNL I believe anyone with Confluence access and in the 'intel' group should be able to edit it. Please make any corrections as needed. kevin

1 0

[JLSE-Intel-NDA] Fwd: Intel Meeting Notes
by Kumaran, Kalyan 09 May '16

09 May '16

We have our bi-weekly meeting with Intel tomorrow on our KNL white box. Date: May 10th, 2016 Time: 2:30pm to 3:00pm Room: 1172 Please see action items below on topics for discussion. If you have other issues to discuss please let me know. Thanks, Kumar Begin forwarded message: From: "Harms, Kevin" <harms(a)alcf.anl.gov<mailto:[email protected]>> Subject: [JLSE-Intel-NDA] Intel Meeting Notes Date: April 26, 2016 at 3:56:28 PM CDT To: "intel-nda(a)lists.jlse.anl.gov<mailto:[email protected]>" <intel-nda(a)lists.jlse.anl.gov<mailto:[email protected]>> Meeting with Avinash and Intel - 2016/4/26 Discussion: - We indicated that we are able to get 420,000 MB/s from stream in Flat All2All and Quadrant modes since the recent BIOS update - Intel would like us to investigate the Quadrant model in Flat mode because they think we should get 440,000 MB/s - Intel examined our NUMA output from the system and believes that we are not running a newer firmware because the logical processes should be nicely ordered. - Intel said they only expect at 5-6% variance between NUMA clusters. - Vitali present his HACC results on KNL (which were all good and have improved since A0) - Irv from Intel joined to help debug our BIOS setup. - He indicated that our BIOS is provided by PCSD division which does there own packaging and he doesn't know what they have. - Intel development runs BIOS created by him. - After some discussion, we need someone from Intel to help us debug the issue - Avinash will work to assign someone to help us - It appears Intel development has their own development boards and do not use Adams Pass boards Action Items: - Argonne: Run our stream test cases in Flat-Quadrant configuration and try to achieve 440,000 MB/s using 1 or 2 threads per core - Intel: Assign someone to help investigate the BIOS version/configuration on our box - Argonne/Intel - setup a meeting in two weeks to discuss progress. Scott will present his NEK results. kevin _______________________________________________ Intel-nda mailing list Intel-nda(a)lists.jlse.anl.gov<mailto:[email protected]> https://lists.jlse.anl.gov/mailman/listinfo/intel-nda

1 0

[JLSE-Intel-NDA] LAMMPS and QBox on KNL
by Christopher Knight 09 May '16

09 May '16

Hello, Here’s quick summary of what I’ve seen so far with LAMMPS and QBox on KNL. WEST code is work in progress. For the LAMMPS rhodo benchmark, I see core-to-core speedup of 3.39x on KNL and a node-to-node speedup of 5.40x using 64 MPI ranks and 2 OpenMP threads on KNL. Using Intel optimized code on KNL (can’t run on BG/Q), I see core-to-core speedup of 10.49x and node-to-node speedup of 12.47x. For the LAMMPS reaxc benchmark using multi-threaded package, I see core-to-core speedup of 4.05x on KNL and a node-to-node speedup of 5.19x using 16 MPI ranks and 8 OpenMP threads on KNL. For QBox, I’m using inputs related to an ESP project. I see core-to-core speedups in the range of 2.5 – 3.5x on KNL and a node-to-node speedups in range of 3.8 – 4.5x depending on model, system, and compilation flags. Runtimes sensitive to FFT interface selected for MKL. chris

1 0

Re: [JLSE-Intel-NDA] ESP - KNL00 questions
by Luo, Ye 05 May '16

05 May '16

Hi Fabien, I'd like to share my experience of vtune. I ran the vtune 2016 by collecting hotspots without problems but memory-access is not supported. In my case, I only launch 1 MPI and use all the cores+HT via threads so mpirun is even not needed. amplxe-cl -collect hotspots my_app or when I use flat quadrant mode and use the HBM. amplxe-cl -collect hotspots numactl -m 1 my_app vtune post-processing is almost single threaded. So please try to make your run short to reduce the time. Then you can just run amplxe-gui to visualize then measurement. Ye =================== Ye Luo, Ph.D. Leadership Computing Facility Argonne National Laboratory ________________________________ From: intel-nda-bounces(a)lists.jlse.anl.gov [intel-nda-bounces(a)lists.jlse.anl.gov] on behalf of Delalondre Fabien Jonathan [fabien.delalondre(a)epfl.ch] Sent: Wednesday, May 04, 2016 2:44 PM To: intel-nda(a)lists.jlse.anl.gov Subject: [JLSE-Intel-NDA] ESP - KNL00 questions Hi All, For those of you who dont know me, I am Fabien Delalondre from EPFL working on brain simulation in Switzerland. I have just got access to knl00 as part of the ALCF Early Science Program. I have a few questions related to KNL00. I apologize already if they sound like silly questions ... Appreciate all the help I can get ... 0) Is there any wiki accessible where users would add interesting facts/experience/general documentation etc ? 1) What is the recommended placeholder for the files to be written to ? Should I use home ? What is the file system/expected bandwidth on this file system 2) Is there a possibility to change the card configuration from user space ? Or should I use intel-nda(a)lists.jlse.anl.gov<mailto:[email protected]> mailing list every time I want to ask a different configuration ? 3) When the /soft/scheduling/knl/knl.txt text says knl00 node status: in use by William Scullin until 11:30 am CT 4 May 2016 - e-mail to to preempt running benchmark. Can I still use the system or is there a policy defined ? How can I "reserve" the KNL ? Should I send an email to the mailing list ? With the time difference between the US and Chicago I could use it every night in Chicago :-) 4) I understand there is no scheduler on this system up to now. Please let me know if this is not correct. 5) Can you point me to someone who has lots of experience with VTune on KNL ? Vitali was mentioning to me it does not produce the results he expected. I d like to know more about this. Any other profiling tool you would recommend ? Any documentation available on performance counters available ? 6) Is there a debugger available on this node ? If the answer is yes, which one ? Can I use run a vnc server to connect and improve response time of the graphical interfaces ? 7) Is memkind/autoHBM installed on the system (http://www.nersc.gov/users/computational-systems/cori/application-porting-a…)<http://www.nersc.gov/users/computational-systems/cori/application-porting-a…>? Any recommendation regarding using or not using them ? Thanks again for your help ! Best, Fabien

1 0

[JLSE-Intel-NDA] Fwd: Two new KNL papers
by Allen, Benjamin S. 05 May '16

05 May '16

Some howtos from Colfax. Ben Begin forwarded message: From: Gautam Shah <shah(a)colfax-intl.com<mailto:[email protected]>> Subject: Two new KNL papers Date: May 5, 2016 at 1:56:29 AM CDT To: "Allen, Benjamin S." <bsallen(a)alcf.anl.gov<mailto:[email protected]>> Gautam Shah Colfax International 408-730-2275 x 141 shah(a)colfax-intl.com<mailto:[email protected]> www.colfax-intl.com<http://www.colfax-intl.com> www.colfaxdirect.com<http://www.colfaxdirect.com> www.colfaxresearch.com<http://www.colfaxresearch.com>

1 0

Re: [JLSE-Intel-NDA] ESP - KNL00 questions
by Vitali A. Morozov 04 May '16

04 May '16

> Can you point me to someone who has lots of experience with VTune on KNL ? Vitali was mentioning to me it does not produce the results he expected. I d like to know more about this. Any other profiling tool you would recommend ? Any documentation available on performance counters available ? I am using /home/morozov/lib/libmpitrace.a profiling tool, and I am quite happy with this so far. It has similar functionality as identical BG/Q libmpitrace.a, except no counters are implemented yet. Vitali On 05/04/2016 02:44 PM, Delalondre Fabien Jonathan wrote: > Hi All, > > For those of you who dont know me, I am Fabien Delalondre from EPFL > working on brain simulation in Switzerland. I have just got access to > knl00 as part of the ALCF Early Science Program. I have a few > questions related to KNL00. I apologize already if they sound like > silly questions ... Appreciate all the help I can get ... > > 0) Is there any wiki accessible where users would add interesting > facts/experience/general documentation etc ? > 1) What is the recommended placeholder for the files to be written to > ? Should I use home ? What is the file system/expected bandwidth on > this file system > 2) Is there a possibility to change the card configuration from user > space ? Or should I use intel-nda(a)lists.jlse.anl.gov > <mailto:[email protected]> mailing list every time I want > to ask a different configuration ? > 3) When the /soft/scheduling/knl/knl.txt text says knl00 node status: > in use by William Scullin until 11:30 am CT 4 May 2016 - e-mail to to > preempt running benchmark. Can I still use the system or is there a > policy defined ? How can I "reserve" the KNL ? Should I send an email > to the mailing list ? With the time difference between the US and > Chicago I could use it every night in Chicago :-) > 4) I understand there is no scheduler on this system up to now. Please > let me know if this is not correct. > 5) Can you point me to someone who has lots of experience with VTune > on KNL ? Vitali was mentioning to me it does not produce the results > he expected. I d like to know more about this. Any other profiling > tool you would recommend ? Any documentation available on performance > counters available ? > 6) Is there a debugger available on this node ? If the answer is yes, > which one ? Can I use run a vnc server to connect and improve response > time of the graphical interfaces ? > 7) Is memkind/autoHBM installed on the system > (http://www.nersc.gov/users/computational-systems/cori/application-porting-a…) > <http://www.nersc.gov/users/computational-systems/cori/application-porting-a…>? > > Any recommendation regarding using or not using them ? > > Thanks again for your help ! > > > Best, > > > Fabien > > > > _______________________________________________ > Intel-nda mailing list > Intel-nda(a)lists.jlse.anl.gov > https://lists.jlse.anl.gov/mailman/listinfo/intel-nda

1 0

Re: [JLSE-Intel-NDA] ESP - KNL00 questions
by Allen, Benjamin S. 04 May '16

04 May '16

You would have received your password in an email when you were automatically subscribed to the list. This doesn't use the same credentials as https://accounts.jlse.anl.gov unfortunately. Ben > On May 4, 2016, at 4:17 PM, Delalondre Fabien Jonathan <fabien.delalondre(a)epfl.ch> wrote: > > Looks like I cant log in even with my jlse email/password. > I am happy to contribute to a wiki if you guys want to do this. > > Best, > > Fabien > ________________________________________ > From: intel-nda-bounces(a)lists.jlse.anl.gov <intel-nda-bounces(a)lists.jlse.anl.gov> on behalf of Allen, Benjamin S. <bsallen(a)alcf.anl.gov> > Sent: Wednesday, May 4, 2016 11:02 PM > To: Kumaran, Kalyan > Cc: intel-nda(a)lists.jlse.anl.gov > Subject: Re: [JLSE-Intel-NDA] ESP - KNL00 questions > > Yea it is, https://lists.jlse.anl.gov/mailman/private/intel-nda/. > > Ben > >> On May 4, 2016, at 3:57 PM, Kumaran, Kalyan <kumaran(a)alcf.anl.gov> wrote: >> >> I hope the list is being archived and folks can search the archives! We should include the link to the archives in the introductory email. >> >> -Kumar >> >> >>> On May 4, 2016, at 3:08 PM, Allen, Benjamin S. <bsallen(a)alcf.anl.gov> wrote: >>> >>> Hi Fabien, >>> >>> Answered in-line below: >>> >>>> For those of you who dont know me, I am Fabien Delalondre from EPFL working on brain simulation in Switzerland. I have just got access to knl00 as part of the ALCF Early Science Program. I have a few questions related to KNL00. I apologize already if they sound like silly questions ... Appreciate all the help I can get ... >>>> >>>> 0) Is there any wiki accessible where users would add interesting facts/experience/general documentation etc ? >>> >>> Nothing setup, but we could if there's interest from the group. >>> >>>> 1) What is the recommended placeholder for the files to be written to ? Should I use home ? What is the file system/expected bandwidth on this file system >>> >>> Use the local SSD for any temp files. This is just a Intel DC S3710 SATA 200GB. /home is fine, but don't expect it to be fast, nor rely on it for any I/O sensitive work. >>> >>>> 2) Is there a possibility to change the card configuration from user space ? Or should I use intel-nda(a)lists.jlse.anl.gov mailing list every time I want to ask a different configuration ? >>> >>> Assume when you say card you're referring to the KNL system itself, and I'm assuming when you say configuration you're asking about MCDRAM's mode and model. If so yes this is possible if needed. It requires BIOS changes, but we have a utility to change those settings in Linux. Changes are applied on reboot. >>> >>>> 3) When the /soft/scheduling/knl/knl.txt text says knl00 node status: in use by William Scullin until 11:30 am CT 4 May 2016 - e-mail to to preempt running benchmark. Can I still use the system or is there a policy defined ? How can I "reserve" the KNL ? Should I send an email to the mailing list ? With the time difference between the US and Chicago I could use it every night in Chicago :-) >>> >>> No you cannot use the system if someone has listed that they're using it in that file. You can of course email this list asking to preempt whoever using it if you think you need to. For reservations just add text to the bottom of the same file stating the reservation date, start and end times. >>> >>>> 4) I understand there is no scheduler on this system up to now. Please let me know if this is not correct. >>> >>> Correct no scheduler. >>> >>>> 5) Can you point me to someone who has lots of experience with VTune on KNL ? Vitali was mentioning to me it does not produce the results he expected. I d like to know more about this. Any other profiling tool you would recommend ? Any documentation available on performance counters available ? >>> >>> We still don't have access to Vtune 2016's special KNL add-on, so we're a bit out of luck so far. As Kevin said, we're at the forefront of KNL knowledge. Unsure if Advisor is any better. >>> >>>> 6) Is there a debugger available on this node ? If the answer is yes, which one ? Can I use run a vnc server to connect and improve response time of the graphical interfaces ? >>> >>> No VNC. Use X11 forwarding. >>> >>>> 7) Is memkind/autoHBM installed on the system (http://www.nersc.gov/users/computational-systems/cori/application-porting-a… >>>> Any recommendation regarding using or not using them ? >>> >>> Yea they're installed. >>> >>> >>> >>> >>> Ben Allen >>> >>> HPC Systems Administrator >>> ALCF Operations - Joint Laboratory for Systems Evaluation >>> Argonne National Laboratory >>> Email: bsallen(a)alcf.anl.gov >>> Office: (630) 252-0554 >>> >>> >>> >>> >>> _______________________________________________ >>> Intel-nda mailing list >>> Intel-nda(a)lists.jlse.anl.gov >>> https://lists.jlse.anl.gov/mailman/listinfo/intel-nda >> > > _______________________________________________ > Intel-nda mailing list > Intel-nda(a)lists.jlse.anl.gov > https://lists.jlse.anl.gov/mailman/listinfo/intel-nda

1 0

Re: [JLSE-Intel-NDA] ESP - KNL00 questions
by Delalondre Fabien Jonathan 04 May '16

04 May '16

Looks like I cant log in even with my jlse email/password. I am happy to contribute to a wiki if you guys want to do this. Best, Fabien ________________________________________ From: intel-nda-bounces(a)lists.jlse.anl.gov <intel-nda-bounces(a)lists.jlse.anl.gov> on behalf of Allen, Benjamin S. <bsallen(a)alcf.anl.gov> Sent: Wednesday, May 4, 2016 11:02 PM To: Kumaran, Kalyan Cc: intel-nda(a)lists.jlse.anl.gov Subject: Re: [JLSE-Intel-NDA] ESP - KNL00 questions Yea it is, https://lists.jlse.anl.gov/mailman/private/intel-nda/. Ben > On May 4, 2016, at 3:57 PM, Kumaran, Kalyan <kumaran(a)alcf.anl.gov> wrote: > > I hope the list is being archived and folks can search the archives! We should include the link to the archives in the introductory email. > > -Kumar > > >> On May 4, 2016, at 3:08 PM, Allen, Benjamin S. <bsallen(a)alcf.anl.gov> wrote: >> >> Hi Fabien, >> >> Answered in-line below: >> >>> For those of you who dont know me, I am Fabien Delalondre from EPFL working on brain simulation in Switzerland. I have just got access to knl00 as part of the ALCF Early Science Program. I have a few questions related to KNL00. I apologize already if they sound like silly questions ... Appreciate all the help I can get ... >>> >>> 0) Is there any wiki accessible where users would add interesting facts/experience/general documentation etc ? >> >> Nothing setup, but we could if there's interest from the group. >> >>> 1) What is the recommended placeholder for the files to be written to ? Should I use home ? What is the file system/expected bandwidth on this file system >> >> Use the local SSD for any temp files. This is just a Intel DC S3710 SATA 200GB. /home is fine, but don't expect it to be fast, nor rely on it for any I/O sensitive work. >> >>> 2) Is there a possibility to change the card configuration from user space ? Or should I use intel-nda(a)lists.jlse.anl.gov mailing list every time I want to ask a different configuration ? >> >> Assume when you say card you're referring to the KNL system itself, and I'm assuming when you say configuration you're asking about MCDRAM's mode and model. If so yes this is possible if needed. It requires BIOS changes, but we have a utility to change those settings in Linux. Changes are applied on reboot. >> >>> 3) When the /soft/scheduling/knl/knl.txt text says knl00 node status: in use by William Scullin until 11:30 am CT 4 May 2016 - e-mail to to preempt running benchmark. Can I still use the system or is there a policy defined ? How can I "reserve" the KNL ? Should I send an email to the mailing list ? With the time difference between the US and Chicago I could use it every night in Chicago :-) >> >> No you cannot use the system if someone has listed that they're using it in that file. You can of course email this list asking to preempt whoever using it if you think you need to. For reservations just add text to the bottom of the same file stating the reservation date, start and end times. >> >>> 4) I understand there is no scheduler on this system up to now. Please let me know if this is not correct. >> >> Correct no scheduler. >> >>> 5) Can you point me to someone who has lots of experience with VTune on KNL ? Vitali was mentioning to me it does not produce the results he expected. I d like to know more about this. Any other profiling tool you would recommend ? Any documentation available on performance counters available ? >> >> We still don't have access to Vtune 2016's special KNL add-on, so we're a bit out of luck so far. As Kevin said, we're at the forefront of KNL knowledge. Unsure if Advisor is any better. >> >>> 6) Is there a debugger available on this node ? If the answer is yes, which one ? Can I use run a vnc server to connect and improve response time of the graphical interfaces ? >> >> No VNC. Use X11 forwarding. >> >>> 7) Is memkind/autoHBM installed on the system (http://www.nersc.gov/users/computational-systems/cori/application-porting-a… >>> Any recommendation regarding using or not using them ? >> >> Yea they're installed. >> >> >> >> >> Ben Allen >> >> HPC Systems Administrator >> ALCF Operations - Joint Laboratory for Systems Evaluation >> Argonne National Laboratory >> Email: bsallen(a)alcf.anl.gov >> Office: (630) 252-0554 >> >> >> >> >> _______________________________________________ >> Intel-nda mailing list >> Intel-nda(a)lists.jlse.anl.gov >> https://lists.jlse.anl.gov/mailman/listinfo/intel-nda > _______________________________________________ Intel-nda mailing list Intel-nda(a)lists.jlse.anl.gov https://lists.jlse.anl.gov/mailman/listinfo/intel-nda

1 0