August 2016 - Intel-nda - mailman3.cels.anl.gov

[JLSE-Intel-NDA] Fwd: User level KNL memory mode checker
by Kumaran, Kalyan 31 Aug '16

31 Aug '16

Begin forwarded message: From: "Nam, Hai Ah" <hnam(a)lanl.gov<mailto:[email protected]>> Subject: FW: User level KNL memory mode checker Date: August 31, 2016 at 12:44:44 PM CDT To: Jack Deslippe <jrdeslippe(a)lbl.gov<mailto:[email protected]>>, Kalyan Kumaran <kumaran(a)anl.gov<mailto:[email protected]>> Something our admin wrote to help identify the flavor of node. Thought it would be helpful to share with your user communities. --------------------------------------- Hai Ah Nam, Ph.D. Scientist, CCS-2 Los Alamos National Laboratory Phone: (505) 667-3768 Email: hnam(a)lanl.gov<mailto:[email protected]> From: rtc <rtc(a)lanl.gov<mailto:[email protected]>> Date: Tuesday, August 30, 2016 at 2:57 PM To: "Peltz Jr., Paul" <peltz(a)lanl.gov<mailto:[email protected]>> Cc: "Vernon, Louis James" <vernon(a)lanl.gov<mailto:[email protected]>>, Hai Ah Nam <hnam(a)lanl.gov<mailto:[email protected]>>, "Berry, Mike" <mrberry(a)lanl.gov<mailto:[email protected]>> Subject: Re: User level KNL memory mode checker Paul, this is great, thank you! Will update the docs. rob _______________________ Robert Cunningham HPC Consulting Team, HPC-DEV Los Alamos National Security, LLC http://hpc.lanl.gov<http://hpc.lanl.gov/> 505-665-4444 x05704 On Aug 30, 2016, at 1:22 PM, Peltz Jr., Paul <peltz(a)lanl.gov<mailto:[email protected]>> wrote: I heard during part of the boot camp today that users wanted to verify memory modes a node is configured in. I’ve adapted some simple bash scripts to do this when I do my verification testing. There are two ways to do this easily. First I have a function in my .bashrc file for interactive sessions. function knl_mode() { if [ -f /.hwinfo.cray ] ; then CNAME=`/usr/bin/cat /proc/cray_xt/cname` NODE=`echo -n $CNAME | tail -c1` NID=`/usr/bin/cat /proc/cray_xt/nid` NUMA=`/usr/bin/grep "numa_cfg\[$NODE" /.hwinfo.cray | awk -F "=" '{print $2}'` MCDRAM=`/usr/bin/grep "mcdram_cfg\[$NODE" /.hwinfo.cray| awk -F "=" '{print $2}'` if [ $NUMA ] ; then echo $NID "CLE_"$NUMA"_"$MCDRAM else echo "This is not a KNL compute node." fi else echo "This can only be used on a compute node." fi } And the same script for non-interactive sessions in my home area knl_mode.sh: #!/bin/bash if [ -f /.hwinfo.cray ] ; then CNAME=`/usr/bin/cat /proc/cray_xt/cname` NODE=`echo -n $CNAME | tail -c1` NID=`/usr/bin/cat /proc/cray_xt/nid` NUMA=`/usr/bin/grep "numa_cfg\[$NODE" /.hwinfo.cray | awk -F "=" '{print $2}'` MCDRAM=`/usr/bin/grep "mcdram_cfg\[$NODE" /.hwinfo.cray| awk -F "=" '{print $2}'` if [ $NUMA ] ; then echo $NID "CLE_"$NUMA"_"$MCDRAM else echo "This is not a KNL compute node." fi else echo "This can only be used on a compute node." fi CSH versions would need to be made, but this is what I’m doing currently and you might want to add this option to your documentation. Other options such as apstat -M don’t give as useful info. The memory mode isn’t obvious with this. Thanks, Paul =========================== Paul Peltz Jr HPC-DES Systems Integration Los Alamos National Laboratory ===========================

1 0

[JLSE-Intel-NDA] Fwd: parallelism training + DAAL, and Optimized Python
by Kumaran, Kalyan 31 Aug '16

31 Aug '16

Please contact Chris if you would like to attend and run into any issues. Begin forwarded message: From: "Allison, CM" <cm.allison(a)intel.com<mailto:[email protected]>> Subject: parallelism training + DAAL, and Optimized Python Argonne team - You (and your Chicago-area colleagues) are invited to participate in this free training we’ve organized with the Univ of Chicago: http://colfaxresearch.com/events/uchicago-16-10/ If you have questions, please contact me. I have coordinated this opportunity with Birali. You and your colleagues can find details and register at the above URL. Best regards, - Chris Chris Allison Research Computing, Field Intel Corporation 816-835-1106

1 0

Re: [JLSE-Intel-NDA] Update XPPSL (MPSP) to 1.4.1
by Hammond, Jeff R 25 Aug '16

25 Aug '16

The free list sort patch for Linux is really important. Please test it like crazy and let us know if there any major regressions. Jeff Sent from my iPhone > On Aug 25, 2016, at 10:56 AM, Allen, Benjamin S. <bsallen(a)alcf.anl.gov> wrote: > > Intel's XPPSL (MPSP) has been updated to 1.4.1. The KNL nodes will get the updates on their next reboot. > > Intel included a new kernel module to help deal with the cached memory mode performance degradation over time issue named zonesort_module. See page 21 in the attached doc. > > Note, I plan on loading this module and running "echo 0 > /sys/kernel/zone_sort_free_pages/nodeid" on job start where the node is not rebooted, and the mode is cached. If someone is interested in testing this out before I get to implementing it, let me know. > > Interesting known issues: > > - The hbmalloc library does not guarantee to allocate memory from the closest MCDRAM node when SNC-2 or SNC-4 Memory Mode is used. > > Ben > > > <xppsl_user_guide.pdf> > _______________________________________________ > Intel-nda mailing list > Intel-nda(a)lists.jlse.anl.gov > https://lists.jlse.anl.gov/mailman/listinfo/intel-nda

1 0

[JLSE-Intel-NDA] KNL BIOS Updates
by Allen, Benjamin S. 25 Aug '16

25 Aug '16

Fresh set of BIOS, BMC, and FRUSDR updates for the Adam's Pass KNL machines. They're not yet installed. Ben

1 0

[JLSE-Intel-NDA] Update XPPSL (MPSP) to 1.4.1
by Allen, Benjamin S. 25 Aug '16

25 Aug '16

Intel's XPPSL (MPSP) has been updated to 1.4.1. The KNL nodes will get the updates on their next reboot. Intel included a new kernel module to help deal with the cached memory mode performance degradation over time issue named zonesort_module. See page 21 in the attached doc. Note, I plan on loading this module and running "echo 0 > /sys/kernel/zone_sort_free_pages/nodeid" on job start where the node is not rebooted, and the mode is cached. If someone is interested in testing this out before I get to implementing it, let me know. Interesting known issues: - The hbmalloc library does not guarantee to allocate memory from the closest MCDRAM node when SNC-2 or SNC-4 Memory Mode is used. Ben

1 0

Re: [JLSE-Intel-NDA] Location of Vtune?
by Allen, Benjamin S. 24 Aug '16

24 Aug '16

Updated the wiki. Note, source /soft/compilers/intel/vtune_amplifier_xe/amplxe-vars.sh Points to the latest version. Ben > On Aug 24, 2016, at 9:36 AM, Delalondre Fabien Jonathan <fabien.delalondre(a)epfl.ch> wrote: > > Not sure my last email went through ... > So, the command is > source /soft/compilers/intel/vtune_amplifier_xe_2016.4.0.470476/amplxe-vars.sh > > This will put it in your path. > Please make sure to update the wiki accordingly. > > Best, > > Fabien > ________________________________________ > From: intel-nda-bounces(a)lists.jlse.anl.gov <intel-nda-bounces(a)lists.jlse.anl.gov> on behalf of Tom LeCompte <lecompte(a)anl.gov> > Sent: Wednesday, August 24, 2016 4:26 PM > To: intel-nda(a)jlse.anl.gov > Subject: [JLSE-Intel-NDA] Location of Vtune? > > Hi everyone- > > I have an application I would like to profile. The Wiki says "Option 1" > is to use Vtune and gives instructions > > execute the following command via Cobalt submission script > > $ amplxe-cl -c memory-access ... > > This givea a command not found for amplxe-cl. Is there something I need > to set up? > > Cheers, > > Tom > > > _______________________________________________ > Intel-nda mailing list > Intel-nda(a)lists.jlse.anl.gov > https://lists.jlse.anl.gov/mailman/listinfo/intel-nda > _______________________________________________ > Intel-nda mailing list > Intel-nda(a)lists.jlse.anl.gov > https://lists.jlse.anl.gov/mailman/listinfo/intel-nda

1 0

[JLSE-Intel-NDA] Intel Parallel Studio 2017
by Allen, Benjamin S. 24 Aug '16

24 Aug '16

Intel provided us an early copy of the final release of Parallel Studio 2017. It's under NDA until officially released. As such it's installed under /soft/restricted/intel_nda. Currently the /soft/restricted mount is only available on the KNL nodes. Let me know if it is needed elsewhere. Ben

1 0

Re: [JLSE-Intel-NDA] Location of Vtune?
by Delalondre Fabien Jonathan 24 Aug '16

24 Aug '16

Not sure my last email went through ... So, the command is source /soft/compilers/intel/vtune_amplifier_xe_2016.4.0.470476/amplxe-vars.sh This will put it in your path. Please make sure to update the wiki accordingly. Best, Fabien ________________________________________ From: intel-nda-bounces(a)lists.jlse.anl.gov <intel-nda-bounces(a)lists.jlse.anl.gov> on behalf of Tom LeCompte <lecompte(a)anl.gov> Sent: Wednesday, August 24, 2016 4:26 PM To: intel-nda(a)jlse.anl.gov Subject: [JLSE-Intel-NDA] Location of Vtune? Hi everyone- I have an application I would like to profile. The Wiki says "Option 1" is to use Vtune and gives instructions execute the following command via Cobalt submission script $ amplxe-cl -c memory-access ... This givea a command not found for amplxe-cl. Is there something I need to set up? Cheers, Tom _______________________________________________ Intel-nda mailing list Intel-nda(a)lists.jlse.anl.gov https://lists.jlse.anl.gov/mailman/listinfo/intel-nda

1 0

Re: [JLSE-Intel-NDA] Location of Vtune?
by Tom LeCompte 24 Aug '16

24 Aug '16

Thanks. (And to you too, Scott) I gave up my confluence access when we ran out of access, though, so someone else will need to update it. On 8/24/2016 9:33 AM, Delalondre Fabien Jonathan wrote: > source /soft/compilers/intel/vtune_amplifier_xe_2016.4.0.470476/amplxe-vars.sh > Can you please add this to the confluence page instructions ? > ________________________________________ > From: intel-nda-bounces(a)lists.jlse.anl.gov <intel-nda-bounces(a)lists.jlse.anl.gov> on behalf of Tom LeCompte <lecompte(a)anl.gov> > Sent: Wednesday, August 24, 2016 4:26 PM > To: intel-nda(a)jlse.anl.gov > Subject: [JLSE-Intel-NDA] Location of Vtune? > > Hi everyone- > > I have an application I would like to profile. The Wiki says "Option 1" > is to use Vtune and gives instructions > > execute the following command via Cobalt submission script > > $ amplxe-cl -c memory-access ... > > This givea a command not found for amplxe-cl. Is there something I need > to set up? > > Cheers, > > Tom > > > _______________________________________________ > Intel-nda mailing list > Intel-nda(a)lists.jlse.anl.gov > https://lists.jlse.anl.gov/mailman/listinfo/intel-nda

1 0

Re: [JLSE-Intel-NDA] Location of Vtune?
by Scott Parker 24 Aug '16

24 Aug '16

To setup the Vtune environment the simplest way is to use the command : source /soft/compilers/intel/vtune_amplifier_xe_2016.4.0.470476/amplxe-vars.sh I believe you should be able to just add it to your Cobalt script before the amplxe-cl command. -Scott On 8/24/16 9:26 AM, Tom LeCompte wrote: > Hi everyone- > > I have an application I would like to profile. The Wiki says "Option > 1" is to use Vtune and gives instructions > > execute the following command via Cobalt submission script > > $ amplxe-cl -c memory-access ... > > This givea a command not found for amplxe-cl. Is there something I > need to set up? > > Cheers, > > Tom > > > _______________________________________________ > Intel-nda mailing list > Intel-nda(a)lists.jlse.anl.gov > https://lists.jlse.anl.gov/mailman/listinfo/intel-nda

1 0