June 2016 - Intel-nda - mailman3.cels.anl.gov

Re: [JLSE-Intel-NDA] Help on knl00 segfault
by Harms, Kevin 16 Jun '16

16 Jun '16

Steve, First thing, try the latest compiler. source /soft/compilers/intel_beta/compilers_and_libraries/linux/bin/compilervars.sh The 2017 beta has explicit KNL support. Second, have you tried running it in gdb? kevin >I am trying to run a Fortran program that exercises some of my computational >kernels. The program does not use MPI, just OpenMP. I have compiled >it for a small case and that runs fine and gives the correct results. >However when I recompile it for a realistic cases (the same source >is used, only a bunch of parameter statements are different), it >segfaults at line 13 of the main program: > >[knl00 tests] ../mc/x.testwf.12c-11_states-pn c12_11states-pn.in >forrtl: severe (174): SIGSEGV, segmentation fault occurred >Image PC Routine Line Source > >libintlc.so.5 00007F569D060C05 Unknown Unknown Unknown >libintlc.so.5 00007F569D05E827 Unknown Unknown Unknown >libifcoremt.so.5 00007F569DEE1962 Unknown Unknown Unknown >libifcoremt.so.5 00007F569DEE17B6 Unknown Unknown Unknown >libifcoremt.so.5 00007F569DE36CAD Unknown Unknown Unknown >libifcoremt.so.5 00007F569DE48FAD Unknown Unknown Unknown >libpthread.so.0 00007F569F2D7100 Unknown Unknown Unknown >x.testwf.12c-11_s 00000000004030F0 MAIN__ 13 testwf.f >x.testwf.12c-11_s 0000000000402C0E Unknown Unknown Unknown >libc.so.6 00007F569CAA3B15 Unknown Unknown Unknown >x.testwf.12c-11_s 0000000000402B19 Unknown Unknown Unknown >Aborted (core dumped) >[knl00 tests] > > >Line 13 is the very beginning of declarations, not an executable statement. >It is the first non-comment line and is: > > use random2 > >random2 is a module that defines some subroutines. It is not changed >by the parameter statements that are different for the working and failing >cases. > >Here are the working and failing executable sizes: > > text data bss dec hex filename >2459352 124772 24979752 27563876 1a49764 x.testwf.4he-pn >2192689 92004 623277384 625562077 254951dd x.testwf.12c-11_states-pn > >Here is how the main program is compiled and module linked: > >ifort -Imods.libgen -FR -g -O0 -qopenmp -lpthread -qopenmp-report -r8 -WB -traceback -fpe3 -I../common/ -I../mc/ -fpp -DSPIN_SUF= -DPARAMNUC=nclude/paramnuc.12c-11_states-pn -DM0_DIV=1 -DDOCASE=0 -DPARAMCASE=nclude/paramcase.12c-11_states-pn -c -g testwf.f > >ifort -g -mcmodel=medium -traceback -fpe3 -qopenmp -lpthread -qopenmp-report -o x.testwf.12c-11_states-pn testwf.o do_adlb_no.o dmem_fake.o adlb_fake.o \ > ../common/wall_second-nompi.o ../common/stopnompi.o ../common/mpi_wtime_fake.o -L . -l12c-11_states-pn -lgen -l12c-11_states-pn -L ../common/ -lcomgen enable_abort_fake.o -lsvml > > >[knl00 mc] which ifort >/soft/compilers/intel/bin/ifort > > >This is in ~b21541/gfmc/mc . > >This test case runs fine on blues. > >Any suggestions will be gratefully received. > >Steve >-- >Steven C. Pieper, Senior Physicist SO: spieper(a)anl.gov >Argonne National Laboratory, Physics Division, Bldg. 203, Argonne, IL 60439 >Phone: 630-252-4232 Fax -6008 >Secretary, Debra Beres, -4100

1 0

[JLSE-Intel-NDA] IPMI access
by Alvaro 16 Jun '16

16 Jun '16

Hi , Could I have IPMI access in jlse? my user is : vama Thanks -- -Alvaro http://web.alcf.anl.gov/~vama/

1 0

[JLSE-Intel-NDA] Help on knl00 segfault
by Steven C. Pieper 16 Jun '16

16 Jun '16

I am trying to run a Fortran program that exercises some of my computational kernels. The program does not use MPI, just OpenMP. I have compiled it for a small case and that runs fine and gives the correct results. However when I recompile it for a realistic cases (the same source is used, only a bunch of parameter statements are different), it segfaults at line 13 of the main program: [knl00 tests] ../mc/x.testwf.12c-11_states-pn c12_11states-pn.in forrtl: severe (174): SIGSEGV, segmentation fault occurred Image PC Routine Line Source libintlc.so.5 00007F569D060C05 Unknown Unknown Unknown libintlc.so.5 00007F569D05E827 Unknown Unknown Unknown libifcoremt.so.5 00007F569DEE1962 Unknown Unknown Unknown libifcoremt.so.5 00007F569DEE17B6 Unknown Unknown Unknown libifcoremt.so.5 00007F569DE36CAD Unknown Unknown Unknown libifcoremt.so.5 00007F569DE48FAD Unknown Unknown Unknown libpthread.so.0 00007F569F2D7100 Unknown Unknown Unknown x.testwf.12c-11_s 00000000004030F0 MAIN__ 13 testwf.f x.testwf.12c-11_s 0000000000402C0E Unknown Unknown Unknown libc.so.6 00007F569CAA3B15 Unknown Unknown Unknown x.testwf.12c-11_s 0000000000402B19 Unknown Unknown Unknown Aborted (core dumped) [knl00 tests] Line 13 is the very beginning of declarations, not an executable statement. It is the first non-comment line and is: use random2 random2 is a module that defines some subroutines. It is not changed by the parameter statements that are different for the working and failing cases. Here are the working and failing executable sizes: text data bss dec hex filename 2459352 124772 24979752 27563876 1a49764 x.testwf.4he-pn 2192689 92004 623277384 625562077 254951dd x.testwf.12c-11_states-pn Here is how the main program is compiled and module linked: ifort -Imods.libgen -FR -g -O0 -qopenmp -lpthread -qopenmp-report -r8 -WB -traceback -fpe3 -I../common/ -I../mc/ -fpp -DSPIN_SUF= -DPARAMNUC=nclude/paramnuc.12c-11_states-pn -DM0_DIV=1 -DDOCASE=0 -DPARAMCASE=nclude/paramcase.12c-11_states-pn -c -g testwf.f ifort -g -mcmodel=medium -traceback -fpe3 -qopenmp -lpthread -qopenmp-report -o x.testwf.12c-11_states-pn testwf.o do_adlb_no.o dmem_fake.o adlb_fake.o \ ../common/wall_second-nompi.o ../common/stopnompi.o ../common/mpi_wtime_fake.o -L . -l12c-11_states-pn -lgen -l12c-11_states-pn -L ../common/ -lcomgen enable_abort_fake.o -lsvml [knl00 mc] which ifort /soft/compilers/intel/bin/ifort This is in ~b21541/gfmc/mc . This test case runs fine on blues. Any suggestions will be gratefully received. Steve -- Steven C. Pieper, Senior Physicist SO: spieper(a)anl.gov Argonne National Laboratory, Physics Division, Bldg. 203, Argonne, IL 60439 Phone: 630-252-4232 Fax -6008 Secretary, Debra Beres, -4100

1 0

[JLSE-Intel-NDA] VTune: Memory Access
by Vitali A. Morozov 08 Jun '16

08 Jun '16

Hello, I have tried to go through the "Memory Access" feature of the NDA version of VTune and found that 1) it works! 2) it works as expected! 3) it is very simple to use if everything is configured right. The down side is two fold: too much data is collected due to tracing and the GUI provides too much unnecessary data - hard to find what you really want. I have put a step-by-step instruction on how to use it into https://wiki.jlse.anl.gov/display/knl/Intel+KNL I also found that not all objects, that caused excessive L2 cache misses can help increase performance when being mapped to MCDRAM. I have seen several cases when the object does cause L2 cache misses but the true performance bottleneck is the piepieline dependency stalls; therefore placing the object to MCDRAM does not give any. As always, this is just a tool, the judgement is yours. Vitali

1 0

[JLSE-Intel-NDA] Fwd: WW23-24 Xeon Phi Biweekly Product Update
by Kumaran, Kalyan 07 Jun '16

07 Jun '16

Begin forwarded message: From: "Fromkin, Russ" <russ.fromkin(a)intel.com<mailto:[email protected]>> Subject: FW: WW23-24 Xeon Phi Biweekly Product Update Date: June 7, 2016 at 9:03:47 PM CDT To: 'Susan Coghlan' <smc(a)anl.gov<mailto:[email protected]>>, "'Kumaran, Kalyan'" <kumaran(a)alcf.anl.gov<mailto:[email protected]>>, "'hfinkel(a)anl.gov<mailto:[email protected]>'" <hfinkel(a)anl.gov<mailto:[email protected]>> From: Linthwaite, Kirsten A Sent: Tuesday, June 7, 2016 6:17 PM Subject: WW23-24 Xeon Phi Biweekly Product Update [cid:[email protected]] Xeon Phi™ Home Page<http://www.intel.com/content/www/us/en/processors/xeon/xeon-phi-detail.html> | Xeon Phi™ App Catalogs<https://software.intel.com/en-us/mic-developer/app-catalogs> | Developer Zone<http://software.intel.com/en-us/mic-developer> | IPCCs<http://software.intel.com/en-us/articles/intel-parallel-computing-centers> | Product Overview (IBL)<http://www.intel.com/cd/edesign/library/asmo-na/eng/538824.htm> [cid:[email protected]] Spotlight Knights Landing Launch: Less than 2 weeks to the launch of KNL at ISC’16 in Frankfurt! See below for a summary of different events at launch in support of KNL and code modernization. Visit the Intel Booth throughout the event to see KNL demos, hear a variety of talks in the Collaboration Hub, and access a DAP system at the Expert “Bar”. From the Intel booth you can also kick off your self-guided Fellow Traveler Tour where you can visit all the Intel partners to learn more about the broad Intel ecosystem and systems available. KNL “Official Launch” will be during Raj Hazra’s keynote address at 6pm on 6/20. ISC ’16 IXPUG BoF sessions: Gearing Up Application Performance for Intel Xeon Phi (KNL) Supercomputers: • Location: Frankfurt am Main, Germany Date: Wednesday, June 22, 2016, 8:30-9:30 am (Frankfurt time) Venue: Substanz 1+2, Forum • Agenda to include: Lightning talks presenting data, including KNL data; Community discussion; Working groups highlights; Call for participation in IXPUG events and the broader IXPUG community • Submissions: Please start making submissions now. Submission URL: EasyChair IXPUG Workshop ISC16<https://easychair.org/conferences/?conf=ixpugisc16>. Please select BoF only, in the Topics selection box. Presentation template: IXPUG PowerPoint template<https://drive.google.com/file/d/0B9D5EnxRqcaZUV9nNExZT0hOWFE/view?usp=shari…> - we've found that those who include content in this template have the smoothest reviews IXPUG Workshop: "Application Performance on Intel Xeon Phi– Being Prepared for KNL and Beyond": The workshop will bring together software developers and technology experts to share challenges, experiences and best-practice methods for the optimization of HPC workloads on the Intel Xeon Phi. The workshop will cover application performance and scalability challenges at all levels - from single processor, to moderately-scaled cluster, up to large HPC configurations with many Xeon Phi devices. The keynote will present recent information about the KNL processor. The submitted talks cover optimization and scalability topics in real-world HPC applications, e.g. data layouts and code restructuring for efficient SIMD operation, work distribution and thread management. Aspects related to KNL features (e.g. high-bandwidth memory) are of particular interest. The usability of tools for development, debugging and performance analysis will be covered. A planned panel session provides an opportunity to discuss optimization strategies for Phi and to provide feedback to the toolchain developers. • Important Note: The workshop is held in conjunction with the ISC'16 in Frankfurt (Main). For attending the workshop, you have to register for ISC Workshops (registration is open since March 1st). More information is on the ISC'16 Conference site<http://www.isc-hpc.com/>. • For more info visit: https://www.ixpug.org/events/ixpug-isc-2016 More Launch Activities: The Texas Advanced Computing Center (TACC) at the University of Texas at Austin, in partnership with Intel Corporation, is proud to announce a special tutorial event as part of the Intel Xeon Phi User group Meeting (IXPUG) at ISC16 in Frankfurt, Germany. This FREE event will provide hands-on experience on the latest Intel product, the second generation Intel Xeon Phi, also known as Knights Landing (KNL) and access to the TACC Stampede system. Attendees to this half-day event will be some of the first researchers in the world to have access to the KNL platform. The event will be June 19th from 8am-12pm at the Frankfurt Hotel Marriott. You can register for the event here<https://portal.tacc.utexas.edu/training/#/session/27>. Be sure to follow @IntelHPC on Twitter throughout the event for more updates and event info! More News FREE TRAINING: Colfax Research continues to offer their “Developer’s Guide to the Next-Generation Intel® Xeon Phi™ Processor” webinars. This is a one-hour FREE webinar that is held at various times to accommodate developers worldwide. The webinars are both an introduction to KNL and a programmer’s guide to KNL. You can find the times and register on the Colfax website<http://colfaxresearch.com/knl-webinar/>. Colfax has also opened registration<https://attendee.gotowebinar.com/register/5359149790859160067> for their June Hands-On Workshop (HOW), which focuses on parallel programming and optimization for Intel Architecture. The course runs for 3 weeks and provides 20 hours of web-based instruction and remote access to training servers. The course begins June 20th and runs through July 1st. Training Highlight FREE Intel® Xeon Phi™ Code Modernization Workshops Upcoming Dates Location Link June 8-9 Toulouse, France Register<http://www.inteldevconference.com/events/toulouse/> June 15-16 Zurich, Switzerland Register<http://www.inteldevconference.com/events/zurich-hpc-code-modernization/> July 6-7 Cambridge, England Register<http://www.inteldevconference.com/events/cambridge-united-kingdom/> Intel® Software Development Conference June 28-29 Tel Aviv, Israel Register<http://ildevcon.com/> Knights Landing Key Information Software Resources • Developer Access Program (DAP)<http://dap.xeonphi.com/#platformspecs> • NEW! Order DAP System<http://dap.xeonphi.com/> • Knights Landing – What’s Public<https://software.intel.com/en-us/articles/what-disclosures-has-intel-made-a…> • Intel® Parallel Studio XE Tools and Libraries<https://software.intel.com/en-us/free_tools_and_libraries> • Intel® MIC Developer Zone<http://software.intel.com/mic-developer> • Intel® Modern Code<https://software.intel.com/en-us/modern-code> • Intel® Xeon Phi™ Software Code Recipes<http://software.intel.com/en-us/articles/code-recipes-for-intelr-xeon-phitm…> • Intel® Xeon Phi™ SW Tools and Libraries<http://software.intel.com/en-us/articles/intel-and-third-party-tools-and-li…> • Intel® Software Tools Technical Webinar Series<http://software.intel.com/en-us/articles/intel-software-tools-technical-web…> • Intel® MIC Technical Update (CDI doc 473027) – CNDA<http://www.intel.com/cd/edesign/library/asmo-na/eng/473027.htm> • Intel® Xeon Phi™ Coprocessor – Applications and Solutions Catalog<https://software.intel.com/en-us/articles/intel-xeon-phi-coprocessor-applic…> • Intel® Xeon Phi™ Processor Product Family Early Release Software and Tools<https://software.intel.com/en-us/xeon-phi-nda/tools> (request access to the site here<https://software.intel.com/en-us/form/intel-xeon-phi-nda-site-access-request>) Knights Corner Collateral • Intel® Xeon Phi™ Public Performance Data<http://www.intel.com/content/www/us/en/benchmarks/server/xeon-phi/xeon-phi-…> • Accelerating Energy Exploration with Intel® Xeon Phi™ Coprocessors Case Study<https://software.intel.com/sites/default/files/managed/f1/e6/Intel-Cirrasca…> • Taking Oil and Gas Exploration to the Next Level Case Study<https://www-ssl.intel.com/content/www/us/en/high-performance-computing/dug-…> • SGI® Unveiling the Early Universe: Optimizing Cosmology Workloads for Intel® Xeon Phi™ Coprocessors Case Study<http://www.sgi.com/pdfs/4523.pdf> • OEM Platform Supporting Intel® Xeon Phi™ – CNDA<http://www.intel.com/cd/edesign/library/asmo-na/eng/540160.htm> Copyright © 2016 Intel Corporation. All rights reserved. Legal<http://targetmailer.intel.com/TMService/Redir.aspx?ID=1266509>. Privacy.<http://targetmailer.intel.com/TMService/Redir.aspx?ID=1266510> Intel Confidential – For Use under CNDA Only *Other names and brands may be claimed as the property of others.

1 0

Re: [JLSE-Intel-NDA] Method for Determining Current Memory and Cluster Mode
by Allen, Benjamin S. 06 Jun '16

06 Jun '16

Digging into this a bit more this is actually generated by: $ sudo /soft/restricted/intel-vtune-nda/vtune_amplifier_xe_2016.3.0.463186/sepdk/src/read_dmisysfs.py ClusterMode=Quadrant MemoryMode=Flat MCDRAMCache=0 The "sep" command below is just printing the contents of /opt/intel/sep4_0/dmi_config.txt, which is generated on boot by insmod-sep using the above python. For now /opt/intel/sep4_0/dmi_config.txt is readable by normal users and since at least on knl00 it is updated on every boot, it should be safe place to find the current running config. Ben > On Jun 6, 2016, at 10:55 AM, Allen, Benjamin S. <bsallen(a)alcf.anl.gov> wrote: > > [bsallen@knl00 ~]$ /soft/restricted/intel-vtune-nda/vtune_amplifier_xe_2016.3.0.463186/bin64/sep -version > Sampling Enabling Product version: 4.0 (private) built by patbbinn on Apr 20 2016 02:39:28 > SEP User Mode Version: 4.0.0 > SEP Driver Version: 4.0.0 > PAX Driver Version: 1.0.1 > Platform type: 102 > CPU name: Intel(R) Unreleased Processor > PMU: knl > Cluster mode: Quadrant > Memory mode: Flat > > > _______________________________________________ > Intel-nda mailing list > Intel-nda(a)lists.jlse.anl.gov > https://lists.jlse.anl.gov/mailman/listinfo/intel-nda

1 0

[JLSE-Intel-NDA] Intel MPSP 1.3.2 Updates
by Allen, Benjamin S. 06 Jun '16

06 Jun '16

Will be installing this later today. Previously installed was 1.3.0. Ben

1 0

[JLSE-Intel-NDA] Method for Determining Current Memory and Cluster Mode
by Allen, Benjamin S. 06 Jun '16

06 Jun '16

[bsallen@knl00 ~]$ /soft/restricted/intel-vtune-nda/vtune_amplifier_xe_2016.3.0.463186/bin64/sep -version Sampling Enabling Product version: 4.0 (private) built by patbbinn on Apr 20 2016 02:39:28 SEP User Mode Version: 4.0.0 SEP Driver Version: 4.0.0 PAX Driver Version: 1.0.1 Platform type: 102 CPU name: Intel(R) Unreleased Processor PMU: knl Cluster mode: Quadrant Memory mode: Flat

1 0

Re: [JLSE-Intel-NDA] !DIR$ ATTRIBUTES FASTMEM in cache mode
by Vitali A. Morozov 03 Jun '16

03 Jun '16

Hello, FYI: I have confirmed with the memkind developers that the policy is broken at this moment and documentation is not reflecting the boundary behaviour of the hbw_malloc for out-of-memory conditions. Vitali On 05/03/2016 09:07 AM, Vitali A. Morozov wrote: > Thanks for reporting it, Marta - I have been distracted yesterday and > failed sending a message on this issue. > > We are facing with tho issues. > > First of, we see a non-intuitive behaviour of the default policy > HBW_POLICY_PREFERRED. As documented > > "If insufficient memory is available from the high bandwidth NUMA node > closest at allocation time, fall back to standard memory (default) > with the smallest NUMA distance." > > This brings two questions. > > What happens if there is no HBW memory on the system? The expected > answer - hbw_malloc (and its Fortran equivalent ATTRIBUTES FASTMEM) > should behave equivalently to malloc. > > What happens on KNL in cache mode, where the is no memory other then > DDR memory? The expected answer - hbw_malloc (and its Fortran > equivalent) should behave equivalently to malloc. > > Marta and I have found that on both Xeon and KNL in cache mode both > Fortran and C codes failed to allocate a small chunk of memory with > hbw_malloc interface. We think this is a bug and it need to be > reported and fixed. > > We found a workaround. Additionally to the mentioned conditional > compilation (which is not an appropriate way to fix it), the user may > define > > MEMKIND_HBW_NODE=0 > > This seems to direct all memory requests to numa node 0, and this > exactly what hbw_malloc is expected to do if it runs out of memory on > other numa nodes. > > Secondly, we seems to fail setting a policy from Fortran (not that the > policy works as follows from the first report, but still). In C, the > function "hbw_set_policy" changes the policy and the following > "hbw_get_policy" returns a new policy. In Fortran the equivalent > > ierr = hbw_set_policy( %VAL(1) ) > > returns an error and the following > > ierr = hbw_get_policy() > > returns the default policy as if the previous function is ignored. > > The correspondent codes are in JLSE servers in > /home/morozov/KNL/HSCD-intel/test5 for those of you who want to verify > the reports. > > Best, > > Vitali > > > On 05/02/2016 10:19 PM, Garcia Martinez, Marta wrote: >> Hi, >> >> Vitali and I are using FASTMEM in a subroutine and it works well in >> flat mode. However, trying to run the same test in cache mode is >> causing problems (issues with not enough memory) at runtime (not >> compilation). >> >> !DIR$ ATTRIBUTES FASTMEM :: var >> >> A turnaround for the moment is to enclose that between an if defined >> declaration like this: >> >> !DEC$ IF DEFINED(name) >> !DIR$ ATTRIBUTES FASTMEM :: var >> !DEC$ ENDIF >> >> and then compile with the variable ‘name'. >> >> Marta >> >> >> _______________________________________________ >> Intel-nda mailing list >> Intel-nda(a)lists.jlse.anl.gov >> https://lists.jlse.anl.gov/mailman/listinfo/intel-nda >

1 0

[JLSE-Intel-NDA] New KNL Machines
by Allen, Benjamin S. 02 Jun '16

02 Jun '16

An update for everyone here. We've received two additional KNL machines. They're Intel Software Development Platform (SDP) machines, which means they're not quite the same as knl00. They have "Groveport" main boards instead of Adam's Pass. Groveport refers to the Intel internal reference platform, where Adam's pass is the public (or soon to be) product. This of course means they have a different BIOS from our existing Adam's Pass board, so we're still working out how that relates to all the bugs we've fixed on the Adam's Pass via BIOS updates. It's also not clear yet how we change the MCDRAM Memory and Cluster modes. One good bit is these systems have "bin1" Intel Xeon Phi 7250 68c 1.4Ghz CPUs. We're working out if we can flash over the Groveport BIOS with an Adam's Pass BIOS or vice versa, and the other details above. I'll let you know when they're to a point where some early testing (STREAM, etc) is needed. Thanks, Ben

1 0