- Intel-nda - mailman3.cels.anl.gov

[JLSE-Intel-NDA] misleading traceback running MPI on KNL
by Steven C. Pieper 22 Jul '16

22 Jul '16

I got the following misleading traceback on KNL10. I was running a two-rank mpi job with both ranks on KNL10. The job consistantly failed with the following in stderr: 13:46:14.989 128.314 Slave 001: obsrv start loop 1 1 1 12 0 0 0 13:46:14.989 128.314 Slave 001: obsrv, psil_p 1 1 1 4096 924 forrtl: error (69): process interrupted (SIGINT) Image PC Routine Line Source libintlc.so.5 00002B92469D6621 Unknown Unknown Unknown libintlc.so.5 00002B92469D4487 Unknown Unknown Unknown Unknown 00002B9245BE0F62 Unknown Unknown Unknown libifcoremt.so.5 00002B9245BE0DB6 Unknown Unknown Unknown libifcoremt.so.5 00002B9245B3DA22 Unknown Unknown Unknown libifcoremt.so.5 00002B9245B4FA92 Unknown Unknown Unknown Unknown 00002B924395C100 Unknown Unknown Unknown libc.so.6 00002B9246D05B77 Unknown Unknown Unknown libmpi.so.12 00002B9244F2B476 Unknown Unknown Unknown Unknown 00002B924514A617 Unknown Unknown Unknown libmpi.so.12 00002B92451F020E Unknown Unknown Unknown libmpifort.so.12 00002B9244B977A8 Unknown Unknown Unknown x.mc.12c-11_state 00000000004381D8 MAIN__ 3338 master.f x.mc.12c-11_state 000000000040425E Unknown Unknown Unknown libc.so.6 00002B9246C4BB15 Unknown Unknown Unknown x.mc.12c-11_state 0000000000404169 Unknown Unknown Unknown The 3338 master.f is an mpi_probe being done on rank 0; it was started about 90 seconds before the failure. call MPI_PROBE( MPI_ANY_SOURCE, MPI_ANY_TAG, app_comm, status, ierr ) The actual cause of the failure is an openmp block being done on rank 1 in a different subroutine. The traceback gives me no hint of that and I had to painfully isolate the problem with print statements. I should have received a traceback for rank 1 instead of or in addition to the rank 0 traceback. BTW: The openmp block works fine on BG/Q. The loop works fine if I remove the !$OMP argound it. It also works fine with smaller dimensions. So this also appears to be another ifort OMP problem, but my complaint here is the traceback. Steve -- Steven C. Pieper, Senior Physicist Emeritus: spieper(a)anl.gov Argonne National Laboratory, Physics Division, Bldg. 203, Argonne, IL 60439 Phone: 630-252-4232 Fax -6008 Secretary, Debra Beres, -4100 http://www.phy.anl.gov/theory/staff/scp.html

1 0

[JLSE-Intel-NDA] Fwd: WW29-30 Xeon Phi Biweekly Product Update
by Kumaran, Kalyan 22 Jul '16

22 Jul '16

Begin forwarded message: From: Linthwaite, Kirsten A Sent: Friday, July 22, 2016 2:58 PM Subject: WW29-30 Xeon Phi Biweekly Product Update [cid:[email protected]] Xeon Phi™ Home Page<http://www.intel.com/content/www/us/en/processors/xeon/xeon-phi-detail.html> | Xeon Phi™ App Catalogs<https://software.intel.com/en-us/mic-developer/app-catalogs> | Developer Zone<http://software.intel.com/en-us/mic-developer> | IPCCs<http://software.intel.com/en-us/articles/intel-parallel-computing-centers> | Product Overview (IBL)<http://www.intel.com/cd/edesign/library/asmo-na/eng/538824.htm> [cid:[email protected]] Spotlight Now Available!: The new book "Intel® Xeon Phi™ Processor High Performance Programming, Knights Landing Edition”, by Jim Jeffers, James Reinders, and Avinash Sodani is now available. The book has three sections. The first section focuses on Knights Landing itself and dives into the architecture, the high bandwidth memory, clustering modes, and integrated fabric. The second section focuses on application programming with consideration for the scale of many-core. And the third section parallel programming in full applications with examples with notes on Knights Landing specific results and optimizations. You can find more details on the book on http://lotsofcores.com<http://lotsofcores.com/>, as well as information on ordering a copy. IXPUG: Registration is now open for the IXPUG Annual Meeting, which will be held September 19-23, 2016 at Argonne National Labs in Chicago, IL. Details on registration and how you can contribute to the agenda can be found on the IXPUG site<https://www.ixpug.org/events/ixpug-2016>. New Publication: Colfax Research recently published a new white paper titled “Intel® Python on 2nd Generation Intel® Xeon Phi™ Processors: Out-of-the-Box Performance”. This new paper reports on the value and performance for computational applications of the Intel® distribution for Python* 2017 Beta on the new Intel® Xeon Phi™ processors (formerly codenamed Knights Landing). Benchmarks of LU decomposition, Cholesky decomposition, singular value decomposition and double precision general matrix-matrix multiplication routines in the SciPy and NumPy libraries are presented, and tuning methodology for use with high-bandwidth memory (HBM) is laid out. Developer Access Program: Just a reminder that all DAP customers receive access to Intel Parallel Studio XE 2016 Cluster Edition. We are seeing some great performance results from DAP system users, such as the STREAM performance that was captured at the Advanced Computing Research Centre at the University of Bristol (see below): [cid:[email protected]] More News FREE TRAINING: Colfax Research is expanding their “Hands-On Workshop (HOW)” with a new topic: “Performance Optimization for Intel® Xeon Phi™ Processor”. This 2-hour webinar provides best practices for performance optimization for Intel® Xeon Phi processor. To find out more and register for the webinar, please visit Colfax’s website<http://colfaxresearch.com/how-knl-16-07/>. Colfax is also offering a Deep Dive HOW Series that will offer 10 x 2-hour sessions including remote access to a system titled “Parallel Programming and Optimization for Intel® Architecture”. The July course runs from July 25th – August 5th. Please visit the Colfax site<http://colfaxresearch.com/how-16-07/> to register. Save the Date: Date City Country Partner 07/22/16 Seoul South Korea Dae Han Consulting 08/03/16 Sao Paulo Brazil UNESP 08/05/16 Sao Paulo Brazil UNESP 08/09/16 Denver USA Colfax International 08/12/16 Seoul South Korea Dae Han Consulting 08/19/16 Seoul South Korea Dae Han Consulting 08/26/16 Seoul South Korea Dae Han Consulting 09/02/16 Seoul South Korea Dae Han Consulting 09/09/16 Seoul South Korea Dae Han Consulting 09/20/16 Beijing PRC Paratera 09/22/16 Beijing PRC Inspur 09/23/16 Seoul South Korea Dae Han Consulting 09/25/16 Dammam Saudi Arabia Bayncore 09/27/16 Djedda Saudi Arabia Bayncore 09/30/16 Seoul South Korea Dae Han Consulting 09/TBD/16 Boston/New England USA Colfax International 09/TBD/16 Chicago USA Colfax International Knights Landing Key Information Software Resources • Intel® Xeon Phi™ Product Family Main Page<http://www.intel.com/content/www/us/en/processors/xeon/xeon-phi-detail.html> • Intel® Xeon Phi™ Processor Press Kit<https://newsroom.intel.com/press-kits/intel-isc/> • Intel® Xeon Phi™ Processor SKU Details<http://mark.intel.com/products/family/92650/Intel-Xeon-Phi-Product-Family-x…> • Intel® Xeon Phi™ Application Proof Points<https://software.intel.com/en-us/articles/intel-xeon-phi-processor-applicat…> • NEW Intel® Xeon Phi™ Public Performance Data<http://www.intel.com/content/www/us/en/benchmarks/server/xeon-phi/xeon-phi-…> • Updated! <http://www.intel.com/content/www/us/en/benchmarks/server/xeon-phi/xeon-phi-…> Developer Access Program (DAP)<http://dap.xeonphi.com/#platformspecs> • Knights Landing – Public Disclosures<https://software.intel.com/en-us/articles/what-disclosures-has-intel-made-a…> • Intel® Modern Code<https://software.intel.com/en-us/modern-code> • Intel® Xeon Phi™ Software Code Recipes<https://software.intel.com/en-us/xeon-phi/x200-processor> • Intel® Xeon Phi™ SW Tools and Libraries<https://software.intel.com/en-us/xeon-phi/x200-processor> • Machine Learning<https://software.intel.com/en-us/machine-learning> • Intel® Xeon Phi™ Processor Software<https://software.intel.com/en-us/articles/intel-xeon-phi-processor-software> • Intel® Software Tools Technical Webinar Series<http://software.intel.com/en-us/articles/intel-software-tools-technical-web…> • Intel® Xeon Phi™ Coprocessor – Applications and Solutions Catalog<https://software.intel.com/en-us/articles/intel-xeon-phi-coprocessor-applic…> • Intel® MIC Developer Zone<http://software.intel.com/mic-developer> • Intel® Xeon Phi™ Processor Product Family Early Release Software and Tools<https://software.intel.com/en-us/xeon-phi-nda/tools> (request access to the site here<https://software.intel.com/en-us/form/intel-xeon-phi-nda-site-access-request>) • Intel® Parallel Studio XE Tools and Libraries<https://software.intel.com/en-us/free_tools_and_libraries> Knights Corner Collateral • Accelerating Energy Exploration with Intel® Xeon Phi™ Coprocessors Case Study<https://software.intel.com/sites/default/files/managed/f1/e6/Intel-Cirrasca…> • Taking Oil and Gas Exploration to the Next Level Case Study<https://www-ssl.intel.com/content/www/us/en/high-performance-computing/dug-…> • SGI® Unveiling the Early Universe: Optimizing Cosmology Workloads for Intel® Xeon Phi™ Coprocessors Case Study<http://www.sgi.com/pdfs/4523.pdf> Copyright © 2016 Intel Corporation. All rights reserved. Legal<http://targetmailer.intel.com/TMService/Redir.aspx?ID=1266509>. Privacy.<http://targetmailer.intel.com/TMService/Redir.aspx?ID=1266510> Intel Confidential – For Use under CNDA Only *Other names and brands may be claimed as the property of others.

1 0

Re: [JLSE-Intel-NDA] OpenMP bug in Intel Fortran compiler
by Kumaran, Kalyan 22 Jul '16

22 Jul '16

Steve, I’m figuring out with Intel the easiest way users can report compiler bugs. -Kumar On Jul 12, 2016, at 1:55 PM, Steven C. Pieper <spieper(a)anl.gov<mailto:[email protected]>> wrote: I have attached a simplified source code and demonstration script to show an OpenMP bug in the Intel Fortran compiler. I am using /soft/compilers/intel_beta/compilers_and_libraries_2017.0.042/linux/bin/intel64/ifort The bug occurs on both the jlse login node and knl00. It occurs using two or more OpenMP threads, but not if only one thread is used. The demonstration uses no optimization (O0) but the bug also happens with optimization. It does not happen on the IBM BG/Q. The bug is a segfault at line 113 of the source file: !$OMP PARALLEL WORKSHARE cof(:,:,1:min(2,num_op_emew),:,:) = 0 cfcl(:,:,:,:) = 0.d0 !$OMP END PARALLEL WORKSHARE It is triggered by line 128 when that line is activated (in the source file it is commented): cof(:,:,2,iqcross(:),1) = cof(:,:,1,iqcross(:),1) + cof(:,:,2,iqcross(:),1) The problem is the index iqcross(:); iqcross is a 2-element array with valid values as are printed. If this line is replaced by the equivalent two lines that follow: cof(:,:,2,iqcross(1),1) = cof(:,:,1,iqcross(1),1) + cof(:,:,2,iqcross(1),1) cof(:,:,2,iqcross(2),1) = cof(:,:,1,iqcross(2),1) + cof(:,:,2,iqcross(2),1) no segfault occurs. Note also that in fact this section of the code will never be reached due to the stop at line 119 and the if at line 121 which is false. In the original large code, this second OMP section was more than 100 lines removed from the first with a large intermediate OMP section. The script first compiles and runs the source with the commented line (bug-ok.f). This shows that things work with the line removed. Then the script uses sed to uncomment the line into bug-bad.f and compiles and runs that. This causes the segfault. Steve -- Steven C. Pieper, Senior Physicist Emeritus: spieper(a)anl.gov<mailto:[email protected]> Argonne National Laboratory, Physics Division, Bldg. 203, Argonne, IL 60439 Phone: 630-252-4232 Fax -6008 Secretary, Debra Beres, -4100 http://www.phy.anl.gov/theory/staff/scp.html <makebug.txt><bug-ok.f>_______________________________________________ Intel-nda mailing list Intel-nda(a)lists.jlse.anl.gov<mailto:[email protected]> https://lists.jlse.anl.gov/mailman/listinfo/intel-nda

1 0

Re: [JLSE-Intel-NDA] Last version of compiler & Vtune working with memory analysis
by Allen, Benjamin S. 22 Jul '16

22 Jul '16

VTune 2016 Update 4 added KNL support, so the NDA'ed add-on bits are no longer needed. The compilers in 2017 and 2016 haven't been updated recently, so they should be the same as when you last worked with them. Ben > On Jul 22, 2016, at 7:15 AM, Kumaran, Kalyan <kumaran(a)alcf.anl.gov> wrote: > > You can check, but I think Release 16 is a sure bet and had full KNL support. > > -Kumar > > >> On Jul 21, 2016, at 6:00 PM, Delalondre Fabien Jonathan <fabien.delalondre(a)epfl.ch> wrote: >> >> Hi All, >> >> Sorry to ask but ... Can someone confirm which version of the compiler and Vtune work fine together for the data memory analysis. >> >> I double check Vitali's comments on the wiki but I have been told everything should work fine with the latest 2017 versions. >> >> Thanks ! >> >> Fabien >> _______________________________________________ >> Intel-nda mailing list >> Intel-nda(a)lists.jlse.anl.gov >> https://lists.jlse.anl.gov/mailman/listinfo/intel-nda > > _______________________________________________ > Intel-nda mailing list > Intel-nda(a)lists.jlse.anl.gov > https://lists.jlse.anl.gov/mailman/listinfo/intel-nda

1 0

Re: [JLSE-Intel-NDA] Last version of compiler & Vtune working with memory analysis
by Kumaran, Kalyan 22 Jul '16

22 Jul '16

You can check, but I think Release 16 is a sure bet and had full KNL support. -Kumar On Jul 21, 2016, at 6:00 PM, Delalondre Fabien Jonathan <fabien.delalondre(a)epfl.ch<mailto:[email protected]>> wrote: Hi All, Sorry to ask but ... Can someone confirm which version of the compiler and Vtune work fine together for the data memory analysis. I double check Vitali's comments on the wiki but I have been told everything should work fine with the latest 2017 versions. Thanks ! Fabien _______________________________________________ Intel-nda mailing list Intel-nda(a)lists.jlse.anl.gov<mailto:[email protected]> https://lists.jlse.anl.gov/mailman/listinfo/intel-nda

1 0

[JLSE-Intel-NDA] Last version of compiler & Vtune working with memory analysis
by Delalondre Fabien Jonathan 21 Jul '16

21 Jul '16

Hi All, Sorry to ask but ... Can someone confirm which version of the compiler and Vtune work fine together for the data memory analysis. I double check Vitali's comments on the wiki but I have been told everything should work fine with the latest 2017 versions. Thanks ! Fabien

1 0

[JLSE-Intel-NDA] changing memory model
by Steven C. Pieper 14 Jul '16

14 Jul '16

I would like to set the memory model on knl10 to quadratn, cache (or have it done for me). I have reserved it to 3:00pm today. Please give me the necessary passwords or someone do it and let me know. THanks, Steve -- Steven C. Pieper, Senior Physicist Emeritus: spieper(a)anl.gov Argonne National Laboratory, Physics Division, Bldg. 203, Argonne, IL 60439 Phone: 630-252-4232 Fax -6008 Secretary, Debra Beres, -4100 http://www.phy.anl.gov/theory/staff/scp.html

1 0

[JLSE-Intel-NDA] VTune Amplifier XE 2016 Update 4
by Vitali A. Morozov 13 Jul '16

13 Jul '16

Hello, Now that Intel Compiler Release 16 Update 4 has been released with full KNL support, the VTune Amplifier team has been updated a deck of slides with very many useful hints and techniques on how to use it. Check it out. Vitali

1 0

[JLSE-Intel-NDA] OpenMP bug in Intel Fortran compiler
by Steven C. Pieper 12 Jul '16

12 Jul '16

I have attached a simplified source code and demonstration script to show an OpenMP bug in the Intel Fortran compiler. I am using /soft/compilers/intel_beta/compilers_and_libraries_2017.0.042/linux/bin/intel64/ifort The bug occurs on both the jlse login node and knl00. It occurs using two or more OpenMP threads, but not if only one thread is used. The demonstration uses no optimization (O0) but the bug also happens with optimization. It does not happen on the IBM BG/Q. The bug is a segfault at line 113 of the source file: !$OMP PARALLEL WORKSHARE cof(:,:,1:min(2,num_op_emew),:,:) = 0 cfcl(:,:,:,:) = 0.d0 !$OMP END PARALLEL WORKSHARE It is triggered by line 128 when that line is activated (in the source file it is commented): cof(:,:,2,iqcross(:),1) = cof(:,:,1,iqcross(:),1) + cof(:,:,2,iqcross(:),1) The problem is the index iqcross(:); iqcross is a 2-element array with valid values as are printed. If this line is replaced by the equivalent two lines that follow: cof(:,:,2,iqcross(1),1) = cof(:,:,1,iqcross(1),1) + cof(:,:,2,iqcross(1),1) cof(:,:,2,iqcross(2),1) = cof(:,:,1,iqcross(2),1) + cof(:,:,2,iqcross(2),1) no segfault occurs. Note also that in fact this section of the code will never be reached due to the stop at line 119 and the if at line 121 which is false. In the original large code, this second OMP section was more than 100 lines removed from the first with a large intermediate OMP section. The script first compiles and runs the source with the commented line (bug-ok.f). This shows that things work with the line removed. Then the script uses sed to uncomment the line into bug-bad.f and compiles and runs that. This causes the segfault. Steve -- Steven C. Pieper, Senior Physicist Emeritus: spieper(a)anl.gov Argonne National Laboratory, Physics Division, Bldg. 203, Argonne, IL 60439 Phone: 630-252-4232 Fax -6008 Secretary, Debra Beres, -4100 http://www.phy.anl.gov/theory/staff/scp.html

1 0

Re: [JLSE-Intel-NDA] Adding knl01 - knl08
by Allen, Benjamin S. 11 Jul '16

11 Jul '16

Yes, fixed this. It should now be working on these new nodes. Ben > On Jul 11, 2016, at 9:32 AM, Swann Perarnau <swann(a)anl.gov> wrote: > > Ben, > > Is IPMI available on those nodes too ? I can't get a power status: > >> [perarnau@jlselogin2 ~]$ ipmitool -U jlseuser -I lanplus -H knl03.fto power status >> Error: Unable to establish IPMI v2 / RMCP+ session >> Unable to get Chassis Power Status > > Thanks, > > -- > Swann > > On Fri, Jul 08, 2016 at 09:16:28PM +0000, Allen, Benjamin S. wrote: >> knl01 through knl08 are now installed. Note, I'm keeping knl07 and knl08 reserved as we'll be using them for integration testing as we get Cobalt setup for these KNL nodes. >> >> knl01 through knl08 match knl00 specs, i.e. Xeon Phi 7210 (bin3), 192GB DDR4-2133, 200GB SSD. >> >> Also note all of the KNL nodes now have an active OmniPath interface. IPs: 172.27.6.104 - 172.27.6.114, DNS: knl<N>.jop.alcf.anl.gov. >> >> Ben >> _______________________________________________ >> Intel-nda mailing list >> Intel-nda(a)lists.jlse.anl.gov >> https://lists.jlse.anl.gov/mailman/listinfo/intel-nda

1 0