- Intel-nda - mailman3.cels.anl.gov

[JLSE-Intel-NDA] Fwd: VTune guide for KNL
by Kumaran, Kalyan 20 Jun '16

20 Jun '16

Begin forwarded message: Ben, Kumar: I may have sent you this already but, in case I haven’t – see attached. This is the VTune guide for KNL. Access to the attached file is granted via the Web, as well: https://software.intel.com/en-us/form/intel-xeon-phi-nda-site-access-request Regards, - Chris Chris Allison High Performance Computing, Field Intel Corporation 816-835-1106

1 0

[JLSE-Intel-NDA] knl09 and knl10, new OS image
by Allen, Benjamin S. 20 Jun '16

20 Jun '16

All, We now have knl09 and knl10 available. These have the "bin1" 68-core 1.4Ghz CPUs and 192GB of DDR4-2400 memory. Please note all the nodes (including knl00) are running an updated OS image. This image is re-provisioned on reboot now, so the local SSD will be wiped on repo. Please let me know if you find anything missing. Things I know that are missing is the sep4_0 kernel module for Vtune, and the status banner on login. Ben

1 0

Re: [JLSE-Intel-NDA] Help on knl00 segfault
by Steven Pieper 18 Jun '16

18 Jun '16

I will try various things when I can next get on. I don't believe AVX can be it: 1) The smaller case was compiled with the same options and works. 2) It hasn't gotten started at all, so I don't see how any AVX instructions could have been encountered. Steve Steven C. Pieper: spieper(a)anl.gov Argonne National Laboratory, Physics Division, Bldg. 203, Argonne, IL 60439 Phone: 630-252-4232 Fax -6008 Secretary, Debra Beres, -4100 On 06/17/2016 06:09 PM, Harms, Kevin wrote: > Steve, > > I have no further good ideas. You could try simplifying your compile options to see if that will help produce a runnable binary. The KNL should support AVX but maybe the first thing to try is removing all the AVX flags. If that works, you can try adding -xMIC-AVX512. > > kevin > >> Well further tests have to wait until Monday now, but >> >> 1) I used both mcmodel=medium and large -- both fail the same way >> 2) It runs on blues (haswell nodes and Intell 11 compiler) with mcmodel=medium >> 3) I forget the number of OMP threads I was using -- I think 32. >> 4) The size command does not show a huge executable (the big stuff will come >> later from allocates): >>>>>> text data bss dec hex filename >>>>>> 2459352 124772 24979752 27563876 1a49764 x.testwf.4he-pn<<<< this works >>>>>> 2192689 92004 623277384 625562077 254951dd x.testwf.12c-11_states-pn<<< this fails >> >> Steve >> >> Steven C. Pieper: spieper(a)anl.gov >> Argonne National Laboratory, Physics Division, Bldg. 203, Argonne, IL 60439 >> Phone: 630-252-4232 Fax -6008 >> Secretary, Debra Beres, -4100 >> >> On 06/16/2016 04:41 PM, Vitali A. Morozov wrote: >>> >>> I would say it has something to do with the memory model and the limitations that the large segments of the code cannot be correctly located by the OS. The problems shows up at the first executable statement. >>> >>> Two suggestions: try to set the number of OMP threads to some small number (4 or 8 to start with), and try -mcmodel large. >>> >>> Vitali >>> >>> >>> >>> On 06/16/2016 04:10 PM, Steven C. Pieper wrote: >>>> I'm not sure what you mean. random2.f is written in standard >>>> fortran 90. >>>> >>>> However the error doesn't seem to have anything to do with random2.f per se. >>>> I commented that line out and now the traceback points to the next >>>> non-comment line. So I think the segfault is happening at load time >>>> and the traceback just points to the first non comment line. >>>> >>>> I have to go now; I guess this will have to wait until next week. >>>> >>>> Thanks, >>>> Steve >>>> Steven C. Pieper, Senior Physicist SO:[email protected] >>>> Argonne National Laboratory, Physics Division, Bldg. 203, Argonne, IL 60439 >>>> Phone: 630-252-4232 Fax -6008 >>>> Secretary, Debra Beres, -4100 >>>> On 06/16/16 16:02, Harms, Kevin wrote: >>>>> Steve, >>>>> >>>>> I don�t know fortran well, is random2 part of the standard fortran or your code? >>>>> >>>>> kevin >>>>> >>>>>> OK, with mpiifort it compiles but it still fails the same way. >>>>>> Here are the results from gdb: >>>>>> >>>>>> [knl00 tests] gdb ../mc/x.testwf.12c-11_states-pn >>>>>> GNU gdb (GDB) Red Hat Enterprise Linux 7.6.1-80.el7 >>>>>> Copyright (C) 2013 Free Software Foundation, Inc. >>>>>> License GPLv3+: GNU GPL version 3 or later >>>>>> <http://gnu.org/licenses/gpl.html> <http://gnu.org/licenses/gpl.html> >>>>>> This is free software: you are free to change and redistribute it. >>>>>> There is NO WARRANTY, to the extent permitted by law. Type"show copying" >>>>>> and"show warranty" for details. >>>>>> This GDB was configured as"x86_64-redhat-linux-gnu". >>>>>> For bug reporting instructions, please see: >>>>>> <http://www.gnu.org/software/gdb/bugs/> <http://www.gnu.org/software/gdb/bugs/>... >>>>>> Reading symbols from /home/b21541/gfmc/mc/x.testwf.12c-11_states-pn...done. >>>>>> (gdb) run >>>>>> Starting program: /home/b21541/gfmc/tests/../mc/x.testwf.12c-11_states-pn >>>>>> [Thread debugging using libthread_db enabled] >>>>>> Using host libthread_db library"/lib64/libthread_db.so.1". >>>>>> >>>>>> Program received signal SIGSEGV, Segmentation fault. >>>>>> 0x00000000004030f0 in _unnamed_main$$ () at testwf.f:13 >>>>>> 13 use random2 >>>>>> Missing separate debuginfos, use: debuginfo-install glibc-2.17-106.el7_2.4.x86_64 libgcc-4.8.5-4.el7.x86_64 >>>>>> (gdb) >>>>>> >>>>>> (gdb) where >>>>>> #0 0x00000000004030f0 in _unnamed_main$$ () at testwf.f:13 >>>>>> #1 0x0000000000402c0e in main () >>>>>> #2 0x00007ffff53b7b15 in __libc_start_main () from /lib64/libc.so.6 >>>>>> #3 0x0000000000402b19 in _start () >>>>>> (gdb) >>>>>> >>>>>> >>>>>> I can come over to Bldg 240 if that seems helpful. >>>>>> >>>>>> Steve >>>>>> >>>>>> >>>>>> Steven C. Pieper, Senior Physicist SO:[email protected] >>>>>> Argonne National Laboratory, Physics Division, Bldg. 203, Argonne, IL 60439 >>>>>> Phone: 630-252-4232 Fax -6008 >>>>>> Secretary, Debra Beres, -4100 >>>>>> On 06/16/16 15:08, Luo, Ye wrote: >>>>>> >>>>>> >>>>>> Try mpiifort. >>>>>> You can verify the use of ifort by"mpiifort -show". >>>>>> Ye >>>>>> =================== >>>>>> Ye Luo, Ph.D. >>>>>> Leadership Computing Facility >>>>>> Argonne National Laboratory >>>>>> >>>>>> >>>>>> >>>>>> ________________________________________ >>>>>> From: >>>>>> intel-nda-bounces(a)lists.jlse.anl.gov<mailto:[email protected]> [intel-nda-bounces(a)lists.jlse.anl.gov] on behalf of Steven C. Pieper [spieper(a)anl.gov] >>>>>> Sent: Thursday, June 16, 2016 3:04 PM >>>>>> To: Harms, Kevin; >>>>>> intel-nda(a)lists.jlse.anl.gov<mailto:[email protected]>; Osborn, James C. >>>>>> Subject: Re: [JLSE-Intel-NDA] Help on knl00 segfault >>>>>> >>>>>> >>>>>> Hi Kevin, >>>>>> >>>>>> I did the source command and ifort gets the beta compiler OK. However mpif90 >>>>>> is using gfortran: >>>>>> >>>>>> mpif90 -FR -g -O4 -mavx -xAVX -axAVX -qopenmp -lpthread -qopenmp-report -r8 -WB -traceback -fpe3 -I../common/ -I../mc/ -fpp -DSPIN_SUF= -DPARAMNUC=nclude/paramnuc.12c-11_states-pn -DM0_DIV=1 -DDOCASE=0 -DPARAMCASE=nclude/paramcase.12c-11_states-pn >>>>>> -I../adlb/ -I../dmem/ -c kinetic.f >>>>>> gfortran: error: unrecognized command line option '-axAVX' >>>>>> gfortran: error: unrecognized command line option '-qopenmp' >>>>>> gfortran: error: unrecognized command line option '-qopenmp-report' >>>>>> gfortran: error: unrecognized command line option '-r8' >>>>>> gfortran: error: unrecognized command line option '-WB' >>>>>> gfortran: error: unrecognized command line option '-traceback' >>>>>> gfortran: error: unrecognized command line option '-fpe3' >>>>>> gfortran: error: unrecognized command line option '-fpp' >>>>>> make: *** [lib12c-11_states-pn.a(kinetic.o)] Error 1 >>>>>> [knl00 mc] which mpif90 >>>>>> /soft/compilers/intel_beta/compilers_and_libraries_2017.0.042/linux/mpi/intel64/bin/mpif90 >>>>>> >>>>>> >>>>>> Steve >>>>>> Steven C. Pieper, Senior Physicist SO:[email protected] >>>>>> Argonne National Laboratory, Physics Division, Bldg. 203, Argonne, IL 60439 >>>>>> Phone: 630-252-4232 Fax -6008 >>>>>> Secretary, Debra Beres, -4100 >>>>>> On 06/16/16 14:53, Harms, Kevin wrote: >>>>>> >>>>>> >>>>>> Oops. I forgot to put �intel64� at the end. The wiki has some useful notes on it: >>>>>> >>>>>> https://wiki.jlse.anl.gov/display/knl/Intel+KNL >>>>>> >>>>>> kevin >>>>>> >>>>>> Steve, >>>>>> >>>>>> First thing, try the latest compiler. >>>>>> >>>>>> source /soft/compilers/intel_beta/compilers_and_libraries/linux/bin/compilervars.sh >>>>>> >>>>>> The 2017 beta has explicit KNL support. Second, have you tried running it in gdb? >>>>>> >>>>>> kevin >>>>>> >>>>>> I am trying to run a Fortran program that exercises some of my computational >>>>>> kernels. The program does not use MPI, just OpenMP. I have compiled >>>>>> it for a small case and that runs fine and gives the correct results. >>>>>> However when I recompile it for a realistic cases (the same source >>>>>> is used, only a bunch of parameter statements are different), it >>>>>> segfaults at line 13 of the main program: >>>>>> >>>>>> [knl00 tests] ../mc/x.testwf.12c-11_states-pn c12_11states-pn.in >>>>>> forrtl: severe (174): SIGSEGV, segmentation fault occurred >>>>>> Image PC Routine Line Source >>>>>> >>>>>> libintlc.so.5 00007F569D060C05 Unknown Unknown Unknown >>>>>> libintlc.so.5 00007F569D05E827 Unknown Unknown Unknown >>>>>> libifcoremt.so.5 00007F569DEE1962 Unknown Unknown Unknown >>>>>> libifcoremt.so.5 00007F569DEE17B6 Unknown Unknown Unknown >>>>>> libifcoremt.so.5 00007F569DE36CAD Unknown Unknown Unknown >>>>>> libifcoremt.so.5 00007F569DE48FAD Unknown Unknown Unknown >>>>>> libpthread.so.0 00007F569F2D7100 Unknown Unknown Unknown >>>>>> x.testwf.12c-11_s 00000000004030F0 MAIN__ 13 testwf.f >>>>>> x.testwf.12c-11_s 0000000000402C0E Unknown Unknown Unknown >>>>>> libc.so.6 00007F569CAA3B15 Unknown Unknown Unknown >>>>>> x.testwf.12c-11_s 0000000000402B19 Unknown Unknown Unknown >>>>>> Aborted (core dumped) >>>>>> [knl00 tests] >>>>>> >>>>>> >>>>>> Line 13 is the very beginning of declarations, not an executable statement. >>>>>> It is the first non-comment line and is: >>>>>> >>>>>> use random2 >>>>>> >>>>>> random2 is a module that defines some subroutines. It is not changed >>>>>> by the parameter statements that are different for the working and failing >>>>>> cases. >>>>>> >>>>>> Here are the working and failing executable sizes: >>>>>> >>>>>> text data bss dec hex filename >>>>>> 2459352 124772 24979752 27563876 1a49764 x.testwf.4he-pn >>>>>> 2192689 92004 623277384 625562077 254951dd x.testwf.12c-11_states-pn >>>>>> >>>>>> Here is how the main program is compiled and module linked: >>>>>> >>>>>> ifort -Imods.libgen -FR -g -O0 -qopenmp -lpthread -qopenmp-report -r8 -WB -traceback -fpe3 -I../common/ -I../mc/ -fpp -DSPIN_SUF= -DPARAMNUC=nclude/paramnuc.12c-11_states-pn -DM0_DIV=1 -DDOCASE=0 -DPARAMCASE=nclude/paramcase.12c-11_states-pn -c -g testwf.f >>>>>> >>>>>> ifort -g -mcmodel=medium -traceback -fpe3 -qopenmp -lpthread -qopenmp-report -o x.testwf.12c-11_states-pn testwf.o do_adlb_no.o dmem_fake.o adlb_fake.o \ >>>>>> ../common/wall_second-nompi.o ../common/stopnompi.o ../common/mpi_wtime_fake.o -L . -l12c-11_states-pn -lgen -l12c-11_states-pn -L ../common/ -lcomgen enable_abort_fake.o -lsvml >>>>>> >>>>>> >>>>>> [knl00 mc] which ifort >>>>>> /soft/compilers/intel/bin/ifort >>>>>> >>>>>> >>>>>> This is in ~b21541/gfmc/mc . >>>>>> >>>>>> This test case runs fine on blues. >>>>>> >>>>>> Any suggestions will be gratefully received. >>>>>> >>>>>> Steve >>>>>> -- >>>>>> Steven C. Pieper, Senior Physicist SO:[email protected] >>>>>> Argonne National Laboratory, Physics Division, Bldg. 203, Argonne, IL 60439 >>>>>> Phone: 630-252-4232 Fax -6008 >>>>>> Secretary, Debra Beres, -4100 >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>> >>>> >>>> >>>> _______________________________________________ >>>> Intel-nda mailing list >>>> Intel-nda(a)lists.jlse.anl.gov >>>> https://lists.jlse.anl.gov/mailman/listinfo/intel-nda >>>

1 0

Re: [JLSE-Intel-NDA] Help on knl00 segfault
by Harms, Kevin 17 Jun '16

17 Jun '16

Steve, I have no further good ideas. You could try simplifying your compile options to see if that will help produce a runnable binary. The KNL should support AVX but maybe the first thing to try is removing all the AVX flags. If that works, you can try adding -xMIC-AVX512. kevin >Well further tests have to wait until Monday now, but > >1) I used both mcmodel=medium and large -- both fail the same way >2) It runs on blues (haswell nodes and Intell 11 compiler) with mcmodel=medium >3) I forget the number of OMP threads I was using -- I think 32. >4) The size command does not show a huge executable (the big stuff will come > later from allocates): >>>>> text data bss dec hex filename >>>>> 2459352 124772 24979752 27563876 1a49764 x.testwf.4he-pn <<<< this works >>>>> 2192689 92004 623277384 625562077 254951dd x.testwf.12c-11_states-pn <<< this fails > >Steve > >Steven C. Pieper: spieper(a)anl.gov >Argonne National Laboratory, Physics Division, Bldg. 203, Argonne, IL 60439 >Phone: 630-252-4232 Fax -6008 >Secretary, Debra Beres, -4100 > >On 06/16/2016 04:41 PM, Vitali A. Morozov wrote: >> >> I would say it has something to do with the memory model and the limitations that the large segments of the code cannot be correctly located by the OS. The problems shows up at the first executable statement. >> >> Two suggestions: try to set the number of OMP threads to some small number (4 or 8 to start with), and try -mcmodel large. >> >> Vitali >> >> >> >> On 06/16/2016 04:10 PM, Steven C. Pieper wrote: >>> I'm not sure what you mean. random2.f is written in standard >>> fortran 90. >>> >>> However the error doesn't seem to have anything to do with random2.f per se. >>> I commented that line out and now the traceback points to the next >>> non-comment line. So I think the segfault is happening at load time >>> and the traceback just points to the first non comment line. >>> >>> I have to go now; I guess this will have to wait until next week. >>> >>> Thanks, >>> Steve >>> Steven C. Pieper, Senior Physicist SO:[email protected] >>> Argonne National Laboratory, Physics Division, Bldg. 203, Argonne, IL 60439 >>> Phone: 630-252-4232 Fax -6008 >>> Secretary, Debra Beres, -4100 >>> On 06/16/16 16:02, Harms, Kevin wrote: >>>> Steve, >>>> >>>> I don�t know fortran well, is random2 part of the standard fortran or your code? >>>> >>>> kevin >>>> >>>>> OK, with mpiifort it compiles but it still fails the same way. >>>>> Here are the results from gdb: >>>>> >>>>> [knl00 tests] gdb ../mc/x.testwf.12c-11_states-pn >>>>> GNU gdb (GDB) Red Hat Enterprise Linux 7.6.1-80.el7 >>>>> Copyright (C) 2013 Free Software Foundation, Inc. >>>>> License GPLv3+: GNU GPL version 3 or later >>>>> <http://gnu.org/licenses/gpl.html> <http://gnu.org/licenses/gpl.html> >>>>> This is free software: you are free to change and redistribute it. >>>>> There is NO WARRANTY, to the extent permitted by law. Type"show copying" >>>>> and"show warranty" for details. >>>>> This GDB was configured as"x86_64-redhat-linux-gnu". >>>>> For bug reporting instructions, please see: >>>>> <http://www.gnu.org/software/gdb/bugs/> <http://www.gnu.org/software/gdb/bugs/>... >>>>> Reading symbols from /home/b21541/gfmc/mc/x.testwf.12c-11_states-pn...done. >>>>> (gdb) run >>>>> Starting program: /home/b21541/gfmc/tests/../mc/x.testwf.12c-11_states-pn >>>>> [Thread debugging using libthread_db enabled] >>>>> Using host libthread_db library"/lib64/libthread_db.so.1". >>>>> >>>>> Program received signal SIGSEGV, Segmentation fault. >>>>> 0x00000000004030f0 in _unnamed_main$$ () at testwf.f:13 >>>>> 13 use random2 >>>>> Missing separate debuginfos, use: debuginfo-install glibc-2.17-106.el7_2.4.x86_64 libgcc-4.8.5-4.el7.x86_64 >>>>> (gdb) >>>>> >>>>> (gdb) where >>>>> #0 0x00000000004030f0 in _unnamed_main$$ () at testwf.f:13 >>>>> #1 0x0000000000402c0e in main () >>>>> #2 0x00007ffff53b7b15 in __libc_start_main () from /lib64/libc.so.6 >>>>> #3 0x0000000000402b19 in _start () >>>>> (gdb) >>>>> >>>>> >>>>> I can come over to Bldg 240 if that seems helpful. >>>>> >>>>> Steve >>>>> >>>>> >>>>> Steven C. Pieper, Senior Physicist SO:[email protected] >>>>> Argonne National Laboratory, Physics Division, Bldg. 203, Argonne, IL 60439 >>>>> Phone: 630-252-4232 Fax -6008 >>>>> Secretary, Debra Beres, -4100 >>>>> On 06/16/16 15:08, Luo, Ye wrote: >>>>> >>>>> >>>>> Try mpiifort. >>>>> You can verify the use of ifort by"mpiifort -show". >>>>> Ye >>>>> =================== >>>>> Ye Luo, Ph.D. >>>>> Leadership Computing Facility >>>>> Argonne National Laboratory >>>>> >>>>> >>>>> >>>>> ________________________________________ >>>>> From: >>>>> intel-nda-bounces(a)lists.jlse.anl.gov <mailto:[email protected]> [intel-nda-bounces(a)lists.jlse.anl.gov] on behalf of Steven C. Pieper [spieper(a)anl.gov] >>>>> Sent: Thursday, June 16, 2016 3:04 PM >>>>> To: Harms, Kevin; >>>>> intel-nda(a)lists.jlse.anl.gov <mailto:[email protected]>; Osborn, James C. >>>>> Subject: Re: [JLSE-Intel-NDA] Help on knl00 segfault >>>>> >>>>> >>>>> Hi Kevin, >>>>> >>>>> I did the source command and ifort gets the beta compiler OK. However mpif90 >>>>> is using gfortran: >>>>> >>>>> mpif90 -FR -g -O4 -mavx -xAVX -axAVX -qopenmp -lpthread -qopenmp-report -r8 -WB -traceback -fpe3 -I../common/ -I../mc/ -fpp -DSPIN_SUF= -DPARAMNUC=nclude/paramnuc.12c-11_states-pn -DM0_DIV=1 -DDOCASE=0 -DPARAMCASE=nclude/paramcase.12c-11_states-pn >>>>> -I../adlb/ -I../dmem/ -c kinetic.f >>>>> gfortran: error: unrecognized command line option '-axAVX' >>>>> gfortran: error: unrecognized command line option '-qopenmp' >>>>> gfortran: error: unrecognized command line option '-qopenmp-report' >>>>> gfortran: error: unrecognized command line option '-r8' >>>>> gfortran: error: unrecognized command line option '-WB' >>>>> gfortran: error: unrecognized command line option '-traceback' >>>>> gfortran: error: unrecognized command line option '-fpe3' >>>>> gfortran: error: unrecognized command line option '-fpp' >>>>> make: *** [lib12c-11_states-pn.a(kinetic.o)] Error 1 >>>>> [knl00 mc] which mpif90 >>>>> /soft/compilers/intel_beta/compilers_and_libraries_2017.0.042/linux/mpi/intel64/bin/mpif90 >>>>> >>>>> >>>>> Steve >>>>> Steven C. Pieper, Senior Physicist SO:[email protected] >>>>> Argonne National Laboratory, Physics Division, Bldg. 203, Argonne, IL 60439 >>>>> Phone: 630-252-4232 Fax -6008 >>>>> Secretary, Debra Beres, -4100 >>>>> On 06/16/16 14:53, Harms, Kevin wrote: >>>>> >>>>> >>>>> Oops. I forgot to put �intel64� at the end. The wiki has some useful notes on it: >>>>> >>>>> https://wiki.jlse.anl.gov/display/knl/Intel+KNL >>>>> >>>>> kevin >>>>> >>>>> Steve, >>>>> >>>>> First thing, try the latest compiler. >>>>> >>>>> source /soft/compilers/intel_beta/compilers_and_libraries/linux/bin/compilervars.sh >>>>> >>>>> The 2017 beta has explicit KNL support. Second, have you tried running it in gdb? >>>>> >>>>> kevin >>>>> >>>>> I am trying to run a Fortran program that exercises some of my computational >>>>> kernels. The program does not use MPI, just OpenMP. I have compiled >>>>> it for a small case and that runs fine and gives the correct results. >>>>> However when I recompile it for a realistic cases (the same source >>>>> is used, only a bunch of parameter statements are different), it >>>>> segfaults at line 13 of the main program: >>>>> >>>>> [knl00 tests] ../mc/x.testwf.12c-11_states-pn c12_11states-pn.in >>>>> forrtl: severe (174): SIGSEGV, segmentation fault occurred >>>>> Image PC Routine Line Source >>>>> >>>>> libintlc.so.5 00007F569D060C05 Unknown Unknown Unknown >>>>> libintlc.so.5 00007F569D05E827 Unknown Unknown Unknown >>>>> libifcoremt.so.5 00007F569DEE1962 Unknown Unknown Unknown >>>>> libifcoremt.so.5 00007F569DEE17B6 Unknown Unknown Unknown >>>>> libifcoremt.so.5 00007F569DE36CAD Unknown Unknown Unknown >>>>> libifcoremt.so.5 00007F569DE48FAD Unknown Unknown Unknown >>>>> libpthread.so.0 00007F569F2D7100 Unknown Unknown Unknown >>>>> x.testwf.12c-11_s 00000000004030F0 MAIN__ 13 testwf.f >>>>> x.testwf.12c-11_s 0000000000402C0E Unknown Unknown Unknown >>>>> libc.so.6 00007F569CAA3B15 Unknown Unknown Unknown >>>>> x.testwf.12c-11_s 0000000000402B19 Unknown Unknown Unknown >>>>> Aborted (core dumped) >>>>> [knl00 tests] >>>>> >>>>> >>>>> Line 13 is the very beginning of declarations, not an executable statement. >>>>> It is the first non-comment line and is: >>>>> >>>>> use random2 >>>>> >>>>> random2 is a module that defines some subroutines. It is not changed >>>>> by the parameter statements that are different for the working and failing >>>>> cases. >>>>> >>>>> Here are the working and failing executable sizes: >>>>> >>>>> text data bss dec hex filename >>>>> 2459352 124772 24979752 27563876 1a49764 x.testwf.4he-pn >>>>> 2192689 92004 623277384 625562077 254951dd x.testwf.12c-11_states-pn >>>>> >>>>> Here is how the main program is compiled and module linked: >>>>> >>>>> ifort -Imods.libgen -FR -g -O0 -qopenmp -lpthread -qopenmp-report -r8 -WB -traceback -fpe3 -I../common/ -I../mc/ -fpp -DSPIN_SUF= -DPARAMNUC=nclude/paramnuc.12c-11_states-pn -DM0_DIV=1 -DDOCASE=0 -DPARAMCASE=nclude/paramcase.12c-11_states-pn -c -g testwf.f >>>>> >>>>> ifort -g -mcmodel=medium -traceback -fpe3 -qopenmp -lpthread -qopenmp-report -o x.testwf.12c-11_states-pn testwf.o do_adlb_no.o dmem_fake.o adlb_fake.o \ >>>>> ../common/wall_second-nompi.o ../common/stopnompi.o ../common/mpi_wtime_fake.o -L . -l12c-11_states-pn -lgen -l12c-11_states-pn -L ../common/ -lcomgen enable_abort_fake.o -lsvml >>>>> >>>>> >>>>> [knl00 mc] which ifort >>>>> /soft/compilers/intel/bin/ifort >>>>> >>>>> >>>>> This is in ~b21541/gfmc/mc . >>>>> >>>>> This test case runs fine on blues. >>>>> >>>>> Any suggestions will be gratefully received. >>>>> >>>>> Steve >>>>> -- >>>>> Steven C. Pieper, Senior Physicist SO:[email protected] >>>>> Argonne National Laboratory, Physics Division, Bldg. 203, Argonne, IL 60439 >>>>> Phone: 630-252-4232 Fax -6008 >>>>> Secretary, Debra Beres, -4100 >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>> > >>> >>> >>> >>> _______________________________________________ >>> Intel-nda mailing list >>> Intel-nda(a)lists.jlse.anl.gov >>> https://lists.jlse.anl.gov/mailman/listinfo/intel-nda >>

1 0

Re: [JLSE-Intel-NDA] Help on knl00 segfault -- more
by Steven Pieper 16 Jun '16

16 Jun '16

I just ran it with up to 64 threads on a haswell node on blues. Steve Steven C. Pieper: spieper(a)anl.gov Argonne National Laboratory, Physics Division, Bldg. 203, Argonne, IL 60439 Phone: 630-252-4232 Fax -6008 Secretary, Debra Beres, -4100 On 06/16/2016 07:56 PM, Steven Pieper wrote: > Well further tests have to wait until Monday now, but > > 1) I used both mcmodel=medium and large -- both fail the same way > 2) It runs on blues (haswell nodes and Intell 11 compiler) with mcmodel=medium > 3) I forget the number of OMP threads I was using -- I think 32. > 4) The size command does not show a huge executable (the big stuff will come > later from allocates): >>>>> text data bss dec hex filename >>>>> 2459352 124772 24979752 27563876 1a49764 x.testwf.4he-pn <<<< this works >>>>> 2192689 92004 623277384 625562077 254951dd x.testwf.12c-11_states-pn <<< this fails > > Steve > > Steven C. Pieper: spieper(a)anl.gov > Argonne National Laboratory, Physics Division, Bldg. 203, Argonne, IL 60439 > Phone: 630-252-4232 Fax -6008 > Secretary, Debra Beres, -4100 > > On 06/16/2016 04:41 PM, Vitali A. Morozov wrote: >> >> I would say it has something to do with the memory model and the limitations that the large segments of the code cannot be correctly located by the OS. The problems shows up at the first executable statement. >> >> Two suggestions: try to set the number of OMP threads to some small number (4 or 8 to start with), and try -mcmodel large. >> >> Vitali >> >> >> >> On 06/16/2016 04:10 PM, Steven C. Pieper wrote: >>> I'm not sure what you mean. random2.f is written in standard >>> fortran 90. >>> >>> However the error doesn't seem to have anything to do with random2.f per se. >>> I commented that line out and now the traceback points to the next >>> non-comment line. So I think the segfault is happening at load time >>> and the traceback just points to the first non comment line. >>> >>> I have to go now; I guess this will have to wait until next week. >>> >>> Thanks, >>> Steve >>> Steven C. Pieper, Senior Physicist SO:[email protected] >>> Argonne National Laboratory, Physics Division, Bldg. 203, Argonne, IL 60439 >>> Phone: 630-252-4232 Fax -6008 >>> Secretary, Debra Beres, -4100 >>> On 06/16/16 16:02, Harms, Kevin wrote: >>>> Steve, >>>> >>>> I don�t know fortran well, is random2 part of the standard fortran or your code? >>>> >>>> kevin >>>> >>>>> OK, with mpiifort it compiles but it still fails the same way. >>>>> Here are the results from gdb: >>>>> >>>>> [knl00 tests] gdb ../mc/x.testwf.12c-11_states-pn >>>>> GNU gdb (GDB) Red Hat Enterprise Linux 7.6.1-80.el7 >>>>> Copyright (C) 2013 Free Software Foundation, Inc. >>>>> License GPLv3+: GNU GPL version 3 or later >>>>> <http://gnu.org/licenses/gpl.html> <http://gnu.org/licenses/gpl.html> >>>>> This is free software: you are free to change and redistribute it. >>>>> There is NO WARRANTY, to the extent permitted by law. Type"show copying" >>>>> and"show warranty" for details. >>>>> This GDB was configured as"x86_64-redhat-linux-gnu". >>>>> For bug reporting instructions, please see: >>>>> <http://www.gnu.org/software/gdb/bugs/> <http://www.gnu.org/software/gdb/bugs/>... >>>>> Reading symbols from /home/b21541/gfmc/mc/x.testwf.12c-11_states-pn...done. >>>>> (gdb) run >>>>> Starting program: /home/b21541/gfmc/tests/../mc/x.testwf.12c-11_states-pn >>>>> [Thread debugging using libthread_db enabled] >>>>> Using host libthread_db library"/lib64/libthread_db.so.1". >>>>> >>>>> Program received signal SIGSEGV, Segmentation fault. >>>>> 0x00000000004030f0 in _unnamed_main$$ () at testwf.f:13 >>>>> 13 use random2 >>>>> Missing separate debuginfos, use: debuginfo-install glibc-2.17-106.el7_2.4.x86_64 libgcc-4.8.5-4.el7.x86_64 >>>>> (gdb) >>>>> >>>>> (gdb) where >>>>> #0 0x00000000004030f0 in _unnamed_main$$ () at testwf.f:13 >>>>> #1 0x0000000000402c0e in main () >>>>> #2 0x00007ffff53b7b15 in __libc_start_main () from /lib64/libc.so.6 >>>>> #3 0x0000000000402b19 in _start () >>>>> (gdb) >>>>> >>>>> >>>>> I can come over to Bldg 240 if that seems helpful. >>>>> >>>>> Steve >>>>> >>>>> >>>>> Steven C. Pieper, Senior Physicist SO:[email protected] >>>>> Argonne National Laboratory, Physics Division, Bldg. 203, Argonne, IL 60439 >>>>> Phone: 630-252-4232 Fax -6008 >>>>> Secretary, Debra Beres, -4100 >>>>> On 06/16/16 15:08, Luo, Ye wrote: >>>>> >>>>> >>>>> Try mpiifort. >>>>> You can verify the use of ifort by"mpiifort -show". >>>>> Ye >>>>> =================== >>>>> Ye Luo, Ph.D. >>>>> Leadership Computing Facility >>>>> Argonne National Laboratory >>>>> >>>>> >>>>> >>>>> ________________________________________ >>>>> From: >>>>> intel-nda-bounces(a)lists.jlse.anl.gov <mailto:[email protected]> [intel-nda-bounces(a)lists.jlse.anl.gov] on behalf of Steven C. Pieper [spieper(a)anl.gov] >>>>> Sent: Thursday, June 16, 2016 3:04 PM >>>>> To: Harms, Kevin; >>>>> intel-nda(a)lists.jlse.anl.gov <mailto:[email protected]>; Osborn, James C. >>>>> Subject: Re: [JLSE-Intel-NDA] Help on knl00 segfault >>>>> >>>>> >>>>> Hi Kevin, >>>>> >>>>> I did the source command and ifort gets the beta compiler OK. However mpif90 >>>>> is using gfortran: >>>>> >>>>> mpif90 -FR -g -O4 -mavx -xAVX -axAVX -qopenmp -lpthread -qopenmp-report -r8 -WB -traceback -fpe3 -I../common/ -I../mc/ -fpp -DSPIN_SUF= -DPARAMNUC=nclude/paramnuc.12c-11_states-pn -DM0_DIV=1 -DDOCASE=0 -DPARAMCASE=nclude/paramcase.12c-11_states-pn >>>>> -I../adlb/ -I../dmem/ -c kinetic.f >>>>> gfortran: error: unrecognized command line option '-axAVX' >>>>> gfortran: error: unrecognized command line option '-qopenmp' >>>>> gfortran: error: unrecognized command line option '-qopenmp-report' >>>>> gfortran: error: unrecognized command line option '-r8' >>>>> gfortran: error: unrecognized command line option '-WB' >>>>> gfortran: error: unrecognized command line option '-traceback' >>>>> gfortran: error: unrecognized command line option '-fpe3' >>>>> gfortran: error: unrecognized command line option '-fpp' >>>>> make: *** [lib12c-11_states-pn.a(kinetic.o)] Error 1 >>>>> [knl00 mc] which mpif90 >>>>> /soft/compilers/intel_beta/compilers_and_libraries_2017.0.042/linux/mpi/intel64/bin/mpif90 >>>>> >>>>> >>>>> Steve >>>>> Steven C. Pieper, Senior Physicist SO:[email protected] >>>>> Argonne National Laboratory, Physics Division, Bldg. 203, Argonne, IL 60439 >>>>> Phone: 630-252-4232 Fax -6008 >>>>> Secretary, Debra Beres, -4100 >>>>> On 06/16/16 14:53, Harms, Kevin wrote: >>>>> >>>>> >>>>> Oops. I forgot to put �intel64� at the end. The wiki has some useful notes on it: >>>>> >>>>> https://wiki.jlse.anl.gov/display/knl/Intel+KNL >>>>> >>>>> kevin >>>>> >>>>> Steve, >>>>> >>>>> First thing, try the latest compiler. >>>>> >>>>> source /soft/compilers/intel_beta/compilers_and_libraries/linux/bin/compilervars.sh >>>>> >>>>> The 2017 beta has explicit KNL support. Second, have you tried running it in gdb? >>>>> >>>>> kevin >>>>> >>>>> I am trying to run a Fortran program that exercises some of my computational >>>>> kernels. The program does not use MPI, just OpenMP. I have compiled >>>>> it for a small case and that runs fine and gives the correct results. >>>>> However when I recompile it for a realistic cases (the same source >>>>> is used, only a bunch of parameter statements are different), it >>>>> segfaults at line 13 of the main program: >>>>> >>>>> [knl00 tests] ../mc/x.testwf.12c-11_states-pn c12_11states-pn.in >>>>> forrtl: severe (174): SIGSEGV, segmentation fault occurred >>>>> Image PC Routine Line Source >>>>> >>>>> libintlc.so.5 00007F569D060C05 Unknown Unknown Unknown >>>>> libintlc.so.5 00007F569D05E827 Unknown Unknown Unknown >>>>> libifcoremt.so.5 00007F569DEE1962 Unknown Unknown Unknown >>>>> libifcoremt.so.5 00007F569DEE17B6 Unknown Unknown Unknown >>>>> libifcoremt.so.5 00007F569DE36CAD Unknown Unknown Unknown >>>>> libifcoremt.so.5 00007F569DE48FAD Unknown Unknown Unknown >>>>> libpthread.so.0 00007F569F2D7100 Unknown Unknown Unknown >>>>> x.testwf.12c-11_s 00000000004030F0 MAIN__ 13 testwf.f >>>>> x.testwf.12c-11_s 0000000000402C0E Unknown Unknown Unknown >>>>> libc.so.6 00007F569CAA3B15 Unknown Unknown Unknown >>>>> x.testwf.12c-11_s 0000000000402B19 Unknown Unknown Unknown >>>>> Aborted (core dumped) >>>>> [knl00 tests] >>>>> >>>>> >>>>> Line 13 is the very beginning of declarations, not an executable statement. >>>>> It is the first non-comment line and is: >>>>> >>>>> use random2 >>>>> >>>>> random2 is a module that defines some subroutines. It is not changed >>>>> by the parameter statements that are different for the working and failing >>>>> cases. >>>>> >>>>> Here are the working and failing executable sizes: >>>>> >>>>> text data bss dec hex filename >>>>> 2459352 124772 24979752 27563876 1a49764 x.testwf.4he-pn >>>>> 2192689 92004 623277384 625562077 254951dd x.testwf.12c-11_states-pn >>>>> >>>>> Here is how the main program is compiled and module linked: >>>>> >>>>> ifort -Imods.libgen -FR -g -O0 -qopenmp -lpthread -qopenmp-report -r8 -WB -traceback -fpe3 -I../common/ -I../mc/ -fpp -DSPIN_SUF= -DPARAMNUC=nclude/paramnuc.12c-11_states-pn -DM0_DIV=1 -DDOCASE=0 -DPARAMCASE=nclude/paramcase.12c-11_states-pn -c -g testwf.f >>>>> >>>>> ifort -g -mcmodel=medium -traceback -fpe3 -qopenmp -lpthread -qopenmp-report -o x.testwf.12c-11_states-pn testwf.o do_adlb_no.o dmem_fake.o adlb_fake.o \ >>>>> ../common/wall_second-nompi.o ../common/stopnompi.o ../common/mpi_wtime_fake.o -L . -l12c-11_states-pn -lgen -l12c-11_states-pn -L ../common/ -lcomgen enable_abort_fake.o -lsvml >>>>> >>>>> >>>>> [knl00 mc] which ifort >>>>> /soft/compilers/intel/bin/ifort >>>>> >>>>> >>>>> This is in ~b21541/gfmc/mc . >>>>> >>>>> This test case runs fine on blues. >>>>> >>>>> Any suggestions will be gratefully received. >>>>> >>>>> Steve >>>>> -- >>>>> Steven C. Pieper, Senior Physicist SO:[email protected] >>>>> Argonne National Laboratory, Physics Division, Bldg. 203, Argonne, IL 60439 >>>>> Phone: 630-252-4232 Fax -6008 >>>>> Secretary, Debra Beres, -4100 >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>> > >>> >>> >>> >>> _______________________________________________ >>> Intel-nda mailing list >>> Intel-nda(a)lists.jlse.anl.gov >>> https://lists.jlse.anl.gov/mailman/listinfo/intel-nda >> > _______________________________________________ > Intel-nda mailing list > Intel-nda(a)lists.jlse.anl.gov > https://lists.jlse.anl.gov/mailman/listinfo/intel-nda

1 0

Re: [JLSE-Intel-NDA] Help on knl00 segfault
by Steven Pieper 16 Jun '16

16 Jun '16

Well further tests have to wait until Monday now, but 1) I used both mcmodel=medium and large -- both fail the same way 2) It runs on blues (haswell nodes and Intell 11 compiler) with mcmodel=medium 3) I forget the number of OMP threads I was using -- I think 32. 4) The size command does not show a huge executable (the big stuff will come later from allocates): >>>> text data bss dec hex filename >>>> 2459352 124772 24979752 27563876 1a49764 x.testwf.4he-pn <<<< this works >>>> 2192689 92004 623277384 625562077 254951dd x.testwf.12c-11_states-pn <<< this fails Steve Steven C. Pieper: spieper(a)anl.gov Argonne National Laboratory, Physics Division, Bldg. 203, Argonne, IL 60439 Phone: 630-252-4232 Fax -6008 Secretary, Debra Beres, -4100 On 06/16/2016 04:41 PM, Vitali A. Morozov wrote: > > I would say it has something to do with the memory model and the limitations that the large segments of the code cannot be correctly located by the OS. The problems shows up at the first executable statement. > > Two suggestions: try to set the number of OMP threads to some small number (4 or 8 to start with), and try -mcmodel large. > > Vitali > > > > On 06/16/2016 04:10 PM, Steven C. Pieper wrote: >> I'm not sure what you mean. random2.f is written in standard >> fortran 90. >> >> However the error doesn't seem to have anything to do with random2.f per se. >> I commented that line out and now the traceback points to the next >> non-comment line. So I think the segfault is happening at load time >> and the traceback just points to the first non comment line. >> >> I have to go now; I guess this will have to wait until next week. >> >> Thanks, >> Steve >> Steven C. Pieper, Senior Physicist SO:[email protected] >> Argonne National Laboratory, Physics Division, Bldg. 203, Argonne, IL 60439 >> Phone: 630-252-4232 Fax -6008 >> Secretary, Debra Beres, -4100 >> On 06/16/16 16:02, Harms, Kevin wrote: >>> Steve, >>> >>> I don�t know fortran well, is random2 part of the standard fortran or your code? >>> >>> kevin >>> >>>> OK, with mpiifort it compiles but it still fails the same way. >>>> Here are the results from gdb: >>>> >>>> [knl00 tests] gdb ../mc/x.testwf.12c-11_states-pn >>>> GNU gdb (GDB) Red Hat Enterprise Linux 7.6.1-80.el7 >>>> Copyright (C) 2013 Free Software Foundation, Inc. >>>> License GPLv3+: GNU GPL version 3 or later >>>> <http://gnu.org/licenses/gpl.html> <http://gnu.org/licenses/gpl.html> >>>> This is free software: you are free to change and redistribute it. >>>> There is NO WARRANTY, to the extent permitted by law. Type"show copying" >>>> and"show warranty" for details. >>>> This GDB was configured as"x86_64-redhat-linux-gnu". >>>> For bug reporting instructions, please see: >>>> <http://www.gnu.org/software/gdb/bugs/> <http://www.gnu.org/software/gdb/bugs/>... >>>> Reading symbols from /home/b21541/gfmc/mc/x.testwf.12c-11_states-pn...done. >>>> (gdb) run >>>> Starting program: /home/b21541/gfmc/tests/../mc/x.testwf.12c-11_states-pn >>>> [Thread debugging using libthread_db enabled] >>>> Using host libthread_db library"/lib64/libthread_db.so.1". >>>> >>>> Program received signal SIGSEGV, Segmentation fault. >>>> 0x00000000004030f0 in _unnamed_main$$ () at testwf.f:13 >>>> 13 use random2 >>>> Missing separate debuginfos, use: debuginfo-install glibc-2.17-106.el7_2.4.x86_64 libgcc-4.8.5-4.el7.x86_64 >>>> (gdb) >>>> >>>> (gdb) where >>>> #0 0x00000000004030f0 in _unnamed_main$$ () at testwf.f:13 >>>> #1 0x0000000000402c0e in main () >>>> #2 0x00007ffff53b7b15 in __libc_start_main () from /lib64/libc.so.6 >>>> #3 0x0000000000402b19 in _start () >>>> (gdb) >>>> >>>> >>>> I can come over to Bldg 240 if that seems helpful. >>>> >>>> Steve >>>> >>>> >>>> Steven C. Pieper, Senior Physicist SO:[email protected] >>>> Argonne National Laboratory, Physics Division, Bldg. 203, Argonne, IL 60439 >>>> Phone: 630-252-4232 Fax -6008 >>>> Secretary, Debra Beres, -4100 >>>> On 06/16/16 15:08, Luo, Ye wrote: >>>> >>>> >>>> Try mpiifort. >>>> You can verify the use of ifort by"mpiifort -show". >>>> Ye >>>> =================== >>>> Ye Luo, Ph.D. >>>> Leadership Computing Facility >>>> Argonne National Laboratory >>>> >>>> >>>> >>>> ________________________________________ >>>> From: >>>> intel-nda-bounces(a)lists.jlse.anl.gov <mailto:[email protected]> [intel-nda-bounces(a)lists.jlse.anl.gov] on behalf of Steven C. Pieper [spieper(a)anl.gov] >>>> Sent: Thursday, June 16, 2016 3:04 PM >>>> To: Harms, Kevin; >>>> intel-nda(a)lists.jlse.anl.gov <mailto:[email protected]>; Osborn, James C. >>>> Subject: Re: [JLSE-Intel-NDA] Help on knl00 segfault >>>> >>>> >>>> Hi Kevin, >>>> >>>> I did the source command and ifort gets the beta compiler OK. However mpif90 >>>> is using gfortran: >>>> >>>> mpif90 -FR -g -O4 -mavx -xAVX -axAVX -qopenmp -lpthread -qopenmp-report -r8 -WB -traceback -fpe3 -I../common/ -I../mc/ -fpp -DSPIN_SUF= -DPARAMNUC=nclude/paramnuc.12c-11_states-pn -DM0_DIV=1 -DDOCASE=0 -DPARAMCASE=nclude/paramcase.12c-11_states-pn >>>> -I../adlb/ -I../dmem/ -c kinetic.f >>>> gfortran: error: unrecognized command line option '-axAVX' >>>> gfortran: error: unrecognized command line option '-qopenmp' >>>> gfortran: error: unrecognized command line option '-qopenmp-report' >>>> gfortran: error: unrecognized command line option '-r8' >>>> gfortran: error: unrecognized command line option '-WB' >>>> gfortran: error: unrecognized command line option '-traceback' >>>> gfortran: error: unrecognized command line option '-fpe3' >>>> gfortran: error: unrecognized command line option '-fpp' >>>> make: *** [lib12c-11_states-pn.a(kinetic.o)] Error 1 >>>> [knl00 mc] which mpif90 >>>> /soft/compilers/intel_beta/compilers_and_libraries_2017.0.042/linux/mpi/intel64/bin/mpif90 >>>> >>>> >>>> Steve >>>> Steven C. Pieper, Senior Physicist SO:[email protected] >>>> Argonne National Laboratory, Physics Division, Bldg. 203, Argonne, IL 60439 >>>> Phone: 630-252-4232 Fax -6008 >>>> Secretary, Debra Beres, -4100 >>>> On 06/16/16 14:53, Harms, Kevin wrote: >>>> >>>> >>>> Oops. I forgot to put �intel64� at the end. The wiki has some useful notes on it: >>>> >>>> https://wiki.jlse.anl.gov/display/knl/Intel+KNL >>>> >>>> kevin >>>> >>>> Steve, >>>> >>>> First thing, try the latest compiler. >>>> >>>> source /soft/compilers/intel_beta/compilers_and_libraries/linux/bin/compilervars.sh >>>> >>>> The 2017 beta has explicit KNL support. Second, have you tried running it in gdb? >>>> >>>> kevin >>>> >>>> I am trying to run a Fortran program that exercises some of my computational >>>> kernels. The program does not use MPI, just OpenMP. I have compiled >>>> it for a small case and that runs fine and gives the correct results. >>>> However when I recompile it for a realistic cases (the same source >>>> is used, only a bunch of parameter statements are different), it >>>> segfaults at line 13 of the main program: >>>> >>>> [knl00 tests] ../mc/x.testwf.12c-11_states-pn c12_11states-pn.in >>>> forrtl: severe (174): SIGSEGV, segmentation fault occurred >>>> Image PC Routine Line Source >>>> >>>> libintlc.so.5 00007F569D060C05 Unknown Unknown Unknown >>>> libintlc.so.5 00007F569D05E827 Unknown Unknown Unknown >>>> libifcoremt.so.5 00007F569DEE1962 Unknown Unknown Unknown >>>> libifcoremt.so.5 00007F569DEE17B6 Unknown Unknown Unknown >>>> libifcoremt.so.5 00007F569DE36CAD Unknown Unknown Unknown >>>> libifcoremt.so.5 00007F569DE48FAD Unknown Unknown Unknown >>>> libpthread.so.0 00007F569F2D7100 Unknown Unknown Unknown >>>> x.testwf.12c-11_s 00000000004030F0 MAIN__ 13 testwf.f >>>> x.testwf.12c-11_s 0000000000402C0E Unknown Unknown Unknown >>>> libc.so.6 00007F569CAA3B15 Unknown Unknown Unknown >>>> x.testwf.12c-11_s 0000000000402B19 Unknown Unknown Unknown >>>> Aborted (core dumped) >>>> [knl00 tests] >>>> >>>> >>>> Line 13 is the very beginning of declarations, not an executable statement. >>>> It is the first non-comment line and is: >>>> >>>> use random2 >>>> >>>> random2 is a module that defines some subroutines. It is not changed >>>> by the parameter statements that are different for the working and failing >>>> cases. >>>> >>>> Here are the working and failing executable sizes: >>>> >>>> text data bss dec hex filename >>>> 2459352 124772 24979752 27563876 1a49764 x.testwf.4he-pn >>>> 2192689 92004 623277384 625562077 254951dd x.testwf.12c-11_states-pn >>>> >>>> Here is how the main program is compiled and module linked: >>>> >>>> ifort -Imods.libgen -FR -g -O0 -qopenmp -lpthread -qopenmp-report -r8 -WB -traceback -fpe3 -I../common/ -I../mc/ -fpp -DSPIN_SUF= -DPARAMNUC=nclude/paramnuc.12c-11_states-pn -DM0_DIV=1 -DDOCASE=0 -DPARAMCASE=nclude/paramcase.12c-11_states-pn -c -g testwf.f >>>> >>>> ifort -g -mcmodel=medium -traceback -fpe3 -qopenmp -lpthread -qopenmp-report -o x.testwf.12c-11_states-pn testwf.o do_adlb_no.o dmem_fake.o adlb_fake.o \ >>>> ../common/wall_second-nompi.o ../common/stopnompi.o ../common/mpi_wtime_fake.o -L . -l12c-11_states-pn -lgen -l12c-11_states-pn -L ../common/ -lcomgen enable_abort_fake.o -lsvml >>>> >>>> >>>> [knl00 mc] which ifort >>>> /soft/compilers/intel/bin/ifort >>>> >>>> >>>> This is in ~b21541/gfmc/mc . >>>> >>>> This test case runs fine on blues. >>>> >>>> Any suggestions will be gratefully received. >>>> >>>> Steve >>>> -- >>>> Steven C. Pieper, Senior Physicist SO:[email protected] >>>> Argonne National Laboratory, Physics Division, Bldg. 203, Argonne, IL 60439 >>>> Phone: 630-252-4232 Fax -6008 >>>> Secretary, Debra Beres, -4100 >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>> > >> >> >> >> _______________________________________________ >> Intel-nda mailing list >> Intel-nda(a)lists.jlse.anl.gov >> https://lists.jlse.anl.gov/mailman/listinfo/intel-nda >

1 0

Re: [JLSE-Intel-NDA] Help on knl00 segfault
by Vitali A. Morozov 16 Jun '16

16 Jun '16

I would say it has something to do with the memory model and the limitations that the large segments of the code cannot be correctly located by the OS. The problems shows up at the first executable statement. Two suggestions: try to set the number of OMP threads to some small number (4 or 8 to start with), and try -mcmodel large. Vitali On 06/16/2016 04:10 PM, Steven C. Pieper wrote: > I'm not sure what you mean. random2.f is written in standard > fortran 90. > > However the error doesn't seem to have anything to do with random2.f > per se. > I commented that line out and now the traceback points to the next > non-comment line. So I think the segfault is happening at load time > and the traceback just points to the first non comment line. > > I have to go now; I guess this will have to wait until next week. > > Thanks, > Steve > Steven C. Pieper, Senior Physicist SO:[email protected] > Argonne National Laboratory, Physics Division, Bldg. 203, Argonne, IL 60439 > Phone: 630-252-4232 Fax -6008 > Secretary, Debra Beres, -4100 > On 06/16/16 16:02, Harms, Kevin wrote: >> Steve, >> >> I donï¿½t know fortran well, is random2 part of the standard fortran or your code? >> >> kevin >> >>> OK, with mpiifort it compiles but it still fails the same way. >>> Here are the results from gdb: >>> >>> [knl00 tests] gdb ../mc/x.testwf.12c-11_states-pn >>> GNU gdb (GDB) Red Hat Enterprise Linux 7.6.1-80.el7 >>> Copyright (C) 2013 Free Software Foundation, Inc. >>> License GPLv3+: GNU GPL version 3 or later >>> <http://gnu.org/licenses/gpl.html> <http://gnu.org/licenses/gpl.html> >>> This is free software: you are free to change and redistribute it. >>> There is NO WARRANTY, to the extent permitted by law. Type "show copying" >>> and "show warranty" for details. >>> This GDB was configured as "x86_64-redhat-linux-gnu". >>> For bug reporting instructions, please see: >>> <http://www.gnu.org/software/gdb/bugs/> <http://www.gnu.org/software/gdb/bugs/>... >>> Reading symbols from /home/b21541/gfmc/mc/x.testwf.12c-11_states-pn...done. >>> (gdb) run >>> Starting program: /home/b21541/gfmc/tests/../mc/x.testwf.12c-11_states-pn >>> [Thread debugging using libthread_db enabled] >>> Using host libthread_db library "/lib64/libthread_db.so.1". >>> >>> Program received signal SIGSEGV, Segmentation fault. >>> 0x00000000004030f0 in _unnamed_main$$ () at testwf.f:13 >>> 13 use random2 >>> Missing separate debuginfos, use: debuginfo-install glibc-2.17-106.el7_2.4.x86_64 libgcc-4.8.5-4.el7.x86_64 >>> (gdb) >>> >>> (gdb) where >>> #0 0x00000000004030f0 in _unnamed_main$$ () at testwf.f:13 >>> #1 0x0000000000402c0e in main () >>> #2 0x00007ffff53b7b15 in __libc_start_main () from /lib64/libc.so.6 >>> #3 0x0000000000402b19 in _start () >>> (gdb) >>> >>> >>> I can come over to Bldg 240 if that seems helpful. >>> >>> Steve >>> >>> >>> Steven C. Pieper, Senior Physicist SO:[email protected] >>> Argonne National Laboratory, Physics Division, Bldg. 203, Argonne, IL 60439 >>> Phone: 630-252-4232 Fax -6008 >>> Secretary, Debra Beres, -4100 >>> On 06/16/16 15:08, Luo, Ye wrote: >>> >>> >>> Try mpiifort. >>> You can verify the use of ifort by "mpiifort -show". >>> Ye >>> =================== >>> Ye Luo, Ph.D. >>> Leadership Computing Facility >>> Argonne National Laboratory >>> >>> >>> >>> ________________________________________ >>> From: >>> intel-nda-bounces(a)lists.jlse.anl.gov <mailto:[email protected]> [intel-nda-bounces(a)lists.jlse.anl.gov] on behalf of Steven C. Pieper [spieper(a)anl.gov] >>> Sent: Thursday, June 16, 2016 3:04 PM >>> To: Harms, Kevin; >>> intel-nda(a)lists.jlse.anl.gov <mailto:[email protected]>; Osborn, James C. >>> Subject: Re: [JLSE-Intel-NDA] Help on knl00 segfault >>> >>> >>> Hi Kevin, >>> >>> I did the source command and ifort gets the beta compiler OK. However mpif90 >>> is using gfortran: >>> >>> mpif90 -FR -g -O4 -mavx -xAVX -axAVX -qopenmp -lpthread -qopenmp-report -r8 -WB -traceback -fpe3 -I../common/ -I../mc/ -fpp -DSPIN_SUF= -DPARAMNUC=nclude/paramnuc.12c-11_states-pn -DM0_DIV=1 -DDOCASE=0 -DPARAMCASE=nclude/paramcase.12c-11_states-pn >>> -I../adlb/ -I../dmem/ -c kinetic.f >>> gfortran: error: unrecognized command line option '-axAVX' >>> gfortran: error: unrecognized command line option '-qopenmp' >>> gfortran: error: unrecognized command line option '-qopenmp-report' >>> gfortran: error: unrecognized command line option '-r8' >>> gfortran: error: unrecognized command line option '-WB' >>> gfortran: error: unrecognized command line option '-traceback' >>> gfortran: error: unrecognized command line option '-fpe3' >>> gfortran: error: unrecognized command line option '-fpp' >>> make: *** [lib12c-11_states-pn.a(kinetic.o)] Error 1 >>> [knl00 mc] which mpif90 >>> /soft/compilers/intel_beta/compilers_and_libraries_2017.0.042/linux/mpi/intel64/bin/mpif90 >>> >>> >>> Steve >>> Steven C. Pieper, Senior Physicist SO:[email protected] >>> Argonne National Laboratory, Physics Division, Bldg. 203, Argonne, IL 60439 >>> Phone: 630-252-4232 Fax -6008 >>> Secretary, Debra Beres, -4100 >>> On 06/16/16 14:53, Harms, Kevin wrote: >>> >>> >>> Oops. I forgot to put ï¿½intel64ï¿½ at the end. The wiki has some useful notes on it: >>> >>> https://wiki.jlse.anl.gov/display/knl/Intel+KNL >>> >>> kevin >>> >>> Steve, >>> >>> First thing, try the latest compiler. >>> >>> source /soft/compilers/intel_beta/compilers_and_libraries/linux/bin/compilervars.sh >>> >>> The 2017 beta has explicit KNL support. Second, have you tried running it in gdb? >>> >>> kevin >>> >>> I am trying to run a Fortran program that exercises some of my computational >>> kernels. The program does not use MPI, just OpenMP. I have compiled >>> it for a small case and that runs fine and gives the correct results. >>> However when I recompile it for a realistic cases (the same source >>> is used, only a bunch of parameter statements are different), it >>> segfaults at line 13 of the main program: >>> >>> [knl00 tests] ../mc/x.testwf.12c-11_states-pn c12_11states-pn.in >>> forrtl: severe (174): SIGSEGV, segmentation fault occurred >>> Image PC Routine Line Source >>> >>> libintlc.so.5 00007F569D060C05 Unknown Unknown Unknown >>> libintlc.so.5 00007F569D05E827 Unknown Unknown Unknown >>> libifcoremt.so.5 00007F569DEE1962 Unknown Unknown Unknown >>> libifcoremt.so.5 00007F569DEE17B6 Unknown Unknown Unknown >>> libifcoremt.so.5 00007F569DE36CAD Unknown Unknown Unknown >>> libifcoremt.so.5 00007F569DE48FAD Unknown Unknown Unknown >>> libpthread.so.0 00007F569F2D7100 Unknown Unknown Unknown >>> x.testwf.12c-11_s 00000000004030F0 MAIN__ 13 testwf.f >>> x.testwf.12c-11_s 0000000000402C0E Unknown Unknown Unknown >>> libc.so.6 00007F569CAA3B15 Unknown Unknown Unknown >>> x.testwf.12c-11_s 0000000000402B19 Unknown Unknown Unknown >>> Aborted (core dumped) >>> [knl00 tests] >>> >>> >>> Line 13 is the very beginning of declarations, not an executable statement. >>> It is the first non-comment line and is: >>> >>> use random2 >>> >>> random2 is a module that defines some subroutines. It is not changed >>> by the parameter statements that are different for the working and failing >>> cases. >>> >>> Here are the working and failing executable sizes: >>> >>> text data bss dec hex filename >>> 2459352 124772 24979752 27563876 1a49764 x.testwf.4he-pn >>> 2192689 92004 623277384 625562077 254951dd x.testwf.12c-11_states-pn >>> >>> Here is how the main program is compiled and module linked: >>> >>> ifort -Imods.libgen -FR -g -O0 -qopenmp -lpthread -qopenmp-report -r8 -WB -traceback -fpe3 -I../common/ -I../mc/ -fpp -DSPIN_SUF= -DPARAMNUC=nclude/paramnuc.12c-11_states-pn -DM0_DIV=1 -DDOCASE=0 -DPARAMCASE=nclude/paramcase.12c-11_states-pn -c -g testwf.f >>> >>> ifort -g -mcmodel=medium -traceback -fpe3 -qopenmp -lpthread -qopenmp-report -o x.testwf.12c-11_states-pn testwf.o do_adlb_no.o dmem_fake.o adlb_fake.o \ >>> ../common/wall_second-nompi.o ../common/stopnompi.o ../common/mpi_wtime_fake.o -L . -l12c-11_states-pn -lgen -l12c-11_states-pn -L ../common/ -lcomgen enable_abort_fake.o -lsvml >>> >>> >>> [knl00 mc] which ifort >>> /soft/compilers/intel/bin/ifort >>> >>> >>> This is in ~b21541/gfmc/mc . >>> >>> This test case runs fine on blues. >>> >>> Any suggestions will be gratefully received. >>> >>> Steve >>> -- >>> Steven C. Pieper, Senior Physicist SO:[email protected] >>> Argonne National Laboratory, Physics Division, Bldg. 203, Argonne, IL 60439 >>> Phone: 630-252-4232 Fax -6008 >>> Secretary, Debra Beres, -4100 >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >> > > > > > _______________________________________________ > Intel-nda mailing list > Intel-nda(a)lists.jlse.anl.gov > https://lists.jlse.anl.gov/mailman/listinfo/intel-nda

1 0

Re: [JLSE-Intel-NDA] Help on knl00 segfault
by Steven C. Pieper 16 Jun '16

16 Jun '16

I'm not sure what you mean. random2.f is written in standard fortran 90. However the error doesn't seem to have anything to do with random2.f per se. I commented that line out and now the traceback points to the next non-comment line. So I think the segfault is happening at load time and the traceback just points to the first non comment line. I have to go now; I guess this will have to wait until next week. Thanks, Steve Steven C. Pieper, Senior Physicist SO: spieper(a)anl.gov Argonne National Laboratory, Physics Division, Bldg. 203, Argonne, IL 60439 Phone: 630-252-4232 Fax -6008 Secretary, Debra Beres, -4100 On 06/16/16 16:02, Harms, Kevin wrote: > Steve, > > I don’t know fortran well, is random2 part of the standard fortran or your code? > > kevin > >> OK, with mpiifort it compiles but it still fails the same way. >> Here are the results from gdb: >> >> [knl00 tests] gdb ../mc/x.testwf.12c-11_states-pn >> GNU gdb (GDB) Red Hat Enterprise Linux 7.6.1-80.el7 >> Copyright (C) 2013 Free Software Foundation, Inc. >> License GPLv3+: GNU GPL version 3 or later >> <http://gnu.org/licenses/gpl.html> <http://gnu.org/licenses/gpl.html> >> This is free software: you are free to change and redistribute it. >> There is NO WARRANTY, to the extent permitted by law. Type "show copying" >> and "show warranty" for details. >> This GDB was configured as "x86_64-redhat-linux-gnu". >> For bug reporting instructions, please see: >> <http://www.gnu.org/software/gdb/bugs/> <http://www.gnu.org/software/gdb/bugs/>... >> Reading symbols from /home/b21541/gfmc/mc/x.testwf.12c-11_states-pn...done. >> (gdb) run >> Starting program: /home/b21541/gfmc/tests/../mc/x.testwf.12c-11_states-pn >> [Thread debugging using libthread_db enabled] >> Using host libthread_db library "/lib64/libthread_db.so.1". >> >> Program received signal SIGSEGV, Segmentation fault. >> 0x00000000004030f0 in _unnamed_main$$ () at testwf.f:13 >> 13 use random2 >> Missing separate debuginfos, use: debuginfo-install glibc-2.17-106.el7_2.4.x86_64 libgcc-4.8.5-4.el7.x86_64 >> (gdb) >> >> (gdb) where >> #0 0x00000000004030f0 in _unnamed_main$$ () at testwf.f:13 >> #1 0x0000000000402c0e in main () >> #2 0x00007ffff53b7b15 in __libc_start_main () from /lib64/libc.so.6 >> #3 0x0000000000402b19 in _start () >> (gdb) >> >> >> I can come over to Bldg 240 if that seems helpful. >> >> Steve >> >> >> Steven C. Pieper, Senior Physicist SO: spieper(a)anl.gov >> Argonne National Laboratory, Physics Division, Bldg. 203, Argonne, IL 60439 >> Phone: 630-252-4232 Fax -6008 >> Secretary, Debra Beres, -4100 >> On 06/16/16 15:08, Luo, Ye wrote: >> >> >> Try mpiifort. >> You can verify the use of ifort by "mpiifort -show". >> Ye >> =================== >> Ye Luo, Ph.D. >> Leadership Computing Facility >> Argonne National Laboratory >> >> >> >> ________________________________________ >> From: >> intel-nda-bounces(a)lists.jlse.anl.gov <mailto:[email protected]> [intel-nda-bounces(a)lists.jlse.anl.gov] on behalf of Steven C. Pieper [spieper(a)anl.gov] >> Sent: Thursday, June 16, 2016 3:04 PM >> To: Harms, Kevin; >> intel-nda(a)lists.jlse.anl.gov <mailto:[email protected]>; Osborn, James C. >> Subject: Re: [JLSE-Intel-NDA] Help on knl00 segfault >> >> >> Hi Kevin, >> >> I did the source command and ifort gets the beta compiler OK. However mpif90 >> is using gfortran: >> >> mpif90 -FR -g -O4 -mavx -xAVX -axAVX -qopenmp -lpthread -qopenmp-report -r8 -WB -traceback -fpe3 -I../common/ -I../mc/ -fpp -DSPIN_SUF= -DPARAMNUC=nclude/paramnuc.12c-11_states-pn -DM0_DIV=1 -DDOCASE=0 -DPARAMCASE=nclude/paramcase.12c-11_states-pn >> -I../adlb/ -I../dmem/ -c kinetic.f >> gfortran: error: unrecognized command line option '-axAVX' >> gfortran: error: unrecognized command line option '-qopenmp' >> gfortran: error: unrecognized command line option '-qopenmp-report' >> gfortran: error: unrecognized command line option '-r8' >> gfortran: error: unrecognized command line option '-WB' >> gfortran: error: unrecognized command line option '-traceback' >> gfortran: error: unrecognized command line option '-fpe3' >> gfortran: error: unrecognized command line option '-fpp' >> make: *** [lib12c-11_states-pn.a(kinetic.o)] Error 1 >> [knl00 mc] which mpif90 >> /soft/compilers/intel_beta/compilers_and_libraries_2017.0.042/linux/mpi/intel64/bin/mpif90 >> >> >> Steve >> Steven C. Pieper, Senior Physicist SO: spieper(a)anl.gov >> Argonne National Laboratory, Physics Division, Bldg. 203, Argonne, IL 60439 >> Phone: 630-252-4232 Fax -6008 >> Secretary, Debra Beres, -4100 >> On 06/16/16 14:53, Harms, Kevin wrote: >> >> >> Oops. I forgot to put “intel64” at the end. The wiki has some useful notes on it: >> >> https://wiki.jlse.anl.gov/display/knl/Intel+KNL >> >> kevin >> >> Steve, >> >> First thing, try the latest compiler. >> >> source /soft/compilers/intel_beta/compilers_and_libraries/linux/bin/compilervars.sh >> >> The 2017 beta has explicit KNL support. Second, have you tried running it in gdb? >> >> kevin >> >> I am trying to run a Fortran program that exercises some of my computational >> kernels. The program does not use MPI, just OpenMP. I have compiled >> it for a small case and that runs fine and gives the correct results. >> However when I recompile it for a realistic cases (the same source >> is used, only a bunch of parameter statements are different), it >> segfaults at line 13 of the main program: >> >> [knl00 tests] ../mc/x.testwf.12c-11_states-pn c12_11states-pn.in >> forrtl: severe (174): SIGSEGV, segmentation fault occurred >> Image PC Routine Line Source >> >> libintlc.so.5 00007F569D060C05 Unknown Unknown Unknown >> libintlc.so.5 00007F569D05E827 Unknown Unknown Unknown >> libifcoremt.so.5 00007F569DEE1962 Unknown Unknown Unknown >> libifcoremt.so.5 00007F569DEE17B6 Unknown Unknown Unknown >> libifcoremt.so.5 00007F569DE36CAD Unknown Unknown Unknown >> libifcoremt.so.5 00007F569DE48FAD Unknown Unknown Unknown >> libpthread.so.0 00007F569F2D7100 Unknown Unknown Unknown >> x.testwf.12c-11_s 00000000004030F0 MAIN__ 13 testwf.f >> x.testwf.12c-11_s 0000000000402C0E Unknown Unknown Unknown >> libc.so.6 00007F569CAA3B15 Unknown Unknown Unknown >> x.testwf.12c-11_s 0000000000402B19 Unknown Unknown Unknown >> Aborted (core dumped) >> [knl00 tests] >> >> >> Line 13 is the very beginning of declarations, not an executable statement. >> It is the first non-comment line and is: >> >> use random2 >> >> random2 is a module that defines some subroutines. It is not changed >> by the parameter statements that are different for the working and failing >> cases. >> >> Here are the working and failing executable sizes: >> >> text data bss dec hex filename >> 2459352 124772 24979752 27563876 1a49764 x.testwf.4he-pn >> 2192689 92004 623277384 625562077 254951dd x.testwf.12c-11_states-pn >> >> Here is how the main program is compiled and module linked: >> >> ifort -Imods.libgen -FR -g -O0 -qopenmp -lpthread -qopenmp-report -r8 -WB -traceback -fpe3 -I../common/ -I../mc/ -fpp -DSPIN_SUF= -DPARAMNUC=nclude/paramnuc.12c-11_states-pn -DM0_DIV=1 -DDOCASE=0 -DPARAMCASE=nclude/paramcase.12c-11_states-pn -c -g testwf.f >> >> ifort -g -mcmodel=medium -traceback -fpe3 -qopenmp -lpthread -qopenmp-report -o x.testwf.12c-11_states-pn testwf.o do_adlb_no.o dmem_fake.o adlb_fake.o \ >> ../common/wall_second-nompi.o ../common/stopnompi.o ../common/mpi_wtime_fake.o -L . -l12c-11_states-pn -lgen -l12c-11_states-pn -L ../common/ -lcomgen enable_abort_fake.o -lsvml >> >> >> [knl00 mc] which ifort >> /soft/compilers/intel/bin/ifort >> >> >> This is in ~b21541/gfmc/mc . >> >> This test case runs fine on blues. >> >> Any suggestions will be gratefully received. >> >> Steve >> -- >> Steven C. Pieper, Senior Physicist SO: spieper(a)anl.gov >> Argonne National Laboratory, Physics Division, Bldg. 203, Argonne, IL 60439 >> Phone: 630-252-4232 Fax -6008 >> Secretary, Debra Beres, -4100 >> >> >> >> >> >> >> >> >> >> >> >> > >

1 0

Re: [JLSE-Intel-NDA] Help on knl00 segfault
by Harms, Kevin 16 Jun '16

16 Jun '16

Steve, I don’t know fortran well, is random2 part of the standard fortran or your code? kevin >OK, with mpiifort it compiles but it still fails the same way. >Here are the results from gdb: > >[knl00 tests] gdb ../mc/x.testwf.12c-11_states-pn >GNU gdb (GDB) Red Hat Enterprise Linux 7.6.1-80.el7 >Copyright (C) 2013 Free Software Foundation, Inc. >License GPLv3+: GNU GPL version 3 or later ><http://gnu.org/licenses/gpl.html> <http://gnu.org/licenses/gpl.html> >This is free software: you are free to change and redistribute it. >There is NO WARRANTY, to the extent permitted by law. Type "show copying" >and "show warranty" for details. >This GDB was configured as "x86_64-redhat-linux-gnu". >For bug reporting instructions, please see: ><http://www.gnu.org/software/gdb/bugs/> <http://www.gnu.org/software/gdb/bugs/>... >Reading symbols from /home/b21541/gfmc/mc/x.testwf.12c-11_states-pn...done. >(gdb) run >Starting program: /home/b21541/gfmc/tests/../mc/x.testwf.12c-11_states-pn >[Thread debugging using libthread_db enabled] >Using host libthread_db library "/lib64/libthread_db.so.1". > >Program received signal SIGSEGV, Segmentation fault. >0x00000000004030f0 in _unnamed_main$$ () at testwf.f:13 >13 use random2 >Missing separate debuginfos, use: debuginfo-install glibc-2.17-106.el7_2.4.x86_64 libgcc-4.8.5-4.el7.x86_64 >(gdb) > >(gdb) where >#0 0x00000000004030f0 in _unnamed_main$$ () at testwf.f:13 >#1 0x0000000000402c0e in main () >#2 0x00007ffff53b7b15 in __libc_start_main () from /lib64/libc.so.6 >#3 0x0000000000402b19 in _start () >(gdb) > > >I can come over to Bldg 240 if that seems helpful. > >Steve > > >Steven C. Pieper, Senior Physicist SO: spieper(a)anl.gov >Argonne National Laboratory, Physics Division, Bldg. 203, Argonne, IL 60439 >Phone: 630-252-4232 Fax -6008 >Secretary, Debra Beres, -4100 >On 06/16/16 15:08, Luo, Ye wrote: > > >Try mpiifort. >You can verify the use of ifort by "mpiifort -show". >Ye >=================== >Ye Luo, Ph.D. >Leadership Computing Facility >Argonne National Laboratory > > > >________________________________________ >From: >intel-nda-bounces(a)lists.jlse.anl.gov <mailto:[email protected]> [intel-nda-bounces(a)lists.jlse.anl.gov] on behalf of Steven C. Pieper [spieper(a)anl.gov] >Sent: Thursday, June 16, 2016 3:04 PM >To: Harms, Kevin; >intel-nda(a)lists.jlse.anl.gov <mailto:[email protected]>; Osborn, James C. >Subject: Re: [JLSE-Intel-NDA] Help on knl00 segfault > > >Hi Kevin, > >I did the source command and ifort gets the beta compiler OK. However mpif90 >is using gfortran: > >mpif90 -FR -g -O4 -mavx -xAVX -axAVX -qopenmp -lpthread -qopenmp-report -r8 -WB -traceback -fpe3 -I../common/ -I../mc/ -fpp -DSPIN_SUF= -DPARAMNUC=nclude/paramnuc.12c-11_states-pn -DM0_DIV=1 -DDOCASE=0 -DPARAMCASE=nclude/paramcase.12c-11_states-pn > -I../adlb/ -I../dmem/ -c kinetic.f >gfortran: error: unrecognized command line option '-axAVX' >gfortran: error: unrecognized command line option '-qopenmp' >gfortran: error: unrecognized command line option '-qopenmp-report' >gfortran: error: unrecognized command line option '-r8' >gfortran: error: unrecognized command line option '-WB' >gfortran: error: unrecognized command line option '-traceback' >gfortran: error: unrecognized command line option '-fpe3' >gfortran: error: unrecognized command line option '-fpp' >make: *** [lib12c-11_states-pn.a(kinetic.o)] Error 1 >[knl00 mc] which mpif90 >/soft/compilers/intel_beta/compilers_and_libraries_2017.0.042/linux/mpi/intel64/bin/mpif90 > > >Steve >Steven C. Pieper, Senior Physicist SO: spieper(a)anl.gov >Argonne National Laboratory, Physics Division, Bldg. 203, Argonne, IL 60439 >Phone: 630-252-4232 Fax -6008 >Secretary, Debra Beres, -4100 >On 06/16/16 14:53, Harms, Kevin wrote: > > > Oops. I forgot to put “intel64” at the end. The wiki has some useful notes on it: > > https://wiki.jlse.anl.gov/display/knl/Intel+KNL > >kevin > >Steve, > > First thing, try the latest compiler. > > source /soft/compilers/intel_beta/compilers_and_libraries/linux/bin/compilervars.sh > > The 2017 beta has explicit KNL support. Second, have you tried running it in gdb? > >kevin > >I am trying to run a Fortran program that exercises some of my computational >kernels. The program does not use MPI, just OpenMP. I have compiled >it for a small case and that runs fine and gives the correct results. >However when I recompile it for a realistic cases (the same source >is used, only a bunch of parameter statements are different), it >segfaults at line 13 of the main program: > >[knl00 tests] ../mc/x.testwf.12c-11_states-pn c12_11states-pn.in >forrtl: severe (174): SIGSEGV, segmentation fault occurred >Image PC Routine Line Source > >libintlc.so.5 00007F569D060C05 Unknown Unknown Unknown >libintlc.so.5 00007F569D05E827 Unknown Unknown Unknown >libifcoremt.so.5 00007F569DEE1962 Unknown Unknown Unknown >libifcoremt.so.5 00007F569DEE17B6 Unknown Unknown Unknown >libifcoremt.so.5 00007F569DE36CAD Unknown Unknown Unknown >libifcoremt.so.5 00007F569DE48FAD Unknown Unknown Unknown >libpthread.so.0 00007F569F2D7100 Unknown Unknown Unknown >x.testwf.12c-11_s 00000000004030F0 MAIN__ 13 testwf.f >x.testwf.12c-11_s 0000000000402C0E Unknown Unknown Unknown >libc.so.6 00007F569CAA3B15 Unknown Unknown Unknown >x.testwf.12c-11_s 0000000000402B19 Unknown Unknown Unknown >Aborted (core dumped) >[knl00 tests] > > >Line 13 is the very beginning of declarations, not an executable statement. >It is the first non-comment line and is: > > use random2 > >random2 is a module that defines some subroutines. It is not changed >by the parameter statements that are different for the working and failing >cases. > >Here are the working and failing executable sizes: > > text data bss dec hex filename >2459352 124772 24979752 27563876 1a49764 x.testwf.4he-pn >2192689 92004 623277384 625562077 254951dd x.testwf.12c-11_states-pn > >Here is how the main program is compiled and module linked: > >ifort -Imods.libgen -FR -g -O0 -qopenmp -lpthread -qopenmp-report -r8 -WB -traceback -fpe3 -I../common/ -I../mc/ -fpp -DSPIN_SUF= -DPARAMNUC=nclude/paramnuc.12c-11_states-pn -DM0_DIV=1 -DDOCASE=0 -DPARAMCASE=nclude/paramcase.12c-11_states-pn -c -g testwf.f > >ifort -g -mcmodel=medium -traceback -fpe3 -qopenmp -lpthread -qopenmp-report -o x.testwf.12c-11_states-pn testwf.o do_adlb_no.o dmem_fake.o adlb_fake.o \ > ../common/wall_second-nompi.o ../common/stopnompi.o ../common/mpi_wtime_fake.o -L . -l12c-11_states-pn -lgen -l12c-11_states-pn -L ../common/ -lcomgen enable_abort_fake.o -lsvml > > >[knl00 mc] which ifort >/soft/compilers/intel/bin/ifort > > >This is in ~b21541/gfmc/mc . > >This test case runs fine on blues. > >Any suggestions will be gratefully received. > >Steve >-- >Steven C. Pieper, Senior Physicist SO: spieper(a)anl.gov >Argonne National Laboratory, Physics Division, Bldg. 203, Argonne, IL 60439 >Phone: 630-252-4232 Fax -6008 >Secretary, Debra Beres, -4100 > > > > > > > > > > > > >

1 0

Re: [JLSE-Intel-NDA] Help on knl00 segfault
by Steven C. Pieper 16 Jun '16

16 Jun '16

OK, with mpiifort it compiles but it still fails the same way. Here are the results from gdb: [knl00 tests] gdb ../mc/x.testwf.12c-11_states-pn GNU gdb (GDB) Red Hat Enterprise Linux 7.6.1-80.el7 Copyright (C) 2013 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-redhat-linux-gnu". For bug reporting instructions, please see: <http://www.gnu.org/software/gdb/bugs/>... Reading symbols from /home/b21541/gfmc/mc/x.testwf.12c-11_states-pn...done. (gdb) run Starting program: /home/b21541/gfmc/tests/../mc/x.testwf.12c-11_states-pn [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". Program received signal SIGSEGV, Segmentation fault. 0x00000000004030f0 in _unnamed_main$$ () at testwf.f:13 13 use random2 Missing separate debuginfos, use: debuginfo-install glibc-2.17-106.el7_2.4.x86_64 libgcc-4.8.5-4.el7.x86_64 (gdb) (gdb) where #0 0x00000000004030f0 in _unnamed_main$$ () at testwf.f:13 #1 0x0000000000402c0e in main () #2 0x00007ffff53b7b15 in __libc_start_main () from /lib64/libc.so.6 #3 0x0000000000402b19 in _start () (gdb) I can come over to Bldg 240 if that seems helpful. Steve Steven C. Pieper, Senior Physicist SO: spieper(a)anl.gov Argonne National Laboratory, Physics Division, Bldg. 203, Argonne, IL 60439 Phone: 630-252-4232 Fax -6008 Secretary, Debra Beres, -4100 On 06/16/16 15:08, Luo, Ye wrote: > Try mpiifort. > You can verify the use of ifort by "mpiifort -show". > Ye > =================== > Ye Luo, Ph.D. > Leadership Computing Facility > Argonne National Laboratory > ------------------------------------------------------------------------ > *From:* intel-nda-bounces(a)lists.jlse.anl.gov > [intel-nda-bounces(a)lists.jlse.anl.gov] on behalf of Steven C. Pieper > [spieper(a)anl.gov] > *Sent:* Thursday, June 16, 2016 3:04 PM > *To:* Harms, Kevin; intel-nda(a)lists.jlse.anl.gov; Osborn, James C. > *Subject:* Re: [JLSE-Intel-NDA] Help on knl00 segfault > > Hi Kevin, > > I did the source command and ifort gets the beta compiler OK. However > mpif90 > is using gfortran: > > mpif90 -FR -g -O4 -mavx -xAVX -axAVX -qopenmp -lpthread > -qopenmp-report -r8 -WB -traceback -fpe3 -I../common/ -I../mc/ -fpp > -DSPIN_SUF= -DPARAMNUC=nclude/paramnuc.12c-11_states-pn -DM0_DIV=1 > -DDOCASE=0 -DPARAMCASE=nclude/paramcase.12c-11_states-pn -I../adlb/ > -I../dmem/ -c kinetic.f > gfortran: error: unrecognized command line option '-axAVX' > gfortran: error: unrecognized command line option '-qopenmp' > gfortran: error: unrecognized command line option '-qopenmp-report' > gfortran: error: unrecognized command line option '-r8' > gfortran: error: unrecognized command line option '-WB' > gfortran: error: unrecognized command line option '-traceback' > gfortran: error: unrecognized command line option '-fpe3' > gfortran: error: unrecognized command line option '-fpp' > make: *** [lib12c-11_states-pn.a(kinetic.o)] Error 1 > [knl00 mc] which mpif90 > /soft/compilers/intel_beta/compilers_and_libraries_2017.0.042/linux/mpi/intel64/bin/mpif90 > > > Steve > Steven C. Pieper, Senior Physicist SO:[email protected] > Argonne National Laboratory, Physics Division, Bldg. 203, Argonne, IL 60439 > Phone: 630-252-4232 Fax -6008 > Secretary, Debra Beres, -4100 > On 06/16/16 14:53, Harms, Kevin wrote: >> Oops. I forgot to put ï¿½intel64ï¿½ at the end. The wiki has some useful notes on it: >> >> https://wiki.jlse.anl.gov/display/knl/Intel+KNL >> >> kevin >> >>> Steve, >>> >>> First thing, try the latest compiler. >>> >>> source /soft/compilers/intel_beta/compilers_and_libraries/linux/bin/compilervars.sh >>> >>> The 2017 beta has explicit KNL support. Second, have you tried running it in gdb? >>> >>> kevin >>> >>>> I am trying to run a Fortran program that exercises some of my computational >>>> kernels. The program does not use MPI, just OpenMP. I have compiled >>>> it for a small case and that runs fine and gives the correct results. >>>> However when I recompile it for a realistic cases (the same source >>>> is used, only a bunch of parameter statements are different), it >>>> segfaults at line 13 of the main program: >>>> >>>> [knl00 tests] ../mc/x.testwf.12c-11_states-pn c12_11states-pn.in >>>> forrtl: severe (174): SIGSEGV, segmentation fault occurred >>>> Image PC Routine Line Source >>>> >>>> libintlc.so.5 00007F569D060C05 Unknown Unknown Unknown >>>> libintlc.so.5 00007F569D05E827 Unknown Unknown Unknown >>>> libifcoremt.so.5 00007F569DEE1962 Unknown Unknown Unknown >>>> libifcoremt.so.5 00007F569DEE17B6 Unknown Unknown Unknown >>>> libifcoremt.so.5 00007F569DE36CAD Unknown Unknown Unknown >>>> libifcoremt.so.5 00007F569DE48FAD Unknown Unknown Unknown >>>> libpthread.so.0 00007F569F2D7100 Unknown Unknown Unknown >>>> x.testwf.12c-11_s 00000000004030F0 MAIN__ 13 testwf.f >>>> x.testwf.12c-11_s 0000000000402C0E Unknown Unknown Unknown >>>> libc.so.6 00007F569CAA3B15 Unknown Unknown Unknown >>>> x.testwf.12c-11_s 0000000000402B19 Unknown Unknown Unknown >>>> Aborted (core dumped) >>>> [knl00 tests] >>>> >>>> >>>> Line 13 is the very beginning of declarations, not an executable statement. >>>> It is the first non-comment line and is: >>>> >>>> use random2 >>>> >>>> random2 is a module that defines some subroutines. It is not changed >>>> by the parameter statements that are different for the working and failing >>>> cases. >>>> >>>> Here are the working and failing executable sizes: >>>> >>>> text data bss dec hex filename >>>> 2459352 124772 24979752 27563876 1a49764 x.testwf.4he-pn >>>> 2192689 92004 623277384 625562077 254951dd x.testwf.12c-11_states-pn >>>> >>>> Here is how the main program is compiled and module linked: >>>> >>>> ifort -Imods.libgen -FR -g -O0 -qopenmp -lpthread -qopenmp-report -r8 -WB -traceback -fpe3 -I../common/ -I../mc/ -fpp -DSPIN_SUF= -DPARAMNUC=nclude/paramnuc.12c-11_states-pn -DM0_DIV=1 -DDOCASE=0 -DPARAMCASE=nclude/paramcase.12c-11_states-pn -c -g testwf.f >>>> >>>> ifort -g -mcmodel=medium -traceback -fpe3 -qopenmp -lpthread -qopenmp-report -o x.testwf.12c-11_states-pn testwf.o do_adlb_no.o dmem_fake.o adlb_fake.o \ >>>> ../common/wall_second-nompi.o ../common/stopnompi.o ../common/mpi_wtime_fake.o -L . -l12c-11_states-pn -lgen -l12c-11_states-pn -L ../common/ -lcomgen enable_abort_fake.o -lsvml >>>> >>>> >>>> [knl00 mc] which ifort >>>> /soft/compilers/intel/bin/ifort >>>> >>>> >>>> This is in ~b21541/gfmc/mc . >>>> >>>> This test case runs fine on blues. >>>> >>>> Any suggestions will be gratefully received. >>>> >>>> Steve >>>> -- >>>> Steven C. Pieper, Senior Physicist SO:[email protected] >>>> Argonne National Laboratory, Physics Division, Bldg. 203, Argonne, IL 60439 >>>> Phone: 630-252-4232 Fax -6008 >>>> Secretary, Debra Beres, -4100 >

1 0