Hi Daniel, Ok, thanks. The code is part of the SVM app on JLSE. The nested loop kernel offloaded to the csa is in this dir /home/elise/SVM/test Best wishes, Elise ________________________________ From: Woodworth, Daniel <[email protected]> Sent: Monday, December 18, 2017 12:49:35 PM To: Jennings, Elise; '[email protected]' Subject: RE: __builtin_csa_spmdization(k) with nested loops Ah, this assert is the one that triggers if the compiler runs out of lics: since we currently represent lics as physical registers, we can only have a finite number of them and this error happens if a function tries to use too many. We have someone working on improving this, but in practice code that hits the lic limits like this will usually also be too large to fit on the dataflow array anyway. You may need to decrease your worker count. Usually SPMDization should produce less code than unrolling, though, since it doesn’t need cleanup loops. What does the code that you’re working on look like? From: Jennings, Elise [mailto:[email protected]] Sent: Monday, December 18, 2017 1:09 PM To: Woodworth, Daniel <[email protected]>; '[email protected]' <[email protected]> Subject: Re: __builtin_csa_spmdization(k) with nested loops Hi Daniel, Error message is below. Thanks, Elise icpc -g -O3 -I/soft/restricted/intel_csa/comp/fsim/2017-12-04-release -c svm.cpp icpc -g -O3 -I/soft/restricted/intel_csa/comp/fsim/2017-12-04-release -c matrix_precomputation.cpp icpc -g -O3 -I/soft/restricted/intel_csa/comp/fsim/2017-12-04-release svm-train.c svm.o -o svm-train -lm -L/soft/restricted/intel_csa/comp/fsim/2017-12-04-release -lcsasim -lrt -ldl csa-clang -S -O2 csa_kernel.cpp clang-6.0: /nfs/hd/disks/csaobj/comp/tools/2017-12-04-release/tools/src/llvm/include/llvm/MC/MCRegisterInfo.h:64: unsigned int llvm::MCRegisterClass::getRegister(unsigned int) const: Assertion `i < getNumRegs() && "Register number out of range!"' failed. #0 0x00000000031e58f8 llvm::sys::PrintStackTrace(llvm::raw_ostream&) (/soft/restricted/intel_csa/comp/tools/2017-12-04-release/tools/install/bin/clang-6.0+0x31e58f8) #1 0x00000000031e5989 PrintStackTraceSignalHandler(void*) (/soft/restricted/intel_csa/comp/tools/2017-12-04-release/tools/install/bin/clang-6.0+0x31e5989) #2 0x00000000031e3e1c llvm::sys::RunSignalHandlers() (/soft/restricted/intel_csa/comp/tools/2017-12-04-release/tools/install/bin/clang-6.0+0x31e3e1c) #3 0x00000000031e51df SignalHandler(int) (/soft/restricted/intel_csa/comp/tools/2017-12-04-release/tools/install/bin/clang-6.0+0x31e51df) #4 0x00007f6963b055e0 __restore_rt (/lib64/libpthread.so.0+0xf5e0) #5 0x00007f6962c1d1f7 __GI_raise (/lib64/libc.so.6+0x351f7) #6 0x00007f6962c1e8e8 __GI_abort (/lib64/libc.so.6+0x368e8) #7 0x00007f6962c16266 __assert_fail_base (/lib64/libc.so.6+0x2e266) #8 0x00007f6962c16312 (/lib64/libc.so.6+0x2e312) #9 0x0000000001e7e097 llvm::MCRegisterClass::getRegister(unsigned int) const (/soft/restricted/intel_csa/comp/tools/2017-12-04-release/tools/install/bin/clang-6.0+0x1e7e097) #10 0x0000000001e7e113 llvm::TargetRegisterClass::getRegister(unsigned int) const (/soft/restricted/intel_csa/comp/tools/2017-12-04-release/tools/install/bin/clang-6.0+0x1e7e113) #11 0x0000000001e7e576 llvm::CSAMachineFunctionInfo::allocateLIC(llvm::TargetRegisterClass const*) (/soft/restricted/intel_csa/comp/tools/2017-12-04-release/tools/install/bin/clang-6.0+0x1e7e576) #12 0x0000000001f035a8 llvm::CSACvtCFDFPass::assignLicForDF() (/soft/restricted/intel_csa/comp/tools/2017-12-04-release/tools/install/bin/clang-6.0+0x1f035a8) #13 0x0000000001efad89 llvm::CSACvtCFDFPass::runOnMachineFunction(llvm::MachineFunction&) (/soft/restricted/intel_csa/comp/tools/2017-12-04-release/tools/install/bin/clang-6.0+0x1efad89) #14 0x00000000026bdb07 llvm::MachineFunctionPass::runOnFunction(llvm::Function&) (/soft/restricted/intel_csa/comp/tools/2017-12-04-release/tools/install/bin/clang-6.0+0x26bdb07) #15 0x0000000002b53260 llvm::FPPassManager::runOnFunction(llvm::Function&) (/soft/restricted/intel_csa/comp/tools/2017-12-04-release/tools/install/bin/clang-6.0+0x2b53260) #16 0x0000000002b533f3 llvm::FPPassManager::runOnModule(llvm::Module&) (/soft/restricted/intel_csa/comp/tools/2017-12-04-release/tools/install/bin/clang-6.0+0x2b533f3) #17 0x0000000002b5377f (anonymous namespace)::MPPassManager::runOnModule(llvm::Module&) (/soft/restricted/intel_csa/comp/tools/2017-12-04-release/tools/install/bin/clang-6.0+0x2b5377f) #18 0x0000000002b53e8a llvm::legacy::PassManagerImpl::run(llvm::Module&) (/soft/restricted/intel_csa/comp/tools/2017-12-04-release/tools/install/bin/clang-6.0+0x2b53e8a) #19 0x0000000002b540cb llvm::legacy::PassManager::run(llvm::Module&) (/soft/restricted/intel_csa/comp/tools/2017-12-04-release/tools/install/bin/clang-6.0+0x2b540cb) #20 0x0000000003464bbb (anonymous namespace)::EmitAssemblyHelper::EmitAssembly(clang::BackendAction, std::unique_ptr<llvm::raw_pwrite_stream, std::default_delete<llvm::raw_pwrite_stream> >) (/soft/restricted/intel_csa/comp/tools/2017-12-04-release/tools/install/bin/clang-6.0+0x3464bbb) #21 0x000000000346737e clang::EmitBackendOutput(clang::DiagnosticsEngine&, clang::HeaderSearchOptions const&, clang::CodeGenOptions const&, clang::TargetOptions const&, clang::LangOptions const&, llvm::DataLayout const&, llvm::Module*, clang::BackendAction, std::unique_ptr<llvm::raw_pwrite_stream, std::default_delete<llvm::raw_pwrite_stream> >) (/soft/restricted/intel_csa/comp/tools/2017-12-04-release/tools/install/bin/clang-6.0+0x346737e) #22 0x0000000003f2c4df clang::BackendConsumer::HandleTranslationUnit(clang::ASTContext&) (/soft/restricted/intel_csa/comp/tools/2017-12-04-release/tools/install/bin/clang-6.0+0x3f2c4df) #23 0x0000000004b827ec clang::ParseAST(clang::Sema&, bool, bool) (/soft/restricted/intel_csa/comp/tools/2017-12-04-release/tools/install/bin/clang-6.0+0x4b827ec) #24 0x0000000003a5fd10 clang::ASTFrontendAction::ExecuteAction() (/soft/restricted/intel_csa/comp/tools/2017-12-04-release/tools/install/bin/clang-6.0+0x3a5fd10) #25 0x0000000003f2fcc6 clang::CodeGenAction::ExecuteAction() (/soft/restricted/intel_csa/comp/tools/2017-12-04-release/tools/install/bin/clang-6.0+0x3f2fcc6) #26 0x0000000003a5f6ff clang::FrontendAction::Execute() (/soft/restricted/intel_csa/comp/tools/2017-12-04-release/tools/install/bin/clang-6.0+0x3a5f6ff) #27 0x0000000003a073b6 clang::CompilerInstance::ExecuteAction(clang::FrontendAction&) (/soft/restricted/intel_csa/comp/tools/2017-12-04-release/tools/install/bin/clang-6.0+0x3a073b6) #28 0x0000000003b8f740 clang::ExecuteCompilerInvocation(clang::CompilerInstance*) (/soft/restricted/intel_csa/comp/tools/2017-12-04-release/tools/install/bin/clang-6.0+0x3b8f740) #29 0x0000000001e63868 cc1_main(llvm::ArrayRef<char const*>, char const*, void*) (/soft/restricted/intel_csa/comp/tools/2017-12-04-release/tools/install/bin/clang-6.0+0x1e63868) #30 0x0000000001e59f96 ExecuteCC1Tool(llvm::ArrayRef<char const*>, llvm::StringRef) (/soft/restricted/intel_csa/comp/tools/2017-12-04-release/tools/install/bin/clang-6.0+0x1e59f96) #31 0x0000000001e5ae92 main (/soft/restricted/intel_csa/comp/tools/2017-12-04-release/tools/install/bin/clang-6.0+0x1e5ae92) #32 0x00007f6962c09c05 __libc_start_main (/lib64/libc.so.6+0x21c05) #33 0x0000000001e561b1 _start /usr/src/packages/BUILD/glibc-2.11.3/csu/../sysdeps/x86_64/elf/start.S:116:0 Stack dump: 0. Program arguments: /soft/restricted/intel_csa/comp/tools/2017-12-04-release/tools/install/bin/clang-6.0 -cc1 -triple csa -S -disable-free -main-file-name csa_kernel.cpp -mrelocation-model static -mthread-model posix -mdisable-fp-elim -no-integrated-as -mconstructor-aliases -fno-builtin-memset -fno-builtin-memcpy -include __clang_csa_math.h -dwarf-column-info -debugger-tuning=gdb -coverage-notes-file /home/elise/WFG/SVM/test/csa_kernel.gcno -resource-dir /soft/restricted/intel_csa/comp/tools/2017-12-04-release/tools/install/lib/clang/6.0.0 -I/soft/compilers/intel/compilers_and_libraries_2018.1.163/linux/ipp/include -I/soft/compilers/intel/compilers_and_libraries_2018.1.163/linux/mkl/include -I/soft/compilers/intel/compilers_and_libraries_2018.1.163/linux/pstl/include -I/soft/compilers/intel/compilers_and_libraries_2018.1.163/linux/tbb/include -I/soft/compilers/intel/compilers_and_libraries_2018.1.163/linux/tbb/include -I/soft/compilers/intel/compilers_and_libraries_2018.1.163/linux/daal/include -I/usr/intel/pkgs/ics/2013.1e/composer_xe_2013_sp1.2.144/tbb/include/ -internal-isystem /usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../include/c++/4.8.5 -internal-isystem /usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../include/c++/4.8.5/x86_64-redhat-linux -internal-isystem /usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../include/c++/4.8.5/backward -internal-isystem /usr/local/include -internal-isystem /soft/restricted/intel_csa/comp/tools/2017-12-04-release/tools/install/lib/clang/6.0.0/include -internal-externc-isystem /include -internal-externc-isystem /usr/include -O2 -fdeprecated-macro -fno-dwarf-directory-asm -fdebug-compilation-dir /home/elise/WFG/SVM/test -ferror-limit 19 -fmessage-length 164 -fobjc-runtime=gcc -fcxx-exceptions -fexceptions -fdiagnostics-show-option -fcolor-diagnostics -vectorize-loops -vectorize-slp -o csa_kernel.s -x c++ csa_kernel.cpp 1. <eof> parser at end of file 2. Code generation 3. Running pass 'Function Pass Manager' on module 'csa_kernel.cpp'. 4. Running pass 'CSA Convert Control Flow to Data Flow' on function '@csa_kernel' clang-6.0: error: unable to execute command: Aborted clang-6.0: error: clang frontend command failed due to signal (use -v to see invocation) clang version 6.0.0 (ssh://git-amr-3.devtools.intel.com:29418/dcg-knp-arch-lpu-clang fe29d5a51c355eecac19d23a9cd0999976335069) (ssh://git-amr-3.devtools.intel.com:29418/dcg-knp-arch-lpu-llvm 719142b6d797956bb47096cbd9a93c2e7ab8ce51) Target: csa Thread model: posix InstalledDir: /soft/restricted/intel_csa/comp/tools/2017-12-04-release/tools/local/bin clang-6.0: note: diagnostic msg: PLEASE submit a bug report to http://llvm.org/bugs/ and include the crash backtrace, preprocessed source, and associated run script. clang-6.0: note: diagnostic msg: ******************** PLEASE ATTACH THE FOLLOWING FILES TO THE BUG REPORT: Preprocessed source(s) and associated run script(s) are located at: clang-6.0: note: diagnostic msg: /tmp/csa_kernel-22f338.cpp clang-6.0: note: diagnostic msg: /tmp/csa_kernel-22f338.sh clang-6.0: note: diagnostic msg: ******************** make: *** [csa_kernel.s] Error 254 [elise@jlselogin1 test]$ ________________________________ From: Woodworth, Daniel <[email protected]<mailto:[email protected]>> Sent: Monday, December 18, 2017 10:09:36 AM To: Jennings, Elise; '[email protected]' Subject: RE: __builtin_csa_spmdization(k) with nested loops You should be able to. What error message are you getting? From: anl-intel-csa [mailto:[email protected]] On Behalf Of Jennings, Elise Sent: Monday, December 18, 2017 10:56 AM To: '[email protected]' <[email protected]<mailto:[email protected]>> Subject: [Anl-intel-csa] __builtin_csa_spmdization(k) with nested loops Hi all, Can I use __builtin_csa_spmdization(k) on nested loops? Previously I used two __builtin_csa_parallel_region entry calls each with different unroll factors. When I replace both with SPMDization I get errors when compiling. If I replace only one __builtin_cas_parallel region is works fine. Thanks, Elise ________________________________ From: anl-intel-csa <[email protected]<mailto:[email protected]>> on behalf of Glossop, Kent <[email protected]<mailto:[email protected]>> Sent: Monday, December 4, 2017 9:42 PM To: '[email protected]' Subject: [Anl-intel-csa] CSA SDK Release Notes for 2017/12/04 CSA SDK Release Notes for 2017/12/04 version The default release has been changed to 2017-12-04-release. Relevant changes since the 2017-11-21-release: SDK Environment • The SDK environmental setup now provides a “csaenv” command. After invoking the csasetup script, the “latest” tools will be selected. The “csaenv” command can be used to show what is set, and to switch between versions if necessary. o Once csasetup.bash is sourced, user can use ‘csaenv’ to switch between environments o Examples: • `csaenv list` • Shows all available environments in system • `csaenv set fsim/2017-12-04-release tools/2017-11-29-release’ • Sets current environment with fsim built in 2017-12-04 and tools built in 2017-11-29 o Execute `csaenv help` for more details” Compiler / Runtime • Added a warning when the SPMDization region is not structured code; the code should be OpenMP-conforming (branches to or from the SPMDization region (continue, break, etc.) are illegal) • Fixes: o Add the zero trip count guard to each of the generated workers. Now, SPMDization should generate correct code when 0 <= #iterations < #workers Advisor • Introduced special mode for Dependencies analysis (-loop-call-count-limit=N, switched off by default) which helps to dramatically reduce Spatial Advisor Dependencies overhead o Example: advixe-cl --collect dependencies -filter-reductions -loops=”scalar,loop-height=0” -loop-call-count-limit=16 … • Improved SAPPC html report output and included new metrics (e.g. tripcounts, normalized total time) • Improved SAPPC performance by disabling Advisor “recommendations engine” (which is not yet used in SAPPC)