Reinstalled the kernel modules, and fixed kernel.perf_event_paranoid again on knl00. Looks like /proc/sys/kernel/perf_event_mlock_kb is 516 currently. Possibly related ulimit -l is also only 64k currently. Ben
On Jul 25, 2016, at 1:41 PM, Delalondre Fabien Jonathan <[email protected]> wrote:
Hi Ben,
So I tried the following test on KNL00 then rebooted (and forgot the image would be wiped out...). Sorry about that Ben, can you please re-apply the image with fix on KNL00 so that I can test the quadrant/cache configuration because I believe the issue I have been running into (related to heap memory in SNC4/cache mode) over the week end is a different one...
I am using the following compilers and vtune versions source /soft/compilers/intel_beta/compilers_and_libraries/linux/bin/compilervars.sh intel64 source /soft/compilers/intel_beta/compilers_and_libraries/linux/mpi/bin64/mpivars.sh source /soft/compilers/intel/vtune_amplifier_xe_2016.4.0.470476/amplxe-vars.sh
** Test 1 - SNC4/Cache I had configured the system in SNC4-Cache last Friday. Did some testing over the week end but ran into issues related to heap allocation. I d like to possibly discuss those after I am done with reading the Intel KNL book.
Nevertheless, executed the following command on this configuration (which is a test I could execute fine last May): amplxe-cl -c memory-access -knob analyze-mem-objects=true -knob sampling-interval=10 -data-limit=0 -d 60 -- mpirun -n $NUM_NODES ... <EXE> <EXE_PARAM>
I get the following error is: amplxe: Collection started. To stop the collection, either press CTRL-C or enter from another console window: amplxe-cl -r /home/delalond/test_knl_cn/Result4.knl00.ftm.alcf.anl.gov -command stop. amplxe: Error: Permission error mapping pages. Consider increasing /proc/sys/kernel/perf_event_mlock_kb, or try again with a smaller value of -m/--mmap_pages. (current value: 4294967295)
amplxe: Error: Permission error mapping pages. Consider increasing /proc/sys/kernel/perf_event_mlock_kb, or try again with a smaller value of -m/--mmap_pages. (current value: 4294967295)
amplxe: Collection failed. amplxe: Collection failed. amplxe: Internal Error amplxe: Internal Error amplxe: Error: Permission error mapping pages. Consider increasing /proc/sys/kernel/perf_event_mlock_kb, or try again with a smaller value of -m/--mmap_pages. (current value: 4294967295)
amplxe: Collection failed. amplxe: Internal Error
________________________________________ From: Allen, Benjamin S. <[email protected]> Sent: Monday, July 25, 2016 7:04 PM To: Delalondre Fabien Jonathan Cc: [email protected]; Kumaran, Kalyan Subject: Re: [JLSE-Intel-NDA] Last version of compiler & Vtune working with memory analysis
Hi Fabien,
Could you test this on knl00? I installed the sampling driver and set perf_event_paranoid to 0 on the running OS image (i.e. it will go away if rebooted). I'd like to see if everything is working before adding it the the OS image.
Thanks,
Ben
On Jul 22, 2016, at 4:44 PM, Delalondre Fabien Jonathan <[email protected]> wrote:
Cool ! Thanks ben. Let me know when I can give it a try again.
Best,
Fabien ________________________________________ From: Allen, Benjamin S. <[email protected]> Sent: Friday, July 22, 2016 11:39 PM To: Delalondre Fabien Jonathan Cc: [email protected]; Kumaran, Kalyan Subject: Re: [JLSE-Intel-NDA] Last version of compiler & Vtune working with memory analysis
Hi Fabien,
Both these items are on my todo list.
1. Loading the sep4_0 driver 2. Setting perf_event_paranoid to 0
Should get to them sometime next week.
Ben
On Jul 22, 2016, at 3:55 PM, Delalondre Fabien Jonathan <[email protected]> wrote:
Thanks All.
So I face a couple of issues. Please let me know if this is coming from me or something else:
On knl00 (configured, SNC4-Cache) I use: compiler (2017): source /soft/compilers/intel_beta/compilers_and_libraries/linux/bin/compilervars.sh intel64 source /soft/compilers/intel_beta/compilers_and_libraries/linux/mpi/bin64/mpivars.sh vtune (2016 v4): source /soft/compilers/intel/vtune_amplifier_xe_2016.4.0.470476/amplxe-vars.sh
When I execute the an MPI+OpenMP application 1) It works (running basic vtune hot spot analysis) amplxe-cl -collect hotspots -- mpirun -n $NODES <EXEC> <ParamEXEC>
2) It used to work in May (with Vtune patched NDA - now it returns an error) amplxe-cl -c memory-access -knob analyze-mem-objects=true -knob sampling-interval=10 -data-limit=0 -d 60 -- mpirun -n $NODES <EXEC> <ParamEXEC>
amplxe: Error: Cannot enable Hardware Event-based Sampling: problem with the driver (sep*/sepdrv*). Check that the driver is running and the driver group is in the current user group list. See "Building and Managing the Sampling Driver" help topic for further details.
3) Instructions provided on wiki by Vitali do not work for me due to lack of permission (set to 1 in X file)
mpirun -n $NODE_COUNT -env KMP_AFFINITY=balanced,granularity=fine -gtool "/soft/restricted/intel-vtune-nda/vtune_amplifier_xe/bin64/amplxe-cl -collect memory-access -knob sampling-interval=10 -knob analyze-mem-objects=true -data-limit=1500 -mrte-mode=native -result-dir=Result4:0-3" <EXEC> <EXEC_PARAM>
Error message: amplxe: Error: Not enough permissions to run uncore events collection. Consider setting the /proc/sys/kernel/perf_event_paranoid value to 0 or less.
Any hint ? Can we change the permissions to run uncore events collection on all KNL? Let me know if I am doing something wrong.
Thanks,
Fabien
________________________________________ From: Allen, Benjamin S. <[email protected]> Sent: Friday, July 22, 2016 3:15 PM To: Delalondre Fabien Jonathan Cc: [email protected]; Kumaran, Kalyan Subject: Re: [JLSE-Intel-NDA] Last version of compiler & Vtune working with memory analysis
VTune 2016 Update 4 added KNL support, so the NDA'ed add-on bits are no longer needed. The compilers in 2017 and 2016 haven't been updated recently, so they should be the same as when you last worked with them.
Ben
On Jul 22, 2016, at 7:15 AM, Kumaran, Kalyan <[email protected]> wrote:
You can check, but I think Release 16 is a sure bet and had full KNL support.
-Kumar
On Jul 21, 2016, at 6:00 PM, Delalondre Fabien Jonathan <[email protected]> wrote:
Hi All,
Sorry to ask but ... Can someone confirm which version of the compiler and Vtune work fine together for the data memory analysis.
I double check Vitali's comments on the wiki but I have been told everything should work fine with the latest 2017 versions.
Thanks !
Fabien _______________________________________________ Intel-nda mailing list [email protected] https://lists.jlse.anl.gov/mailman/listinfo/intel-nda
_______________________________________________ Intel-nda mailing list [email protected] https://lists.jlse.anl.gov/mailman/listinfo/intel-nda