April 2016 - Intel-nda - mailman3.cels.anl.gov

Re: [JLSE-Intel-NDA] Intel MPSP
by Vitali A. Morozov 12 Apr '16

12 Apr '16

#include <hbwmalloc.h> -lmemkind Vitali On 04/12/2016 11:16 AM, Allen, Benjamin S. wrote: > Missing libraries? Or does it actually need to be on a KNL? > > Ben > >> On Apr 12, 2016, at 11:14 AM, Vitali A. Morozov <morozov(a)anl.gov> wrote: >> >> One interesting side effect is that we cannot compile the code on regular Xeon box if it uses MPSP. >> >> Vitali >> >> >> On 04/11/2016 03:57 PM, Allen, Benjamin S. wrote: >>> MPSP is now install on knl00. >>> >>> Ben >>> >>> >>>> On Apr 11, 2016, at 12:49 PM, Allen, Benjamin S. <bsallen(a)alcf.anl.gov> >>>> wrote: >>>> >>>> It includes the kernel SRPM if you want to go spelunking for patches. >>>> >>>> Ben >>>> >>>> >>>>> On Apr 11, 2016, at 12:45 PM, Harms, Kevin <harms(a)alcf.anl.gov> >>>>> wrote: >>>>> >>>>> >>>>> Getting the updated jemalloc and memkind libraries would be worth it if we are going to test the flat mode. It might be interesting to get and run the STREAM test in the micperf package to demonstrate the memory bandwidth seems to be incorrect. >>>>> >>>>> The document doesn't really go into what the kernel changes were other than vague statements of fixes or support of additional features. >>>>> >>>>> kevin >>>>> >>>>> >>>>> >>>>> >>>>> >>>>>> Intel has a patched RHEL7.2 kernel and a number of other tools that appear to better support KNL. Attached the user guide. Let me know if I should install this on knl00. >>>>>> >>>>>> Ben >>>>>> >>>> _______________________________________________ >>>> Intel-nda mailing list >>>> >>>> Intel-nda(a)lists.jlse.anl.gov >>>> https://lists.jlse.anl.gov/mailman/listinfo/intel-nda >>> >>> _______________________________________________ >>> Intel-nda mailing list >>> >>> Intel-nda(a)lists.jlse.anl.gov >>> https://lists.jlse.anl.gov/mailman/listinfo/intel-nda

1 0

Re: [JLSE-Intel-NDA] Intel MPSP
by Allen, Benjamin S. 12 Apr '16

12 Apr '16

Missing libraries? Or does it actually need to be on a KNL? Ben > On Apr 12, 2016, at 11:14 AM, Vitali A. Morozov <morozov(a)anl.gov> wrote: > > One interesting side effect is that we cannot compile the code on regular Xeon box if it uses MPSP. > > Vitali > > > On 04/11/2016 03:57 PM, Allen, Benjamin S. wrote: >> MPSP is now install on knl00. >> >> Ben >> >> >>> On Apr 11, 2016, at 12:49 PM, Allen, Benjamin S. <bsallen(a)alcf.anl.gov> >>> wrote: >>> >>> It includes the kernel SRPM if you want to go spelunking for patches. >>> >>> Ben >>> >>> >>>> On Apr 11, 2016, at 12:45 PM, Harms, Kevin <harms(a)alcf.anl.gov> >>>> wrote: >>>> >>>> >>>> Getting the updated jemalloc and memkind libraries would be worth it if we are going to test the flat mode. It might be interesting to get and run the STREAM test in the micperf package to demonstrate the memory bandwidth seems to be incorrect. >>>> >>>> The document doesn't really go into what the kernel changes were other than vague statements of fixes or support of additional features. >>>> >>>> kevin >>>> >>>> >>>> >>>> >>>> >>>>> Intel has a patched RHEL7.2 kernel and a number of other tools that appear to better support KNL. Attached the user guide. Let me know if I should install this on knl00. >>>>> >>>>> Ben >>>>> >>> _______________________________________________ >>> Intel-nda mailing list >>> >>> Intel-nda(a)lists.jlse.anl.gov >>> https://lists.jlse.anl.gov/mailman/listinfo/intel-nda >> >> >> _______________________________________________ >> Intel-nda mailing list >> >> Intel-nda(a)lists.jlse.anl.gov >> https://lists.jlse.anl.gov/mailman/listinfo/intel-nda >

1 0

Re: [JLSE-Intel-NDA] Intel MPSP
by Vitali A. Morozov 12 Apr '16

12 Apr '16

One interesting side effect is that we cannot compile the code on regular Xeon box if it uses MPSP. Vitali On 04/11/2016 03:57 PM, Allen, Benjamin S. wrote: > MPSP is now install on knl00. > > Ben > >> On Apr 11, 2016, at 12:49 PM, Allen, Benjamin S. <bsallen(a)alcf.anl.gov> wrote: >> >> It includes the kernel SRPM if you want to go spelunking for patches. >> >> Ben >> >>> On Apr 11, 2016, at 12:45 PM, Harms, Kevin <harms(a)alcf.anl.gov> wrote: >>> >>> >>> Getting the updated jemalloc and memkind libraries would be worth it if we are going to test the flat mode. It might be interesting to get and run the STREAM test in the micperf package to demonstrate the memory bandwidth seems to be incorrect. >>> >>> The document doesn't really go into what the kernel changes were other than vague statements of fixes or support of additional features. >>> >>> kevin >>> >>> >>> >>> >>>> Intel has a patched RHEL7.2 kernel and a number of other tools that appear to better support KNL. Attached the user guide. Let me know if I should install this on knl00. >>>> >>>> Ben >> _______________________________________________ >> Intel-nda mailing list >> Intel-nda(a)lists.jlse.anl.gov >> https://lists.jlse.anl.gov/mailman/listinfo/intel-nda > > > _______________________________________________ > Intel-nda mailing list > Intel-nda(a)lists.jlse.anl.gov > https://lists.jlse.anl.gov/mailman/listinfo/intel-nda

1 0

[JLSE-Intel-NDA] Fwd: KNL Status
by Kumaran, Kalyan 12 Apr '16

12 Apr '16

Keeping the list abreast on discussions with Intel on Stream results. Begin forwarded message: From: "Raman, Karthik" <karthik.raman(a)intel.com<mailto:[email protected]>> Subject: RE: [JLSE-Intel-NDA] KNL Status Date: April 12, 2016 at 1:51:44 AM CDT To: "Chinthamani, Sundaram" <sundaram.chinthamani(a)intel.com<mailto:[email protected]>>, Vitali Morozov <morozov(a)anl.gov<mailto:[email protected]>>, "Kumaran, Kalyan" <kumaran(a)alcf.anl.gov<mailto:[email protected]>> Cc: "Sodani, Avinash" <avinash.sodani(a)intel.com<mailto:[email protected]>>, "Fromkin, Russ" <russ.fromkin(a)intel.com<mailto:[email protected]>>, "Harms, Kevin" <harms(a)alcf.anl.gov<mailto:[email protected]>>, "Morozov, Vitali A." <morozov(a)alcf.anl.gov<mailto:[email protected]>>, "Allen, Benjamin S." <bsallen(a)alcf.anl.gov<mailto:[email protected]>> Also could you please modify the memory allocations in STREAM to align at 2MB boundary. For example: static allocations: static STREAM_TYPE a[STREAM_ARRAY_SIZE+OFFSET] __attribute__((aligned(2097152))) dynamic allocations: a = (STREAM_TYPE *)_mm_malloc(sizeof(STREAM_TYPE)*(STREAM_ARRAY_SIZE+OFFSET),2097152); memkind/hbwalloc: (STREAM_TYPE *)hbw_posix_memalign((void **)&a, 2097152, sizeof(STREAM_TYPE)*(STREAM_ARRAY_SIZE+OFFSET)) For memkind/hbwmalloc to work out of the box, you need to have the PMTT tables populated so that it can identify the high bandwidth nodes. Alternatively you can set the MEMKIND_HBW_NODES environment variable to the high bandwidth node (MCDRAM). e.g. All2All/Flat or Quad/Flat : export MEMKIND_HBW_NODES=1 ; ./app_with_hbwmalloc SNC4/Flat: export MEMKIND_HBW_NODES=4,5,6,7 ; ./app_with_hbw_malloc Thanks Karthik From: Chinthamani, Sundaram Sent: Monday, April 11, 2016 8:15 PM To: Vitali Morozov <morozov(a)anl.gov<mailto:[email protected]>>; Kumaran, Kalyan <kumaran(a)alcf.anl.gov<mailto:[email protected]>> Cc: Sodani, Avinash <avinash.sodani(a)intel.com<mailto:[email protected]>>; Fromkin, Russ <russ.fromkin(a)intel.com<mailto:[email protected]>>; Raman, Karthik <karthik.raman(a)intel.com<mailto:[email protected]>>; Harms, Kevin <harms(a)alcf.anl.gov<mailto:[email protected]>>; Morozov, Vitali A. <morozov(a)alcf.anl.gov<mailto:[email protected]>>; Allen, Benjamin S. <bsallen(a)alcf.anl.gov<mailto:[email protected]>>; Chinthamani, Sundaram <sundaram.chinthamani(a)intel.com<mailto:[email protected]>> Subject: RE: [JLSE-Intel-NDA] KNL Status You should use numactl –m 4,5,6,7 to get the memory bandwidth out of MC-DRAM. What kind of part is it? Is it bin1 or bin3? Even in All2All mode, we have measured up to 470 GB/s of stream triad bandwidth. Quad mode should get 490 GB/s. DDR-stream triad bandwidth on Bin1-part should be close to 90 GB/s. -Sundaram From: Vitali Morozov [mailto:[email protected]] Sent: Monday, April 11, 2016 6:59 PM To: Kumaran, Kalyan <kumaran(a)alcf.anl.gov<mailto:[email protected]>> Cc: Sodani, Avinash <avinash.sodani(a)intel.com<mailto:[email protected]>>; Fromkin, Russ <russ.fromkin(a)intel.com<mailto:[email protected]>>; Chinthamani, Sundaram <sundaram.chinthamani(a)intel.com<mailto:[email protected]>>; Raman, Karthik <karthik.raman(a)intel.com<mailto:[email protected]>>; Harms, Kevin <harms(a)alcf.anl.gov<mailto:[email protected]>>; Morozov, Vitali A. <morozov(a)alcf.anl.gov<mailto:[email protected]>>; Allen, Benjamin S. <bsallen(a)alcf.anl.gov<mailto:[email protected]>> Subject: Re: [JLSE-Intel-NDA] KNL Status Avinash and others, here is what we have so far for Flat more, All2All. The results from memory were obtained by using default STREAM with static allocation. The results from MCDRAM were obtained by using modified STREAM with hbwmalloc.h interface. 1 MPI process, 64 threads, scatter affinity, no numactl is used. Most results are as expected: 80 GB/s from DDR memory, 420 GB/s from MCDRAM, the MCDRAM size is about 16 GB, the aggregate L2 cache size is 32 MB. Noticed inconsistencies for hbwmalloc benchmarks: 1. The result is not validated - will spend more time on that tomorrow. 2. hbw_malloc continues allocating data beyond the expected 16 GB up to 48 GB. This means it fails back to DDR if it cannot allocate in MCDRAM - not expected default behavior. It must fail if it does not have enough space. The problem shows up when I am running static allocation and use numactl --membind=1 to ask for MCDRAM-only allocation. The variation is very large and inconsistent with the result observed when using hbw_malloc API: # MB N GB/s numactl GB/s hbw_malloc 228.88 10000000 255555.46 421009.18 457.76 20000000 363930.93 416997.91 686.65 30000000 309415.87 427808.31 915.53 40000000 371691.30 423310.75 1144.41 50000000 187002.22 431032.35 1373.29 60000000 385954.23 427172.91 1602.17 70000000 388254.49 426823.59 1831.05 80000000 374491.43 428103.96 2059.94 90000000 379622.74 428152.02 2288.82 100000000 80868.34 430185.03 We also tried SNC-4 instead of All2All, but could not get more than 40 GB/s from MCDRAM even with hbw_malloc API. Raw data: # flat memory, running from memory (default), alltoall # MB N GB/s 2.29 100000 272062.96 4.58 200000 491040.47 6.87 300000 603979.78 9.16 400000 745654.04 11.44 500000 811800.77 13.73 600000 901462.35 16.02 700000 891953.26 18.31 800000 1019375.15 20.60 900000 984749.63 22.89 1000000 923516.48 25.18 1100000 758422.09 27.47 1200000 779328.74 29.75 1300000 513185.43 32.04 1400000 155550.35 34.33 1500000 103209.12 36.62 1600000 97435.74 38.91 1700000 83968.40 41.20 1800000 83692.35 43.49 1900000 80734.60 45.78 2000000 76317.89 68.66 3000000 81180.08 91.55 4000000 78889.73 114.44 5000000 79891.50 137.33 6000000 79954.96 160.22 7000000 79477.00 183.11 8000000 79145.59 205.99 9000000 79821.12 228.88 10000000 79519.15 228.88 10000000 79212.54 457.76 20000000 79692.27 686.65 30000000 79866.15 915.53 40000000 79828.15 1144.41 50000000 80214.91 1373.29 60000000 80226.84 1602.17 70000000 80178.77 1831.05 80000000 80177.06 2059.94 90000000 80109.79 2288.82 100000000 80175.94 # flat memory, running from MCDRAM, alltoall, hbwmalloc API # MB N GB/s 228.88 10000000 421009.18 457.76 20000000 416997.91 686.65 30000000 427808.31 915.53 40000000 423310.75 1144.41 50000000 431032.35 1373.29 60000000 427172.91 1602.17 70000000 426823.59 1831.05 80000000 428103.96 2059.94 90000000 428152.02 2288.82 100000000 430185.03 2343.75 102400000 425999.98 4687.50 204800000 427821.10 9155.27 400000000 429318.13 9375.00 409600000 428376.70 11444.09 500000000 428052.08 13732.91 600000000 266061.01 16021.73 700000000 200257.79 18310.55 800000000 168164.19 20599.37 900000000 150192.91 22888.18 1000000000 137862.07 25177.00 1100000000 129418.12 27465.82 1200000000 123019.53 29754.64 1300000000 118268.79 32043.46 1400000000 114343.33 34332.28 1500000000 110991.25 36621.09 1600000000 108538.36 38909.91 1700000000 106249.72 41198.73 1800000000 104455.07 43487.55 1900000000 102828.93 45776.37 2000000000 101348.56 48065.19 2100000000 100013.73 We appreciate any help on that. Best, Vitali On 04/11/2016 08:42 PM, Kumaran, Kalyan wrote: Avinash, thanks for the prompt reply. Kevin/Vitali can provide all the details. The short summary is we are unable to reproduce the STREAM results from IPM or DDR. It would be great if Karthik or Sundaram can advise us on how to run (which mode etc) or if they could log into the machine and make sure we have set up things correctly. Thanks, Kumar On Apr 11, 2016, at 8:07 PM, Sodani, Avinash <avinash.sodani(a)intel.com<mailto:[email protected]>> wrote: Copying Sundaram/Karthik from our side. Kumar, Can you send us more information on this? -Avinash From: Kumaran, Kalyan [mailto:[email protected]] Sent: Monday, April 11, 2016 5:32 PM To: Fromkin, Russ; Sodani, Avinash Cc: Harms, Kevin; Morozov, Vitali A. Subject: Fwd: [JLSE-Intel-NDA] KNL Status Russ, Avinash, we need help in understanding how to obtain STREAM results. Can anyone help? Thanks, Kumar Begin forwarded message: From: "Harms, Kevin" <harms(a)alcf.anl.gov<mailto:[email protected]>> Subject: [JLSE-Intel-NDA] KNL Status Date: April 11, 2016 at 5:42:15 PM CDT To: "intel-nda(a)jlse.anl.gov<mailto:[email protected]>" <intel-nda(a)jlse.anl.gov<mailto:[email protected]>> The knl00 node is currently in Flat mode and All2All model. With this setup we can get 420 GB/s from IPM memory on STREAM Triad when using hbw_malloc and friends which was confirmed by both Vitali and I. Changing the BIOS settings and warm rebooting seem to be problematic. Vitali and I were running earlier and could not get over 80 GB/s with any method even though memory capacity indicated flat mode. Using numactl --membind is also problematic. The STREAM results when using this to launch the application were erratic, so it is also suggested not to use that. That means you will need to modify your code to use dynamic memory and hbw_malloc to exploit the IPM. Will followup with more detailed results later. kevin _______________________________________________ Intel-nda mailing list Intel-nda(a)lists.jlse.anl.gov<mailto:[email protected]> https://lists.jlse.anl.gov/mailman/listinfo/intel-nda

1 0

Re: [JLSE-Intel-NDA] KNL Status
by Vitali Morozov 11 Apr '16

11 Apr '16

A little more results for STREAM. Flat more, alltoall. The results from memory were obtained by using default STREAM with static allocation. The results from MCDRAM were obtained by using modified STREAM with hbwmalloc.h interface. 1 MPI process, 64 threads, scatter affinity, no numactl is used. Most results are as expected: 80 GB/s from DDR memory, 420 GB/s from MCDRAM, the MCDRAM size is about 16 GB, the aggregate L2 cache size is 32 MB. Noticed inconsistencies for hbwmalloc benchmarks: 1. The result is not validated - will spend more time on that tomorrow. 2. hbw_malloc continues allocating data beyond the expected 16 GB up to 48 GB. This means it fails back to DDR if it cannot allocate in MCDRAM - not expected default behavior. It must fail if it does not have enough space. Vitali Raw data: # flat memory, running from memory (default), alltoall # MB N GB/s 2.29 100000 272062.96 4.58 200000 491040.47 6.87 300000 603979.78 9.16 400000 745654.04 11.44 500000 811800.77 13.73 600000 901462.35 16.02 700000 891953.26 18.31 800000 1019375.15 20.60 900000 984749.63 22.89 1000000 923516.48 25.18 1100000 758422.09 27.47 1200000 779328.74 29.75 1300000 513185.43 32.04 1400000 155550.35 34.33 1500000 103209.12 36.62 1600000 97435.74 38.91 1700000 83968.40 41.20 1800000 83692.35 43.49 1900000 80734.60 45.78 2000000 76317.89 68.66 3000000 81180.08 91.55 4000000 78889.73 114.44 5000000 79891.50 137.33 6000000 79954.96 160.22 7000000 79477.00 183.11 8000000 79145.59 205.99 9000000 79821.12 228.88 10000000 79519.15 228.88 10000000 79212.54 457.76 20000000 79692.27 686.65 30000000 79866.15 915.53 40000000 79828.15 1144.41 50000000 80214.91 1373.29 60000000 80226.84 1602.17 70000000 80178.77 1831.05 80000000 80177.06 2059.94 90000000 80109.79 2288.82 100000000 80175.94 # flat memory, running from MCDRAM, alltoall, hbwmalloc API # MB N GB/s 228.88 10000000 421009.18 457.76 20000000 416997.91 686.65 30000000 427808.31 915.53 40000000 423310.75 1144.41 50000000 431032.35 1373.29 60000000 427172.91 1602.17 70000000 426823.59 1831.05 80000000 428103.96 2059.94 90000000 428152.02 2288.82 100000000 430185.03 2343.75 102400000 425999.98 4687.50 204800000 427821.10 9155.27 400000000 429318.13 9375.00 409600000 428376.70 11444.09 500000000 428052.08 13732.91 600000000 266061.01 16021.73 700000000 200257.79 18310.55 800000000 168164.19 20599.37 900000000 150192.91 22888.18 1000000000 137862.07 25177.00 1100000000 129418.12 27465.82 1200000000 123019.53 29754.64 1300000000 118268.79 32043.46 1400000000 114343.33 34332.28 1500000000 110991.25 36621.09 1600000000 108538.36 38909.91 1700000000 106249.72 41198.73 1800000000 104455.07 43487.55 1900000000 102828.93 45776.37 2000000000 101348.56 48065.19 2100000000 100013.73 On 04/11/2016 05:42 PM, Harms, Kevin wrote: > The knl00 node is currently in Flat mode and All2All model. With this setup we can get 420 GB/s from IPM memory on STREAM Triad when using hbw_malloc and friends which was confirmed by both Vitali and I. > > Changing the BIOS settings and warm rebooting seem to be problematic. Vitali and I were running earlier and could not get over 80 GB/s with any method even though memory capacity indicated flat mode. > > Using numactl --membind is also problematic. The STREAM results when using this to launch the application were erratic, so it is also suggested not to use that. That means you will need to modify your code to use dynamic memory and hbw_malloc to exploit the IPM. > > Will followup with more detailed results later. > > kevin > > > _______________________________________________ > Intel-nda mailing list > Intel-nda(a)lists.jlse.anl.gov > https://lists.jlse.anl.gov/mailman/listinfo/intel-nda

1 0

[JLSE-Intel-NDA] KNL Status
by Harms, Kevin 11 Apr '16

11 Apr '16

The knl00 node is currently in Flat mode and All2All model. With this setup we can get 420 GB/s from IPM memory on STREAM Triad when using hbw_malloc and friends which was confirmed by both Vitali and I. Changing the BIOS settings and warm rebooting seem to be problematic. Vitali and I were running earlier and could not get over 80 GB/s with any method even though memory capacity indicated flat mode. Using numactl --membind is also problematic. The STREAM results when using this to launch the application were erratic, so it is also suggested not to use that. That means you will need to modify your code to use dynamic memory and hbw_malloc to exploit the IPM. Will followup with more detailed results later. kevin

1 0

Re: [JLSE-Intel-NDA] Intel MPSP
by Vitali A. Morozov 11 Apr '16

11 Apr '16

Sounds great - thanks! On 04/11/2016 04:38 PM, Allen, Benjamin S. wrote: > Rebooting the box now. It appears on warm reboot, it often hangs or takes an extremely long time for the kernel to boot. With a ipmitool power cycle, it boots normally. > > After the machine is back up you're free to test. > > Ben > >> On Apr 11, 2016, at 4:26 PM, Vitali A. Morozov <morozov(a)anl.gov> wrote: >> >> Hi Ben, >> >> just talked to Kevin about STREAM - he suggests me to try it as well. The knl.txt says you own it until 5 pm. >> >> Should I wait? >> >> Thanks, >> >> Vitali >> >> >> On 04/11/2016 03:57 PM, Allen, Benjamin S. wrote: >>> MPSP is now install on knl00. >>> >>> Ben >>> >>> >>>> On Apr 11, 2016, at 12:49 PM, Allen, Benjamin S. <bsallen(a)alcf.anl.gov> >>>> wrote: >>>> >>>> It includes the kernel SRPM if you want to go spelunking for patches. >>>> >>>> Ben >>>> >>>> >>>>> On Apr 11, 2016, at 12:45 PM, Harms, Kevin <harms(a)alcf.anl.gov> >>>>> wrote: >>>>> >>>>> >>>>> Getting the updated jemalloc and memkind libraries would be worth it if we are going to test the flat mode. It might be interesting to get and run the STREAM test in the micperf package to demonstrate the memory bandwidth seems to be incorrect. >>>>> >>>>> The document doesn't really go into what the kernel changes were other than vague statements of fixes or support of additional features. >>>>> >>>>> kevin >>>>> >>>>> >>>>> >>>>> >>>>> >>>>>> Intel has a patched RHEL7.2 kernel and a number of other tools that appear to better support KNL. Attached the user guide. Let me know if I should install this on knl00. >>>>>> >>>>>> Ben >>>>>> >>>> _______________________________________________ >>>> Intel-nda mailing list >>>> >>>> Intel-nda(a)lists.jlse.anl.gov >>>> https://lists.jlse.anl.gov/mailman/listinfo/intel-nda >>> >>> _______________________________________________ >>> Intel-nda mailing list >>> >>> Intel-nda(a)lists.jlse.anl.gov >>> https://lists.jlse.anl.gov/mailman/listinfo/intel-nda

1 0

Re: [JLSE-Intel-NDA] Intel MPSP
by Allen, Benjamin S. 11 Apr '16

11 Apr '16

Rebooting the box now. It appears on warm reboot, it often hangs or takes an extremely long time for the kernel to boot. With a ipmitool power cycle, it boots normally. After the machine is back up you're free to test. Ben > On Apr 11, 2016, at 4:26 PM, Vitali A. Morozov <morozov(a)anl.gov> wrote: > > Hi Ben, > > just talked to Kevin about STREAM - he suggests me to try it as well. The knl.txt says you own it until 5 pm. > > Should I wait? > > Thanks, > > Vitali > > > On 04/11/2016 03:57 PM, Allen, Benjamin S. wrote: >> MPSP is now install on knl00. >> >> Ben >> >> >>> On Apr 11, 2016, at 12:49 PM, Allen, Benjamin S. <bsallen(a)alcf.anl.gov> >>> wrote: >>> >>> It includes the kernel SRPM if you want to go spelunking for patches. >>> >>> Ben >>> >>> >>>> On Apr 11, 2016, at 12:45 PM, Harms, Kevin <harms(a)alcf.anl.gov> >>>> wrote: >>>> >>>> >>>> Getting the updated jemalloc and memkind libraries would be worth it if we are going to test the flat mode. It might be interesting to get and run the STREAM test in the micperf package to demonstrate the memory bandwidth seems to be incorrect. >>>> >>>> The document doesn't really go into what the kernel changes were other than vague statements of fixes or support of additional features. >>>> >>>> kevin >>>> >>>> >>>> >>>> >>>> >>>>> Intel has a patched RHEL7.2 kernel and a number of other tools that appear to better support KNL. Attached the user guide. Let me know if I should install this on knl00. >>>>> >>>>> Ben >>>>> >>> _______________________________________________ >>> Intel-nda mailing list >>> >>> Intel-nda(a)lists.jlse.anl.gov >>> https://lists.jlse.anl.gov/mailman/listinfo/intel-nda >> >> >> _______________________________________________ >> Intel-nda mailing list >> >> Intel-nda(a)lists.jlse.anl.gov >> https://lists.jlse.anl.gov/mailman/listinfo/intel-nda >

1 0

Re: [JLSE-Intel-NDA] Intel MPSP
by Vitali A. Morozov 11 Apr '16

11 Apr '16

Hi Ben, just talked to Kevin about STREAM - he suggests me to try it as well. The knl.txt says you own it until 5 pm. Should I wait? Thanks, Vitali On 04/11/2016 03:57 PM, Allen, Benjamin S. wrote: > MPSP is now install on knl00. > > Ben > >> On Apr 11, 2016, at 12:49 PM, Allen, Benjamin S. <bsallen(a)alcf.anl.gov> wrote: >> >> It includes the kernel SRPM if you want to go spelunking for patches. >> >> Ben >> >>> On Apr 11, 2016, at 12:45 PM, Harms, Kevin <harms(a)alcf.anl.gov> wrote: >>> >>> >>> Getting the updated jemalloc and memkind libraries would be worth it if we are going to test the flat mode. It might be interesting to get and run the STREAM test in the micperf package to demonstrate the memory bandwidth seems to be incorrect. >>> >>> The document doesn't really go into what the kernel changes were other than vague statements of fixes or support of additional features. >>> >>> kevin >>> >>> >>> >>> >>>> Intel has a patched RHEL7.2 kernel and a number of other tools that appear to better support KNL. Attached the user guide. Let me know if I should install this on knl00. >>>> >>>> Ben >> _______________________________________________ >> Intel-nda mailing list >> Intel-nda(a)lists.jlse.anl.gov >> https://lists.jlse.anl.gov/mailman/listinfo/intel-nda > > > _______________________________________________ > Intel-nda mailing list > Intel-nda(a)lists.jlse.anl.gov > https://lists.jlse.anl.gov/mailman/listinfo/intel-nda

1 0

Re: [JLSE-Intel-NDA] Intel MPSP
by Allen, Benjamin S. 11 Apr '16

11 Apr '16

MPSP is now install on knl00. Ben > On Apr 11, 2016, at 12:49 PM, Allen, Benjamin S. <bsallen(a)alcf.anl.gov> wrote: > > It includes the kernel SRPM if you want to go spelunking for patches. > > Ben > >> On Apr 11, 2016, at 12:45 PM, Harms, Kevin <harms(a)alcf.anl.gov> wrote: >> >> >> Getting the updated jemalloc and memkind libraries would be worth it if we are going to test the flat mode. It might be interesting to get and run the STREAM test in the micperf package to demonstrate the memory bandwidth seems to be incorrect. >> >> The document doesn't really go into what the kernel changes were other than vague statements of fixes or support of additional features. >> >> kevin >> >> >> >> >>> Intel has a patched RHEL7.2 kernel and a number of other tools that appear to better support KNL. Attached the user guide. Let me know if I should install this on knl00. >>> >>> Ben > > _______________________________________________ > Intel-nda mailing list > Intel-nda(a)lists.jlse.anl.gov > https://lists.jlse.anl.gov/mailman/listinfo/intel-nda

1 0