June 2016 - Intel-nda - mailman3.cels.anl.gov

Re: [JLSE-Intel-NDA] Intel DAP boxes set up
by Susan Coghlan 29 Jun '16

29 Jun '16

Unfortunately, a reboot is 15 mins. That's a lot of time so I think we don't want to reboot for each job. We've asked Cray and Intel to analyze and document where the time is spent during the boot to see if there is a way to reduce it, but for now we are stuck with 15 min boots. I do like the idea of allowing the user to request a reboot even if they are running in the default mode. That seems like it should be doable. -Susan Kamil Iskra wrote: > On Tue, Jun 28, 2016 at 19:42:29 -0500, Hal Finkel wrote: > >> Description: Application Performance can degrade over time when >> MCDRAM is used in cache mode >> Impact: The direct-mapped nature of MCDRAM-as-cache is by >> design. Real applications that need more than 16GB >> of memory will not exhibit this time-based >> degradation; it is really only a problem for >> benchmarking when running smaller datasets. >> Mitigation: Workaround is to reboot the node >> >> That seems scary. I believe we were considering booting the nodes in cache mode by default on Theta. Maybe that's a bad idea. > > If I understand it correctly then, this is due to physical memory > fragmentation of DDR, typical for page-based memory management as used in > Linux. This effect can be observed with any non-fully-associative cache > (including regular CPU caches), although it will be the worst for > one-way-associative (direct-mapped) one. > > For the most predictable, and generally best, performance, it may make > sense to consider rebooting nodes between each job on Theta, or at least > make node reboot a user-accessible option at job submission time... > > Kamil > _______________________________________________ > Intel-nda mailing list > Intel-nda(a)lists.jlse.anl.gov > https://lists.jlse.anl.gov/mailman/listinfo/intel-nda

1 0

Re: [JLSE-Intel-NDA] Intel DAP boxes set up
by Scott Parker 29 Jun '16

29 Jun '16

This seem like something we should understand better. The descriptions is a little ambiguous, I wonder if this mean a single application running in cache mode for multiple hours will see performance degradation? Or it's the standard Linux memory fragmentation that occurs on systems over weeks or months of having numerous processes allocate and deallocate memory? Does it only occur when applications are in cache mode and use less that 16 GB? -Scott On 6/29/16 10:27 AM, Kamil Iskra wrote: > On Tue, Jun 28, 2016 at 19:42:29 -0500, Hal Finkel wrote: > >> Description: Application Performance can degrade over time when >> MCDRAM is used in cache mode >> Impact: The direct-mapped nature of MCDRAM-as-cache is by >> design. Real applications that need more than 16GB >> of memory will not exhibit this time-based >> degradation; it is really only a problem for >> benchmarking when running smaller datasets. >> Mitigation: Workaround is to reboot the node >> >> That seems scary. I believe we were considering booting the nodes in cache mode by default on Theta. Maybe that's a bad idea. > If I understand it correctly then, this is due to physical memory > fragmentation of DDR, typical for page-based memory management as used in > Linux. This effect can be observed with any non-fully-associative cache > (including regular CPU caches), although it will be the worst for > one-way-associative (direct-mapped) one. > > For the most predictable, and generally best, performance, it may make > sense to consider rebooting nodes between each job on Theta, or at least > make node reboot a user-accessible option at job submission time... > > Kamil > _______________________________________________ > Intel-nda mailing list > Intel-nda(a)lists.jlse.anl.gov > https://lists.jlse.anl.gov/mailman/listinfo/intel-nda

1 0

Re: [JLSE-Intel-NDA] Intel DAP boxes set up
by Swann Perarnau 29 Jun '16

29 Jun '16

On Wed, Jun 29, 2016 at 10:27:00AM -0500, Kamil Iskra wrote: > On Tue, Jun 28, 2016 at 19:42:29 -0500, Hal Finkel wrote: > > > Description: Application Performance can degrade over time when > > MCDRAM is used in cache mode > > Impact: The direct-mapped nature of MCDRAM-as-cache is by > > design. Real applications that need more than 16GB > > of memory will not exhibit this time-based > > degradation; it is really only a problem for > > benchmarking when running smaller datasets. > > Mitigation: Workaround is to reboot the node > > > > That seems scary. I believe we were considering booting the nodes in cache mode by default on Theta. Maybe that's a bad idea. > > If I understand it correctly then, this is due to physical memory > fragmentation of DDR, typical for page-based memory management as used in > Linux. This effect can be observed with any non-fully-associative cache > (including regular CPU caches), although it will be the worst for > one-way-associative (direct-mapped) one. > > For the most predictable, and generally best, performance, it may make > sense to consider rebooting nodes between each job on Theta, or at least > make node reboot a user-accessible option at job submission time... > If direct-mapped MCDRAM-as-cache is a long term feature of Xeon Phis, you could try to tell Intel to implement page coloring in Linux. Torvalds only killed 4 attempts at it in the last ten years :p -- Swann

1 0

Re: [JLSE-Intel-NDA] Intel DAP boxes set up
by Kamil Iskra 29 Jun '16

29 Jun '16

On Tue, Jun 28, 2016 at 19:42:29 -0500, Hal Finkel wrote: > Description: Application Performance can degrade over time when > MCDRAM is used in cache mode > Impact: The direct-mapped nature of MCDRAM-as-cache is by > design. Real applications that need more than 16GB > of memory will not exhibit this time-based > degradation; it is really only a problem for > benchmarking when running smaller datasets. > Mitigation: Workaround is to reboot the node > > That seems scary. I believe we were considering booting the nodes in cache mode by default on Theta. Maybe that's a bad idea. If I understand it correctly then, this is due to physical memory fragmentation of DDR, typical for page-based memory management as used in Linux. This effect can be observed with any non-fully-associative cache (including regular CPU caches), although it will be the worst for one-way-associative (direct-mapped) one. For the most predictable, and generally best, performance, it may make sense to consider rebooting nodes between each job on Theta, or at least make node reboot a user-accessible option at job submission time... Kamil

1 0

Re: [JLSE-Intel-NDA] Intel DAP boxes set up
by Hal Finkel 28 Jun '16

28 Jun '16

----- Original Message ----- > From: "Hal Finkel" <hfinkel(a)anl.gov> > To: "Vitali Morozov" <morozov(a)anl.gov> > Cc: "Vitali A. Morozov" <morozov(a)alcf.anl.gov>, intel-nda(a)lists.jlse.anl.gov > Sent: Tuesday, June 28, 2016 8:41:03 PM > Subject: Re: [JLSE-Intel-NDA] Intel DAP boxes set up > > ----- Original Message ----- > > From: "Vitali Morozov" <morozov(a)anl.gov> > > To: "Hal Finkel" <hfinkel(a)anl.gov> > > Cc: "Kevin Harms" <harms(a)alcf.anl.gov>, "Vitali A. Morozov" > > <morozov(a)alcf.anl.gov>, intel-nda(a)lists.jlse.anl.gov > > Sent: Tuesday, June 28, 2016 8:29:17 PM > > Subject: Re: [JLSE-Intel-NDA] Intel DAP boxes set up > > > > > > Interesting, what are you doing such that others do not? Why no one > > else > > reports it? > > I'm just building LLVM trunk with the system gcc. make -j256. Also, the build directory is in /dev/shm (source directory is in my home directory): $ df -h /dev/shm Filesystem Size Used Avail Use% Mounted on tmpfs 55G 3.1G 52G 6% /dev/shm That's how I build on the regular Xeons and on the POWER machines. It looks like there's still plenty of room. -Hal > > -Hal > > > > > Vitali > > > > > > On 06/28/2016 07:18 PM, Hal Finkel wrote: > > > Hi Kevin, > > > > > > FWIW, no, the problem happened again with the new kernel: > > > > > > Message from syslogd@kingly at Jun 28 19:12:49 ... > > > kernel:BUG: soft lockup - CPU#104 stuck for 23s! > > > [cc1plus:77778] > > > > > > Message from syslogd@kingly at Jun 28 19:12:51 ... > > > kernel:BUG: soft lockup - CPU#177 stuck for 23s! > > > [cc1plus:77662] > > > > > > ... > > > > > > What's special about this kernel? Could I build a mainline kernel > > > and expect it to work reasonably? > > > > > > -Hal > > > > > > ----- Original Message ----- > > >> From: "Hal Finkel" <hfinkel(a)anl.gov> > > >> To: "Kevin Harms" <harms(a)alcf.anl.gov> > > >> Cc: "Vitali A. Morozov" <morozov(a)alcf.anl.gov>, > > >> intel-nda(a)lists.jlse.anl.gov > > >> Sent: Tuesday, June 28, 2016 12:47:02 PM > > >> Subject: Re: [JLSE-Intel-NDA] Intel DAP boxes set up > > >> > > >> Thanks! I'll upgrade. > > >> > > >> -Hal > > >> > > >> ----- Original Message ----- > > >>> From: "Kevin Harms" <harms(a)alcf.anl.gov> > > >>> To: "Hal J. Finkel" <hfinkel(a)anl.gov>, "Vitali A. Morozov" > > >>> <morozov(a)alcf.anl.gov> > > >>> Cc: intel-nda(a)lists.jlse.anl.gov > > >>> Sent: Tuesday, June 28, 2016 11:59:20 AM > > >>> Subject: Re: [JLSE-Intel-NDA] Intel DAP boxes set up > > >>> > > >>> > > >>> I updated this page to indicate how to upgrade to the mpsp > > >>> 1.3.2 > > >>> kernel. > > >>> https://wiki.jlse.anl.gov/display/knl/Intel+DAP+Setup > > >>> > > >>> No idea, if that will fix this problem though :) > > >>> > > >>> kevin > > >>> > > >>>> FYI: James and I have been experimenting with our box. We had > > >>>> to > > >>>> reset it after we hit some kernel bug: > > >>>> > > >>>> Message from syslogd@kingly at Jun 27 14:29:44 ... > > >>>> kernel:BUG: soft lockup - CPU#77 stuck for 22s! > > >>>> [cc1plus:15316] > > >>>> > > >>>> Message from syslogd@kingly at Jun 27 14:30:43 ... > > >>>> kernel:BUG: soft lockup - CPU#149 stuck for 23s! > > >>>> [cc1plus:14840] > > >>>> > > >>>> Message from syslogd@kingly at Jun 27 14:30:44 ... > > >>>> kernel:BUG: soft lockup - CPU#8 stuck for 23s! [cc1plus:15337] > > >>>> > > >>>> Message from syslogd@kingly at Jun 27 14:30:46 ... > > >>>> kernel:BUG: soft lockup - CPU#49 stuck for 22s! [as:16806] > > >>>> > > >>>> Message from syslogd@kingly at Jun 27 14:30:47 ... > > >>>> kernel:BUG: soft lockup - CPU#27 stuck for 22s! > > >>>> [cc1plus:16548] > > >>>> > > >>>> Message from syslogd@kingly at Jun 27 14:30:50 ... > > >>>> kernel:BUG: soft lockup - CPU#34 stuck for 22s! [cmake:16807] > > >>>> > > >>>> Message from syslogd@kingly at Jun 27 14:30:52 ... > > >>>> kernel:BUG: soft lockup - CPU#154 stuck for 22s! [cmake:16810] > > >>>> > > >>>> Message from syslogd@kingly at Jun 27 14:30:54 ... > > >>>> kernel:BUG: soft lockup - CPU#35 stuck for 23s! > > >>>> [cc1plus:15807] > > >>>> > > >>>> Message from syslogd@kingly at Jun 27 14:30:55 ... > > >>>> kernel:BUG: soft lockup - CPU#77 stuck for 22s! > > >>>> [cc1plus:15316] > > >>>> > > >>>> Message from syslogd@kingly at Jun 27 14:30:55 ... > > >>>> kernel:BUG: soft lockup - CPU#91 stuck for 22s! [cmake:16813] > > >>>> > > >>>> Message from syslogd@kingly at Jun 27 14:30:58 ... > > >>>> kernel:BUG: soft lockup - CPU#124 stuck for 23s! [make:16811] > > >>>> > > >>>> Message from syslogd@kingly at Jun 27 14:31:00 ... > > >>>> kernel:BUG: soft lockup - CPU#149 stuck for 23s! > > >>>> [cc1plus:14840] > > >>>> > > >>>> ... (this went on for a while) > > >>>> > > >>>> The kernel (3.10.0-327.el7.centos.mpsp_1.3.1.45.x86_64) seems > > >>>> like > > >>>> some special build? > > >>>> > > >>>> -Hal > > >>>> > > >>>> ----- Original Message ----- > > >>>>> From: "Vitali A. Morozov" <morozov(a)anl.gov> > > >>>>> To: "Kalyan Kumaran" <kumaran(a)alcf.anl.gov>, > > >>>>> intel-nda(a)lists.jlse.anl.gov > > >>>>> Sent: Thursday, June 23, 2016 10:47:37 AM > > >>>>> Subject: Re: [JLSE-Intel-NDA] Intel DAP boxes set up > > >>>>> > > >>>>> > > >>>>> > > >>>>> I have updated the Ethernet port issue that I have > > >>>>> encountered. > > >>>>> > > >>>>> Vitali > > >>>>> > > >>>>> On 06/23/2016 10:42 AM, Kumaran, Kalyan wrote: > > >>>>> > > >>>>> > > >>>>> For folks setting these up please see instructions at: > > >>>>> > > >>>>> > > >>>>> https://wiki.jlse.anl.gov/display/knl/Intel+DAP+Setup > > >>>>> > > >>>>> > > >>>>> > > >>>>> Kevin, Scott and Vitali will be adding information as they > > >>>>> set > > >>>>> the > > >>>>> machines up. Others are welcome to add stuff. > > >>>>> > > >>>>> > > >>>>> -Kumar > > >>>>> > > >>>>> _______________________________________________ > > >>>>> Intel-nda mailing list Intel-nda(a)lists.jlse.anl.gov > > >>>>> https://lists.jlse.anl.gov/mailman/listinfo/intel-nda > > >>>>> > > >>>>> _______________________________________________ > > >>>>> Intel-nda mailing list > > >>>>> Intel-nda(a)lists.jlse.anl.gov > > >>>>> https://lists.jlse.anl.gov/mailman/listinfo/intel-nda > > >>>>> > > >>>> -- > > >>>> Hal Finkel > > >>>> Assistant Computational Scientist > > >>>> Leadership Computing Facility > > >>>> Argonne National Laboratory > > >>>> _______________________________________________ > > >>>> Intel-nda mailing list > > >>>> Intel-nda(a)lists.jlse.anl.gov > > >>>> https://lists.jlse.anl.gov/mailman/listinfo/intel-nda > > >> -- > > >> Hal Finkel > > >> Assistant Computational Scientist > > >> Leadership Computing Facility > > >> Argonne National Laboratory > > >> _______________________________________________ > > >> Intel-nda mailing list > > >> Intel-nda(a)lists.jlse.anl.gov > > >> https://lists.jlse.anl.gov/mailman/listinfo/intel-nda > > >> > > > > > > -- > Hal Finkel > Assistant Computational Scientist > Leadership Computing Facility > Argonne National Laboratory > _______________________________________________ > Intel-nda mailing list > Intel-nda(a)lists.jlse.anl.gov > https://lists.jlse.anl.gov/mailman/listinfo/intel-nda > -- Hal Finkel Assistant Computational Scientist Leadership Computing Facility Argonne National Laboratory

1 0

Re: [JLSE-Intel-NDA] Intel DAP boxes set up
by Hal Finkel 28 Jun '16

28 Jun '16

----- Original Message ----- > From: "Vitali Morozov" <morozov(a)anl.gov> > To: "Hal Finkel" <hfinkel(a)anl.gov> > Cc: "Kevin Harms" <harms(a)alcf.anl.gov>, "Vitali A. Morozov" <morozov(a)alcf.anl.gov>, intel-nda(a)lists.jlse.anl.gov > Sent: Tuesday, June 28, 2016 8:29:17 PM > Subject: Re: [JLSE-Intel-NDA] Intel DAP boxes set up > > > Interesting, what are you doing such that others do not? Why no one > else > reports it? I'm just building LLVM trunk with the system gcc. make -j256. -Hal > > Vitali > > > On 06/28/2016 07:18 PM, Hal Finkel wrote: > > Hi Kevin, > > > > FWIW, no, the problem happened again with the new kernel: > > > > Message from syslogd@kingly at Jun 28 19:12:49 ... > > kernel:BUG: soft lockup - CPU#104 stuck for 23s! [cc1plus:77778] > > > > Message from syslogd@kingly at Jun 28 19:12:51 ... > > kernel:BUG: soft lockup - CPU#177 stuck for 23s! [cc1plus:77662] > > > > ... > > > > What's special about this kernel? Could I build a mainline kernel > > and expect it to work reasonably? > > > > -Hal > > > > ----- Original Message ----- > >> From: "Hal Finkel" <hfinkel(a)anl.gov> > >> To: "Kevin Harms" <harms(a)alcf.anl.gov> > >> Cc: "Vitali A. Morozov" <morozov(a)alcf.anl.gov>, > >> intel-nda(a)lists.jlse.anl.gov > >> Sent: Tuesday, June 28, 2016 12:47:02 PM > >> Subject: Re: [JLSE-Intel-NDA] Intel DAP boxes set up > >> > >> Thanks! I'll upgrade. > >> > >> -Hal > >> > >> ----- Original Message ----- > >>> From: "Kevin Harms" <harms(a)alcf.anl.gov> > >>> To: "Hal J. Finkel" <hfinkel(a)anl.gov>, "Vitali A. Morozov" > >>> <morozov(a)alcf.anl.gov> > >>> Cc: intel-nda(a)lists.jlse.anl.gov > >>> Sent: Tuesday, June 28, 2016 11:59:20 AM > >>> Subject: Re: [JLSE-Intel-NDA] Intel DAP boxes set up > >>> > >>> > >>> I updated this page to indicate how to upgrade to the mpsp > >>> 1.3.2 > >>> kernel. https://wiki.jlse.anl.gov/display/knl/Intel+DAP+Setup > >>> > >>> No idea, if that will fix this problem though :) > >>> > >>> kevin > >>> > >>>> FYI: James and I have been experimenting with our box. We had to > >>>> reset it after we hit some kernel bug: > >>>> > >>>> Message from syslogd@kingly at Jun 27 14:29:44 ... > >>>> kernel:BUG: soft lockup - CPU#77 stuck for 22s! [cc1plus:15316] > >>>> > >>>> Message from syslogd@kingly at Jun 27 14:30:43 ... > >>>> kernel:BUG: soft lockup - CPU#149 stuck for 23s! [cc1plus:14840] > >>>> > >>>> Message from syslogd@kingly at Jun 27 14:30:44 ... > >>>> kernel:BUG: soft lockup - CPU#8 stuck for 23s! [cc1plus:15337] > >>>> > >>>> Message from syslogd@kingly at Jun 27 14:30:46 ... > >>>> kernel:BUG: soft lockup - CPU#49 stuck for 22s! [as:16806] > >>>> > >>>> Message from syslogd@kingly at Jun 27 14:30:47 ... > >>>> kernel:BUG: soft lockup - CPU#27 stuck for 22s! [cc1plus:16548] > >>>> > >>>> Message from syslogd@kingly at Jun 27 14:30:50 ... > >>>> kernel:BUG: soft lockup - CPU#34 stuck for 22s! [cmake:16807] > >>>> > >>>> Message from syslogd@kingly at Jun 27 14:30:52 ... > >>>> kernel:BUG: soft lockup - CPU#154 stuck for 22s! [cmake:16810] > >>>> > >>>> Message from syslogd@kingly at Jun 27 14:30:54 ... > >>>> kernel:BUG: soft lockup - CPU#35 stuck for 23s! [cc1plus:15807] > >>>> > >>>> Message from syslogd@kingly at Jun 27 14:30:55 ... > >>>> kernel:BUG: soft lockup - CPU#77 stuck for 22s! [cc1plus:15316] > >>>> > >>>> Message from syslogd@kingly at Jun 27 14:30:55 ... > >>>> kernel:BUG: soft lockup - CPU#91 stuck for 22s! [cmake:16813] > >>>> > >>>> Message from syslogd@kingly at Jun 27 14:30:58 ... > >>>> kernel:BUG: soft lockup - CPU#124 stuck for 23s! [make:16811] > >>>> > >>>> Message from syslogd@kingly at Jun 27 14:31:00 ... > >>>> kernel:BUG: soft lockup - CPU#149 stuck for 23s! [cc1plus:14840] > >>>> > >>>> ... (this went on for a while) > >>>> > >>>> The kernel (3.10.0-327.el7.centos.mpsp_1.3.1.45.x86_64) seems > >>>> like > >>>> some special build? > >>>> > >>>> -Hal > >>>> > >>>> ----- Original Message ----- > >>>>> From: "Vitali A. Morozov" <morozov(a)anl.gov> > >>>>> To: "Kalyan Kumaran" <kumaran(a)alcf.anl.gov>, > >>>>> intel-nda(a)lists.jlse.anl.gov > >>>>> Sent: Thursday, June 23, 2016 10:47:37 AM > >>>>> Subject: Re: [JLSE-Intel-NDA] Intel DAP boxes set up > >>>>> > >>>>> > >>>>> > >>>>> I have updated the Ethernet port issue that I have encountered. > >>>>> > >>>>> Vitali > >>>>> > >>>>> On 06/23/2016 10:42 AM, Kumaran, Kalyan wrote: > >>>>> > >>>>> > >>>>> For folks setting these up please see instructions at: > >>>>> > >>>>> > >>>>> https://wiki.jlse.anl.gov/display/knl/Intel+DAP+Setup > >>>>> > >>>>> > >>>>> > >>>>> Kevin, Scott and Vitali will be adding information as they set > >>>>> the > >>>>> machines up. Others are welcome to add stuff. > >>>>> > >>>>> > >>>>> -Kumar > >>>>> > >>>>> _______________________________________________ > >>>>> Intel-nda mailing list Intel-nda(a)lists.jlse.anl.gov > >>>>> https://lists.jlse.anl.gov/mailman/listinfo/intel-nda > >>>>> > >>>>> _______________________________________________ > >>>>> Intel-nda mailing list > >>>>> Intel-nda(a)lists.jlse.anl.gov > >>>>> https://lists.jlse.anl.gov/mailman/listinfo/intel-nda > >>>>> > >>>> -- > >>>> Hal Finkel > >>>> Assistant Computational Scientist > >>>> Leadership Computing Facility > >>>> Argonne National Laboratory > >>>> _______________________________________________ > >>>> Intel-nda mailing list > >>>> Intel-nda(a)lists.jlse.anl.gov > >>>> https://lists.jlse.anl.gov/mailman/listinfo/intel-nda > >> -- > >> Hal Finkel > >> Assistant Computational Scientist > >> Leadership Computing Facility > >> Argonne National Laboratory > >> _______________________________________________ > >> Intel-nda mailing list > >> Intel-nda(a)lists.jlse.anl.gov > >> https://lists.jlse.anl.gov/mailman/listinfo/intel-nda > >> > > -- Hal Finkel Assistant Computational Scientist Leadership Computing Facility Argonne National Laboratory

1 0

Re: [JLSE-Intel-NDA] Intel DAP boxes set up
by Vitali Morozov 28 Jun '16

28 Jun '16

Interesting, what are you doing such that others do not? Why no one else reports it? Vitali On 06/28/2016 07:18 PM, Hal Finkel wrote: > Hi Kevin, > > FWIW, no, the problem happened again with the new kernel: > > Message from syslogd@kingly at Jun 28 19:12:49 ... > kernel:BUG: soft lockup - CPU#104 stuck for 23s! [cc1plus:77778] > > Message from syslogd@kingly at Jun 28 19:12:51 ... > kernel:BUG: soft lockup - CPU#177 stuck for 23s! [cc1plus:77662] > > ... > > What's special about this kernel? Could I build a mainline kernel and expect it to work reasonably? > > -Hal > > ----- Original Message ----- >> From: "Hal Finkel" <hfinkel(a)anl.gov> >> To: "Kevin Harms" <harms(a)alcf.anl.gov> >> Cc: "Vitali A. Morozov" <morozov(a)alcf.anl.gov>, intel-nda(a)lists.jlse.anl.gov >> Sent: Tuesday, June 28, 2016 12:47:02 PM >> Subject: Re: [JLSE-Intel-NDA] Intel DAP boxes set up >> >> Thanks! I'll upgrade. >> >> -Hal >> >> ----- Original Message ----- >>> From: "Kevin Harms" <harms(a)alcf.anl.gov> >>> To: "Hal J. Finkel" <hfinkel(a)anl.gov>, "Vitali A. Morozov" >>> <morozov(a)alcf.anl.gov> >>> Cc: intel-nda(a)lists.jlse.anl.gov >>> Sent: Tuesday, June 28, 2016 11:59:20 AM >>> Subject: Re: [JLSE-Intel-NDA] Intel DAP boxes set up >>> >>> >>> I updated this page to indicate how to upgrade to the mpsp 1.3.2 >>> kernel. https://wiki.jlse.anl.gov/display/knl/Intel+DAP+Setup >>> >>> No idea, if that will fix this problem though :) >>> >>> kevin >>> >>>> FYI: James and I have been experimenting with our box. We had to >>>> reset it after we hit some kernel bug: >>>> >>>> Message from syslogd@kingly at Jun 27 14:29:44 ... >>>> kernel:BUG: soft lockup - CPU#77 stuck for 22s! [cc1plus:15316] >>>> >>>> Message from syslogd@kingly at Jun 27 14:30:43 ... >>>> kernel:BUG: soft lockup - CPU#149 stuck for 23s! [cc1plus:14840] >>>> >>>> Message from syslogd@kingly at Jun 27 14:30:44 ... >>>> kernel:BUG: soft lockup - CPU#8 stuck for 23s! [cc1plus:15337] >>>> >>>> Message from syslogd@kingly at Jun 27 14:30:46 ... >>>> kernel:BUG: soft lockup - CPU#49 stuck for 22s! [as:16806] >>>> >>>> Message from syslogd@kingly at Jun 27 14:30:47 ... >>>> kernel:BUG: soft lockup - CPU#27 stuck for 22s! [cc1plus:16548] >>>> >>>> Message from syslogd@kingly at Jun 27 14:30:50 ... >>>> kernel:BUG: soft lockup - CPU#34 stuck for 22s! [cmake:16807] >>>> >>>> Message from syslogd@kingly at Jun 27 14:30:52 ... >>>> kernel:BUG: soft lockup - CPU#154 stuck for 22s! [cmake:16810] >>>> >>>> Message from syslogd@kingly at Jun 27 14:30:54 ... >>>> kernel:BUG: soft lockup - CPU#35 stuck for 23s! [cc1plus:15807] >>>> >>>> Message from syslogd@kingly at Jun 27 14:30:55 ... >>>> kernel:BUG: soft lockup - CPU#77 stuck for 22s! [cc1plus:15316] >>>> >>>> Message from syslogd@kingly at Jun 27 14:30:55 ... >>>> kernel:BUG: soft lockup - CPU#91 stuck for 22s! [cmake:16813] >>>> >>>> Message from syslogd@kingly at Jun 27 14:30:58 ... >>>> kernel:BUG: soft lockup - CPU#124 stuck for 23s! [make:16811] >>>> >>>> Message from syslogd@kingly at Jun 27 14:31:00 ... >>>> kernel:BUG: soft lockup - CPU#149 stuck for 23s! [cc1plus:14840] >>>> >>>> ... (this went on for a while) >>>> >>>> The kernel (3.10.0-327.el7.centos.mpsp_1.3.1.45.x86_64) seems like >>>> some special build? >>>> >>>> -Hal >>>> >>>> ----- Original Message ----- >>>>> From: "Vitali A. Morozov" <morozov(a)anl.gov> >>>>> To: "Kalyan Kumaran" <kumaran(a)alcf.anl.gov>, >>>>> intel-nda(a)lists.jlse.anl.gov >>>>> Sent: Thursday, June 23, 2016 10:47:37 AM >>>>> Subject: Re: [JLSE-Intel-NDA] Intel DAP boxes set up >>>>> >>>>> >>>>> >>>>> I have updated the Ethernet port issue that I have encountered. >>>>> >>>>> Vitali >>>>> >>>>> On 06/23/2016 10:42 AM, Kumaran, Kalyan wrote: >>>>> >>>>> >>>>> For folks setting these up please see instructions at: >>>>> >>>>> >>>>> https://wiki.jlse.anl.gov/display/knl/Intel+DAP+Setup >>>>> >>>>> >>>>> >>>>> Kevin, Scott and Vitali will be adding information as they set >>>>> the >>>>> machines up. Others are welcome to add stuff. >>>>> >>>>> >>>>> -Kumar >>>>> >>>>> _______________________________________________ >>>>> Intel-nda mailing list Intel-nda(a)lists.jlse.anl.gov >>>>> https://lists.jlse.anl.gov/mailman/listinfo/intel-nda >>>>> >>>>> _______________________________________________ >>>>> Intel-nda mailing list >>>>> Intel-nda(a)lists.jlse.anl.gov >>>>> https://lists.jlse.anl.gov/mailman/listinfo/intel-nda >>>>> >>>> -- >>>> Hal Finkel >>>> Assistant Computational Scientist >>>> Leadership Computing Facility >>>> Argonne National Laboratory >>>> _______________________________________________ >>>> Intel-nda mailing list >>>> Intel-nda(a)lists.jlse.anl.gov >>>> https://lists.jlse.anl.gov/mailman/listinfo/intel-nda >> -- >> Hal Finkel >> Assistant Computational Scientist >> Leadership Computing Facility >> Argonne National Laboratory >> _______________________________________________ >> Intel-nda mailing list >> Intel-nda(a)lists.jlse.anl.gov >> https://lists.jlse.anl.gov/mailman/listinfo/intel-nda >>

1 0

Re: [JLSE-Intel-NDA] Intel DAP boxes set up
by Hal Finkel 28 Jun '16

28 Jun '16

Hi Kevin, Thanks! Also, from the release notes: Issue number: 0000003 Description: Application Performance can degrade over time when MCDRAM is used in cache mode Impact: The direct-mapped nature of MCDRAM-as-cache is by design. Real applications that need more than 16GB of memory will not exhibit this time-based degradation; it is really only a problem for benchmarking when running smaller datasets. Mitigation: Workaround is to reboot the node That seems scary. I believe we were considering booting the nodes in cache mode by default on Theta. Maybe that's a bad idea. Intel Tracking ID: 0001418 Affected OS: CentOS 7.2, RHEL 7.2, SLES 12, SLES 12 SP1 Description: [Tools] rpm rebuild not working Notes: Will not be Fixed; Not a defect Why not? -Hal ----- Original Message ----- > From: "Kevin Harms" <harms(a)alcf.anl.gov> > To: "Hal J. Finkel" <hfinkel(a)anl.gov> > Cc: "Vitali A. Morozov" <morozov(a)alcf.anl.gov>, intel-nda(a)lists.jlse.anl.gov > Sent: Tuesday, June 28, 2016 7:30:30 PM > Subject: Re: [JLSE-Intel-NDA] Intel DAP boxes set up > > > I believe this kernel is just a set of selected backports from the > main kernel. It’s possible this bug is related to a problem with > PTE handling. Intel reported some bug where dirty pages were > handled incorrectly under some conditions. That’s all the info we > got. I haven’t seen the fix yet. The mpsp 1.3.3 is out as well, > renamed xppsl_1.3.3. > > https://software.intel.com/en-us/articles/intel-xeon-phi-processor-software > http://registrationcenter-download.intel.com/akdlm/irc_nas/9325/xppsl-1.3.3… > > > kevin > > >Hi Kevin, > > > >FWIW, no, the problem happened again with the new kernel: > > > >Message from syslogd@kingly at Jun 28 19:12:49 ... > > kernel:BUG: soft lockup - CPU#104 stuck for 23s! [cc1plus:77778] > > > >Message from syslogd@kingly at Jun 28 19:12:51 ... > > kernel:BUG: soft lockup - CPU#177 stuck for 23s! [cc1plus:77662] > > > >... > > > >What's special about this kernel? Could I build a mainline kernel > >and expect it to work reasonably? > > > > -Hal > > > >----- Original Message ----- > >> From: "Hal Finkel" <hfinkel(a)anl.gov> > >> To: "Kevin Harms" <harms(a)alcf.anl.gov> > >> Cc: "Vitali A. Morozov" <morozov(a)alcf.anl.gov>, > >> intel-nda(a)lists.jlse.anl.gov > >> Sent: Tuesday, June 28, 2016 12:47:02 PM > >> Subject: Re: [JLSE-Intel-NDA] Intel DAP boxes set up > >> > >> Thanks! I'll upgrade. > >> > >> -Hal > >> > >> ----- Original Message ----- > >> > From: "Kevin Harms" <harms(a)alcf.anl.gov> > >> > To: "Hal J. Finkel" <hfinkel(a)anl.gov>, "Vitali A. Morozov" > >> > <morozov(a)alcf.anl.gov> > >> > Cc: intel-nda(a)lists.jlse.anl.gov > >> > Sent: Tuesday, June 28, 2016 11:59:20 AM > >> > Subject: Re: [JLSE-Intel-NDA] Intel DAP boxes set up > >> > > >> > > >> > I updated this page to indicate how to upgrade to the mpsp > >> > 1.3.2 > >> > kernel. https://wiki.jlse.anl.gov/display/knl/Intel+DAP+Setup > >> > > >> > No idea, if that will fix this problem though :) > >> > > >> > kevin > >> > > >> > >FYI: James and I have been experimenting with our box. We had > >> > >to > >> > >reset it after we hit some kernel bug: > >> > > > >> > >Message from syslogd@kingly at Jun 27 14:29:44 ... > >> > > kernel:BUG: soft lockup - CPU#77 stuck for 22s! > >> > > [cc1plus:15316] > >> > > > >> > >Message from syslogd@kingly at Jun 27 14:30:43 ... > >> > > kernel:BUG: soft lockup - CPU#149 stuck for 23s! > >> > > [cc1plus:14840] > >> > > > >> > >Message from syslogd@kingly at Jun 27 14:30:44 ... > >> > > kernel:BUG: soft lockup - CPU#8 stuck for 23s! [cc1plus:15337] > >> > > > >> > >Message from syslogd@kingly at Jun 27 14:30:46 ... > >> > > kernel:BUG: soft lockup - CPU#49 stuck for 22s! [as:16806] > >> > > > >> > >Message from syslogd@kingly at Jun 27 14:30:47 ... > >> > > kernel:BUG: soft lockup - CPU#27 stuck for 22s! > >> > > [cc1plus:16548] > >> > > > >> > >Message from syslogd@kingly at Jun 27 14:30:50 ... > >> > > kernel:BUG: soft lockup - CPU#34 stuck for 22s! [cmake:16807] > >> > > > >> > >Message from syslogd@kingly at Jun 27 14:30:52 ... > >> > > kernel:BUG: soft lockup - CPU#154 stuck for 22s! [cmake:16810] > >> > > > >> > >Message from syslogd@kingly at Jun 27 14:30:54 ... > >> > > kernel:BUG: soft lockup - CPU#35 stuck for 23s! > >> > > [cc1plus:15807] > >> > > > >> > >Message from syslogd@kingly at Jun 27 14:30:55 ... > >> > > kernel:BUG: soft lockup - CPU#77 stuck for 22s! > >> > > [cc1plus:15316] > >> > > > >> > >Message from syslogd@kingly at Jun 27 14:30:55 ... > >> > > kernel:BUG: soft lockup - CPU#91 stuck for 22s! [cmake:16813] > >> > > > >> > >Message from syslogd@kingly at Jun 27 14:30:58 ... > >> > > kernel:BUG: soft lockup - CPU#124 stuck for 23s! [make:16811] > >> > > > >> > >Message from syslogd@kingly at Jun 27 14:31:00 ... > >> > > kernel:BUG: soft lockup - CPU#149 stuck for 23s! > >> > > [cc1plus:14840] > >> > > > >> > >... (this went on for a while) > >> > > > >> > >The kernel (3.10.0-327.el7.centos.mpsp_1.3.1.45.x86_64) seems > >> > >like > >> > >some special build? > >> > > > >> > > -Hal > >> > > > >> > >----- Original Message ----- > >> > >> From: "Vitali A. Morozov" <morozov(a)anl.gov> > >> > >> To: "Kalyan Kumaran" <kumaran(a)alcf.anl.gov>, > >> > >> intel-nda(a)lists.jlse.anl.gov > >> > >> Sent: Thursday, June 23, 2016 10:47:37 AM > >> > >> Subject: Re: [JLSE-Intel-NDA] Intel DAP boxes set up > >> > >> > >> > >> > >> > >> > >> > >> I have updated the Ethernet port issue that I have > >> > >> encountered. > >> > >> > >> > >> Vitali > >> > >> > >> > >> On 06/23/2016 10:42 AM, Kumaran, Kalyan wrote: > >> > >> > >> > >> > >> > >> For folks setting these up please see instructions at: > >> > >> > >> > >> > >> > >> https://wiki.jlse.anl.gov/display/knl/Intel+DAP+Setup > >> > >> > >> > >> > >> > >> > >> > >> Kevin, Scott and Vitali will be adding information as they > >> > >> set > >> > >> the > >> > >> machines up. Others are welcome to add stuff. > >> > >> > >> > >> > >> > >> -Kumar > >> > >> > >> > >> _______________________________________________ > >> > >> Intel-nda mailing list Intel-nda(a)lists.jlse.anl.gov > >> > >> https://lists.jlse.anl.gov/mailman/listinfo/intel-nda > >> > >> > >> > >> _______________________________________________ > >> > >> Intel-nda mailing list > >> > >> Intel-nda(a)lists.jlse.anl.gov > >> > >> https://lists.jlse.anl.gov/mailman/listinfo/intel-nda > >> > >> > >> > > > >> > >-- > >> > >Hal Finkel > >> > >Assistant Computational Scientist > >> > >Leadership Computing Facility > >> > >Argonne National Laboratory > >> > >_______________________________________________ > >> > >Intel-nda mailing list > >> > >Intel-nda(a)lists.jlse.anl.gov > >> > >https://lists.jlse.anl.gov/mailman/listinfo/intel-nda > >> > > >> > >> -- > >> Hal Finkel > >> Assistant Computational Scientist > >> Leadership Computing Facility > >> Argonne National Laboratory > >> _______________________________________________ > >> Intel-nda mailing list > >> Intel-nda(a)lists.jlse.anl.gov > >> https://lists.jlse.anl.gov/mailman/listinfo/intel-nda > >> > > > >-- > >Hal Finkel > >Assistant Computational Scientist > >Leadership Computing Facility > >Argonne National Laboratory > -- Hal Finkel Assistant Computational Scientist Leadership Computing Facility Argonne National Laboratory

1 0

Re: [JLSE-Intel-NDA] Intel DAP boxes set up
by Harms, Kevin 28 Jun '16

28 Jun '16

I believe this kernel is just a set of selected backports from the main kernel. It’s possible this bug is related to a problem with PTE handling. Intel reported some bug where dirty pages were handled incorrectly under some conditions. That’s all the info we got. I haven’t seen the fix yet. The mpsp 1.3.3 is out as well, renamed xppsl_1.3.3. https://software.intel.com/en-us/articles/intel-xeon-phi-processor-software http://registrationcenter-download.intel.com/akdlm/irc_nas/9325/xppsl-1.3.3… kevin >Hi Kevin, > >FWIW, no, the problem happened again with the new kernel: > >Message from syslogd@kingly at Jun 28 19:12:49 ... > kernel:BUG: soft lockup - CPU#104 stuck for 23s! [cc1plus:77778] > >Message from syslogd@kingly at Jun 28 19:12:51 ... > kernel:BUG: soft lockup - CPU#177 stuck for 23s! [cc1plus:77662] > >... > >What's special about this kernel? Could I build a mainline kernel and expect it to work reasonably? > > -Hal > >----- Original Message ----- >> From: "Hal Finkel" <hfinkel(a)anl.gov> >> To: "Kevin Harms" <harms(a)alcf.anl.gov> >> Cc: "Vitali A. Morozov" <morozov(a)alcf.anl.gov>, intel-nda(a)lists.jlse.anl.gov >> Sent: Tuesday, June 28, 2016 12:47:02 PM >> Subject: Re: [JLSE-Intel-NDA] Intel DAP boxes set up >> >> Thanks! I'll upgrade. >> >> -Hal >> >> ----- Original Message ----- >> > From: "Kevin Harms" <harms(a)alcf.anl.gov> >> > To: "Hal J. Finkel" <hfinkel(a)anl.gov>, "Vitali A. Morozov" >> > <morozov(a)alcf.anl.gov> >> > Cc: intel-nda(a)lists.jlse.anl.gov >> > Sent: Tuesday, June 28, 2016 11:59:20 AM >> > Subject: Re: [JLSE-Intel-NDA] Intel DAP boxes set up >> > >> > >> > I updated this page to indicate how to upgrade to the mpsp 1.3.2 >> > kernel. https://wiki.jlse.anl.gov/display/knl/Intel+DAP+Setup >> > >> > No idea, if that will fix this problem though :) >> > >> > kevin >> > >> > >FYI: James and I have been experimenting with our box. We had to >> > >reset it after we hit some kernel bug: >> > > >> > >Message from syslogd@kingly at Jun 27 14:29:44 ... >> > > kernel:BUG: soft lockup - CPU#77 stuck for 22s! [cc1plus:15316] >> > > >> > >Message from syslogd@kingly at Jun 27 14:30:43 ... >> > > kernel:BUG: soft lockup - CPU#149 stuck for 23s! [cc1plus:14840] >> > > >> > >Message from syslogd@kingly at Jun 27 14:30:44 ... >> > > kernel:BUG: soft lockup - CPU#8 stuck for 23s! [cc1plus:15337] >> > > >> > >Message from syslogd@kingly at Jun 27 14:30:46 ... >> > > kernel:BUG: soft lockup - CPU#49 stuck for 22s! [as:16806] >> > > >> > >Message from syslogd@kingly at Jun 27 14:30:47 ... >> > > kernel:BUG: soft lockup - CPU#27 stuck for 22s! [cc1plus:16548] >> > > >> > >Message from syslogd@kingly at Jun 27 14:30:50 ... >> > > kernel:BUG: soft lockup - CPU#34 stuck for 22s! [cmake:16807] >> > > >> > >Message from syslogd@kingly at Jun 27 14:30:52 ... >> > > kernel:BUG: soft lockup - CPU#154 stuck for 22s! [cmake:16810] >> > > >> > >Message from syslogd@kingly at Jun 27 14:30:54 ... >> > > kernel:BUG: soft lockup - CPU#35 stuck for 23s! [cc1plus:15807] >> > > >> > >Message from syslogd@kingly at Jun 27 14:30:55 ... >> > > kernel:BUG: soft lockup - CPU#77 stuck for 22s! [cc1plus:15316] >> > > >> > >Message from syslogd@kingly at Jun 27 14:30:55 ... >> > > kernel:BUG: soft lockup - CPU#91 stuck for 22s! [cmake:16813] >> > > >> > >Message from syslogd@kingly at Jun 27 14:30:58 ... >> > > kernel:BUG: soft lockup - CPU#124 stuck for 23s! [make:16811] >> > > >> > >Message from syslogd@kingly at Jun 27 14:31:00 ... >> > > kernel:BUG: soft lockup - CPU#149 stuck for 23s! [cc1plus:14840] >> > > >> > >... (this went on for a while) >> > > >> > >The kernel (3.10.0-327.el7.centos.mpsp_1.3.1.45.x86_64) seems like >> > >some special build? >> > > >> > > -Hal >> > > >> > >----- Original Message ----- >> > >> From: "Vitali A. Morozov" <morozov(a)anl.gov> >> > >> To: "Kalyan Kumaran" <kumaran(a)alcf.anl.gov>, >> > >> intel-nda(a)lists.jlse.anl.gov >> > >> Sent: Thursday, June 23, 2016 10:47:37 AM >> > >> Subject: Re: [JLSE-Intel-NDA] Intel DAP boxes set up >> > >> >> > >> >> > >> >> > >> I have updated the Ethernet port issue that I have encountered. >> > >> >> > >> Vitali >> > >> >> > >> On 06/23/2016 10:42 AM, Kumaran, Kalyan wrote: >> > >> >> > >> >> > >> For folks setting these up please see instructions at: >> > >> >> > >> >> > >> https://wiki.jlse.anl.gov/display/knl/Intel+DAP+Setup >> > >> >> > >> >> > >> >> > >> Kevin, Scott and Vitali will be adding information as they set >> > >> the >> > >> machines up. Others are welcome to add stuff. >> > >> >> > >> >> > >> -Kumar >> > >> >> > >> _______________________________________________ >> > >> Intel-nda mailing list Intel-nda(a)lists.jlse.anl.gov >> > >> https://lists.jlse.anl.gov/mailman/listinfo/intel-nda >> > >> >> > >> _______________________________________________ >> > >> Intel-nda mailing list >> > >> Intel-nda(a)lists.jlse.anl.gov >> > >> https://lists.jlse.anl.gov/mailman/listinfo/intel-nda >> > >> >> > > >> > >-- >> > >Hal Finkel >> > >Assistant Computational Scientist >> > >Leadership Computing Facility >> > >Argonne National Laboratory >> > >_______________________________________________ >> > >Intel-nda mailing list >> > >Intel-nda(a)lists.jlse.anl.gov >> > >https://lists.jlse.anl.gov/mailman/listinfo/intel-nda >> > >> >> -- >> Hal Finkel >> Assistant Computational Scientist >> Leadership Computing Facility >> Argonne National Laboratory >> _______________________________________________ >> Intel-nda mailing list >> Intel-nda(a)lists.jlse.anl.gov >> https://lists.jlse.anl.gov/mailman/listinfo/intel-nda >> > >-- >Hal Finkel >Assistant Computational Scientist >Leadership Computing Facility >Argonne National Laboratory

1 0

Re: [JLSE-Intel-NDA] Intel DAP boxes set up
by Hal Finkel 28 Jun '16

28 Jun '16

Hi Kevin, FWIW, no, the problem happened again with the new kernel: Message from syslogd@kingly at Jun 28 19:12:49 ... kernel:BUG: soft lockup - CPU#104 stuck for 23s! [cc1plus:77778] Message from syslogd@kingly at Jun 28 19:12:51 ... kernel:BUG: soft lockup - CPU#177 stuck for 23s! [cc1plus:77662] ... What's special about this kernel? Could I build a mainline kernel and expect it to work reasonably? -Hal ----- Original Message ----- > From: "Hal Finkel" <hfinkel(a)anl.gov> > To: "Kevin Harms" <harms(a)alcf.anl.gov> > Cc: "Vitali A. Morozov" <morozov(a)alcf.anl.gov>, intel-nda(a)lists.jlse.anl.gov > Sent: Tuesday, June 28, 2016 12:47:02 PM > Subject: Re: [JLSE-Intel-NDA] Intel DAP boxes set up > > Thanks! I'll upgrade. > > -Hal > > ----- Original Message ----- > > From: "Kevin Harms" <harms(a)alcf.anl.gov> > > To: "Hal J. Finkel" <hfinkel(a)anl.gov>, "Vitali A. Morozov" > > <morozov(a)alcf.anl.gov> > > Cc: intel-nda(a)lists.jlse.anl.gov > > Sent: Tuesday, June 28, 2016 11:59:20 AM > > Subject: Re: [JLSE-Intel-NDA] Intel DAP boxes set up > > > > > > I updated this page to indicate how to upgrade to the mpsp 1.3.2 > > kernel. https://wiki.jlse.anl.gov/display/knl/Intel+DAP+Setup > > > > No idea, if that will fix this problem though :) > > > > kevin > > > > >FYI: James and I have been experimenting with our box. We had to > > >reset it after we hit some kernel bug: > > > > > >Message from syslogd@kingly at Jun 27 14:29:44 ... > > > kernel:BUG: soft lockup - CPU#77 stuck for 22s! [cc1plus:15316] > > > > > >Message from syslogd@kingly at Jun 27 14:30:43 ... > > > kernel:BUG: soft lockup - CPU#149 stuck for 23s! [cc1plus:14840] > > > > > >Message from syslogd@kingly at Jun 27 14:30:44 ... > > > kernel:BUG: soft lockup - CPU#8 stuck for 23s! [cc1plus:15337] > > > > > >Message from syslogd@kingly at Jun 27 14:30:46 ... > > > kernel:BUG: soft lockup - CPU#49 stuck for 22s! [as:16806] > > > > > >Message from syslogd@kingly at Jun 27 14:30:47 ... > > > kernel:BUG: soft lockup - CPU#27 stuck for 22s! [cc1plus:16548] > > > > > >Message from syslogd@kingly at Jun 27 14:30:50 ... > > > kernel:BUG: soft lockup - CPU#34 stuck for 22s! [cmake:16807] > > > > > >Message from syslogd@kingly at Jun 27 14:30:52 ... > > > kernel:BUG: soft lockup - CPU#154 stuck for 22s! [cmake:16810] > > > > > >Message from syslogd@kingly at Jun 27 14:30:54 ... > > > kernel:BUG: soft lockup - CPU#35 stuck for 23s! [cc1plus:15807] > > > > > >Message from syslogd@kingly at Jun 27 14:30:55 ... > > > kernel:BUG: soft lockup - CPU#77 stuck for 22s! [cc1plus:15316] > > > > > >Message from syslogd@kingly at Jun 27 14:30:55 ... > > > kernel:BUG: soft lockup - CPU#91 stuck for 22s! [cmake:16813] > > > > > >Message from syslogd@kingly at Jun 27 14:30:58 ... > > > kernel:BUG: soft lockup - CPU#124 stuck for 23s! [make:16811] > > > > > >Message from syslogd@kingly at Jun 27 14:31:00 ... > > > kernel:BUG: soft lockup - CPU#149 stuck for 23s! [cc1plus:14840] > > > > > >... (this went on for a while) > > > > > >The kernel (3.10.0-327.el7.centos.mpsp_1.3.1.45.x86_64) seems like > > >some special build? > > > > > > -Hal > > > > > >----- Original Message ----- > > >> From: "Vitali A. Morozov" <morozov(a)anl.gov> > > >> To: "Kalyan Kumaran" <kumaran(a)alcf.anl.gov>, > > >> intel-nda(a)lists.jlse.anl.gov > > >> Sent: Thursday, June 23, 2016 10:47:37 AM > > >> Subject: Re: [JLSE-Intel-NDA] Intel DAP boxes set up > > >> > > >> > > >> > > >> I have updated the Ethernet port issue that I have encountered. > > >> > > >> Vitali > > >> > > >> On 06/23/2016 10:42 AM, Kumaran, Kalyan wrote: > > >> > > >> > > >> For folks setting these up please see instructions at: > > >> > > >> > > >> https://wiki.jlse.anl.gov/display/knl/Intel+DAP+Setup > > >> > > >> > > >> > > >> Kevin, Scott and Vitali will be adding information as they set > > >> the > > >> machines up. Others are welcome to add stuff. > > >> > > >> > > >> -Kumar > > >> > > >> _______________________________________________ > > >> Intel-nda mailing list Intel-nda(a)lists.jlse.anl.gov > > >> https://lists.jlse.anl.gov/mailman/listinfo/intel-nda > > >> > > >> _______________________________________________ > > >> Intel-nda mailing list > > >> Intel-nda(a)lists.jlse.anl.gov > > >> https://lists.jlse.anl.gov/mailman/listinfo/intel-nda > > >> > > > > > >-- > > >Hal Finkel > > >Assistant Computational Scientist > > >Leadership Computing Facility > > >Argonne National Laboratory > > >_______________________________________________ > > >Intel-nda mailing list > > >Intel-nda(a)lists.jlse.anl.gov > > >https://lists.jlse.anl.gov/mailman/listinfo/intel-nda > > > > -- > Hal Finkel > Assistant Computational Scientist > Leadership Computing Facility > Argonne National Laboratory > _______________________________________________ > Intel-nda mailing list > Intel-nda(a)lists.jlse.anl.gov > https://lists.jlse.anl.gov/mailman/listinfo/intel-nda > -- Hal Finkel Assistant Computational Scientist Leadership Computing Facility Argonne National Laboratory

1 0