March 2016 - Intel-nda - mailman3.cels.anl.gov

Re: [JLSE-Intel-NDA] Kernel Panic on KNL Box
by Allen, Benjamin S. 29 Mar '16

29 Mar '16

Vitali, Upgraded the BIOS with A0 and swapped the B0 back in. Appears to be a lot more happy now. I can't reproduce the panic as I had before. Various release notes and updated manuals attached. Feel free to start testing. Thanks, Ben

1 0

Re: [JLSE-Intel-NDA] Kernel Panic on KNL Box
by Allen, Benjamin S. 28 Mar '16

28 Mar '16

A0 chip is installed. We don't have instructions yet from Intel on how to go about the BIOS upgrade (or where to get it from). The A0 appears to be 66 cores @1.20Ghz. Memory is coming up at 1866 MT/s. 16GB MCDRAM. Ben > On Mar 28, 2016, at 1:16 PM, Vitali A. Morozov <morozov(a)anl.gov> wrote: > > Again... > > Interestingly, this same job passed through in configuration of 16 processes 4 threads each and 8 processes 8 thread each. I am trying 8 processes 4 threads each - cannot make it for 3 times in a row, every time dying in a different spot. > > Vitali > > > > On 03/28/2016 01:01 PM, Allen, Benjamin S. wrote: >> I see that. Power cycling now. >> >> Ben >> >>> On Mar 28, 2016, at 1:00 PM, Vitali A. Morozov <morozov(a)anl.gov> wrote: >>> >>> Died again... >>> >>> >>> On 03/28/2016 11:37 AM, Vitali A. Morozov wrote: >>>> I seems to "kernel panic" the box... >>>> >>>> Vitali >>>> >>>> >>>> On 03/28/2016 11:35 AM, Kumaran, Kalyan wrote: >>>>> Looks like our A0 to update the BIOS has/should arrive(d) today. >>>>> >>>>> -Kumar >>>>> >>>>> >>>>> >>>>>> Begin forwarded message: >>>>>> >>>>>> From: "Allen, Benjamin S." <bsallen(a)alcf.anl.gov> >>>>>> Subject: Re: Kernel Panic on KNL Box >>>>>> Date: March 25, 2016 at 6:29:54 PM CDT >>>>>> To: "Fromkin, Russ" <russ.fromkin(a)intel.com> >>>>>> Cc: "Kumaran, Kalyan" <kumaran(a)alcf.anl.gov>, "Kirkendall, Keith G" <keith.g.kirkendall(a)intel.com>, "Arney, Rebecca" <rebecca.arney(a)intel.com>, "Knight, Christopher J." <knight(a)mcs.anl.gov>, "Allison, CM" <cm.allison(a)intel.com> >>>>>> >>>>>> Thanks: https://www.fedex.com/apps/fedextrack/?tracknumbers=6431-3664-8770 >>>>>> >>>>>> Ben >>>>>> >>>>>>> On Mar 25, 2016, at 6:28 PM, Fromkin, Russ <russ.fromkin(a)intel.com> wrote: >>>>>>> >>>>>>> Here is what I have: >>>>>>> >>>>>>> Shipping Memo (Delivery Number) #1303398298 >>>>>>> Shipment Document #7000316160 >>>>>>> Tracking643136648770 >>>>>>> CarrierFM >>>>>>> ETA03/28/2016 10:00 >>>>>>> >>>>>>> -----Original Message----- >>>>>>> From: Allen, Benjamin S. [mailto:[email protected]] >>>>>>> Sent: Friday, March 25, 2016 7:15 PM >>>>>>> To: Fromkin, Russ <russ.fromkin(a)intel.com> >>>>>>> Cc: Kumaran, Kalyan <kumaran(a)alcf.anl.gov>; Kirkendall, Keith G <keith.g.kirkendall(a)intel.com>; Arney, Rebecca <rebecca.arney(a)intel.com>; Knight, Christopher J. <knight(a)mcs.anl.gov>; Allison, CM <cm.allison(a)intel.com> >>>>>>> Subject: Re: Kernel Panic on KNL Box >>>>>>> >>>>>>> Thanks Russ, can you share the tracking number? >>>>>>> >>>>>>> Ben >>>>>>> >>>>>>>> On Mar 25, 2016, at 5:57 PM, Fromkin, Russ <russ.fromkin(a)intel.com> wrote: >>>>>>>> >>>>>>>> Just received the shipping notice. It should arrive on the 28th. >>>>>>>> >>>>>>>> Russ >>>>>>>> >>>>>>>> -----Original Message----- >>>>>>>> From: Kumaran, Kalyan [mailto:[email protected]] >>>>>>>> Sent: Wednesday, March 23, 2016 12:49 PM >>>>>>>> To: Allen, Benjamin S. <bsallen(a)alcf.anl.gov> >>>>>>>> Cc: Fromkin, Russ <russ.fromkin(a)intel.com>; Kirkendall, Keith G <keith.g.kirkendall(a)intel.com>; Arney, Rebecca <rebecca.arney(a)intel.com>; Knight, Christopher J. <knight(a)mcs.anl.gov>; Allison, CM <cm.allison(a)intel.com> >>>>>>>> Subject: Re: Kernel Panic on KNL Box >>>>>>>> >>>>>>>> If it is being shipped to me I have not got any emails indicating a shipment. >>>>>>>> >>>>>>>> -Kumar >>>>>>>> >>>>>>>> >>>>>>>>> On Mar 23, 2016, at 11:45 AM, Allen, Benjamin S. <bsallen(a)alcf.anl.gov> wrote: >>>>>>>>> >>>>>>>>> Hi Russ, >>>>>>>>> >>>>>>>>> I have a handful of vmcores from the panics if there's any interest from the team. We can also provide remote access to Intel staff if needed. >>>>>>>>> >>>>>>>>> Agreed on the concern about the BIOS. Hopefully we'll get the A0 soon to get it upgraded. >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> >>>>>>>>> Ben >>>>>>>>> >>>>>>>>>> On Mar 23, 2016, at 11:41 AM, Fromkin, Russ <russ.fromkin(a)intel.com> wrote: >>>>>>>>>> >>>>>>>>>> Hi Ben, >>>>>>>>>> >>>>>>>>>> We agree and are trying to work the issue and our LAMMPS team agreed it was not the code. >>>>>>>>>> >>>>>>>>>> I am still worried about the B0 inside the system with a bios that was installed for A0. That being said, I do believe a few others with this machine are having some issues. >>>>>>>>>> >>>>>>>>>> I need to read through the handful of emails from today to see if there is anything of value to share. >>>>>>>>>> >>>>>>>>>> Russ >>>>>>>>>> >>>>>>>>>> -----Original Message----- >>>>>>>>>> From: Allen, Benjamin S. [mailto:[email protected]] >>>>>>>>>> Sent: Wednesday, March 23, 2016 12:33 PM >>>>>>>>>> To: Fromkin, Russ <russ.fromkin(a)intel.com> >>>>>>>>>> Cc: Kirkendall, Keith G <keith.g.kirkendall(a)intel.com>; Arney, Rebecca <rebecca.arney(a)intel.com>; Kumaran, Kalyan <kumaran(a)alcf.anl.gov>; Knight, Christopher J. <knight(a)mcs.anl.gov>; Allison, CM <cm.allison(a)intel.com> >>>>>>>>>> Subject: Re: Kernel Panic on KNL Box >>>>>>>>>> >>>>>>>>>> Hi Russ, >>>>>>>>>> >>>>>>>>>> We can get a similar kernel panic by just running a 128 rank mpi_hello_world app, so I don't think this is actually related to LAMPPS. I'm going to hold off doing additional testing and troubleshooting on this until we get the BIOS upgraded. We're also still seeing Machine Check Errors "Generic CACHE Level-2 Generic Error". >>>>>>>>>> >>>>>>>>>> Ben >>>>>>>>>> >>>>>>>>>>> On Mar 22, 2016, at 3:40 PM, Fromkin, Russ <russ.fromkin(a)intel.com> wrote: >>>>>>>>>>> >>>>>>>>>>> Pulling Rebecca onto the string, as I want to make sure we keep any of this work and early science work together. >>>>>>>>>>> >>>>>>>>>>> We will confirm this the next steps shortly. >>>>>>>>>>> >>>>>>>>>>> Russ >>>>>>>>>>> >>>>>>>>>>> -----Original Message----- >>>>>>>>>>> From: Allen, Benjamin S. [mailto:[email protected]] >>>>>>>>>>> Sent: Tuesday, March 22, 2016 3:30 PM >>>>>>>>>>> To: Fromkin, Russ <russ.fromkin(a)intel.com>; Kirkendall, Keith G <keith.g.kirkendall(a)intel.com> >>>>>>>>>>> Cc: Kumaran, Kalyan <kumaran(a)alcf.anl.gov>; Knight, Christopher J. <knight(a)mcs.anl.gov>; Allison, CM <cm.allison(a)intel.com> >>>>>>>>>>> Subject: Kernel Panic on KNL Box >>>>>>>>>>> >>>>>>>>>>> Russ and Keith, >>>>>>>>>>> >>>>>>>>>>> We have a reproducible kernel panic when running LAMMPS on the KNL B0 box, since this hardware is NDA and not-released we can't really ask for help from RedHat. Who can we get help from? >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> >>>>>>>>>>> Ben >>>>> >>>>> >>>>> _______________________________________________ >>>>> Intel-nda mailing list >>>>> >>>>> Intel-nda(a)lists.jlse.anl.gov >>>>> https://lists.jlse.anl.gov/mailman/listinfo/intel-nda >>>> >>>> >>>> _______________________________________________ >>>> Intel-nda mailing list >>>> >>>> Intel-nda(a)lists.jlse.anl.gov >>>> https://lists.jlse.anl.gov/mailman/listinfo/intel-nda >

1 0

Re: [JLSE-Intel-NDA] Kernel Panic on KNL Box
by Allen, Benjamin S. 28 Mar '16

28 Mar '16

Swapping CPUs. It's up with the A0 now. Ben > On Mar 28, 2016, at 3:28 PM, Vitali A. Morozov <morozov(a)anl.gov> wrote: > > Hi Ben, > > still hanging on "no route to host". > > Vitali > > > > On 03/28/2016 02:21 PM, Allen, Benjamin S. wrote: >> Power cycling now. >> >> Ben >> >>> On Mar 28, 2016, at 1:16 PM, Vitali A. Morozov <morozov(a)anl.gov> wrote: >>> >>> Again... >>> >>> Interestingly, this same job passed through in configuration of 16 processes 4 threads each and 8 processes 8 thread each. I am trying 8 processes 4 threads each - cannot make it for 3 times in a row, every time dying in a different spot. >>> >>> Vitali >>> >>> >>> >>> On 03/28/2016 01:01 PM, Allen, Benjamin S. wrote: >>>> I see that. Power cycling now. >>>> >>>> Ben >>>> >>>>> On Mar 28, 2016, at 1:00 PM, Vitali A. Morozov <morozov(a)anl.gov> wrote: >>>>> >>>>> Died again... >>>>> >>>>> >>>>> On 03/28/2016 11:37 AM, Vitali A. Morozov wrote: >>>>>> I seems to "kernel panic" the box... >>>>>> >>>>>> Vitali >>>>>> >>>>>> >>>>>> On 03/28/2016 11:35 AM, Kumaran, Kalyan wrote: >>>>>>> Looks like our A0 to update the BIOS has/should arrive(d) today. >>>>>>> >>>>>>> -Kumar >>>>>>> >>>>>>> >>>>>>> >>>>>>>> Begin forwarded message: >>>>>>>> >>>>>>>> From: "Allen, Benjamin S." <bsallen(a)alcf.anl.gov> >>>>>>>> Subject: Re: Kernel Panic on KNL Box >>>>>>>> Date: March 25, 2016 at 6:29:54 PM CDT >>>>>>>> To: "Fromkin, Russ" <russ.fromkin(a)intel.com> >>>>>>>> Cc: "Kumaran, Kalyan" <kumaran(a)alcf.anl.gov>, "Kirkendall, Keith G" <keith.g.kirkendall(a)intel.com>, "Arney, Rebecca" <rebecca.arney(a)intel.com>, "Knight, Christopher J." <knight(a)mcs.anl.gov>, "Allison, CM" <cm.allison(a)intel.com> >>>>>>>> >>>>>>>> Thanks: https://www.fedex.com/apps/fedextrack/?tracknumbers=6431-3664-8770 >>>>>>>> >>>>>>>> Ben >>>>>>>> >>>>>>>>> On Mar 25, 2016, at 6:28 PM, Fromkin, Russ <russ.fromkin(a)intel.com> wrote: >>>>>>>>> >>>>>>>>> Here is what I have: >>>>>>>>> >>>>>>>>> Shipping Memo (Delivery Number) #1303398298 >>>>>>>>> Shipment Document #7000316160 >>>>>>>>> Tracking643136648770 >>>>>>>>> CarrierFM >>>>>>>>> ETA03/28/2016 10:00 >>>>>>>>> >>>>>>>>> -----Original Message----- >>>>>>>>> From: Allen, Benjamin S. [mailto:[email protected]] >>>>>>>>> Sent: Friday, March 25, 2016 7:15 PM >>>>>>>>> To: Fromkin, Russ <russ.fromkin(a)intel.com> >>>>>>>>> Cc: Kumaran, Kalyan <kumaran(a)alcf.anl.gov>; Kirkendall, Keith G <keith.g.kirkendall(a)intel.com>; Arney, Rebecca <rebecca.arney(a)intel.com>; Knight, Christopher J. <knight(a)mcs.anl.gov>; Allison, CM <cm.allison(a)intel.com> >>>>>>>>> Subject: Re: Kernel Panic on KNL Box >>>>>>>>> >>>>>>>>> Thanks Russ, can you share the tracking number? >>>>>>>>> >>>>>>>>> Ben >>>>>>>>> >>>>>>>>>> On Mar 25, 2016, at 5:57 PM, Fromkin, Russ <russ.fromkin(a)intel.com> wrote: >>>>>>>>>> >>>>>>>>>> Just received the shipping notice. It should arrive on the 28th. >>>>>>>>>> >>>>>>>>>> Russ >>>>>>>>>> >>>>>>>>>> -----Original Message----- >>>>>>>>>> From: Kumaran, Kalyan [mailto:[email protected]] >>>>>>>>>> Sent: Wednesday, March 23, 2016 12:49 PM >>>>>>>>>> To: Allen, Benjamin S. <bsallen(a)alcf.anl.gov> >>>>>>>>>> Cc: Fromkin, Russ <russ.fromkin(a)intel.com>; Kirkendall, Keith G <keith.g.kirkendall(a)intel.com>; Arney, Rebecca <rebecca.arney(a)intel.com>; Knight, Christopher J. <knight(a)mcs.anl.gov>; Allison, CM <cm.allison(a)intel.com> >>>>>>>>>> Subject: Re: Kernel Panic on KNL Box >>>>>>>>>> >>>>>>>>>> If it is being shipped to me I have not got any emails indicating a shipment. >>>>>>>>>> >>>>>>>>>> -Kumar >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> On Mar 23, 2016, at 11:45 AM, Allen, Benjamin S. <bsallen(a)alcf.anl.gov> wrote: >>>>>>>>>>> >>>>>>>>>>> Hi Russ, >>>>>>>>>>> >>>>>>>>>>> I have a handful of vmcores from the panics if there's any interest from the team. We can also provide remote access to Intel staff if needed. >>>>>>>>>>> >>>>>>>>>>> Agreed on the concern about the BIOS. Hopefully we'll get the A0 soon to get it upgraded. >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> >>>>>>>>>>> Ben >>>>>>>>>>> >>>>>>>>>>>> On Mar 23, 2016, at 11:41 AM, Fromkin, Russ <russ.fromkin(a)intel.com> wrote: >>>>>>>>>>>> >>>>>>>>>>>> Hi Ben, >>>>>>>>>>>> >>>>>>>>>>>> We agree and are trying to work the issue and our LAMMPS team agreed it was not the code. >>>>>>>>>>>> >>>>>>>>>>>> I am still worried about the B0 inside the system with a bios that was installed for A0. That being said, I do believe a few others with this machine are having some issues. >>>>>>>>>>>> >>>>>>>>>>>> I need to read through the handful of emails from today to see if there is anything of value to share. >>>>>>>>>>>> >>>>>>>>>>>> Russ >>>>>>>>>>>> >>>>>>>>>>>> -----Original Message----- >>>>>>>>>>>> From: Allen, Benjamin S. [mailto:[email protected]] >>>>>>>>>>>> Sent: Wednesday, March 23, 2016 12:33 PM >>>>>>>>>>>> To: Fromkin, Russ <russ.fromkin(a)intel.com> >>>>>>>>>>>> Cc: Kirkendall, Keith G <keith.g.kirkendall(a)intel.com>; Arney, Rebecca <rebecca.arney(a)intel.com>; Kumaran, Kalyan <kumaran(a)alcf.anl.gov>; Knight, Christopher J. <knight(a)mcs.anl.gov>; Allison, CM <cm.allison(a)intel.com> >>>>>>>>>>>> Subject: Re: Kernel Panic on KNL Box >>>>>>>>>>>> >>>>>>>>>>>> Hi Russ, >>>>>>>>>>>> >>>>>>>>>>>> We can get a similar kernel panic by just running a 128 rank mpi_hello_world app, so I don't think this is actually related to LAMPPS. I'm going to hold off doing additional testing and troubleshooting on this until we get the BIOS upgraded. We're also still seeing Machine Check Errors "Generic CACHE Level-2 Generic Error". >>>>>>>>>>>> >>>>>>>>>>>> Ben >>>>>>>>>>>> >>>>>>>>>>>>> On Mar 22, 2016, at 3:40 PM, Fromkin, Russ <russ.fromkin(a)intel.com> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>> Pulling Rebecca onto the string, as I want to make sure we keep any of this work and early science work together. >>>>>>>>>>>>> >>>>>>>>>>>>> We will confirm this the next steps shortly. >>>>>>>>>>>>> >>>>>>>>>>>>> Russ >>>>>>>>>>>>> >>>>>>>>>>>>> -----Original Message----- >>>>>>>>>>>>> From: Allen, Benjamin S. [mailto:[email protected]] >>>>>>>>>>>>> Sent: Tuesday, March 22, 2016 3:30 PM >>>>>>>>>>>>> To: Fromkin, Russ <russ.fromkin(a)intel.com>; Kirkendall, Keith G <keith.g.kirkendall(a)intel.com> >>>>>>>>>>>>> Cc: Kumaran, Kalyan <kumaran(a)alcf.anl.gov>; Knight, Christopher J. <knight(a)mcs.anl.gov>; Allison, CM <cm.allison(a)intel.com> >>>>>>>>>>>>> Subject: Kernel Panic on KNL Box >>>>>>>>>>>>> >>>>>>>>>>>>> Russ and Keith, >>>>>>>>>>>>> >>>>>>>>>>>>> We have a reproducible kernel panic when running LAMMPS on the KNL B0 box, since this hardware is NDA and not-released we can't really ask for help from RedHat. Who can we get help from? >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks, >>>>>>>>>>>>> >>>>>>>>>>>>> Ben >>>>>>> >>>>>>> _______________________________________________ >>>>>>> Intel-nda mailing list >>>>>>> >>>>>>> Intel-nda(a)lists.jlse.anl.gov >>>>>>> https://lists.jlse.anl.gov/mailman/listinfo/intel-nda >>>>>> >>>>>> _______________________________________________ >>>>>> Intel-nda mailing list >>>>>> >>>>>> Intel-nda(a)lists.jlse.anl.gov >>>>>> https://lists.jlse.anl.gov/mailman/listinfo/intel-nda >

1 0

Re: [JLSE-Intel-NDA] Kernel Panic on KNL Box
by Vitali A. Morozov 28 Mar '16

28 Mar '16

Hi Ben, still hanging on "no route to host". Vitali On 03/28/2016 02:21 PM, Allen, Benjamin S. wrote: > Power cycling now. > > Ben > >> On Mar 28, 2016, at 1:16 PM, Vitali A. Morozov <morozov(a)anl.gov> wrote: >> >> Again... >> >> Interestingly, this same job passed through in configuration of 16 processes 4 threads each and 8 processes 8 thread each. I am trying 8 processes 4 threads each - cannot make it for 3 times in a row, every time dying in a different spot. >> >> Vitali >> >> >> >> On 03/28/2016 01:01 PM, Allen, Benjamin S. wrote: >>> I see that. Power cycling now. >>> >>> Ben >>> >>>> On Mar 28, 2016, at 1:00 PM, Vitali A. Morozov <morozov(a)anl.gov> wrote: >>>> >>>> Died again... >>>> >>>> >>>> On 03/28/2016 11:37 AM, Vitali A. Morozov wrote: >>>>> I seems to "kernel panic" the box... >>>>> >>>>> Vitali >>>>> >>>>> >>>>> On 03/28/2016 11:35 AM, Kumaran, Kalyan wrote: >>>>>> Looks like our A0 to update the BIOS has/should arrive(d) today. >>>>>> >>>>>> -Kumar >>>>>> >>>>>> >>>>>> >>>>>>> Begin forwarded message: >>>>>>> >>>>>>> From: "Allen, Benjamin S." <bsallen(a)alcf.anl.gov> >>>>>>> Subject: Re: Kernel Panic on KNL Box >>>>>>> Date: March 25, 2016 at 6:29:54 PM CDT >>>>>>> To: "Fromkin, Russ" <russ.fromkin(a)intel.com> >>>>>>> Cc: "Kumaran, Kalyan" <kumaran(a)alcf.anl.gov>, "Kirkendall, Keith G" <keith.g.kirkendall(a)intel.com>, "Arney, Rebecca" <rebecca.arney(a)intel.com>, "Knight, Christopher J." <knight(a)mcs.anl.gov>, "Allison, CM" <cm.allison(a)intel.com> >>>>>>> >>>>>>> Thanks: https://www.fedex.com/apps/fedextrack/?tracknumbers=6431-3664-8770 >>>>>>> >>>>>>> Ben >>>>>>> >>>>>>>> On Mar 25, 2016, at 6:28 PM, Fromkin, Russ <russ.fromkin(a)intel.com> wrote: >>>>>>>> >>>>>>>> Here is what I have: >>>>>>>> >>>>>>>> Shipping Memo (Delivery Number) #1303398298 >>>>>>>> Shipment Document #7000316160 >>>>>>>> Tracking643136648770 >>>>>>>> CarrierFM >>>>>>>> ETA03/28/2016 10:00 >>>>>>>> >>>>>>>> -----Original Message----- >>>>>>>> From: Allen, Benjamin S. [mailto:[email protected]] >>>>>>>> Sent: Friday, March 25, 2016 7:15 PM >>>>>>>> To: Fromkin, Russ <russ.fromkin(a)intel.com> >>>>>>>> Cc: Kumaran, Kalyan <kumaran(a)alcf.anl.gov>; Kirkendall, Keith G <keith.g.kirkendall(a)intel.com>; Arney, Rebecca <rebecca.arney(a)intel.com>; Knight, Christopher J. <knight(a)mcs.anl.gov>; Allison, CM <cm.allison(a)intel.com> >>>>>>>> Subject: Re: Kernel Panic on KNL Box >>>>>>>> >>>>>>>> Thanks Russ, can you share the tracking number? >>>>>>>> >>>>>>>> Ben >>>>>>>> >>>>>>>>> On Mar 25, 2016, at 5:57 PM, Fromkin, Russ <russ.fromkin(a)intel.com> wrote: >>>>>>>>> >>>>>>>>> Just received the shipping notice. It should arrive on the 28th. >>>>>>>>> >>>>>>>>> Russ >>>>>>>>> >>>>>>>>> -----Original Message----- >>>>>>>>> From: Kumaran, Kalyan [mailto:[email protected]] >>>>>>>>> Sent: Wednesday, March 23, 2016 12:49 PM >>>>>>>>> To: Allen, Benjamin S. <bsallen(a)alcf.anl.gov> >>>>>>>>> Cc: Fromkin, Russ <russ.fromkin(a)intel.com>; Kirkendall, Keith G <keith.g.kirkendall(a)intel.com>; Arney, Rebecca <rebecca.arney(a)intel.com>; Knight, Christopher J. <knight(a)mcs.anl.gov>; Allison, CM <cm.allison(a)intel.com> >>>>>>>>> Subject: Re: Kernel Panic on KNL Box >>>>>>>>> >>>>>>>>> If it is being shipped to me I have not got any emails indicating a shipment. >>>>>>>>> >>>>>>>>> -Kumar >>>>>>>>> >>>>>>>>> >>>>>>>>>> On Mar 23, 2016, at 11:45 AM, Allen, Benjamin S. <bsallen(a)alcf.anl.gov> wrote: >>>>>>>>>> >>>>>>>>>> Hi Russ, >>>>>>>>>> >>>>>>>>>> I have a handful of vmcores from the panics if there's any interest from the team. We can also provide remote access to Intel staff if needed. >>>>>>>>>> >>>>>>>>>> Agreed on the concern about the BIOS. Hopefully we'll get the A0 soon to get it upgraded. >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> >>>>>>>>>> Ben >>>>>>>>>> >>>>>>>>>>> On Mar 23, 2016, at 11:41 AM, Fromkin, Russ <russ.fromkin(a)intel.com> wrote: >>>>>>>>>>> >>>>>>>>>>> Hi Ben, >>>>>>>>>>> >>>>>>>>>>> We agree and are trying to work the issue and our LAMMPS team agreed it was not the code. >>>>>>>>>>> >>>>>>>>>>> I am still worried about the B0 inside the system with a bios that was installed for A0. That being said, I do believe a few others with this machine are having some issues. >>>>>>>>>>> >>>>>>>>>>> I need to read through the handful of emails from today to see if there is anything of value to share. >>>>>>>>>>> >>>>>>>>>>> Russ >>>>>>>>>>> >>>>>>>>>>> -----Original Message----- >>>>>>>>>>> From: Allen, Benjamin S. [mailto:[email protected]] >>>>>>>>>>> Sent: Wednesday, March 23, 2016 12:33 PM >>>>>>>>>>> To: Fromkin, Russ <russ.fromkin(a)intel.com> >>>>>>>>>>> Cc: Kirkendall, Keith G <keith.g.kirkendall(a)intel.com>; Arney, Rebecca <rebecca.arney(a)intel.com>; Kumaran, Kalyan <kumaran(a)alcf.anl.gov>; Knight, Christopher J. <knight(a)mcs.anl.gov>; Allison, CM <cm.allison(a)intel.com> >>>>>>>>>>> Subject: Re: Kernel Panic on KNL Box >>>>>>>>>>> >>>>>>>>>>> Hi Russ, >>>>>>>>>>> >>>>>>>>>>> We can get a similar kernel panic by just running a 128 rank mpi_hello_world app, so I don't think this is actually related to LAMPPS. I'm going to hold off doing additional testing and troubleshooting on this until we get the BIOS upgraded. We're also still seeing Machine Check Errors "Generic CACHE Level-2 Generic Error". >>>>>>>>>>> >>>>>>>>>>> Ben >>>>>>>>>>> >>>>>>>>>>>> On Mar 22, 2016, at 3:40 PM, Fromkin, Russ <russ.fromkin(a)intel.com> wrote: >>>>>>>>>>>> >>>>>>>>>>>> Pulling Rebecca onto the string, as I want to make sure we keep any of this work and early science work together. >>>>>>>>>>>> >>>>>>>>>>>> We will confirm this the next steps shortly. >>>>>>>>>>>> >>>>>>>>>>>> Russ >>>>>>>>>>>> >>>>>>>>>>>> -----Original Message----- >>>>>>>>>>>> From: Allen, Benjamin S. [mailto:[email protected]] >>>>>>>>>>>> Sent: Tuesday, March 22, 2016 3:30 PM >>>>>>>>>>>> To: Fromkin, Russ <russ.fromkin(a)intel.com>; Kirkendall, Keith G <keith.g.kirkendall(a)intel.com> >>>>>>>>>>>> Cc: Kumaran, Kalyan <kumaran(a)alcf.anl.gov>; Knight, Christopher J. <knight(a)mcs.anl.gov>; Allison, CM <cm.allison(a)intel.com> >>>>>>>>>>>> Subject: Kernel Panic on KNL Box >>>>>>>>>>>> >>>>>>>>>>>> Russ and Keith, >>>>>>>>>>>> >>>>>>>>>>>> We have a reproducible kernel panic when running LAMMPS on the KNL B0 box, since this hardware is NDA and not-released we can't really ask for help from RedHat. Who can we get help from? >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> >>>>>>>>>>>> Ben >>>>>> >>>>>> _______________________________________________ >>>>>> Intel-nda mailing list >>>>>> >>>>>> Intel-nda(a)lists.jlse.anl.gov >>>>>> https://lists.jlse.anl.gov/mailman/listinfo/intel-nda >>>>> >>>>> _______________________________________________ >>>>> Intel-nda mailing list >>>>> >>>>> Intel-nda(a)lists.jlse.anl.gov >>>>> https://lists.jlse.anl.gov/mailman/listinfo/intel-nda

1 0

Re: [JLSE-Intel-NDA] Kernel Panic on KNL Box
by Allen, Benjamin S. 28 Mar '16

28 Mar '16

Power cycling now. Ben > On Mar 28, 2016, at 1:16 PM, Vitali A. Morozov <morozov(a)anl.gov> wrote: > > Again... > > Interestingly, this same job passed through in configuration of 16 processes 4 threads each and 8 processes 8 thread each. I am trying 8 processes 4 threads each - cannot make it for 3 times in a row, every time dying in a different spot. > > Vitali > > > > On 03/28/2016 01:01 PM, Allen, Benjamin S. wrote: >> I see that. Power cycling now. >> >> Ben >> >>> On Mar 28, 2016, at 1:00 PM, Vitali A. Morozov <morozov(a)anl.gov> wrote: >>> >>> Died again... >>> >>> >>> On 03/28/2016 11:37 AM, Vitali A. Morozov wrote: >>>> I seems to "kernel panic" the box... >>>> >>>> Vitali >>>> >>>> >>>> On 03/28/2016 11:35 AM, Kumaran, Kalyan wrote: >>>>> Looks like our A0 to update the BIOS has/should arrive(d) today. >>>>> >>>>> -Kumar >>>>> >>>>> >>>>> >>>>>> Begin forwarded message: >>>>>> >>>>>> From: "Allen, Benjamin S." <bsallen(a)alcf.anl.gov> >>>>>> Subject: Re: Kernel Panic on KNL Box >>>>>> Date: March 25, 2016 at 6:29:54 PM CDT >>>>>> To: "Fromkin, Russ" <russ.fromkin(a)intel.com> >>>>>> Cc: "Kumaran, Kalyan" <kumaran(a)alcf.anl.gov>, "Kirkendall, Keith G" <keith.g.kirkendall(a)intel.com>, "Arney, Rebecca" <rebecca.arney(a)intel.com>, "Knight, Christopher J." <knight(a)mcs.anl.gov>, "Allison, CM" <cm.allison(a)intel.com> >>>>>> >>>>>> Thanks: https://www.fedex.com/apps/fedextrack/?tracknumbers=6431-3664-8770 >>>>>> >>>>>> Ben >>>>>> >>>>>>> On Mar 25, 2016, at 6:28 PM, Fromkin, Russ <russ.fromkin(a)intel.com> wrote: >>>>>>> >>>>>>> Here is what I have: >>>>>>> >>>>>>> Shipping Memo (Delivery Number) #1303398298 >>>>>>> Shipment Document #7000316160 >>>>>>> Tracking643136648770 >>>>>>> CarrierFM >>>>>>> ETA03/28/2016 10:00 >>>>>>> >>>>>>> -----Original Message----- >>>>>>> From: Allen, Benjamin S. [mailto:[email protected]] >>>>>>> Sent: Friday, March 25, 2016 7:15 PM >>>>>>> To: Fromkin, Russ <russ.fromkin(a)intel.com> >>>>>>> Cc: Kumaran, Kalyan <kumaran(a)alcf.anl.gov>; Kirkendall, Keith G <keith.g.kirkendall(a)intel.com>; Arney, Rebecca <rebecca.arney(a)intel.com>; Knight, Christopher J. <knight(a)mcs.anl.gov>; Allison, CM <cm.allison(a)intel.com> >>>>>>> Subject: Re: Kernel Panic on KNL Box >>>>>>> >>>>>>> Thanks Russ, can you share the tracking number? >>>>>>> >>>>>>> Ben >>>>>>> >>>>>>>> On Mar 25, 2016, at 5:57 PM, Fromkin, Russ <russ.fromkin(a)intel.com> wrote: >>>>>>>> >>>>>>>> Just received the shipping notice. It should arrive on the 28th. >>>>>>>> >>>>>>>> Russ >>>>>>>> >>>>>>>> -----Original Message----- >>>>>>>> From: Kumaran, Kalyan [mailto:[email protected]] >>>>>>>> Sent: Wednesday, March 23, 2016 12:49 PM >>>>>>>> To: Allen, Benjamin S. <bsallen(a)alcf.anl.gov> >>>>>>>> Cc: Fromkin, Russ <russ.fromkin(a)intel.com>; Kirkendall, Keith G <keith.g.kirkendall(a)intel.com>; Arney, Rebecca <rebecca.arney(a)intel.com>; Knight, Christopher J. <knight(a)mcs.anl.gov>; Allison, CM <cm.allison(a)intel.com> >>>>>>>> Subject: Re: Kernel Panic on KNL Box >>>>>>>> >>>>>>>> If it is being shipped to me I have not got any emails indicating a shipment. >>>>>>>> >>>>>>>> -Kumar >>>>>>>> >>>>>>>> >>>>>>>>> On Mar 23, 2016, at 11:45 AM, Allen, Benjamin S. <bsallen(a)alcf.anl.gov> wrote: >>>>>>>>> >>>>>>>>> Hi Russ, >>>>>>>>> >>>>>>>>> I have a handful of vmcores from the panics if there's any interest from the team. We can also provide remote access to Intel staff if needed. >>>>>>>>> >>>>>>>>> Agreed on the concern about the BIOS. Hopefully we'll get the A0 soon to get it upgraded. >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> >>>>>>>>> Ben >>>>>>>>> >>>>>>>>>> On Mar 23, 2016, at 11:41 AM, Fromkin, Russ <russ.fromkin(a)intel.com> wrote: >>>>>>>>>> >>>>>>>>>> Hi Ben, >>>>>>>>>> >>>>>>>>>> We agree and are trying to work the issue and our LAMMPS team agreed it was not the code. >>>>>>>>>> >>>>>>>>>> I am still worried about the B0 inside the system with a bios that was installed for A0. That being said, I do believe a few others with this machine are having some issues. >>>>>>>>>> >>>>>>>>>> I need to read through the handful of emails from today to see if there is anything of value to share. >>>>>>>>>> >>>>>>>>>> Russ >>>>>>>>>> >>>>>>>>>> -----Original Message----- >>>>>>>>>> From: Allen, Benjamin S. [mailto:[email protected]] >>>>>>>>>> Sent: Wednesday, March 23, 2016 12:33 PM >>>>>>>>>> To: Fromkin, Russ <russ.fromkin(a)intel.com> >>>>>>>>>> Cc: Kirkendall, Keith G <keith.g.kirkendall(a)intel.com>; Arney, Rebecca <rebecca.arney(a)intel.com>; Kumaran, Kalyan <kumaran(a)alcf.anl.gov>; Knight, Christopher J. <knight(a)mcs.anl.gov>; Allison, CM <cm.allison(a)intel.com> >>>>>>>>>> Subject: Re: Kernel Panic on KNL Box >>>>>>>>>> >>>>>>>>>> Hi Russ, >>>>>>>>>> >>>>>>>>>> We can get a similar kernel panic by just running a 128 rank mpi_hello_world app, so I don't think this is actually related to LAMPPS. I'm going to hold off doing additional testing and troubleshooting on this until we get the BIOS upgraded. We're also still seeing Machine Check Errors "Generic CACHE Level-2 Generic Error". >>>>>>>>>> >>>>>>>>>> Ben >>>>>>>>>> >>>>>>>>>>> On Mar 22, 2016, at 3:40 PM, Fromkin, Russ <russ.fromkin(a)intel.com> wrote: >>>>>>>>>>> >>>>>>>>>>> Pulling Rebecca onto the string, as I want to make sure we keep any of this work and early science work together. >>>>>>>>>>> >>>>>>>>>>> We will confirm this the next steps shortly. >>>>>>>>>>> >>>>>>>>>>> Russ >>>>>>>>>>> >>>>>>>>>>> -----Original Message----- >>>>>>>>>>> From: Allen, Benjamin S. [mailto:[email protected]] >>>>>>>>>>> Sent: Tuesday, March 22, 2016 3:30 PM >>>>>>>>>>> To: Fromkin, Russ <russ.fromkin(a)intel.com>; Kirkendall, Keith G <keith.g.kirkendall(a)intel.com> >>>>>>>>>>> Cc: Kumaran, Kalyan <kumaran(a)alcf.anl.gov>; Knight, Christopher J. <knight(a)mcs.anl.gov>; Allison, CM <cm.allison(a)intel.com> >>>>>>>>>>> Subject: Kernel Panic on KNL Box >>>>>>>>>>> >>>>>>>>>>> Russ and Keith, >>>>>>>>>>> >>>>>>>>>>> We have a reproducible kernel panic when running LAMMPS on the KNL B0 box, since this hardware is NDA and not-released we can't really ask for help from RedHat. Who can we get help from? >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> >>>>>>>>>>> Ben >>>>> >>>>> >>>>> _______________________________________________ >>>>> Intel-nda mailing list >>>>> >>>>> Intel-nda(a)lists.jlse.anl.gov >>>>> https://lists.jlse.anl.gov/mailman/listinfo/intel-nda >>>> >>>> >>>> _______________________________________________ >>>> Intel-nda mailing list >>>> >>>> Intel-nda(a)lists.jlse.anl.gov >>>> https://lists.jlse.anl.gov/mailman/listinfo/intel-nda >

1 0

Re: [JLSE-Intel-NDA] Fwd: Kernel Panic on KNL Box
by Vitali A. Morozov 28 Mar '16

28 Mar '16

Again... Interestingly, this same job passed through in configuration of 16 processes 4 threads each and 8 processes 8 thread each. I am trying 8 processes 4 threads each - cannot make it for 3 times in a row, every time dying in a different spot. Vitali On 03/28/2016 01:01 PM, Allen, Benjamin S. wrote: > I see that. Power cycling now. > > Ben > >> On Mar 28, 2016, at 1:00 PM, Vitali A. Morozov <morozov(a)anl.gov> wrote: >> >> Died again... >> >> >> On 03/28/2016 11:37 AM, Vitali A. Morozov wrote: >>> I seems to "kernel panic" the box... >>> >>> Vitali >>> >>> >>> On 03/28/2016 11:35 AM, Kumaran, Kalyan wrote: >>>> Looks like our A0 to update the BIOS has/should arrive(d) today. >>>> >>>> -Kumar >>>> >>>> >>>> >>>>> Begin forwarded message: >>>>> >>>>> From: "Allen, Benjamin S." <bsallen(a)alcf.anl.gov> >>>>> Subject: Re: Kernel Panic on KNL Box >>>>> Date: March 25, 2016 at 6:29:54 PM CDT >>>>> To: "Fromkin, Russ" <russ.fromkin(a)intel.com> >>>>> Cc: "Kumaran, Kalyan" <kumaran(a)alcf.anl.gov>, "Kirkendall, Keith G" <keith.g.kirkendall(a)intel.com>, "Arney, Rebecca" <rebecca.arney(a)intel.com>, "Knight, Christopher J." <knight(a)mcs.anl.gov>, "Allison, CM" <cm.allison(a)intel.com> >>>>> >>>>> Thanks: https://www.fedex.com/apps/fedextrack/?tracknumbers=6431-3664-8770 >>>>> >>>>> Ben >>>>> >>>>>> On Mar 25, 2016, at 6:28 PM, Fromkin, Russ <russ.fromkin(a)intel.com> wrote: >>>>>> >>>>>> Here is what I have: >>>>>> >>>>>> Shipping Memo (Delivery Number) #1303398298 >>>>>> Shipment Document #7000316160 >>>>>> Tracking643136648770 >>>>>> CarrierFM >>>>>> ETA03/28/2016 10:00 >>>>>> >>>>>> -----Original Message----- >>>>>> From: Allen, Benjamin S. [mailto:[email protected]] >>>>>> Sent: Friday, March 25, 2016 7:15 PM >>>>>> To: Fromkin, Russ <russ.fromkin(a)intel.com> >>>>>> Cc: Kumaran, Kalyan <kumaran(a)alcf.anl.gov>; Kirkendall, Keith G <keith.g.kirkendall(a)intel.com>; Arney, Rebecca <rebecca.arney(a)intel.com>; Knight, Christopher J. <knight(a)mcs.anl.gov>; Allison, CM <cm.allison(a)intel.com> >>>>>> Subject: Re: Kernel Panic on KNL Box >>>>>> >>>>>> Thanks Russ, can you share the tracking number? >>>>>> >>>>>> Ben >>>>>> >>>>>>> On Mar 25, 2016, at 5:57 PM, Fromkin, Russ <russ.fromkin(a)intel.com> wrote: >>>>>>> >>>>>>> Just received the shipping notice. It should arrive on the 28th. >>>>>>> >>>>>>> Russ >>>>>>> >>>>>>> -----Original Message----- >>>>>>> From: Kumaran, Kalyan [mailto:[email protected]] >>>>>>> Sent: Wednesday, March 23, 2016 12:49 PM >>>>>>> To: Allen, Benjamin S. <bsallen(a)alcf.anl.gov> >>>>>>> Cc: Fromkin, Russ <russ.fromkin(a)intel.com>; Kirkendall, Keith G <keith.g.kirkendall(a)intel.com>; Arney, Rebecca <rebecca.arney(a)intel.com>; Knight, Christopher J. <knight(a)mcs.anl.gov>; Allison, CM <cm.allison(a)intel.com> >>>>>>> Subject: Re: Kernel Panic on KNL Box >>>>>>> >>>>>>> If it is being shipped to me I have not got any emails indicating a shipment. >>>>>>> >>>>>>> -Kumar >>>>>>> >>>>>>> >>>>>>>> On Mar 23, 2016, at 11:45 AM, Allen, Benjamin S. <bsallen(a)alcf.anl.gov> wrote: >>>>>>>> >>>>>>>> Hi Russ, >>>>>>>> >>>>>>>> I have a handful of vmcores from the panics if there's any interest from the team. We can also provide remote access to Intel staff if needed. >>>>>>>> >>>>>>>> Agreed on the concern about the BIOS. Hopefully we'll get the A0 soon to get it upgraded. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> >>>>>>>> Ben >>>>>>>> >>>>>>>>> On Mar 23, 2016, at 11:41 AM, Fromkin, Russ <russ.fromkin(a)intel.com> wrote: >>>>>>>>> >>>>>>>>> Hi Ben, >>>>>>>>> >>>>>>>>> We agree and are trying to work the issue and our LAMMPS team agreed it was not the code. >>>>>>>>> >>>>>>>>> I am still worried about the B0 inside the system with a bios that was installed for A0. That being said, I do believe a few others with this machine are having some issues. >>>>>>>>> >>>>>>>>> I need to read through the handful of emails from today to see if there is anything of value to share. >>>>>>>>> >>>>>>>>> Russ >>>>>>>>> >>>>>>>>> -----Original Message----- >>>>>>>>> From: Allen, Benjamin S. [mailto:[email protected]] >>>>>>>>> Sent: Wednesday, March 23, 2016 12:33 PM >>>>>>>>> To: Fromkin, Russ <russ.fromkin(a)intel.com> >>>>>>>>> Cc: Kirkendall, Keith G <keith.g.kirkendall(a)intel.com>; Arney, Rebecca <rebecca.arney(a)intel.com>; Kumaran, Kalyan <kumaran(a)alcf.anl.gov>; Knight, Christopher J. <knight(a)mcs.anl.gov>; Allison, CM <cm.allison(a)intel.com> >>>>>>>>> Subject: Re: Kernel Panic on KNL Box >>>>>>>>> >>>>>>>>> Hi Russ, >>>>>>>>> >>>>>>>>> We can get a similar kernel panic by just running a 128 rank mpi_hello_world app, so I don't think this is actually related to LAMPPS. I'm going to hold off doing additional testing and troubleshooting on this until we get the BIOS upgraded. We're also still seeing Machine Check Errors "Generic CACHE Level-2 Generic Error". >>>>>>>>> >>>>>>>>> Ben >>>>>>>>> >>>>>>>>>> On Mar 22, 2016, at 3:40 PM, Fromkin, Russ <russ.fromkin(a)intel.com> wrote: >>>>>>>>>> >>>>>>>>>> Pulling Rebecca onto the string, as I want to make sure we keep any of this work and early science work together. >>>>>>>>>> >>>>>>>>>> We will confirm this the next steps shortly. >>>>>>>>>> >>>>>>>>>> Russ >>>>>>>>>> >>>>>>>>>> -----Original Message----- >>>>>>>>>> From: Allen, Benjamin S. [mailto:[email protected]] >>>>>>>>>> Sent: Tuesday, March 22, 2016 3:30 PM >>>>>>>>>> To: Fromkin, Russ <russ.fromkin(a)intel.com>; Kirkendall, Keith G <keith.g.kirkendall(a)intel.com> >>>>>>>>>> Cc: Kumaran, Kalyan <kumaran(a)alcf.anl.gov>; Knight, Christopher J. <knight(a)mcs.anl.gov>; Allison, CM <cm.allison(a)intel.com> >>>>>>>>>> Subject: Kernel Panic on KNL Box >>>>>>>>>> >>>>>>>>>> Russ and Keith, >>>>>>>>>> >>>>>>>>>> We have a reproducible kernel panic when running LAMMPS on the KNL B0 box, since this hardware is NDA and not-released we can't really ask for help from RedHat. Who can we get help from? >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> >>>>>>>>>> Ben >>>> >>>> >>>> _______________________________________________ >>>> Intel-nda mailing list >>>> >>>> Intel-nda(a)lists.jlse.anl.gov >>>> https://lists.jlse.anl.gov/mailman/listinfo/intel-nda >>> >>> >>> _______________________________________________ >>> Intel-nda mailing list >>> >>> Intel-nda(a)lists.jlse.anl.gov >>> https://lists.jlse.anl.gov/mailman/listinfo/intel-nda

1 0

Re: [JLSE-Intel-NDA] Fwd: Kernel Panic on KNL Box
by Allen, Benjamin S. 28 Mar '16

28 Mar '16

I see that. Power cycling now. Ben > On Mar 28, 2016, at 1:00 PM, Vitali A. Morozov <morozov(a)anl.gov> wrote: > > Died again... > > > On 03/28/2016 11:37 AM, Vitali A. Morozov wrote: >> I seems to "kernel panic" the box... >> >> Vitali >> >> >> On 03/28/2016 11:35 AM, Kumaran, Kalyan wrote: >>> Looks like our A0 to update the BIOS has/should arrive(d) today. >>> >>> -Kumar >>> >>> >>> >>>> Begin forwarded message: >>>> >>>> From: "Allen, Benjamin S." <bsallen(a)alcf.anl.gov> >>>> Subject: Re: Kernel Panic on KNL Box >>>> Date: March 25, 2016 at 6:29:54 PM CDT >>>> To: "Fromkin, Russ" <russ.fromkin(a)intel.com> >>>> Cc: "Kumaran, Kalyan" <kumaran(a)alcf.anl.gov>, "Kirkendall, Keith G" <keith.g.kirkendall(a)intel.com>, "Arney, Rebecca" <rebecca.arney(a)intel.com>, "Knight, Christopher J." <knight(a)mcs.anl.gov>, "Allison, CM" <cm.allison(a)intel.com> >>>> >>>> Thanks: https://www.fedex.com/apps/fedextrack/?tracknumbers=6431-3664-8770 >>>> >>>> Ben >>>> >>>>> On Mar 25, 2016, at 6:28 PM, Fromkin, Russ <russ.fromkin(a)intel.com> wrote: >>>>> >>>>> Here is what I have: >>>>> >>>>> Shipping Memo (Delivery Number) #1303398298 >>>>> Shipment Document #7000316160 >>>>> Tracking643136648770 >>>>> CarrierFM >>>>> ETA03/28/2016 10:00 >>>>> >>>>> -----Original Message----- >>>>> From: Allen, Benjamin S. [mailto:[email protected]] >>>>> Sent: Friday, March 25, 2016 7:15 PM >>>>> To: Fromkin, Russ <russ.fromkin(a)intel.com> >>>>> Cc: Kumaran, Kalyan <kumaran(a)alcf.anl.gov>; Kirkendall, Keith G <keith.g.kirkendall(a)intel.com>; Arney, Rebecca <rebecca.arney(a)intel.com>; Knight, Christopher J. <knight(a)mcs.anl.gov>; Allison, CM <cm.allison(a)intel.com> >>>>> Subject: Re: Kernel Panic on KNL Box >>>>> >>>>> Thanks Russ, can you share the tracking number? >>>>> >>>>> Ben >>>>> >>>>>> On Mar 25, 2016, at 5:57 PM, Fromkin, Russ <russ.fromkin(a)intel.com> wrote: >>>>>> >>>>>> Just received the shipping notice. It should arrive on the 28th. >>>>>> >>>>>> Russ >>>>>> >>>>>> -----Original Message----- >>>>>> From: Kumaran, Kalyan [mailto:[email protected]] >>>>>> Sent: Wednesday, March 23, 2016 12:49 PM >>>>>> To: Allen, Benjamin S. <bsallen(a)alcf.anl.gov> >>>>>> Cc: Fromkin, Russ <russ.fromkin(a)intel.com>; Kirkendall, Keith G <keith.g.kirkendall(a)intel.com>; Arney, Rebecca <rebecca.arney(a)intel.com>; Knight, Christopher J. <knight(a)mcs.anl.gov>; Allison, CM <cm.allison(a)intel.com> >>>>>> Subject: Re: Kernel Panic on KNL Box >>>>>> >>>>>> If it is being shipped to me I have not got any emails indicating a shipment. >>>>>> >>>>>> -Kumar >>>>>> >>>>>> >>>>>>> On Mar 23, 2016, at 11:45 AM, Allen, Benjamin S. <bsallen(a)alcf.anl.gov> wrote: >>>>>>> >>>>>>> Hi Russ, >>>>>>> >>>>>>> I have a handful of vmcores from the panics if there's any interest from the team. We can also provide remote access to Intel staff if needed. >>>>>>> >>>>>>> Agreed on the concern about the BIOS. Hopefully we'll get the A0 soon to get it upgraded. >>>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> Ben >>>>>>> >>>>>>>> On Mar 23, 2016, at 11:41 AM, Fromkin, Russ <russ.fromkin(a)intel.com> wrote: >>>>>>>> >>>>>>>> Hi Ben, >>>>>>>> >>>>>>>> We agree and are trying to work the issue and our LAMMPS team agreed it was not the code. >>>>>>>> >>>>>>>> I am still worried about the B0 inside the system with a bios that was installed for A0. That being said, I do believe a few others with this machine are having some issues. >>>>>>>> >>>>>>>> I need to read through the handful of emails from today to see if there is anything of value to share. >>>>>>>> >>>>>>>> Russ >>>>>>>> >>>>>>>> -----Original Message----- >>>>>>>> From: Allen, Benjamin S. [mailto:[email protected]] >>>>>>>> Sent: Wednesday, March 23, 2016 12:33 PM >>>>>>>> To: Fromkin, Russ <russ.fromkin(a)intel.com> >>>>>>>> Cc: Kirkendall, Keith G <keith.g.kirkendall(a)intel.com>; Arney, Rebecca <rebecca.arney(a)intel.com>; Kumaran, Kalyan <kumaran(a)alcf.anl.gov>; Knight, Christopher J. <knight(a)mcs.anl.gov>; Allison, CM <cm.allison(a)intel.com> >>>>>>>> Subject: Re: Kernel Panic on KNL Box >>>>>>>> >>>>>>>> Hi Russ, >>>>>>>> >>>>>>>> We can get a similar kernel panic by just running a 128 rank mpi_hello_world app, so I don't think this is actually related to LAMPPS. I'm going to hold off doing additional testing and troubleshooting on this until we get the BIOS upgraded. We're also still seeing Machine Check Errors "Generic CACHE Level-2 Generic Error". >>>>>>>> >>>>>>>> Ben >>>>>>>> >>>>>>>>> On Mar 22, 2016, at 3:40 PM, Fromkin, Russ <russ.fromkin(a)intel.com> wrote: >>>>>>>>> >>>>>>>>> Pulling Rebecca onto the string, as I want to make sure we keep any of this work and early science work together. >>>>>>>>> >>>>>>>>> We will confirm this the next steps shortly. >>>>>>>>> >>>>>>>>> Russ >>>>>>>>> >>>>>>>>> -----Original Message----- >>>>>>>>> From: Allen, Benjamin S. [mailto:[email protected]] >>>>>>>>> Sent: Tuesday, March 22, 2016 3:30 PM >>>>>>>>> To: Fromkin, Russ <russ.fromkin(a)intel.com>; Kirkendall, Keith G <keith.g.kirkendall(a)intel.com> >>>>>>>>> Cc: Kumaran, Kalyan <kumaran(a)alcf.anl.gov>; Knight, Christopher J. <knight(a)mcs.anl.gov>; Allison, CM <cm.allison(a)intel.com> >>>>>>>>> Subject: Kernel Panic on KNL Box >>>>>>>>> >>>>>>>>> Russ and Keith, >>>>>>>>> >>>>>>>>> We have a reproducible kernel panic when running LAMMPS on the KNL B0 box, since this hardware is NDA and not-released we can't really ask for help from RedHat. Who can we get help from? >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> >>>>>>>>> Ben >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >>> >>> >>> _______________________________________________ >>> Intel-nda mailing list >>> >>> Intel-nda(a)lists.jlse.anl.gov >>> https://lists.jlse.anl.gov/mailman/listinfo/intel-nda >> >> >> >> _______________________________________________ >> Intel-nda mailing list >> >> Intel-nda(a)lists.jlse.anl.gov >> https://lists.jlse.anl.gov/mailman/listinfo/intel-nda >

1 0

Re: [JLSE-Intel-NDA] Fwd: Kernel Panic on KNL Box
by Vitali A. Morozov 28 Mar '16

28 Mar '16

Died again... On 03/28/2016 11:37 AM, Vitali A. Morozov wrote: > I seems to "kernel panic" the box... > > Vitali > > > On 03/28/2016 11:35 AM, Kumaran, Kalyan wrote: >> Looks like our A0 to update the BIOS has/should arrive(d) today. >> >> -Kumar >> >> >> >>> Begin forwarded message: >>> >>> *From: *"Allen, Benjamin S." <bsallen(a)alcf.anl.gov >>> <mailto:[email protected]>> >>> *Subject: **Re: Kernel Panic on KNL Box* >>> *Date: *March 25, 2016 at 6:29:54 PM CDT >>> *To: *"Fromkin, Russ" <russ.fromkin(a)intel.com >>> <mailto:[email protected]>> >>> *Cc: *"Kumaran, Kalyan" <kumaran(a)alcf.anl.gov >>> <mailto:[email protected]>>, "Kirkendall, Keith G" >>> <keith.g.kirkendall(a)intel.com >>> <mailto:[email protected]>>, "Arney, Rebecca" >>> <rebecca.arney(a)intel.com <mailto:[email protected]>>, "Knight, >>> Christopher J." <knight(a)mcs.anl.gov <mailto:[email protected]>>, >>> "Allison, CM" <cm.allison(a)intel.com <mailto:[email protected]>> >>> >>> Thanks: >>> https://www.fedex.com/apps/fedextrack/?tracknumbers=6431-3664-8770 >>> >>> Ben >>> >>>> On Mar 25, 2016, at 6:28 PM, Fromkin, Russ <russ.fromkin(a)intel.com >>>> <mailto:[email protected]>> wrote: >>>> >>>> Here is what I have: >>>> >>>> Shipping Memo (Delivery Number) #1303398298 >>>> Shipment Document #7000316160 >>>> Tracking643136648770 >>>> CarrierFM >>>> ETA03/28/2016 10:00 >>>> >>>> -----Original Message----- >>>> From: Allen, Benjamin S. [mailto:[email protected]] >>>> Sent: Friday, March 25, 2016 7:15 PM >>>> To: Fromkin, Russ <russ.fromkin(a)intel.com >>>> <mailto:[email protected]>> >>>> Cc: Kumaran, Kalyan <kumaran(a)alcf.anl.gov >>>> <mailto:[email protected]>>; Kirkendall, Keith G >>>> <keith.g.kirkendall(a)intel.com >>>> <mailto:[email protected]>>; Arney, Rebecca >>>> <rebecca.arney(a)intel.com <mailto:[email protected]>>; Knight, >>>> Christopher J. <knight(a)mcs.anl.gov <mailto:[email protected]>>; >>>> Allison, CM <cm.allison(a)intel.com <mailto:[email protected]>> >>>> Subject: Re: Kernel Panic on KNL Box >>>> >>>> Thanks Russ, can you share the tracking number? >>>> >>>> Ben >>>> >>>>> On Mar 25, 2016, at 5:57 PM, Fromkin, Russ <russ.fromkin(a)intel.com >>>>> <mailto:[email protected]>> wrote: >>>>> >>>>> Just received the shipping notice. It should arrive on the 28th. >>>>> >>>>> Russ >>>>> >>>>> -----Original Message----- >>>>> From: Kumaran, Kalyan [mailto:[email protected]] >>>>> Sent: Wednesday, March 23, 2016 12:49 PM >>>>> To: Allen, Benjamin S. <bsallen(a)alcf.anl.gov >>>>> <mailto:[email protected]>> >>>>> Cc: Fromkin, Russ <russ.fromkin(a)intel.com >>>>> <mailto:[email protected]>>; Kirkendall, Keith G >>>>> <keith.g.kirkendall(a)intel.com >>>>> <mailto:[email protected]>>; Arney, Rebecca >>>>> <rebecca.arney(a)intel.com <mailto:[email protected]>>; >>>>> Knight, Christopher J. <knight(a)mcs.anl.gov >>>>> <mailto:[email protected]>>; Allison, CM <cm.allison(a)intel.com >>>>> <mailto:[email protected]>> >>>>> Subject: Re: Kernel Panic on KNL Box >>>>> >>>>> If it is being shipped to me I have not got any emails indicating >>>>> a shipment. >>>>> >>>>> -Kumar >>>>> >>>>> >>>>>> On Mar 23, 2016, at 11:45 AM, Allen, Benjamin S. >>>>>> <bsallen(a)alcf.anl.gov <mailto:[email protected]>> wrote: >>>>>> >>>>>> Hi Russ, >>>>>> >>>>>> I have a handful of vmcores from the panics if there's any >>>>>> interest from the team. We can also provide remote access to >>>>>> Intel staff if needed. >>>>>> >>>>>> Agreed on the concern about the BIOS. Hopefully we'll get the A0 >>>>>> soon to get it upgraded. >>>>>> >>>>>> Thanks, >>>>>> >>>>>> Ben >>>>>> >>>>>>> On Mar 23, 2016, at 11:41 AM, Fromkin, Russ >>>>>>> <russ.fromkin(a)intel.com <mailto:[email protected]>> wrote: >>>>>>> >>>>>>> Hi Ben, >>>>>>> >>>>>>> We agree and are trying to work the issue and our LAMMPS team >>>>>>> agreed it was not the code. >>>>>>> >>>>>>> I am still worried about the B0 inside the system with a bios >>>>>>> that was installed for A0. That being said, I do believe a few >>>>>>> others with this machine are having some issues. >>>>>>> >>>>>>> I need to read through the handful of emails from today to see >>>>>>> if there is anything of value to share. >>>>>>> >>>>>>> Russ >>>>>>> >>>>>>> -----Original Message----- >>>>>>> From: Allen, Benjamin S. [mailto:[email protected]] >>>>>>> Sent: Wednesday, March 23, 2016 12:33 PM >>>>>>> To: Fromkin, Russ <russ.fromkin(a)intel.com >>>>>>> <mailto:[email protected]>> >>>>>>> Cc: Kirkendall, Keith G <keith.g.kirkendall(a)intel.com >>>>>>> <mailto:[email protected]>>; Arney, Rebecca >>>>>>> <rebecca.arney(a)intel.com <mailto:[email protected]>>; >>>>>>> Kumaran, Kalyan <kumaran(a)alcf.anl.gov >>>>>>> <mailto:[email protected]>>; Knight, Christopher J. >>>>>>> <knight(a)mcs.anl.gov <mailto:[email protected]>>; Allison, CM >>>>>>> <cm.allison(a)intel.com <mailto:[email protected]>> >>>>>>> Subject: Re: Kernel Panic on KNL Box >>>>>>> >>>>>>> Hi Russ, >>>>>>> >>>>>>> We can get a similar kernel panic by just running a 128 rank >>>>>>> mpi_hello_world app, so I don't think this is actually related >>>>>>> to LAMPPS. I'm going to hold off doing additional testing and >>>>>>> troubleshooting on this until we get the BIOS upgraded. We're >>>>>>> also still seeing Machine Check Errors "Generic CACHE Level-2 >>>>>>> Generic Error". >>>>>>> >>>>>>> Ben >>>>>>> >>>>>>>> On Mar 22, 2016, at 3:40 PM, Fromkin, Russ >>>>>>>> <russ.fromkin(a)intel.com <mailto:[email protected]>> wrote: >>>>>>>> >>>>>>>> Pulling Rebecca onto the string, as I want to make sure we keep >>>>>>>> any of this work and early science work together. >>>>>>>> >>>>>>>> We will confirm this the next steps shortly. >>>>>>>> >>>>>>>> Russ >>>>>>>> >>>>>>>> -----Original Message----- >>>>>>>> From: Allen, Benjamin S. [mailto:[email protected]] >>>>>>>> Sent: Tuesday, March 22, 2016 3:30 PM >>>>>>>> To: Fromkin, Russ <russ.fromkin(a)intel.com >>>>>>>> <mailto:[email protected]>>; Kirkendall, Keith G >>>>>>>> <keith.g.kirkendall(a)intel.com >>>>>>>> <mailto:[email protected]>> >>>>>>>> Cc: Kumaran, Kalyan <kumaran(a)alcf.anl.gov >>>>>>>> <mailto:[email protected]>>; Knight, Christopher J. >>>>>>>> <knight(a)mcs.anl.gov <mailto:[email protected]>>; Allison, CM >>>>>>>> <cm.allison(a)intel.com <mailto:[email protected]>> >>>>>>>> Subject: Kernel Panic on KNL Box >>>>>>>> >>>>>>>> Russ and Keith, >>>>>>>> >>>>>>>> We have a reproducible kernel panic when running LAMMPS on the >>>>>>>> KNL B0 box, since this hardware is NDA and not-released we >>>>>>>> can't really ask for help from RedHat. Who can we get help from? >>>>>>>> >>>>>>>> Thanks, >>>>>>>> >>>>>>>> Ben >>>>>>> >>>>>> >>>>> >>>> >>> >> >> >> >> _______________________________________________ >> Intel-nda mailing list >> Intel-nda(a)lists.jlse.anl.gov >> https://lists.jlse.anl.gov/mailman/listinfo/intel-nda > > > > _______________________________________________ > Intel-nda mailing list > Intel-nda(a)lists.jlse.anl.gov > https://lists.jlse.anl.gov/mailman/listinfo/intel-nda

1 0

Re: [JLSE-Intel-NDA] Fwd: Kernel Panic on KNL Box
by Vitali A. Morozov 28 Mar '16

28 Mar '16

I seems to "kernel panic" the box... Vitali On 03/28/2016 11:35 AM, Kumaran, Kalyan wrote: > Looks like our A0 to update the BIOS has/should arrive(d) today. > > -Kumar > > > >> Begin forwarded message: >> >> *From: *"Allen, Benjamin S." <bsallen(a)alcf.anl.gov >> <mailto:[email protected]>> >> *Subject: **Re: Kernel Panic on KNL Box* >> *Date: *March 25, 2016 at 6:29:54 PM CDT >> *To: *"Fromkin, Russ" <russ.fromkin(a)intel.com >> <mailto:[email protected]>> >> *Cc: *"Kumaran, Kalyan" <kumaran(a)alcf.anl.gov >> <mailto:[email protected]>>, "Kirkendall, Keith G" >> <keith.g.kirkendall(a)intel.com <mailto:[email protected]>>, >> "Arney, Rebecca" <rebecca.arney(a)intel.com >> <mailto:[email protected]>>, "Knight, Christopher J." >> <knight(a)mcs.anl.gov <mailto:[email protected]>>, "Allison, CM" >> <cm.allison(a)intel.com <mailto:[email protected]>> >> >> Thanks: >> https://www.fedex.com/apps/fedextrack/?tracknumbers=6431-3664-8770 >> >> Ben >> >>> On Mar 25, 2016, at 6:28 PM, Fromkin, Russ <russ.fromkin(a)intel.com >>> <mailto:[email protected]>> wrote: >>> >>> Here is what I have: >>> >>> Shipping Memo (Delivery Number) #1303398298 >>> Shipment Document #7000316160 >>> Tracking643136648770 >>> CarrierFM >>> ETA03/28/2016 10:00 >>> >>> -----Original Message----- >>> From: Allen, Benjamin S. [mailto:[email protected]] >>> Sent: Friday, March 25, 2016 7:15 PM >>> To: Fromkin, Russ <russ.fromkin(a)intel.com >>> <mailto:[email protected]>> >>> Cc: Kumaran, Kalyan <kumaran(a)alcf.anl.gov >>> <mailto:[email protected]>>; Kirkendall, Keith G >>> <keith.g.kirkendall(a)intel.com >>> <mailto:[email protected]>>; Arney, Rebecca >>> <rebecca.arney(a)intel.com <mailto:[email protected]>>; Knight, >>> Christopher J. <knight(a)mcs.anl.gov <mailto:[email protected]>>; >>> Allison, CM <cm.allison(a)intel.com <mailto:[email protected]>> >>> Subject: Re: Kernel Panic on KNL Box >>> >>> Thanks Russ, can you share the tracking number? >>> >>> Ben >>> >>>> On Mar 25, 2016, at 5:57 PM, Fromkin, Russ <russ.fromkin(a)intel.com >>>> <mailto:[email protected]>> wrote: >>>> >>>> Just received the shipping notice. It should arrive on the 28th. >>>> >>>> Russ >>>> >>>> -----Original Message----- >>>> From: Kumaran, Kalyan [mailto:[email protected]] >>>> Sent: Wednesday, March 23, 2016 12:49 PM >>>> To: Allen, Benjamin S. <bsallen(a)alcf.anl.gov >>>> <mailto:[email protected]>> >>>> Cc: Fromkin, Russ <russ.fromkin(a)intel.com >>>> <mailto:[email protected]>>; Kirkendall, Keith G >>>> <keith.g.kirkendall(a)intel.com >>>> <mailto:[email protected]>>; Arney, Rebecca >>>> <rebecca.arney(a)intel.com <mailto:[email protected]>>; Knight, >>>> Christopher J. <knight(a)mcs.anl.gov <mailto:[email protected]>>; >>>> Allison, CM <cm.allison(a)intel.com <mailto:[email protected]>> >>>> Subject: Re: Kernel Panic on KNL Box >>>> >>>> If it is being shipped to me I have not got any emails indicating a >>>> shipment. >>>> >>>> -Kumar >>>> >>>> >>>>> On Mar 23, 2016, at 11:45 AM, Allen, Benjamin S. >>>>> <bsallen(a)alcf.anl.gov <mailto:[email protected]>> wrote: >>>>> >>>>> Hi Russ, >>>>> >>>>> I have a handful of vmcores from the panics if there's any >>>>> interest from the team. We can also provide remote access to Intel >>>>> staff if needed. >>>>> >>>>> Agreed on the concern about the BIOS. Hopefully we'll get the A0 >>>>> soon to get it upgraded. >>>>> >>>>> Thanks, >>>>> >>>>> Ben >>>>> >>>>>> On Mar 23, 2016, at 11:41 AM, Fromkin, Russ >>>>>> <russ.fromkin(a)intel.com <mailto:[email protected]>> wrote: >>>>>> >>>>>> Hi Ben, >>>>>> >>>>>> We agree and are trying to work the issue and our LAMMPS team >>>>>> agreed it was not the code. >>>>>> >>>>>> I am still worried about the B0 inside the system with a bios >>>>>> that was installed for A0. That being said, I do believe a few >>>>>> others with this machine are having some issues. >>>>>> >>>>>> I need to read through the handful of emails from today to see if >>>>>> there is anything of value to share. >>>>>> >>>>>> Russ >>>>>> >>>>>> -----Original Message----- >>>>>> From: Allen, Benjamin S. [mailto:[email protected]] >>>>>> Sent: Wednesday, March 23, 2016 12:33 PM >>>>>> To: Fromkin, Russ <russ.fromkin(a)intel.com >>>>>> <mailto:[email protected]>> >>>>>> Cc: Kirkendall, Keith G <keith.g.kirkendall(a)intel.com >>>>>> <mailto:[email protected]>>; Arney, Rebecca >>>>>> <rebecca.arney(a)intel.com <mailto:[email protected]>>; >>>>>> Kumaran, Kalyan <kumaran(a)alcf.anl.gov >>>>>> <mailto:[email protected]>>; Knight, Christopher J. >>>>>> <knight(a)mcs.anl.gov <mailto:[email protected]>>; Allison, CM >>>>>> <cm.allison(a)intel.com <mailto:[email protected]>> >>>>>> Subject: Re: Kernel Panic on KNL Box >>>>>> >>>>>> Hi Russ, >>>>>> >>>>>> We can get a similar kernel panic by just running a 128 rank >>>>>> mpi_hello_world app, so I don't think this is actually related to >>>>>> LAMPPS. I'm going to hold off doing additional testing and >>>>>> troubleshooting on this until we get the BIOS upgraded. We're >>>>>> also still seeing Machine Check Errors "Generic CACHE Level-2 >>>>>> Generic Error". >>>>>> >>>>>> Ben >>>>>> >>>>>>> On Mar 22, 2016, at 3:40 PM, Fromkin, Russ >>>>>>> <russ.fromkin(a)intel.com <mailto:[email protected]>> wrote: >>>>>>> >>>>>>> Pulling Rebecca onto the string, as I want to make sure we keep >>>>>>> any of this work and early science work together. >>>>>>> >>>>>>> We will confirm this the next steps shortly. >>>>>>> >>>>>>> Russ >>>>>>> >>>>>>> -----Original Message----- >>>>>>> From: Allen, Benjamin S. [mailto:[email protected]] >>>>>>> Sent: Tuesday, March 22, 2016 3:30 PM >>>>>>> To: Fromkin, Russ <russ.fromkin(a)intel.com >>>>>>> <mailto:[email protected]>>; Kirkendall, Keith G >>>>>>> <keith.g.kirkendall(a)intel.com <mailto:[email protected]>> >>>>>>> Cc: Kumaran, Kalyan <kumaran(a)alcf.anl.gov >>>>>>> <mailto:[email protected]>>; Knight, Christopher J. >>>>>>> <knight(a)mcs.anl.gov <mailto:[email protected]>>; Allison, CM >>>>>>> <cm.allison(a)intel.com <mailto:[email protected]>> >>>>>>> Subject: Kernel Panic on KNL Box >>>>>>> >>>>>>> Russ and Keith, >>>>>>> >>>>>>> We have a reproducible kernel panic when running LAMMPS on the >>>>>>> KNL B0 box, since this hardware is NDA and not-released we can't >>>>>>> really ask for help from RedHat. Who can we get help from? >>>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> Ben >>>>>> >>>>> >>>> >>> >> > > > > _______________________________________________ > Intel-nda mailing list > Intel-nda(a)lists.jlse.anl.gov > https://lists.jlse.anl.gov/mailman/listinfo/intel-nda

1 0

[JLSE-Intel-NDA] Fwd: Kernel Panic on KNL Box
by Kumaran, Kalyan 28 Mar '16

28 Mar '16

Looks like our A0 to update the BIOS has/should arrive(d) today. -Kumar Begin forwarded message: From: "Allen, Benjamin S." <bsallen(a)alcf.anl.gov<mailto:[email protected]>> Subject: Re: Kernel Panic on KNL Box Date: March 25, 2016 at 6:29:54 PM CDT To: "Fromkin, Russ" <russ.fromkin(a)intel.com<mailto:[email protected]>> Cc: "Kumaran, Kalyan" <kumaran(a)alcf.anl.gov<mailto:[email protected]>>, "Kirkendall, Keith G" <keith.g.kirkendall(a)intel.com<mailto:[email protected]>>, "Arney, Rebecca" <rebecca.arney(a)intel.com<mailto:[email protected]>>, "Knight, Christopher J." <knight(a)mcs.anl.gov<mailto:[email protected]>>, "Allison, CM" <cm.allison(a)intel.com<mailto:[email protected]>> Thanks: https://www.fedex.com/apps/fedextrack/?tracknumbers=6431-3664-8770 Ben On Mar 25, 2016, at 6:28 PM, Fromkin, Russ <russ.fromkin(a)intel.com<mailto:[email protected]>> wrote: Here is what I have: Shipping Memo (Delivery Number) #1303398298 Shipment Document #7000316160 Tracking643136648770 CarrierFM ETA03/28/2016 10:00 -----Original Message----- From: Allen, Benjamin S. [mailto:[email protected]] Sent: Friday, March 25, 2016 7:15 PM To: Fromkin, Russ <russ.fromkin(a)intel.com<mailto:[email protected]>> Cc: Kumaran, Kalyan <kumaran(a)alcf.anl.gov<mailto:[email protected]>>; Kirkendall, Keith G <keith.g.kirkendall(a)intel.com<mailto:[email protected]>>; Arney, Rebecca <rebecca.arney(a)intel.com<mailto:[email protected]>>; Knight, Christopher J. <knight(a)mcs.anl.gov<mailto:[email protected]>>; Allison, CM <cm.allison(a)intel.com<mailto:[email protected]>> Subject: Re: Kernel Panic on KNL Box Thanks Russ, can you share the tracking number? Ben On Mar 25, 2016, at 5:57 PM, Fromkin, Russ <russ.fromkin(a)intel.com<mailto:[email protected]>> wrote: Just received the shipping notice. It should arrive on the 28th. Russ -----Original Message----- From: Kumaran, Kalyan [mailto:[email protected]] Sent: Wednesday, March 23, 2016 12:49 PM To: Allen, Benjamin S. <bsallen(a)alcf.anl.gov<mailto:[email protected]>> Cc: Fromkin, Russ <russ.fromkin(a)intel.com<mailto:[email protected]>>; Kirkendall, Keith G <keith.g.kirkendall(a)intel.com<mailto:[email protected]>>; Arney, Rebecca <rebecca.arney(a)intel.com<mailto:[email protected]>>; Knight, Christopher J. <knight(a)mcs.anl.gov<mailto:[email protected]>>; Allison, CM <cm.allison(a)intel.com<mailto:[email protected]>> Subject: Re: Kernel Panic on KNL Box If it is being shipped to me I have not got any emails indicating a shipment. -Kumar On Mar 23, 2016, at 11:45 AM, Allen, Benjamin S. <bsallen(a)alcf.anl.gov<mailto:[email protected]>> wrote: Hi Russ, I have a handful of vmcores from the panics if there's any interest from the team. We can also provide remote access to Intel staff if needed. Agreed on the concern about the BIOS. Hopefully we'll get the A0 soon to get it upgraded. Thanks, Ben On Mar 23, 2016, at 11:41 AM, Fromkin, Russ <russ.fromkin(a)intel.com<mailto:[email protected]>> wrote: Hi Ben, We agree and are trying to work the issue and our LAMMPS team agreed it was not the code. I am still worried about the B0 inside the system with a bios that was installed for A0. That being said, I do believe a few others with this machine are having some issues. I need to read through the handful of emails from today to see if there is anything of value to share. Russ -----Original Message----- From: Allen, Benjamin S. [mailto:[email protected]] Sent: Wednesday, March 23, 2016 12:33 PM To: Fromkin, Russ <russ.fromkin(a)intel.com<mailto:[email protected]>> Cc: Kirkendall, Keith G <keith.g.kirkendall(a)intel.com<mailto:[email protected]>>; Arney, Rebecca <rebecca.arney(a)intel.com<mailto:[email protected]>>; Kumaran, Kalyan <kumaran(a)alcf.anl.gov<mailto:[email protected]>>; Knight, Christopher J. <knight(a)mcs.anl.gov<mailto:[email protected]>>; Allison, CM <cm.allison(a)intel.com<mailto:[email protected]>> Subject: Re: Kernel Panic on KNL Box Hi Russ, We can get a similar kernel panic by just running a 128 rank mpi_hello_world app, so I don't think this is actually related to LAMPPS. I'm going to hold off doing additional testing and troubleshooting on this until we get the BIOS upgraded. We're also still seeing Machine Check Errors "Generic CACHE Level-2 Generic Error". Ben On Mar 22, 2016, at 3:40 PM, Fromkin, Russ <russ.fromkin(a)intel.com<mailto:[email protected]>> wrote: Pulling Rebecca onto the string, as I want to make sure we keep any of this work and early science work together. We will confirm this the next steps shortly. Russ -----Original Message----- From: Allen, Benjamin S. [mailto:[email protected]] Sent: Tuesday, March 22, 2016 3:30 PM To: Fromkin, Russ <russ.fromkin(a)intel.com<mailto:[email protected]>>; Kirkendall, Keith G <keith.g.kirkendall(a)intel.com<mailto:[email protected]>> Cc: Kumaran, Kalyan <kumaran(a)alcf.anl.gov<mailto:[email protected]>>; Knight, Christopher J. <knight(a)mcs.anl.gov<mailto:[email protected]>>; Allison, CM <cm.allison(a)intel.com<mailto:[email protected]>> Subject: Kernel Panic on KNL Box Russ and Keith, We have a reproducible kernel panic when running LAMMPS on the KNL B0 box, since this hardware is NDA and not-released we can't really ask for help from RedHat. Who can we get help from? Thanks, Ben

1 0