August 2016 - Intel-nda - mailman3.cels.anl.gov

Re: [JLSE-Intel-NDA] Location of Vtune?
by Delalondre Fabien Jonathan 24 Aug '16

24 Aug '16

source /soft/compilers/intel/vtune_amplifier_xe_2016.4.0.470476/amplxe-vars.sh Can you please add this to the confluence page instructions ? ________________________________________ From: intel-nda-bounces(a)lists.jlse.anl.gov <intel-nda-bounces(a)lists.jlse.anl.gov> on behalf of Tom LeCompte <lecompte(a)anl.gov> Sent: Wednesday, August 24, 2016 4:26 PM To: intel-nda(a)jlse.anl.gov Subject: [JLSE-Intel-NDA] Location of Vtune? Hi everyone- I have an application I would like to profile. The Wiki says "Option 1" is to use Vtune and gives instructions execute the following command via Cobalt submission script $ amplxe-cl -c memory-access ... This givea a command not found for amplxe-cl. Is there something I need to set up? Cheers, Tom _______________________________________________ Intel-nda mailing list Intel-nda(a)lists.jlse.anl.gov https://lists.jlse.anl.gov/mailman/listinfo/intel-nda

1 0

[JLSE-Intel-NDA] Location of Vtune?
by Tom LeCompte 24 Aug '16

24 Aug '16

Hi everyone- I have an application I would like to profile. The Wiki says "Option 1" is to use Vtune and gives instructions execute the following command via Cobalt submission script $ amplxe-cl -c memory-access ... This givea a command not found for amplxe-cl. Is there something I need to set up? Cheers, Tom

1 0

Re: [JLSE-Intel-NDA] jlse nodes
by Alekseev, Yury 23 Aug '16

23 Aug '16

This problem is sorted out now. Yuri ________________________________________ From: intel-nda-bounces(a)lists.jlse.anl.gov [intel-nda-bounces(a)lists.jlse.anl.gov] on behalf of Hammond, Jeff R [jeff.r.hammond(a)intel.com] Sent: Tuesday, August 23, 2016 12:38 PM To: Kumaran, Kalyan Cc: intel-nda(a)lists.jlse.anl.gov Subject: Re: [JLSE-Intel-NDA] jlse nodes Sarom is GAMESS developer Sent from my iPhone > On Aug 23, 2016, at 10:24 AM, Kumaran, Kalyan <kumaran(a)alcf.anl.gov> wrote: > > Which project is he part of? ESP or other? > > >> On Aug 23, 2016, at 12:22 PM, Allen, Benjamin S. <bsallen(a)alcf.anl.gov> wrote: >> >> Fixed the permissions on the knl_7250 queue, brought knl04 and knl09 back. knl03 and knl05 were panic'ed. Rebooting them now. >> >> I'll leave the policy question on Sarom Leang's 6 48-hour jobs to Tim and Kumar. >> >> Ben >> >>> On Aug 23, 2016, at 12:17 PM, Kenneth E Jansen <kenneth.jansen(a)colorado.edu> wrote: >>> >>> looks like 3 7210 nodes are down, a single user has submitted 6, 48 hour INTERACTIVE jobs to the only up 7210 nodes. >>> >>> one of the 7250 nodes is down, the other idle but trying to submit to it yields >>> >>> [kjansen@jlselogin1 64]$ qsub -n 1 -q knl_7250 -t 60 -I >>> <Fault 1001: "You are not allowed to submit to the 'knl_7250' queue\n"> >>> >>> >>> ok while writing this email it seems knl04 is back but are we really ok with 6 48 hour interactive one node jobs? >>> >>> And is it correct that I (other users too) can’t use the knl_7250 queue…thought I did before but maybe miss-remembering). >>> >>> Best regards, >>> >>> Ken >>> >>> ---------------------------------------------------------------- >>> Kenneth E. Jansen, Professor of Aerospace Engineering Sciences >>> ECAE 161 OFFICE (303) 492-4359 >>> 429 UCB FAX (303) 492-4990 >>> University of Colorado at Boulder jansenke(a)colorado.edu >>> Boulder, CO, 80309-0429 http://www.colorado.edu/aerospace >>> ---------------------------------------------------------------- >>> >>> >>> >>> _______________________________________________ >>> Intel-nda mailing list >>> Intel-nda(a)lists.jlse.anl.gov >>> https://lists.jlse.anl.gov/mailman/listinfo/intel-nda > > _______________________________________________ > Intel-nda mailing list > Intel-nda(a)lists.jlse.anl.gov > https://lists.jlse.anl.gov/mailman/listinfo/intel-nda _______________________________________________ Intel-nda mailing list Intel-nda(a)lists.jlse.anl.gov https://lists.jlse.anl.gov/mailman/listinfo/intel-nda

1 0

Re: [JLSE-Intel-NDA] jlse nodes
by Hammond, Jeff R 23 Aug '16

23 Aug '16

Sarom is GAMESS developer Sent from my iPhone > On Aug 23, 2016, at 10:24 AM, Kumaran, Kalyan <kumaran(a)alcf.anl.gov> wrote: > > Which project is he part of? ESP or other? > > >> On Aug 23, 2016, at 12:22 PM, Allen, Benjamin S. <bsallen(a)alcf.anl.gov> wrote: >> >> Fixed the permissions on the knl_7250 queue, brought knl04 and knl09 back. knl03 and knl05 were panic'ed. Rebooting them now. >> >> I'll leave the policy question on Sarom Leang's 6 48-hour jobs to Tim and Kumar. >> >> Ben >> >>> On Aug 23, 2016, at 12:17 PM, Kenneth E Jansen <kenneth.jansen(a)colorado.edu> wrote: >>> >>> looks like 3 7210 nodes are down, a single user has submitted 6, 48 hour INTERACTIVE jobs to the only up 7210 nodes. >>> >>> one of the 7250 nodes is down, the other idle but trying to submit to it yields >>> >>> [kjansen@jlselogin1 64]$ qsub -n 1 -q knl_7250 -t 60 -I >>> <Fault 1001: "You are not allowed to submit to the 'knl_7250' queue\n"> >>> >>> >>> ok while writing this email it seems knl04 is back but are we really ok with 6 48 hour interactive one node jobs? >>> >>> And is it correct that I (other users too) can’t use the knl_7250 queue…thought I did before but maybe miss-remembering). >>> >>> Best regards, >>> >>> Ken >>> >>> ---------------------------------------------------------------- >>> Kenneth E. Jansen, Professor of Aerospace Engineering Sciences >>> ECAE 161 OFFICE (303) 492-4359 >>> 429 UCB FAX (303) 492-4990 >>> University of Colorado at Boulder jansenke(a)colorado.edu >>> Boulder, CO, 80309-0429 http://www.colorado.edu/aerospace >>> ---------------------------------------------------------------- >>> >>> >>> >>> _______________________________________________ >>> Intel-nda mailing list >>> Intel-nda(a)lists.jlse.anl.gov >>> https://lists.jlse.anl.gov/mailman/listinfo/intel-nda > > _______________________________________________ > Intel-nda mailing list > Intel-nda(a)lists.jlse.anl.gov > https://lists.jlse.anl.gov/mailman/listinfo/intel-nda

1 0

Re: [JLSE-Intel-NDA] jlse nodes
by Kumaran, Kalyan 23 Aug '16

23 Aug '16

Which project is he part of? ESP or other? > On Aug 23, 2016, at 12:22 PM, Allen, Benjamin S. <bsallen(a)alcf.anl.gov> wrote: > > Fixed the permissions on the knl_7250 queue, brought knl04 and knl09 back. knl03 and knl05 were panic'ed. Rebooting them now. > > I'll leave the policy question on Sarom Leang's 6 48-hour jobs to Tim and Kumar. > > Ben > >> On Aug 23, 2016, at 12:17 PM, Kenneth E Jansen <kenneth.jansen(a)colorado.edu> wrote: >> >> looks like 3 7210 nodes are down, a single user has submitted 6, 48 hour INTERACTIVE jobs to the only up 7210 nodes. >> >> one of the 7250 nodes is down, the other idle but trying to submit to it yields >> >> [kjansen@jlselogin1 64]$ qsub -n 1 -q knl_7250 -t 60 -I >> <Fault 1001: "You are not allowed to submit to the 'knl_7250' queue\n"> >> >> >> ok while writing this email it seems knl04 is back but are we really ok with 6 48 hour interactive one node jobs? >> >> And is it correct that I (other users too) can’t use the knl_7250 queue…thought I did before but maybe miss-remembering). >> >> Best regards, >> >> Ken >> >> ---------------------------------------------------------------- >> Kenneth E. Jansen, Professor of Aerospace Engineering Sciences >> ECAE 161 OFFICE (303) 492-4359 >> 429 UCB FAX (303) 492-4990 >> University of Colorado at Boulder jansenke(a)colorado.edu >> Boulder, CO, 80309-0429 http://www.colorado.edu/aerospace >> ---------------------------------------------------------------- >> >> >> >> _______________________________________________ >> Intel-nda mailing list >> Intel-nda(a)lists.jlse.anl.gov >> https://lists.jlse.anl.gov/mailman/listinfo/intel-nda >

1 0

Re: [JLSE-Intel-NDA] jlse nodes
by Allen, Benjamin S. 23 Aug '16

23 Aug '16

Fixed the permissions on the knl_7250 queue, brought knl04 and knl09 back. knl03 and knl05 were panic'ed. Rebooting them now. I'll leave the policy question on Sarom Leang's 6 48-hour jobs to Tim and Kumar. Ben > On Aug 23, 2016, at 12:17 PM, Kenneth E Jansen <kenneth.jansen(a)colorado.edu> wrote: > > looks like 3 7210 nodes are down, a single user has submitted 6, 48 hour INTERACTIVE jobs to the only up 7210 nodes. > > one of the 7250 nodes is down, the other idle but trying to submit to it yields > > [kjansen@jlselogin1 64]$ qsub -n 1 -q knl_7250 -t 60 -I > <Fault 1001: "You are not allowed to submit to the 'knl_7250' queue\n"> > > > ok while writing this email it seems knl04 is back but are we really ok with 6 48 hour interactive one node jobs? > > And is it correct that I (other users too) can’t use the knl_7250 queue…thought I did before but maybe miss-remembering). > > Best regards, > > Ken > > ---------------------------------------------------------------- > Kenneth E. Jansen, Professor of Aerospace Engineering Sciences > ECAE 161 OFFICE (303) 492-4359 > 429 UCB FAX (303) 492-4990 > University of Colorado at Boulder jansenke(a)colorado.edu > Boulder, CO, 80309-0429 http://www.colorado.edu/aerospace > ---------------------------------------------------------------- > > > > _______________________________________________ > Intel-nda mailing list > Intel-nda(a)lists.jlse.anl.gov > https://lists.jlse.anl.gov/mailman/listinfo/intel-nda

1 0

[JLSE-Intel-NDA] jlse nodes
by Kenneth E Jansen 23 Aug '16

23 Aug '16

looks like 3 7210 nodes are down, a single user has submitted 6, 48 hour INTERACTIVE jobs to the only up 7210 nodes. one of the 7250 nodes is down, the other idle but trying to submit to it yields [kjansen@jlselogin1 64]$ qsub -n 1 -q knl_7250 -t 60 -I <Fault 1001: "You are not allowed to submit to the 'knl_7250' queue\n"> ok while writing this email it seems knl04 is back but are we really ok with 6 48 hour interactive one node jobs? And is it correct that I (other users too) can’t use the knl_7250 queue…thought I did before but maybe miss-remembering). Best regards, Ken ---------------------------------------------------------------- Kenneth E. Jansen, Professor of Aerospace Engineering Sciences ECAE 161 OFFICE (303) 492-4359 429 UCB FAX (303) 492-4990 University of Colorado at Boulder jansenke(a)colorado.edu Boulder, CO, 80309-0429 http://www.colorado.edu/aerospace ----------------------------------------------------------------

1 0

Re: [JLSE-Intel-NDA] JSLE KNL Scheduling Paused
by Allen, Benjamin S. 17 Aug '16

17 Aug '16

This is done, scheduling un-paused. Now at worst, if a node goes down when idle (between jobs), it will only cause one job to fail. Ben > On Aug 17, 2016, at 3:19 PM, Allen, Benjamin S. <bsallen(a)alcf.anl.gov> wrote: > > I've stopped scheduling for a bit while I test some changes to fix problems Steve and Ye Luo reported. Guessing less than 30 minutes. Running jobs are unaffected. > > Ben > _______________________________________________ > Intel-nda mailing list > Intel-nda(a)lists.jlse.anl.gov > https://lists.jlse.anl.gov/mailman/listinfo/intel-nda

1 0

[JLSE-Intel-NDA] JSLE KNL Scheduling Paused
by Allen, Benjamin S. 17 Aug '16

17 Aug '16

I've stopped scheduling for a bit while I test some changes to fix problems Steve and Ye Luo reported. Guessing less than 30 minutes. Running jobs are unaffected. Ben

1 0

Re: [JLSE-Intel-NDA] threading issue on KNL
by Kumaran, Kalyan 17 Aug '16

17 Aug '16

Maybe Jeff Hammond can help... > On Aug 17, 2016, at 1:41 PM, Allen, Benjamin S. <bsallen(a)alcf.anl.gov> wrote: > > We have very little experience with OmniPath so far. To use the OmniPath interfaces via Intel MPI add "-iface ib0 -PSM" to your mpirun. > > Unsure about the rank/thread placement questions. > > Ben > >> On Aug 17, 2016, at 12:35 PM, Luo, Ye <yeluo(a)anl.gov> wrote: >> >> Hi all, >> >> I just noticed a very strange behavior of the mpirun on KNL nodes. >> If I run 1 node with 1 MPI and 128 threads, everything goes well. >> If I run 2 nodes with 1 MPI and 128 threads on each nodes, all the 128 threads are actually running on a single core. >> This behavior is confirmed with KMP_AFFINITY=verbose. >> >> An extra question, is the currect mpirun aware of the omni path? I'm using the intel mpi comes with the compiler. >> >> Ye >> >> =================== >> Ye Luo, Ph.D. >> Leadership Computing Facility >> Argonne National Laboratory >> _______________________________________________ >> Intel-nda mailing list >> Intel-nda(a)lists.jlse.anl.gov >> https://lists.jlse.anl.gov/mailman/listinfo/intel-nda > > _______________________________________________ > Intel-nda mailing list > Intel-nda(a)lists.jlse.anl.gov > https://lists.jlse.anl.gov/mailman/listinfo/intel-nda

1 0