May 2015 - Allocations-admins - mailman3.cels.anl.gov

Re: [allocations-admins] [LCRC Accounts] Project Allocation Request SepCFD
by Aithal, Shashikant M. 06 May '15

06 May '15

I had several email exchanges with Kent yesterday on this matter. He has apparently been using the INL cluster since he does not have to wait in queue. He plans to have a summer student/post-doc who will use this time from mid-May to late August for some OpenFoam/multiphase flow simulations. ________________________________________ From: allocations-admins-bounces(a)lcrc.anl.gov [allocations-admins-bounces(a)lcrc.anl.gov] on behalf of Bair, Raymond A. [rbair(a)anl.gov] Sent: Wednesday, May 06, 2015 9:11 AM To: LCRC Allocations Admins Subject: Re: [allocations-admins] [LCRC Accounts] Project Allocation Request SepCFD In FY2014 this project had a big allocation but only used 37 core-hours (out of 450K). That presents a policy Catch-22 Does a project need to submit a report if it did not do anything? I think not, but their future allocation may reflect their apparent readiness. Also I suspect that JV was waiting for their report; but we should not require one. Hence, lets grant the requested 75K and see if it gets used. Regards, Ray ----------------------------------------- Ray Bair Computing, Environment, and Life Sciences Argonne National Laboratory and the University of Chicago TCS Building 240, Room 4122 9700 South Cass Avenue Argonne, IL 60439 email: rbair(at)anl.gov Phone: (630)252-5751 On 5/5/15, 4:45 PM, "accounts(a)lcrc.anl.gov" <accounts(a)lcrc.anl.gov> wrote: >Hello, > >A change in allocation has been requested: > > Requester: wardle (Kent E. Wardle) > Project: SepCFD > Title: CFD Simulation of Separations Processes > Description: This project involves multiphase, multi-scale simulations >related to complex flows in nuclear fuel cycle and separations processes. > Simulations of liquid-liquid mixing phenomena in complex devices will be >performed. In addition, simulations of multiphase flows in a sub-critical >nuclear solution with natural convection will also be performed. > >The code package OpenFOAM uses a C++ object-oriented framework with >dynamically linked libraries and high-level solvers to enable the user to >write custom solvers and combine various physics models and equations >enabling the evaluation of unique combinations of models such as are >required for the development of advanced liquid-liquid mixing simulation >capability as targeted in this work. The hybrid solver developed in >OpenFOAM and other custom OpenFOAM solvers will be used for this work. > >This allocation supports nuclear fuel cycle simulations efforts under >DOE-NE as well as simulations supporting projects funded under DOE-NNSA >related to medical isotope production. This latter project component will >be the focus of a concerted effort during the Summer of 2015 with the >participation of a Graduate Research Aide. > Current: undetermined amount >Justification: > > Requested: 75000 > >A specific reason has been given: >Along with initial scoping simulations, it is anticipated that a total of >~5 simulations on 128 cores and with a wall-time of approximately 100hrs >each will be conducted using this requested allocation. These >simulations are to be done during the period from May to ~Aug 2015. > >This needs to be approved and the final allocation amount decided upon. > >Thank You, >The LCRC Accounts System >_______________________________________________ >allocations-admins mailing list >allocations-admins(a)lcrc.anl.gov >https://lists.lcrc.anl.gov/mailman/listinfo/allocations-admins _______________________________________________ allocations-admins mailing list allocations-admins(a)lcrc.anl.gov https://lists.lcrc.anl.gov/mailman/listinfo/allocations-admins

1 0

Re: [allocations-admins] [LCRC Accounts] Project Allocation Request SepCFD
by Bair, Raymond A. 06 May '15

06 May '15

In FY2014 this project had a big allocation but only used 37 core-hours (out of 450K). That presents a policy Catch-22 Does a project need to submit a report if it did not do anything? I think not, but their future allocation may reflect their apparent readiness. Also I suspect that JV was waiting for their report; but we should not require one. Hence, lets grant the requested 75K and see if it gets used. Regards, Ray ----------------------------------------- Ray Bair Computing, Environment, and Life Sciences Argonne National Laboratory and the University of Chicago TCS Building 240, Room 4122 9700 South Cass Avenue Argonne, IL 60439 email: rbair(at)anl.gov Phone: (630)252-5751 On 5/5/15, 4:45 PM, "accounts(a)lcrc.anl.gov" <accounts(a)lcrc.anl.gov> wrote: >Hello, > >A change in allocation has been requested: > > Requester: wardle (Kent E. Wardle) > Project: SepCFD > Title: CFD Simulation of Separations Processes > Description: This project involves multiphase, multi-scale simulations >related to complex flows in nuclear fuel cycle and separations processes. > Simulations of liquid-liquid mixing phenomena in complex devices will be >performed. In addition, simulations of multiphase flows in a sub-critical >nuclear solution with natural convection will also be performed. > >The code package OpenFOAM uses a C++ object-oriented framework with >dynamically linked libraries and high-level solvers to enable the user to >write custom solvers and combine various physics models and equations >enabling the evaluation of unique combinations of models such as are >required for the development of advanced liquid-liquid mixing simulation >capability as targeted in this work. The hybrid solver developed in >OpenFOAM and other custom OpenFOAM solvers will be used for this work. > >This allocation supports nuclear fuel cycle simulations efforts under >DOE-NE as well as simulations supporting projects funded under DOE-NNSA >related to medical isotope production. This latter project component will >be the focus of a concerted effort during the Summer of 2015 with the >participation of a Graduate Research Aide. > Current: undetermined amount >Justification: > > Requested: 75000 > >A specific reason has been given: >Along with initial scoping simulations, it is anticipated that a total of >~5 simulations on 128 cores and with a wall-time of approximately 100hrs >each will be conducted using this requested allocation. These >simulations are to be done during the period from May to ~Aug 2015. > >This needs to be approved and the final allocation amount decided upon. > >Thank You, >The LCRC Accounts System >_______________________________________________ >allocations-admins mailing list >allocations-admins(a)lcrc.anl.gov >https://lists.lcrc.anl.gov/mailman/listinfo/allocations-admins

1 0

Re: [allocations-admins] [LCRC Accounts] Project Request: NEKCEM
by John Blaas 05 May '15

05 May '15

Hi all, The project has been activated. - JB On Mon, May 4, 2015 at 2:53 PM, Bair, Raymond A. <rbair(a)anl.gov> wrote: > I suggest that we approve this request > > Ray

1 0

[LCRC Accounts] Project Allocation Request
by accounts＠lcrc.anl.gov 05 May '15

05 May '15

Hello, A change in allocation has been requested: Requester: wardle (Kent E. Wardle) Project: SepCFD Title: CFD Simulation of Separations Processes Description: This project involves multiphase, multi-scale simulations related to complex flows in nuclear fuel cycle and separations processes. Simulations of liquid-liquid mixing phenomena in complex devices will be performed. In addition, simulations of multiphase flows in a sub-critical nuclear solution with natural convection will also be performed. The code package OpenFOAM uses a C++ object-oriented framework with dynamically linked libraries and high-level solvers to enable the user to write custom solvers and combine various physics models and equations enabling the evaluation of unique combinations of models such as are required for the development of advanced liquid-liquid mixing simulation capability as targeted in this work. The hybrid solver developed in OpenFOAM and other custom OpenFOAM solvers will be used for this work. This allocation supports nuclear fuel cycle simulations efforts under DOE-NE as well as simulations supporting projects funded under DOE-NNSA related to medical isotope production. This latter project component will be the focus of a concerted effort during the Summer of 2015 with the participation of a Graduate Research Aide. Current: undetermined amount Justification: Requested: 75000 A specific reason has been given: Along with initial scoping simulations, it is anticipated that a total of ~5 simulations on 128 cores and with a wall-time of approximately 100hrs each will be conducted using this requested allocation. These simulations are to be done during the period from May to ~Aug 2015. This needs to be approved and the final allocation amount decided upon. Thank You, The LCRC Accounts System

1 0

Re: [allocations-admins] [LCRC Accounts] Project Request: NEKCEM
by Aithal, Shashikant M. 05 May '15

05 May '15

Thanks, John. Shashi ________________________________________ From: allocations-admins-bounces(a)lcrc.anl.gov [allocations-admins-bounces(a)lcrc.anl.gov] on behalf of John Blaas [wilshire(a)mcs.anl.gov] Sent: Tuesday, May 05, 2015 8:10 AM To: LCRC Allocations Admins Subject: Re: [allocations-admins] [LCRC Accounts] Project Request: NEKCEM I'll activate this project later today. - JB On Mon, May 4, 2015 at 2:53 PM, Bair, Raymond A. <rbair(a)anl.gov> wrote: > I suggest that we approve this request > > Ray > > > > > > On 5/2/15, 1:35 PM, "accounts(a)lcrc.anl.gov" <accounts(a)lcrc.anl.gov> wrote: > >>Hello, >> >>A new project on the LCRC cluster has been requested. Please forward >>the information on to the LCRC Allocation sub-committee. >> >> Applicant's name: Misun Min >>Applicant's institution: ANL >> Applicant's division: MCS >> Project Name: NEKCEM >> Project title: NEKCEM >> Associated funding: DOE ASCR, DOE CESAR >> Other Systems: OLCF Titan, ALCF BG/P, ALCF BG/Q Vesta >> Science: NEKCEM is a high-order spectral-element (SE) >>discretization basis code for solving nanoscale solar cells, plasmonic >>nanoparticle devices, and quantum electrodynamic systems, with scalable >>CPU performance up to 262,144 CPU cores and scalable GPU performance up >>to 16,384 GPUs. >> >> Project description: NEKCEM is a high-order spectral-element (SE) >>discretization basis code for solving nanoscale solar cells, plasmonic >>nanoparticle devices, and quantum electrodynamic systems, with scalable >>CPU performance up to 262,144 CPU cores and scalable GPU performance up >>to 16,384 GPUs. >> >>We have been developing a newly tuned gather-scatter kernel of NekCEM >>that can be portable for GPU runs at large scale and recently have >>completed scalability tests on OLCF Titan using 16,384 GPUs with >>GPUDirect and GPUDirect-disabled communication, compared to CPU-only runs >>on Titan using the same number of nodes (262,144 MPI ranks). We are now >>ready to perform intensive tests of the newly tuned communication kernel >>on different systems for CPU runs. >> >>Especially, NekCEM's quantum mechanical solver is a newly-developed >>solver based on density matrix formalism that is designed for solving >>multi-state multi-quantum dot system models at exascale. Our algorithms >>use a compressed-sparse-row format allowing any quantum systems to be >>defined in a reduced dense form for the Hamiltonian, work scaling as >>O(N^2). Using this solver, Argonne scientists and university >>collaborators were able to explore a new physical scheme for creating >>significant bipartite entanglements at arbitrary time using repeated >>pulse in the hybrid quantum/plasmonic nanoparticle systems, not been >>studied before. >> >>We also developed a spectral-element (SE) based drift-diffusion solver in >>time domain. Efficient timestepping algorithms are developed by treating >>nonlinear drift term explicitly and diffusion term implicitly, including >>projection and SE multigrid preconditioners. Full 3D simulations were >>performed for a potassium ion channel in biological membrane, validating >>other groupsâ€™ works that are based on dimension-reduction approach. >> >> >> Industry partnership: We request 50,000 allocation hours. >> >>Using LCRC computational resources, we'll test our newly tuned >>gather-scatter communication kernel for scalability on CPUs, quantum >>density matrix solver for large-scale quantum systems, drift-diffusion >>solver for nanowire solar cells. >> >> Project URL: http://nekcem.mcs.anl.gov/ >> Requested allocation: 60000 >> Q1: 0 >> Q2: 0 >> Q3: 20000 >> Q4: 40000 >> Justification: >> Storage requirements: >> >> >>The requester has used undetermined amount hours of their initial startup >>project. >> >>In addition to approving an initial amount, please specify a Category >>and Subcategory for this project. For a list of the current selection >>of approved categories, please see: >> >> https://wiki.lcrc.anl.gov/wiki/Processes/Categories >> >>Once the Allocation committee has approved the project, please go to >>the Project Management page to create it: >> >>https://accounts.lcrc.anl.gov/projects.php >> >>Thank You, >>The LCRC Accounts System > > _______________________________________________ > allocations-admins mailing list > allocations-admins(a)lcrc.anl.gov > https://lists.lcrc.anl.gov/mailman/listinfo/allocations-admins _______________________________________________ allocations-admins mailing list allocations-admins(a)lcrc.anl.gov https://lists.lcrc.anl.gov/mailman/listinfo/allocations-admins

1 0

Re: [allocations-admins] [LCRC Accounts] Project Request: NEKCEM
by John Blaas 05 May '15

05 May '15

I'll activate this project later today. - JB On Mon, May 4, 2015 at 2:53 PM, Bair, Raymond A. <rbair(a)anl.gov> wrote: > I suggest that we approve this request > > Ray > > > > > > On 5/2/15, 1:35 PM, "accounts(a)lcrc.anl.gov" <accounts(a)lcrc.anl.gov> wrote: > >>Hello, >> >>A new project on the LCRC cluster has been requested. Please forward >>the information on to the LCRC Allocation sub-committee. >> >> Applicant's name: Misun Min >>Applicant's institution: ANL >> Applicant's division: MCS >> Project Name: NEKCEM >> Project title: NEKCEM >> Associated funding: DOE ASCR, DOE CESAR >> Other Systems: OLCF Titan, ALCF BG/P, ALCF BG/Q Vesta >> Science: NEKCEM is a high-order spectral-element (SE) >>discretization basis code for solving nanoscale solar cells, plasmonic >>nanoparticle devices, and quantum electrodynamic systems, with scalable >>CPU performance up to 262,144 CPU cores and scalable GPU performance up >>to 16,384 GPUs. >> >> Project description: NEKCEM is a high-order spectral-element (SE) >>discretization basis code for solving nanoscale solar cells, plasmonic >>nanoparticle devices, and quantum electrodynamic systems, with scalable >>CPU performance up to 262,144 CPU cores and scalable GPU performance up >>to 16,384 GPUs. >> >>We have been developing a newly tuned gather-scatter kernel of NekCEM >>that can be portable for GPU runs at large scale and recently have >>completed scalability tests on OLCF Titan using 16,384 GPUs with >>GPUDirect and GPUDirect-disabled communication, compared to CPU-only runs >>on Titan using the same number of nodes (262,144 MPI ranks). We are now >>ready to perform intensive tests of the newly tuned communication kernel >>on different systems for CPU runs. >> >>Especially, NekCEM's quantum mechanical solver is a newly-developed >>solver based on density matrix formalism that is designed for solving >>multi-state multi-quantum dot system models at exascale. Our algorithms >>use a compressed-sparse-row format allowing any quantum systems to be >>defined in a reduced dense form for the Hamiltonian, work scaling as >>O(N^2). Using this solver, Argonne scientists and university >>collaborators were able to explore a new physical scheme for creating >>significant bipartite entanglements at arbitrary time using repeated >>pulse in the hybrid quantum/plasmonic nanoparticle systems, not been >>studied before. >> >>We also developed a spectral-element (SE) based drift-diffusion solver in >>time domain. Efficient timestepping algorithms are developed by treating >>nonlinear drift term explicitly and diffusion term implicitly, including >>projection and SE multigrid preconditioners. Full 3D simulations were >>performed for a potassium ion channel in biological membrane, validating >>other groupsâ€™ works that are based on dimension-reduction approach. >> >> >> Industry partnership: We request 50,000 allocation hours. >> >>Using LCRC computational resources, we'll test our newly tuned >>gather-scatter communication kernel for scalability on CPUs, quantum >>density matrix solver for large-scale quantum systems, drift-diffusion >>solver for nanowire solar cells. >> >> Project URL: http://nekcem.mcs.anl.gov/ >> Requested allocation: 60000 >> Q1: 0 >> Q2: 0 >> Q3: 20000 >> Q4: 40000 >> Justification: >> Storage requirements: >> >> >>The requester has used undetermined amount hours of their initial startup >>project. >> >>In addition to approving an initial amount, please specify a Category >>and Subcategory for this project. For a list of the current selection >>of approved categories, please see: >> >> https://wiki.lcrc.anl.gov/wiki/Processes/Categories >> >>Once the Allocation committee has approved the project, please go to >>the Project Management page to create it: >> >>https://accounts.lcrc.anl.gov/projects.php >> >>Thank You, >>The LCRC Accounts System > > _______________________________________________ > allocations-admins mailing list > allocations-admins(a)lcrc.anl.gov > https://lists.lcrc.anl.gov/mailman/listinfo/allocations-admins

1 0

Re: [allocations-admins] [LCRC Accounts] Project Request: NEKCEM
by Bair, Raymond A. 04 May '15

04 May '15

I suggest that we approve this request Ray On 5/2/15, 1:35 PM, "accounts(a)lcrc.anl.gov" <accounts(a)lcrc.anl.gov> wrote: >Hello, > >A new project on the LCRC cluster has been requested. Please forward >the information on to the LCRC Allocation sub-committee. > > Applicant's name: Misun Min >Applicant's institution: ANL > Applicant's division: MCS > Project Name: NEKCEM > Project title: NEKCEM > Associated funding: DOE ASCR, DOE CESAR > Other Systems: OLCF Titan, ALCF BG/P, ALCF BG/Q Vesta > Science: NEKCEM is a high-order spectral-element (SE) >discretization basis code for solving nanoscale solar cells, plasmonic >nanoparticle devices, and quantum electrodynamic systems, with scalable >CPU performance up to 262,144 CPU cores and scalable GPU performance up >to 16,384 GPUs. > > Project description: NEKCEM is a high-order spectral-element (SE) >discretization basis code for solving nanoscale solar cells, plasmonic >nanoparticle devices, and quantum electrodynamic systems, with scalable >CPU performance up to 262,144 CPU cores and scalable GPU performance up >to 16,384 GPUs. > >We have been developing a newly tuned gather-scatter kernel of NekCEM >that can be portable for GPU runs at large scale and recently have >completed scalability tests on OLCF Titan using 16,384 GPUs with >GPUDirect and GPUDirect-disabled communication, compared to CPU-only runs >on Titan using the same number of nodes (262,144 MPI ranks). We are now >ready to perform intensive tests of the newly tuned communication kernel >on different systems for CPU runs. > >Especially, NekCEM's quantum mechanical solver is a newly-developed >solver based on density matrix formalism that is designed for solving >multi-state multi-quantum dot system models at exascale. Our algorithms >use a compressed-sparse-row format allowing any quantum systems to be >defined in a reduced dense form for the Hamiltonian, work scaling as >O(N^2). Using this solver, Argonne scientists and university >collaborators were able to explore a new physical scheme for creating >significant bipartite entanglements at arbitrary time using repeated >pulse in the hybrid quantum/plasmonic nanoparticle systems, not been >studied before. > >We also developed a spectral-element (SE) based drift-diffusion solver in >time domain. Efficient timestepping algorithms are developed by treating >nonlinear drift term explicitly and diffusion term implicitly, including >projection and SE multigrid preconditioners. Full 3D simulations were >performed for a potassium ion channel in biological membrane, validating >other groupsâ€™ works that are based on dimension-reduction approach. > > > Industry partnership: We request 50,000 allocation hours. > >Using LCRC computational resources, we'll test our newly tuned >gather-scatter communication kernel for scalability on CPUs, quantum >density matrix solver for large-scale quantum systems, drift-diffusion >solver for nanowire solar cells. > > Project URL: http://nekcem.mcs.anl.gov/ > Requested allocation: 60000 > Q1: 0 > Q2: 0 > Q3: 20000 > Q4: 40000 > Justification: > Storage requirements: > > >The requester has used undetermined amount hours of their initial startup >project. > >In addition to approving an initial amount, please specify a Category >and Subcategory for this project. For a list of the current selection >of approved categories, please see: > > https://wiki.lcrc.anl.gov/wiki/Processes/Categories > >Once the Allocation committee has approved the project, please go to >the Project Management page to create it: > >https://accounts.lcrc.anl.gov/projects.php > >Thank You, >The LCRC Accounts System

1 0

[LCRC Accounts] Project Request: NEKCEM
by accounts＠lcrc.anl.gov 02 May '15

02 May '15

Hello, A new project on the LCRC cluster has been requested. Please forward the information on to the LCRC Allocation sub-committee. Applicant's name: Misun Min Applicant's institution: ANL Applicant's division: MCS Project Name: NEKCEM Project title: NEKCEM Associated funding: DOE ASCR, DOE CESAR Other Systems: OLCF Titan, ALCF BG/P, ALCF BG/Q Vesta Science: NEKCEM is a high-order spectral-element (SE) discretization basis code for solving nanoscale solar cells, plasmonic nanoparticle devices, and quantum electrodynamic systems, with scalable CPU performance up to 262,144 CPU cores and scalable GPU performance up to 16,384 GPUs. Project description: NEKCEM is a high-order spectral-element (SE) discretization basis code for solving nanoscale solar cells, plasmonic nanoparticle devices, and quantum electrodynamic systems, with scalable CPU performance up to 262,144 CPU cores and scalable GPU performance up to 16,384 GPUs. We have been developing a newly tuned gather-scatter kernel of NekCEM that can be portable for GPU runs at large scale and recently have completed scalability tests on OLCF Titan using 16,384 GPUs with GPUDirect and GPUDirect-disabled communication, compared to CPU-only runs on Titan using the same number of nodes (262,144 MPI ranks). We are now ready to perform intensive tests of the newly tuned communication kernel on different systems for CPU runs. Especially, NekCEM's quantum mechanical solver is a newly-developed solver based on density matrix formalism that is designed for solving multi-state multi-quantum dot system models at exascale. Our algorithms use a compressed-sparse-row format allowing any quantum systems to be defined in a reduced dense form for the Hamiltonian, work scaling as O(N^2). Using this solver, Argonne scientists and university collaborators were able to explore a new physical scheme for creating significant bipartite entanglements at arbitrary time using repeated pulse in the hybrid quantum/plasmonic nanoparticle systems, not been studied before. We also developed a spectral-element (SE) based drift-diffusion solver in time domain. Efficient timestepping algorithms are developed by treating nonlinear drift term explicitly and diffusion term implicitly, including projection and SE multigrid preconditioners. Full 3D simulations were performed for a potassium ion channel in biological membrane, validating other groups’ works that are based on dimension-reduction approach. Industry partnership: We request 50,000 allocation hours. Using LCRC computational resources, we'll test our newly tuned gather-scatter communication kernel for scalability on CPUs, quantum density matrix solver for large-scale quantum systems, drift-diffusion solver for nanowire solar cells. Project URL: http://nekcem.mcs.anl.gov/ Requested allocation: 60000 Q1: 0 Q2: 0 Q3: 20000 Q4: 40000 Justification: Storage requirements: The requester has used undetermined amount hours of their initial startup project. In addition to approving an initial amount, please specify a Category and Subcategory for this project. For a list of the current selection of approved categories, please see: https://wiki.lcrc.anl.gov/wiki/Processes/Categories Once the Allocation committee has approved the project, please go to the Project Management page to create it: https://accounts.lcrc.anl.gov/projects.php Thank You, The LCRC Accounts System

1 0