September 2016 - Allocations-admins - mailman3.cels.anl.gov

[LCRC Accounts] Project Request: grbdft
by accounts＠lcrc.anl.gov 07 Sep '16

07 Sep '16

Hello, A new project on the LCRC cluster has been requested. Please forward the information on to the LCRC Allocation sub-committee. Applicant's name: Fatih Sen Applicant's institution: ANL Applicant's division: NST Project Name: grbdft Project title: Atomistic modeling of grain boundaries in CdTe Associated funding: LDRD Other Systems: NERSC (500,000 core hours) Stampede (50,000 core hours) Science: The efficiency of polycrystalline CdTe solar cells has been below their theoretical limits [1-3] due to three primary factors: Shockley-Read-Hall (SRH) [4, 5] recombination at grain boundaries and within grains, difficulty in doping, and difficulty in forming ohmic and transparent contacts. A major uncertainty in poly-CdTe is a careful atomistic understanding of grain boundaries and their effects. Much insight on the role of grain boundaries can be gained by combining DFT calculations with experiments as to why certain behaviors are observed in poly-CdTe and why cells are far from their theoretical limit. With underlying mechanisms revealed, new techniques can be developed which may eliminate recombination centers and allow for high-performance CdTe devices. The goal of this project is to determine defect and impurity thermodynamics and their effects on electronic structures at grain boundaries in CdTe. Atomic structure models of grain boundaries in CdTe will be built based on STEM images, and structural relaxation and energetics analysis will be performed using density functional theory (DFT) calculations. 1. C. Ferekides and J. Britt, Solar Energy Materials and Solar Cells 35, 255 (1994). 2. X. Wu, Solar Energy 77, 803 (2004). 3. W. Shockley and H. J. Queisser, Journal of Applied Physics 32, 510 (1961). 4. W. Shockley and W. T. Read, Jr., Physical Review 87, 835 (1952). 5. R. N. Hall, Physical Review 87, 387 (1952). Project description: The proposed research aims to investigate simplified model structures of representative grain boundaries in CdTe, towards a detailed understanding of complete grain boundary networks present in poly-CdTe solar cells. We are going to screen the created grain boundaries based on their interface energies and will carry out detailed electronic structure analysis for the best candidate grain boundary structures within different categories. The electronic structures of the lowest-energy models will be also investigated using hybrid functionals, which give more accurate values of the band gaps and band edges than local and semilocal functionals. Site- and angular-momentum projected DOS analysis will allow us to determine the atomic orbitals from which mid-gap states arise, and suggest strategies for passivation. For possible passivants, the thermodynamics of incorporation in grain boundary structures, and effects on mid-gap states, will also be studied. In this way preferred, GB orientations can be found, as well as passivation techniques for given GB orientations. This will allow passivation techniques to be developed to exploit the behavior and encourage doping while reducing recombination. All DFT calculations will be carried out using a parallel plane wave code Vienna Ab Initio Simulation Package (VASP), which is already available in LCRC resources. Accordingly, our proposed work can be listed as: i) DFT relaxations on grain boundaries to find lowest energy structures. For 4.8º tilt (110)||(110) grain boundary, each full atomic relaxation does approximately take 20 hours using 512 processors. We are estimating to carry out 5 such calculations for three types of dislocation cores, which would take 20 hours *512 processors * 5 trials * 3 cores = 153,600 core hours. ii) For 2.0º tilt (111)||(111) grain boundary, each atomic relaxation takes approximately 20 hours using 128 processors. We estimate to do 20 trials to obtain the accurate atomic structure at the interface, such calculations, which would take 20 trials * 128 processors * 20 hours=51,200 SUs. iii) Possible passivants at 4.8º tilt (110)||(110) grain boundary will be considered for both the three types of dislocation cores. We are planning to try 5 different dopants at each dislocation core. Each atomic relaxation and electronic structure calculation is expected to take 20 hours using 512 processors. For each dopant we will also consider two different charged states. As a result the time requested for these calculations will be: 5 dopants * 3 dislocation cores * 3 charge states * 10 hours *512 processors = 230,400 SUs. iv) Hybrid functional calculations of selected systems. 5 system * 100 hours * 128 cores = 64,000 core hours. Total computation time requested: 450,000 core hours. Industry partnership: Project URL: Requested allocation: 450000 Q1: 112500 Q2: 112500 Q3: 112500 Q4: 112500 Justification: Storage requirements: 1 TB The requester has used undetermined amount hours of their initial startup project. In addition to approving an initial amount, please specify a Category and Subcategory for this project. For a list of the current selection of approved categories, please see: https://wiki.lcrc.anl.gov/wiki/Processes/Categories Once the Allocation committee has approved the project, please go to the Project Management page to create it: https://accounts.lcrc.anl.gov/projects.php Thank You, The LCRC Accounts System

1 0

[LCRC Accounts] Project Request: perovskite_scat
by accounts＠lcrc.anl.gov 07 Sep '16

07 Sep '16

Hello, A new project on the LCRC cluster has been requested. Please forward the information on to the LCRC Allocation sub-committee. Applicant's name: Sridhar Sadasivam Applicant's institution: ANL Applicant's division: NST Project Name: perovskite_scat Project title: Electron-phonon and phonon-phonon scattering in lead halide perovskites Associated funding: Department of Energy (LDRD) Other Systems: Carbon cluster, Center for Nanoscale Materials Science: In this proposal, we seek to understand the energy cascade mechanisms between electrons, optical and acoustic phonons in methylammonium lead halide (MAPbI3) perovskites. Organic-inorganic halide perovskite materials are promising candidates for third-generation photovoltaics. Perovskite solar cells are manufactured using inexpensive solution-based techniques [1], and their energy conversion efficiency has increased from 3.8% in 2009 to 19% presently [2], with enormous potential for further improvements. One of many remarkable properties of these materials is the slow relaxation of high-energy charge carriers that could allow their use in highly efficient hot carrier solar cells [3]. However, intuition regarding the dynamics of energy cascade processes in perovskite materials is still lacking, and predictive first-principles calculations of the scattering strength between energy carriers are scarce. In this proposal, we seek to perform first-principles calculations under the density functional theory (DFT) framework to predict electron-phonon and phonon-phonon scattering times in methyl ammonium lead iodide perovskites. Electron-phonon scattering calculations will be performed under the framework of density functional perturbation theory (DFPT) with a Wannier interpolation of electron-phonon matrix elements to dense electron and phonon wavevector grids [4-6]. Phonon-phonon scattering calculations will be performed using a real-space supercell approach where finite differences of forces due to atomic displacements will be used to obtain the third-order interatomic force constants [7]. The proposed work is expected to provide a fundamental understanding of energy transfer processes between electrons and phonons in halide perovskite materials, and also provide insights that could guide experimentalists to engineer materials for hot-carrier solar cell applications. References: 1. M.M. Lee, J. Teuscher, T. Miyasaka, T.N. Murakami, H.J. Snaith, Efficient Hybrid Solar Cells Based on Meso-Superstructured Organometal Halide Perovskites, Science, 338, 643-647, 2012 2. H. Zhou, Q. Chen, G. Li, S. Luo, T. Song, H. Duan, Z. Hong, J. You, Y. Liu & Y. Yang, Interface engineering of highly efficient perovskite solar cells, Science, 345, 542-546, 2014 3. Y. Yang, D.P. Ostrowski, R.M. France, K. Zhu, J. van de Lagemaat, J.M. Luther & M.C. Beard, Observation of a hot-phonon bottleneck in lead-iodide perovskites, Nature Photonics, 10, 53-59, 2015 4. M. Bernardi, D. Vigil-Fowler, C.S. Ong, J.B. Neaton & S.G. Louie, Ab Initio Study of Hot Electrons in GaAs, Proceedings of the National Academy of Sciences, 112, 5291-5296, 2015 5. F. Giustino, M.L. Cohen & S.G. Louie, Electron-phonon interactions using Wannier functions, Physical Review B, 76, 165108, 2007 6. J. Noffsinger, F. Giustino, B.D. Malone, C-H. Park, S.G. Louie & M.L. Cohen, EPW: A program for calculating the electron-phonon coupling using maximally localized Wannier functions, Computer Physics Communications, 181, 2140-2148, 2010 7. A. Togo, L. Chaput & I. Tanaka, Distribution of phonon lifetimes in Brillouin zones, Physical Review B, 91, 094306, 2015 Project description: The proposed work involves two primary tasks as outlined below. Both these calculations will involve use of the planewave DFT code Quantum Espresso with fully relativistic pseudopotentials for Pb and I atoms. The inclusion of spin-orbit coupling requires the use of large memory nodes and the 64 GB nodes on the Blues cluster are expected to be ideal for such calculations. 1. Computation of electron-phonon scattering rates: The first step in the calculation of electron-phonon scattering is the computation of phonon bandstructure along with the variation in self-consistent potential with respect to atomic displacements. Both these calculations will be performed under the framework of DFPT and is already implemented within the phonon module of Quantum Espresso. All DFT calculations would also involve spin-orbit coupling, since relativistic effects are important to consider for the heavy Pb and I atoms. Considering a 12 atom cubic unit cell of MAPbI3, the number of DFPT calculations required for a 4x4x4 phonon wavector grid is 36. Preliminary tests on the Blues cluster indicate that dynamical matrix calculations at a single phonon wavevector on 1 node (16 cores) takes approximately 150 hours. Hence, a full phonon dispersion calculation over 36 wavevectors will require a total time of approximately 36 x 16 x 150 = 86,400 hours. The second part of the computations will involve the use of Electron-Phonon Wannier (EPW) code to interpolate electron-phonon matrix elements from the coarse grid used in DFPT calculations to dense grids needed for convergence of electron and phonon self-energies. EPW calculations will need to be parallelized using the same number of processors used in the computation of phonon dispersion. The time needed for Wannier interpolation scales linearly with the number of grid points at which the electron-phonon matrix elements need to be obtained. Assuming that four calculations on different grid densities will be needed for studying the convergence of electron-phonon scattering rates, a total time of about 20,000 hours is expected to be sufficient for calculation of electron and phonon scattering self-energies. 2. Computation of phonon-phonon scattering rates Unlike the electron-phonon scattering rates that were obtained using DFPT, the phonon-phonon scattering rates will be computed using a real-space supercell approach. Assuming a 2 x 2 x 2 supercell of 96 atoms, along with a 5 Angstrom cutoff distance for third-order force constants, the number of supercell DFT calculations needed is approximately 5200 (after considering symmetries of the crystal structure). Preliminary tests on the Blues cluster indicate that each supercell calculation requires about two hours on 2 nodes (32 cores). Hence the computation of phonon-phonon scattering rates is expected to require about 5200 x 32 x 2 = 332800 hours. The estimated time required for the various aspects of the project are summarized below: 1) DFPT calculations of phonon dispersion: 86400 core hours 2) EPW calculation of electron-phonon scattering rates: 20000 core hours 3) Phonon-phonon scattering rate calculations using real-space supercell approach: 332800 core hours The total requested time is 440,000 core-hours. Dr. Sridhar Sadasivam and Dr. Pierre Darancet will be responsible for carrying out the simulations. Industry partnership: Project URL: Requested allocation: 440000 Q1: 100000 Q2: 150000 Q3: 100000 Q4: 90000 Justification: Storage requirements: The requester has used undetermined amount hours of their initial startup project. In addition to approving an initial amount, please specify a Category and Subcategory for this project. For a list of the current selection of approved categories, please see: https://wiki.lcrc.anl.gov/wiki/Processes/Categories Once the Allocation committee has approved the project, please go to the Project Management page to create it: https://accounts.lcrc.anl.gov/projects.php Thank You, The LCRC Accounts System

1 0

[LCRC Accounts] Yearly Allocation Request for MRSA
by accounts＠lcrc.anl.gov 07 Sep '16

07 Sep '16

Hello, A yearly allocation for the LCRC cluster has been requested with the following updated information: Submitter/PI: Nicholson Collier Project Name: MRSA Division: DIS Project title: Agent-based Modeling of Community-Associated Methicillin-Resistant Staphylococcus Aureus Epidemiology Associated funding: NIH Other Systems: Science: This project is developing an agent-based model of the spread of community-associated methicillin-resistant Staphylococcus aureus throughout Cook County. Each agent represents an individual person in a synthetic population. The scientific objective is to better understand the epidemiology of the disease transmission. Current scenarios simulate 2.9 million people and 1.2 locations throughout Cook County for 10 years on an hourly basis. We have generalized the simulation to other infectious diseases transmitted by human-to-human contact with the development of the chiSIM framework. Project description: This project is using Repast for High Performance Computing (HPC (http://repast.sourceforge.net) Repast HPC C++ agent-based modeling system that is designed for use on large computing clusters and supercomputers. Several papers and presentations have been produced so far. Among these are: David, Michael Z., Robert S. Daum, Michael J. North, Nicholson Collier, Vanja M. Dukic, Philip Schumm, James A. Evans, Duane T. Wegener, Jocelyn R. Wilder, Charles M. Macal, Diane S. Lauderdale. 2015 (Draft). Estimating the Contribution of Urban Jails to the Spread of Community-Associated MRSA, 2001-2010, using an Agent-Based Model of Chicago. Presented chiSIM talk at "Big Question: Enhancing community resilience to traumatic events" workshop at the University of Chicago on 8/2/16 (http://big.uchicago.edu/projects/enhancing-community-resilience-to-traumati…) Collier, N.T., J. Ozik, and C.M. Macal. Large-scale ABM with RepastHPC: A Case-study in Parallelizing a Distributed ABM, in PADABS 2015 Proceedings. Vienna, Austria, 24-28 August 2015 (in press). Macal, C.M., M.J. North, N.T. Collier, V.M. Dukic, D.T. Wegner, M.Z. David, R.S. Daum, P. Shumm, J.A. Evans, J.R. Wilder, D.S. Lauderdale, L.G. Miller, and S.J. Eels, “Reducing Transmission of Community-Associated Methicillin-Resistant Staphylococcus aureus: A Dynamic Agent-based Simulation,” Journal of Translational Medicine (2014). Macal et al., et al., “Modeling the Spread of Community-Associated MRSA,” Proceedings of the 2012 Winter Simulation Conference, C. Laroque, J. Himmelspach, R. Pasupathy, O. Rose, and A.M. Uhrmacher, eds., IEEE/ACM, Berlin, FRG (December 2012). We are currently refining the jail working paper mentioned above, experimenting with different simulated infection and contact scenarios within the jail. The new allocation will allow us to execute these scenarios. Industry partnership: Project URL: Current FY Hours Used: undetermined amount New FY Requested allocation: 160000 Q1: 40000 Q2: 40000 Q3: 40000 Q4: 40000 Justification: Storage requirements: Thank You, The LCRC Accounts System

1 0

[LCRC Accounts] Yearly Allocation Request for cavitation-modeling
by accounts＠lcrc.anl.gov 07 Sep '16

07 Sep '16

Hello, A yearly allocation for the LCRC cluster has been requested with the following updated information: Submitter/PI: Sibendu Som Project Name: cavitation-modeling Division: ES Project title: Computation Fluid Dynamics Modeling of Internal Flow of Diesel injectors. Associated funding: DOE - Office of Vehicle Technologies Other Systems: CMT, Convergent Science Inc. Science: This project will use a Computation Fluid Dynamics (CFD) software called CONVERGE to model the in-nozzle transients due to needle movement in a piezo-direct-acting diesel injector. The simulation results will be corroborated with experimental data. Project description: A new injector technology was developed a few years ago. It consists of controlling the needle motion directly with the displacement of a piezo-stack. This technology adds one degree of freedom to control the injected mass in to the engine, for example, the shape of the rate of injection can be modified (‘boot’ shape) and the needle opening/closing speeds can be also precisely controlled. The influence of nozzle geometry and needle dynamics on cavitation phenomenon, the spray formation and the combustion efficiency has been reported in the literature for conventional solenoid or piezoelectric driven injectors. In those cases, the needle lift could not be varied independently of injection conditions. Under the framework of ‘cavitation-modeling’ project, simulations of this new injector technology have been performed to prove that Computational Fluid Dynamics (CFD) is a predictive tool also when this new degree of freedom is added. Thus, they can b e used to investigate the relationship amongst the needle position (defined by the voltage applied to the piezo-stack), the shape of the rate of injection and the nozzle flow characteristics. Cavitation often plays a significant role in this type of injectors because very low needle lift values can be achieved for all injection conditions, generating large pressure drop at the needle seat, so cavitation phenomenon takes place also there and not only at the orifice inlet. Furthermore, the maximum needle lift is smaller than in conventional injectors (less than half). Thus, the discharge coefficient of the nozzle decreases, turbulence levels increase and velocity distribution at the nozzle exit is modified. It is known that cavitation patterns also vary with needle wobble, which is significant for these injectors due to their short needle lifts. In the following year efforts will be made to further optimize the settings of the numerical framework to ensure faster turn-around t ime and higher accuracy. The focus will be to capture the SOI trends in response to the applied piezo voltage and the resultant needle movement. Following the completion of the in-nozzle simulations, the results of these simulations will be applied as inputs for spray simulations to verify the effectiveness of the in-nozzle simulation to capture the spray characteristics in a more physically accurate manner. We achieved significant progress in both these parts in FY16 and hope to continue working on FY17 with the LCRC allocation. Industry partnership: Convergent Science Inc. Project URL: http://www.transportation.anl.gov/engines/multi_dim_model_home.html Current FY Hours Used: undetermined amount New FY Requested allocation: 450000 Q1: 125000 Q2: 125000 Q3: 100000 Q4: 100000 Justification: Storage requirements: 5 TB Thank You, The LCRC Accounts System

1 0

[LCRC Accounts] Yearly Allocation Request for cfd_enginemodeling
by accounts＠lcrc.anl.gov 07 Sep '16

07 Sep '16

Hello, A yearly allocation for the LCRC cluster has been requested with the following updated information: Submitter/PI: Sibendu Som Project Name: cfd_enginemodeling Division: ES Project title: Modeling of coupled in-nozzle flows and spray formation of GDI fuel injectors Associated funding: DOE – Office of Vehicle Technologies Other Systems: Science: Modeling of Gasoline Direct Injection spray formation is a relatively new area of research compared to the direct injection diesel systems. The challenges posed in GDI systems require extensive scientific investigation in terms of model development as well optimization of the model parameters. Rigorous Eulerian-Eulerian and Eulerian-Lagrangian spray simulations will be carried out using the CONVERGE computational fluid dynamics (CFD) code. Project description: The project is divided in two parts: In the first part, high fidelity in-nozzle Eulerian simulations will be carried out for fixed and moving needle scenarios under flashing and non-flashing scenarios. Different turbulence modeling approaches, such as RANS (standard and RNG k-ε) and Large Eddy Simulation (LES) will be carried out to capture the transient flow features during the Start-of-Injection (SOI) and End-of-Injection (EOI) periods. Optimization study involving different numerical techniques will undergo to ensure faster turn-around time yet accurate predictions. Promising moving needle simulation results have been obtained in the preliminary studies carried out in the current year. In the second part, the in-nozzle simulation data will be provided as boundary and initial conditions for running Lagrangian spray simulations with modified spray-models accounting additional physics due to superheated condition which currently does not exist in the mo deling framework. For non-flashing conditions encouraging agreement in prediction has been achieved when compared with the experimental data. However, model developments are required for flashing scenarios for capturing the effect of phase change phenomenon inside the nozzle orifices. Effort will be made to consider this aspect. We achieved significant progress in both these parts in FY16 and hope to continue working on FY17 with the LCRC allocation. Industry partnership: none Project URL: http://www.transportation.anl.gov/engines/multi_dim_model_home.html Current FY Hours Used: undetermined amount New FY Requested allocation: 1000000 Q1: 250000 Q2: 250000 Q3: 250000 Q4: 250000 Justification: Our recent publication (J. Kodavasal, K. Harms, P. Srivastava, S. Som, S. Quan, K.J. Richards, M. Garcia, “Development of stiffness-based chemistry load balancing scheme, and optimization of I/O and communication, to enable massively parallel high-fidelity internal combustion engine simulations,” Journal of Energy Resource Technology; JERT-16-1022, 2016) together with MCS and Convergent Science discusses the improvements to the Converge tool that has resulted in significant improvement in scaling. Currently, we are able to scale the code up to 4096 processors on Mira with about 70% scaling efficiency for a fixed mesh size. This was achieved due to the implementation of: (1) MPI I/O, (2) Improved Communication, (3) METIS load balancing scheme, (4) Development of a new chemistry load balancing scheme. Storage requirements: 5 TB Thank You, The LCRC Accounts System

1 0

[LCRC Accounts] Yearly Allocation Request for spray-modeling
by accounts＠lcrc.anl.gov 07 Sep '16

07 Sep '16

Hello, A yearly allocation for the LCRC cluster has been requested with the following updated information: Submitter/PI: Sibendu Som Project Name: spray-modeling Division: ES Project title: Modeling of Flash Boiling of single component and blended surrogate fuels for Gasoline Direct Injection Applications Associated funding: DOE-FOA award with UIUC Other Systems: UIUC Science: Blending conventional hydrocarbon-based fuels with ethanol is an increasing trend these days to cope with the increment in the green-house gas emissions as well as the depleting fossil fuel reserves. Ethanol contains approximately 33% less energy per unit volume compared to pure gasoline, but higher octane rating of ethanol provides leverage by operating at higher compression ratio (using turbocharger). Increment of ethanol content is one of the directives of Department of Energy (DOE) and Environmental Protection Agency (EPA). This project will use a Computation Fluid Dynamics (CFD) software called CONVERGE to model phase change phenomenon to capture the effects of different levels of volatility of blended fuels depending on the fuel composition. Modified spray-models will be developed to take into account the effect of fuel composition variations. Project description: The project is divided in two parts: In the first part, sector simulations in Eulerian framework will be carried out to capture the different flashing tendencies of different fuels using single component (iso-octane and ethanol) and blended (varying % of ethanol by volume) gasoline surrogates. Flash boiling is a common phenomenon in GDI systems, where a high temperature fuel undergoes bulk vaporization in a short span of time when subjected to superheated thermodynamic conditions in the combustion chamber. The current study uses an Eulerian multiphase mixture framework in conjunction with homogenous relaxation model for modeling phase change in pure and blended fuels. A pseudo-single component is used to represent the physical properties of blends (varying proportions of ethanol and iso-octane). The volatility of blended components is sometimes higher than the constituent species due to the inherent non-equilibrium state of the mixture. This strange beha vior has led to the prediction of enhanced flashing for specific blends compared to the individual species and the other blends in the current numerical study. The varied levels of flashing tendencies depending on fuel composition therefore, helps in further optimization of spray and combustion research in GDI systems. In the second part, the in-nozzle simulation data will be used to run Lagrangian spray simulations with modified spray-models accounting for blended fuel effects. The additional physics due to superheated condition and multi-component mixture will be incorporated in the sub-models, which currently does not exist in the modeling framework. We achieved significant progress in both these parts in FY16 and hope to continue working on FY17 with the LCRC allocation. Industry partnership: None Project URL: http://www.transportation.anl.gov/engines/multi_dim_model_home.html Current FY Hours Used: undetermined amount New FY Requested allocation: 1000000 Q1: 250000 Q2: 250000 Q3: 250000 Q4: 250000 Justification: Our recent publication (J. Kodavasal, K. Harms, P. Srivastava, S. Som, S. Quan, K.J. Richards, M. Garcia, “Development of stiffness-based chemistry load balancing scheme, and optimization of I/O and communication, to enable massively parallel high-fidelity internal combustion engine simulations,” Journal of Energy Resource Technology; JERT-16-1022, 2016) together with MCS and Convergent Science discusses the improvements to the Converge tool that has resulted in significant improvement in scaling. Currently, we are able to scale the code up to 4096 processors on Mira with about 70% scaling efficiency for a fixed mesh size. This was achieved due to the implementation of: (1) MPI I/O, (2) Improved Communication, (3) METIS load balancing scheme, (4) Development of a new chemistry load balancing scheme. Storage requirements: 5 TB Thank You, The LCRC Accounts System

1 0

[LCRC Accounts] Project Request: CrysGrowKMC
by accounts＠lcrc.anl.gov 07 Sep '16

07 Sep '16

Hello, A new project on the LCRC cluster has been requested. Please forward the information on to the LCRC Allocation sub-committee. Applicant's name: Dongwei Xu Applicant's institution: ANL Applicant's division: MSD Project Name: CrysGrowKMC Project title: Kinetic Monte Carlo simulation on the crystal thin film growth Associated funding: DOE Other Systems: Science: Understanding growth of films on surfaces of various crystal orientations is essential to technologies such as GaN-based light-emitting diodes and high-power electronics. Kinetics Monte Carlo (KMC) simulation is a powerful method to simulate the film growth. There are several fundamental questions are worthy to explore, i.e., the temperature effect in the competition of atom diffusion and deposition during the growth, what affects the island shape, how to connect the step edge barrier to the behavior of the steps and what is the correlation between the island during the growth. The KMC simulation will help to understand these questions. By scaling the simulation parameters according to the experiments, we can give a connection between the experiment and KMC simulation. Our simulation results will help to design and guide experimental studies. Project description: The project started from the beginning of 2015 and we have performed calculations using Kinetic Monte Carlo simulation on the thin film growth on c- and m-planes of GaN. Next, we will study the time correlation of the surface step evolutions at equilibrium and during the growth. We will also study the situation when evaporation and condensation are both included in the model. The Stochastic Parallel PARticle Kinetic Simulator (SPPARKS) will be used to do the simulation. SPPARKS is an open source code developed at Sandia National Lab. It is installed at LCRC and we can modify the code to install in our own directory as needed. The code can only ran in one single core when the atom deposition is included. However, we have written a parallel script to run 16 (or any other) single jobs at the same time using the same node. The computation time is size and parameters dependent. The typical running time is 15 hours for one simulation condition and there are about 64 different situations for both c- and m-planes. Matlab, which is also installed at LCRC, is used to do the analysis. We can only use single core right now and it is very memory consuming (sometimes, we need to use BIGgpu). We plan to include the evaporation in our simulation, which is more time consuming. For each configuration, we need about 25 hours of run time when using 16 cores. Therefore, expected time estimate for simulation with evaporation is (16 cores/node * 25 hours of run time *64*2) 512,00 core-hours. We would also study the steps and the time correlation for both c- and m-planes, which needs about 102,400 core-hours. The time needed for the matlab program is about (16 cores/node * 3 hours of run time *64*3*2) 18,432 core-hours. We also want to do some tests on larger system sizes, which adds about 20,000 core-hours. The total time requested is about 200,000 core-hours. There will be two to three members in the project. Industry partnership: Project URL: Requested allocation: 200000 Q1: 40000 Q2: 60000 Q3: 60000 Q4: 40000 Justification: Storage requirements: 20TB. Every single output is a very large data file (about 1.8GB) that will be used to analysis later. We need at least 16 runs for one condition and there are 64*2*3=384 situations. The total will be about 11TB. We will also calculate some larger system which will have lager output files. The requester has used undetermined amount hours of their initial startup project. In addition to approving an initial amount, please specify a Category and Subcategory for this project. For a list of the current selection of approved categories, please see: https://wiki.lcrc.anl.gov/wiki/Processes/Categories Once the Allocation committee has approved the project, please go to the Project Management page to create it: https://accounts.lcrc.anl.gov/projects.php Thank You, The LCRC Accounts System

1 0

[LCRC Accounts] Yearly Allocation Request for UNEDF_Optimization
by accounts＠lcrc.anl.gov 07 Sep '16

07 Sep '16

Hello, A yearly allocation for the LCRC cluster has been requested with the following updated information: Submitter/PI: Jason Sarich Project Name: UNEDF_Optimization Division: MCS Project title: Optimization for the NUCLEI SciDAC-3 Associated funding: DOE ASCR/NP/NNSA SciDAC-3 project NUCLEI DOE ASCR SciDAC Institutes FASTMath and SUPER Other Systems: cetus.alcf.anl.gov -- limited time for development and testing Science: The NUCLEI (NUclear Computational Low-Energy Initiative) project will build upon recent successes in large-scale computations of atomic nuclei to provide results critical to nuclear science and nuclear astrophysics, and to nuclear applications in energy and national security. The large-scale computations we envision will transform the fields of low-energy nuclear physics and astrophysics. Physics topics to be addressed include nuclear interactions and their uncertainties, ab-initio studies of light nuclei and their reactions and of nucleonic matter and its astrophysical properties. We will also fundamentally advance the studies of neutron-rich nuclei and the fission of heavy nuclei, and the key nuclear physics issues in neutron stars and tests of fundamental symmetries. Project description: We are building upon our optimization successes under the SciDAC-2 UNEDF (Universal Nuclear Energy Density Functional) project in the successor project NUCLEI. In this project, optimizations over parameter spaces corresponding to energy density functional parameters will be performed using the Argonne-developed POUNDERS optimization algorithm. We will work with DFT codes HFBTHO and HFODD and begin work with ab initio codes such as MFDn. We will also be performing sensitivity analysis and exploration of the design space, we have recently implemented load-balancing for the efficient computation of simultaneous parameter sets on thousands of nodes. We expect to use 120k core-hours for this task. For our parameter optimizations, we typically have 10-20 parameters and require on the order of 300 simulations. Based on the current simulation time, we would thus require 120 8-core nodes for up to 6 hours, or 6k core-hours for each version of the functional we want to optimize. We expect the number of core-hours will remain the same when using the new 16-core nodes. Past experience has let us to expect to do roughly 10 such optimizations in FY17, for a total of 60k core-hours for this task. POUNDERS is available in the Toolkit of Advanced Optimization (TAO), but we still prototype specialized versions of the algorithm using MATLAB/Octave on a single, master node. We will test updates to the TAO version on Blues and compare the performance and correctness against the MATLAB/Octave versions. As in the past, we will have roughly 4 users for this project. Industry partnership: Project URL: http://computingnuclei.org Current FY Hours Used: undetermined amount New FY Requested allocation: 180000 Q1: 45000 Q2: 45000 Q3: 45000 Q4: 45000 Justification: Storage requirements: Thank You, The LCRC Accounts System

1 0

[LCRC Accounts] Yearly Allocation Request for SRF-materials
by accounts＠lcrc.anl.gov 07 Sep '16

07 Sep '16

Hello, A yearly allocation for the LCRC cluster has been requested with the following updated information: Submitter/PI: Peter Zapol Project Name: SRF-materials Division: MSD Project title: Modeling of Impurity Structures in Metals Associated funding: DOE-HEP Other Systems: ANL CNM - Carbon Science: The objective of this project is to model the effects of chemical impurities and interface properties on the materials. While initial focus was on materials used to construct superconducting radio-frequency (SRF) cavities, we have also included impurities in other metals. In particular, in addition to hydride formation in Nb, we investigate hydride formation in Pd to shed light on common mechanisms. Niobium SRF cavities are a key technology for high-performance linear particle accelerators, and will be used in high energy physics applications such as a proposed 8 GeV proton source at Fermilab, and potentially in accelerators for waste management from nuclear power sources, for synchrotron light sources, and for neutron sources. SRF cavity performance is characterized by the accelerating gradient and the quality factor (a measure of the stored energy versus the dissipated power), both of which are highly sensitive to the structure and composition of the top ~100 nm of a cavity’s surface. Absorption of chemical impurities during processing can affect both the maximum accelerating gradient and the quality factor. In addition, materials synthesis and processing affects these properties. Two examples, which will be the focus of this project, are (i) the formation of a surface oxide film, approximately 20 nm thick, which is composed of layers of Nb2O5, NbO2, NbO, and a gra dient of dissolved oxygen in the niobium bulk; and (ii) the formation of hydride precipitates. The niobium oxides can accommodate substantial off-stoichiometry, which results in some cases in local magnetic moments. This is detrimental for superconductivity because magnetic impurities create normal conducting states within the superconducting energy gap. Hydride precipitation is detrimental because the hydrides with an ordered structure are not superconducting above 2 K. On the other hand, the presence of interstitial oxygen, nitrogen, and carbon may be beneficial to SRF cavity performance at low concentrations. A steep drop-off of the quality factor at the onset of the RF field can be observed in cavities made from high purity niobium, but not typically in cavities from lower purity reactor grade niobium. This is because oxygen, nitrogen, and possibly carbon can trap hydrogen atoms, which prevents their diffusion and precipitate formation. The tetragonal and cubic niobium nitrides also have superconducting transition temperatures that are higher than that of pure niobium, so formation of these phases may potentially benefit niobium SRF cavities. In addition, different nitride phases form interfaces that can have either beneficial or detrimental properties. Understanding of these interfaces is important for both Nb and metals. Project description: This project will utilize VASP to model the interactions of chemical impurities in niobium and other metals to investigate the formation of the phases and to shed light on processes that may positively or negatively affect the performance. Our previous studies focused on chemical impurities in Nb. This year the project will utilize VASP to model the interfaces of different nitride phases and other metals. This phase of the project will focus on surface and interface properties. We are particularly interested in the change in properties at surfaces and across the interfaces between the nitrides, hydrides and other materials. Since the crystal lattices may deform considerably near the interfaces, we will consider bulk models of under strain, as well as full interface models. The models will consist of up ~100 atoms. Density functional theory in VASP will be used to optimize the geometries of the models and calculate their electronic properties. Phonon DOS and dispersions will be evaluated using Phonopy, which uses the force constants calculated for atomic perturbations via density functional theory in VASP. Since defects, including interfaces, reduce the symmetry of the systems, many perturbations will need to be performed for some of these models to accurately assess the phonon properties. Abinit will also be used to calculate the superconducting properties (electron-p honon coupling and transition temperature) of select structures. Accurate determination of the electron-phonon coupling is very computationally demanding, requiring dense k-point and q-point grids. Abinit parallelizes over k-points, bands, and perturbations; so it is particularly suitable for use on large machines with many cores. Industry partnership: Project URL: Current FY Hours Used: undetermined amount New FY Requested allocation: 800000 Q1: 200000 Q2: 200000 Q3: 200000 Q4: 200000 Justification: This project will primarily use the density functional theory (DFT) implemented in electronic structure code VASP. The system sizes of our calculations ranges from 100 to 300 atoms. VASP has been tested to scale very well in parallel processing on Blues. The optimum number of processors used for these systems is 16-64 processors per calculation. Storage requirements: Thank You, The LCRC Accounts System

1 0

[LCRC Accounts] Yearly Allocation Request for madman
by accounts＠lcrc.anl.gov 07 Sep '16

07 Sep '16

Hello, A yearly allocation for the LCRC cluster has been requested with the following updated information: Submitter/PI: Angel Yanguas-Gil Project Name: madman Division: ES Project title: Modeling the advanced manufacturing of energy materials Associated funding: This effort is supported through LDRD and EERE funds Other Systems: None Science: This project aims at developing multiscale predictive models for advanced manufacturing applications, specifically nanomanufacturing and additive manufacturing. Our research has two goals: 1) improve the current understanding of the fundamental processes controlling the synthesis and microstructure of materials, 2) develop models that can predict the scale up of manufacturing process as well as the performance of the resulting materials. Project description: Goals ===== The goal of this project is to develop predictive models for advanced manufacturing applications, specifically nanomanufacturing and additive manufacturing as part of three funded activities supported by DOE's EERE office (Sunshot program) and through Argonne's LDRD program. Project overview ================ The scope of this project covers the following three research lines: Multiscale simulations of ALD on high surface area materials ------------------------------------------------------------ ### Main approach One of the main applications of thin film deposition techniques such as ALD is the coating of high surface area materials. We have developed a simulation tool to couple reactive transport at a reactor scale and inside high surface area materials for the coating and functionalization of high surface area materials. Our reactor scale component of our simulation code is based on OpenFOAM, the coupling between reactor scale and the nanoscale taking place at the boundary condition. Reactive transport inside high surface area materials is carried out using a combination of approaches: - We are using LAMMPS to model the growth of high surface area materials based on nanoparticle spheres, using a molecular dynamic approach to model the settling of nanoparticles in solution through a combination of interatomic potentials, viscous drag and an external force that forces the sedimentation. Typical number of particles involve ranges between 20000 - 100000 spheres. - For nanoparticle based substrates, we then calculate the view factors using kinetic Monte Carlo simulations. This is an embarrasingly parallel problem that can take advantage of multiple cores available in the cluster, and our prototype simulation takes advantage of the parallel programming capabilities built-in in julia to share the load between cores. - The resulting view factors are then incorporated into a model that treats the reactive transport of species as an absorbing Markov Chain. This provides the probability that particles react at a certain depth of our high surface area material. - For deterministic structures such as circular pores or rectangular trenches, view factors are computed analytically and directly fed into the Markov chain model. All these components were tested in Blues during the prior cycle. ### Applications We are currently applying this model to two separate cases: #### Predictive scale up of the synthesis of nanostructured materials using roll-to-roll compatible gas-phase manufacturing techniques. In particular, we are looking at the impact of microstructure and reactivity on the spatial ALD of high surface area materials. #### Conformal coating of large size tubes using self-limited and non self-limited processes for nuclear engineering applications. Models of additive manufacturing -------------------------------- ### Main approach We are developing physics-based models to simulate the material and energy flow during part processing through additive manufacturing. Our focus here is in developing coarse-scale models in the time and length scale of the actual printing of parts. Our approach combines finite volume approach through OpenFOAM with the development of custom coarse-scale simulations that model that incorporate the rastering of the nozzle or laser in a similar way as it is carried out in a 3D printing process. This will allow us to understand the impact of rastering sequence on the heat load seen at every point of the part as it gets printed and correlate these values with the microstructure of 3D printed parts. These models will then be used as an input for reduced order models that will pair down the complexity of the model even further as part of our approach to bring more advanced simulations of the 3D printing of materials to industry. Project members =============== We expect three people working on this project. Allocation ========== We are asking for 60,000 core hours for the next fiscal years. Of these, we expect that 10% will be used in runs involving single-core processes running concurrently, 30% will involve single node calculations, with the remaining extending through multiple nodes. Industry partnership: Project URL: Current FY Hours Used: undetermined amount New FY Requested allocation: 60000 Q1: 15000 Q2: 15000 Q3: 15000 Q4: 15000 Justification: Storage requirements: Thank You, The LCRC Accounts System

1 0