August 2012 - Allocations-admins - mailman3.cels.anl.gov

[LCRC Accounts] Yearly Allocation Request from parvis
by accounts＠lcrc.anl.gov 28 Aug '12

28 Aug '12

Hello, A yearly allocation for the LCRC cluster has been requested with the following updated information: Submitter/PI: Robert Jacob Project Name: parvis Division: MCS Project title: Parallel Analysis Tools for Ultra-Large Climate Data Sets Associated funding: DOE BER Other Systems: NCAR, NERSC, ALCF, OLCF Science: Like many scientific applications, climate models rely on numerical methods to solve their equations. But insights about the simulated climate are revealed only after performing additional calculations on several years to decades of the multivariate, multidimensional time series output by the climate model. The tools currently used to perform the climate data analysis and visualization (DAV) are serial and can no longer handle the data sizes being generated. We will be developing a new parallel version of a widely used climate DAV system called NCL which will speed up the climate science workflow for researchers throughout the community. Project description: NCL (the NCAR Command Language) is an interpreted scripting language that allows Matlab-like operations on arrays, can read many climate and weather data formats and provides special functions for plotting climate and weather quantities over maps of the earth. NCL produces publication-quality output. To parallelize NCL, we will use two existing parallel libraries. MOAB, the Mesh Oriented Database, contains datatypes and methods for representing unstructured grids in parallel. Intrepid is part of the Trilinos system and is a library for computing operators acting on discretized fields. MOAB and Intrepid both use MPI for their parallelism and MOAB, Intrepid and NCL are written in C++. We will be developing new code to interface NCL with MOAB/Intrepid. Along with developing a parallel version of NCL, we are also investigating the use of task parallelism using the Swift language. We are converting current c-shell scripts that analyze climate data into Swift scripts that can perform the steps in parallel. We are also substituting some of the serial commands in the scripts with parallel versions so the resulting script will be both task- and data- parallel. Are goal is for a scalable DAV tool that can perform all I/O and calculation in parallel while still rending publication quality 2D plots. If we distribute the data evenly among processors in parallel, then all time-domain calculation will be embarrassingly parallel while most spatial calculation will need only nearest neighbor exchanges. Our requested time will be use to test and time the scaling of our software as it is developed. Project URL: http://trac.mcs.anl.gov/projects/parvis Current FY Hours Used: undetermined amount New FY Requested allocation: 4000 Q1: 1000 Q2: 1000 Q3: 1000 Q4: 1000 Justification: Thank You, The LCRC Accounts System

1 0

[LCRC Accounts] Yearly Allocation Request from MD_Potential_Oxide
by accounts＠lcrc.anl.gov 28 Aug '12

28 Aug '12

Hello, A yearly allocation for the LCRC cluster has been requested with the following updated information: Submitter/PI: Milind Malshe Project Name: MD_Potential_Oxide Division: CSE Project title: Synthesis Science of Functional Layered Complex Oxides Associated funding: Strategic Initiative on Materials for Energy (LDRD-2012-188-R1) Other Systems: NERSC: ~ 400000 core-hours Science: The principal objective of this project is development of new, more flexible and more accurate many-body potentials for the description of interatomic interactions in various inhomogeneous systems and use of these potentials in large scale molecular dynamics (MD) and Monte Carlo (MC) simulations of the synthesis and properties of new functional materials such, e.g., as layered complex oxides. The results of the proposed studies will aid the ongoing collaborative experimental effort at MSD, APS, CNM and CSE in defining synthesis schedules and conditions – types of elements, temperature, pressure, etc. - that will result in materials with the desired properties and functionalities. We will also explore the properties of new materials as a function of the chemical identity and number/thickness of layers, strain, and nature and concentration of defects, including the mechanisms of formation of defects (e.g., as induced by temperature or pressure). Project description: Initial efforts are devoted on exploring the structural and dynamical/thermal properties of surfaces and thin films of complex layered oxides using empirical potentials. We focused on the study of strontium titanate (STO) as a paradigm. This choice was motivated by the synthesis and characterization work on this system currently underway within our experimental component. Surface relaxations in a thin film of STO are analyzed and the role of epitaxial strain arising due to mismatch between the lattice constants of the film and substrate material is being investigated. Large-scale MD runs are being performed using the parallelized version of the DL_POLY and LAMMPS packages. The high scalability of these packages with the system size is well documented. The number of atoms in our planned simulations will vary from a few tens of thousands to hundreds of thousands and possibly a few millions. These MD runs will be complemented by the state-of-the art densit y functional theory (DFT) calculations on these systems. The planned DFT computations will be carried out with the VASP and Quantum Espresso packages. Both the packages exhibit high parallelization scalability for the electronic structure computations needed for this project. Concurrently, we are also pursuing the development of new, robust, flexible, and more accurate potentials for mimicking the interatomic interactions in layered complex oxide materials. The effort will be pursued in two directions. The first aims at incorporation of variable charges and polarizabilities into the existing potentials, thereby generalizing their applicability for simulating defects in oxide materials which are known to play an important role in governing materials properties. The second is a principally new approach based on the neural networks (NN) methodology and evolutionary (specifically, genetic algorithms (GA)) techniques. A particularly attractive aspect of the NN approach is that no a priori assumption about the functional form of the potential is made, which results in a much higher degree of flexibility and robustness. The codes NN and GA based potentials will be developed and implemented in MATLAB. An additional algorithmic advantage of the NN and GA m ethodologies is that they are inherently amenable to high degree of parallelization. Work on this aspect and the related methodological developments will be an important part of the planned activities. The number of project members is 4. Project URL: http://blogs.anl.gov/major_initiatives/materials-for-energy/ Current FY Hours Used: undetermined amount New FY Requested allocation: 480000 Q1: 120000 Q2: 120000 Q3: 120000 Q4: 120000 Justification: The allocation request is based on MD test runs of 2 ns for a thin film of SrTiO3 consisting of 20,000 atoms performed with the DL_POLY package on Carver at NERSC: Cores Wall time (s) 8 38880 16 31629 32 24843 64 16282 128 8698 256 3895 (A better parallelization scaling is achieved for this system, as evaluated over runs of 2 ns in duration, with a larger number of cores.) Thank You, The LCRC Accounts System

1 0

[LCRC Accounts] Yearly Allocation Request from 1atomCat
by accounts＠lcrc.anl.gov 28 Aug '12

28 Aug '12

Hello, A yearly allocation for the LCRC cluster has been requested with the following updated information: Submitter/PI: Ujjal Das Project Name: 1atomCat Division: MSD Project title: Supported Single-atom Catalysts for Activation of Small Molecules, including Alkanes Associated funding: BES Chemical Sciences Other Systems: NA Science: Supported single-atom catalysts show exceptionally high catalytic activity compared to metal surfaces and sub-nanometer metal clusters. This high activity is attributed to large surface area of an isolated atom as well as its low coordination number. Standard techniques have been devised to prepare thermally stable single-atom catalysts using atomic layer deposition (ALD). In brief, supported single-atom catalysts show potential to activate very difficult bonds, including the ones in alkanes, with very high turn-over frequency. However, very little is known about the background working principles of such single-atom catalysts. For example, how the nature of bonding of the single-atom affects its catalytic activity, how local charge density varies for different single-atom types, the role of a support, and an atomic level understanding of the mechanism of activation of small molecules. All these information are useful to control the activity and selectivity of the c atalyst as well as to establish a structure-function relationship to understand the way such catalysts work. We plan to obtain all these information using high-level quantum chemical calculations for silica supported single-transition-metal catalysts. Project description: 1. The computational methods and programming models remain same from our previous submission and have been described below: Density functional theory (DFT) calculations will be performed to determine structure, bonding, and reaction energies of supported single-atom catalysts. Periodic boundary condition (PBC) will be imposed in order to maintain periodicity of the solid support. B3LYP hybrid functionals will be used in general, however, we may also switch to gradient corrected density functionals (GGA), where, B3LYP is either very expensive or not applicable. The Nudged Elastic Band (NEB) method of Henkelman and coworkers will be used to calculate transition states and activation barriers. Surface phonons will be calculated for accurate estimation of reaction free energies. Bader charge analysis and NBO population analysis will be performed to obtain information about local charge density and orbital electron density of the single-atom catalyst. 2. Software requirement mostly remains same. We request one additional program (Lammps): (a) VASP 5.0 or later (b) Gaussian-09 (c) Matlab (d) Lammps Additional Comments: We have problems running parallel VASP-5.2.11 on Fusion. This has been notified to LCRC support staff. We request updates from them on this issue. 3. Performance achieved: We have made significant progress since the allocations were made on June, 2012. We have developed accurate cluster models representing amorphous silica surface. Using these models, we have successfully identified the defect sites on dehydroxylated silica. The defect sites mostly contain strained three- and four-membered rings of Si-O bonds. Our calculated surface Raman spectra of the defect sites match well with the experimental results obtained by our collaborators. Using the cluster models, we have also identified the binding sites for metals on the silica surface. In general, the metals prefer to bind at the defect sites forming metal-O bonds. The binding process is highly exothermic as it leads to the opening of the strained rings. So far, we have completed structure evaluations for iron (FeIII) and zinc (ZnII). Future studies will focus on Ni, Co, and Cu. For iron, we have calculated different vibrational modes of silica adsorbed Fe(III) atom and found that only the sym metric Fe-O stretching modes, in different combinations, appear in the experimentally observed UV-resonace Raman spectra. This is a new and exciting observation and will be published soon. 4. Future Research: In the FY-2013, we will mainly study catalytic activity of different silica supported single-atoms. We will begin with hydrogenation and dehydrogenation of propylene by Zn. First, we will identify all the intermediates in the catalytic cycle. Their thermodynamic stability will be assessed from the heat of formation energy. Activation barriers will be computed to see if the formations of these species are kinetically feasible. We will also verify if the intermediates play any role in poisoning the catalyst. CO oxidation is another important reaction often used to study catalytic activity. We will identify the CO binding sites on Zn and develop a complete mechanism of CO oxidation to CO2. The role of the support in this process will be investigated in details. CO oxidation and hydrogenation will be studied for Fe(III) also. In summary, FY-2013 resources will be used mainly for the following research activities: 1. Zn as a catalyst (hydrogenation/CO oxidation) 2. Fe as a catalyst (hydrogenation/CO oxidation) 3. Identify structures of Ni/Co/Cu on silica 4. Identify the role of support on catalysis 5. Requests and justifications: Energy optimization using VASP takes approximately 1000 CPU hours. Hessian (second order energy derivative) evaluation takes roughly 1500 CPU hours. Optimization and Hessian evaluation using Gaussian take approximately the same time for model clusters, 150 CPU hrs. For each quarter, we estimate to run 50 geometry optimizations and 25 Hessian calculations using VASP, and 200 Gaussian calculations, a total of 117500 CPU hrs. Therefore, our total request for FY-2013 is 470000 CPU hrs. The PI of the project is Dr. Ujjal Das. Dr. Larry Curtiss, a Distinguished Fellow at the Argonne National Lab, is a co-PI and the contact person for this project (curtiss(a)anl.gov) We will notify if a new member joins this project or if additional resources are required. Project URL: Current FY Hours Used: undetermined amount New FY Requested allocation: 470000 Q1: 117500 Q2: 117500 Q3: 117500 Q4: 117500 Justification: Thank You, The LCRC Accounts System

1 0

[LCRC Accounts] Yearly Allocation Request from md_pls_dft_for_TiO2
by accounts＠lcrc.anl.gov 28 Aug '12

28 Aug '12

Hello, A yearly allocation for the LCRC cluster has been requested with the following updated information: Submitter/PI: Handan Yildirim Project Name: md_pls_dft_for_TiO2 Division: CNM Project title: First Principles and Classical Molecular Dynamics Calculations of Lithium Intercalation into TiO2 Surfaces and Nanostructures Associated funding: BES Other Systems: Carbon 150 000 Science: First principles calculations based on Density Functional Theory (DFT) and classical Molecular Dynamic (MD) simulations (using polarizable potentials) will be performed to study diffusion and intercalation of Li in TiO2 crystalline and amorphous nanoparticles and nanotubes of several sizes. These simulations are directly relevant to the exploration of the properties of titania as an alternative electrode for energy storage for Li ion batteries. Our calculations will be the extension of our earlier study at which Li diffusion and intercalation characteristics are derived for bulk TiO2 polymorphs. The effect of Li concentration on the diffusion energetics and dynamics in the nanostructures will be studied for better understanding of their electrochemical performance. We will compare our results obtained those for the bulk anatase, rutile and amorphous TiO2 to extract the role of nanostructuring and morphology on the diffusivity of Li ions. Additionally, the structural transitions upon certain concentration of Li intercalation into these structures will be evaluated, and compared to that observed in the amorphous TiO2. We will also extend our study to explore pressure-induced structural transitions and role of nanostructuring on the transition pressure. The present study requires detailed analysis of the energetics, dynamics of Li in TiO2 under various concentration and temperature conditions. For these calculations, we will vastly benefit from the ability of classical MD simulations that allow simulation of several thousands of atoms. We will explore the size effects on Li intercalation properties. We will use MD simulations (for temperatures ranging form 300K to 900K) for a long enough time (~ 5ns at the current studies) to explore the details of the Li diffusion mechanisms in both crystalline and amorphous TiO2 nanoparticles and nanotubes for concentrations ranging from 25% to 100%. The observed mechanisms of diffusion will be supplie d as an input to calculate the corresponding energetics and dynamics using periodic Density Functional Theory calculations. Project description: In our preliminary simulations, we have obtained insights into the diffusivity of Li in the TiO2 polymorphs including amorphous (bulk) and nanostructures. The concentration effect, mechanisms for diffusion in different polymorphs, and host morphology effects on the diffusivity of Li ions are explored by comparing the results obtained for anatase, rutile and amorphous bulk TiO2 and anatase and amorphous nanostructures. The structural as a function of Li concentration are studied extensively. The structural properties and the diffusivity of Li ions in the new crystal phase are explored in details. The results of MD simulations at relatively low temperatures are analyzed to identify Li diffusion mechanisms in these systems, and the obtained mechanism is used to calculate the corresponding isolated Li diffusion energetics using DFT. Our MD results, in summary, showed that Li ions diffuse with different mechanism at each system that is mostly determined by the host lattice structure. We could also trace clearly the effect induced by nanostructuring on the diffusivity of Li ions. We performed classical MD simulations to study Li diffusion energetics and dynamics under several conditions using the DL-POLY code. We have gained significant insights into the role of concentration and morphology on the diffusivity of Li ions in the bulk TiO2 polymorphs. Additionally, phase transition with high Li load and under certain pressure range is observed with these studied for nanosecond time scales. These simulations are indeed computationally intensive as the systems have large number of atoms. Our calculations for this part of the study are performed with a very fine time step (~0.2 fs in general), requiring long simulation hours even for 5ns (due to the type of the potential used). For the calculations of nearly 5000 to 10000 atoms for simulating 5ns, 25 to 50 millions simulation steps are needed. As our simulations extend to several temperature and concentration ranges, the need for intensive computational time can be understood. For the proposed project on nanoparticles and tubes, we have tested the reliability of the potential by studying different sizes of nanoparticles ranging from 2nm to 10nm. The detailed structural analysis and the diffusivity are explored for each Li concentration in the small nanoparticles up to 6nm. Concentration dependent diffusivity profile of Li ions are extracted and compared to those obtained for bulk TiO2. We now would like to extend these simulations and test the diffusivity profile and structural transitions for larger nanoparticles particularly for disordered nanoparticles, and those particles under pressure. Each simulation will be conducted for at least 1 ns and expected not to exceed 5ns. As the size of the particles are fairly large, significant computational resources are needed. For instance, in order to complete the calculations for 6nm nanoparticle for 5 concentrations and 6 temperatures, we required of about 80,000 cpu hours. We anticipate that we will be r unning of about 10 to 15 jobs per day to complete the proposed research. Project URL: Current FY Hours Used: undetermined amount New FY Requested allocation: 400 Q1: 150 Q2: 150 Q3: 50 Q4: 50 Justification: Thank You, The LCRC Accounts System

1 0

[LCRC Accounts] Yearly Allocation Request from ESMF
by accounts＠lcrc.anl.gov 28 Aug '12

28 Aug '12

Hello, A yearly allocation for the LCRC cluster has been requested with the following updated information: Submitter/PI: Silverio Vasquez Project Name: ESMF Division: MCS Project title: Earth System Modeling Framework Associated funding: DOD, NASA, NSF, NOAA Other Systems: external: NERSC, NCAR, ORNL, NCSA, NASA Science: The Earth System Modeling Framework (ESMF) enables the coupling of model components in high performance Earth system modeling codes. ESMF component interfaces have been implemented in most of the prominent climate and weather codes in the U.S.: CCSM, WRF, NASA GEOS-5, EMC GFS/NEMS, NRL COAMPS and the GFDL MOM4 ocean. In general, componentization with ESMF has been implemented at the level of major physical domains, where simulated interactions require inter-component data communications (e.g. atmosphere, ocean). This implementation is compatible with codes such as WRF that impose their own interoperability conventions at the level of individual physics parameterizations. The codes listed above, plus the full GFDL modeling system under FMS, can now operate within a component-based, hierarchical architecture in which large-scale components can be called as subroutines. This was not the case when ESMF began. Over the years ESMF has evolved and now can be used in multiple ways: 1) to create interoperable component-based modeling systems; 2) as a source of remapping, time management, metadata handling, and other functions that can be used independently of the component constructs; and 3) as a file-based offline generator of interpolation weights for many different kinds of grids. This tool reads in two grid files in NetCDF format and outputs weights for interpolation between the two grids. The recent technical focus has been on continued development of ESMF grid classes and a regridding capability. These are built on a base of two flexible data representations that were constructed in previous years: an unstructured finite element mesh library and a general array class designed to support fields on logically rectangular grids. At this point ESMF can produce conservative and non-conservative interpolation weights, in parallel or serial, in 2D for logically rectangular grids or meshes composed of triangles or quadrilaterals and in 3D for logically rectangular grids or meshes composed of tetrahedra or hexahedra. Project description: The goal of the Earth System Modeling Framework project is to build high performance, flexible software infrastructure to increase ease of use, performance portability, interoperability, and reuse in climate, numerical weather prediction, data assimilation, and other Earth science applications. The framework is intended for use by individual researchers as well as major operational and research centers. ESMF products include an API specification, a reference implementation accompanied by extensive documentation, and an exhaustive regression test suite that includes 4600+ unit tests, system tests, and examples. The ESMF software consists of a superstructure for coupling and exchanging data between models components (e.g., atmosphere, ocean) and an infrastructure consisting of 1) data structures for representing grids and fields and 2) an optimized, portable set of low-level utilities. The data constructs and low-level utilities can be used by the coupling superstructure and may also be used separately to compose scientific applications. As computing architectures evolve, optimizations of the ESMF software is ongoing. Future changes include updates for codes running on computers with general purpose GPUs or accelerators, where data movement in and out of device memory is the source of the major performance bottleneck. Accelerators provide increased computational power which can be leveraged by ESMF in the intensive portion of the code, such as the grid remapping algorithms. However, the main focus in ongoing performance optimization of ESMF is to ensure that the software scales readily and does not introduce overhead as codes start to routinely run on hundreds of thousands of processors. In addition to ESMF full Fortran 90 and partial C/C++ interfaces, a prototype ESMF-Python (ESMP) interface has been implemented. ESMP provides support for Grids, Meshes, Fields, and Field regridding. The regridding capability includes bilinear, higher-order patch recovery and first-order conservative regridding methods and the choice to ignore unmapped points. The ESMP is being used in the Ultrascale Visualization - Climate Data Analysis Tools 1.0, which receives primary support from the Department of Energy. Fault-tolerant extension between ESMF Components interfaces has been implemented. The objective is to allow ensemble drivers running in different executables to complete a run in a controlled manner even after the other side of the connection has become unresponsive. Project URL: http://www.earthsystemmodeling.org Current FY Hours Used: undetermined amount New FY Requested allocation: 17750 Q1: 4000 Q2: 4250 Q3: 4500 Q4: 5000 Justification: Thank You, The LCRC Accounts System

1 0

[LCRC Accounts] Yearly Allocation Request from Climate_Emulator
by accounts＠lcrc.anl.gov 28 Aug '12

28 Aug '12

Hello, A yearly allocation for the LCRC cluster has been requested with the following updated information: Submitter/PI: Robert Jacob Project Name: Climate_Emulator Division: MCS Project title: Building a climate emulator for use in energy policy analysis Associated funding: NSF Other Systems: Science: State-of-the-art general circulation models (GCMs) provide the best estimates of future climate change. While they are valuable tools for assessing impacts of climate change for a given scenario of future greenhouse gas emissions, they cannot be used in iterative economic models since a single multi-century simulation can take weeks to months to run and the iterative models require hundreds of thousands of iterations. For this reason, we aim to develop a statistical emulator that will accurately and efficiently reproduce GCM forecasts of regional climate change for an arbitrary trajectory of CO2 concentrations by using a library of pre-computed GCM output. Library based statistical emulation has not been possible to this date because there is no consistent and comprehensive library of climate model runs corresponding to a range of scenarios of future CO2 concentrations. Computing time on Fusion will enable us to build this unique resource by performing a series of s imulations using the Community Climate System Model (CCSM3). Since library-based statistical emulation is currently not possible, the most common approach to emulating regional climate uses linear pattern scaling, in which changes in regional temperature and precipitation are assumed to be linearly proportional to the change in global mean temperature (GMT). That is, regional climate is effectively assumed to depend solely on GMT, and not on the rate or duration of warming. Our preliminary analysis of CCSM3 output from the Climate Model Intercomparison Project (CMIP3) archive shows that this assumption is not valid for temperature projections over time scales of multiple centuries and breaks down for projections of precipitation even over decadal time scales. The CMIP3 archive, however, is not sufficient for full statistical emulation, since it is both restricted in number of runs and inconsistent in its treatment of regional climate forcing. The carefully designed library we propose here will be used to develop and test a number of alternative approaches for emulating climate model responses, including (i) nearest neighbor interpolation, (ii) a variant of pattern scaling that accounts for the rate of warming, and (iii) principal component analysis (PCA). The intended outcome of this research is a new climate emulator that will aid in the evaluation of energy policies for greenhouse gas mitigation. We anticipate that there will be a wide interest in a user-friendly, computationally tractable emulation tool from the policy, economics, and climate communities. The library itself will also be made public for further use by the community Project description: This request is for the third year of a continuing project on the study and emulation of transient climates – climate after a change in CO2 or other radiative forcing but before the 1000s of years need to reach a new equilibrium. The primary goal of the project has been to build tools to facilitate analysis of climate-related policies. Fusion runs enabled us to pioneer the technique of library-based climate model emulation, in which a library of pre-computed model runs is used to generate a statistical tool that reproduces the output of complex models in a fraction of the time it takes to run the full model. We used Fusion computing time in FY2011 to develop a prototype climate library from runs of CCSM3 (the open-source Community Climate System Model 3 built by the National Center for Atmospheric Research). We demonstrated climate emulation on a subcontinental scale in FY2011 and on at model grid scale in FY2012 (2 papers in preparation) The model runs have also facilitated much synergistic science on the physics of transient climates, including demonstration of the cause of nonlinearities with global mean temperatures for the evolution of both precipitation and global heat uptake. This work directly addresses major outstanding questions in climate science. (1 submitted paper, 2 in preparation). In FY2012 we used much of our Fusion allocation to do several very long (multi-1000-year) climate runs with different forcing scenarios (different sizes or CO2 increases and one long run with solar forcing). Those runs were not discussed in our FY2012 request but proved necessary for the science that emerged from the project, and the timeliness and urgency of this work prompted a reordering of priorities, as these issues are urgent concerns in climate science at present and Fusion computing resources gave us the possibility of answering them. In FY2013 we would like to return to the project we discussed in our FY2012 request, to extend our emulation and study of climate model behavior to cases where climate model parameters are themselves varied. We will continue to work with a relatively low spatial resolution climate model version (CCSM3 at T31, i.e. 3.5ox3.5o resolution) to permit testing more cases and performing longer runs. The dominant computational requirements in CCSM3 are its atmosphere and ocean components. The atmosphere component is the Community Atmosphere Model, which currently supports three different numerical cores for solving the basic equations: a spectral transform method, a Semi-Lagrangian scheme, and a finite-volume scheme. The ocean component is a finite-difference model called the Parallel Ocean Program and was developed at Los Alamos. CCSM3 requires Fortran90 and C compilers and the MPI and NetCDF libraries to run. The study we will return to is the exploration of the importance of parametric uncertainty on climate model projections (especially in parameters relating to cloud coverage and water content, which have been shown to be key drivers of uncertainty in climate projections). The only analogous previous work is the “climateprediction.net” project which used community-donated computing time over many years to do a similar experiment on a higher-resolution version of a British climate model. Lowering the resolution to T31 allows us to be competitive in terms of physical insights with more modest computing demands. In FY2013 we will perform T31 CCSM3 simulations with three values for each of three cloud parameters (nine combinations). For each combination we will spin-up the climate using a 500 year control run and a 140 historical forcing run (1870-2010). Based on results from our FY2011 simulations, we will require three scenarios of future CO2 concentrations from 2010-2160, with three realizations of each scenario, in order to “train” our statistical model. To test the emulator we will require another two scenarios, but will consider only one realization of each. The total requirements for this experiment are 9 * (500 + 140 + 3*150*3 + 2*150) = 21,000 model years. The T31 resolution version of CCSM3 requires 38 core hours per model year, which gives a total of 21,000*38 ~= 800,000 core hours. In FY2012 we requested and received a total allocation of 800K hours, received in two increments, and used all of it. We also received an emergency allocation of 300K hours on August 2; this was a response to the loss of 20% of our climate library in a PADS failure and allowed us to recalculate the runs that were lost. We have approximately 100K left from that allocation and will use all of it by the end of the fiscal year. We have been storing data on the Argonne/UChicago PADS system, which is currently both near capacity and near shutdown. We are therefore in the process of moving to UChicago’s Reseach Computing Center (RCC), which will give us continuing sufficient disk space to store the output offsite. Project URL: Current FY Hours Used: undetermined amount New FY Requested allocation: 800 Q1: 200 Q2: 200 Q3: 200 Q4: 200 Justification: Our allocations so far have allowed us to perform thousands of years of simulations using CCSM3 at T31. This provides us with detailed statistics on the time required to run this model configuration on Fusion. Using 8 nodes (64 processors) takes an average of 38 core-hours for a simulation of one model year. Several simulations can be run at the same time to scale the project out to the full machine if available. We successfully used over 1 million core-hours in FY12. CCSM3 was the recipient of a great deal of tuning and evaluation by combined DOE and NCAR computational scientists so we do not expect further improvements in the per node performance. Thank You, The LCRC Accounts System

1 0

New LCRC report submitted for CAT-engine-modeling
by support＠lcrc.anl.gov 28 Aug '12

28 Aug '12

Full project report at https://accounts.lcrc.anl.gov/reporting/project/2012/?project=CAT-engine-mo…

1 0

[LCRC Accounts] Yearly Allocation Request from CAT-engine-modeling
by accounts＠lcrc.anl.gov 28 Aug '12

28 Aug '12

Hello, A yearly allocation for the LCRC cluster has been requested with the following updated information: Submitter/PI: Sibendu Som Project Name: CAT-engine-modeling Division: ES, MCS Project title: Simulation of Internal Combustion Engines with High-Performance Computing Tools Associated funding: Other Systems: CATERPILLAR Cluster, CONVERGENT Science Cluster Science: The primary goal of this project is to study scaling issues using the commercial code CONVERGE with detailed chemistry. As a part of this exercise, complete 3-D simulation of a single cylinder diesel engine will be conducted. A series of chemistry models with increasing complexity (species and reactions) will be included. The objective is to find the optimum reaction mechanism and cores that will provide useful design information not obtained from 2-D and reduced chemistry models. Project description: Traditional Lagrangian spray modeling approaches for internal combustion engines (ICEs) are highly grid-dependent due to insufficient resolution in the near nozzle region. This is primarily because of inherent restrictions of volume fraction with the Lagrangian assumption together with high computational costs associated with small grid sizes. A state-of-the-art grid-convergent spray modeling approach was recently developed and implemented by the co-PIs in CONVERGE software. The key features of the methodology include Adaptive Mesh Refinement (AMR), advanced liquid-gas momentum coupling, and improved distribution of the liquid phase, which enables use of cell sizes smaller than the nozzle diameter. This modeling approach was rigorously validated against non-evaporating, evaporating, and reacting data from the literature. The current numerical study focused on further demonstration of the grid-convergent modeling approach for simulating a single-cylinder Caterpillar compression ignition engine. Grid-Convergence of several critical engine performance and emission parameters such as pressure, heat release rate, ignition delay, and NOx and soot emissions was investigated. The goal of this study is to recommend grid-settings for accuracy/runtime trade-off for industrial partners. The use of grid-convergent modeling approach is possible by using grid sizes smaller than the nozzle diameter (d). A minimum resolution of “~0.5d” resulted in a peak cell count of 50 million. HPC in such an environment is quite challenging due to stringent demands on the load balancing algorithm. Traditional load balancing approach in CONVERGE resulted in poor scalability beyond 50 processors for the problems investigated. This bottleneck was circumvented by implementing the METIS algorithm during this project. Improved load balancing was observed up to 256 processors for cases with minimum resolution of “~d” with a peak cell count of about 16 million. The METIS algorithm will be available in the next version release of CONVERGE, thus highly benefitting the CONVERGE user community. To the best of our knowledge this is a pioneering study in the field aimed at demonstrating both grid-convergence and high-scalability of ICE simulations. Project URL: http://www.transportation.anl.gov/engines/multi_dim_model_home.html Current FY Hours Used: undetermined amount New FY Requested allocation: 495000 Q1: 125000 Q2: 125000 Q3: 125000 Q4: 120000 Justification: Thank You, The LCRC Accounts System

1 0

[LCRC Accounts] Yearly Allocation Request from kinetic
by accounts＠lcrc.anl.gov 28 Aug '12

28 Aug '12

Hello, A yearly allocation for the LCRC cluster has been requested with the following updated information: Submitter/PI: Zeke Insepov Project Name: kinetic Division: MCS Project title: Kinetic Monte Carlo simulation of radiation defects evolution and thin film deposition Associated funding: RERTR AFCI Other Systems: Blue Gene, aniki Science: As a part of the continuing effort to study the microstructure evolution of Molybdenum (Mo) under irradiation of Xenon (Xe) Ions at different temperature regimes and different radiation doses, an on-lattice Kinetic Monte Carlo model is being built for that purpose. The first objective of this model is to investigate the bimodal gas bubble size distribution observed experimentally by simulating defects diffusion through the matrix during the so-called kinetic phase, in contrast to ballistic phase. The "ballistic phase" accounts for the creation of defects through cascade of collisions made by a primary knock-on atom (PKA), while the "kinetic phase", accounts for the diffusion-controlled dynamical evolution of such defects. The on-lattice model employs space discretization technique and hence allows the defect to exist only on a defined set of points in the space depending on the type of the defect. Normally, this set of points consists of certain interstitial sites, in addition to lattice sites. KMC algorithm is then used to simulate defects hops between these points. Understanding the mechanisms involved in defects diffusion process is so crucial, as different mechanisms,at the same temperature and defect population, could result in completely different behavior. The existence of thermally activated slipping systems in Mo, which can at finite temperature compete with the crowdion and <111> dumbbell diffusion mechanism, adds more complication to the simulation of interstitial diffusion. KMC gives the capability of isolating different mechanisms, in order to explore the contribution of individual mechanisms to a certain phenomenon. The first step focused on studying the embryonic stage (incubation and nucleation stage) of voids and followed by tracking the spatial arrangement and the void size distribution as a function of temperature and displacements per atom (dpa). The objective of this part is to investigate the possibility of void superlattice formation and the role played by hypothesized shadowing effect in this case. Project description: We will use the kinetic Monte Carlo and Metropolis Monte Carlo methods that were realized as a SPPARKS package at SANDIA Labs. SPPARKS is a parallel Monte Carlo code for on-lattice and off-lattice models that includes algorithms for kinetic Monte Carlo (KMC), rejection kinetic Monte Carlo (rKMC), and Metropolis Monte Carlo (MMC), coupling of this code with MD-Synchronous parallel kMC. The scaling properties of KMC solvers currently available in SPPARKS : linear search, O(N) tree search, O(logN) composition-rejection search, O(1) We plan to run 150-200 jobs of 256 cores x 5-10 hours. For validation and fortification of this KMC model, comparison with experimental results is needed. The in situ TEM study of Xe implantation in pure Mo, took place at ANL facilities, gives valuable data on Xe bubble size distribution in Mo, which can be used as a reference data to be compared with the results of this model. Accelerator experiments are planned to be conducted at Argonne IVEM and Atlas machines, to verify the simulation results. SPPARKS will run on any parallel machine that compiles C++ and supports the MPI message-passing library. This includes distributed- or shared-memory machines. We expect 4-5 users that will be working on this project. Project URL: Current FY Hours Used: undetermined amount New FY Requested allocation: 250000 Q1: 50000 Q2: 75000 Q3: 50000 Q4: 75000 Justification: Thank You, The LCRC Accounts System

1 0

[LCRC Accounts] Yearly Allocation Request from xenon
by accounts＠lcrc.anl.gov 28 Aug '12

28 Aug '12

Hello, A yearly allocation for the LCRC cluster has been requested with the following updated information: Submitter/PI: Zeke Insepov Project Name: xenon Division: MCS, NE Project title: Multi-scale simulation of radiation damage in nuclear fuels Associated funding: RERTR, AFCI, NEAMS, Argonne LDRD 2013-2015 Other Systems: Blue Gene/aniki Science: Summary of the project: The formation of defects in bcc Mo lattice as a result of 300 keV Xe ion bombardment will be studied via classical atomistic simulation molecular dynamics and kinetic Monte Carlo simulations, with an interatomic potential developed using the force-matching Ab initio based approach. Radiation defect formation via electronic and nuclear stopping power and evolution of the defects in the cascade will be described. Diffusion and interaction of Frenkel pairs, such as self-interstitials, vacancies, and clusters of defects will be analyzed. Stable forms of clusters of defects will be studied that show one-dimensional diffusion with a very high diffusion coefficient and that escape quickly to an open surface. Single vacancies have lower diffusivities and will not aggregate into large clusters. Large prismatic vacancy loop formation near the impact surface will be studied that may be considered as a result of fast recrystallization. The mobility of the vacancy dislocation loop segments will be found, the motion of the entire loops will be studied that strongly hinders the neighbor point defects. A new multiscale concept for irradiated materials simulation will further be developed that will be based on coupling molecular dynamics simulations (MD) where the potential was obtained from Ab initio data of energies of the basic defect structures, with kinetic mesoscale models. The evolution of a system containing self-interstitial atoms (SIAs) and vacancies in crystalline molybdenum will be investigated by means of MD. The kinetics of formation of a dimer clusters (built of two self-interstitial atoms) (di-SIA) and SIA–vacancy recombination will be analyzed via approaches used in the kinetic theory of radiation ageing. The effects of 1D diffusion of SIAs, temperature, and defect concentrations on the reaction rates will also be studied. This approach can validate both the kinetic mechanisms and the appropriate kinetic coefficients, offering the potential to significantly reduce the uncertainty of the kinetic methodology and providing a powerful predictive tool for simulating irradiation behavior of nuclear materials. Project description: The following calculations will be performed for interatomic potentials Mo-Mo, Mo-Xe, U/Mo/Xe in the form of EAM potential: 1. Choosing a structure fitting the potential, calculations of energies and forces on atoms in this configurations from DFT by VASP and potfit codes. 2. Creating the potential by force-matching software PotFit. 3. Verification of the U-Mo-Xe EAM potential obtained by comparison with the DFT data: formation energies of vacancies, self-interstitial atoms (SIA) and their clusters, energies of the basic structures of SIAs, their hierarchy. 4. Verification of the potential with the experimental data on diffusion of defects (diffusion constants, threshold temperatures to induce long-range migration) and production of defects in radiation cascades (structure of the defects and sizes). 5.Improvement of the reference structures to reduce the discrepancies revealed in 3 and 4. computational methods: Density Functional Molecular Dynamics simulations using the siesta method programming model: parallel experiments planned: Argonne IVEM and Atlas experiments, Electron Microscopy software requirements: C++, fortran compiler (e.g. openmpi) sizes of calculations run on and/or planned for LCRC or other systems: hundreds of nano-seconds time scale for classical MD simulations on 1e7-1e8 atomic systems expected number of project members: 7 We expect a total of 7 project members (two from University of Illinois, and several members from the New University of Astana and Russian Academy of Sciences. Results of assessments are expected to lead to joint proposals to NSF, DOE (Office of Science, and NE), and DARPA. Project URL: Current FY Hours Used: undetermined amount New FY Requested allocation: 300000 Q1: 75000 Q2: 75000 Q3: 75000 Q4: 75000 Justification: Thank You, The LCRC Accounts System

1 0