Allocations-admins

Download

[email protected]

August 2011

1 participants
12 discussions

[LCRC Accounts] Project Allocation Request
by accounts＠lcrc.anl.gov 03 Aug '11

03 Aug '11

Hello, A change in allocation has been requested: Requester: yuxwang (Yuxin Wang) Project: SOFC Title: Imaging, Analysis and Simulation of Heterogeneous Functional Materials Description: In the proposed work, a transmission x-ray microscope (TXM) with high resolution x-ray nanotomography will be used to obtain a detailed geometric description of SOFC electrodes at 30 nm resolution useing a tunable synchrotron light source at APS beamline 32-ID-C. The XCT technique is a non-destructive method used to image and reconstruct actual SOFC microstructures. X-rays penetrate the sample mounted on a rotation stage, then passes through a scintillation screen and captured by a camera. The sample is rotated through 180 degrees with sample projections produced at specified angle intervals and exposure times. Images are captured over a field of view of up to 100 um by using stitching methods and reconstructed into a 3-D volume of the pore structure. To achieve elemental sensitivity for Ni at the Ni-YSZ interface, the microscopy experiments were carried at 8.317 keV and 8.357 keV, corresponding to 16 eV below and 24 eV above the Ni K-edge (8.333 keV), respecti vely. Once the TXM data has been obtained, slice-by-slice reconstruction will be carried out to convert the image data to digitized 3D reconstructed volumes. This approach has been successfully validated. The digitized 3D geometry will be used as model input. The nanotomography data obtained require further processing before the microstructure of the samples can be fully characterized. Chromatic focusing of the TXM zoneplate lens during the measurements can cause non-negligible variation in the magnification of the sample over the energy range investigated. The images were scaled to eliminate this variation. The transmission images from two different energy levels were processed to isolate the Ni phases to produce two separate nanotomography data sets, one representing Ni and another phase. Each of these data sets was aligned on a central rotation axis, and reconstructed using the iterative algebraic reconstruction technique (i-ART) algorithm. The output from the reconstruction is two image stacks corresponding to each phase of interest. These stacks are processed separately with segmentation steps performed using the ImageJ software. After performing fine alignment to account for minor variations in stage positioning, the p rocessed stacks are merged to produce one final volumetric data set, with discrete identification of each phase. This volumetric data is used as an input to previously developed microstructural characterization codes. Image processing is a compute intensive task, but necessary to obtain useful structures and elemental/chemical information for quantum chemical simulations, atomistic simulations, and mesoscale phase-field and lattice Boltzmann simulations. Quantum chemical simulations and Quantum mechanical calculations It is the objective of the quantum chemical calculations for this project to rationally design new nano-structured heterogeneous functional materials from the "atom-up" and to identify the scientific relationship between structure and property/function. Quantum chemical simulations will be based on density functional theory (DFT) and transition state theory which permits connecting atomistic information with meso- and macroscopic information and therefore ultimately the simulation of materials at experimental time and length scales. For the anode electrode of the SOFC, key properties of interest are high electronic conductivity, oxygen ion mobility, tolerance to sulfur impurities in the fuel stream, and tolerance to carbon poisoning. For the cathode electrode, key properties are high electronic conductivity, oxygen ion mobility, and oxygen dissociation rate. We aim at identifying a minimal set of independent descriptors for these material properties that allows rational m aterial selection, particularly dopant selection. Unfortunately, no one quantum chemical method can calculate accurately all properties of interest. The workhorse method for condensed matter electronic structure calculations at present is DFT within the generalized gradient approximation (GGA) to electron exchange and correlation. And while the SOFC electrolyte, YSZ, is well described by DFT-GGA, many materials of interest for the anode and cathode electrode of the SOFC are reduced oxides with transition metal or lanthanide ions with open shell 3d- and 4f- electrons that are described incorrectly by DFT-GGA because their approximate exchange energies do not exactly cancel the spurious Coulomb repulsion of each open-shell d-electron with itself, leading to the so-called "self-interaction error (SIE)". The consequences of the inexact cancellation are not just quantitative but qualitative failure, e.g., with semiconductors sometimes predicted to be metals (e.g., NiO and FeO). This class of materials is referred to as strong ly correlated, and cannot be described properly in a static mean field theory, such as DFT. Although several first principles quantum mechanics methods of treating such materials exist, we will employ electronic structure methods that provide the correct physics at minimal extra cost beyond a conventional DFT calculation. These are the periodic electrostatic embedded cluster (PEEC) method with hybrid and double-hybrid density functionals and the DFT+U method, originally proposed by Anisimov, Lichtenstein and coworkers,[1-3] in which the spurious intra-atomic self-Coulomb repulsion of the open-shell d and/or f electrons is cancelled by using exact intra-atomic Hartree-Fock (HF) exchange, while the rest of the electrons are described within DFT. Materials of interest for the anode side of the SOFC are doped perovskite and double perovskite structures such as SrTiO3 and Sr2Fe1.5Mo0.5O6-δ. Double perovskite structures based on Mo can easily be doped with various transition metals, permitting a significant variation of the material properties. Materials of interest for the cathode side are gadolinium-doped ceria (Gd-doped CeO2 or GDC), lanthanum strontium cobaltate (La0.6Sr0.4CoO3 LSC, the Fe-doped analog (LSCF), Sr2Fe1.5Mo0.5O6-δ, and lanthanum strontium manganate (LSM). We will employ DFT+U based calculations to characterize the bulk and surface electronic structure and kinetic processes of the SOFC cathode materials LSCF and LSM. DFT+U requires as input a parameter that is the difference between the averaged Coulomb (U) and exchange (J) interactions for the electrons of a specific angular momentum localized on each atomic site. We use the method of Mosey, Liao, and Carter[4] for calculating U and J ab initio from unrestricted Hartree-Fock (UHF) calculations on electrostatically embedded clusters, so as to render the DFT+U method nonempirical. We are interested in improving oxygen ion transport in mixed electron-ion conductors such as LSCF and LSM by doping or alloying these parent materials. To predict the kinetics of oxygen diffusion through the SOFC cathode material, we will employ the nudged elastic band (NEB)[5] method to locate the minimum energy path for diffusion. This technique determines the transition state from which we can calculate the activation energy and, by calculating the vibrational frequencies associated with the minimum and transition state, the pre-exponential factor for the diffusion constant.[6-7] DFT with the GGA and LDA functionals also fail to accurately describe the bond energy of dioxygen. This failure occurs not only because of the SIE mentioned earlier but also the open-shell triplet ground state of O2 is not a pure spin eigenfunction of the single determinant wavefunction employed in DFT. To overcome this failure, we will employ embedded configuration interaction (ECI)[8] calculations. Here the system is broken into a small region of interest treated with a highly accurate quantum mechanical method (e.g., multi-reference single and double excitation CI, MRSDCI) and a background region described with a lower-level method (DFT+U or electrostatic embedding) that defines an embedding potential for the region of interest. In addition to providing an accurate description of oxygen bond dissociation energetics, this method allows for the explicit inclusion of an electron at the LSCF and LSM surface. In this way, we can model the electrochemical reduction processes occurring at the SOFC cathode. Using these methods, we aim to predict the kinetics of the different cathode processes which influence the overall performance of the SOFC. From our calculations, we will seek to understand how material composition affects the different kinetic parameters as well as determining which processes may serve as key descriptors of the overall cathode kinetics. Mesoscale Phase-field Simulations In order to address the issue of microstructural stability in SOFCs over extended operation times, a coarse-grained continuum approach is required. The so-called "diffuse-interface" or "phase-field" (PF) modeling framework, in which Prof. Mikko Haataja has extensive experience[12,13,14], has become very popular over the past several years as a tool to simulate evolving microstructures in multicomponent, multiphase materials. In this approach, several continuum fields ("order parameters") are first introduced to account for both the local phase (solid vs. pore) and composition/structure (Ni vs. YSZ). Then, a thermodynamic free energy, which accounts for the thermodynamics of bulk phases and relevant interfacial energies, is constructed. Finally, the dynamics of the order parameters are obtained from the free energy via variational principles, leading to coupled non-linear partial differential equations defined in the whole computational domain. One of the strengths of the PF method is that it conveniently handles complex topological changes occurring during the microstructural evolution process, while the numerical challenge is to integrate stiff PDEs over extended simulation times in a 3D computational domain. Lattice Boltzmann calculations Over the past decade, the lattice Boltzmann method (LBM) has been applied to solve an astonishingly large variety of problems in various disciplines, particularly at the mesoscale. LBM techniques are CPU and memory intensive. This often restricts the lattice resolution that can be used on standard cluster computers. Now that massively parallel computers such as the FUSION system are available, these CPU and memory issues can be addressed so problems that were previously intractable using the LBM can be brought within reach. The LBM scheme has been shown to scale well on large computing systems. Excellent scale-up was observed up to 1024 processors by Oliker et al. using a 512^3 lattice to study flow turbulence[15]. For a turbulence application, a sustained performance of 2 TFlops was achieved on 256 NEC vector CPUs. This indicates that the LBM is a very promising candidate for large scale computing platforms like FUSION. Mathematical modeling of transport phenomenon in a fuel cell is very useful for analyzing and optimizing fuel cells. For example, team member Prof. Wilson K. S. Chiu has used three-dimensional (3D) LBM models to investigate multi-component mass transport with electrochemistry and reformation in porous SOFCs[16-18]. The imaging technology required to reconstruct the 3D SOFC electrode geometry is available[19], and 3D LBM simulations would significantly increase the accuracy of the SOFC mass transport model. The goal of our research is to model the diffusive flow of three gaseous species (H2, H2O, and N2) through the porous anode structure of a SOFC. Our model uses a 3D representation of the porous anode, obtained from x-ray nanotomography at APS beamline 32-ID-C, with resolution as low as 30 nm[19]. Such analysis is often very expensive on scalar and smaller cluster systems and often restricts the resolution that can be used to model the problem of interest. However, now that massively parallel computers such as FUSION are available, these memory issues can be addressed and problems that were intractable using the LBM only a few years ago are within reach. Computational Detail - Resource Needs Image processing All image processing will be performed using Matlab with three toolboxes (image processing toolbox, parallel computing (version 2010b), and curve fitting). We will reconstruct an average of 20 samples per week requiring an average of 48 node hours per calculation (50000 node hours per year). We will also be performing additional image processing of each sample requiring approximately 30 node hours per calculation (30000 node hours per year). All calculations will require up to 100 GB of memory. Quantum chemical simulations and Quantum mechanical calculations Our DFT calculations will be performed using the Vienna Ab Initio Simulation Package (VASP)20,21 which solves the Kohn-Sham equations subject to periodic boundary conditions using a planewave basis set. We will perform an average of 40 single-point energy calculations each week requiring an average of 20 node hours per calculation (41600 node hours per year). We will also be performing an average of three nudged elastic band calculations each week requiring approximately 1200 node hours per calculation ( node hours per year). All PEECM calculations will be performed with the Turbomole code (based on Gaussian-type basis sets) using hybrid and double-hybrid density functionals. Transition states required for the calculation of reaction rates will be located with our own highly efficient growing string[22] and improved dimer[23] algorithms. Recent tests on EMSL Chinook confirmed that both VASP and Turbomole scale well up to hundreds of nodes. Overall, we expect that computations required for quantum chemical calculations on the anode side of the SOFC will require approximately 750,000 node-hours per year and 500 GB of memory. This estimate is based on submitting 150 geometry optimizations and 20 transition state searches per year for each the perovskite and double perovskite structures (optimization of a perovskite structure requires on FUSION about 5000 node hours; optimization for a double perovskite structure requires about 12,500 node hours; transition state searches require about 10 times mo re time than a structure optimization). Phase-field Simulations Our continuum PF simulations will be performed with a parallel MPI code written in-house. While we have not benchmarked this particular code, our experience with similar codes in the past suggests that it should scale well up to 32 or 64 CPUs on the FUSION system. Furthermore, we estimate that a simulation with 128^3 (256^3) grid points will require 240 (1920) node-hours wall clock time and 100 GB (800GB) of memory. On an annual basis, we expect that our simulations of SOFC anode microstructural evolution phenomena require on the order of 200,000 node hours and 1 TB of memory on the FUSION system, corresponding to 100 high-resolution simulations. Lattice Boltzmann Calculations Our LBM simulation of 3D mass transport and electrochemistry in solid oxide fuel cells scales well up to 64 CPU on a local SGI Altix 850 system. For the FUSION system, we estimate that a single case using low resolution 81^3 lattice will require 350 node-hours wall-clock time and 100 GB memory on the FUSION system. A high resolution 674^3 lattice will enable us to simulate realistic physical domain sizes and lead to better accuracy of the model predictions, but each case will require 22,400 node-hours wall-clock time and 1 TB memory on the FUSION system. Our annual estimated EFRC project needs on the FUSION system are 300,000 node hours: two low resolution lattice cases per week (36,400 node-hours wall-clock time per year), and one high resolution lattice case per month (268,800 node-hours wall-clock time per year). Summary of Computing Resource Needs (March 2011 to September 2011) Software Requirements Node Hours Storage GB Image Processing Matlab with toolboxes 30,000 100 Quantum chemical simulations Turbomole 140,000 500 Quantum mechanical calculations VASP 40,000 500 Phase-field simulations Intel C++ Compiler 45,000 1500 Lattice Boltzmann calculations Intel Fortran Compiler 45,000 1500 Total 300,000 3000 Matlab will require the (1) image processing toolbox, (2) parallel computing toolbox (version 2010b), and (3) curve fitting toolbox. References 1. V. Anisimov, J. Zaanen, O. K. Andersen, "Band theory and Mott insulators: Hubbard U instead of Stoner I", Physical Review B, vol. 44, pp. 943-954 (1991). 2. V. Anisimov, F. Aryasetiawan, A. I. Lichtenstein, "First-principles calculations of the electronic structure and spectra of strongly correlated systems: the LDA+U method", Journal of Physics: Condensed Matter, vol. 9, pp. 767-808 (1997). 3. S. L. Dudarev, A. I. Lichtenstein, M. R. Castell, G. A. D. Briggs, A. P. Sutton, "Surface states on NiO (100) and the origin of the contrast reversal in atomically resolved scanning tunneling microscope images", Physical Review B, vol. 56, pp. 4900-4908 (1997). 4. N. J. Mosey, P. Liao, E. A. Carter, "Rotationally invariant ab initio evaluation of Coulomb and exchange parameters for DFT+U calculations", Journal of Chemical Physics, vol. 129, pp. 014103 (2008). 5. H. Jo'nsson, G. Mills, K. Jacobsen, "Nudged elastic band method for finding minimum energy paths of transitions" in Classical and Quantum Dynamics in Condensed Phase Simulations, pp. 385-404, World Scientific, 1998. 6. K. A. Marino and E. A. Carter, "The effect of platinum on diffusion kinetics is beta-NiAl: Implications for thermal barrier coating lifetimes", ChemPhysChem, vol. 10, pp. 226, (2009). 7. K. A. Marino and E. A. Carter, "First-principles characterization of Ni diffusion kinetics in beta-NiAl", Physical Review B, vol. 78, pp. 184105 (2008). 8. P. Huang and E. A. Carter, "Self-consistent embedding theory for locally correlated configuration interaction wave functions in condensed matter", Journal of Chemical Physics, vol. 125, pp. 084102 (2006). 9. R. M. Shroll, and T. P. Straatsma, "Molecular Dynamics Simulations of the Goethite-Water Interface", Molecular Simulation, 29, 1-11 (2003) 10. R. M. Shroll, and T. P. Straatsma, "Molecular Basis for Microbial Adhesion to Geochemical Surfaces: Computer Simulation of Pseudomonas aeruginosa Adhesion to Goethite", Biophysical Journal, 84, 1765-1772 (2003). 11. T. P. Straatsma and J. A. McCammon, "Multiconfiguration thermodynamic integration", J. Chem. Phys., 95, 1175-1188 (1991). 12. S. Sreekala and M. Haataja, "Recrystallization kinetics: A coupled coarse-grained dislocation density and phase-field approach", Phys. Rev. B 76, 094109 (2007). 13. M. Haataja and F. Le'onard, "Spinodal decomposition in the presence of mobile dislocations", Phys. Rev. B 69, 081201 (R) (2004). 14. M. Haataja, J. Mueller, A. D. Rutenberg, and M. Grant, "Dislocations and morphological instabilities: Continuum modeling of misfitting heteroepitaxial films", Phys. Rev. B 65, 165414 (2002). 15. L. Oliker, A. Canning, J. Carter, C. Iancu, M. Lijewski, S. Kamil, J. Shalf, H. Shan, E. Strohmaier, S. Ethier, T. Goodale (2007) Scientific Application Performance on Candidate Petascale Platforms, International Parallel and Distributed Processing Symposium (IPDPS), March 24 – 30, 2007, Long Beach, CA. 16. A. S. Joshi, K. N. Grew, J. R. Izzo, Jr., A. A. Peracchio, and W. K. S. Chiu, "Lattice Boltzmann Modeling of Three-Dimensional, Multicomponent Mass Diffusion in a Solid Oxide Fuel Cell Anode," ASME Journal of Fuel Cell Science and Technology, vol. 7, pp. 011006-1-011006-8 (2010). 17. A. S. Joshi, K. N. Grew, A. A. Peracchio and W. K. S. Chiu, "Lattice Boltzmann Modeling of 2D Gas Transport in a Solid Oxide Fuel Cell Anode," Journal of Power Sources, vol. 164, pp. 631-638 (2007). 18. K. N. Grew, A. S. Joshi, A. A. Peracchio, and W. K. S. Chiu, "Pore-Scale Investigation of Mass Transport and Electrochemistry in a Solid Oxide Fuel Cell Anode," Journal of Power Sources, vol. 195, pp. 2331–2345 (2010). 19. K. N. Grew, Y. S. Chu, J. Yi, A. A. Peracchio, J. R. Izzo, Jr., F. De Carlo, Y. Hwu and W. K. S. Chiu, "Nondestructive Nanoscale 3D Elemental Mapping and Analysis of a Solid Oxide Fuel Cell Anode," Journal of the Electrochemical Society, vol. 157(5), (2010). 20. G. Kresse and J. Furthmueller, "Efficiency of ab-initio total energy calculations for metals and semiconductors using a plane-wave basis set", Computational Materials Science, vol. 6, pp. 15-50, 1996. 21. G. Kresse and J. Furthmueller, "Efficient iterative schemes for ab initio total-energy calculations using a plane-wave basis set", Physical Review B, vol. 54, pp. 11169-11186 (1996). 22. Peters, B., Heyden, A., Bell, A. T., Chakraborty, A., A growing string method for determining transition states: Comparison to the nudged elastic band and string methods. J. Chem. Phys. 120, 7877-7886 (2004). 23. Heyden, A., Bell, A. T., Keil, F. J., Efficient methods for finding transition states in chemical reactions: Comparison of modified dimer method and partitioned rational function optimization method. J. Chem. Phys. 123, 224101-14 (2005). Current: undetermined amount Justification: N/A Requested: 250000 A specific reason has been given: Allocated time depleted from high levels of use by several users from different institutions. This needs to be approved and the final allocation amount decided upon. Thank You, The LCRC Accounts System

1 0

[LCRC Accounts] Project Request: ScalaGAUSS
by accounts＠lcrc.anl.gov 01 Aug '11

01 Aug '11

Hello, A new project on the LCRC cluster has been requested. Please forward the information on to the LCRC Allocation sub-committee. Applicant's name: Jie Chen Applicant's institution: ANL Applicant's division: MCS Project Name: ScalaGAUSS Project title: Scalable Gaussian Process Data Analysis Associated funding: Scalable Gaussian Process Analysis of Spatio-Temporal Data (PI: Mihai Anitescu) Other Systems: MCS workstations Science: This project uses a maximum likelihood approach to perform Gaussian process analysis on both synthetic and real data. The observations considered for analysis are simulation and data from nuclear engineering and climate science. The resulting analysis is used to describe the spatial correlation of the sampled data, to predict and to understand uncertainty in very high fidelity phenomena. Project description: We model data as Gaussian process model using several covariance models, including a Matern covariance kernel. The goal of the computation is to estimate the parameters of the covariance kernel. We employ a maximum likelihood method, which is reformulated by using a sample average approximation approach. The novelty of the approach is that it allows for scalable analysis without compromise in the accuracy of the correlation. This reduces to solving linear systems that have the same size of the grid from which the data are sampled. The linear systems have multiple right-hand sides, and in the case of a regular grid, they are multilevel Toeplitz matrices. We employ a block CG solver with multilevel circulant preconditioners. Previous experiments indicate a linear scalability of computational costs for 2D grids of size up to $10^3times10^3$ on a single desktop machine in a Matlab environment. We have demonstrated that up to 1 million data points our ap proach scales virtually perfectly. We have developed parallel C++ codes based on the Trilinos project and will investigate the scalability of the algorithm for 4D data set with on the order of $10^9-10^{12}$ data points; beyond the $10^6$ data sets we can manipulate at the moment. For being able to compile our code, we need (in addition to standard compilers) Trilinos, and ideally also PETSC. The ScalaGauss code can be used in numerous other applications (chemistry, power grids, materials science) that require a Gaussian process statistical models. Project URL: Requested allocation: 201000 Justification: The requester has used 0 hours of their initial startup project. In addition to approving an initial amount, please specify a Category and Subcategory for this project. For a list of the current selection of approved categories, please see: https://wiki.lcrc.anl.gov/wiki/Processes/Categories Once the Allocation committee has approved the project, please go to the Project Management page to create it: https://accounts.lcrc.anl.gov/projects.php Thank You, The LCRC Accounts System

1 0