- Allocations-admins - mailman3.cels.anl.gov

[LCRC Accounts] Yearly Allocation Request for Supernova_DES
by accounts＠lcrc.anl.gov 07 Sep '17

07 Sep '17

Hello, A yearly allocation for the LCRC cluster has been requested with the following updated information: Submitter/PI: Stephen Kuhlmann Project Name: Supernova_DES Division: HEP Project title: Supernova analysis and modeling for Dark Energy Survey Associated funding: DOE/Office of Science/High Energy Physics Other Systems: 32 nodes ANL/HEP division, 50% time Science: The Dark Energy Survey is a 5-year DOE project to study cosmology, dark energy, and dark matter. It has just completed it's fourth year of data collection, and it has discovered many thousands of new supernova explosions in distant galaxies. The key to supernova cosmology is the measurement of the brightness of the explosion. We are determining this brightness using data packages on the LCRC cluster, as well as simulating the explosions using a detailed parametric model. These packages led to four publications in the past year of these data. Project description: The project is to define the supernova cosmology uncertainties for the Dark Energy Survey, using a combination of data analysis and parametric simulations. This project uses three software packages, all of which are compiled and running successfully at LCRC. We do not anticipate needing further packages. The SNCosmo and SNANA packages were compiled by the project members, and the MEEP package was compiled by LCRC support and is a softenv product. The use of these resources led directly to four publications in the past year, as described in the project report. The supernova light-curve fits for the supernova data set from years 1-3 data take 2K-coreHrs each. The fourth year data set is already becoming available for analysis. After analysis, we determine about 10% of the raw data are good supernova candidates. For simulations, we need x10 more statistics than good data candidates to define cosmology uncertainties, using at least 10 different simulation parameters. With these numbers we estimate we need 80K coreHrs per quarter. This project will involve five members of the ANL/HEP DES collaboration. Industry partnership: Project URL: http://www.hep.anl.gov/des/ Current FY Hours Used: undetermined amount New FY Requested allocation: 320000 Q1: 80000 Q2: 80000 Q3: 80000 Q4: 80000 Justification: Storage requirements: Thank You, The LCRC Accounts System

1 0

New LCRC report submitted for ACPC
by support＠lcrc.anl.gov 07 Sep '17

07 Sep '17

Full project report at https://accounts.lcrc.anl.gov/reporting/project/2017/?project=ACPC

1 0

[LCRC Accounts] Yearly Allocation Request for Irridates
by accounts＠lcrc.anl.gov 07 Sep '17

07 Sep '17

Hello, A yearly allocation for the LCRC cluster has been requested with the following updated information: Submitter/PI: Alejandro Lopez Bezanilla Project Name: Irridates Division: PSE Project title: Metal-insulator phase transitions in irridates Associated funding: DoE Other Systems: Science: Pyrochlore iridates (R2Ir2O7 where R is rare earth element or Y) are a paradigm system for studying the interplay of spin-orbit coupling and electron correlation, potentially leading to important topological effects in the electronic structure. Theoretical model-system calculations have reported interesting phases including chiral spin liquids, Weyl semimetal, and axion insulators. Systematic experimental studies have been also conducted for these compounds and revealed a strong dependence of physical properties on the choice of rare earth/alkali ion R. In our previous LCRC work we performed realistic density functional plus dynamical mean field (DFT+DMFT) calculations we obtained an ab-initio many-body phase diagram in the plane of chemical composition and on-site interaction U that features metallic magnetic phases with all-in/all-out magnetic ordering and electronic structure characterized by tilted Weyl cones, and we studied the R-dependence of the metal-insul ator transitions, concluding that intersite effects normally neglected in DFT+DMFT calculations play an important role in understanding material-dependence in experiments for the Y and Eu compound. Project description: We request an additional allocation of 490,000 CPU hours to continue and extend our DFT+DMFT studies of pyrochlore materials with strong spin-orbit coupling and electron correlations. The pyrochlore oxide Cd2Re2O7 is of intense current interest because it is a correlated material that undergoes a continuous phase transition at a temperature T=200K from an ideal pyrochlore structure to a distorted, parity-violating structure. The energetics and indeed the physics of this transition are the subject of intense current interest, with the relative importance of lattice-driven displace effects and multipolar nematicity now debate. We aim to apply state of the art many-body methods to obtain an understanding of the electronic structure of both phases of this material and to obtain an estimate of the energetics of the transition. Cd2Re2O7 has a similar crystal structure to the pyrochlore iridates (R2Ir2O7) that were the subject of our previous density functional plus dynamical means theory studies. Building on our previous work, we will carry out fully relativistic DFT calculations, including on-site U via the DFT+U method, for the 2 structures above and below transition, and then building on these results we will first verify that the J=1/2 and 3/2 band complexes are sufficiently separated, so that a frontier orbital approach similar to that used in the irradiates applies. Assuming it does we will perform 4-site cluster DMFT calculations. If the orbitals cannot be separated we will include the entire J- multiplets and perform single-site DMFT calculations. The exact diagonalization (ED) impurity solver will be employed to solve the self consistent equations in DMFT. This solver has been developed and validated in our previous study of pyrochlore iridates and it enables access to large clusters and a greate number of bath orbitals which makes cluster calculations of this type possible. According to our experience from pyrochlore iridates, we propose to use eight correlated and sixteen bath orbitals, for a reasonable accuracy. According to previous cluster DMFT calculations on Blues, a single DMFT iteration takes 128 CPU hours (0.5 hour with 256 cores). The number of iterations for a converged solution largely depends on the external parameters (U: Coulomb interaction, mu: chemical potential) and the initial condition, but it is usually between 10 to 100. Therefore, we count about 5k CPU hours for a typical calculation. For each structure, we will run the calculations with different sets of external parameters and the initial condition, as much as the allocation allows. In total, the calculates will require 490,000 CPU hours. We will use software already installed and tested in LCRC clusters to perform the DFT and DMFT calculations, which require LAPACK, CLAPACK, GSL, BLAS and OPENMPI packages (all available on Blues and Bebop). For the DFT calculations we will use VASP_5 or the latest version of the code in Bebop once compatibility has been checked. We are currently porting and testing our codes to Bebob. We intend to intensively use the new cluster. Industry partnership: Project URL: Current FY Hours Used: undetermined amount New FY Requested allocation: 495000 Q1: 125000 Q2: 125000 Q3: 125000 Q4: 120000 Justification: Storage requirements: 1 TB Thank You, The LCRC Accounts System

1 0

[LCRC Accounts] Yearly Allocation Request for rainfall
by accounts＠lcrc.anl.gov 07 Sep '17

07 Sep '17

Hello, A yearly allocation for the LCRC cluster has been requested with the following updated information: Submitter/PI: Scott Collis Project Name: rainfall Division: EVS Project title: Developing radar based observations for climate model development Associated funding: DoE-CESD Climate Model Development and Validation (CMDV) Other Systems: Assorted ANL systems (my own 32 core machine). Systems at OLCF used for formal ARM work (open cluster with no quota system..) Science: The primary goal of the CMDV LLNL/ANL/BNL consortium project is to investigate if a very high resolution climate model (The DoE ACME model with regionally refined meshes at 10km) can reproduce mesoscale flows and features (~50km) that we observe in nature. Argonne's role is to create tailored retrievals and analyses to feed into a metrics based validation system. We will be focusing on the decadal record of radar data from the Australian Bureau of Meteorology's C-Band Radar in Darwin Australia. Project description: The project will use the Python-ARM Radar Toolkit (Py-ART) and NASA’s MultiDop package to ingest, correct and retrieve vertical velocities from dual polarimetric radars. Since time steps (~10 minutes) are independent the methodology is pleasantly parallel. We will map the radar problem using dask. Dask manages execution at the nodes and Py-ART and Multidop manage the saving and reducing of the data. During the past Rainfall project, vertical velocities from 4 years of data had been retrieved using Py-ART and Multidop. However, there are potential errors in the generated data that are caused by errors in Py-ART’s ability to dealias Doppler velocities. Therefore, the allocation will be used to reprocess these data using an improved dealiasing algorithms from codes developed by the Australian Bureau of Meteorology. The Py-ART codebase is fairly efficient with higher demand components written in C and FORTRAN. It is, however, memory intensive with 2-4GB per job needed. Fortunately this is perfect for the configuration of Bebop. Most importantly Py-ART and multidop are thread-safe. We have already mapped the radar problem to Bebop using 1500 cores effectively, radically reducing the time to retrieve the vertical velocities. We will be using this allocation to work through many years (10+) of radar data, roughly 350,000 radar "Volumes". The overwhelming majority of core hours will be used by jobs with greater than 100 cores. The team consists of myself and my supervisor, Scott Collis. Industry partnership: Project URL: http://climatemodeling.science.energy.gov/news/new-funding-opportunity-anno… Current FY Hours Used: undetermined amount New FY Requested allocation: 120000 Q1: 30000 Q2: 30000 Q3: 30000 Q4: 30000 Justification: Storage requirements: None, we have paid for a large storage system Thank You, The LCRC Accounts System

1 0

[LCRC Accounts] Yearly Allocation Request for MODAL-Azores
by accounts＠lcrc.anl.gov 07 Sep '17

07 Sep '17

Hello, A yearly allocation for the LCRC cluster has been requested with the following updated information: Submitter/PI: Scott Collis Project Name: MODAL-Azores Division: EVS Project title: Models to Observations: A study of the impact of island flow on observed cloud fraction. Associated funding: LDRD-Prime Other Systems: No other large resources. Science: The project Models to Observations: A Digital Atmospheric Library (MODAL) seeks to create weather scenes and instrument simulators that create synthetic observations from the model data. One application of MODAL, and the application which is the subject of this allocation request, is investigating the impact of landscape on observed cloud properties. ARM collects data from an island site in the Eastern Northern Atlantic (ENA) which it assumes is representative of free ocean conditions. Understanding flow behavior and atmospheric conditions near complex island topography poses a challenging forecasting problem to the modeling community. We have been conducting simulations at large-eddy scale (LES) resolutions since the beginning of 2017 to study to marine and island boundary layer evolutions. The aim of the project is to create a modeling framework at true measurement resolutions to couple with cloud instrument simulators to investigate the impact of the terrain on o bserved cloud climatologies. Project description: Prior to the development of the digital library, we focused on studying the various atmospheric conditions that naturally occur due to the presence of the Graciosa island. Ideal and real configurations of flow and terrain are performed using high-resolution, large-eddy resolving (e.g., Δ < 100 meter) simulations. Ideal configurations include model initializations with ideal dry and moist temperature and wind profiles to capture flow features over an island-like topography. Real configurations use observations from different climatological background states over the ENA-ARM site on the island. Initial small-domain large-eddy simulations (LES) of dry airflow produce cold-pool formation upstream of an ideal two-kilometer island, with von Kármán like vortices propagation downstream. Although the peak height of Graciosa is less than half kilometer, the Azores island chain has a mountain over 2 km, which may be leading to more complex flow patterns when sim ulations are extended to a larger domain. Preliminary moist simulations that are performed to capture diurnal evolution of island boundary layer show distinct land/sea breeze formations under quiescent flow conditions. Further numerical experiments are planned to extend moist simulations to include realistic atmospheric profiles and observations of surface fluxes coupled with radiative effects. This work is intended to produce a useful simulation framework coupled with instruments to guide airborne and ground sampling strategies during the ACE-ENA field campaign which is aimed to better characterize marine boundary layer clouds. Although previous studies have investigated flow behavior around idealized islands, including 2D flow characteristics, LES of real island topography studies have been made possible more recently due to extensive computational requirements. Such high-resolutions simulations allow us to capture flow behavior at land/sea interface in detail, any perturbations that arises from fine-scale topography variations, as well as test the robustness of physical parameterization schemes. In the initial phase of the project, we focused our attention around a single island, yet we are planning to extend our simulations to cover 10 times the model domain, including additional islands of interest. Even though the numerical model heavily relies on OpenMP + MPI parallelism, the data processing and visualization tasks are carried out on single processors. We are planning to deploy small-domain ensemble simulations to assess the sensitivity of process details to varying initial conditions. Analysi s of ensemble datasets will necessitate programming solutions that are based on parallel data tools. Similarly, as a part of the MODAL framework, observed, remotely sensed, and modelled atmospheric datasets will be meshed on Blues/Bebop with the Python based parallel processing libraries. The bulk of the allocation will be used in running Cloud Model 1 (CM1) and Weather Research and Forecasting (WRF) models. Both models are highly amenable to using many cores, and parallelism is achieved by decomposing the model domain horizontally among available processors. We have scalability experiments that are ranging from a single node to full 32-nodes, yet the optimum performance (considering the queue wait times) were obtained by requesting 16-nodes. The initial tests show the pure distributed memory parallelism yields the best timings. However, we will test the hybrid parallelism option, once larger-domain simulations are considered. Handling data IO at high-frequency and large volumes still poses a challenge before we take the simulations to a next-step. We used near 100k compute hours to conduct small to mid-scale numerical experiments in the previous year. With the planned ensemble experiments about two times more computational resources can be required. The accounts on Blues and Bebop clusters will serve as a good test base for the larger-domain and higher-resolution experiments that will be performed on the ALCF’s Theta platform. The team will consist of Scott Collis, Gökhan Sever, and Virendra Ghate. Industry partnership: Project URL: Current FY Hours Used: undetermined amount New FY Requested allocation: 120000 Q1: 30000 Q2: 30000 Q3: 30000 Q4: 30000 Justification: Storage requirements: We have a 320TB store attached to Blues. This will be our primary storage location. Thank You, The LCRC Accounts System

1 0

[LCRC Accounts] Yearly Allocation Request for PACC
by accounts＠lcrc.anl.gov 07 Sep '17

07 Sep '17

Hello, A yearly allocation for the LCRC cluster has been requested with the following updated information: Submitter/PI: Stephen Klippenstein Project Name: PACC Division: CSE Project title: Predictive Automated Combustion Chemistry Associated funding: DOE-ASCR Other Systems: group owned linus cluster (48 nodes) group owned nodes on blues (30 nodes) I have access to about 1/5 of the CPU on these nodes Science: The recent rise of theoretical gas phase chemical kinetics to a predictive science allows us to move beyond the empiricism (and its concomitant limitations) of all prior mechanism development efforts. Here we are developing a suite of chemistry codes and scripts for automatically generating high fidelity elementary gas phase chemical mechanisms via large-scale, multiscale, high-level, a priori theoretical chemical kinetics predictions. This code exploits massively parallel computer architectures to simultaneously examine the hundreds or thousands of relevant chemical reactions, ushering in the holistic development of mechanisms. Currently, mechanisms are generally validated against experimental data that often falls outside the range of end-use conditions and are developed sequentially one reaction at a time. The enhanced fidelity of a high level theory-derived (i.e., a priori) mechanism, coupled with uncertainty driven improvement schemes, can directly access arbit rary conditions with dramatically improved and rapidly constructed mechanisms for the end-use conditions. Project description: Our procedure involves the iterative application of mechanism generator routines (Reaction Mechanism Generator, RMG), an a priori theoretical kinetics routine (EStokTP), mechanism reduction routines (DRGASA) and chemical system solvers, and uncertainty quantification/global sensitivity analysis (UQ/GSA) software (Dakota). In the initial step, a reference set of reactions for the mechanism is generated through an automated mechanism generator code (RMG). In recent (low-CPU) work, we have modified RMG to produce output in a style that facilitates subsequent steps in the workflow. The subsequent steps will require significant computational resources, and these are now handled by a new code called QTC. QTC parses output of quantum chemistry calculations (Gaussian, Molpro, NWChem) and stores this information in a database. The scope of the initial implementation of QTC and of our current database is limited to thermochemistry. QTC is capable of complex compos ite theoretical chemistry calculations, which are comprised of results obtained using GAUSSIAN, MOLPRO, and/or NWCHEM. Advanced strategies for treating anharmonic motions such as torsions from EStokTP and MESS have been incorporated, and a new interface code (TorsScan) between QTC and EStokTP was written. A general code for anharmonic corrections using Monte Carlo phase space integration (MCPSI) was implemented, parallelized, and tested. In the next period, we will begin applying QTC “at scale” using a version we have just started to integrate with SWIFT. Our applications involve several levels of electronic structure theory, with the lowest level theories applied more broadly and the highest level theories applied to just a few important species. Our proposed strategy is as follows: (i) Thermochemistry for all (~1000) species will be calculated using cbs-qb3. (ii) Thermochemistry for the ~250 core species will be recomputed using ccsd(t)-f12/cc-pvtz-f12//B2PLYPD3/cc-pVTZ with rigid rotor/harmonic oscillator state counts. (iii) Thermochemistry for the top 100 species will be recomputed using a composite method (ccsd(t)-f12/cc-pvtz-f12 + mp2/cc-pvqz-f12 - mp2/cc-pvtz-f12 // b2plypd3/cc-pvtz) with one-dimensional hindered rotors. (iv) Thermochemistry for the top 50 species will be recomputed using a similar composite method as above but with larger basis sets and again with one-dimensional hindered rotors. (v) Thermochemistry for the top 20 species will be recomputed using a complete-basis-set-limit composite method and multidimensional hindered rotors. (vi) Thermochemistry for the top 10 species will be recomputed using this same composite method but with improved treatments of anharmonicity for the non-torsion modes along with the multidimensional hindered rotors. CPU costs scale significantly with system size, such that the cost of the total calculation can be estimated based on the cost of the largest species. We have characterized the distribution of system sizes in our sets and estimated the cost of the 6 steps described above as follows: (1) 1000 species x 1 hr/species = 1,000 core-hours, (2) 250 species x 4 hrs/species = 1,000 core-hours, (3) 100 species x 10 torsion scan steps/species x 20 hrs/step = 20,000 core-hours, (4) 50 species x 10 torsion scan steps/species x 80 hrs/step = 40,000 core-hours, (5) 20 species x 40 torsion samples/species x 160 hrs/sample = 128,000 core-hours, and (6) 10 species x 40 torsion samples/species x 640 hrs/sample = 256,000 core-hours. The total is 446,000 core-hours. Industry partnership: Project URL: Current FY Hours Used: undetermined amount New FY Requested allocation: 446000 Q1: 111500 Q2: 111500 Q3: 111500 Q4: 111500 Justification: Storage requirements: Thank You, The LCRC Accounts System

1 0

[LCRC Accounts] Yearly Allocation Request for Comp_Nanocatalysis
by accounts＠lcrc.anl.gov 07 Sep '17

07 Sep '17

Hello, A yearly allocation for the LCRC cluster has been requested with the following updated information: Submitter/PI: Aslihan Sumer Project Name: Comp_Nanocatalysis Division: CSE Project title: Theoretical/Computational Studies of the Fundamentals of One-Component and Alloy Metal Based Nanocatalysis Relevant to Alternative Sources of Energy Associated funding: BES/DOE Other Systems: NERSC, ~1000000 core-hours Science: The central objective of this project is to perform modeling and computational studies that aim at understanding and characterization of the chemical reactivity and catalytic functionality of pure and alloy metal nanoparticles at the microscopic (i.e., atomic and electronic) level. Atomic and molecular nanoparticles and related systems and phenomena are the subject of a rapidly developing research field in catalysis, owing to the unique properties of these materials at different sizes, conformation and compositions. The complexity of these systems, arising from the multiplicity and variety of their structural forms and the reactive sites, offer a large number of questions which are hardly explored in catalysis. We particularly aim to address the factors that govern the preferred morphology and the role of morphology in controlling catalytic functionality, through computations within the density functional framework (DFT). Project description: The changes in the electronic structure of metal nanoparticles as a result of charge addition or depletion can lead to changes in the morphology of metal nanoparticles. Last year, through a collaborative approach with Dr. André Fielicke and his spectroscopy group at Fritz-Haber Institute, Germany, we studied the relations between the morphology, electronic structure and charge state of sub-nano size platinum particles (n=2-13 atoms). The computations reproduced the experimental findings of spectroscopy (performed on anionic and neutral Pt clusters) at an accuracy exceeding the literature and reveals important information on the electronic and physical transformations during charge transfer in platinum clusters. The collaborative approach validated our computational method and proved its suitability to other metal clusters involved in specific catalytic reactions. This year we plan to expand the scope of our work to Pd nanoparticles. Palladium nanoparticles are important for many applications such as catalysis, hydrogen storage and sensors. We plan to map out the potential energy surface for Pd nanoparticles (n=3-26 atoms) at neutral and anionic states to determine the minimum energy structures and optimum spin states using DFT. In a parallel fashion, spectroscopic experiments will be performed and the observed charge distributions and ionization potentials will be correlated with the computational results. There are a number of published papers in literature on the stable isomeric forms of Pd clusters. However the majority of these studies focus on the neutral states of the structures and how charge addition/depletion affects the morphology still is under investigation, particularly at relatively larger sizes. We will re-optimize the geometrical arrangements reported in the literature at neutral and anionic states. To determine the preferred structural arrangement of Pd atoms at relatively larger sizes (i.e. n>13 atoms), we plan to take advantage of the simulated annealing module of NWChem to further expand our database of stable Pd isomeric forms. The project members are Julius Jellinek and Aslihan Sumer. For the next year, we are planning to use VASP, NWchem, CP2K and Crystal09 softwares to model and study the systems of interest. These codes offer high scaling efficiency and advanced methodological capabilities, e.g. availability of high-level exchange-correlation functionals and dispersion corrections, which we will take advantage of to deal with the increased complexity of our models. Industry partnership: Project URL: Current FY Hours Used: undetermined amount New FY Requested allocation: 800000 Q1: 200000 Q2: 200000 Q3: 200000 Q4: 200000 Justification: The allocation request is based on previous DFT test runs with NWChem and VASP on LCRC and NERSC clusters. Scaling efficiency of parallel computations with NWChem and VASP are overall high. Tests for with NWChem on Blues and Nersc systems using various number of processors show a substantial increase in performance with the increase in the number of processors up to hundreds of processors: Pt7-glycerol-water complex (77 atoms and total of 469 basis functions) optimized with NWchem on Blues: Cores Average SCF cycle (s) 64 9 128 6 Pt7-glycerol system (20 atoms and total of 544 basis functions) optimized with NWChem on Nersc/Edison: Cores Average SCF cycle (s) 120 15 240 9 Storage requirements: 1 TB (default) Thank You, The LCRC Accounts System

1 0

[LCRC Accounts] Yearly Allocation Request for Comp_POP_Catalysts
by accounts＠lcrc.anl.gov 07 Sep '17

07 Sep '17

Hello, A yearly allocation for the LCRC cluster has been requested with the following updated information: Submitter/PI: Fanjie Kong Project Name: Comp_POP_Catalysts Division: CSE Project title: Computational studies of copper metalated catechol-containing porous organic polymers catalysts Associated funding: Catalysts Other Systems: None Science: The goal of this project is to explore, understand, and characterize, through modeling and simulations, the complex correlations between the structure, electronic features, and chemical reactivity of porous organic polymers (POPs) decorated with high densities of copper-(catecholate) groups. These discrete, monocatecholated complexes would be highly useful for applications such as gas storage,catalysis, or gas separations owing to their high specific surface area, good chemical stability and high structure control during synthesis. The central aim is to aid the ongoing experimental effort currently underway at CSE on synthesis of novel heterogeneous – primarily copper metalated catechol-containing POPs with superior activity and selectivity. Among the targeted catalytic transformations is selective oxidation of benzyl alcohol to benzaldehyde, which is a process of importance in various chemical technologies including those relevant to novel sources of energy. Project description: The planned computations will explore the structural, energetic, electronic, and catalytic functionality of two Cu centers in close proximity supported on catechol porous organic polymers (catPOPs). Our studies will study the connection of morphology, electronic features and catalytic functionality of two Cu centers catPOPs. The candidates for catalytic transformations (these systems will be tested) are the selective oxidation of benzyl alcohol to benzaldehyde. The activity and selectivity in the oxidation reaction of benzyl alcohol will be explored. The choices for the catechol-containing POPs and catalytic transformations are made based on the planned experimental investigations, some of which are currently undergoing. Within the planned studies, the issues that will be addressed are: 1) The atomic and electronic structures of a series of multinuclear aryl complexes of Cu(I) [Cun(Mes)n] will be explored. In solution they exhibit dimeric oligomers as the prevailing structure. In constrast, their solid state structures in crystals obtained from toluene solution show higer nuclearities such as [Cu(Mes)]5 which slowly turns toward the corresponding dimer in the respective solvent as observed by NMR in solution (the conversion of the pentamer to dimer). These stable homoleptic macrocycles exhibits quite symmetric arrays which can be roughly considered as D5h. The addtion of catPOP to the toluene solution resulted in the formation of CuI2catPOP(MesH) with the protolysis and elimination of MesH. What is the atomic arangement of the dimeric Cu(I) complex [Cu(Mes)]2 and the mechanism of the reaction for the formation of CuI2catPOP(MesH)? 2) The possible mechanism for the synthesis of catechol containing POPs CuIIcatPOP using a cobalt-catalyzed acetylene trimerization strategy will be addressed. 3) Use selective aerobic oxidation of benzyl alcohol to benzaldehyde to probe the catalytic activity and selectivity of CuI2catPOP(MesH).The benzyl alcohol can be oxidized selectively to the corresponding benzaldehyde at room temperature (25 ゜C) with air in acetonitrile solvent using a CuI2catPOP(MesH) and TEMPO as cocatalysts. The aim of the calculations will reveal the mechanism in the following respects as for the CuI2catPOP(MesH) catalyst: 1. What are the roles of TEMPO and CuI2catPOP(MesH) in the chemical transformation? 2. Why is this catalytic system so selective? 3 . What is the effect of the solvent(CH2CN) and ligand (“backbone”) on the reactivity? 4. What is the effect of the oxidation state of Cu on the reactivity? 4) Use propylene/propane as mixture to probe the selectivities of the single copper catPOP, the molecular packing as well as the shape and size of the pores in the amorphous single copper catPOP will be analyzed.The structural model of catpop framework materials has not yet been conclusively determined, there are few computational approaches to underpin the molecular design of amorphous porous solids, despite their possible practical advantages. For example, amorphous porous polymers can form robust, solution-processable separation membranes. A computational methodology for modeling the solid-state,amorphous packing of porous organic cages will be used to analyze void connectivity and simulate the diffusion of gases within the pore structure of the single copper catPOP. The computational studies of the structural, energetic, and electronic structure aspects will be performed using density functional theory. The packages that will be utilized include VASP, NWChem, CP2k and Crystal. Naturally, we will benefit from the efficient scaling of the codes and take advantage of it in order to explore systems with still increasing number of atoms. Thus in our project: 1) The study of the chemical properties for two Cu centers catPOPs, and 2) the selective oxidation of benzyl alcohol to benzaldehyde is a priority research program which suits well with the high-performance infrastructure of the computational resources. Industry partnership: Project URL: Current FY Hours Used: undetermined amount New FY Requested allocation: 640000 Q1: 160000 Q2: 160000 Q3: 160000 Q4: 160000 Justification: The scaling efficiency is indicated by the following example for NWChem as evaluated by us for One Cu catechol POP system (total of 962 basis functions): Cores Average SCF cycle (s) 96 1500 120 1000 Storage requirements: Thank You, The LCRC Accounts System

1 0

[LCRC Accounts] Yearly Allocation Request for ApsRenewalLattice
by accounts＠lcrc.anl.gov 07 Sep '17

07 Sep '17

Hello, A yearly allocation for the LCRC cluster has been requested with the following updated information: Submitter/PI: Michael Borland Project Name: ApsRenewalLattice Division: ASD Project title: Lattice Development and Beam Dynamics for APS Upgrade Associated funding: Department of Energy,Office of Science, Office of Basic Energy Sciences Other Systems: ASD "weed" cluster, operated by ASD/AOP group, has 1112 cores with a lustre filesystem and infiniband network; we can use approximately 80% of this cluster full time. We also have full-time use of a 960-core subcluster that is part of Blues. The allocation request assumes the use of these systems at the levels described. Science: As part of the APS Upgrade, the accelerator lattice (the location and powering of the magnets) may be completely replaced with a multi-bend achromat (MBA) configuration. This configuration promises dramatically lower emittance and dramatically higher x-ray brightness. If successful, it would make APS the brightest storage ring hard x-ray source in the world. The goal of this project is to design and validate an MBA lattice suitable for incorporation into the APS Upgrade. FY18 tasks will emphasize refinement of the design, including iteration with engineering constraints, as well as increased detail in the simulations. Project description: APS accelerator simulation activities center on the code Pelegant, which is capable of optimization, single-particle dynamics, and collective dynamics simulations. Some of the relevant uses include multi-objective genetic algorithms (MOGA) for optimization of accelerator lattices, evaluation of robustness of optimized lattices in the presence of errors, computation of particle loss distributions, and simulation of single- and multi-bunch collective instabilities. Pelegant and related codes have been used for years on Blues to successfully develop and understand the present preliminary design for the APS Upgrade. Tests of Pelegant on Bebop in the early user period showed even better performance. In FY18, we will continue refinement and analysis of APS upgrade lattice. Among the important issues we will address are the following: (1) Continue MOGA optimization of the lattice to adapt to changes originating in evolving engineering constraints. (2) Perform optimization and evaluation for different chromaticity values, from 3 to 7, in addition to the present value of 5. (3) Perform optimization and evaluation for various sextupole terms in the dipole magnets. (4) Determine whether less-performant cases from ensemble evaluation can be improved by reapplication of MOGA optimization. (5) Evaluate performance with field errors derived from simulations of mechanical stack-up in the magnets. (6) Continue optimization of collimation scheme for localization of beam losses. Perform simulations of beam dumps with various collimator shapes. (7) Continue simulations of longitudinal feedback system options and situations, including effects of various sets of higher-order modes, 324 bunches with gaps, phase- and energy-based detection, and noise effects. (8) Simulate the effect of canted insertion devices. (9) Refine and verify field quality requirements for insertion devices, particularly for polarized devices. (10) Perform Touschek scattering simulations for many error ensembles, to understand the loss distribution more fully. (11) Include soft-edge effects on the multipole magnets. Requires reoptimization of the nonlinear dynamics, then reevaluation. (12) Perform simulations using generalized gradient expansion of the fields of the dipole magnets. (13) Compute the beam-stay-clear envelope using injection simulations. (14) Simulate injection with measured kicker waveforms and field non-uniformity. Industry partnership: Project URL: Current FY Hours Used: undetermined amount New FY Requested allocation: 40000000 Q1: 10000000 Q2: 10000000 Q3: 10000000 Q4: 10000000 Justification: Pelegant has been used for years on Blues, Mira, and other computers with good parallel efficiency. The number of cores that a computation can utilize efficiently depends on the type of simulation and the problem size. Many of the computations are embarrassingly parallel and scale well up to hundreds of cores; a recent example is the newly-added simulation of elastic gas scattering, which shows no significant variation in efficiency between 16 and 640 cores (the largest number tested). Less common types of simulations involve collective effects; with typical problem parameters, these show greater than 50% efficiency up to 144 cores, limited by the need to frequently share particle distribution information. Most of our simulations make use of multiple jobs to explore parameter space or different error seeds, with each job using on the order of a hundred cores. For example, MOGA optimization will typically involve 30-60 simultaneous jobs using two nodes each; th e efficiency in this case is very high. Storage requirements: 3 TB, needed for storage of large data files generated by collective effects simulation. Thank You, The LCRC Accounts System

1 0

[LCRC Accounts] Project Request: WATER_EAM
by accounts＠lcrc.anl.gov 06 Sep '17

06 Sep '17

Hello, A new project on the LCRC cluster has been requested. Please forward the information on to the LCRC Allocation sub-committee. Applicant's name: Yuepeng Zhang Applicant's institution: ANL Applicant's division: Energy Systems Division (ES) Project Name: WATER_EAM Project title: Application of high performance computing to manage water infrastructure networks Associated funding: This project is associated with a CRADA project currently under development and negotiation with California Water Service Company (Cal Water) Other Systems: Our collaborators at Cal Water has regular access to the company’s internal business network. The network is mainly comprised of dual-core and quad-core HP desktop computers, equipped with a variety of modeling and data analytical software. However, this network does not include any regular form of (or access to) high-performance computing (HPC) capabilities, except through project-specific requests and contract services; that is the basis for this request. Science: The primary science objectives for this project is to examine the intersection/overlap of big data and high performance computing capabilities, and long-range programmatic planning and optimization of water system infrastructure. As noted in the description below, specific topics will encompass code refinement and optimization within the context of HPC, code implementation, sensitivity analysis, and rich visualization. Project description: Infrastructure is the physical framework upon which the American economy operates, and the nation’s standard of living depends. As highlighted by recent reports issued by both the American Society of Civil Engineers (ASCE) and the American Water Works Association (AWWA), the aging of these physical assets represents one of the most critical technical, managerial, financial, and safety issues currently facing the nation. According to the most recent ASCE report, the cost of building new infrastructure to service increasing populations and expanded economic activity, as well as to maintain or rebuild existing infrastructure can be as high as one trillion dollars by 2020 and roughly five trillion dollars by 2040. Within the water sector alone, these funding shortfalls are expected to be $84 billion and $144 billion, respectively. Drawing upon these considerations, of note and significance is that enterprise asset management (EAM) programs can be a significant and effective form of mitigation against the above issues and cost. More specifically, asset management programs that incorporate asset criticality, risk, life-cycle analysis, grid optimization, and other advanced techniques, can produce significant insight into infrastructure management and improvements. Our current LCRC request is framed around addressing the computational requirements associated with comprehensive EAM programs. For example, a medium to large water system may have as much as 1,000 miles of water main, 50-200 pumps, 20-100 storage tanks, hundreds of line and control valves, etc. Applying criticality, risk, and optimization techniques to systems of this scale quickly becomes computationally unapproachable for most water utilities. When taken to full national scale, this unapproachability is amplified even further. The objectives of this application are multi-dimensional, and are intended to provide industry reflection and future guidance as motivated by suitably-chosen case studies. Specific topics will encompass code refinement and optimization within the context of HPC, code implementation, sensitivity analysis, and rich visualization. Collectively, these steps will help measure the associated technical, analytical, and procedural steps and improvements (“lessons learned”) that can be used as industry guidance for future and similarly situated EAM considerations. Programming model includes algorithmic design and use, combined with considerations of modularity (function, subroutine, and library calls). This approach largely springs from the modular and customizable design of the EPANET code/engine, combined with an objective to optimize any newly-developed or modified code. The main experiments planned for this work will be a relatively small number of example water network models that can be used for testing proposed computational frameworks, methods, and solutions. The water network models will be provided by the industry partner. No field-based or laboratory-based experiments are envisioned at this time. Given the somewhat pilot nature of the project, there will be no scalability issues anticipated at this stage. The software requirements for this project will be associated with EPANET, a public domain hydraulic network analysis engine, written by the USEPA. Supporting software will also be required as follows: - Standard MS Office Suite for simple data analysis and documentation - ANSI C Compiler - FORTRAN compiler - MATLAB and/or Mathematica for post-processing - Python or similar scripting language - ESRI ArcGIS or similar/compatible package that can read spatial data Regarding performance evaluation, no previous requests of this computational scale have ever been proposed by the industry partner. Thus, no specific performance benchmarks or metrics are available. Developing these benchmarks and metrics will be one of several deliverables produced by this project (combined iteratively with some code optimization). As noted in the core-hour request, this project envisions a total demand of slightly over 23,000 core-hours. While some initial testing will be required, such as code dimensionality and efficiency, most runs are expected to be complete within 24 hours or less. An estimated 30 runs will be required on 16 cores, 2 nodes. Code optimization and efficiency improvement will be one aspect of this project. For instance, it is not immediately known whether the EPANET code/solver is compatible with a parallel computing or other high-performance computing (HPC) framework. Thus, it is likely that some amount of code development, refinement, and debugging within an HPC framework will be required. Over 200 core-hours have been allocated to this task. The overall project team has four people, in the roles of framework and model development as well as code development and debugging. Two people will primarily submit job requests to the computing cluster. The Argonne PIs are Yuepeng Zhang and Kaizhong Gao. The industry contact is Jonathan Keck at the California Water Service Company. Industry partnership: California Water Service Company, San Jose, CA, USA Project URL: Requested allocation: 26220 Q1: 5414 Q2: 7846 Q3: 7776 Q4: 5184 Justification: Not Applicable; this request falls below 500,000 core-hours. Storage requirements: 1 TB is adequate; no additional storage resources are required. The requester has used undetermined amount hours of their initial startup project. In addition to approving an initial amount, please specify a Category and Subcategory for this project. For a list of the current selection of approved categories, please see: https://wiki.lcrc.anl.gov/wiki/Processes/Categories Once the Allocation committee has approved the project, please go to the Project Management page to create it: https://accounts.lcrc.anl.gov/projects.php Thank You, The LCRC Accounts System

1 0