September 2015 - Allocations-admins - mailman3.cels.anl.gov

[LCRC Accounts] Yearly Allocation Request for Meso
by accounts＠lcrc.anl.gov 03 Sep '15

03 Sep '15

Hello, A yearly allocation for the LCRC cluster has been requested with the following updated information: Submitter/PI: Olle Heinonen Project Name: Meso Division: PSE Project title: Mesoscale computational materials science Associated funding: LDRD, BES (FWP 59001-00-105), CHiMaD (NIST), BES (Computational Materials Hub MiCCom) Other Systems: serafina cluster (MSD), BG/Q, CNM cluster Science: - Simulations of mesoscale coupled elastic/ferroic systems - Oxygen defect motion in metal/oxide/metal heterostructures - Magnetization behavior and dynamics in nanostructured magnetic systems. Project description: This project has three thrusts: (i) development of computational codes and algorithms for mesoscale modeling of inhomogeneous materials systems and applications of these codes, (ii) modeling of magnetic micron- and nano-scale systems, and (iii) first-principle and molecular dynamics modeling of metal/oxide/metal heterostructures. In the first thrust, we are developing scalable finite element codes for coupled elastic-ferroic systems, as well as charged particles immersed in a dielectric fluid subjected to electric fields. The code developments build on libraries such as libmesh and PETSc to achieve portable, scalable and highly efficiently parallelized codes. In particular, we are developing codes that combine fast-multipole methods with boundary element methods (FMM-BEM)to calculate long-range electrostatic and magnetostatic fields from bound and free surface charges. Within the MOOSE framework, we are also developing solvers for coupled ferroelectric/ferromagnetic-elastic systems. We will in the coming year integrate the FMM-BEM method with the MOOSE framework and apply the codes to study ferroelectric or ferromagnetic thin films and nanoparticles, as well as inhomogeneous composite multiferroic systems. We are in contact with a number of experimental groups with which we expect to collaborate. We a re also developing fast generalized geometry Ewald methods to coupled particles and long molecules, such as polymers, to fluids. This latter part is in close collaboration with Juan de Pablo and his group. The work in this first thrust will also be integrated into codes under the MiCCom computational materials hub led by Giulia Galli. The codes developed under this thrust are also massively scalable to leadership-class computer platforms. The programming language for these developments is C++. As we move from development to production phases we expect to run on up to 248 cores on fusion and blues. The MOOSE framework has been shown to scale almost linearly up to some 100k cores on BG/Q without any BG/Q-specific optimization. For the FMM-BEM method we have already shown linear scaling with the number of cores (using hybrid MPI-threading) to hundreds of cores. In the second thrust we are modeling static and dynamic magnet configurations in a variety of magnetic systems. This work is in collaboration with a number of experimental groups at Argonne and around the world. The workhorse of this thrust is a homemade micromagnetic code, or set of codes, in FORTRAN90 parallellized using OpenMP in addition to mkl libraries (in particular mkl dft); they only run on one node at a time but have been optimized and are highly efficient, typically running at 95% efficiency on eight to 16 cores. These runs typically consume one node on fusion or blues (all cores on the node) for one to 30 wall-hours. In thrust (iii), we are predominantly using the SIESTA/smeagol (a development version) and FHI-aims density functional theory based codes. Both have been optimized for blues/fusion and we can routinely run on up to a few hundred cores (up to some tens of thousand cores on BG/Q) using hybrid parallelization (smeagol) or just MPI tasks (FHI-aims). We plan to model finite-bias conductance in TiN-HfO2-Ta-TiN structures using non-equilibrium Green's functions (NEGF) in smeagol on systems containing 600-1000 atoms (we have already tested the performance on blues); the largest runs will use up to 512 cores for a few days wall time on blues. We will also do nudged elastic band calculations of oxygen diffusion barriers in the HfO2 using SIESTA/smeagol with about the same requirements for the largest runs. We will also combine molecular dynamics (MD) simulations using LAMMPS with NEGF; from an MD perspective, runs with some 1000 atoms are very small and the time required is negligible compared to the requirements for smeagol. We expect there to be about eight members in this project (PI, two co-PIs, four post-docs and one graduate student). Industry partnership: N/A Project URL: Current FY Hours Used: undetermined amount New FY Requested allocation: 600000 Q1: 150000 Q2: 150000 Q3: 150000 Q4: 150000 Justification: We have worked extensively with one of the smeagol developers, Ivan Rungger (who is also one of the co-PIs) to optimize SIESTA/smeagol and we now have very good scaling for up to some tens of thousand cores. FHI-ames has been optimized on blues with help from ALCF/LCRC staff and scales very well for up to some hundreds of cores (the larges we have tried). For the development under the MOOSE framework, we constantly check scaling of the developed codes to make sure they scale basically linearly to hundreds of cores before we start production runs. We wasted some core-hours FY2015 by running some codes (especially FEniCs) inefficiently; I have since gone after post-docs to make sure the optimize preconditioners and MPI/thread balance, and also spend more time on preparation using small number of cores or desktops/laptops, before starting to consume larger resources fusion/blues. The allocation request is large, not because we will do a number of runs that will c onsume a very large number of cores, but because we (a) spend a lot of time on code development, and (b) we have a relatively large number of project members who work extensively on blues/fusion. Storage requirements: 2 TB. We have already used up 1TB. I am constantly asking project members to do house cleaning, delete obsolete data, compress and archive data to be kept. Thank You, The LCRC Accounts System

1 0

[LCRC Accounts] Yearly Allocation Request for ApsRenewalLattice
by accounts＠lcrc.anl.gov 02 Sep '15

02 Sep '15

Hello, A yearly allocation for the LCRC cluster has been requested with the following updated information: Submitter/PI: Michael Borland Project Name: ApsRenewalLattice Division: ASD Project title: Lattice Development and Beam Dynamics for APS Upgrade Associated funding: Department of Energy,Office of Science, Office of Basic Energy Sciences Other Systems: ASD "weed" cluster, operated by ASD/AOP group, has 1070 cores with a lustre filesystem and infiniband network; we can use approximately 80% of this cluster full time. We also have full-time use of a 40-node (640 core) subcluster that is part of Blues. The allocation request assumes the use of these systems at the levels described. Science: As part of the APS Upgrade, the accelerator lattice (the location and powering of the magnets) may be completely replaced with a multi-bend achromat (MBA) configuration. This configuration promises dramatically lower emittance and dramatically higher x-ray brightness. If successful, it would make APS the brightest storage ring hard x-ray source in the world. The goal of this project is to design and validate an MBA lattice suitable for incorporation into the APS Upgrade. Project description: APS has developed a method of using multi-objective genetic algorithms (MOGA) for optimization of accelerator lattices. In past years, this method has been applied successfully using blues, fusion, intrepid, and a small APS cluster to development and evaluation of lattices for the APS Upgrade. In the coming year, we will contniue working on a multi-bend achromat (MBA) lattice design that promises to push APS x-ray brightness to world-leading levels. Significant progress has already been made developing a design that is consistent with engineering constraints. The design has been presented at external reviews related to the on-going upgrade project. Our approach is to create a series of lattices that gradually push performance up while simultaneously incorporating on-going constraints derived from on-going interactions with magnet and vacuum system designers. As such, the process is naturally iterative, and we have completed a series of iterations to date to conform to engineering limits. Further iterations are on-going, with interest in reducing cost also being relevant. In addition to MOGA, we must perform ensemble evaluation to assess the robustness of solutions, understand magnet quality requirements, and evaluate effects of insertion devices on the newly-developed solutions. Finalization of the lattice design and preparation of the preliminary design report is anticipated in FY16. The codes being used for MOGA and ensemble evaluation have all been used in past years on fusion. These include Pelegant, a parallel accelerator simulation code used primarily for particle tracking and acceptance determination, as well as a Tcl script used for genetic optimization. The simulations are close to embarrassingly parallel and thus scale very well. In addition to lattice development, we model the effects of the higher harmonic cavity (HHC), which plays a vital role in increasing the beam lifetime and reducing emittance blow-up. The simulations involve tracking beams consisting of 48 or more bunches of 100,000 or more particles each, and computing the collective electromagnetic interaction of these particles with the vacuum chamber and rf cavities. The methods are well advanced and will be employed with new lattices as they are generated. Beam loss analysis began in FY15 and will continue in FY16. This involves detailed and time-intensive simulation of the injection process and the Touschek scattering process. It also involves evaluation of beam loss collimation schemes, which are needed in order to protect accelerator components and reduce radiation outside the shielding wall. Industry partnership: Project URL: Current FY Hours Used: undetermined amount New FY Requested allocation: 4000000 Q1: 1000000 Q2: 1000000 Q3: 1000000 Q4: 1000000 Justification: The primary code being used, ELEGANT, has run successfully on fusion, blues, intrepid, mira, and other systems. Most simulations are close to embarrassingly parallel, since they are of single-particle dynamics only. Simulations with collective effects show greater than 70% efficiency for 256 or more cores, depending on problem details. Storage requirements: 3 TB, needed for storage of large data files generated by collective effects simulation. Thank You, The LCRC Accounts System

1 0

New LCRC report submitted for Atmos_Chem
by support＠lcrc.anl.gov 02 Sep '15

02 Sep '15

Full project report at https://accounts.lcrc.anl.gov/reporting/project/2015/?project=Atmos_Chem

1 0

[LCRC Accounts] Yearly Allocation Request for support
by accounts＠lcrc.anl.gov 02 Sep '15

02 Sep '15

Hello, A yearly allocation for the LCRC cluster has been requested with the following updated information: Submitter/PI: John Blaas Project Name: support Division: LCRC Project title: Systems administration tasks on the LCRC Cluster(s). Associated funding: Other Systems: Science: Testing to support scientists Project description: Lots of time used for home migration and testing software Industry partnership: Project URL: http://www.lcrc.anl.gov/ Current FY Hours Used: undetermined amount New FY Requested allocation: 32000 Q1: 8000 Q2: 8000 Q3: 8000 Q4: 8000 Justification: Storage requirements: 1TB Thank You, The LCRC Accounts System

1 0

New LCRC report submitted for LowF_VRC-TST
by support＠lcrc.anl.gov 01 Sep '15

01 Sep '15

Full project report at https://accounts.lcrc.anl.gov/reporting/project/2015/?project=LowF_VRC-TST

1 0

[LCRC Accounts] Yearly Allocation Request for LowF_VRC-TST
by accounts＠lcrc.anl.gov 01 Sep '15

01 Sep '15

Hello, A yearly allocation for the LCRC cluster has been requested with the following updated information: Submitter/PI: Marco Verdicchio Project Name: LowF_VRC-TST Division: CSE Project title: Low frequencies modes coupling in Variable Reaction Coordinate TST Associated funding: DOE-BES Other Systems: pople.tcg.anl.gov, linus.tcg.anl.gov Science: Radical-radical association reactions and unimolecular dissociations to free radicals are known to be of great importance in the chemistry of hydrocarbons combustion and planetary atmospheres. This kind of reactions, however, often show very low or absent local potential energy barrier, which makes traditional Transition State Theory (TST) calculations unfeasible. The variable reaction coordinate (VRC) TST approach has been proven to be of considerable utility in estimating the kinetic of barrierless reactions. The transition state dividing surface, defined by a fixed distance between a set of pivot points on each fragment, provide a valuable approximate division of the phase space into reactants and products. The position of each pivot in the molecular frame of the corresponding fragment determines the shape of the dividing surface. The optimization of both pivot point locations and distances between them provides a great flexibility in the definition of the dividing surface and allow the investigation of multiple binding sites system (i.e. addition reactions of resonantly stabilized, multiple side attack radical additions, etc.). A common problem for reactions involving radicals is the presence of a low frequency umbrella mode that is poorly treated in the harmonic oscillator approximation. Moreover this mode is often coupled to torsional modes and it is not clear how this coupling affect the partition functions. Our goal, in this project, is to develop and apply a new computational method to efficiently handle the large anharmonicities arising in molecule with strong coupled low frequencies modes. We propose to explore these issues for a number of prototypical radical-radical association reactions relevant in combustion and atmospheric chemistry. Project description: - Computational methods: We intend to use VRC-TST theory to study the effect of coupled low frequencies motions on EJ-resolved, microcanonical and canonical kinetic constants for several radical-radical association reactions. The central focus of VRC-TST is the evaluation of the reactive flux through an arbitrary dividing surface for several temperatures. In particular a multifaceted dividing surface approach, which allow for the incorporation of multiple dividing surfaces for each binding site, will be adopted. This scheme allows high flexibility on the dividing surface shape and the possibility to take into account additional reaction pathways (i.e. H atom abstraction). Unfortunately the computational cost of each kinetic calculation increases with the number of considered dividing surfaces. A variational minimization of the reactive flux is then performed with respect to both the location of the pivot points and the distances between them. The previous allocation let us to investigate the low-frequencies effect on the rate constants for several systems: - CH3 + H - CH3 + CH3 - CH2CH3 + H - CH2CH3 + CH3 - H +C2H3 with excellent results. The Blues machine has been found to be of great help for this type of studies allowing us to use a large number of processors (128 cpu for an average of 24h per job) and speed up considerably our calculations. In this third stage of the project, we would like to move our attention to different systems like: - CH3 + CH2OH - H + CH2OH These systems, in fact, have been already investigated within the previous allocation, but the complexity of their potential energy surface require additional work in order to work out an accurate rate constant (the low-freq effect has a strong dependence on the potential energy surface). Moreover the extension of this method to oxygenated systems: - O2 + CH3 - O2 + CH2CH3 requires the use of high level ab initio calculations (because of the presence of the O2 molecule) which are highly computationally demanding. - Technical details: Kinetic calculations will be performed using the VRC-TST method as implemented in the computer code VaReCoF (Y. Georgievskii, S.J. Klippenstein, VaReCoF, Sandia National Laboratories and Argonne National Laboratory, 2006). The code makes use of the MOLPRO or GAUSSIAN quantum chemistry software for on-the-fly evaluation of the potential energy. A Python interface to the code has been implemented for the management of the input files (reference energy evaluation, pivot points positions, etc.), for the generation of the input different structures ([Phi,Chi] grid of points) and for results post processing. The code does not require large amount of memory (less than 1GB) and works very efficiently on parallel architectures. The code makes use of MPI and shows a speed-up efficiency of about 60% for up to 96 CPU’s (test performed on a 8 cpu/cores AMD Operton 2354 machine). Improvement is expected with better CPUs and fastest internode connections. - Expected number of project members:1 Industry partnership: Project URL: Current FY Hours Used: undetermined amount New FY Requested allocation: 150000 Q1: 25000 Q2: 25000 Q3: 50000 Q4: 50000 Justification: Storage requirements: Thank You, The LCRC Accounts System

1 0

New LCRC report submitted for SHARP-IF
by support＠lcrc.anl.gov 31 Aug '15

31 Aug '15

Full project report at https://accounts.lcrc.anl.gov/reporting/project/2015/?project=SHARP-IF

1 0

New LCRC report submitted for SHARP-IF
by support＠lcrc.anl.gov 31 Aug '15

31 Aug '15

Full project report at https://accounts.lcrc.anl.gov/reporting/project/2015/?project=SHARP-IF

1 0

New LCRC report submitted for ElectrolyteRedoxFlow
by support＠lcrc.anl.gov 31 Aug '15

31 Aug '15

Full project report at https://accounts.lcrc.anl.gov/reporting/project/2015/?project=ElectrolyteRe…

1 0

New LCRC report submitted for metadissolve
by support＠lcrc.anl.gov 31 Aug '15

31 Aug '15

Full project report at https://accounts.lcrc.anl.gov/reporting/project/2015/?project=metadissolve

1 0