- Allocations-admins - mailman3.cels.anl.gov

[LCRC Accounts] Project Request: DDM
by accounts＠lcrc.anl.gov 22 Jul '10

22 Jul '10

Hello, A new project on the LCRC cluster has been requested. Please forward the information on to the LCRC Allocation sub-committee. Applicant's name: Ran Huo Applicant's institution: ANL Applicant's division: HEP Project Name: DDM Project title: Decaying Dark Matter Associated funding: Research Assistent from uchicago Other Systems: At bangcompute.uchicago.edu, tested for 1k CPUh Science: We have evidence that the universe consists of Dark Matter (DM), but we don't know the property of DM as particle. One issue is whether the DM particle is stable, or it has undergone decay. Given that the two product particles are collisionless, if the DM decays at the medium stage of the universe evolution, namely, just around Cosmic Microwave Background (CMB) and Large Scale Structure (LSS) formation, current observation can set stringent bound on its property. Basically the decay will introduce free streaming which suppress the small scale structure on LSS, and introduce relativisitc component which lifts the acoustic peaks on CMB. Compared with the literature, our analysis will be comprehensive and the first to set numerical bound on mass ratio between the parent particle and the product particles. Project description: We run the modified public code "cosmomc", which is a Markov Chain Monte Carlo engine for exploring cosmological parameter space. It calls modified lib "camb" to compute theoretical spectrum for parameters, and analysize with data from WMAP and other observation to finally determine a parameter likelihood distribution. The code has internal scaling, and since it is parallel, the scaling is arbitrary. The public library used are LAPACK -- Linear Algebra PACKage, CFITSIO - A FITS File Subroutine Library, MKL - Intel Math Kernel Library. A test on a small cluster for 2 days is completed and it ran for 3000 parameters. The test is partially MPI, which is, the theoretical "camb" part is MPI, and not the data analysis. With the full power of LCRC, hopefully several days work will give enough parameter points, which would be order 100k. Now only one member is expected. Project URL: Requested allocation: 280000 Justification: The requester has used 0 hours of their initial startup project. In addition to approving an initial amount, please specify a Category and Subcategory for this project. For a list of the current selection of approved categories, please see: https://wiki.lcrc.anl.gov/wiki/Processes/Categories Once the Allocation committee has approved the project, please go to the Project Management page to create it: https://accounts.lcrc.anl.gov/projects.php Thank You, The LCRC Accounts System

1 0

[LCRC Accounts] Project Allocation Request
by accounts＠lcrc.anl.gov 22 Jul '10

22 Jul '10

Hello, A change in allocation has been requested: Requester: rrankin (Rees Rankin) Project: DFT_TM_Catal_Screen Title: DFT Screening Investigation on Catalytic and Electrocatalytic Phenomena of Transition Metals Description: We intend to perform these calculations to study multiple important reactions in the context of fundamental catalytic as well as electrocatalytic applications. Specifically, we began in FY2009 by studying the reduction of hydrogen peroxide (H2O2) on these surfaces. The applications of studying this reaction were numerous. It is well known that H2O2 historically has not been synthesized efficiently via direct conversion of H2 and O2. Recent work in the literature suggests metallic alloy nanoparticles can create new chemistry in the selective synthesis of H2O2 from H2 and O2. After completing our initial work (flat, and stepped surfaces of 'pure' single component TM species) on this reaction in FY2009, we are ready to move forward quickly with screening of alloys/skins for this reaction, and to extend our now-developed methods to look at related oxygen reduction (ORR) reactions and the interesting reactions involved with the electrochemical reduction of CO2 (CDR) towards more economically and environmentally beneficial target species. Additionally, by building on the knowledge generated from such fundamental reactions, it will be much more feasible to predict the performance of novel catalysts in other reactions which involve the same or similar reactants and intermediates but different products, or conversely, differing reactants with similar intermediates and products. Our work will continue to supplement the computational materials-by-design database being developed. All calculations will be performed using the Vienna ab initio Simulation Program (VASP). This program uses a periodic plane-wave representation of the electronic wavefunctions of materials with a self-consistent minimization/solution under the Density Functional Theory (DFT) formalism. Recent improvements in the parallelization of VASP combined with the improved hardware capabilities in the LCRC Fusion system (compared to Jazz) allow for reasonably efficient near-linear scaling for jobs running on up to 80 cores (10 nodes). Superlinear scaling can now even be witnessed for small jobs in a single compute node with appropriate cache sizes. Additional calculations in the future may utilize the Dacapo DFT code, which performs similarly to VASP in the above regards. GPAW, the successor to Dacapo, may become a viable option within FY 2010 as well; GPAW is inherently designed to scale well beyond Dacapo or Vasp in terms of parallelization efficiency on large HPC systems such a s the BlueGene. Preliminary calculations for this work have already been performed on the HPC Carbon cluster available at the CNM (ANL), and the JAZZ system at LCRC (ANL) in FY2009, and are continuing into FY2010. However, the resources available at CNM-ANL Carbon cluster are currently significantly oversubscribed and will likely remain so even with the upcoming hardware expansion; small scale jobs can actually sit in the submission queue for time intervals longer than their execution related wall time. An allocation of 650,000 core hours on LCRC-Fusion would ostensibly allow for approximately 1600 calculation-sets (1 calculation-set here can be envisioned as scanning several related materials (such as differently structured surface alloys of the same stoichiometry) +adsorbate over many adsorption sites, or scanning a single material surface +adsorption site over many adsorbates, or as a series of coupled calculations to determine the transition state/activation energy between 2 local minim a). As our initial work has determined the best single component Transition Metals relevant for H2O2 selective synthesis, we are now ready to significantly augment the work by screening through vast combinations of alloy/skins/etc derived from these materials. Additionally, we will extend our work to look at related reactions on these surfaces such as the electro-reduction of CO2, which is far more complicated and will hence require more calculations to initially understand. The ability to augment our knowledge base with the incorporation of this many additional calculation-sets would be paramount in developing the detailed understanding necessary to move forward in a purely computational catalysis by design (or broader materials by design) project. As a note, we used our FY2009 Allocation of 96000 hours effectively in ~6 months, and we propose to do effectively ~20x the number of total calculations in FY2010 now that we have a detailed initial understanding of our method s and some basic insights into fundamental reaction mechanisms/TM species we wish to study in vastly greater detail. The improved efficiency of these calculations with respect to Fusion's higher core clock speeds and improved intra-node and inter-node communication should make this feasible under the requested allocation. Current: undetermined amount Justification: As a brief summary of typical results, Vasp 4.6 or Vasp 5.x+ with appropriate compilation, Infiniband, and modern processor cores scales well (near linear) out to approximately 80 cores or 10 (Fusion) nodes for most typical calculations on these similar HPC systems. VASP scaling does fall significantly in the regime of calculations with severely large file I/O overhead--however, this is not expected to apply to the jobs performed in the proposed work. I am not certain how to attach a file to this web template (I am using Firefox and see no options for a file upload?) I can certainly provide, via email if necessary, some parallelization benchmarks datafiles or references for VASP calculations on similar modern HPC installations (such as EMSL @ PNL, or the Carbon cluster at ANL's CNM). I expect most jobs submitted would use between 32-64 cores (4-8 Fusion nodes)--a regime where again, VASP is expected to work as well, or better, on Fusion than it does on similar HPC systems where I use the same level of resources and parallelization is acceptably good. Requested: 240000 A specific reason has been given: The current project is proceeding well and manuscripts are being prepared. However we are also currently investigating more detailed kinetics of the system than initially expected. Thus we would like to perform more NEB calculations than initially budgeted in our proposal. The current request for additional allocation time should cover the requisite new NEB calculations. This needs to be approved and the final allocation amount decided upon. Thank You, The LCRC Accounts System

1 0

[LCRC Accounts] Project Request: Climate_Emulator
by accounts＠lcrc.anl.gov 21 Jul '10

21 Jul '10

Hello, A new project on the LCRC cluster has been requested. Please forward the information on to the LCRC Allocation sub-committee. Applicant's name: Robert Jacob Applicant's institution: ANL Applicant's division: MCS Project Name: Climate_Emulator Project title: Building a climate emulator for use in energy policy analysis Associated funding: NSF Other Systems: pending application with NCAR Science: State-of-the-art general circulation models (GCMs) provide the best estimates of future climate change. While they are valuable tools for assessing impacts of climate change for a given scenario of future greenhouse gas emissions, they cannot be used in iterative economic models since a single multi-century simulation can take weeks to months to run and the iterative models require hundreds of thousands of iterations. For this reason, we aim to develop a statistical emulator that will accurately and efficiently reproduce GCM forecasts of regional climate change for an arbitrary trajectory of CO2 concentrations by using a library of pre-computed GCM output. Library based statistical emulation has not been possible to this date because there is no consistent and comprehensive library of climate model runs corresponding to a range of scenarios of future CO2 concentrations. Computing time on Fusion will enable us to build this unique resource by perform ing a series of simulations using the Community Climate System Model (CCSM3). Since library-based statistical emulation is currently not possible, the most common approach to emulating regional climate uses linear pattern scaling, in which changes in regional temperature and precipitation are assumed to be linearly proportional to the change in global mean temperature (GMT). That is, regional climate is effectively assumed to depend solely on GMT, and not on the rate or duration of warming. Our preliminary analysis of CCSM3 output from the Climate Model Intercomparison Project (CMIP3) archive shows that this assumption is not valid for temperature projections over time scales of multiple centuries and breaks down for projections of precipitation even over decadal time scales. The CMIP3 archive, however, is not sufficient for full statistical emulation, since it is both restricted in number of runs and inconsistent in its treatment of regional climate forcing. The carefully designed library we propose here will be used to develop and test a number of alternative approaches for emulating climate model responses, including (i) nearest neighbor interpolation, (ii) a variant of pattern scaling that accounts for the rate of warming, and (iii) principal component analysis (PCA). The intended outcome of this research is a new climate emulator that will aid in the evaluation of energy policies for greenhouse gas mitigation. We anticipate that there will be a wide interest in a user-friendly, computationally tractable emulation tool from the policy, economics, and climate communities. The library itself will also be made public for further use by the community Project description: Building a new climate library is a prerequisite for developing and testing climate emulation techniques due to the limitations of existing climate libraries. We are applying here for computing time to build a library of CCSM3 runs with a large number of scenarios, multiple realizations of each scenario, and multi-century simulations. This would represent a substantial advance over the library of CCSM3 runs in the CMIP3 archive, which contains only three scenarios for future CO2 concentration growth, does not study CO2 growth beyond 2100, and considers different regional sulphate aerosol emissions for each scenario, complicating any understanding of regional climate patterns expected in a warming world. By addressing these limitations, the consistent and comprehensive climate library we propose here will make possible new emulation methods. CCSM's atmosphere component is the Community Atmosphere Model. CAM currently supports three different numerical cores for solving the basic equations: a spectral transform method, a Semi-Lagrangian scheme, and a finite-volume scheme. The ocean component is a finite-difference model called the Parallel Ocean Program and was developed at Los Alamos. CCSM's sea ice and land surface models are each among the most complex and complete models in existence. CCSM requires Fortran90 and C compilers and the MPI and NetCDF libraries to run. In order to make the project computationally tractable we will use the low-resolution (T31) version of CCSM3. Scenarios of future CO2 concentration will follow a tree design. We will consider two scenarios between 2010 and 2050, branch to six scenarios between 2050 and 2100, and branch again to 12 scenarios between 2100 and 2200. Finally, we will consider six scenarios between 2200 and 2400, for the study of very long-term effects. The tree design allows significant savings to be made in computing resources as multiple scenarios can use the same restart files, and allows examination of hysteresis (i.e., the importance of CO2 history) for scenarios with overlapping branches. Multiple realizations of each scenario (obtained by running the model with different initial conditions) are required to allow robust separation of true signals from internal variability in the climate system. We will take 10 realizations of each scenario between 2010 and 2200 and five realizations for scenarios between 2200 and 2400, when we expect long-term trends to be more pronounced. Finally, we will conduct a detailed exploration on the importance of natural variability by considering a much larger number of realizations (30) for a single 200 year scenario. This yields a total of (40*3*10) + (50*6*10) + (100*12*10) + (200*6*5) + (200*1*30) = 28,200 model years. We have sufficient disk space to store the output offsite. Project URL: Requested allocation: 830000 Justification: Preliminary testing of CCSM3 at T31 on Fusion using 4 nodes (32 processors) required 55 minutes for a simulation of one model year. Therefore, we are requesting 28,200*32*(55/60) ~= 830,000 CPU hours for the full suite of simulations. Several simulations can be run at the same time to scale the project out to the full machine if available. CCSM3 was the recipient of a great deal of tuning and evaluation by combined DOE and NCAR computational scientists so we do not expect further improvements in the per node performance. The requester has used 0 hours of their initial startup project. In addition to approving an initial amount, please specify a Category and Subcategory for this project. For a list of the current selection of approved categories, please see: https://wiki.lcrc.anl.gov/wiki/Processes/Categories Once the Allocation committee has approved the project, please go to the Project Management page to create it: https://accounts.lcrc.anl.gov/projects.php Thank You, The LCRC Accounts System

1 0

[LCRC Accounts] Project Allocation Request
by accounts＠lcrc.anl.gov 21 Jul '10

21 Jul '10

Hello, A change in allocation has been requested: Requester: jared (Jared Wilkening) Project: MG-RAST Title: Analysis of metagenomes using MG-RAST Description: Goal is to provide initial analysis for the two large data sets that represent the current state of the art in soil metagenome analysis. Both projects are well aligned with Argonne's strategic plan the BESI metagenomics initiative goals. Current: undetermined amount Justification: During the last allocation we have scaled up from running single node jobs to jobs of an arbitrary size. Additionally we worked to efficiently utilize backfill time slots due to the flexible run time of our code. Over the next allocation we plan to experiment with new more efficient algorithms that will make use of the large memory nodes. Requested: 350000 A specific reason has been given: Nearly used up current allocation and expect to continue to need the core-hours. This needs to be approved and the final allocation amount decided upon. Thank You, The LCRC Accounts System

1 0

[LCRC Accounts] Project Request: SepMD
by accounts＠lcrc.anl.gov 19 Jul '10

19 Jul '10

Hello, A new project on the LCRC cluster has been requested. Please forward the information on to the LCRC Allocation sub-committee. Applicant's name: Kent E. Wardle Applicant's institution: ANL Applicant's division: CSE Project Name: SepMD Project title: Molecular Dynamics Simulation of Separations Processes Associated funding: DOE-NE Nuclear Energy Advanced Modeling and Simulation (NEAMS) Program Other Systems: MCS Full-disclosure (SiCortex 5832) Science: Liquid-liquid extraction is used in a variety of chemical processes relevant to energy technology and most notably is the primary means for recycle of used nuclear fuel. Computational science-based simulation of such processes can lead to key advancements in fuel recycle technology through increased understanding of the scientific foundations of such physical phenomena. Predictive simulation of a complex process such as liquid-liquid extraction requires accurate representation of the fundamental physics and chemistry on a number of scales ranging from the molecular level to the continuum and engineering scale and covering an equally broad range of temporal scales. This project seeks to develop accurate intermolecular potentials incorporating molecular polarizability for use in molecular dynamics simulations of liquid-liquid interfacial phenomena. Electronic structure calculations for key molecular species of interest will be used to calculate pol arizabilities. It is anticipated that physically realistic molecular models as developed here will help to move toward greater predictive capability and ultimately enable the design of advanced extractants for efficient separations processes. Project description: Liquid-liquid extraction is used in a variety of chemical processes relevant to energy technology and most notably is the primary means for recycle of used nuclear fuel. Computational science-based simulation of such processes can lead to key advancements in fuel recycle technology through increased understanding of the scientific foundations of such physical phenomena. Predictive simulation of a complex process such as liquid-liquid extraction requires accurate representation of the fundamental physics and chemistry on a number of scales ranging from the molecular level to the continuum and engineering scale and covering an equally broad range of temporal scales. This project seeks to develop accurate intermolecular potentials incorporating molecular polarizability for use in molecular dynamics simulations of liquid-liquid interfacial phenomena. Polarizable potentials will be developed using a Drude oscillator implementation which has recently been incorporated in the NAMD code. While polarizable models have been developed for a number of the required species such as water and alkanes, models for extractant molecules are not available. This project has started by developing a polarizable model for a key extractant for nuclear-related separations process, namely, tributyl phosphate (TBP). At present, a preliminary model has been developed and testing is underway. The development of this and other models involves electronic structure calculations for calculation of molecular polarizabilities. Electronic structure calculations are done using Gaussian. While the efforts to date have focused on static runs, the requested allocation will enable us to perform more rigorous calculations and explore a wider configuration space. In general, these simulations will use only 8-16 processors but require multiple runs for man y tens of hours. Following Gaussian simulations, MD model development and testing will be performed in CHARMM and NAMD. CHARMM will be used primarily for system configuration setup and small-scale testing while NAMD will be used for full-scale simulations. The allocation requested here is primarily for the Gaussian work as we are currently running NAMD very successfully on the SiCortex machine Full-Disclosure. As this project is still in the startup/exploratory phase, the requested allocation is quite small but increased resources may be required in the future. This project has one PI and perhaps 1-2 additional researchers. Project URL: Requested allocation: 50000 Justification: The requester has used 0 hours of their initial startup project. In addition to approving an initial amount, please specify a Category and Subcategory for this project. For a list of the current selection of approved categories, please see: https://wiki.lcrc.anl.gov/wiki/Processes/Categories Once the Allocation committee has approved the project, please go to the Project Management page to create it: https://accounts.lcrc.anl.gov/projects.php Thank You, The LCRC Accounts System

1 0

[LCRC Accounts] Project Request: forms
by accounts＠lcrc.anl.gov 02 Jul '10

02 Jul '10

Hello, A new project on the LCRC cluster has been requested. Please forward the information on to the LCRC Allocation sub-committee. Applicant's name: Gustav Meglicki Applicant's institution: ANL Applicant's division: MCS Project Name: forms Project title: Forms, the co-programmable FDTD solver Associated funding: DoE Other Systems: Teragrid, IU Quarry, ANL MCS workstations Science: Program Forms is a Scheme-extensible FDTD electromagnetic solver, based on Chombo and Guile, and developed for nanophotonics and astrophysics. Project description: This is a small request. The program was developed for nanophotonic research at ANL, the work was done with Boyana Norris and Stephen Gray, and the objective of this project is to run code tests for the already completed, but not yet tested, code installation on Fusion, in /fusion/gpfs/project/photonic. The following tests will be attempted: (1) sequential run (through PBS) (2) parallel multi-core run on a single node (3) parallel single- and multi-core run on multiple nodes (4) some limited scalability, performance and IO tests--not encroaching on production Project URL: http://perth.ovpit.indiana.edu/gustav/Forms.pdf Requested allocation: 40000 Justification: The requester has used 0 hours of their initial startup project. In addition to approving an initial amount, please specify a Category and Subcategory for this project. For a list of the current selection of approved categories, please see: https://wiki.lcrc.anl.gov/wiki/Processes/Categories Once the Allocation committee has approved the project, please go to the Project Management page to create it: https://accounts.lcrc.anl.gov/projects.php Thank You, The LCRC Accounts System

1 0

[LCRC Accounts] Project Allocation Request
by accounts＠lcrc.anl.gov 30 Jun '10

30 Jun '10

Hello, A change in allocation has been requested: Requester: roux (Benoit Roux) Project: kv Title: Studies of voltage-vgated potassium channel Description: Voltage-gated potassium (Kv) channels are membrane proteins that respond to changes in the transmembrane potential by altering their conformation to allow the passive conduction of K+ ions across the cell membrane. These channels are tetrameric proteins, in which each subunit comprises six transmembrane (TM) helical segments (S1-S6). The ion conduction pore is located at the center of the tetrameric structure and is formed by the S5 and S6 helices from the four subunits. The first four segments (S1-S4) in each subunit form a voltage-sensing domain (VSD) that is located in the lipid membrane at the periphery of the central pore domain (PD). Upon depolarization of the membrane, the VSD in each subunit undergoes a voltage-dependent transition from a resting to an active conformation, which then leads the opening of the intracellular gate of the ion conduction pore. The conformational changes associated with the activation of Kv channels result in the transfer of an electric charge Delta-Q across the membrane, that can be measured experimentally as a small transient capacitive current. In the Shaker K+ channel, the "gating charge" corresponds to the transfer of 12-14 elementary charge (e) along the transmembrane electric field. Correspondingly, a change V in the membrane potential shifts the relative free energy of the closed and open conformations by V*Delta-Q. The gating charge DeltaQ is, thus, a key concept to explain how channel activation is coupled to the membrane potential. Initially postulated by Hodgkin and Huxley in 1952, it was first detected and measured more than 20 years later by Armstrong and Bezanilla. Ultimately, explaining the voltage-gating mechanism of Kv channels in molecular terms requires gaining knowledge of the active and resting conformations, and then showing how those conformations are able to account for the experimentally observed gating charge Delta-Q. Our strategy is to first carefully construct atomic models of the active and resting states of Kv channels that are consistent with all available experimental information using protein structure prediction algorithms. We also have a novel method to compute the gating charge using all-atom MD. Pathak et al. have generated detailed atomic models of Kv1.2 in the open/active and closed/resting states using the Rosetta-Membrane structure prediction program. The model of the open/active state complements the information missing from the X-ray structure of the Kv1.2 channel used as a template, while the S1-S2, S2-S3, and S3-S4 loops in the voltage sensing domain were modeled de novo. These results offer a promising starting point to expand our understanding of voltage gating in Kv channels. Some critical issues must be addressed. Recently, the stability of those structural models in the complex dynamical environment of the lipid bilayer was ascertained using molecular dynamics (Khalili-Araghi et al, 2010) and the gating charge was calculated using a new theory of the membrane potential (Roux, 2008). Our long-term goal is to refine the atomic models of the closed/resting and open/activated states and test their ability to account for the experimentally observed gating charge using all-atom MD and explicit lipid membrane environment. In the coming year, we have 3 specific sub-projects: 1) We want to incorporate additional experimental constraints in the refinement of the resting down-state of the voltage sensor (Macpos et al, 2007). Ernesto Vargas (a graduate student at the University of Chicago) is carrying out this project. 2) We want to predict the effect of side-directed mutations on the channels, namely the effect of substitution along the transmembrane helix S4. David Medovoy (a graduate student at the University of Chicago) is carrying out this project. 3) We want we want to refine the 3d structure of the bacterial voltage-gated channel KvAP using electron paramagnetic resonance (EPR). We will use a new mmethod that has been develop by us (Sompornpisut et al, 2008) and tested on other channels (Vasquez et al, 2008). This work will be carried out by a visiting scholar (Rong Shen). We had originally requested 250K nodehours for the fiscal year, but only half of that, 125K was granted by the Allocation committee. These systems run NAMD on 25 cores at a rate of about 1 day/nanosecond of MD. For a total of 50 ns of simulation total for the 3 projects means that we need about 300,000 nodehours. REFERENCES Pathak, M.M., V. Yarov-Yarovoy, G. Agarwal, B. Roux, P. Barth, S. Kohout, F. Tombola, and E. Y. Isacoff. (2007). "Closing in on the Resting State of the Shaker K+ Channel." Neuron 56, 124-140. Campos, F.V., B. Chanda, B. Roux, and F. Bezanilla. (2007). "Two Atomic Constraints Unambiguously Position the S4 Segment Relative to S1 and S2 Segments in the Closed State of Shaker K Channel." Proceedings of the National Academy of Sciences of the United States of America 104, 7904-7909. Sompornpisut, P., B. Roux, and E. Perozo. 2008. Structural refinement of membrane proteins by restrained molecular dynamics and solvent accessibility data. Biophys 95:5349-5361. Vasquez, V., M. Sotomayor, D. M. Cortes, B. Roux, K. Schulten, and E. Perozo. 2008. Three-dimensional architecture of membrane-embedded MscS in the closed conformation. J Mol Biol 378:55-70. Pan, A.C., D. Sezer, and B. Roux. (2008). "Finding Transition Pathways Using the String Method with Swarms of Trajectories." J Phys Chem B 112, 3432-3440. Roux, B. 2008. The membrane potential and its representation by a constant electric field in computer simulations. Biophys J 95:4205-4216. Khalili-Araghi, F., V. Jogini, V. Yarov-Yarovoy, E. Tajkhorshid, B. Roux, and K. Schulten. 2010. Calculation of the gating charge for the Kv1.2 voltage-activated potassium channel. Biophys J 98:2189-2198. Current: undetermined amount Justification: We had originally requested 250K nodehours for the fiscal year, but only half of that, 125K was granted by the Allocation committee. These systems run NAMD on 25 cores at a rate of about 1 day/nanosecond of MD. For a total of 50 ns of simulation total for the 3 projects means that we need about 300,000 nodehours. For the resources, we expect to be using NAMD runs of up to 256 CPU. This amounts to: 256 CPU * 24 hours * 50 days = ~300,000 Fusion will be an excellent platform to establish and execute the string method algorithm. Requested: 300000 A specific reason has been given: This is a very active project with several graduate students involved. It is also a collaborative project with experimentalists at the University of Chicago (Perozo and Bezanilla). This needs to be approved and the final allocation amount decided upon. Thank You, The LCRC Accounts System

1 0

[LCRC Accounts] Project Allocation Request
by accounts＠lcrc.anl.gov 30 Jun '10

30 Jun '10

Hello, A change in allocation has been requested: Requester: roux (Benoit Roux) Project: Drude Title: Development and validation of new polarizable force field for molecular dynamics Description: At the present time, MD simulations are not only limited by the computational resources but also by the physical correctness and accuracy of the underlying atomic models. Without a systematic approach to generate, test and validate the accuracy of the force fields, the value of MD simulations will always remain in question. The development of potential functions for computer simulations has a long history and is still a very active field. Essentially all of the MD simulations of complex molecular systems (biological or not) are currently carried out using fixed-charge non-polarizable force fields, even though there is a broad consensus that electronic polarization effect ought to be included. The important role of polarization is particularly obvious for self-assembling amphiphilic molecules, containing polar or charged groups as well as nonpolar moieties (see below). A number of research groups are working at developing new algorithm to treat polarization, eit her with inducible point dipoles, fluctuating charges, or classical Drude oscillators. The latter approach, which we have adopted, is the most advantageous due to its simplicity and computational efficiency. In this project, we will establish a protocol to automatically optimize the parameters of the molecules of interest using simulations. We will develop a global derivative-free optimization optimization search method to drive the parameter search relying on information from the large scale simulations. This procedure will involve a parallelized Hamiltonian-Replica exchange algorithm. The goal is to develop, optimize, and validate the functional form and parameters of a new force field accounting for induced polarization in molecular dynamics simulations must be for a range of biomolecular systems (lipid membranes, protein in solution, liquid phases, etc...). # computational methods The computational method involves ab initio quantum mechanics, as well as classical molecular dynamics simulations and free energy perturbation. The induced polarization is represented by introducing a small auxiliary charged particle attached to the nucleus by a spring. The pair forms a Drude oscillator. It can be shown that this construction accounts properly for the physics of induced polarization in molecular interactions. # scalability issues The new force field is developed to be implementable in a wide range of existing molecular dynamics codes. Currently, the model is supported by CHARMM, which has modest scalability. Our initial implementation of the methods in NAMD has allowed to run some benchmark systems with 75,000 water molecules with the polarizable force field. These NAMD simulatinos scale up to 1,000 cores without problems. # experiments planned We must simulate fully hydrated bilayer and monolayer membranes of DPPC, DPPE, DOPC and POPC and PS to optimized the force field for phospholipid molecules. To increase the sampling, we will probably run those simulations in parallel-parallel mode with replica-exchange MD simulations. The planned tests on the large solvated systems are absolutely necessary. We are also developing ion solvation models in liquid N-methylacetamide. # software requirements We use Gaussian, Qchem, CHARMM, NAMD. # sizes of calculations run on and/or planned for Jazz or other systems On single MD runs, we expect to use up to 64 cores in parallel. On replica-exchange MD runs, we could easily use up to 512 cores or more. Each of the simulations are expected to require about several weeks non-stop. # expected number of project members The number of members is expected to be 5-6, including Benoit Roux, Yun Luo and Albert Lau and Janamejaya Chowdhary Chris Rowley, from University of Chicago. Current: undetermined amount Justification: MD simulation of lipid membrane bilayers with the polarizable force field . The rate is 1.5 days/ns with 256 cores, for a total of 4,608 SUs for each system. We have to examine different sets of parameters to optimize the lipid force field. Each must be simulated for about 5-10 ns, which amounts to a total of 300,000 SUs. MD simulation of ion solvation with the polarizable force field in N-methylacetamide (NMA) and water for 3 alkali ion, 4 alkali earth, and 6 transition metals. The rate is 3 days/ns with 64 cores, for a total of 4,608 SUs for each system. There are 13 systems total, yielding 60,000 SUs. Ab initio calculations on NMA-Ion clusters using the program Gaussian. Here we mainly carry out geometry optimization with different level of theories: DFT hybrid, DFT pure, DFT+D, and the most computationally expensive, RI-MP2. We have to optimize about 80 clusters (different ions with different numbers of NMA and configuration around the ion) with each level. This amounts to a total of 60,000 SUs. Requested: 300000 A specific reason has been given: This is a very active project with 5 people currently involved. This is a key year for the development of the force field because we are almost done. This needs to be approved and the final allocation amount decided upon. Thank You, The LCRC Accounts System

1 0

[LCRC Accounts] Project Allocation Request
by accounts＠lcrc.anl.gov 30 Jun '10

30 Jun '10

Hello, A change in allocation has been requested: Requester: iddir (hakim iddir) Project: TIES Title: Tailored Interfaces for Energy Storage (TIES) Description: We propose to investigate the above mentioned properties (SEI formation, structure, Li transport, and reactions at the interface) on model systems using density functional theory (DFT) as implemented in the Vienna Ab Initio Simulation Package (VASP) currently available on JAZZ. We will particularly focus on the most currently used anode material (graphite) and its interface with Li2CO3, as well as Li2CO3 properties themselves. We will also investigate Si based materials, such as Si carbides, Si single crystal surfaces, and their interface with liquid electrolyte. In this case we will particularly focus on the electrochemical aspect of the modeling. The scale of these calculations exceeds small computer cluster capacity. The capabilities and already tested performance of JAZZ are extremely suitable for the type of calculations we propose. In particular, VASP, the package that we will be using in this study does not scale very well beyond 48 processors, precluding efficient use of short-time large-scale capabilities typical for general use computer centers. In fact, the scaling of the calculations is dramatically reduced with the number of processors beyond 48. Preliminary calculations for this work show that at least 750000 core hours would be necessary to complete the proposed work. Expected number of project members: 4 Current: undetermined amount Justification: Preliminary calculations ran on the PNNL machines, show that a typical open shell structure optimization run for a single Li2CO3 monomer adsorbed on an edge site of a four-layer graphite substrate (4x2x2 super-cell, 166 atoms, 512 bands), requires on average about 1000 node hours. The study of Li2CO3 growth on graphite alone (basal and edge faces) will require the investigation of about 40 different configurations [(Li2CO3)x, x=1,2,3,4]. The notoriously CPU-expensive NEB calculations (charged cells- slow convergence) will require about 2000 node hours to study the diffusion of Li-ion in Li2CO3. The study of the binding and frequency calculations of several adsorbents on both faces of the graphite support, as well as the studies on Si single crystal surfaces, and Si based materials and their interface with liquid electrolyte will require at least an equal amount of time 40000 node hours. A total of about 750000 core hours would be needed to perform these calculations. Requested: 150000 A specific reason has been given: We have made good progress on this project using the initial allocated time on Fusion. However, the systems under investigation seem to require more computer time than initially estimated. Both the systems size as well as the number of configurations that need to be investigated in this research project turned out to be larger than expected. For instance Li2CO3/Graphite interface structures with two different film thicknesses of Li2CO3 and different relaxation constraints on graphite with 406 and 335 atoms for the thick and thin films, respectively, required about 48000 core hours each so far, and will need about 20000 to 30000 hours to complete the structure optimization. The completion of the structure optimization for these two systems with two different configurations (4 large systems total) alone would require about 80000 to 120000 hours. The study of the binding and frequency calculations of several adsorbents on both faces of the graphite support will require at least 40000 hours. A total of about 150000 core hours would be needed to continue these calculations. This needs to be approved and the final allocation amount decided upon. Thank You, The LCRC Accounts System

1 0

[LCRC Accounts] Project Allocation Request
by accounts＠lcrc.anl.gov 29 Jun '10

29 Jun '10

Hello, A change in allocation has been requested: Requester: sinclair (Donald Sinclair) Project: Lattice-QCD Title: Lattice simulations of Conformal and Walking Technicolor. Description: We perform simulations to evaluate the functional integrals of QCD-like theories formulated on a discrete space-time lattice to enable determination of the non-perturbative aspects these theories. These include the properties of these theories at non-zero temperature including the scales of confinement and chiral symmetry breaking, and such zero temperature properties as spectra, decay constants and the running of the gauge coupling constant. We are particularly interested in those theories where the coupling constant evolves particularly slowly, since these are candidate 'Walking Technicolor' theories. Related to these are theories with an infrared fixed point. Of special interest is to differentiate these two different types of behaviour. We are performing simulations of QCD-like theories which are models for Walking or Conformal Technicolor. We have been studying theories which are essentially QCD but with colour-sextet rather than colour-triplet quarks. We hope to measure the running of the QCD coupling constant. For 2 or 3 flavours, 2-loop perturbation theory predicts an infrared fixed point. For 2 flavours, it is possible that a chiral condensate forms before this fixed point is reached. If so the fixed point is avoided, the theory is confining and chiral symmetry breaks spontaneously. However, there is a region where the coupling constant evolves very slowly. These are the properties required for a walking technicolor theory. Simulations we have performed so far at finite temperature suggest that this theory does indeed walk. This is an ongoing program at NERSC. We wish to run small-lattice simulations of the 3-flavour theory, which we expect to have conformal, rather than walking behaviour on Fusion, for comparison. If we are correct, the properties of the 2 and 3 flavour theories should look rather different. Our sextet quark codes are based on our earlier triplet quark codes and use the RHMC simulation method. The Rational Hybrid Monte Carlo (RHMC) is a stochastic molecular dynamics algorithm. The functional integral of QCD is written as a partition function of a classical field theory evolving in a fictitious time. The determinant of the Dirac operator raised to a fractional power is calculated by introducing bosonic fields (pseudofermions), and sandwiching this Dirac operator raised to minus said fractional power between them. This fractional power of the Dirac operator is approximated to machine accuracy by a rational approximation. After defining this this theory on a discrete space-time lattice, the inversions required by the partial-fraction expansion of the rational approximation are performed using Krylov space methods, in particular a multi-shift extension of the conjugate gradient algorithm. A global Metropolis Monte-Carlo accept/reject step applied at the end of each t rajectory removes discretization errors introduced by the numerical integration of these stochastic equations of motion. We parallelize the code by assigning a fixed number of adjacent lattice sites to each MPI task. Network bandwidth ultimately limits how small a chunk of the lattice can be assigned to each task. Benchmarks of this code on NERSC's Cray XT4 (Franklin) give 68 Gflops on 48 cores for a 12^3*4 lattice. This is a fortran code with the parallelization performed using MPI. Current: undetermined amount Justification: The CPU requirement is the estimated time required to perform finite temperature simulations of the 3-flavour theory on a 12^3*4 lattice, and is based on the CPU time which we used to perform our simulations on the same size lattice for 2 flavours on Franklin. This assumes that the code performance on Fusion and Franklin will be similar. We will also need to perform simulations on an 8^3*4 lattice, but expect this will be done during the break-in period. The reason we are proposing to perform this project on Fusion is that the other machines we have access to schedule jobs so as to discourage such small jobs as ours. Running these jobs on Fusion will allow us to run larger lattice simulations on these other platforms. We have benchmarked our 12^3*4 codes on Franklin and found the following performances on various numbers of CPUs 8 CPUs = 10 Gflops 12 CPUs = 17 Gflops 16 CPUs = 26 Gflops 24 CPUs = 41 Gflops 48 CPUs = 68 Gflops On 48 CPUs, the measured communication overhead is 47%, indicating that we are approaching the point where communications are becoming a bottleneck. Requested: 500000 A specific reason has been given: I am currently running low on resources and wish to try and complete the project on which I am working before the end of FY2010. This needs to be approved and the final allocation amount decided upon. Thank You, The LCRC Accounts System

1 0