June 2011 - Allocations-admins - mailman3.cels.anl.gov

[LCRC Accounts] Project Allocation Request
by accounts＠lcrc.anl.gov 30 Jun '11

30 Jun '11

Hello, A change in allocation has been requested: Requester: sinclair (Donald Sinclair) Project: Lattice-QCD Title: Lattice simulations of Conformal and Walking Technicolor. Description: We perform simulations to evaluate the functional integrals of QCD-like theories formulated on a discrete space-time lattice to enable determination of the non-perturbative aspects these theories. These include the properties of these theories at non-zero temperature including the scales of confinement and chiral symmetry breaking, and such zero temperature properties as spectra, decay constants and the running of the gauge coupling constant. We are particularly interested in those theories where the coupling constant evolves particularly slowly, since these are candidate 'Walking Technicolor' theories. Related to these are theories with an infrared fixed point. Of special interest is to differentiate these two different types of behaviour. We are performing simulations of QCD-like theories which are models for Walking or Conformal Technicolor. We have been studying theories which are essentially QCD but with colour-sextet rather than colour-triplet quarks. We hope to measure the running of the QCD coupling constant. For 2 or 3 flavours, 2-loop perturbation theory predicts an infrared fixed point. For 2 flavours, it is possible that a chiral condensate forms before this fixed point is reached. If so the fixed point is avoided, the theory is confining and chiral symmetry breaks spontaneously. However, there is a region where the coupling constant evolves very slowly. These are the properties required for a walking technicolor theory. Simulations we have performed so far at finite temperature suggest that this theory does indeed walk. This is an ongoing program at NERSC. We wish to run small-lattice simulations of the 3-flavour theory, which we expect to have conformal, rather than walking behaviour on Fusion, for comparison. If we are correct, the properties of the 2 and 3 flavour theories should look rather different. Our sextet quark codes are based on our earlier triplet quark codes and use the RHMC simulation method. The Rational Hybrid Monte Carlo (RHMC) is a stochastic molecular dynamics algorithm. The functional integral of QCD is written as a partition function of a classical field theory evolving in a fictitious time. The determinant of the Dirac operator raised to a fractional power is calculated by introducing bosonic fields (pseudofermions), and sandwiching this Dirac operator raised to minus said fractional power between them. This fractional power of the Dirac operator is approximated to machine accuracy by a rational approximation. After defining this this theory on a discrete space-time lattice, the inversions required by the partial-fraction expansion of the rational approximation are performed using Krylov space methods, in particular a multi-shift extension of the conjugate gradient algorithm. A global Metropolis Monte-Carlo accept/reject step applied at the end of each t rajectory removes discretization errors introduced by the numerical integration of these stochastic equations of motion. We parallelize the code by assigning a fixed number of adjacent lattice sites to each MPI task. Network bandwidth ultimately limits how small a chunk of the lattice can be assigned to each task. Benchmarks of this code on Fusion for the 24^3*12 lattice planned for some of our FY2011 runs give: 24 cores = 46 Gflops 48 cores = 95 Gflops 72 cores = 148 Gflops 96 cores = 220 Gflops 144 cores = 369 Gflops 288 cores = 784 Gflops We plan to run the 24^3*12 lattice jobs on 288 cores (36 nodes). Our smaller lattice 16^3*8 runs would be run on 128 cores (16 nodes). Current: undetermined amount Justification: This allocation request is based on having one 288 core job or two 128 core jobs running at all times. Our project is expected to use time at NERSC and NICS as well. LCRC has the advantage that it provides facilities for jobs which have medium levels of parallelism such as the 128 and 288 core jobs indicated here, which are discriminated against on larger machines such as Franklin and Hopper(stage 2) at NERSC, and Kraken at NICS. Carver at NERSC provides a similar platform to Fusion, but the large number of users and potential users mean that we cannot rely on it to provide the bulk of our computing at NERSC. Numbers given in the description section indicate that scaling is excellent. We have plans to increase the level of parallelism for this code, but this is probably irrelevant to our Fusion use. For codes which have smaller portions of the lattice on each core, such as our 16^3*8 code, performances of around 3 Gflop/core are sustained. Requested: 500000 A specific reason has been given: To complete 24^3*8 lattice runs at m=0.0025, beta=6.7, needed to determine if runs on 16^3*8 lattice suffer from finite volume effects. Current allocation is exhausted. This needs to be approved and the final allocation amount decided upon. Thank You, The LCRC Accounts System

1 0

[LCRC Accounts] Project Request: WRF-CHEM-GVAX
by accounts＠lcrc.anl.gov 29 Jun '11

29 Jun '11

Hello, A new project on the LCRC cluster has been requested. Please forward the information on to the LCRC Allocation sub-committee. Applicant's name: Rao Kotamarthi Applicant's institution: ANL Applicant's division: EVS Project Name: WRF-CHEM-GVAX Project title: Weather and Aerosol forecast products for the Ganges Valley Aerosol Project Associated funding: DOE-BER Other Systems: NERSC Science: The Ganges valley region is one of the largest and most rapidly developing sections of the Indian subcontinent. Impacts of changes in precipitation patterns, temperature, and the flow of the snow-fed rivers could be immense. Recent satellite-based measurements have indicated that the upper Ganges valley has some of the highest persistently observed aerosol optical depth values. The aerosol layer covers a vast region, extending across the Indo-Gangetic Plain to the Bay of Bengal during the winter and early spring of each year. This is one of the few regions showing a trend toward increasing surface dimming and enhanced mid-tropospheric warming. The consequences of aerosols and associated pollution for surface insolation over the Ganges valley and monsoons in particular are not well understood. Our proposed field study will provide critical data to address these hypotheses and will contribute to developing better parameterizations for tropical clouds , convection, and aerosol-cloud interactions. The primary science questions for the mission are as follows: (a) What are the sources and sinks of the aerosols comprising the aerosol cloud over this region and what effects does it have on clear-sky radiation fields, as well as on cloud micro- and macrophysical properties? What effect does the diurnal PBL cycle in the Ganges valley have on the atmospheric distribution of aerosols? (b) What effect do increasing aerosols in the Ganges Valley have on the ISM? What effect do aerosols have on shortwave radiative forcing, mid-tropospheric heating and convective activity over continental India? To achieve these science objectives an implementation has been devised in collaboration with scientists in India. The primary anchor facility for the project is the ARM mobile facility. A deployment starting in approximately June 2011 and lasting to April 2012 at the ARIES observatory located in Nainital is anticipated for AMF ground operations. During this one-year deployment of the AMF there will be an intensive periods, extending from February 1st, 2012 to April 1st 2012 (winter intensive). Project description: The Ganges Valley Aerosol eXperiment (GVAX), led by Dr. Rao Kotamarthi, will begin its one-year ground-based observation with the Atmospheric Radiation Monitoring (ARM) mobile facility at Nainital, India,starting June, 2011. This major field study involving multiple Department of Energy laboratories will focus on investigating the potential impact of aerosols on regional scale radiative forcing changes over the Ganges Valley. Measurements of radiative flux, cloud, convection, and aerosol physical, chemical, and optical properties will be made to develop core data sets for scientific analysis. The measurements will range from those made at surface locations that yield point and profile data sets to a few selected aircraft platform-based measurements. However, these measurements and the subsequent data sets must be placed in a larger regional context in order to relate the data to a particular dynamic state of the atmosphere and begin developing hyp otheses that explain observed phenomena. The regional context will be developed within the conditions generated by a 3-D regional scale climate model. Baseline simulations of 3-D regional scale model will provide the necessary analytical support for many of the scientific studies done under GVAX. In addition, the 3-D model will be a test bed for high performance computing data storage and visualization. We have implemented and run the latest 3.3 beta version of the Weather Research and Forecasting (WRF) model with an online capability of simulating chemistry and aerosols (WRF-Chem) on the Argonne Fusion clusters. The WRF-Chem model has been optimized and uses domain decomposition to divide total work load on the distributed-memory systems. Without chemistry and aerosol modules, the standard WRF CONUS benchmark (48 hr and 12 km forecast over Continental US) completes in 15 minutes on a teraflop system (~250 processors). For the GVAX campaign, the principal investigator will simulate a model domain from 55E to 95E and 0 to 36N, divided into 367x331 12-km grid cells with 28 vertical layers from the surface to the top of the atmosphere. The model boundary conditions will be updated on a daily basis from the gridded NOAA/NCEP global forecasts, which are available online within about 36 hrs to 48 hrs after real time. Therefore, there is a need to perform at least 72 hr model simulations in order to provide 36 hr forecasts required for the field experiment. The test runs on the WRF-Chem model have been completed on a number of nodes. The regional model scales up to 250 cores and needs ~10 hours to complete a 72-hr forecast. The speedup achieved with 72 cores is about 13 relative to 4 cores. Memory used on each node decreases from 4.5 GB, 1.5GB, 1.1 GB, to 0.9 GB while the number of nodes increases from 4, 16, 36, to 72 cores. The ideal computing time to finish daily forecasts is around 3~5 hours. Thus, the daily forecasts is expected to require about 200-250 cores. Because of gas-phase and aerosol chemistry for pollutant forecast output, the WRF-Chem is much more expensive in computation time compared to running the WRF model only. However, forecasts of the geographic distribution and transport pattern of atmospheric gases and aerosols are required for the GVAX campaign. The above estimated time does not include pre-processing of the GFS meteorology fields, post-processing and visualization of the forecast results. In addition t o the model runs, these processes will be done in serial and cost an additional several hours on Fusion. We anticipate using the ESG node getting established here at Argonne by Dr. Ian Foster and his group as a potential data server for assessing the model output by users in the ARM community and other scientists. 3-D regional scale model simulations will provide the necessary analytical support for many of the scientific studies done under GVAX. These modeling results of air pollution and meteorological fields will become one of the central ‘data’ products for the experiments archived at the ARM websites. The expectation is that there will be a significant number of scientists from around the world who will be interested in GVAX data. In addition, the 3-D model will be a test bed for high performance computing data storage and visualization. The large volumes of model outputs generated would be used to test or assess the feasibility of using the cloud computing systems (such as Magellan) to perform/extract a pre-determined set of forecast products as and when needed by scientists distributed at disparate locations world-wide. Project URL: http://www.arm.gov/sites/amf/pgh/ Requested allocation: 150000 Justification: The requester has used 0 hours of their initial startup project. In addition to approving an initial amount, please specify a Category and Subcategory for this project. For a list of the current selection of approved categories, please see: https://wiki.lcrc.anl.gov/wiki/Processes/Categories Once the Allocation committee has approved the project, please go to the Project Management page to create it: https://accounts.lcrc.anl.gov/projects.php Thank You, The LCRC Accounts System

1 0

[LCRC Accounts] Project Allocation Request
by accounts＠lcrc.anl.gov 28 Jun '11

28 Jun '11

Hello, A change in allocation has been requested: Requester: mchan (Maria Chan) Project: LiAirLDRD-TMOxides Title: Investigations of transition-metal oxides as electrocatalysts for Li-air batteries Description: We will use density functional theory (DFT) with the Hubbard U (+U) correction and hybrid functionals which have been demonstrated to produce accurate energetics and reasonably accurate electronic structures in correlated systems such as TM oxides. We will perform DFT+U total-energy calculations involving Li-O_x insertion and removal from Fe/Mn oxides, which involves significant configurational sampling. An estimated 500 such calculations requiring 100-500 core-hours each are needed. Where feasible, accurate energetics and electronic structures will be obtained from optimal structures using hybrid functionals, which are more computationally expensive than DFT/+U. Approximately 50 such calculations requiring 1000-3000 core-hours each are needed. Ab initio molecular dynamics calculations will be used to extract the radial distribution functions and diffusion coefficients. Approximately 100 such calculations requiring 500-1000 core-hours each are needed. Using the projector-augmented wave method, only valence electrons will be included which reduces computational costs. We will use well-established DFT codes VASP and GPAW, which use a plane wave basis set and real space grids respectively. For large systems such as nanostructures, VASP has been tested on Fusion for computation involving systems with up to 7000 electrons. For routine calculations, hard scaling efficiency of 70%-80% has been demonstrated for 64 to 128 cores compared to 8 cores, thus enabling efficient calculations of large numbers of configurations. Current: undetermined amount Justification: Requested: 150000 A specific reason has been given: The allocation for this project for fiscal year 2011 has been exhausted. Further work is necessary to understand the electrocatalysts before and during operation in Li-oxygen batteries. This needs to be approved and the final allocation amount decided upon. Thank You, The LCRC Accounts System

1 0

[LCRC Accounts] Project Allocation Request
by accounts＠lcrc.anl.gov 17 Jun '11

17 Jun '11

Hello, A change in allocation has been requested: Requester: bliu (Bin Liu) Project: dft_biomass Title: DFT studies of catalytic biomass conversion Description: As part of the IACT (short for the Institute for Atom-efficient Chemical Transformations) supported by Energy Frontier Research Center at Argonne, the DFT calculations will be performed using the Vienna ab initio Simulation Program (VASP), which implements DFT in the periodic Bloch-type plane-wave, self-consistent Kohn-Sham framework. DFT studies conducted on Fusion cluster will focus on obtaining the thermodynamics and kinetics of the intermediate species derived from glycerol decomposition on Pt(111) and furfural hydrogenation on Pd(111) with an ultimate goal of constructing relevant free energy surfaces for their respective reaction network. In both cases, the research will involve a thorough investigation of the most stable adsorption configurations of the intermediate and searches for key transition states (using the Nudged Elastic Band algorithm) to obtain elementary reaction energy barriers. The computational demand of such investigation is substantial, given the relatively large unit cells (either a 2 or 3-layer p(4x4) periodic slab) required, together with the large numbers of proposed reaction intermediates of interest (on the order of hundreds by considering all the possible intermediates). However, it is possible to reduce the computational workload by using insights gained from our preliminary investigations on the stable geometrical motifs of some of these intermediates, together with approximate yet useful correlations that can be used to estimate the binding energies and reaction energy barriers to make the work tractable and practical within the time requested (see below). The classical MD simulations of amorphous alumina and zirconia on Fusion, although also being of great interest to the IACT, will require significantly less time than that requested for the DFT work; we estimate that 5-10% of the total computing time will be needed to run these classical MD simulations with DL-POLY (we have already verified the accuracy and suitability of our potentials with DL-POLY. These jobs will typically run on 4~8 Fusion nodes for large-scale production). Dr. Bin Liu of CNM will be the primary invesitgator of this project with Dr. Jeff Greeley as the co-investigator. To complete the above work, we are requesting 600,000 core hours for FY2011. This is an estimate based on our preliminary work running on Fusion in the past 3 months, during which time approximately 120,000 core hours have been used. We anticipate that our need for Fusion resources will increase somewhat in the coming FY due an even greater level of scientific activity (the project has been ramping up but is now at steady state), as well as due to an oversubscription of other compute clusters (e.g., the Carbon cluster at the CNM). Current: undetermined amount Justification: Recent improvements in the parallelization of VASP combined with the improved hardware capabilities (e.g. the expanded memory) allow for reasonably efficient near-linear scaling for jobs running on up to 80 cores (10 nodes). Typically, a DFT job uses 4~8 nodes at a timescale of 48 hours. Furture usage will require larger number of nodes (8~10 nodes) due to a shift of our focus on transition state calculations. As the computation time for a single job of such calculation can be a week or even more. Large allocation is preferred to guarantee the completion of this investigation. In the mean time, we will efficiently use the requested allocation by carefully selecting the key reactions to produce outstanding science. We hope to continue to use Fusion cluster to satisfy our scientific interest. Requested: 80000 A specific reason has been given: We have used all the cpu hours allocated to the project DFT_biomass FY 2012. Most of the cpu hours have been spent on computationally intensive and highly parallelized programs, such as, NEB. Fusion allows us to perform these calculations effectively. We would like to request the above additional amount (80K) to continue the investigation in this area. Thanks very much for your consideration! This needs to be approved and the final allocation amount decided upon. Thank You, The LCRC Accounts System

1 0

[LCRC Accounts] Project Allocation Request
by accounts＠lcrc.anl.gov 08 Jun '11

08 Jun '11

Hello, A change in allocation has been requested: Requester: insepov (Zeke Insepov) Project: xenon Title: Ab-initio simulation of bubbles in dense media Description: The following calculations will be performed for interatomic potentials Mo-Mo, Mo-Xe, U-U in the form of embedded atom method: 1. Choosing of ‘configuration set’ – the structures to be taken into account during fitting of the potential, calculations of energies and forces on atoms in this configurations from DFT by VASP. 2. Creating of the potential with the use of the ‘configuration set’ by means of force-matching software PotFit. 3. Verification of the potential obtained by comparison with the DFT data: formation energies of vacancies, self-interstitial atoms (SIA) and their clusters, energies of the basic structures of SIAs, their hierarchy. Numerous MD and DFT calculations, basically static, will be performed on this step. 4. Verification of the potential with the experimental data on diffusion of defects (diffusivities, threshold temperatures to induce long-range migration) and production of defects in radiation cascades (structure of the defects and sizes). One more possibility is the threshold energies of the primary knocked atom required to initiate the displacement cascade as a function of the direction angle of the primary atom. All this calculations are dynamic, molecular dynamics package LAMMPS will be used. 5.Improvement of the ‘configuration set’ to reduce the discrepancies revealed in 3 and 4. computational methods: Density Functional Molecular Dynamics simulations using the siesta method programming model: parallel experiments planned: molecular dynamics simulations software requirements: fortran compiler (e.g. openmpi) sizes of calculations run on and/or planned for LCRC or other systems: hundreds of picoseconds time scale MD simulations on 100-500 atoms systems expected number of project members: 6 We expect a total of 6 project members (two including a student from University of Illinois, and five staff members from Russian Academy of Sciences. Results of assessments are expected to lead to joint proposals to NSF, DoE (Office of Science, and NE), and DARPA. Current: undetermined amount Justification: We recently purchased a new software - vasp - that will be widely applied to nuclear fuel simulation. One of the applications of the vasp software is the new interatomic potential for the Molybdenum-Xenon system that has been recognized as a breakthrough achievement for the future metal fuel simulations. The size of the simulation box should vary from ~ 10^2 (PotFit) to 10^3 (VASP) or 10^7 (LAMMPS) depending on the size of defect cluster, to ensure there are no interactions with the periodic images via an associated stress fields. All the software involved demonstrates good parallel performance and scalability for the tasks described above. So the typical number of CPUs varies from tens to hundreds (VASP, PotFit) or thousands (LAMMPS). Requested: 125000 A specific reason has been given: We are expecting five new member of our group joining this project this summer. This needs to be approved and the final allocation amount decided upon. Thank You, The LCRC Accounts System

1 0

[LCRC Accounts] Project Allocation Request
by accounts＠lcrc.anl.gov 08 Jun '11

08 Jun '11

Hello, A change in allocation has been requested: Requester: insepov (Zeke Insepov) Project: Bubble Title: Ab-initio simulation of bubbles in dense media Description: computational methods: Density Functional Molecular Dynamics simulations using the siesta method programming model: parallel experiments planned: molecular dynamics simulations software requirements: fortran compiler (e.g. openmpi) sizes of calculations run on and/or planned for LCRC or other systems: hundreds of picoseconds time scale MD simulations on 100-500 atoms systems expected number of project members: 5 We expect a total of 5 project members (two including a student from Purdue University, and three staff from the University of Palermo). Results of assessments are expected to lead to joint proposals to NSF, DoE (Office of Science, and NE), and DARPA. Current: undetermined amount Justification: We recently purchased a new software - vasp - that will be widely applied to nuclear fuel simulation. One of the applications of the vasp software is the new interatomic potential for the Molybdenum-Xenon system that has been recognized as a breakthrough achievement for the future metal fuel simulations. Requested: 125000 A specific reason has been given: We are expecting five new users to join our two projects. This needs to be approved and the final allocation amount decided upon. Thank You, The LCRC Accounts System

1 0

[LCRC Accounts] Project Request: MEDOC
by accounts＠lcrc.anl.gov 01 Jun '11

01 Jun '11

Hello, A new project on the LCRC cluster has been requested. Please forward the information on to the LCRC Allocation sub-committee. Applicant's name: Anouar Benali Applicant's institution: ANL Applicant's division: ALCF Project Name: MEDOC Project title: Mesocopic Devices Optimisation Code Associated funding: Other Systems: BGP-Intrepid/Surveyor (Early Science Project) 2M hours Science: With the new generation of semiconductors based on nanoscaled heterostructures, modelling the electronic transport in such devices using the Boltzmann classical approaches is no longer available. The contacts effects in such components cannot be neglected. Moreover, classical or semi-classical methods do not take into account the energy levels modification when under a high electrical field. All these effects can be taken into consideration using the Density functional theory (DFT) within the Non Equilibrium Green Functions Formalism (NEGFF) and effective mass approximation. I have studied the electronic transport in an AlGaN/GaN heterostructure in a ballistic regime where no carrier is subject to the scattering centres by briefly presenting the principal aspects of the DFT and the NEGF formalism. I have written a sequential code in C++ simulating a high frequency resonant tunneling effect diode. The obtained results are in very good agreement with the available experimental data for the 1D electronic transport in an AlGaN/GaN diode. In order to model larger and more sophisticated components, a parallel version of the code is being tested. If the tests are conclusive, the code will be able to investigate quantum transport in 3D mesoscopic devices with a very low cost in computational time. Project description: I use the Density Functional Theory (DFT) within the effective mass approximation and Non Equilibrium Green Functions to model the electronic transport. The resolution of the Schrodinger/Poisson equations is done by using the finite differences methodes. Due to the nature of the studied systems (presence of non negligible contacts at the end of the device), the Hamiltonian describing all the system (Device + Contacts) is non hermitian. The compute kernel is an eigenvalue solver for a non Hermitian Hamiltonian. it is solved by using the very slow and heavy Lapack function ZGEEV. The second kernel is the Poisson solver using a three-diagonal matrix. The code is sequential for now but is being parallelized using MPI and OpenMP. It requires: -LAPACK -BLACK -ESSL -C++ No sequential job should be used on Fusion. There should be (for now) only one user (myself) on the project. Project URL: Requested allocation: 5000 Justification: The requester has used 0 hours of their initial startup project. In addition to approving an initial amount, please specify a Category and Subcategory for this project. For a list of the current selection of approved categories, please see: https://wiki.lcrc.anl.gov/wiki/Processes/Categories Once the Allocation committee has approved the project, please go to the Project Management page to create it: https://accounts.lcrc.anl.gov/projects.php Thank You, The LCRC Accounts System

1 0

[LCRC Accounts] Project Request: Nanoregime
by accounts＠lcrc.anl.gov 01 Jun '11

01 Jun '11

Hello, A new project on the LCRC cluster has been requested. Please forward the information on to the LCRC Allocation sub-committee. Applicant's name: Anouar Benali Applicant's institution: ANL Applicant's division: ALCF Project Name: Nanoregime Project title: First principal study of Au nanoclusters on Metal oxide substrate Associated funding: Other Systems: BGP-Intrepid/surveyor - 15M hours (nanoregime) BGP-Intrepid/surveyor - 2M hours (ESP) Science: Study of Au/Pt clusters on metal-oxide surfaces. Project description: This allocation request is related to the actual study of nanoclusters on BGP, in association with Dr. Jeffrey Greeley. We will use DFT/LCAO Molecular Dynamics calculations using GPAW simulation code (already installed and highly efficient on Fusion and BGP) to model systems too small to be investigated on BGP. On a comparable machine, a typical run on 1 node (4 CPUs) is: - 50h (200h/CPU) - 50 atoms cluster - 8000 MD steps GPAW is massively parallel and runs on 16k node on BGP and is partially developed at ANL. Project URL: Requested allocation: 200000 Justification: The requester has used 0 hours of their initial startup project. In addition to approving an initial amount, please specify a Category and Subcategory for this project. For a list of the current selection of approved categories, please see: https://wiki.lcrc.anl.gov/wiki/Processes/Categories Once the Allocation committee has approved the project, please go to the Project Management page to create it: https://accounts.lcrc.anl.gov/projects.php Thank You, The LCRC Accounts System

1 0