September 2010 - Allocations-admins - mailman3.cels.anl.gov

[LCRC Accounts] Yearly Allocation Request from kcsa
by accounts＠lcrc.anl.gov 17 Sep '10

17 Sep '10

Hello, A yearly allocation for the LCRC cluster has been requested with the following updated information: Submitter/PI: Benoit Roux Project Name: kcsa Division: BSD Project title: Computational study of ion selectivity in the KcsA channel Associated funding: NIH-GM062342 Other Systems: NCSA, KBT Science: The selective binding of small ions (e.g., Na+, K+, Ca2+, Mg2+) to protein sites plays a key functional role in many important biological processes. The selectivity for is governed by the free energy landscape encountered by the different ions when they associated with the binding site. This, in turn, reflects the balance of ion-protein and ion-solvent interactions. Project description: Understanding quantitatively the microscopic factors controlling ion selectivity remains a great challenge. In simple terms, the concept of selectivity means that the "correct" ion, is able to binding more favorably than an "incorrect" ion. At a physical level, it is to be anticipated that energetics and solvation play a key role in this process, although different aspects of energetics may be highlighted by the various experimental methods used to probe the system. Some experimental measurements are more sensitive to the relative depth of free energy wells, while others are more sensitive to the relative height of free energy barriers. In the case of ion channels, the large hydration energy of the ions contrasts with the small free energy barriers necessary for the fast conduction observed experimentally. This implies that ion-protein association is ultimately controlled by a delicate balance of very strong interactions. Frequently, permeation invo lves the partial dehydration of an ion, followed by the binding to a proteinaceous environment. This archetype is well illustrated by the crystal structure of the KcsA channel. The pore of the KcsA channel comprises a wide aqueous vestibular entryway, lined by non-polar residues on the intracellular side, leading up on the extracellular side to a narrow region lined by backbone carbonyl oxygens. This region of the pore, formed by the residues corresponding to the signature sequence TTVGYG common to all K+ channels, acts as a "selectivity filter" by allowing only the passage of nearly dehydrated K+ ions across the cell membrane. In such narrow molecular pores, a permeating ion must shed most of its surrounding water molecules and the large energetic loss due to dehydration must be compensated by coordination with the backbone carbonyl oxygens. The most informative strategy to address such issues is to compute the multi-ion PMF corresponding to the microscopic process in question (Berneche and Roux, Nature 2001). In particular, the calculation of the PMF enables us to pin-point the location of the largest barrier opposing the passage of a single Na+ ion while there are two K+ ions elsewhere in the pore. We carry those type of calculations using umbrella sampling MD simulations. Another strategy is free energy perturbation (FEP), which allows to compute the relative free energy of different ions at precise location in the pore. There are 2 sub-projects: 1) Compute the multi-ion PM for Na+ permeation through the KcsA channel using the open state structure that was recently determined by X-ray crystallography in the Perozo laboratory. We have already done a similar calculation for the KcsA in the closed state (Egwolf and Roux, JMB 2010). Umbrella sampling MD simulations are used to obtain W(z1,z2,z3) the 3-ion PMF function of the coordinate of the 3 ions in the selectivity filter. To understand the effect of selectivity, two systems must be considered: with 3 K+ ions, and with 1 Na+ and 2 K+ ions. Based on our experience with similar computations, we know that a grand total of 3931 MD window simulations is needed for two systems. Each window should be simulated for 0.6 ns, the first 0.1 ns being used for equilibration and the last 0.5 ns for sampling proper, for total aggregate MD simulation time of 2.4 microseconds for each of the two systems. The unbiased 3D PMFs are then calculated using the weighted histogram method ( WHAM). The simulations are generated with NAMD, which is the most efficient program for classical MD. 2) Use FEP to compute the relative free energy of Na+ and K+ in a KcsA pore with DFT/MM simulations. For this computation we are using CHARMM combined with the DFT program DEMON (developed by Denys Salahub in Calgary). This enables us to treat the ions and the nearby protein atoms with DFT and the remainder with the force field of molecular dynamics. Software requirements NAMD, CHARMM, DEMON Expected number of project members: Benoit Roux, David Medovoy (multi-ion PMF calculation) and Chris Rowley (selectivity with density function theory and MD force field) REFERENCES Noskov, SY; Berneche, S; Roux, B Control of ion selectivity in potassium channels by electrostatic and dynamic properties of carbonyl ligands. NATURE Volume: 431 Issue: 7010 Pages: 830-834 Published: OCT 14 2004 Egwolf, B; Roux, B Ion Selectivity of the KcsA Channel: A Perspective from Multi-Ion Free Energy Landscapes. JOURNAL OF MOLECULAR BIOLOGY Volume: 401 Issue: 5 Pages: 831-842 Published: 2010 Project URL: http://thallium.bsd.uchicago.edu/RouxLab/research.html Current FY Hours Used: undetermined amount New FY Requested allocation: 100000 Justification: The umbrella sampling calculations described here constitute a major part of the PhD thesis of David Medovoy, a graduate student in Biophysics at the University of Chicago. The total requested allocation for this project is: 500,000 SU = (3931 windows) * (0.6 ns/window) * ((8*cores)*(12 hour)/ns). The planned FEP simulations for with DFT/MM will be carried out by Chris Rowley, a postdoc in the lab who has a fellowship from the Canadian NSERC. His work will be a new attack on the problem. Never before has someone computed relative free energies in those channels with DFT/MM. We anticipate this those FEP calculations will require about 500,000 SUs during the year. Demon scales up to about 64 processors, but we are investigating the dependencies on grid density (in DFT) to see how it affects the performance. Thank You, The LCRC Accounts System

1 0

[LCRC Accounts] Yearly Allocation Request from Drude
by accounts＠lcrc.anl.gov 17 Sep '10

17 Sep '10

Hello, A yearly allocation for the LCRC cluster has been requested with the following updated information: Submitter/PI: Benoit Roux Project Name: Drude Division: BIO Project title: Development and validation of new polarizable force field for molecular dynamics Associated funding: LDRD: 2009-047-R1, "Novel Computing Methodologies for the Simulation of Complex Molecular Systems" NIH grant from NIGMS R01-GM072558: Polarizable Force Field For Proteins and Lipids Other Systems: BG/P, NCSA, PSC, KBT Science: The functional form and parameters of a new force field accounting for induced polarization in molecular dynamics simulations must be developed, optimized, and validated for various biomolecular systems. Project description: At the present time, MD simulations are not only limited by the computational resources but also by the physical correctness and accuracy of the underlying atomic models. Without a systematic approach to generate, test and validate the accuracy of the force fields, the value of MD simulations will always remain in question. The development of potential functions for computer simulations has a long history and is still a very active field. Essentially all of the MD simulations of complex molecular systems (biological or not) are currently carried out using fixed-charge non-polarizable force fields, even though there is a broad consensus that electronic polarization effect ought to be included. The important role of polarization is particularly obvious for self-assembling amphiphilic molecules, containing polar or charged groups as well as nonpolar moieties (see below). A number of research groups are working at developing new algorithm to treat polarizati on, either with inducible point dipoles, fluctuating charges, or classical Drude oscillators. The latter approach, which we have adopted, is the most advantageous due to its simplicity and computational efficiency. The goal is to develop, optimize, and validate the functional form and parameters of a new force field accounting for induced polarization in molecular dynamics simulations. We are particularly interested in ion solvation properties. # computational methods The computational method involves ab initio quantum mechanics, as well as classical molecular dynamics simulations and free energy perturbation. The induced polarization is represented by introducing a small auxiliary charged particle attached to the nucleus by a spring. The pair forms a Drude oscillator. It can be shown that this construction accounts properly for the physics of induced polarization in molecular interactions. # scalability issues The new force field is developed to be implementable in a wide range of existing molecular dynamics codes. Currently, the model is supported by CHARMM, which has modest scalability. Our initial implementation of the methods in NAMD has allowed to run some benchmark systems with 75,000 water molecules with the polarizable force field. These NAMD simulatinos scale up to 1,000 cores without problems. # experiments planned We are also developing ion solvation models in water, liquid N-methylacetamide, and liquid ethanol. Those are needed to model biomolecular systems. The work will comprise - divalent ions solvation in N-methylacetamide - CPMD run of K+ in ethanol - CPMD run of Na+ in ethanol - high level ab initio on small clusters - free energy perturbation MD of K+ in ethanol - free energy perturbation MD of Na+ in ethanol - osmotic pressure MD for NaCl and KCl - conductivity of KCl and NaCl # software requirements We use Gaussian, Qchem, CPMD, CHARMM, and NAMD. # sizes of calculations run on and/or planned for Fusion or other systems On single MD runs, we expect to use up to 256 or 512 cores in parallel. The CPMD does not scale as well (perhaps up to 64). The FEP simulations are trivially distributed (thermodynamic integration). The osmotic pressure and the conductivity require several MD at increasingly high ionic concentrations with large systems. # expected number of project members: The number of members is expected to be 5-6, including Benoit Roux, Karen Callahan (ions in ethanol), Yun Luo (osmotic pressure), Chris Rowley (divalent ions), from the University of Chicago. REFERENCES Yu, H; Mazzanti, CL; Whitfield, TW, et al. A Combined Experimental and Theoretical Study of Ion Solvation in Liquid N-Methylacetamide. JOURNAL OF THE AMERICAN CHEMICAL SOCIETY Volume: 132 Issue: 31 Pages: 10847-10856 Published: 2010 Baker, CM; Lopes, PEM; Zhu, X, et al. Accurate Calculation of Hydration Free Energies using Pair-Specific Lennard-Jones Parameters in the CHARMM Drude Polarizable Force Field. JOURNAL OF CHEMICAL THEORY AND COMPUTATION Volume: 6 Issue: 4 Pages: 1181-1198 Published: 2010 Yu, HB; Whitfield, TW; Harder, E, et al. Simulating Monovalent and Divalent Ions in Aqueous Solution Using a Drude Polarizable Force Field. JOURNAL OF CHEMICAL THEORY AND COMPUTATION Volume: 6 Issue: 3 Pages: 774-786 Published: 2010 Harder, E; MacKerell, AD; Roux, B Many-Body Polarization Effects and the Membrane Dipole Potential. JOURNAL OF THE AMERICAN CHEMICAL SOCIETY Volume: 131 Issue: 8 Pages: 2760-+ Published: 2009 Harder, E; Anisimov, VM; Whitfield, TW, et al. Understanding the dielectric properties of liquid amides from a polarizable force field. JOURNAL OF PHYSICAL CHEMISTRY B Volume: 112 Issue: 11 Pages: 3509-3521 Published: 2008 Whitfield, TW; Varma, S; Harder, E, et al. Theoretical study of aqueous solvation of K+ comparing ab initio, polarizable, and fixed-charge models. JOURNAL OF CHEMICAL THEORY AND COMPUTATION Volume: 3 Issue: 6 Pages: 2068-2082 Published: 2007 Project URL: http://thallium.bsd.uchicago.edu/RouxLab/ Current FY Hours Used: undetermined amount New FY Requested allocation: 300000 Justification: We have an excellent track record of publication in this field, and we will make good use of the allocated time. The project involves a variety of different computations (MD, ab initio, CPMD), which are all very important to develop good models. We could easily ask for more resource, and we will definitely utilize all the allocation requested. Thank You, The LCRC Accounts System

1 0

[LCRC Accounts] Project Allocation Request
by accounts＠lcrc.anl.gov 17 Sep '10

17 Sep '10

Hello, A change in allocation has been requested: Requester: roux (Benoit Roux) Project: kv Title: Studies of voltage-vgated potassium channel Description: Voltage-gated potassium (Kv) channels are membrane proteins that respond to changes in the transmembrane potential by altering their conformation to allow the passive conduction of K+ ions across the cell membrane. These channels are tetrameric proteins, in which each subunit comprises six transmembrane (TM) helical segments (S1-S6). The ion conduction pore is located at the center of the tetrameric structure and is formed by the S5 and S6 helices from the four subunits. The first four segments (S1-S4) in each subunit form a voltage-sensing domain (VSD) that is located in the lipid membrane at the periphery of the central pore domain (PD). Upon depolarization of the membrane, the VSD in each subunit undergoes a voltage-dependent transition from a resting to an active conformation, which then leads the opening of the intracellular gate of the ion conduction pore. The conformational changes associated with the activation of Kv channels result in the transfer of an electric charge Delta-Q across the membrane, that can be measured experimentally as a small transient capacitive current. In the Shaker K+ channel, the "gating charge" corresponds to the transfer of 12-14 elementary charge (e) along the transmembrane electric field. Correspondingly, a change V in the membrane potential shifts the relative free energy of the closed and open conformations by V*Delta-Q. The gating charge DeltaQ is, thus, a key concept to explain how channel activation is coupled to the membrane potential. Initially postulated by Hodgkin and Huxley in 1952, it was first detected and measured more than 20 years later by Armstrong and Bezanilla. Ultimately, explaining the voltage-gating mechanism of Kv channels in molecular terms requires gaining knowledge of the active and resting conformations, and then showing how those conformations are able to account for the experimentally observed gating charge Delta-Q. Our strategy is to first carefully construct atomic models of the active and resting states of Kv channels that are consistent with all available experimental information using protein structure prediction algorithms. We also have a novel method to compute the gating charge using all-atom MD. Pathak et al. have generated detailed atomic models of Kv1.2 in the open/active and closed/resting states using the Rosetta-Membrane structure prediction program. The model of the open/active state complements the information missing from the X-ray structure of the Kv1.2 channel used as a template, while the S1-S2, S2-S3, and S3-S4 loops in the voltage sensing domain were modeled de novo. These results offer a promising starting point to expand our understanding of voltage gating in Kv channels. Some critical issues must be addressed. Recently, the stability of those structural models in the complex dynamical environment of the lipid bilayer was ascertained using molecular dynamics (Khalili-Araghi et al, 2010) and the gating charge was calculated using a new theory of the membrane potential (Roux, 2008). Our long-term goal is to refine the atomic models of the closed/resting and open/activated states and test their ability to account for the experimentally observed gating charge using all-atom MD and explicit lipid membrane environment. In the coming year, we have 3 specific sub-projects: 1) We want to incorporate additional experimental constraints in the refinement of the resting down-state of the voltage sensor (Macpos et al, 2007). Ernesto Vargas (a graduate student at the University of Chicago) is carrying out this project. 2) We want to predict the effect of side-directed mutations on the channels, namely the effect of substitution along the transmembrane helix S4. David Medovoy (a graduate student at the University of Chicago) is carrying out this project. 3) We want we want to refine the 3d structure of the bacterial voltage-gated channel KvAP using electron paramagnetic resonance (EPR). We will use a new mmethod that has been develop by us (Sompornpisut et al, 2008) and tested on other channels (Vasquez et al, 2008). This work will be carried out by a visiting scholar (Rong Shen). We had originally requested 250K nodehours for the fiscal year, but only half of that, 125K was granted by the Allocation committee. These systems run NAMD on 25 cores at a rate of about 1 day/nanosecond of MD. For a total of 50 ns of simulation total for the 3 projects means that we need about 300,000 nodehours. REFERENCES Pathak, M.M., V. Yarov-Yarovoy, G. Agarwal, B. Roux, P. Barth, S. Kohout, F. Tombola, and E. Y. Isacoff. (2007). "Closing in on the Resting State of the Shaker K+ Channel." Neuron 56, 124-140. Campos, F.V., B. Chanda, B. Roux, and F. Bezanilla. (2007). "Two Atomic Constraints Unambiguously Position the S4 Segment Relative to S1 and S2 Segments in the Closed State of Shaker K Channel." Proceedings of the National Academy of Sciences of the United States of America 104, 7904-7909. Sompornpisut, P., B. Roux, and E. Perozo. 2008. Structural refinement of membrane proteins by restrained molecular dynamics and solvent accessibility data. Biophys 95:5349-5361. Vasquez, V., M. Sotomayor, D. M. Cortes, B. Roux, K. Schulten, and E. Perozo. 2008. Three-dimensional architecture of membrane-embedded MscS in the closed conformation. J Mol Biol 378:55-70. Pan, A.C., D. Sezer, and B. Roux. (2008). "Finding Transition Pathways Using the String Method with Swarms of Trajectories." J Phys Chem B 112, 3432-3440. Roux, B. 2008. The membrane potential and its representation by a constant electric field in computer simulations. Biophys J 95:4205-4216. Khalili-Araghi, F., V. Jogini, V. Yarov-Yarovoy, E. Tajkhorshid, B. Roux, and K. Schulten. 2010. Calculation of the gating charge for the Kv1.2 voltage-activated potassium channel. Biophys J 98:2189-2198. Current: undetermined amount Justification: We had originally requested 250K nodehours for the fiscal year, but only half of that, 125K was granted by the Allocation committee. These systems run NAMD on 25 cores at a rate of about 1 day/nanosecond of MD. For a total of 50 ns of simulation total for the 3 projects means that we need about 300,000 nodehours. For the resources, we expect to be using NAMD runs of up to 256 CPU. This amounts to: 256 CPU * 24 hours * 50 days = ~300,000 Fusion will be an excellent platform to establish and execute the string method algorithm. Requested: 300000 A specific reason has been given: Allocation will enable the computations required for the 2 sub-projects. This needs to be approved and the final allocation amount decided upon. Thank You, The LCRC Accounts System

1 0

[LCRC Accounts] Project Request: NEAMS_Safety
by accounts＠lcrc.anl.gov 17 Sep '10

17 Sep '10

Hello, A new project on the LCRC cluster has been requested. Please forward the information on to the LCRC Allocation sub-committee. Applicant's name: Justin W Thomas Applicant's institution: ANL Applicant's division: NE Project Name: NEAMS_Safety Project title: CFD Support for Sodium-cooled Fast Reactor Safety Associated funding: DOE-NE Other Systems: Science: Enchance the thermofluid modeling capability of Argonne's liquid-metal fast reactor safety code SAS4A/SASSYS-1 by coupling with the commercial CFD code STAR-CCM+ for problems that involve multidimensional flows, thermal stratification, and natural circulation. This will improve the accuracy of safety-related calculations to support fast reactor design, without requiring significant development work in the SAS4A/SASSYS-1 systems code. Project description: The SAS4A/SASSYS-1 code was developed at Argonne to support fast reactor design. It is used to analyze posited accident scenarios in the fast reactor system, and determine whether or not the reactor system maintains its integrity. This requires the analysis of a nonlinear system (the reactor system itself) and several fields of physics, including reactor physics, thermal-hydraulics, and structural mechanics. For the thermofluid part, SAS4A/SASSYS-1 relies primarily on 1-D and even 0-D models of components in the coolant system. For some scenarios, the accuracy of these calculations could be significantly improved by applying CFD to specific regions of interest, while still using SAS4A/SASSYS-1 to model the majority of the plant. The coupling of STAR-CCM+ and SAS4A/SASSYS-1 has already been demonstrated on a test model, the results of which are being published at the upcoming Winter Meeting of the American Nuclear Society. The scenario under consideration in this case was a loss-of-flow where the primary pumps were shut off, and thermal stratification occurs in the plenum above the reactor core. STAR-CCM+ was applied only to this plenum, modeling natural circulation and thermal stratification. This model was run on only 4 cores for about 40 hours, but production models would typically require 16-32 cores. Project URL: Requested allocation: 20000 Justification: The requester has used 0 hours of their initial startup project. In addition to approving an initial amount, please specify a Category and Subcategory for this project. For a list of the current selection of approved categories, please see: https://wiki.lcrc.anl.gov/wiki/Processes/Categories Once the Allocation committee has approved the project, please go to the Project Management page to create it: https://accounts.lcrc.anl.gov/projects.php Thank You, The LCRC Accounts System

1 0

[LCRC Accounts] Yearly Allocation Request from SHARP_Neutronics
by accounts＠lcrc.anl.gov 17 Sep '10

17 Sep '10

Hello, A yearly allocation for the LCRC cluster has been requested with the following updated information: Submitter/PI: Justin W Thomas Project Name: SHARP_Neutronics Division: NE Project title: SHARP - Neutronics and CFD Coupling Associated funding: DOE-Nuclear Energy Other Systems: Science: To validate the accuracy of the space-angle-energy treatment for neutron transport calculations of nuclear reactors performed with UNIC. To further the application of multiphysics analysis of Very High Temerpature Reactors with the neutron transport code DeCART and the commercial CFD code STAR-CCM+. Project description: The UNIC code is currently undergoing validation and reliability testing on a wide range of parallel machines. While we generally have targeted large scale machines such as BlueGene/P at ANL and Jaguar (XT5) at ORNL, we also have a need to solve smaller benchmark problems to validate the UNIC code such as cross section (material property) generation. These problems are taken directly from experimental data or are mathematical in nature and meant to simulate the physics observed in new reactor concepts currently under study. We envision these calculations to utilize smaller processor counts << 1000 with a general need for more rapid turn around time. These problems inherently have a higher memory load than those that we are currently targeting for execution on BlueGene/P or Jaguar and thus FUSION can meet the needs of those calculations without extensively modifying the code at this point. In this year's efforts, a preconditioner will be added to UNIC's 2nd order discrete ordinates model (SN2ND) to accelerate the calculations. A significant validation and verification effort is planned for the 3D MOC module. A new version of the cross section generation tool MCC-2 (dubbed MCC-3) is under development, and will be merged into UNIC's MOC module. The neutron transport code DeCART and the commercial CFD code STAR-CD have been coupled for multi-physics analysis of Very High Temperature Reactors. In past efforts, the feasiblity of performing STAR-CD/DeCART calculations has been demonstrated. Upcoming effort will focus on STAR-CCM+ coupling, which is the more modern CFD code developed by Adapco. All jobs run on Fusion will be run in parallel on multiple nodes. Eleven users would be expected for this project. Project URL: Current FY Hours Used: undetermined amount New FY Requested allocation: 200000 Justification: The UNIC code is currently undergoing validation and reliability testing on a wide range of parallel machines. While we generally have targeted large scale machines such as BlueGene/P at ANL and Jaguar (XT5) at ORNL, we also have a need to solve smaller benchmark problems to validate the UNIC code such as cross section (material property) generation. These problems are taken directly from experimental data or are mathematical in nature and meant to simulate the physics observed in new reactor concepts currently under study. We envision these calculations to utilize smaller processor counts << 1000 with a general need for more rapid turn around time. These problems inherently have a higher memory load than those that we are currently targeting for execution on BlueGene/P or Jaguar and thus FUSION can meet the needs of those calculations without extensively modifying the code at this point. At Argonne, the CFD codes STAR-CD and STAR-CCM+ are routinely applied to simulations that use 20-100 cores, and good parallel performance has been observed. Over the past year's efforts, the parallel implementation of the MOC module in DeCART was also improved, and now can run on the same number of processors as STAR for coupled calculations. The 1/6 core simulations performed in the past year required a total of 96 cores and required 10-24 hours. Thank You, The LCRC Accounts System

1 0

[LCRC Accounts] Yearly Allocation Request from lcrc-app-engr
by accounts＠lcrc.anl.gov 17 Sep '10

17 Sep '10

Hello, A yearly allocation for the LCRC cluster has been requested with the following updated information: Submitter/PI: John Valdes Project Name: lcrc-app-engr Division: LCRC Project title: LCRC Application Engineer support work Associated funding: Other Systems: Science: Project description: This project will be used by the LCRC application engineers to do support work for users of the LCRC cluster. This support might be in the form of installation and debugging of both LCRC core and user applications, benchmarking and performance analysis, user startup help, etc. Project URL: http://www.lcrc.anl.gov Current FY Hours Used: undetermined amount New FY Requested allocation: 64000 Justification: Thank You, The LCRC Accounts System

1 0

[LCRC Accounts] Yearly Allocation Request from support
by accounts＠lcrc.anl.gov 17 Sep '10

17 Sep '10

Hello, A yearly allocation for the LCRC cluster has been requested with the following updated information: Submitter/PI: John Valdes Project Name: support Division: LCRC Project title: Systems administration tasks on the LCRC Cluster(s). Associated funding: Other Systems: Science: Project description: This project will be used by the support team for systems tasks. This will include debugging problems, testing upgrades and modifications, and running scaling and performance tests for diverse areas such as scheduling algorithms, networks, filesystem upgrades, application installations, system management, etc. Project URL: http://www.lcrc.anl.gov Current FY Hours Used: undetermined amount New FY Requested allocation: 8000 Justification: Thank You, The LCRC Accounts System

1 0

[LCRC Accounts] Yearly Allocation Request from Electromagnetics
by accounts＠lcrc.anl.gov 17 Sep '10

17 Sep '10

Hello, A yearly allocation for the LCRC cluster has been requested with the following updated information: Submitter/PI: Misun Min Project Name: Electromagnetics Division: MCS Project title: Computational Electromagnetics Associated funding: DOE AMR program and LDRD strategic Other Systems: BG/P (Intrepid) Science: This project is to develop NekCEM as an efficient computational tool equipped with advanced scalable numerical algorithms for understanding the fundamental optical properties and predicting optimal designs of next-generation electromagnetic devices, many of which have not been solved due to the limitations of current computational resources and the unavailability of a high-fidelity, high-efficiency code to run on these advanced resources. This project will enable scientists in the relevant computational electromagnetic communities to advance new scientific discoveries through highly efficient and accurate modeling with significant reduction of both the cost and the risk involved in conventional trial-and-error procedures. Project description: This project focuses on developing a massively parallel, open-source, high-fidelity electromagnetic code NekCEM. Numerical schemes are based on high-order approximations in space and time. Spectral-element discretizations are used based on hexahedral element meshes. Recent progress with communication kernel update in NekCEM demonstrate strong scaling with 60% efficiency on 131K cores at Argonne BG/P, even with small number of grid points per core (224 points per core), compared to a base case of 16K cores with 68,250 points per core. Using NekCEM, we will undertake simulation-based investigations for the following two application problems. (1) For the future International Linear Collider (ILC) linac cryomodule for generating high-quality light beams, extensive simulations will be performed to predict beam loss for ultra-short bunch beams, typically less than 0.03 mm in length, and many-meter beam propagations. These billion dollar collider systems are to be built in the 2018-2020 timeframe. Better design in the 2015 timeframe will lead to more effective colliders at lower cost. (2) For subwavelength nanosensors to detect unique signatures of chemical and biological structures of molecules, simulations will focus on optimizing the geometric parameters of metallic, thin-film, nanohole systems for single-molecule detections using surface-enhanced Raman scattering techniques. These will lead to fast and inexpensive detectors for medical tests. Our development will include a new implementation of applications in nano solar cell simulations. Currently, NekCEM project has four main developers (Misun Min, Jin Xu, Mike Fairchild,Jing Fu, Taehun Lee, Kalu Uga). Project URL: Current FY Hours Used: undetermined amount New FY Requested allocation: 350000 Justification: Thank You, The LCRC Accounts System

1 0

[LCRC Accounts] Yearly Allocation Request from Cosmology
by accounts＠lcrc.anl.gov 17 Sep '10

17 Sep '10

Hello, A yearly allocation for the LCRC cluster has been requested with the following updated information: Submitter/PI: Joe Bernstein Project Name: Cosmology Division: HEP Project title: Harnessing Computational Cosmology Simulations to Investigate Cosmic Evolution Associated funding: None external to ANL sources Other Systems: ANL Blue Gene/P (renewal in preparation for 1000K CPU-hrs over 4 months) Science: The evidence for dark energy represents one of the greatest mysteries of modern science. The proposed research will probe the implications of dark energy via analysis of computational cosmology simulations. Several independent observational methods provide evidence pointing to the conclusion that 95% of the current energy density of the Universe exists as part of the "dark sector" that is different from normal, i.e., "baryonic," matter. In particular, the dark sector is thought to be comprised of 22% "dark matter," which interacts with baryonic matter ostensibly only through gravity, and 73% "dark energy," which experiences gravity as a repulsive force. The nature of both dark sector components are essentially complete unknowns. It is presently an exciting time to be involved in cosmology because planned astronomical surveys will effectively result in dark sector probes becoming systematics-limited, making numerical simulations crucial to placing precision constraints on the dark sector. In addition, particle physics theorists predict that the Large Hadron Collider (LHC) and direct dark matter experiments may soon discover a dark matter candidate particle. ANL HEP staff and collaborating U. Illinois faculty are members of the Dark Energy Survey, which will provide the next generation of dark energy and (indirect) dark matter data. Local scientific simulation expertise and high-performance computing further make ANL ideally-suited to pursue answers to questions about the dark sector. The focus of the proposed research is the production and interpretation of computational cosmology simulations in order to elucidate the nature of the dark sector. Project description: The "Cosmology" project on fusion.lcrc.anl.gov (hereafter, "fusion") is in support of the one of the main research efforts of the PI as an ANL Computational Postdoctoral Fellow. The number of project members is currently 3 with a fourth to be added once the user account is approved. We will make use of the requested number of CPU-hrs (85K) in efforts to generate initial conditions and verify scaling results for cosmology simulations run on the ANL Blue Gene/P ("BG/P"). Toward this end, we will harness piecewise-parabolic method (PPM) hydrodynamics and N-body gravity solvers, MPI and OpenMP for inter-processor communication, and HDF5 to implement parallel I/O. We will utilize both the C and FORTRAN90 programming languages. Paul Ricker (U. Illinois) is a computational cosmology expert working with Boyana Norris (MCS) on BG/P cosmology studies. The PI of this proposal transitioned in Feb. 2010 from a classification as an ANL Postdoctoral Appointee to an ANL Computational Postdoctoral Fellow. His duties include collaborating on these efforts with the goal of submitting a DOE INCITE proposal. The ultimate target of the BG/P simulation effort is runs requiring PPM+N-body simulations with a 1 Gpc (1 pc = 3.09x10^16 m) box size, 2048^3 grid cells, and 10^11 dark matter particles. We anticipate employing fusion to generate initial conditions for a suite of a few dozen BG/P simulations. Generating such initial conditions is memory intensive and cannot currently be done on the BG/P. These requirements also push the limits of existing codes. Scaling the cosmology runs to the target level represents a marked increase relative to machines used to date. Verifying scaling results, on a traditional Linux cluster such as fusion, will constitute an important part of our BG/P studies. We are highly confident of fusion's suitability for the proposed simulations based on the FY2010 experience. Progress with the FY2010 fusion allocation was unexpectedly slow to due to unforeseen difficulties with running cosmology simulations on the BG/P architecture. However, with that allocation, we were able to identify the main causes of the lack of scaling on the BG/P and have finally succeeded in pursuing BG/P test runs. Comparison of results on fusion proved to be crucial to that effort. Thus, we anticipate continued use of fusion in such a role. Based on the increased understanding of efficient use of fusion gained in the past 6 months, we propose to complete a scaling study requiring a reduced amount of resources as compared to the FY2010 proposal. The scaling studies will consist 3 runs (256^3, 512^3, and 1024^3 particles) each for strong and weak scaling for a total of 6 runs. Based on Prof. Ricker's experience with PPM+N-body simulations performed at Oak Ridge National Laboratory, we estimate that a run with 2048^3 grid cells will cost approximately 400 CPU-hrs per timestep and require ~30 time steps for a total of 12K CPU-hrs. Assuming reasonable scaling, this implies a total cost of approximately 72K CPU-hrs for the scaling study. Generating initial conditions and performing test simulations will incur a significantly smaller cost by comparison. Thus, an allocation of 85K CPU-hrs will enable significant research progress. Project URL: Current FY Hours Used: undetermined amount New FY Requested allocation: 85000 Justification: Thank You, The LCRC Accounts System

1 0

[LCRC Accounts] Yearly Allocation Request from Neocortex_sim
by accounts＠lcrc.anl.gov 17 Sep '10

17 Sep '10

Hello, A yearly allocation for the LCRC cluster has been requested with the following updated information: Submitter/PI: Mark Hereld Project Name: Neocortex_sim Division: MCS Project title: Neocortical Seizure Simulation Associated funding: LDRD, NSF and UChicago Development Funds Other Systems: Science: Epilepsy remains a difficult disease to treat. Its causes are likely many and its genesis and development is not well understood. We are applying computational techniques to the study of models of neocortex tissue to extend our understanding of root causes and effects that compliment ongoing clinical and laboratory work at the University of Chicago Hospitals. Project description: Development of new scalable methods. Project URL: Current FY Hours Used: undetermined amount New FY Requested allocation: 8000 Justification: Thank You, The LCRC Accounts System

1 0