June 2012 - Allocations-admins - mailman3.cels.anl.gov

Re: [allocations-admins] [LCRC Accounts] Project Request: 7_SEAS_WRF
by Bair, Raymond A. 26 Jun '12

26 Jun '12

This looks fine to me...50K for Q4 FY12. It will have the same classification as the GVAX project, and prio 3. Ray On 6/25/12 4:49 PM, "accounts(a)lcrc.anl.gov" <accounts(a)lcrc.anl.gov> wrote: > Hello, > > A new project on the LCRC cluster has been requested. Please forward > the information on to the LCRC Allocation sub-committee. > > Applicant's name: Rao Kotamarthi > Applicant's institution: ANL > Applicant's division: DIS, PI: David Streets > Project Name: 7_SEAS_WRF > Project title: Forecasting for the 7-SEAS Study > Associated funding: NASA, DOE > Other Systems: None > Science: SEAC4RS will address the influence of Asian > emissions on atmospheric chemistry, clouds, climate and air quality, with > specific focus on studying the redistribution of atmospheric composition under > Asian monsoon circulation and convection conditions. The primary goals of the > experiment are to provide key observations to help validate satellite > observations under these conditions, and to advance our understanding of the > role of deep convection in redistributing pollutants emitted in the tropics > and the subsequent influence of these pollutants on the chemistry of the upper > troposphere, and the influence of feedbacks between the pollutants and local > meteorology through direct, semi-direct and indirect radiative effects. The > successful execution of the mission and the interpretation of the observations > to meet the science objectives require contributions from Chemical Transport > Models (CTMs). CTMs play important roles in the design, execution, and analysi > s of large-scale atmospheric chemistry field studies. They are used in > forecast-mode to enhance flight planning by enabling the representation of > important three-dimensional atmospheric chemical structures (such as dust > storm plumes, polluted air masses associated with large cities, widespread > biomass burning events, and large scale convective redistribution events) and > how they evolve over time. CTM forecasts play the additional important role of > providing a 4-dimensional contextual representation of the experiment. CTMs > also facilitate the integration of the different measurements and measurement > platforms (e.g., aircraft, ground stations and satellite observations). > Finally, CTMs are used to help evaluate and improve emission estimates. > Project description: We have developed an operational regional-scale > forecasting and analysis system to assist in atmospheric field experiments. > The system consists of 3 major components: 1) detailed mesoscale > meteorological model with an off-line CTM and a wide variety of air mass and > emission tracers; 2) detailed 3-dimensional photochemical and radiative > transfer calculations using a CTM; and 3) enhanced emission products that > intimately link emitted amounts and activities to the transport and chemistry > analysis. This system was successfully applied in the design and execution of > the NASA TRACE-P, INTEX A & B and ARCTAS experiments, the NSF ACE-Asia, > MILAGRO, Pacific Dust Experiment (PACDEX) and VOCALS REx experiments, and the > NOAA ITCT-2K2 and ICARTT intensive field experiments. In these previous > applications we have demonstrated that these models can be used effectively in > the execution and analysis of field experiments. Forecast products, emissions > and publications > from these previous missions can be found at > http://www.cgrer.uiowa.edu/ACESS/acess_index.htm. > > We plan to generate SEAC4RS with once per day WRF-Chem forecasts of hourly > meteorology and chemical transport with fully coupled feedbacks. The WRF-Chem > forecasts will be of particular use in this experiment as the model is > well-suited for simulating regional-scale redistribution by convection (see > Section 2.3 for examples) and evaluating feedbacks between pollution and > meteorology. The on-line coupling allows better temporal resolution and > coupling between the processes, and the higher spatial resolution that we plan > to use will directly resolve cloud processes. These features in WRF-Chem will > be used to generate a range of new mission support products tailored to > specific objectives and needs of the SEAC4RS experiment, such as convective > indices, convective outflow regions, and indices of locations and time periods > where new particle activation and aerosol radiative forcing are strongest. > > > > Project URL: http://espo.nasa.gov/missions/seac4rs > Requested allocation: 170000 > Q1: 50000 > Q2: 50000 > Q3: 20000 > Q4: 50000 > Justification: WRF-CHEM model is operational on the Fusion cluster > and used by the Dr. Kotamarthi and Dr. Yan Feng of EVS. The regional climate > model scales up to 72 nodes and needs ~10 hours to complete a 72-hr forecast. > The speedup achieved with 72 processors is about 13 relative to 4 processors. > Memory used on each node decreases from 4.5 GB, 1.5GB, 1.1 GB, to 0.9 GB while > the number of nodes increases from 4, 16, 36, to 72 nodes. The above estimated > time does not include pre-processing of the NCEP meteorology fields, > post-processing and visualization of the forecast results. These numbers are > generated for simulation over domain covering the entire South Asia and we > expect the domain size and time requirements to be similar for the proposed > simulations over Southeast Asia. > > > The requester has used undetermined amount hours of their initial startup > project. > > In addition to approving an initial amount, please specify a Category > and Subcategory for this project. For a list of the current selection > of approved categories, please see: > > https://wiki.lcrc.anl.gov/wiki/Processes/Categories > > Once the Allocation committee has approved the project, please go to > the Project Management page to create it: > > https://accounts.lcrc.anl.gov/projects.php > > Thank You, > The LCRC Accounts System > _______________________________________________ > allocations-admins mailing list > allocations-admins(a)lists.lcrc.anl.gov > https://lists.lcrc.anl.gov/mailman/listinfo/allocations-admins ----------------------------------------- Ray Bair Computing, Environment, and Life Sciences Argonne National Laboratory and the University of Chicago TCS Building 240, Room 4122 9700 South Cass Avenue Argonne, IL 60439 email: rbair(at)anl.gov Phone: (630)252-5751

1 0

[LCRC Accounts] Project Request: 7_SEAS_WRF
by accounts＠lcrc.anl.gov 25 Jun '12

25 Jun '12

Hello, A new project on the LCRC cluster has been requested. Please forward the information on to the LCRC Allocation sub-committee. Applicant's name: Rao Kotamarthi Applicant's institution: ANL Applicant's division: DIS, PI: David Streets Project Name: 7_SEAS_WRF Project title: Forecasting for the 7-SEAS Study Associated funding: NASA, DOE Other Systems: None Science: SEAC4RS will address the influence of Asian emissions on atmospheric chemistry, clouds, climate and air quality, with specific focus on studying the redistribution of atmospheric composition under Asian monsoon circulation and convection conditions. The primary goals of the experiment are to provide key observations to help validate satellite observations under these conditions, and to advance our understanding of the role of deep convection in redistributing pollutants emitted in the tropics and the subsequent influence of these pollutants on the chemistry of the upper troposphere, and the influence of feedbacks between the pollutants and local meteorology through direct, semi-direct and indirect radiative effects. The successful execution of the mission and the interpretation of the observations to meet the science objectives require contributions from Chemical Transport Models (CTMs). CTMs play important roles in the design, execution, and analysi s of large-scale atmospheric chemistry field studies. They are used in forecast-mode to enhance flight planning by enabling the representation of important three-dimensional atmospheric chemical structures (such as dust storm plumes, polluted air masses associated with large cities, widespread biomass burning events, and large scale convective redistribution events) and how they evolve over time. CTM forecasts play the additional important role of providing a 4-dimensional contextual representation of the experiment. CTMs also facilitate the integration of the different measurements and measurement platforms (e.g., aircraft, ground stations and satellite observations). Finally, CTMs are used to help evaluate and improve emission estimates. Project description: We have developed an operational regional-scale forecasting and analysis system to assist in atmospheric field experiments. The system consists of 3 major components: 1) detailed mesoscale meteorological model with an off-line CTM and a wide variety of air mass and emission tracers; 2) detailed 3-dimensional photochemical and radiative transfer calculations using a CTM; and 3) enhanced emission products that intimately link emitted amounts and activities to the transport and chemistry analysis. This system was successfully applied in the design and execution of the NASA TRACE-P, INTEX A & B and ARCTAS experiments, the NSF ACE-Asia, MILAGRO, Pacific Dust Experiment (PACDEX) and VOCALS REx experiments, and the NOAA ITCT-2K2 and ICARTT intensive field experiments. In these previous applications we have demonstrated that these models can be used effectively in the execution and analysis of field experiments. Forecast products, emissions and publications from these previous missions can be found at http://www.cgrer.uiowa.edu/ACESS/acess_index.htm. We plan to generate SEAC4RS with once per day WRF-Chem forecasts of hourly meteorology and chemical transport with fully coupled feedbacks. The WRF-Chem forecasts will be of particular use in this experiment as the model is well-suited for simulating regional-scale redistribution by convection (see Section 2.3 for examples) and evaluating feedbacks between pollution and meteorology. The on-line coupling allows better temporal resolution and coupling between the processes, and the higher spatial resolution that we plan to use will directly resolve cloud processes. These features in WRF-Chem will be used to generate a range of new mission support products tailored to specific objectives and needs of the SEAC4RS experiment, such as convective indices, convective outflow regions, and indices of locations and time periods where new particle activation and aerosol radiative forcing are strongest. Project URL: http://espo.nasa.gov/missions/seac4rs Requested allocation: 170000 Q1: 50000 Q2: 50000 Q3: 20000 Q4: 50000 Justification: WRF-CHEM model is operational on the Fusion cluster and used by the Dr. Kotamarthi and Dr. Yan Feng of EVS. The regional climate model scales up to 72 nodes and needs ~10 hours to complete a 72-hr forecast. The speedup achieved with 72 processors is about 13 relative to 4 processors. Memory used on each node decreases from 4.5 GB, 1.5GB, 1.1 GB, to 0.9 GB while the number of nodes increases from 4, 16, 36, to 72 nodes. The above estimated time does not include pre-processing of the NCEP meteorology fields, post-processing and visualization of the forecast results. These numbers are generated for simulation over domain covering the entire South Asia and we expect the domain size and time requirements to be similar for the proposed simulations over Southeast Asia. The requester has used undetermined amount hours of their initial startup project. In addition to approving an initial amount, please specify a Category and Subcategory for this project. For a list of the current selection of approved categories, please see: https://wiki.lcrc.anl.gov/wiki/Processes/Categories Once the Allocation committee has approved the project, please go to the Project Management page to create it: https://accounts.lcrc.anl.gov/projects.php Thank You, The LCRC Accounts System

1 0

[LCRC Accounts] Project Request: biodiesel-combustion
by accounts＠lcrc.anl.gov 24 Jun '12

24 Jun '12

Hello, A new project on the LCRC cluster has been requested. Please forward the information on to the LCRC Allocation sub-committee. Applicant's name: Sibendu Som Applicant's institution: ANL Applicant's division: Energy Systems Project Name: biodiesel-combustion Project title: Spray and Combustion modeling of Biodiesel from different feedstocks Associated funding: DOE - Office of Vehicle Technologies Other Systems: Science: This project will use an Eulerian-Lagrangian Computation Fluid Dynamics (CFD) software called CONVERGE to model biodiesel spray and combustion processes in a compression ignition engine. The goal is to incorporate detailed chemical kinetic models for biodiesel combustion in order to improve the predictive capability of the simulations. Project description: In this project, mechanisms for biodiesel fuel from different feedstocks will be implemented in CONVERGE. The predictive capability of these mechanisms will assessed based on ignition delay and flame lift-off length data available in literature. Different sizes of mechanisms in terms of number of species and reactions will be tested. Validation of the mechanism will be performed against different geometries such as Caterpillar Single cylinder engine, Ignition Quality tester, and constant volume combustion chamber data available in literature. State-of-the-art spray models developed by the PIs will also be tested together with the chemical kinetic mechanisms. Project URL: http://www.transportation.anl.gov/ Requested allocation: 100000 Q1: 0 Q2: 0 Q3: 0 Q4: 100000 Justification: The requester has used undetermined amount hours of their initial startup project. In addition to approving an initial amount, please specify a Category and Subcategory for this project. For a list of the current selection of approved categories, please see: https://wiki.lcrc.anl.gov/wiki/Processes/Categories Once the Allocation committee has approved the project, please go to the Project Management page to create it: https://accounts.lcrc.anl.gov/projects.php Thank You, The LCRC Accounts System

1 0

[LCRC Accounts] Project Request: cfd_rcmmodeling
by accounts＠lcrc.anl.gov 22 Jun '12

22 Jun '12

Hello, A new project on the LCRC cluster has been requested. Please forward the information on to the LCRC Allocation sub-committee. Applicant's name: Michael Johnson Applicant's institution: ANL Applicant's division: ES Project Name: cfd_rcmmodeling Project title: Computation Fluid Dynamics Modeling of RCM performance Associated funding: DOE funding Other Systems: Science: This project will use an Eulerian-Lagrangian Computation Fluid Dynamics (CFD) software called CONVERGE to model various Rapid Compression Machine (RCM) geometries. The goal is to accurate simulate the RCM operation in order to optimize system geometries, accurately predict heat loss, and model new system performance. Project description: The project is dived in three parts: In the first part the known geometry of the ANL RCM will be studied. Initially, model validation will be done with both coarse and fine meshes without crevices. This will be compared to hand calculations and will use various system parameters. Then creviced piston geometries will be studied and results compared to previous work to validate the approach. The second part will be the development and validation of a physics based heat loss model applicable across a range of system parameters and geometries. Initial work will be checked against previous work, then extended to include new geometries and RCM trajectories. Finally, the third part will be using Converge to optimize new system components to ensure proper functioning of the new equipment. Project URL: http://www.transportation.anl.gov/ Requested allocation: 50000 Q1: 12500 Q2: 12500 Q3: 12500 Q4: 12500 Justification: The requester has used undetermined amount hours of their initial startup project. In addition to approving an initial amount, please specify a Category and Subcategory for this project. For a list of the current selection of approved categories, please see: https://wiki.lcrc.anl.gov/wiki/Processes/Categories Once the Allocation committee has approved the project, please go to the Project Management page to create it: https://accounts.lcrc.anl.gov/projects.php Thank You, The LCRC Accounts System

1 0

[LCRC Accounts] Project Request: DVI-Material
by accounts＠lcrc.anl.gov 08 Jun '12

08 Jun '12

Hello, A new project on the LCRC cluster has been requested. Please forward the information on to the LCRC Allocation sub-committee. Applicant's name: lei wang Applicant's institution: ANL Applicant's division: MCS Project Name: DVI-Material Project title: Dierential Variational Inequalities for Heterogeneous Materials Associated funding: FastMath (PI: Barry Smith) Other Systems: Science: This project uses dierential variational inequalities (DVIs) for heterogeneous materials problems, which describe the mesoscale behavior of irradiate damaged materials. The phase eld approach is used for modeling microstructure in the mesoscale. Since the diuse boundary between phases must be localized, one needs to use double obstacle potentials for generating the free energy functional. This results in a dierential variational inequality. Our aim is to develop software for large-scale DVIs. Project description: Mesoscale materials modeling is an important and challeng- ing problem. It is computationally intensive because the evolution of each diusive boundary between radiation-generated voids and the surrounding matrix must be correctly predicted. We employ coupled Cahn-Hilliard and Allen-Cahn systems with a double obstacle free energy potential to simulate the physics. The model is discretized in time with mixed implicit-explicit integration and in space by nite elements. However, a naive nite element approximation does not guarantee that the discrete solution satises appro- priate constraints. Therefore, we formulate a DVI, which is equivalent to a complementarity problem. This approach in turn allows us to use parallel matrix-free solvers for complementarity problems in PETSc and TAO. We are expanding the non-linear solver part of PETSc and investigating active- set algorithms to solve the algebraic DVIs that arise at each time step of these mesoscale simulations. We have validated the DVI approach agains the baseline results of the Center for Materials Science of Nuclear Fuel (CMSNF). We employed much larger time steps than the conventional nite dierence and nite element methods, which is one of the projected benets of the DVI formulation.We have completed preliminary numerical experiments and mathematical anal- ysis for both the Allen-Cahn and Cahn-Hilliard systems that demonstrate mesh-independent convergence rates for this preconditioner. That is, the work required to solve a single timestep grows linearly with the number of unknowns. Project URL: Requested allocation: 50000 Q1: 10000 Q2: 10000 Q3: 10000 Q4: 20000 Justification: 1. scaling data The requester has used undetermined amount hours of their initial startup project. In addition to approving an initial amount, please specify a Category and Subcategory for this project. For a list of the current selection of approved categories, please see: https://wiki.lcrc.anl.gov/wiki/Processes/Categories Once the Allocation committee has approved the project, please go to the Project Management page to create it: https://accounts.lcrc.anl.gov/projects.php Thank You, The LCRC Accounts System

1 0

[LCRC Accounts] Project Allocation Request
by accounts＠lcrc.anl.gov 08 Jun '12

08 Jun '12

Hello, A change in allocation has been requested: Requester: wild (Stefan Wild) Project: UNEDF_Optimization Title: Optimization of nuclear-energy DFT functionals Description: This DFT parameter optimization is expensive: each simulation of a single nucleus at a particular parameter set using HFBTHO can consume up to .3 CPU-hours. We have recently used OpenMP at the nucleus level, in addition to compiler-based optimization, to bring the wall time per simulation down to 1.5 minutes on a single node of Fusion. Our current optimizations over 14 parameters typically require 200 evaluations of about 100 nuclei, which can be done on Fusion in 5-7.5 hours using 100 8-core nodes. This gives us around 6K CPU-hours for each version of the functional we want to optimize. We note that our current computational needs for a single optimization, 100 nodes for 7.5 hours, yields a significant improvement in wall time over our previous needs of 12 nodes for 90 hours. These optimizations are being done using the derivative-free optimization software POUNDerS developed at Argonne. POUNDerS is available in the Toolkit of Advanced Optimization (TAO), but we still prototype specialized versions of the algorithm using MATLAB on a single, master node. We will test updates to the TAO version on Fusion and compare the performance and correctness against the MATLAB versions. We will also pursue optimizations that depend on nuclear physics using codes other than HFBTHO and plan to begin work with the codes HFODD and MFDn in FY12. Our team of 4 users anticipate doing 40 optimization runs during this project cycle. We also expect to analyze functionals based on more nuclei and depending on more parameters, potentially resulting in further increases to the number of nodes we require. Current: undetermined amount Justification: Requested: 70000 A specific reason has been given: In case it wasn't clear, we will need 70k hours through the end of the fiscal year. I estimate that we will use roughly 10k of these by the end of June and the remaining 60k for the last quarter (of course 70k hours that could be used flexibly across both quarters would be preferred.) This needs to be approved and the final allocation amount decided upon. Thank You, The LCRC Accounts System

1 0

[LCRC Accounts] Project Allocation Request
by accounts＠lcrc.anl.gov 07 Jun '12

07 Jun '12

Hello, A change in allocation has been requested: Requester: wild (Stefan Wild) Project: UNEDF_Optimization Title: Optimization of nuclear-energy DFT functionals Description: This DFT parameter optimization is expensive: each simulation of a single nucleus at a particular parameter set using HFBTHO can consume up to .3 CPU-hours. We have recently used OpenMP at the nucleus level, in addition to compiler-based optimization, to bring the wall time per simulation down to 1.5 minutes on a single node of Fusion. Our current optimizations over 14 parameters typically require 200 evaluations of about 100 nuclei, which can be done on Fusion in 5-7.5 hours using 100 8-core nodes. This gives us around 6K CPU-hours for each version of the functional we want to optimize. We note that our current computational needs for a single optimization, 100 nodes for 7.5 hours, yields a significant improvement in wall time over our previous needs of 12 nodes for 90 hours. These optimizations are being done using the derivative-free optimization software POUNDerS developed at Argonne. POUNDerS is available in the Toolkit of Advanced Optimization (TAO), but we still prototype specialized versions of the algorithm using MATLAB on a single, master node. We will test updates to the TAO version on Fusion and compare the performance and correctness against the MATLAB versions. We will also pursue optimizations that depend on nuclear physics using codes other than HFBTHO and plan to begin work with the codes HFODD and MFDn in FY12. Our team of 4 users anticipate doing 40 optimization runs during this project cycle. We also expect to analyze functionals based on more nuclei and depending on more parameters, potentially resulting in further increases to the number of nodes we require. Current: undetermined amount Justification: Requested: 70000 A specific reason has been given: We are nearing a zero balance and, as expected (note that we requested 140k hours for the second half of FY12 and received roughly half that due to machine constraints), require additional hours to finish off runs for this project and prepare for its successor SciDAC-3 project. Thank you, SW This needs to be approved and the final allocation amount decided upon. Thank You, The LCRC Accounts System

1 0

[LCRC Accounts] Project Allocation Request
by accounts＠lcrc.anl.gov 01 Jun '12

01 Jun '12

Hello, A change in allocation has been requested: Requester: sinclair (Donald Sinclair) Project: Lattice-QCD Title: Lattice simulations of Conformal and Walking Technicolor. Description: We perform simulations to evaluate the functional integrals of QCD-like theories formulated on a discrete space-time lattice to enable determination of the non-perturbative aspects these theories. These include the properties of these theories at non-zero temperature including the scales of confinement and chiral symmetry breaking, and such zero temperature properties as spectra, decay constants and the running of the gauge coupling constant. We are particularly interested in those theories where the coupling constant evolves particularly slowly, since these are candidate 'Walking Technicolor' theories. Related to these are theories with an infrared fixed point. Of special interest is to differentiate these two different types of behaviour. We are performing simulations of QCD-like theories which are models for Walking or Conformal Technicolor. We have been studying theories which are essentially QCD but with colour-sextet rather than colour-triplet quarks. We hope to measure the running of the QCD coupling constant. For 2 or 3 flavours, 2-loop perturbation theory predicts an infrared fixed point. For 2 flavours, it is possible that a chiral condensate forms before this fixed point is reached. If so the fixed point is avoided, the theory is confining and chiral symmetry breaks spontaneously. However, there is a region where the coupling constant evolves very slowly. These are the properties required for a walking technicolor theory. Simulations we have performed so far at finite temperature suggest that this theory does indeed walk. This is an ongoing program at NERSC. We wish to run small-lattice simulations of the 3-flavour theory, which we expect to have conformal, rather than walking behaviour on Fusion, for comparison. If we are correct, the properties of the 2 and 3 flavour theories should look rather different. Our sextet quark codes are based on our earlier triplet quark codes and use the RHMC simulation method. The Rational Hybrid Monte Carlo (RHMC) is a stochastic molecular dynamics algorithm. The functional integral of QCD is written as a partition function of a classical field theory evolving in a fictitious time. The determinant of the Dirac operator raised to a fractional power is calculated by introducing bosonic fields (pseudofermions), and sandwiching this Dirac operator raised to minus said fractional power between them. This fractional power of the Dirac operator is approximated to machine accuracy by a rational approximation. After defining this this theory on a discrete space-time lattice, the inversions required by the partial-fraction expansion of the rational approximation are performed using Krylov space methods, in particular a multi-shift extension of the conjugate gradient algorithm. A global Metropolis Monte-Carlo accept/reject step applied at the end of each t rajectory removes discretization errors introduced by the numerical integration of these stochastic equations of motion. We parallelize the code by assigning a fixed number of adjacent lattice sites to each MPI task. Network bandwidth ultimately limits how small a chunk of the lattice can be assigned to each task. Benchmarks of this code on Fusion for the 24^3*12 lattice planned for some of our FY2011 runs give: 24 cores = 46 Gflops 48 cores = 95 Gflops 72 cores = 148 Gflops 96 cores = 220 Gflops 144 cores = 369 Gflops 288 cores = 784 Gflops Our actual runs planned for Fusion are on a 16^3*8 lattice running on 128 cores. For these runs the per-core performance is about 3 Gflops Current: undetermined amount Justification: The per core performance for the 16^3*8 runs is approximately 3 Gflops on 128 cores. Althought we have not performed such a detailed analysis as for the 24^3*12 lattice, presented above, we expect at worst a linear speedup from the minimum number of cores (8) to the maximum number of cores (128). This code/lattice size has been running effectively on Fusion since it was released to the user community. We will be concentrating on the 3-flavour runs at m=0.01, 0.005, leaving the m=0.0025 runs for Carver at NERSC. The larger lattice runs will be performed on the Crays at NERSC and NICS. This corresponds to having an average of 1 job running for 7500 hours (out of a total of 8784 hours) for the year, which is easy to achieve. Requested: 250000 A specific reason has been given: I have exhausted my current allocation, but have yet to finish the subproject that I am currently running on Fusion. This needs to be approved and the final allocation amount decided upon. Thank You, The LCRC Accounts System

1 0