February 2017 - Allocations-admins - mailman3.cels.anl.gov

Re: [allocations-admins] [LCRC Accounts] Project Allocation Request
by Roberts, John E. 23 Feb '17

23 Feb '17

Time granted. -- John Roberts Argonne National Laboratory CELS Systems jeroberts(a)anl.gov On 2/20/17, 1:11 PM, "allocations-admins-bounces(a)lcrc.anl.gov on behalf of accounts(a)lcrc.anl.gov" <allocations-admins-bounces(a)lcrc.anl.gov on behalf of accounts(a)lcrc.anl.gov> wrote: Hello, A change in allocation has been requested: Requester: hyowon (Hyowon Park) Project: dmft_for_oxides Title: DFT+DMFT Calculations of Transition Metal Oxides Description: A DFT+DMFT code consists of a DFT part and a DMFT part. Most of modern DFT codes are efficiently parallelized such that simulations of large supercells can be performed. A DMFT calculation requires an accurate convergence of the quantum Monte Carlo method. The quantum Monte Carlo simulation is scalable almost linearly as the number of cpus. These codes can be compiled using the Intel compilers and MPI which are already installed in LCRC machines. Therefore, the parallel calculation using the clusters in LCRC will be inevitable for performing precise and efficient DFT+DMFT simulations. During FY 2016-17, the PIs will perform DFT+DMFT calculations to study an accurate description of strongly correlated materials including the ground-state structure, total energy, spectral functions, and magnetism. Specifically, we will study the strong interplay of spin-lattice-charge couplings in various materials including LaCoO3, nickelate superlattices, ferrites, and uranium dioxides. The total energy calculations and the new implementation of atomic force calculations will be adopted for the study of strucutral properties in these materials. These DFT+DMFT outputs can be directly compared to various spectroscopic experiments such as the photoemission, X-ray scattering, and neutron scattering experiments. Current: undetermined amount Justification: Requested: 100000 A specific reason has been given: Our allocation time is almost used up, so we need more time for calculations. This needs to be approved and the final allocation amount decided upon. Thank You, The LCRC Accounts System _______________________________________________ allocations-admins mailing list allocations-admins(a)lcrc.anl.gov https://lists.lcrc.anl.gov/mailman/listinfo/allocations-admins

1 0

Re: [allocations-admins] [LCRC Accounts] Project Allocation Request
by Bair, Raymond A. 23 Feb '17

23 Feb '17

I recommend we approve this request. ----------------------------------------- Ray Bair Argonne National Laboratory On 2/22/17, 9:10 PM, "allocations-admins-bounces(a)lcrc.anl.gov on behalf of accounts(a)lcrc.anl.gov" <allocations-admins-bounces(a)lcrc.anl.gov on behalf of accounts(a)lcrc.anl.gov> wrote: Hello, A change in allocation has been requested: Requester: jacob (Robert Jacob) Project: ACME Title: Accelerated Climate Modeling for Energy Description: ACME started with a branch of the Community Earth System Model (CESM). CESM is a fully-coupled, global climate model that provides state-of-the-art computer simulations of the Earth's past, present, and future climate states. ACME's atmosphere component is the Community Atmosphere Model. The CAM model has flexible formulations for atmospheric dynamics and has recently transitioned to the spectral finite element method coupled to an extensive suite of sub-grid physical parameterizations in its standard configuration. It runs on unstructured quadrilateral grids. The land model, CLM, contains a suite of column process parameterizations running at each grid point with no communication between grid points. CLM represents surface and subsurface water, energy, carbon, and nitrogen dynamics, with a nested hierarchical sub-grid parameterization that allows glaciers, lakes, urban areas, agricultural fields, managed forest, and natural (unmanaged) vegetation types to share space on each grid-cell. Incident radiation is intercepted in a two-layer canopy, with vegetation, soil, snow aging, and black carbon impacts on albedo. Subsurface processes include vertically resolved biogeochemistry, options for carbon and nutrient cycle parameterization, and recently improved treatment of wetlands and permafrost dynamics. In the fully-coupled configuration, the sea-ice and ocean (MPAS-CICE and MPAS-O) component models run with a nominal grid spacing of 1/10° (approximately 15km at the equator and 5km in polar regions) and, for MPAS-O, 100 levels in the vertical. CAM grids have both uniform and variable resolution configurations. The uniform grid with an average grid point spacing at the equator of 1/4° contains 86,400 elements, each with 30 vertical levels. Doubling the resolution to an average grid spacing of 1/8° requires 345,600 elements. The variable resolution configuration allows a small region of 1/8° grid spacing transitioning to an average global resolution of 1°. The CLM is run on the same grid as CAM. The output of the ACME consists of monthly means of several hundred quantities, plus daily averages of a subset of these quantities and hourly output of some key variabl es. ACME requires Fortran90 and C compilers and the MPI library to run. We will be using time on Blues to perform regular testing of ACME. ACME contains a large set of system tests that perform runs of the full model to test various functions. The runs sizes range from a few nodes to a few dozen nodes. The output sizes are relatively small and we will have modest storage needs. In FY17 we will continue regular testing with the gnu compiler and the large "acme_integration" test suite. In Current: undetermined amount Justification: Requested: 80000 A specific reason has been given: We should only need about 11K hours/month to run our regular testing. But a few ACME folks are accidentally running jobs in the batch instead of ACME queue. I'll remind them. This needs to be approved and the final allocation amount decided upon. Thank You, The LCRC Accounts System

1 0

Re: [allocations-admins] [LCRC Accounts] Project Allocation Request
by Bair, Raymond A. 23 Feb '17

23 Feb '17

I suggest we approve this request. ----------------------------------------- Ray Bair Argonne National Laboratory On 2/22/17, 6:56 PM, "allocations-admins-bounces(a)lcrc.anl.gov on behalf of accounts(a)lcrc.anl.gov" <allocations-admins-bounces(a)lcrc.anl.gov on behalf of accounts(a)lcrc.anl.gov> wrote: Hello, A change in allocation has been requested: Requester: neveu (Nicole Neveu) Project: AWA-beam-dynamics Title: High Accuracy Beam Dynamic Simulations Description: The primary goal of this project is to simulate the two linear electron accelerators at the Argonne Wakefield Accelerator facility. To date, a full 3D models of the beam lines has not been simulated. Using OPAL-T and the parallel capabilities of Blues, we seek better agreement between simulations and experiment by accounting for 3D effects, space charge, coherent synchrotron radiation, and wakefields. Current: undetermined amount Justification: Requested: 90000 A specific reason has been given: The current optimization project is larger than expected. In addition to the need for large random sample runs (to probe/define the parameter space), single optimization runs are taking a day or more to converge. Also, 3D RF field maps were recently incorporated, which adds time to any run, including those mentioned above. This needs to be approved and the final allocation amount decided upon. Thank You, The LCRC Accounts System _______________________________________________ allocations-admins mailing list allocations-admins(a)lcrc.anl.gov https://lists.lcrc.anl.gov/mailman/listinfo/allocations-admins

1 0

[LCRC Accounts] Project Allocation Request
by accounts＠lcrc.anl.gov 22 Feb '17

22 Feb '17

Hello, A change in allocation has been requested: Requester: jacob (Robert Jacob) Project: ACME Title: Accelerated Climate Modeling for Energy Description: ACME started with a branch of the Community Earth System Model (CESM). CESM is a fully-coupled, global climate model that provides state-of-the-art computer simulations of the Earth's past, present, and future climate states. ACME's atmosphere component is the Community Atmosphere Model. The CAM model has flexible formulations for atmospheric dynamics and has recently transitioned to the spectral finite element method coupled to an extensive suite of sub-grid physical parameterizations in its standard configuration. It runs on unstructured quadrilateral grids. The land model, CLM, contains a suite of column process parameterizations running at each grid point with no communication between grid points. CLM represents surface and subsurface water, energy, carbon, and nitrogen dynamics, with a nested hierarchical sub-grid parameterization that allows glaciers, lakes, urban areas, agricultural fields, managed forest, and natural (unmanaged) vegetation types to share space on each grid-cell. Incident radiation is intercepted in a two-layer canopy, with vegetation, soil, snow aging, and black carbon impacts on albedo. Subsurface processes include vertically resolved biogeochemistry, options for carbon and nutrient cycle parameterization, and recently improved treatment of wetlands and permafrost dynamics. In the fully-coupled configuration, the sea-ice and ocean (MPAS-CICE and MPAS-O) component models run with a nominal grid spacing of 1/10° (approximately 15km at the equator and 5km in polar regions) and, for MPAS-O, 100 levels in the vertical. CAM grids have both uniform and variable resolution configurations. The uniform grid with an average grid point spacing at the equator of 1/4° contains 86,400 elements, each with 30 vertical levels. Doubling the resolution to an average grid spacing of 1/8° requires 345,600 elements. The variable resolution configuration allows a small region of 1/8° grid spacing transitioning to an average global resolution of 1°. The CLM is run on the same grid as CAM. The output of the ACME consists of monthly means of several hundred quantities, plus daily averages of a subset of these quantities and hourly output of some key variabl es. ACME requires Fortran90 and C compilers and the MPI library to run. We will be using time on Blues to perform regular testing of ACME. ACME contains a large set of system tests that perform runs of the full model to test various functions. The runs sizes range from a few nodes to a few dozen nodes. The output sizes are relatively small and we will have modest storage needs. In FY17 we will continue regular testing with the gnu compiler and the large "acme_integration" test suite. In Current: undetermined amount Justification: Requested: 80000 A specific reason has been given: We should only need about 11K hours/month to run our regular testing. But a few ACME folks are accidentally running jobs in the batch instead of ACME queue. I'll remind them. This needs to be approved and the final allocation amount decided upon. Thank You, The LCRC Accounts System

1 0

[LCRC Accounts] Project Allocation Request
by accounts＠lcrc.anl.gov 22 Feb '17

22 Feb '17

Hello, A change in allocation has been requested: Requester: neveu (Nicole Neveu) Project: AWA-beam-dynamics Title: High Accuracy Beam Dynamic Simulations Description: The primary goal of this project is to simulate the two linear electron accelerators at the Argonne Wakefield Accelerator facility. To date, a full 3D models of the beam lines has not been simulated. Using OPAL-T and the parallel capabilities of Blues, we seek better agreement between simulations and experiment by accounting for 3D effects, space charge, coherent synchrotron radiation, and wakefields. Current: undetermined amount Justification: Requested: 90000 A specific reason has been given: The current optimization project is larger than expected. In addition to the need for large random sample runs (to probe/define the parameter space), single optimization runs are taking a day or more to converge. Also, 3D RF field maps were recently incorporated, which adds time to any run, including those mentioned above. This needs to be approved and the final allocation amount decided upon. Thank You, The LCRC Accounts System

1 0

[LCRC Accounts] Project Allocation Request
by accounts＠lcrc.anl.gov 22 Feb '17

22 Feb '17

Hello, A change in allocation has been requested: Requester: yyu (Yiqi Yu) Project: SHARP_ARC Title: Multi-physics Simulation of SFR Core Radial Expansion with SHARP Description: As the SHARP modeling and simulation toolkit is maturing, the focus is on customer-oriented applications and further validation exercises. This allocation will help with those applications, for performing small to moderate scale simulations and for scaling up to ALCF. The SHARP toolkit is regularly compiled on Blues using Intel or GNU compilers. The components of SHARP—the Nek5000 computational fluid dynamics code and the PROTEUS neutron transport code—have already been demonstrated to scale well not only on Blues but on Blue Gene/P. This project will be used by 4-5 analysts in the NE and MCS Divisions. Current: undetermined amount Justification: FY17 efforts for SHARP are highly focused on solving challenge problems for customers and code validation. FY16 efforts were focused on code development, as there was a major release, and so computational requirements were lower than usual. If the project is successful, we anticipate that FY17 and beyond will be highly resource intensive. The components of SHARP—the Nek5000 computational fluid dynamics code and the PROTEUS neutron transport code—have already been demonstrated to scale well not only on Fusion but on Blue Gene/P. Nek5000 has demonstrated > 80% parallel efficiency on BG/P (strong scaling) out to P=131,072 processors, using only 7300 points/processor at the largest scale. PROTEUS has demonstrated greater than 90% strong space-angle scaling at the 100,000 processer level and was nominated for the Gordon Bell Prize for its parallel performance. Requested: 1000000 A specific reason has been given: Used up all the time for now. have urgent job to finish before March This needs to be approved and the final allocation amount decided upon. Thank You, The LCRC Accounts System

1 0

[LCRC Accounts] Project Allocation Request
by accounts＠lcrc.anl.gov 21 Feb '17

21 Feb '17

Hello, A change in allocation has been requested: Requester: qiaobf (Baofu Qiao) Project: DMDOHEMA_1 Title: Metal Ion Transport through Nano-Structured Lipophilic Systems: Atomistic MD Simulations Description: In the last one year, large-scale atomistic molecular dynamics (MD) simulations were performed using the LCRC cluster. In the simulations, we were aiming to couple computer simulations with synchrotron X-ray scattering technique to understand the aggregation behaviors of lanthanide metal-bearing systems. These systems root their significant role in the refinery of lanthanide metals using the solvent extraction technique. We have shown the large-scale MD simulations are able to reproduce the X-ray scattering data (e.g., SAXS). Consequently, the experimental structural feature could be inferred from the simulation structures. Moreover, the energetics which drive the metal transport from aqueous phase to organic phase have been explained. We have published two papers, and submitted one paper. We are currently preparing some manuscripts, which will be submitted soon. Detailed discussion have been provided in the progress report. In the coming facial year, we are aiming to study two projects. In the first project, we are planning to study the interfacial behavior of alcohol at water-organic interface. The role of water-organic interface for the metal separations in the solvent extraction process is still unknown to us. In the present project, we are studying a simple model system of alcohol/water/organic biphasic solution. We are aiming to study the surface activity of alcohol molecules at water-organic interface. Alcohol molecular was sometimes used as co-extractant to enhance the transport kinetics of metal from aqueous phase to organic phase. In our preliminary simulations using the classical pairwise CGenFF force field, the aggregation of alcohol was observed to form a monolayer at the water-dodecane biphasic solution. More interestingly, a second layer of alcohol is also observed in some systems, even though it is relatively very weak. This suggests that the alcohols are preorganized into an asy mmetric bilayer at the water-organic interface, which might explain the elevated extraction kinetics of metal transport across the interface. Such an asymmetric bilayer has never been reported to the best of our knowledge. In the coming facial year, we are aiming to run MD simulations using polarizable force field. These polarizable force field simulations will be used to double check the influences of the simulations force field. In these project, polarizable molecular dynamics (MD) simulations will be performed using the Drude force field. Drude force field has been well developed for simple systems as in the present project. We will run the simulations using the NAMD simulation package. The NAMD package supports Drude force field, and scales well up to thousands of CPUs. High impact publications are expected based on the novel finding, and its significance in both the fundamental research and technical application. In the second project, we are aiming to continue our existing project to study the surface feature of metallate (platinum group metal). In collaboration with the SFG experiments led by Dr. Ahmet Uysal (Assistant Chemist at CSE), we are aiming to study the surface aggregation feature of different metallate (i.e, PtCl62- and PdCl42-), and understand their roles of in metallate separation. We have obtained some results. And Some more simulations are planned to complete the project. Some papers are expected when appropriate. Current: undetermined amount Justification: Note that this account (DMDOHEMA_1) was very recently granted 200,000 CPU hours. Therefore, I am not asking for the allocation for the first two quarters. Requested: 200000 A specific reason has been given: Hi, Could you please provide me more allocation? I am currently in the middle of finishing some simulations to study the metal-surfactant interactions. But the remaining allocation is apparently not enough. Thanks a lot! regards, Baofu This needs to be approved and the final allocation amount decided upon. Thank You, The LCRC Accounts System

1 0

[LCRC Accounts] Project Request: pozdyn
by accounts＠lcrc.anl.gov 21 Feb '17

21 Feb '17

Hello, A new project on the LCRC cluster has been requested. Please forward the information on to the LCRC Allocation sub-committee. Applicant's name: Mark Pfeifle Applicant's institution: ANL Applicant's division: CSE Project Name: pozdyn Project title: Dissociation Dynamics of Primary Ozonides in Alkene Ozonolysis Associated funding: DAAD (German Academic Exchange Service) Other Systems: Linus (CSE) (no fixed time allocation, depending on current total load) Science: Ozonolysis is a major tropospheric degradation process for alkenes. In the first reaction step, a cyclic primary ozonide (POZ) is formed which is subsequently decomposed into a Criegee intermediate (CI) and a carbonyl compound. The CI population is initially hot due to the exothermic POZ formation. This nascent energy distribution of the CI is essential to adequately model the yields of promptly decomposing and stabilized CI. These quantities in turn are of considerable interest because the thermalized CI acts as an atmospheric oxidant for various species and enhances aerosol formation. It is commonly assumed that the excess energy is distributed statistically among the fragments in the spirit of phase-space theory. However, this assumption is questionable when the exit channel potential energy surface is strongly repulsive, as is the case with POZ decomposition. In this project, direct dynamics calculations for C2H4 + O3 are carried out. Besides con certed POZ cleavage, a step-wise diradical pathway contributes comparably to the reaction flux. Density functional theory based trajectory calculations, initiated at both exit transition states, will serve as a basis to test the statistical partitioning assumption and to provide appropriate input distributions for CI master equation calculations. Project description: The trajectory calculations will be carried out using a self-made driver program by one of our group members (http://tcg.cse.anl.gov/papr/codes/dint.html) It interfaces either the Gaussian 09 or Molpro 2012/2015 program packages to do the actual electronic structure calculations. One typical trajectory run (ca. 1000 time steps using 4 gradient evaluations per iteration) takes about two days on a single processor with density functional theory methods and a moderate-sized one-electron basis set (Gaussian 09). We expect to carry out around 200-300 trajectory runs to achieve statistically significant results. This has to be done for two different transition states that are kinetically relevant. The DFT calculations can be run in parallel in principle. However, according to initial tests, the parallelization efficiency is not optimal for these kind of jobs. Therefore, the quantum-chemical calculations themselves will be run as single-core jobs. Parallelization will be simply achieved by running N separate trajectory jobs on one node (e.g. N=16 for the blues machines). The computational burden within the driver routine is negligible compared to the cost of the electronic structure calculations themselves. In addition to the DFT runs, CASPT2 calculations using Molpro are planned for at least one of the transition states. These are computationally more expensive than DFT calculations, but otherwise the same considerations about parallization apply in this case, too. Presumably, only one member will work on this project. Industry partnership: Project URL: Requested allocation: 54000 Q1: 0 Q2: 18000 Q3: 18000 Q4: 18000 Justification: Storage requirements: The requester has used undetermined amount hours of their initial startup project. In addition to approving an initial amount, please specify a Category and Subcategory for this project. For a list of the current selection of approved categories, please see: https://wiki.lcrc.anl.gov/wiki/Processes/Categories Once the Allocation committee has approved the project, please go to the Project Management page to create it: https://accounts.lcrc.anl.gov/projects.php Thank You, The LCRC Accounts System

1 0

[LCRC Accounts] Project Allocation Request
by accounts＠lcrc.anl.gov 20 Feb '17

20 Feb '17

Hello, A change in allocation has been requested: Requester: pbalapra (Prasanna Balaprakash) Project: perfopt Title: Optimization Algorithms for Empirical Performance Description: There are different strategies for performance model construction. Analytical approaches for performance modeling are cheaper in the sense that they don't suffer from the profiling and model generation cost as in the case of machine learning techniques. But analytical models are often difficult to construct for complex programs and they require detailed understanding of the underlying hardware and implementation of communication functions. Machine learning techniques come into play when it is difficult to construct an analytical model for an application. Models generated using machine learning techniques may not be as highly precise as an analytical model but still they help to understand the performance of an application that in turn help in optimizations such as auto tuning. In this research, we first try to build analytical models for communication of various application kernels. We use architecture-independent parameters as described in well known communication models such as LogGP family to build the analytical models. If the constructed analytical model is accurate enough in predicting the communication time, it saves the huge overhead in gathering training data for the application of machine learning algorithms. We will test the accuracy of the developed analytical models in predicting the communication time of several application kernels from the CORAL benchmarks. If the analytical model is not accurate enough (the error in prediction goes above acceptable range), we switch to gathering profile data and then generate models using machine learning techniques on that data. This hybrid approach gives us a balance between the complexity of building analytical models and the overhead of running expensive machine learning algorithms on huge amount of collected profiling data. Also the machine learning models are limited in the sense that we can't do extrapolation. Fitted analytical models come into play in that case. On Blues, we will conduct an experimental study of communication models. A major benefit Blues brings to this research is the ability to test the impact of MPI implementations and interconnect techniques on Intel-based clusters. Eventually, we hope to deploy our techniques for improving the performance of scientific applications running on Blues. We estimate 100,000 core hours for our experimental study: evaluation of the strengths and limitations of LogGP models = 15,000 core hours; experimental analysis on multiple problem classes on different architectures = 30,000 core hours; empirical analysis for new approaches = 30,000 core hours; tests on parallel performance tuning approaches = 25,000 core hours; We will strongly involve in research dissemination activities such as writing scientific articles and giving seminars at Argonne and in conferences. In the the first and second quarter, our jobs will be 80% parallel. For the third and fourth quarter, we will have 100%. Current: undetermined amount Justification: Requested: 80000 A specific reason has been given: Q2: 15000 Q3: 40000 Q4: 25000 This needs to be approved and the final allocation amount decided upon. Thank You, The LCRC Accounts System

1 0

[LCRC Accounts] Project Allocation Request
by accounts＠lcrc.anl.gov 20 Feb '17

20 Feb '17

Hello, A change in allocation has been requested: Requester: hyowon (Hyowon Park) Project: dmft_for_oxides Title: DFT+DMFT Calculations of Transition Metal Oxides Description: A DFT+DMFT code consists of a DFT part and a DMFT part. Most of modern DFT codes are efficiently parallelized such that simulations of large supercells can be performed. A DMFT calculation requires an accurate convergence of the quantum Monte Carlo method. The quantum Monte Carlo simulation is scalable almost linearly as the number of cpus. These codes can be compiled using the Intel compilers and MPI which are already installed in LCRC machines. Therefore, the parallel calculation using the clusters in LCRC will be inevitable for performing precise and efficient DFT+DMFT simulations. During FY 2016-17, the PIs will perform DFT+DMFT calculations to study an accurate description of strongly correlated materials including the ground-state structure, total energy, spectral functions, and magnetism. Specifically, we will study the strong interplay of spin-lattice-charge couplings in various materials including LaCoO3, nickelate superlattices, ferrites, and uranium dioxides. The total energy calculations and the new implementation of atomic force calculations will be adopted for the study of strucutral properties in these materials. These DFT+DMFT outputs can be directly compared to various spectroscopic experiments such as the photoemission, X-ray scattering, and neutron scattering experiments. Current: undetermined amount Justification: Requested: 100000 A specific reason has been given: Our allocation time is almost used up, so we need more time for calculations. This needs to be approved and the final allocation amount decided upon. Thank You, The LCRC Accounts System

1 0