March 2016 - Allocations-admins - mailman3.cels.anl.gov

[LCRC Accounts] Project Request: CMFD-RANS
by accounts＠lcrc.anl.gov 31 Mar '16

31 Mar '16

Hello, A new project on the LCRC cluster has been requested. Please forward the information on to the LCRC Allocation sub-committee. Applicant's name: Dillon Shaver Applicant's institution: ANL Applicant's division: NE Project Name: CMFD-RANS Project title: Development of Multiphase RANS turbulence models Associated funding: NEAMS FSI Heat Exchanger LDRD Other Systems: Mira - shared project has ~25M CPU hours Science: The objective of this project is to develop and implement Unsteady Reynolds Averaged Navier-Stokes (uRANS) turbulence models in the Nek5000/Nek2P CFD code for use in multi-phase flow simulations. Project description: RANS and uRANS turbulence models are necessary for the accurate CFD simulation of flows on large scales, such as nuclear reactor fuel rod bundles and full cores. The current state-of-the art for simulation of boiling flows utilizes the RANS framework. Currently, a k-omega turbulence model has been demonstrated with Nek in the single-phase formulation. The goal of this project will be to extend that model to a multi-phase formulation. This will enable the further development and validation of multi-phase models with Nek and the subsequent simulation of large, complex problems of interest in both science and industry. This project will use the CFD code Nek5000 and its branch, Nek2P. Written in a combination of Fortran and C, Nek has been demonstrated to be a highly scalable code using the MPI library. Nek5000 has a long history of use and development on the LCRC systems. Proposed simulations will include helical pipe heat exchangers and nuclear reactor subchannel geometries. Meshes for these problems range in size from <100K elements to >1M elements. The project is expected to have between 2-4 members. Industry partnership: Project URL: Requested allocation: 20000 Q1: 0 Q2: 0 Q3: 10000 Q4: 10000 Justification: Storage requirements: The requester has used undetermined amount hours of their initial startup project. In addition to approving an initial amount, please specify a Category and Subcategory for this project. For a list of the current selection of approved categories, please see: https://wiki.lcrc.anl.gov/wiki/Processes/Categories Once the Allocation committee has approved the project, please go to the Project Management page to create it: https://accounts.lcrc.anl.gov/projects.php Thank You, The LCRC Accounts System

1 0

[LCRC Accounts] Project Allocation Request
by accounts＠lcrc.anl.gov 28 Mar '16

28 Mar '16

Hello, A change in allocation has been requested: Requester: zapol (Peter Zapol) Project: glassforms Title: First-principles Calculations of Mineral and Glass Dissolution Description: The first-principles calculations will be carried out using the electronic-structure code VASP. NEB calculations will be performed to calculate the reaction pathways for a variety of composing elements under acidic, neutral and basic conditions. A single optimization calculation takes 48 hours on 32 processors. Last year, we have performed calculations on glass with B, Si and Na. This year, we will add Al and K to these components. We are considering 60 supercell samplings of silicate glasses including multiple components (Si, Al, B, Na, K). The calculation for possible reactants, intermediates, products will take about 200,000 core hours. It takes about 50 hours on 160 processors (5 images) to calculate a reaction barrier along the reaction pathway. The total estimated computer time for barrier calculation is 640,000 core hours. kMC simulation will be performed to simulate the dissolution process of glass for 5 different temperatures and 4 different pH valu es for 72 hours (each simulation should run 50 times in order to get the statistical results). That requires 90,000 core hours. In sum, the total requested allocation for this project is 930,000 core hours. Current: undetermined amount Justification: This project will primarily use the density functional theory (DFT) implemented in electronic structure code VASP. The system sizes of our calculations ranges from 100 to 300 atoms. VASP has been tested to scale very well in parallel processing on Blues. The optimum number of processors used for these systems is 16-64 processors per calculation. The kMC simulation code is written in Fortran. Requested: 250000 A specific reason has been given: We have received considerably smaller allocation than our request and as a result run out of resources. If possible, please consider granting us additional allocation. This needs to be approved and the final allocation amount decided upon. Thank You, The LCRC Accounts System

1 0

[LCRC Accounts] Project Allocation Request
by accounts＠lcrc.anl.gov 28 Mar '16

28 Mar '16

Hello, A change in allocation has been requested: Requester: zapol (Peter Zapol) Project: AlN Title: A DFT study of reaction pathways of NH3 decomposition on AlN (0001) surface Description: Previously, we have performed structure optimizations of the adsorbate-free (0001) surface of AlN and determined reaction pathways for NH3 decomposition using (2 × 2) surface unit cells. Next, we will perform density functional theory (DFT) calculations to determine structure, bonding, and reaction energies of the Al adsorbed species. We will also extend our calculations to other surface orientations, including stepped surfaces. Periodic boundary condition (PBC) will be imposed in order to maintain periodicity of the solid support. Gradient corrected PBE density functional will be used for these calculations. The Climbing Nudged Elastic Band (cNEB) method of Henkelman and coworkers will be used to calculate transition states and activation barriers for formation of different surface species. Each structure in the supported (0001) surface of AlN typically involves ~100 atoms. To obtain accurate reaction energies and barriers, rigorous density functional theory (DFT) calculations are needed. We intend to carry out the DFT calculations using the VASP package. VASP is currently available on Blues cluster and scaling with number of processors is well established. For the MPI version of VASP with the system size of ~100 atoms, a good performance was observed up to 32 processors. Energy optimization in VASP takes approximately 1000 CPU hours and we plan approximately 200 of such calculations for different adsorbate geometries for each of 3 surface orientations. The Climbing Nudged Elastic Band (cNEB) calculation (with 5 to 7 images) takes approximately 3000 CPU hours and we plan to run 60 of these calculations to find reaction pathways. So our total request is 1000*200*3 + 60*3000 = 850,000 core hours Current: undetermined amount Justification: VASP performance on Blues is well established, we use the latest version maintained on Blues. Requested: 100000 A specific reason has been given: We have been given considerably smaller allocation than our request and run out of resources. Please consider our request, if possible. This needs to be approved and the final allocation amount decided upon. Thank You, The LCRC Accounts System

1 0

Re: [allocations-admins] Project Allocation Request - lipidmovement
by Bair, Raymond A. 28 Mar '16

28 Mar '16

Dear Ursula Perez Salas, New allocations start on April 1. You will have more time then. In the meantime you could run in backfill mode. If this is a major problem, please let me know. Regards, Ray ----------------------------------------- Ray Bair Argonne National Laboratory On 3/25/16, 11:48 AM, "allocations-admins-bounces(a)lcrc.anl.gov on behalf of accounts(a)lcrc.anl.gov" <allocations-admins-bounces(a)lcrc.anl.gov on behalf of accounts(a)lcrc.anl.gov> wrote: >Hello, > >A change in allocation has been requested: > > Requester: perezsalas (Ursula Perez Salas) > Project: lipidmovement > Title: Curvature Effect on the Movement of Lipids Within and Between Membranes > Description: Earlier, we have used a unilamellar lipid (DPPC) vesicle, 30 nm in diameter, solvated in water consisting of 10 6 coarse-grained particles, to investigate what effect the curvature has on the free energy of lipid flip-flop across the membrane. This size of vesicle is similar to those used in neutron scattering exper- >iments to which we compare the results of the simulations. We found that there is indeed a >small but noticeable curvature effect on the free energy of DPPC flip-flop. > >Using vesicles similar to the DPPC model vesicle, we will build several new vesicle systems >consisting either of POPS and POPC at two different cholesterol concentrations and use these >systems to calculate the free energy for cholesterol flip-flop and desorption to investigate the >effect of curvature on charged and uncharged membranes. Specifically, we will investigate each >vesicle system at at three different temperatures for cholesterol concentrations, leading to a >total of six systems. > >In order to estimate the rate of direct absorption of cholesterol, we will run a series of small, flat >membrane simulations where we place a cholesterol molecule just outside the membrane and let >it get absorbed without any external influence. Here we will run a set of 25 small simulations >measuring the absorption time per system, with two rate estimations per simulation. > >To investigate the temperature dependence of the absorption rates we will simulate one >POPS-cholesterol system and one POPC-cholesterol system at four different temperatures, >yielding a total of 8 simulation sets. > >To calculate the needed free energy profiles we will use the umbrella sampling technique which >is particularly useful when proper sampling can not be reached in a simple long equilibrium >simulation. To do this, the system is constrained to sample a small region of phase space with a >known external potential added to the system potential. The set of constraints used to sample >the systems are independent of each other, which means that Umbrella Sampling simulations >can be run in parallel. The output sampling data is combined using the Weighted Histogram >Analysis method, post simulation. > > >**Estimate of number of core-hours: > > >Direct absorption simulations in the flat membrane systems: >25 × 8 = 200 simulations are needed. With each simulation running on one node of 8 cores at 24 h/simulation (estimated from a benchmark of 0.0064 days/ns) to get enough data, will yield a need for 200 × 24 × 8 = 38,400 ≈ 40,000 core-hours. > > >Vesicle system equilibration simulations: We need to equilibrate the vesicle systems >before using them in the free energy calculations. A typical equilibration time is 300 - 400 ns, >which with a benchmark time of 0.007 ns/day yields a need for 48 h per simulation. Run on >512 cores, we would need 48 h/simulation × 6 simulation × 512 cores = 147456 ≈ 150,000 core-hours. > > >Cholesterol flip-flop and desorption free energy sampling in the vesicles: 6 systems × >5 simulations/system = 30 sampling simulations are needed. With 30,000 data points per sam- >pling region, collected at every 1 ps as a minimum data requirement and with a benchmark time of 0.06 >days/ns, we need 32 h per simulation, but 35 - 40 h is desirable. A vesicle simulation typically >600 cores to run at this pace, which yields 30 simulations × 40 hours/simulation × 600 cores ≈ >720,000 core-hours. > >In total, we need 40, 000 + 150, 000 + 720, 000 ≈ 910,000 core-hours were the 40,000 >core hours for the absorption estimate simulations are used on independently run small simulations. > >We will use the molecular dynamics program NAMD, which is developed at University of Illinois >at Urbana-Champaign and is freely available for download (http://www.ks.uiuc.edu/Research/namd/) >NAMD is highly parallel and scales efficiently with the number of particles in the system. >With the release of version 2.10 of NAMD we have seen an increase in efficiency of about >15 % on other machines for the proposed vesicle systems. Thus we believe that the >obtained core hours could be used more efficiently with the new version of NAMD also on the >Blues and Fusion clusters. > > >Benchmarks obtained at the UIC Extreme cluster on a 1M coarse-grained particle system: > >Cores | 80 | 160 | 240 | > >NAMD 2.9 | 76.9 ns/day | 108 ns/day | 120.7 ns/day | >NAMD 2.10 | 83.3 ns/day | 129.8 ns/day | 138.8 ns/day | > Current: undetermined amount >Justification: The vesicle systems have an obvious advantage of using the clusters at Argonne, since they need hundreds of cores to reach peak efficiency which is something that is far from feasible even on a high end private desktop computer. > >Current benchmarks used to estimate the core-hours needed for the vesicle system equilibration simulations are from old scaling tests. We have not yet been able to try the new version of NAMD on the blues cluster, but benchmarks obtained at the UIC Extreme cluster on a 1M coarse-grained particle system shows a good scaling up to at least 240 cores and a speedup of 10-20 % compared to NAMD 2.9: > >Cores | 80 | 160 | 240 | > >NAMD 2.9 | 76.9 ns/day | 108 ns/day | 120.7 ns/day | >NAMD 2.10 | 83.3 ns/day | 129.8 ns/day | 138.8 ns/day | > > >The obvious bottleneck in our project is the umbrella sampling of the vesicle systems. The one order of magnitude slowdown that can be seen when comparing the benchmark numbers between the equilibrium simulations with the umbrella sampling simulations is due to the fact that the force calculations needed for the external constraints applied to the systems during sampling cannot be run in parallel. However, the code used to apply the constraints is user made and we plan on revising e.g. the update frequency of the lists needed to perform the force calculations and the output frequency of our sampled variables. > > Requested: 200000 > >A specific reason has been given: >We have about 72k core-hours left to run a few simulations, however this is not enough to get the data needed to draw concrete conclusions on the behavior of cholesterol in our lipid bilayers. We estimate that another 200k core-hours are needed. > >We are very close to finalizing this study and submitting a manuscript. Is it possible to get these additional core-hours? > >Thank you in advance > >This needs to be approved and the final allocation amount decided upon. > >Thank You, >The LCRC Accounts System

1 0

[LCRC Accounts] Project Allocation Request
by accounts＠lcrc.anl.gov 28 Mar '16

28 Mar '16

Hello, A change in allocation has been requested: Requester: svegendla (Prasad Vegendla) Project: CFD-TPM Title: Computational Fluid Dynamics Modeling of Two-phase Boiling Flow and Critical Heat Flux Description: In boiling water reactors, dryout is an important limiting phenomenon in the design of heat exchangers and is associated with the disappearance of the liquid film on the wall in heat exchanger tube. A consequence of this non-wetting heated surface deteriorates the convective heat transfer mechanism resulting in a large rise in the wall temperature of heat flux imposed system such as nuclear reactor fuel rod assemblies. Since dryout is a hydrodynamic phenomenon involving entrainment and redeposition of droplets as well as boiling/evaporation of the liquid, the location of dryout is dynamic and its fluctuation may also give rise to thermal fatigue and corrosion. A large number of experimental and theoretical studies have been conducted over the past several decades to understand and quantify the processes leading to dryout in a heat exchanger tube. * Nek5000 is a highly-scalable open-source transient CFD code developed by MCS-Argonne National Laboratory. This code is based on spectral element method and it is written in basic FORTRAN and C languages. It has more than 200 users world-wide and a long history of development, verification, and validation. * In this project, Nek5000 is used as a platform to develop two-fluid model. * In order to validate the Nek5000 development for two-phase boiling capability, several simulation runs are planned for wide range of pressure, mass flux and wall heat flux. * In FY2015, blues allocated hours are efficiently used for projects of CFD-TPM, NE_RANS and SHARP-IF. * A total of 4 members can use the allocated hours Current: undetermined amount Justification: Nek5000 is a highly scalable tool on fusion. NE users demonstrated good scalability up to several hundred cores. Recent simulations of homogeneous two-fluid boiling model on Fusion cores reported 10% and 58% overhead associated with MPI communication for 128 and 256 cores, respectively. Requested: 20000 A specific reason has been given: Negative balance. Planning to run couple of simulation in next three days. This needs to be approved and the final allocation amount decided upon. Thank You, The LCRC Accounts System

1 0

[LCRC Accounts] Project Allocation Request
by accounts＠lcrc.anl.gov 28 Mar '16

28 Mar '16

Hello, A change in allocation has been requested: Requester: svegendla (Prasad Vegendla) Project: NE_RANS Title: RANS-based CFD for Nuclear Reactors Description: Anticipated CFD support of nuclear reactor analysis in FY16 includes: • Supporting NNSA’s efforts under the GTRI-Convert program to convert research reactors fueled with highly-enriched uranium to instead employ low-enriched uranium, and in doing so reduce the associated proliferation risk. Such reactor conversions require significant analysis to ensure that their cool ability under a range of conditions. Given the range of geometry and flow conditions presented by the various research reactors under consideration, a general-purpose thermofluid analysis capability—like CFD—is required. • Supporting advanced reactor systems modeling by coupling STAR-CCM+ to system codes for transient analysis of postulated adverse events, to ensure that the reactor remains cool even under highly unlikely multi-component failures. • Supporting advanced modeling and simulation efforts under the DOE-NE NEAMS program by integrating STAR-CCM+ into the SHARP reactor analysis package. By including STAR-CCM+ as an optional component in SHARP will provided needed flexibility and enhance the access for new users to the advanced modeling and simulation toolkit. • Support of design and safety analysis of sodium-cooled fast reactors for the DOE-NE Advanced Reactor Concepts program. • Verification and validation of STAR-CCM+ for nuclear rector analysis. • In work for others program, several STAR-CCM+ simulation runs are planned to optimize the aerodynamic and underhood thermal analysis of the heavy-duty vehicles. STAR-CCM+ has been employed by analysts in the Nuclear Engineering Division on LCRC clusters since its inception (both Jazz and Fusion), and has demonstrated good scalability up to several hundred cores. Recent simulations on 720 Fusion cores reported only 15% overhead associated with MPI communication. This project will be used by 5-10 STAR-CCM+ analysts in the NE Division. Current: undetermined amount Justification: STARCCM+ is a highly scalable tool on fusion. NE users demonstrated good scalability up to several hundred cores. Recent simulations on 720 Fusion cores reported only 15% overhead associated with MPI communication. Requested: 20000 A specific reason has been given: Close to zero balance. Planning to run couple of simulation in next three days. This needs to be approved and the final allocation amount decided upon. Thank You, The LCRC Accounts System

1 0

[LCRC Accounts] Project Allocation Request
by accounts＠lcrc.anl.gov 25 Mar '16

25 Mar '16

Hello, A change in allocation has been requested: Requester: perezsalas (Ursula Perez Salas) Project: lipidmovement Title: Curvature Effect on the Movement of Lipids Within and Between Membranes Description: Earlier, we have used a unilamellar lipid (DPPC) vesicle, 30 nm in diameter, solvated in water consisting of 10 6 coarse-grained particles, to investigate what effect the curvature has on the free energy of lipid flip-flop across the membrane. This size of vesicle is similar to those used in neutron scattering exper- iments to which we compare the results of the simulations. We found that there is indeed a small but noticeable curvature effect on the free energy of DPPC flip-flop. Using vesicles similar to the DPPC model vesicle, we will build several new vesicle systems consisting either of POPS and POPC at two different cholesterol concentrations and use these systems to calculate the free energy for cholesterol flip-flop and desorption to investigate the effect of curvature on charged and uncharged membranes. Specifically, we will investigate each vesicle system at at three different temperatures for cholesterol concentrations, leading to a total of six systems. In order to estimate the rate of direct absorption of cholesterol, we will run a series of small, flat membrane simulations where we place a cholesterol molecule just outside the membrane and let it get absorbed without any external influence. Here we will run a set of 25 small simulations measuring the absorption time per system, with two rate estimations per simulation. To investigate the temperature dependence of the absorption rates we will simulate one POPS-cholesterol system and one POPC-cholesterol system at four different temperatures, yielding a total of 8 simulation sets. To calculate the needed free energy profiles we will use the umbrella sampling technique which is particularly useful when proper sampling can not be reached in a simple long equilibrium simulation. To do this, the system is constrained to sample a small region of phase space with a known external potential added to the system potential. The set of constraints used to sample the systems are independent of each other, which means that Umbrella Sampling simulations can be run in parallel. The output sampling data is combined using the Weighted Histogram Analysis method, post simulation. **Estimate of number of core-hours: Direct absorption simulations in the flat membrane systems: 25 × 8 = 200 simulations are needed. With each simulation running on one node of 8 cores at 24 h/simulation (estimated from a benchmark of 0.0064 days/ns) to get enough data, will yield a need for 200 × 24 × 8 = 38,400 ≈ 40,000 core-hours. Vesicle system equilibration simulations: We need to equilibrate the vesicle systems before using them in the free energy calculations. A typical equilibration time is 300 - 400 ns, which with a benchmark time of 0.007 ns/day yields a need for 48 h per simulation. Run on 512 cores, we would need 48 h/simulation × 6 simulation × 512 cores = 147456 ≈ 150,000 core-hours. Cholesterol flip-flop and desorption free energy sampling in the vesicles: 6 systems × 5 simulations/system = 30 sampling simulations are needed. With 30,000 data points per sam- pling region, collected at every 1 ps as a minimum data requirement and with a benchmark time of 0.06 days/ns, we need 32 h per simulation, but 35 - 40 h is desirable. A vesicle simulation typically 600 cores to run at this pace, which yields 30 simulations × 40 hours/simulation × 600 cores ≈ 720,000 core-hours. In total, we need 40, 000 + 150, 000 + 720, 000 ≈ 910,000 core-hours were the 40,000 core hours for the absorption estimate simulations are used on independently run small simulations. We will use the molecular dynamics program NAMD, which is developed at University of Illinois at Urbana-Champaign and is freely available for download (http://www.ks.uiuc.edu/Research/namd/) NAMD is highly parallel and scales efficiently with the number of particles in the system. With the release of version 2.10 of NAMD we have seen an increase in efficiency of about 15 % on other machines for the proposed vesicle systems. Thus we believe that the obtained core hours could be used more efficiently with the new version of NAMD also on the Blues and Fusion clusters. Benchmarks obtained at the UIC Extreme cluster on a 1M coarse-grained particle system: Cores | 80 | 160 | 240 | NAMD 2.9 | 76.9 ns/day | 108 ns/day | 120.7 ns/day | NAMD 2.10 | 83.3 ns/day | 129.8 ns/day | 138.8 ns/day | Current: undetermined amount Justification: The vesicle systems have an obvious advantage of using the clusters at Argonne, since they need hundreds of cores to reach peak efficiency which is something that is far from feasible even on a high end private desktop computer. Current benchmarks used to estimate the core-hours needed for the vesicle system equilibration simulations are from old scaling tests. We have not yet been able to try the new version of NAMD on the blues cluster, but benchmarks obtained at the UIC Extreme cluster on a 1M coarse-grained particle system shows a good scaling up to at least 240 cores and a speedup of 10-20 % compared to NAMD 2.9: Cores | 80 | 160 | 240 | NAMD 2.9 | 76.9 ns/day | 108 ns/day | 120.7 ns/day | NAMD 2.10 | 83.3 ns/day | 129.8 ns/day | 138.8 ns/day | The obvious bottleneck in our project is the umbrella sampling of the vesicle systems. The one order of magnitude slowdown that can be seen when comparing the benchmark numbers between the equilibrium simulations with the umbrella sampling simulations is due to the fact that the force calculations needed for the external constraints applied to the systems during sampling cannot be run in parallel. However, the code used to apply the constraints is user made and we plan on revising e.g. the update frequency of the lists needed to perform the force calculations and the output frequency of our sampled variables. Requested: 200000 A specific reason has been given: We have about 72k core-hours left to run a few simulations, however this is not enough to get the data needed to draw concrete conclusions on the behavior of cholesterol in our lipid bilayers. We estimate that another 200k core-hours are needed. We are very close to finalizing this study and submitting a manuscript. Is it possible to get these additional core-hours? Thank you in advance This needs to be approved and the final allocation amount decided upon. Thank You, The LCRC Accounts System

1 0

[LCRC Accounts] Project Allocation Request
by accounts＠lcrc.anl.gov 25 Mar '16

25 Mar '16

Hello, A change in allocation has been requested: Requester: perezsalas (Ursula Perez Salas) Project: lipidmovement Title: Curvature Effect on the Movement of Lipids Within and Between Membranes Description: Earlier, we have used a unilamellar lipid (DPPC) vesicle, 30 nm in diameter, solvated in water consisting of 10 6 coarse-grained particles, to investigate what effect the curvature has on the free energy of lipid flip-flop across the membrane. This size of vesicle is similar to those used in neutron scattering exper- iments to which we compare the results of the simulations. We found that there is indeed a small but noticeable curvature effect on the free energy of DPPC flip-flop. Using vesicles similar to the DPPC model vesicle, we will build several new vesicle systems consisting either of POPS and POPC at two different cholesterol concentrations and use these systems to calculate the free energy for cholesterol flip-flop and desorption to investigate the effect of curvature on charged and uncharged membranes. Specifically, we will investigate each vesicle system at at three different temperatures for cholesterol concentrations, leading to a total of six systems. In order to estimate the rate of direct absorption of cholesterol, we will run a series of small, flat membrane simulations where we place a cholesterol molecule just outside the membrane and let it get absorbed without any external influence. Here we will run a set of 25 small simulations measuring the absorption time per system, with two rate estimations per simulation. To investigate the temperature dependence of the absorption rates we will simulate one POPS-cholesterol system and one POPC-cholesterol system at four different temperatures, yielding a total of 8 simulation sets. To calculate the needed free energy profiles we will use the umbrella sampling technique which is particularly useful when proper sampling can not be reached in a simple long equilibrium simulation. To do this, the system is constrained to sample a small region of phase space with a known external potential added to the system potential. The set of constraints used to sample the systems are independent of each other, which means that Umbrella Sampling simulations can be run in parallel. The output sampling data is combined using the Weighted Histogram Analysis method, post simulation. **Estimate of number of core-hours: Direct absorption simulations in the flat membrane systems: 25 × 8 = 200 simulations are needed. With each simulation running on one node of 8 cores at 24 h/simulation (estimated from a benchmark of 0.0064 days/ns) to get enough data, will yield a need for 200 × 24 × 8 = 38,400 ≈ 40,000 core-hours. Vesicle system equilibration simulations: We need to equilibrate the vesicle systems before using them in the free energy calculations. A typical equilibration time is 300 - 400 ns, which with a benchmark time of 0.007 ns/day yields a need for 48 h per simulation. Run on 512 cores, we would need 48 h/simulation × 6 simulation × 512 cores = 147456 ≈ 150,000 core-hours. Cholesterol flip-flop and desorption free energy sampling in the vesicles: 6 systems × 5 simulations/system = 30 sampling simulations are needed. With 30,000 data points per sam- pling region, collected at every 1 ps as a minimum data requirement and with a benchmark time of 0.06 days/ns, we need 32 h per simulation, but 35 - 40 h is desirable. A vesicle simulation typically 600 cores to run at this pace, which yields 30 simulations × 40 hours/simulation × 600 cores ≈ 720,000 core-hours. In total, we need 40, 000 + 150, 000 + 720, 000 ≈ 910,000 core-hours were the 40,000 core hours for the absorption estimate simulations are used on independently run small simulations. We will use the molecular dynamics program NAMD, which is developed at University of Illinois at Urbana-Champaign and is freely available for download (http://www.ks.uiuc.edu/Research/namd/) NAMD is highly parallel and scales efficiently with the number of particles in the system. With the release of version 2.10 of NAMD we have seen an increase in efficiency of about 15 % on other machines for the proposed vesicle systems. Thus we believe that the obtained core hours could be used more efficiently with the new version of NAMD also on the Blues and Fusion clusters. Benchmarks obtained at the UIC Extreme cluster on a 1M coarse-grained particle system: Cores | 80 | 160 | 240 | NAMD 2.9 | 76.9 ns/day | 108 ns/day | 120.7 ns/day | NAMD 2.10 | 83.3 ns/day | 129.8 ns/day | 138.8 ns/day | Current: undetermined amount Justification: The vesicle systems have an obvious advantage of using the clusters at Argonne, since they need hundreds of cores to reach peak efficiency which is something that is far from feasible even on a high end private desktop computer. Current benchmarks used to estimate the core-hours needed for the vesicle system equilibration simulations are from old scaling tests. We have not yet been able to try the new version of NAMD on the blues cluster, but benchmarks obtained at the UIC Extreme cluster on a 1M coarse-grained particle system shows a good scaling up to at least 240 cores and a speedup of 10-20 % compared to NAMD 2.9: Cores | 80 | 160 | 240 | NAMD 2.9 | 76.9 ns/day | 108 ns/day | 120.7 ns/day | NAMD 2.10 | 83.3 ns/day | 129.8 ns/day | 138.8 ns/day | The obvious bottleneck in our project is the umbrella sampling of the vesicle systems. The one order of magnitude slowdown that can be seen when comparing the benchmark numbers between the equilibrium simulations with the umbrella sampling simulations is due to the fact that the force calculations needed for the external constraints applied to the systems during sampling cannot be run in parallel. However, the code used to apply the constraints is user made and we plan on revising e.g. the update frequency of the lists needed to perform the force calculations and the output frequency of our sampled variables. Requested: 200000 A specific reason has been given: We have about 72k core-hours left to run a few simulations, however this is not enough to get the data needed to draw concrete conclusions on the behavior of cholesterol in our lipid bilayers. We estimate that another 200k core-hours are needed. We are very close to finalizing this study and submitting a manuscript. Is it possible to get these additional core-hours? Thank you in advance This needs to be approved and the final allocation amount decided upon. Thank You, The LCRC Accounts System

1 0

[LCRC Accounts] Project Allocation Request
by accounts＠lcrc.anl.gov 23 Mar '16

23 Mar '16

Hello, A change in allocation has been requested: Requester: ango (Anh Ngo) Project: ReNiO3 Title: Structural Evolution of rare earth nickelates under high pressure conditions Description: Structure optimizations of ReNiO3 (~80 atoms) with different magnetic configurations will be carried out using (2 x 1x 2) bulk unit cells. By comparing different magnetic ordering energies we will conclude which configuration corresponding to the ground state energy. Once ground state structure is found we will go on to address the structural dependence of ReNiO3 under high pressure. We intend to carry out the density functional theory (DFT) calculations using the VASP package. Periodic boundary condition (PBC) will be imposed in order to maintain periodicity of the solid support. Gradient corrected PBE density functional will be used for these calculations. Those programs are currently available on Blues cluster and scale reasonably well on parallel processors. Energy optimization of ReNiO3 (~80 atoms contain 16 Ni with different spin configurations) using GGA+U takes approximately 4608 CPU hours (16*6*48) and we plan roughly 60 of such calculations. For pressure calculations, since ReNiO3 is a correlated complex oxide, we will perform density functional theory (DFT) calculations using the Heyd-Scuseria-Ernzerhof (HSE) hybrid functional and performed the atomic relaxations within the unit cells. This takes roughly 4608 CPU hours (16*6*48) and we plan to run 80 of these calculations. To investigate metal-insulator transition we plan to run additional 66 HSE calculations for the density of state (DOS) at the ambient pressures of 0 GPa to 3.4 GPa. So our total request is 60*4608 + (80+66)* 4608 =949248 core hours. Current: undetermined amount Justification: VASP package is widely used on Blues and its scaling is well documented. Requested: 150000 A specific reason has been given: We used up the previous allocation analyzing APS data using HSE functional in VASP and need more hours. This needs to be approved and the final allocation amount decided upon. Thank You, The LCRC Accounts System

1 0

[LCRC Accounts] Project Allocation Request
by accounts＠lcrc.anl.gov 23 Mar '16

23 Mar '16

Hello, A change in allocation has been requested: Requester: ango (Anh Ngo) Project: ReNiO3 Title: Structural Evolution of rare earth nickelates under high pressure conditions Description: Structure optimizations of ReNiO3 (~80 atoms) with different magnetic configurations will be carried out using (2 x 1x 2) bulk unit cells. By comparing different magnetic ordering energies we will conclude which configuration corresponding to the ground state energy. Once ground state structure is found we will go on to address the structural dependence of ReNiO3 under high pressure. We intend to carry out the density functional theory (DFT) calculations using the VASP package. Periodic boundary condition (PBC) will be imposed in order to maintain periodicity of the solid support. Gradient corrected PBE density functional will be used for these calculations. Those programs are currently available on Blues cluster and scale reasonably well on parallel processors. Energy optimization of ReNiO3 (~80 atoms contain 16 Ni with different spin configurations) using GGA+U takes approximately 4608 CPU hours (16*6*48) and we plan roughly 60 of such calculations. For pressure calculations, since ReNiO3 is a correlated complex oxide, we will perform density functional theory (DFT) calculations using the Heyd-Scuseria-Ernzerhof (HSE) hybrid functional and performed the atomic relaxations within the unit cells. This takes roughly 4608 CPU hours (16*6*48) and we plan to run 80 of these calculations. To investigate metal-insulator transition we plan to run additional 66 HSE calculations for the density of state (DOS) at the ambient pressures of 0 GPa to 3.4 GPa. So our total request is 60*4608 + (80+66)* 4608 =949248 core hours. Current: undetermined amount Justification: VASP package is widely used on Blues and its scaling is well documented. Requested: 150000 A specific reason has been given: We used up the previous allocation analyzing APS data using HSE functional in VASP and need more hours. This needs to be approved and the final allocation amount decided upon. Thank You, The LCRC Accounts System

1 0