January 2016 - Allocations-admins - mailman3.cels.anl.gov

[LCRC Accounts] Project Allocation Request
by accounts＠lcrc.anl.gov 14 Jan '16

14 Jan '16

Hello, A change in allocation has been requested: Requester: kibaekkim (Kibaek Kim) Project: NEXTGENOPT Title: Next generation optimization Description: Sponsored by a Department of Energy (DOE) Early Career Award, we are currently developing new algorithms for UQ/SO that have a holistic view of the entire data-modeling-optimization process. A particular example of such algorithms is a new clustering-based interior-point strategy for stochastic programs in which scenarios are compressed based on their influence on the first-stage decision, not on data. This has shown to lead to drastic compression rates (and solution times) of above 80%, which cannot be achieved with data compression alone (see Figure 1). The compression is done adaptively, at the linear algebra level, by looking at the different contributions of the scenarios on the Schur complement. We are also currently developing new interior-point algorithms for nonconvex structured optimization capable of dealing with complex multi-scale physics (typically involving large networks of partial differential equations) and stochastic components. These problems arise, for instance, from natural gas inventory (line-pack) management (see Figure 2). This requires of a redesign of existing algorithms and modeling environments capable of conveying problem structure down to the linear algebra kernels. In addition, hybrid linear algebra kernels need to be designed to exploit different structural facets. Problems with billions of variables are being targeted. Current: undetermined amount Justification: Requested: 400000 A specific reason has been given: We are running a large number of optimization instances for a power grid application. The application analyzes the cost efficiency of the large power grid system in California with uncertain wind power production. As a result, a large scale stochastic optimization problem needs to be solved with different seasons and different parameter settings, which requires more core-hours to run such optimization problems. This needs to be approved and the final allocation amount decided upon. Thank You, The LCRC Accounts System

1 0

Re: [allocations-admins] Project Allocation Request for Comp_Soft_Catalysts
by Bair, Raymond A. 11 Jan '16

11 Jan '16

For Comp_Soft_Catalysts Juan needs 200K now and the rest at the next allocation committee meeting. Ray ----------------------------------------- Ray Bair Argonne National Laboratory and the University of Chicago From: Juan M Lopez Encarnacion <juan.lopez15(a)upr.edu<mailto:[email protected]>> Date: Monday, January 11, 2016 at 2:14 PM To: Ray Bair <rbair(a)anl.gov<mailto:[email protected]>> Cc: "Lopez-Encarnacion, Juan Manuel" <JMLopez(a)anl.gov<mailto:[email protected]>>, "jlopez(a)lcrc.anl.gov<mailto:[email protected]>" <jlopez(a)lcrc.anl.gov<mailto:[email protected]>> Subject: Re: Project Allocation Request for Comp_Soft_Catalysts Dear Raymond, OK!, will be good... Thank you in advance, Juan. 2016-01-11 14:08 GMT-06:00 Bair, Raymond A. <rbair(a)anl.gov<mailto:[email protected]>>: Would 200K suffice through March 31? The LCRC staff can approve that amount. Larger amounts require polling the committee. The balance can be part of your Q3 and Q4 allocation. The Allocations Committee meets in mid-March. Regards, Ray ----------------------------------------- Ray Bair Argonne National Laboratory and the University of Chicago From: Juan M Lopez Encarnacion <juan.lopez15(a)upr.edu<mailto:[email protected]>> Date: Monday, January 11, 2016 at 2:04 PM To: Ray Bair <rbair(a)anl.gov<mailto:[email protected]>> Cc: "Lopez-Encarnacion, Juan Manuel" <JMLopez(a)anl.gov<mailto:[email protected]>>, "jlopez(a)lcrc.anl.gov<mailto:[email protected]>" <jlopez(a)lcrc.anl.gov<mailto:[email protected]>> Subject: Re: Project Allocation Request for Comp_Soft_Catalysts Dear Raymond, I may need around 250K. Juan. 2016-01-11 13:39 GMT-06:00 Bair, Raymond A. <rbair(a)anl.gov<mailto:[email protected]>>: Dear Juan, I see that your request is still outstanding. Could you let us know how much of the 400,000 additional core-hours requested will be needed between now and March 31st? Regards, Ray ----------------------------------------- Ray Bair Argonne National Laboratory and the University of Chicago On 1/4/16, 12:44 PM, "allocations-admins-bounces(a)lcrc.anl.gov<mailto:[email protected]> on behalf of Bair, Raymond A." <allocations-admins-bounces(a)lcrc.anl.gov<mailto:[email protected]> on behalf of rbair(a)anl.gov<mailto:[email protected]>> wrote: >Dear Juan, > >The LCRC Allocations Committee makes most decisions about Q1 and Q2 time >in September and Q3 and Q4 time in March. One can request additional time >any time, but we would like to sync with the next meeting. Of the 400K >additional core-hours requested below, how much of that time is needed >before March 31st? > >Regards, > >Ray > >----------------------------------------- >Ray Bair >Computing, Environment, and Life Sciences >Argonne National Laboratory > and the University of Chicago >TCS Building 240, Room 4122 >9700 South Cass Avenue >Argonne, IL 60439 >email: rbair(at)anl.gov<http://anl.gov> >Phone: (630)252-5751<tel:%28630%29252-5751> > > > > > >On 1/4/16, 7:03 AM, "allocations-admins-bounces(a)lcrc.anl.gov<mailto:[email protected]> on behalf of >accounts(a)lcrc.anl.gov<mailto:[email protected]>" <allocations-admins-bounces(a)lcrc.anl.gov<mailto:[email protected]> on behalf >of accounts(a)lcrc.anl.gov<mailto:[email protected]>> wrote: > >>Hello, >> >>A change in allocation has been requested: >> >> Requester: jlopez (Juan Lopez-Encarnacion) >> Project: Comp_Soft_Catalysts >> Title: Soft Catalysts for Green Chemistry and Energy applications >> Description: In the FY2016, the computational plan will continue focus >>on exploration the catalytic performance of different type of >>Ta-Hydride/POP, and early metals-based/POP catalysts in term of reducing >>the energy barriers on the reaction mechanism of two specific chemical >>reactions. One of the targeted processes with Ta-Hydride/POP (solid >>catalyst) is the mechanism for dissociation of gas-phase N2 on the >>presence of H2 gas (i.e., one of the challenging problem in catalysis, >>because the high temperature and high pressure required in the current >>industrial process for ammonia synthesis from air). In addition, we are >>planning to explore early metals based POP catalyst to test hydrogenation >>reactions of alkenes to alkanes. Based on our preliminary results, we >>expect the Ta-hydride and early metals-based reaction centers embedded on >>POP supports to reduce the relevant reaction barriers as compared to >>those of the more common supports, e.g., silica or alumina based. The >>struc >> tural and electronic aspects of reactants, intermediates, transition >>states, and products will be explored and characterized using >>state-of-the-art density functional theory. >> >>The number of atoms treated at a time will vary from a few to a few tens >>(e.g., in organometallic parents of the single-metal reaction centers and >>substrates), from tens to about a hundred (e.g., molecular models of the >>POP alone, and intermediates complexes which involve interactions of >>substrates with the catalyst). The mechanistic aspects of the reactions >>will be evaluated and characterized using >>generalized-gradient-approximation and transition state theory as >>implemented on NWChem. The requested time is based on estimates performed >>on Fusion (see below). The methods used to study the kinetics aspects, >>especially; searching of transition state structures (nudge-elastic-band >>theory and/or constraint-optimization technique) and normal mode analysis >>(harmonic-approximation) of our atomic systems require considerably >>computational power/time. The expected number of project members is >>three (3) or more. >> Current: undetermined amount >>Justification: The scaling efficiency on Fusion was tested using the code >>SIESTA in the full optimization of the atomic structure and cell volume >>of a crystalline porous organic polymer, which contain a total of 584 >>atoms: 192 atoms of H (5 numerical basis function per each H), 360 atoms >>of C (13 numerical basis function per each C), and 32 atoms of O (13 >>numerical basis function per each O) in its unit cell; a total of 6,056 >>basis functions. Substantial speed-up is achieved with the increase in >>the number of cores: >> >>Cores Average elapsed time per optimization step (min) >>16 4.38 >>32 2.70 >>64 2.34 >>128 1.68 >> >> Requested: 400000 >> >>A specific reason has been given: >>We need to run many jobs to complete a set of computations for a >>publication that we are planning to submit in this month. >> >>This needs to be approved and the final allocation amount decided upon. >> >>Thank You, >>The LCRC Accounts System >>_______________________________________________ >>allocations-admins mailing list >>allocations-admins(a)lcrc.anl.gov<mailto:[email protected]> >>https://lists.lcrc.anl.gov/mailman/listinfo/allocations-admins > >_______________________________________________ >allocations-admins mailing list >allocations-admins(a)lcrc.anl.gov<mailto:[email protected]> >https://lists.lcrc.anl.gov/mailman/listinfo/allocations-admins -- Best regards, ------- Juan M. López-Encarnación, PhD Assistant Professor Department of Mathematics-Physics University of Puerto Rico at Cayey E-mail: juan.lopez15(a)upr.edu<mailto:[email protected]> Office: (787) 738-2161, x 3128<tel:%28787%29%20738-2161%2C%20x%203128> -- Best regards, ------- Juan M. López-Encarnación, PhD Assistant Professor Department of Mathematics-Physics University of Puerto Rico at Cayey E-mail: juan.lopez15(a)upr.edu<mailto:[email protected]> Office: (787) 738-2161, x 3128

1 0

Re: [allocations-admins] [LCRC Accounts] Project Allocation Request
by John Blaas 11 Jan '16

11 Jan '16

20k added for Q2. - JB On Mon, Jan 11, 2016 at 1:35 PM, Bair, Raymond A. <rbair(a)anl.gov> wrote: > Lets add the 20K Don requested. > > Ray > > ----------------------------------------- > Ray Bair > Argonne National Laboratory > and the University of Chicago > > > From: <donaldksinclair(a)gmail.com> on behalf of donald sinclair <dks(a)anl.gov> > Date: Monday, January 11, 2016 at 11:28 AM > To: Ray Bair <rbair(a)anl.gov> > Subject: Re: [allocations-admins] [LCRC Accounts] Project Allocation Request > > For the present, yes. It is what I need to finish one particular set of > runs, and I didn't want to wait for the length of time it might take to > consider the much larger amount of time which I would need to finish my > other set of runs. > thanks, Don > > On Mon, Jan 11, 2016 at 8:31 AM, Bair, Raymond A. <rbair(a)anl.gov> wrote: >> >> Don, >> >> Did you intend to request only 20,000 core-hours more? >> >> Ray >> >> >> ----------------------------------------- >> Ray Bair >> Argonne National Laboratory >> and the University of Chicago >> >> >> >> >> >> >> >> >> >> On 1/9/16, 4:23 PM, "allocations-admins-bounces(a)lcrc.anl.gov on behalf of >> accounts(a)lcrc.anl.gov" <allocations-admins-bounces(a)lcrc.anl.gov on behalf of >> accounts(a)lcrc.anl.gov> wrote: >> >> >Hello, >> > >> >A change in allocation has been requested: >> > >> > Requester: sinclair (Donald Sinclair) >> > Project: Lattice-QCD >> > Title: Lattice simulations of Conformal and Walking >> >Technicolor. >> > Description: We perform simulations to evaluate the functional >> > integrals of QCD-like >> >theories formulated on a discrete space-time lattice, to enable >> > determination >> >of the non-perturbative aspects these theories. These include the >> > properties >> >of these theories at non-zero temperature, including the scales of >> > confinement >> >and chiral symmetry breaking, and such zero temperature properties as >> > spectra >> >(including the Higgs mass), decay constants and the running of the gauge >> >coupling constant. We are particularly interested in those theories where >> > the >> >coupling constant evolves very slowly, since these are candidate >> > 'Walking- >> >Technicolor' theories. Related to these are theories with an infrared >> > fixed >> >point (conformal field theories). Our first goal is to differentiate >> > between >> >these two different types of behaviour for candidate theories. >> > >> >We are performing simulations of QCD-like theories which are models for >> >Walking or Conformal Technicolor. We have been studying theories which >> > are >> >essentially QCD but with colour-sextet rather than colour-triplet quarks. >> > We >> >hope to measure the running of the QCD coupling constant. For 2 or 3 >> > flavours, >> >2-loop perturbation theory predicts an infrared fixed point. >> > >> >For 2 flavours, it is possible that a chiral condensate forms before this >> >fixed point is reached. If so, the fixed point is avoided, the theory is >> >confining, and chiral symmetry breaks spontaneously. However, there is a >> >region where the coupling constant evolves very slowly. These are the >> >properties required for a walking technicolor theory. Simulations we have >> >performed so far at finite temperature suggest that this theory might be >> >conformal. So far, however, the results are inconclusive. >> > >> >For 3 flavours we know that the theory should be conformal. Simulations >> > at >> >N_t=6 (12^3 X 6 lattice) and N_t=8 (12^3 X 8 lattice), some of which used >> >Fusion, did not yet show evidence of conformality. We are now simulating >> > at >> >N_t=12 (24^3 X 12 lattice). We started these simulations on Blues, and >> > intend >> >to continue them in FY2016 on Blues to enable continuity during the >> > period >> >when Edison at NERSC is unavailable due to a site move. >> > >> >Our sextet quark codes are based on our earlier triplet quark codes and >> > use >> >the RHMC simulation method. The Rational Hybrid Monte Carlo (RHMC) is a >> >stochastic molecular dynamics algorithm. The functional integral of QCD >> > is >> >written as a partition function of a classical field theory evolving in a >> >fictitious time. The determinant of the Dirac operator raised to a >> > fractional >> >power is calculated by introducing bosonic fields (pseudofermions), and >> >sandwiching this Dirac operator raised to minus said fractional power >> > between >> >them. This fractional power of the Dirac operator is approximated to >> > machine >> >accuracy by a rational approximation. After defining this theory on a >> > discrete >> >space-time lattice, the inversions required by the partial-fraction >> > expansion >> >of the rational approximation are performed using Krylov space methods, >> > in >> >particular a multi-shift extension of the conjugate gradient algorithm. A >> >global Metropolis Monte-Carlo accept/reject step applied at the end of >> > each >> >trajectory removes discretization errors introduced by the numerical >> >integration of these stochastic equations of motion. We parallelize the >> > code >> >by assigning a fixed number of adjacent lattice sites to each MPI task. >> >Network bandwidth ultimately limits how small a chunk of the lattice can >> > be >> >assigned to each task. >> > >> >Our 24^3 X 12 runs will be performed on 288 cores (18 nodes) of Blues. >> > >> >A short benchmark run of the 24^3 X 12 codes yielded 667 Gflops or 2.3 >> >Gflops/core. Since we have seen performances of > 3 Gflops/core on >> > machines >> >with similar processors, and > 4 Gflops/node on Edison, we suspect that >> > at >> >least 3 Gflops/core should be achievable on this machine. Since we have >> > nearly >> >exhausted our allocation on Blues, we cannot perform more extensive >> > tests. >> >Scaling tests on Fusion and on Edison at NERSC for numbers of cores from >> > 24 up >> >to 288 cores indicate that the per core performance increases up to 288 >> > cores, >> >which we believe to be related to cache usage. >> > >> >Our new project simulating QCD at finite mu using Complex Langevin with >> > Gauge >> >Cooling is currently running as serial code on several platforms, >> > including >> >Blues, where the little allocation we have remaining does not allow us to >> > run >> >anything else. The Langevin equation method is essentially the limiting >> > case of >> >the above algorithm in its older form (without the rational >> > approximation), >> >when each trajectory involves a single update of the fields. Its >> > extension to >> >the complex manifold merely involves extending the gauge-field manifold >> > from >> >SU(3) to SL(3,C). The main reason that it is serial code is that this >> > project >> >is still in its exploratory phase, meaning that modifications of the >> > algorithm >> >are still possible. Converting to parallel MPI code will not be >> > difficult, >> >since the compute-intensive kernel is almost identical to that of our >> > older >> >QCD codes, as are many other parts of the code. We do, however, expect >> > some >> >performance degradation, since we have been running this code at 64-bit >> >precision, rather than the 32-bit precision of our QCD codes. When we >> > have >> >converted to parallel code, we will cease our serial runs on Blues. While >> > we >> >perform zero-temperature simulations at NERSC, TACC and SDSC, we plan to >> > start >> >finite temperature simulations on Blues, probably starting on 8^3*4 >> > lattices >> >using 16 or 32 cores, and then moving to 12^3*4 and 12^3*6 lattices on 48 >> > and >> >72 cores respectively. Note that these runs with low-level parallelism >> > are >> >discouraged on the flagship machines at NERSC. >> > >> >Our requested allocation is based on running 2 288-core jobs half of the >> > time >> >for the first 4-months and jobs using a total of 144 cores half the time >> > for >> >the last 8 months. Expected percentage of allocation for serial jobs -- >> > 5%. >> > Current: undetermined amount >> >Justification: We have not yet performed a detailed scaling analysis on >> > Blues, however, we >> >have such analyses performed on Fusion as well as Edison at NERSC. For >> > Fusion, >> >for our QCD with sextet quarks code on a 24^3 X 12 lattice running on >> > Fusion >> >we observed the following performances: >> > 24 cores = 45 Gflops = 1.9 Gflops/core >> > 48 cores = 95 Gflops = 2.0 Gflops/core >> > 72 cores = 148 Gflops = 2.1 Gflops/core >> > 96 cores = 220 Gflops = 2.3 Gflops/core >> >144 cores = 369 Gflops = 2.6 Gflops/core >> >288 cores = 784 Gflops = 2.7 Gflops/core >> >For the same code and lattice size running on Edison at NERSC we observed >> > the >> >following performances. >> > 24 cores = 62 Gflops = 2.6 Gflops/core >> > 48 cores = 129 Gflops = 2.7 Gflops/core >> > 72 cores = 214 Gflops = 3.0 Gflops/core >> > 96 cores = 330 Gflops = 3.4 Gflops/core >> >144 cores = 561 Gflops = 3.9 Gflops/core >> >288 cores = 1219 Gflops = 4.2 Gflops/core >> >We use a custom assignment of tasks to nodes in order to minimize >> >communications. Earlier recoding reduced the number of global reductions >> > in >> >the routines, which use most of the CPU time, by a factor of 2. >> > >> >The new Complex Langevin codes are still serial, but they will be >> > parallelized >> >in the same manner as the RHMC codes. Algorithm changes are under >> > consideration. >> >There is also the possibility that we will prepare an OpenMP version in >> >preparation for running on Cori II at NERSC. >> > >> > Requested: 20000 >> > >> >A specific reason has been given: >> >Needed to finish current set of runs. >> > >> >This needs to be approved and the final allocation amount decided upon. >> > >> >Thank You, >> >The LCRC Accounts System >> >_______________________________________________ >> >allocations-admins mailing list >> >allocations-admins(a)lcrc.anl.gov >> >https://lists.lcrc.anl.gov/mailman/listinfo/allocations-admins > >

1 0

Re: [allocations-admins] [LCRC Accounts] Project Allocation Request
by Bair, Raymond A. 11 Jan '16

11 Jan '16

Lets add the 20K Don requested. Ray ----------------------------------------- Ray Bair Argonne National Laboratory and the University of Chicago From: <donaldksinclair(a)gmail.com<mailto:[email protected]>> on behalf of donald sinclair <dks(a)anl.gov<mailto:[email protected]>> Date: Monday, January 11, 2016 at 11:28 AM To: Ray Bair <rbair(a)anl.gov<mailto:[email protected]>> Subject: Re: [allocations-admins] [LCRC Accounts] Project Allocation Request For the present, yes. It is what I need to finish one particular set of runs, and I didn't want to wait for the length of time it might take to consider the much larger amount of time which I would need to finish my other set of runs. thanks, Don On Mon, Jan 11, 2016 at 8:31 AM, Bair, Raymond A. <rbair(a)anl.gov<mailto:[email protected]>> wrote: Don, Did you intend to request only 20,000 core-hours more? Ray ----------------------------------------- Ray Bair Argonne National Laboratory and the University of Chicago On 1/9/16, 4:23 PM, "allocations-admins-bounces(a)lcrc.anl.gov<mailto:[email protected]> on behalf of accounts(a)lcrc.anl.gov<mailto:[email protected]>" <allocations-admins-bounces(a)lcrc.anl.gov<mailto:[email protected]> on behalf of accounts(a)lcrc.anl.gov<mailto:[email protected]>> wrote: >Hello, > >A change in allocation has been requested: > > Requester: sinclair (Donald Sinclair) > Project: Lattice-QCD > Title: Lattice simulations of Conformal and Walking >Technicolor. > Description: We perform simulations to evaluate the functional integrals of QCD-like >theories formulated on a discrete space-time lattice, to enable determination >of the non-perturbative aspects these theories. These include the properties >of these theories at non-zero temperature, including the scales of confinement >and chiral symmetry breaking, and such zero temperature properties as spectra >(including the Higgs mass), decay constants and the running of the gauge >coupling constant. We are particularly interested in those theories where the >coupling constant evolves very slowly, since these are candidate 'Walking- >Technicolor' theories. Related to these are theories with an infrared fixed >point (conformal field theories). Our first goal is to differentiate between >these two different types of behaviour for candidate theories. > >We are performing simulations of QCD-like theories which are models for >Walking or Conformal Technicolor. We have been studying theories which are >essentially QCD but with colour-sextet rather than colour-triplet quarks. We >hope to measure the running of the QCD coupling constant. For 2 or 3 flavours, >2-loop perturbation theory predicts an infrared fixed point. > >For 2 flavours, it is possible that a chiral condensate forms before this >fixed point is reached. If so, the fixed point is avoided, the theory is >confining, and chiral symmetry breaks spontaneously. However, there is a >region where the coupling constant evolves very slowly. These are the >properties required for a walking technicolor theory. Simulations we have >performed so far at finite temperature suggest that this theory might be >conformal. So far, however, the results are inconclusive. > >For 3 flavours we know that the theory should be conformal. Simulations at >N_t=6 (12^3 X 6 lattice) and N_t=8 (12^3 X 8 lattice), some of which used >Fusion, did not yet show evidence of conformality. We are now simulating at >N_t=12 (24^3 X 12 lattice). We started these simulations on Blues, and intend >to continue them in FY2016 on Blues to enable continuity during the period >when Edison at NERSC is unavailable due to a site move. > >Our sextet quark codes are based on our earlier triplet quark codes and use >the RHMC simulation method. The Rational Hybrid Monte Carlo (RHMC) is a >stochastic molecular dynamics algorithm. The functional integral of QCD is >written as a partition function of a classical field theory evolving in a >fictitious time. The determinant of the Dirac operator raised to a fractional >power is calculated by introducing bosonic fields (pseudofermions), and >sandwiching this Dirac operator raised to minus said fractional power between >them. This fractional power of the Dirac operator is approximated to machine >accuracy by a rational approximation. After defining this theory on a discrete >space-time lattice, the inversions required by the partial-fraction expansion >of the rational approximation are performed using Krylov space methods, in >particular a multi-shift extension of the conjugate gradient algorithm. A >global Metropolis Monte-Carlo accept/reject step applied at the end of each >trajectory removes discretization errors introduced by the numerical >integration of these stochastic equations of motion. We parallelize the code >by assigning a fixed number of adjacent lattice sites to each MPI task. >Network bandwidth ultimately limits how small a chunk of the lattice can be >assigned to each task. > >Our 24^3 X 12 runs will be performed on 288 cores (18 nodes) of Blues. > >A short benchmark run of the 24^3 X 12 codes yielded 667 Gflops or 2.3 >Gflops/core. Since we have seen performances of > 3 Gflops/core on machines >with similar processors, and > 4 Gflops/node on Edison, we suspect that at >least 3 Gflops/core should be achievable on this machine. Since we have nearly >exhausted our allocation on Blues, we cannot perform more extensive tests. >Scaling tests on Fusion and on Edison at NERSC for numbers of cores from 24 up >to 288 cores indicate that the per core performance increases up to 288 cores, >which we believe to be related to cache usage. > >Our new project simulating QCD at finite mu using Complex Langevin with Gauge >Cooling is currently running as serial code on several platforms, including >Blues, where the little allocation we have remaining does not allow us to run >anything else. The Langevin equation method is essentially the limiting case of >the above algorithm in its older form (without the rational approximation), >when each trajectory involves a single update of the fields. Its extension to >the complex manifold merely involves extending the gauge-field manifold from >SU(3) to SL(3,C). The main reason that it is serial code is that this project >is still in its exploratory phase, meaning that modifications of the algorithm >are still possible. Converting to parallel MPI code will not be difficult, >since the compute-intensive kernel is almost identical to that of our older >QCD codes, as are many other parts of the code. We do, however, expect some >performance degradation, since we have been running this code at 64-bit >precision, rather than the 32-bit precision of our QCD codes. When we have >converted to parallel code, we will cease our serial runs on Blues. While we >perform zero-temperature simulations at NERSC, TACC and SDSC, we plan to start >finite temperature simulations on Blues, probably starting on 8^3*4 lattices >using 16 or 32 cores, and then moving to 12^3*4 and 12^3*6 lattices on 48 and >72 cores respectively. Note that these runs with low-level parallelism are >discouraged on the flagship machines at NERSC. > >Our requested allocation is based on running 2 288-core jobs half of the time >for the first 4-months and jobs using a total of 144 cores half the time for >the last 8 months. Expected percentage of allocation for serial jobs -- 5%. > Current: undetermined amount >Justification: We have not yet performed a detailed scaling analysis on Blues, however, we >have such analyses performed on Fusion as well as Edison at NERSC. For Fusion, >for our QCD with sextet quarks code on a 24^3 X 12 lattice running on Fusion >we observed the following performances: > 24 cores = 45 Gflops = 1.9 Gflops/core > 48 cores = 95 Gflops = 2.0 Gflops/core > 72 cores = 148 Gflops = 2.1 Gflops/core > 96 cores = 220 Gflops = 2.3 Gflops/core >144 cores = 369 Gflops = 2.6 Gflops/core >288 cores = 784 Gflops = 2.7 Gflops/core >For the same code and lattice size running on Edison at NERSC we observed the >following performances. > 24 cores = 62 Gflops = 2.6 Gflops/core > 48 cores = 129 Gflops = 2.7 Gflops/core > 72 cores = 214 Gflops = 3.0 Gflops/core > 96 cores = 330 Gflops = 3.4 Gflops/core >144 cores = 561 Gflops = 3.9 Gflops/core >288 cores = 1219 Gflops = 4.2 Gflops/core >We use a custom assignment of tasks to nodes in order to minimize >communications. Earlier recoding reduced the number of global reductions in >the routines, which use most of the CPU time, by a factor of 2. > >The new Complex Langevin codes are still serial, but they will be parallelized >in the same manner as the RHMC codes. Algorithm changes are under consideration. >There is also the possibility that we will prepare an OpenMP version in >preparation for running on Cori II at NERSC. > > Requested: 20000 > >A specific reason has been given: >Needed to finish current set of runs. > >This needs to be approved and the final allocation amount decided upon. > >Thank You, >The LCRC Accounts System >_______________________________________________ >allocations-admins mailing list >allocations-admins(a)lcrc.anl.gov<mailto:[email protected]> >https://lists.lcrc.anl.gov/mailman/listinfo/allocations-admins

1 0

Re: [allocations-admins] [LCRC Accounts] Project Allocation Request - Metadynamics_LC
by John Blaas 11 Jan '16

11 Jan '16

100k granted for Q2, deferring the remaining 400k to AC meeting in march. - JB On Mon, Jan 11, 2016 at 8:58 AM, Bair, Raymond A. <rbair(a)anl.gov> wrote: > Lets add 100K now, and pass the other 400K to the AC for approval in March. > > Ray > > ----------------------------------------- > Ray Bair > Argonne National Laboratory > and the University of Chicago > > > > > > > > > > On 1/11/16, 8:57 AM, "Phillips, Carolyn L." <cphillips(a)anl.gov> wrote: > >>Hi Ray, >> >>Could I get 100K now. >> >>-Carolyn >> >>> On Jan 11, 2016, at 8:56 AM, Bair, Raymond A. <rbair(a)anl.gov> wrote: >>> >>> Carolyn, >>> >>> How much time do you need between now and 3/31/16? The Allocations Committee meets in March to make decisions about Q3 and Q4. So for big requests we provide part now and part through the Committee decision in March. >>> >>> Regards, >>> >>> Ray >>> >>> >>> ----------------------------------------- >>> Ray Bair >>> Argonne National Laboratory >>> and the University of Chicago >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> On 1/11/16, 8:37 AM, "allocations-admins-bounces(a)lcrc.anl.gov on behalf of accounts(a)lcrc.anl.gov" <allocations-admins-bounces(a)lcrc.anl.gov on behalf of accounts(a)lcrc.anl.gov> wrote: >>> >>>> Hello, >>>> >>>> A change in allocation has been requested: >>>> >>>> Requester: cphillips (Carolyn Phillips) >>>> Project: Metadynamics_LC >>>> Title: Using Parameter Free Metadynamics to calculate LC elastic constants >>>> Description: We will be using a parameter-free metadynamics method for an atomistic model of a liquid crystal 4-n-pentyl-4â€™-cyanobiphenyl (5CB), as well as the homologous series (6CB,7CB,8CB). >>>> >>>> Each nCB molecule will be simulated via a coarse-grained united-atom approximation in a classical N-body Molecular Dynamics code. We will be using NAMD + PLUMED2. Simulations will be used to measure the elastic constants of each liquid crystal system using metadynamics. >>>> >>>> NAMD is a well-parallelized stable MD code with a history of use by other users on Blues. Equilibrium simulations of the correct scale of the model described above parallelized across on 4x16-core nodes required ~300 MB of memory and ~36 hours to implement 10 million time steps. However most simulations will also be paired with PLUMED2, a parallelized version of PLUMED1.3 that can be used to implement metadynamics. Metadynamics is not known to perform well in parallel, due to a relatively high frequency of communication across all nodes, so I do not expect to use more than 1-4 nodes for individual "walkers" (simulation implementations). Instead production run simulations will likely use multiple (~20) "walkers" each deployed on 1-4 nodes, thus requiring ~80 nodes at a time. We will be evaluating trade-offs in length of simulation, number of nodes per implementation, and number of walkers to achieve convergence. Thus a better approximation of the allocation required to >>>> complete this project will result from the initial scoping stage of this project. >>>> >>>> Previous MD + metadynamics simulations have been performed on a different model of the system using a different MD code on Midway at UChicago. Small runs using NAMD on Blues and Fusion have already been issued to gain familiarity. >>>> >>>> Expected number of Project Members: 2 >>>> Current: undetermined amount >>>> Justification: A mix of simulation types are currently being run. >>>> >>>> - Large simulations that are scaled over four nodes at a time using NAMD >>>> - Smalls simulations being run over a single node but also using multiple walkers to empirically determine what walker scheme generates the best convergence. >>>> >>>> Using these two schemes I have used 100K this year. >>>> >>>> Soon I will need to run Large Simulations over 4 nodes with multiple walkers. >>>> >>>> Requested: 500000 >>>> >>>> A specific reason has been given: >>>> Testing a scheme for achieving convergence using many walkers and a small system plus equilibrating a large system that will be used to extract data has used 100K already this year. >>>> >>>> This needs to be approved and the final allocation amount decided upon. >>>> >>>> Thank You, >>>> The LCRC Accounts System >> > _______________________________________________ > allocations-admins mailing list > allocations-admins(a)lcrc.anl.gov > https://lists.lcrc.anl.gov/mailman/listinfo/allocations-admins

1 0

Re: [allocations-admins] [LCRC Accounts] Project Allocation Request - Metadynamics_LC
by Bair, Raymond A. 11 Jan '16

11 Jan '16

Lets add 100K now, and pass the other 400K to the AC for approval in March. Ray ----------------------------------------- Ray Bair Argonne National Laboratory and the University of Chicago On 1/11/16, 8:57 AM, "Phillips, Carolyn L." <cphillips(a)anl.gov> wrote: >Hi Ray, > >Could I get 100K now. > >-Carolyn > >> On Jan 11, 2016, at 8:56 AM, Bair, Raymond A. <rbair(a)anl.gov> wrote: >> >> Carolyn, >> >> How much time do you need between now and 3/31/16? The Allocations Committee meets in March to make decisions about Q3 and Q4. So for big requests we provide part now and part through the Committee decision in March. >> >> Regards, >> >> Ray >> >> >> ----------------------------------------- >> Ray Bair >> Argonne National Laboratory >> and the University of Chicago >> >> >> >> >> >> >> >> >> >> On 1/11/16, 8:37 AM, "allocations-admins-bounces(a)lcrc.anl.gov on behalf of accounts(a)lcrc.anl.gov" <allocations-admins-bounces(a)lcrc.anl.gov on behalf of accounts(a)lcrc.anl.gov> wrote: >> >>> Hello, >>> >>> A change in allocation has been requested: >>> >>> Requester: cphillips (Carolyn Phillips) >>> Project: Metadynamics_LC >>> Title: Using Parameter Free Metadynamics to calculate LC elastic constants >>> Description: We will be using a parameter-free metadynamics method for an atomistic model of a liquid crystal 4-n-pentyl-4â€™-cyanobiphenyl (5CB), as well as the homologous series (6CB,7CB,8CB). >>> >>> Each nCB molecule will be simulated via a coarse-grained united-atom approximation in a classical N-body Molecular Dynamics code. We will be using NAMD + PLUMED2. Simulations will be used to measure the elastic constants of each liquid crystal system using metadynamics. >>> >>> NAMD is a well-parallelized stable MD code with a history of use by other users on Blues. Equilibrium simulations of the correct scale of the model described above parallelized across on 4x16-core nodes required ~300 MB of memory and ~36 hours to implement 10 million time steps. However most simulations will also be paired with PLUMED2, a parallelized version of PLUMED1.3 that can be used to implement metadynamics. Metadynamics is not known to perform well in parallel, due to a relatively high frequency of communication across all nodes, so I do not expect to use more than 1-4 nodes for individual "walkers" (simulation implementations). Instead production run simulations will likely use multiple (~20) "walkers" each deployed on 1-4 nodes, thus requiring ~80 nodes at a time. We will be evaluating trade-offs in length of simulation, number of nodes per implementation, and number of walkers to achieve convergence. Thus a better approximation of the allocation required to >>> complete this project will result from the initial scoping stage of this project. >>> >>> Previous MD + metadynamics simulations have been performed on a different model of the system using a different MD code on Midway at UChicago. Small runs using NAMD on Blues and Fusion have already been issued to gain familiarity. >>> >>> Expected number of Project Members: 2 >>> Current: undetermined amount >>> Justification: A mix of simulation types are currently being run. >>> >>> - Large simulations that are scaled over four nodes at a time using NAMD >>> - Smalls simulations being run over a single node but also using multiple walkers to empirically determine what walker scheme generates the best convergence. >>> >>> Using these two schemes I have used 100K this year. >>> >>> Soon I will need to run Large Simulations over 4 nodes with multiple walkers. >>> >>> Requested: 500000 >>> >>> A specific reason has been given: >>> Testing a scheme for achieving convergence using many walkers and a small system plus equilibrating a large system that will be used to extract data has used 100K already this year. >>> >>> This needs to be approved and the final allocation amount decided upon. >>> >>> Thank You, >>> The LCRC Accounts System >

1 0

Re: [allocations-admins] [LCRC Accounts] Project Allocation Request - Metadynamics_LC
by Bair, Raymond A. 11 Jan '16

11 Jan '16

Carolyn, How much time do you need between now and 3/31/16? The Allocations Committee meets in March to make decisions about Q3 and Q4. So for big requests we provide part now and part through the Committee decision in March. Regards, Ray ----------------------------------------- Ray Bair Argonne National Laboratory and the University of Chicago On 1/11/16, 8:37 AM, "allocations-admins-bounces(a)lcrc.anl.gov on behalf of accounts(a)lcrc.anl.gov" <allocations-admins-bounces(a)lcrc.anl.gov on behalf of accounts(a)lcrc.anl.gov> wrote: >Hello, > >A change in allocation has been requested: > > Requester: cphillips (Carolyn Phillips) > Project: Metadynamics_LC > Title: Using Parameter Free Metadynamics to calculate LC elastic constants > Description: We will be using a parameter-free metadynamics method for an atomistic model of a liquid crystal 4-n-pentyl-4â€™-cyanobiphenyl (5CB), as well as the homologous series (6CB,7CB,8CB). > >Each nCB molecule will be simulated via a coarse-grained united-atom approximation in a classical N-body Molecular Dynamics code. We will be using NAMD + PLUMED2. Simulations will be used to measure the elastic constants of each liquid crystal system using metadynamics. > >NAMD is a well-parallelized stable MD code with a history of use by other users on Blues. Equilibrium simulations of the correct scale of the model described above parallelized across on 4x16-core nodes required ~300 MB of memory and ~36 hours to implement 10 million time steps. However most simulations will also be paired with PLUMED2, a parallelized version of PLUMED1.3 that can be used to implement metadynamics. Metadynamics is not known to perform well in parallel, due to a relatively high frequency of communication across all nodes, so I do not expect to use more than 1-4 nodes for individual "walkers" (simulation implementations). Instead production run simulations will likely use multiple (~20) "walkers" each deployed on 1-4 nodes, thus requiring ~80 nodes at a time. We will be evaluating trade-offs in length of simulation, number of nodes per implementation, and number of walkers to achieve convergence. Thus a better approximation of the allocation required to > complete this project will result from the initial scoping stage of this project. > >Previous MD + metadynamics simulations have been performed on a different model of the system using a different MD code on Midway at UChicago. Small runs using NAMD on Blues and Fusion have already been issued to gain familiarity. > >Expected number of Project Members: 2 > Current: undetermined amount >Justification: A mix of simulation types are currently being run. > >- Large simulations that are scaled over four nodes at a time using NAMD >- Smalls simulations being run over a single node but also using multiple walkers to empirically determine what walker scheme generates the best convergence. > >Using these two schemes I have used 100K this year. > >Soon I will need to run Large Simulations over 4 nodes with multiple walkers. > > Requested: 500000 > >A specific reason has been given: >Testing a scheme for achieving convergence using many walkers and a small system plus equilibrating a large system that will be used to extract data has used 100K already this year. > >This needs to be approved and the final allocation amount decided upon. > >Thank You, >The LCRC Accounts System

1 0

[LCRC Accounts] Project Allocation Request
by accounts＠lcrc.anl.gov 11 Jan '16

11 Jan '16

Hello, A change in allocation has been requested: Requester: cphillips (Carolyn Phillips) Project: Metadynamics_LC Title: Using Parameter Free Metadynamics to calculate LC elastic constants Description: We will be using a parameter-free metadynamics method for an atomistic model of a liquid crystal 4-n-pentyl-4’-cyanobiphenyl (5CB), as well as the homologous series (6CB,7CB,8CB). Each nCB molecule will be simulated via a coarse-grained united-atom approximation in a classical N-body Molecular Dynamics code. We will be using NAMD + PLUMED2. Simulations will be used to measure the elastic constants of each liquid crystal system using metadynamics. NAMD is a well-parallelized stable MD code with a history of use by other users on Blues. Equilibrium simulations of the correct scale of the model described above parallelized across on 4x16-core nodes required ~300 MB of memory and ~36 hours to implement 10 million time steps. However most simulations will also be paired with PLUMED2, a parallelized version of PLUMED1.3 that can be used to implement metadynamics. Metadynamics is not known to perform well in parallel, due to a relatively high frequency of communication across all nodes, so I do not expect to use more than 1-4 nodes for individual "walkers" (simulation implementations). Instead production run simulations will likely use multiple (~20) "walkers" each deployed on 1-4 nodes, thus requiring ~80 nodes at a time. We will be evaluating trade-offs in length of simulation, number of nodes per implementation, and number of walkers to achieve convergence. Thus a better approximation of the allocation required to complete this project will result from the initial scoping stage of this project. Previous MD + metadynamics simulations have been performed on a different model of the system using a different MD code on Midway at UChicago. Small runs using NAMD on Blues and Fusion have already been issued to gain familiarity. Expected number of Project Members: 2 Current: undetermined amount Justification: A mix of simulation types are currently being run. - Large simulations that are scaled over four nodes at a time using NAMD - Smalls simulations being run over a single node but also using multiple walkers to empirically determine what walker scheme generates the best convergence. Using these two schemes I have used 100K this year. Soon I will need to run Large Simulations over 4 nodes with multiple walkers. Requested: 500000 A specific reason has been given: Testing a scheme for achieving convergence using many walkers and a small system plus equilibrating a large system that will be used to extract data has used 100K already this year. This needs to be approved and the final allocation amount decided upon. Thank You, The LCRC Accounts System

1 0

Re: [allocations-admins] [LCRC Accounts] Project Allocation Request
by Bair, Raymond A. 11 Jan '16

11 Jan '16

Don, Did you intend to request only 20,000 core-hours more? Ray ----------------------------------------- Ray Bair Argonne National Laboratory and the University of Chicago On 1/9/16, 4:23 PM, "allocations-admins-bounces(a)lcrc.anl.gov on behalf of accounts(a)lcrc.anl.gov" <allocations-admins-bounces(a)lcrc.anl.gov on behalf of accounts(a)lcrc.anl.gov> wrote: >Hello, > >A change in allocation has been requested: > > Requester: sinclair (Donald Sinclair) > Project: Lattice-QCD > Title: Lattice simulations of Conformal and Walking >Technicolor. > Description: We perform simulations to evaluate the functional integrals of QCD-like >theories formulated on a discrete space-time lattice, to enable determination >of the non-perturbative aspects these theories. These include the properties >of these theories at non-zero temperature, including the scales of confinement >and chiral symmetry breaking, and such zero temperature properties as spectra >(including the Higgs mass), decay constants and the running of the gauge >coupling constant. We are particularly interested in those theories where the >coupling constant evolves very slowly, since these are candidate 'Walking- >Technicolor' theories. Related to these are theories with an infrared fixed >point (conformal field theories). Our first goal is to differentiate between >these two different types of behaviour for candidate theories. > >We are performing simulations of QCD-like theories which are models for >Walking or Conformal Technicolor. We have been studying theories which are >essentially QCD but with colour-sextet rather than colour-triplet quarks. We >hope to measure the running of the QCD coupling constant. For 2 or 3 flavours, >2-loop perturbation theory predicts an infrared fixed point. > >For 2 flavours, it is possible that a chiral condensate forms before this >fixed point is reached. If so, the fixed point is avoided, the theory is >confining, and chiral symmetry breaks spontaneously. However, there is a >region where the coupling constant evolves very slowly. These are the >properties required for a walking technicolor theory. Simulations we have >performed so far at finite temperature suggest that this theory might be >conformal. So far, however, the results are inconclusive. > >For 3 flavours we know that the theory should be conformal. Simulations at >N_t=6 (12^3 X 6 lattice) and N_t=8 (12^3 X 8 lattice), some of which used >Fusion, did not yet show evidence of conformality. We are now simulating at >N_t=12 (24^3 X 12 lattice). We started these simulations on Blues, and intend >to continue them in FY2016 on Blues to enable continuity during the period >when Edison at NERSC is unavailable due to a site move. > >Our sextet quark codes are based on our earlier triplet quark codes and use >the RHMC simulation method. The Rational Hybrid Monte Carlo (RHMC) is a >stochastic molecular dynamics algorithm. The functional integral of QCD is >written as a partition function of a classical field theory evolving in a >fictitious time. The determinant of the Dirac operator raised to a fractional >power is calculated by introducing bosonic fields (pseudofermions), and >sandwiching this Dirac operator raised to minus said fractional power between >them. This fractional power of the Dirac operator is approximated to machine >accuracy by a rational approximation. After defining this theory on a discrete >space-time lattice, the inversions required by the partial-fraction expansion >of the rational approximation are performed using Krylov space methods, in >particular a multi-shift extension of the conjugate gradient algorithm. A >global Metropolis Monte-Carlo accept/reject step applied at the end of each >trajectory removes discretization errors introduced by the numerical >integration of these stochastic equations of motion. We parallelize the code >by assigning a fixed number of adjacent lattice sites to each MPI task. >Network bandwidth ultimately limits how small a chunk of the lattice can be >assigned to each task. > >Our 24^3 X 12 runs will be performed on 288 cores (18 nodes) of Blues. > >A short benchmark run of the 24^3 X 12 codes yielded 667 Gflops or 2.3 >Gflops/core. Since we have seen performances of > 3 Gflops/core on machines >with similar processors, and > 4 Gflops/node on Edison, we suspect that at >least 3 Gflops/core should be achievable on this machine. Since we have nearly >exhausted our allocation on Blues, we cannot perform more extensive tests. >Scaling tests on Fusion and on Edison at NERSC for numbers of cores from 24 up >to 288 cores indicate that the per core performance increases up to 288 cores, >which we believe to be related to cache usage. > >Our new project simulating QCD at finite mu using Complex Langevin with Gauge >Cooling is currently running as serial code on several platforms, including >Blues, where the little allocation we have remaining does not allow us to run >anything else. The Langevin equation method is essentially the limiting case of >the above algorithm in its older form (without the rational approximation), >when each trajectory involves a single update of the fields. Its extension to >the complex manifold merely involves extending the gauge-field manifold from >SU(3) to SL(3,C). The main reason that it is serial code is that this project >is still in its exploratory phase, meaning that modifications of the algorithm >are still possible. Converting to parallel MPI code will not be difficult, >since the compute-intensive kernel is almost identical to that of our older >QCD codes, as are many other parts of the code. We do, however, expect some >performance degradation, since we have been running this code at 64-bit >precision, rather than the 32-bit precision of our QCD codes. When we have >converted to parallel code, we will cease our serial runs on Blues. While we >perform zero-temperature simulations at NERSC, TACC and SDSC, we plan to start >finite temperature simulations on Blues, probably starting on 8^3*4 lattices >using 16 or 32 cores, and then moving to 12^3*4 and 12^3*6 lattices on 48 and >72 cores respectively. Note that these runs with low-level parallelism are >discouraged on the flagship machines at NERSC. > >Our requested allocation is based on running 2 288-core jobs half of the time >for the first 4-months and jobs using a total of 144 cores half the time for >the last 8 months. Expected percentage of allocation for serial jobs -- 5%. > Current: undetermined amount >Justification: We have not yet performed a detailed scaling analysis on Blues, however, we >have such analyses performed on Fusion as well as Edison at NERSC. For Fusion, >for our QCD with sextet quarks code on a 24^3 X 12 lattice running on Fusion >we observed the following performances: > 24 cores = 45 Gflops = 1.9 Gflops/core > 48 cores = 95 Gflops = 2.0 Gflops/core > 72 cores = 148 Gflops = 2.1 Gflops/core > 96 cores = 220 Gflops = 2.3 Gflops/core >144 cores = 369 Gflops = 2.6 Gflops/core >288 cores = 784 Gflops = 2.7 Gflops/core >For the same code and lattice size running on Edison at NERSC we observed the >following performances. > 24 cores = 62 Gflops = 2.6 Gflops/core > 48 cores = 129 Gflops = 2.7 Gflops/core > 72 cores = 214 Gflops = 3.0 Gflops/core > 96 cores = 330 Gflops = 3.4 Gflops/core >144 cores = 561 Gflops = 3.9 Gflops/core >288 cores = 1219 Gflops = 4.2 Gflops/core >We use a custom assignment of tasks to nodes in order to minimize >communications. Earlier recoding reduced the number of global reductions in >the routines, which use most of the CPU time, by a factor of 2. > >The new Complex Langevin codes are still serial, but they will be parallelized >in the same manner as the RHMC codes. Algorithm changes are under consideration. >There is also the possibility that we will prepare an OpenMP version in >preparation for running on Cori II at NERSC. > > Requested: 20000 > >A specific reason has been given: >Needed to finish current set of runs. > >This needs to be approved and the final allocation amount decided upon. > >Thank You, >The LCRC Accounts System >_______________________________________________ >allocations-admins mailing list >allocations-admins(a)lcrc.anl.gov >https://lists.lcrc.anl.gov/mailman/listinfo/allocations-admins

1 0

[LCRC Accounts] Project Allocation Request
by accounts＠lcrc.anl.gov 09 Jan '16

09 Jan '16

Hello, A change in allocation has been requested: Requester: sinclair (Donald Sinclair) Project: Lattice-QCD Title: Lattice simulations of Conformal and Walking Technicolor. Description: We perform simulations to evaluate the functional integrals of QCD-like theories formulated on a discrete space-time lattice, to enable determination of the non-perturbative aspects these theories. These include the properties of these theories at non-zero temperature, including the scales of confinement and chiral symmetry breaking, and such zero temperature properties as spectra (including the Higgs mass), decay constants and the running of the gauge coupling constant. We are particularly interested in those theories where the coupling constant evolves very slowly, since these are candidate 'Walking- Technicolor' theories. Related to these are theories with an infrared fixed point (conformal field theories). Our first goal is to differentiate between these two different types of behaviour for candidate theories. We are performing simulations of QCD-like theories which are models for Walking or Conformal Technicolor. We have been studying theories which are essentially QCD but with colour-sextet rather than colour-triplet quarks. We hope to measure the running of the QCD coupling constant. For 2 or 3 flavours, 2-loop perturbation theory predicts an infrared fixed point. For 2 flavours, it is possible that a chiral condensate forms before this fixed point is reached. If so, the fixed point is avoided, the theory is confining, and chiral symmetry breaks spontaneously. However, there is a region where the coupling constant evolves very slowly. These are the properties required for a walking technicolor theory. Simulations we have performed so far at finite temperature suggest that this theory might be conformal. So far, however, the results are inconclusive. For 3 flavours we know that the theory should be conformal. Simulations at N_t=6 (12^3 X 6 lattice) and N_t=8 (12^3 X 8 lattice), some of which used Fusion, did not yet show evidence of conformality. We are now simulating at N_t=12 (24^3 X 12 lattice). We started these simulations on Blues, and intend to continue them in FY2016 on Blues to enable continuity during the period when Edison at NERSC is unavailable due to a site move. Our sextet quark codes are based on our earlier triplet quark codes and use the RHMC simulation method. The Rational Hybrid Monte Carlo (RHMC) is a stochastic molecular dynamics algorithm. The functional integral of QCD is written as a partition function of a classical field theory evolving in a fictitious time. The determinant of the Dirac operator raised to a fractional power is calculated by introducing bosonic fields (pseudofermions), and sandwiching this Dirac operator raised to minus said fractional power between them. This fractional power of the Dirac operator is approximated to machine accuracy by a rational approximation. After defining this theory on a discrete space-time lattice, the inversions required by the partial-fraction expansion of the rational approximation are performed using Krylov space methods, in particular a multi-shift extension of the conjugate gradient algorithm. A global Metropolis Monte-Carlo accept/reject step applied at the end of each trajectory removes discretization errors introduced by the numerical integration of these stochastic equations of motion. We parallelize the code by assigning a fixed number of adjacent lattice sites to each MPI task. Network bandwidth ultimately limits how small a chunk of the lattice can be assigned to each task. Our 24^3 X 12 runs will be performed on 288 cores (18 nodes) of Blues. A short benchmark run of the 24^3 X 12 codes yielded 667 Gflops or 2.3 Gflops/core. Since we have seen performances of > 3 Gflops/core on machines with similar processors, and > 4 Gflops/node on Edison, we suspect that at least 3 Gflops/core should be achievable on this machine. Since we have nearly exhausted our allocation on Blues, we cannot perform more extensive tests. Scaling tests on Fusion and on Edison at NERSC for numbers of cores from 24 up to 288 cores indicate that the per core performance increases up to 288 cores, which we believe to be related to cache usage. Our new project simulating QCD at finite mu using Complex Langevin with Gauge Cooling is currently running as serial code on several platforms, including Blues, where the little allocation we have remaining does not allow us to run anything else. The Langevin equation method is essentially the limiting case of the above algorithm in its older form (without the rational approximation), when each trajectory involves a single update of the fields. Its extension to the complex manifold merely involves extending the gauge-field manifold from SU(3) to SL(3,C). The main reason that it is serial code is that this project is still in its exploratory phase, meaning that modifications of the algorithm are still possible. Converting to parallel MPI code will not be difficult, since the compute-intensive kernel is almost identical to that of our older QCD codes, as are many other parts of the code. We do, however, expect some performance degradation, since we have been running this code at 64-bit precision, rather than the 32-bit precision of our QCD codes. When we have converted to parallel code, we will cease our serial runs on Blues. While we perform zero-temperature simulations at NERSC, TACC and SDSC, we plan to start finite temperature simulations on Blues, probably starting on 8^3*4 lattices using 16 or 32 cores, and then moving to 12^3*4 and 12^3*6 lattices on 48 and 72 cores respectively. Note that these runs with low-level parallelism are discouraged on the flagship machines at NERSC. Our requested allocation is based on running 2 288-core jobs half of the time for the first 4-months and jobs using a total of 144 cores half the time for the last 8 months. Expected percentage of allocation for serial jobs -- 5%. Current: undetermined amount Justification: We have not yet performed a detailed scaling analysis on Blues, however, we have such analyses performed on Fusion as well as Edison at NERSC. For Fusion, for our QCD with sextet quarks code on a 24^3 X 12 lattice running on Fusion we observed the following performances: 24 cores = 45 Gflops = 1.9 Gflops/core 48 cores = 95 Gflops = 2.0 Gflops/core 72 cores = 148 Gflops = 2.1 Gflops/core 96 cores = 220 Gflops = 2.3 Gflops/core 144 cores = 369 Gflops = 2.6 Gflops/core 288 cores = 784 Gflops = 2.7 Gflops/core For the same code and lattice size running on Edison at NERSC we observed the following performances. 24 cores = 62 Gflops = 2.6 Gflops/core 48 cores = 129 Gflops = 2.7 Gflops/core 72 cores = 214 Gflops = 3.0 Gflops/core 96 cores = 330 Gflops = 3.4 Gflops/core 144 cores = 561 Gflops = 3.9 Gflops/core 288 cores = 1219 Gflops = 4.2 Gflops/core We use a custom assignment of tasks to nodes in order to minimize communications. Earlier recoding reduced the number of global reductions in the routines, which use most of the CPU time, by a factor of 2. The new Complex Langevin codes are still serial, but they will be parallelized in the same manner as the RHMC codes. Algorithm changes are under consideration. There is also the possibility that we will prepare an OpenMP version in preparation for running on Cori II at NERSC. Requested: 20000 A specific reason has been given: Needed to finish current set of runs. This needs to be approved and the final allocation amount decided upon. Thank You, The LCRC Accounts System

1 0