Don,

Did you intend to request only 20,000 core-hours more?

Ray


-----------------------------------------
Ray Bair
Argonne National Laboratory
 and the University of Chicago









On 1/9/16, 4:23 PM, "allocations-admins-bounces@lcrc.anl.gov on behalf of accounts@lcrc.anl.gov" <allocations-admins-bounces@lcrc.anl.gov on behalf of accounts@lcrc.anl.gov> wrote:

>Hello,
>
>A change in allocation has been requested:
>
>    Requester: sinclair (Donald Sinclair)
>      Project: Lattice-QCD
>        Title: Lattice simulations of Conformal and Walking
>Technicolor.
>  Description: We perform simulations to evaluate the functional integrals of QCD-like
>theories formulated on a discrete space-time lattice, to enable determination
>of the non-perturbative aspects these theories. These include the properties
>of these theories at non-zero temperature, including the scales of confinement
>and chiral symmetry breaking, and such zero temperature properties as spectra
>(including the Higgs mass), decay constants and the running of the gauge
>coupling constant. We are particularly interested in those theories where the
>coupling constant evolves very slowly, since these are candidate 'Walking-
>Technicolor' theories. Related to these are theories with an infrared fixed
>point (conformal field theories). Our first goal is to differentiate between
>these two different types of behaviour for candidate theories.
>
>We are performing simulations of QCD-like theories which are models for
>Walking or Conformal Technicolor. We have been studying theories which are
>essentially QCD but with colour-sextet rather than colour-triplet quarks. We
>hope to measure the running of the QCD coupling constant. For 2 or 3 flavours,
>2-loop perturbation theory predicts an infrared fixed point.
>
>For 2 flavours, it is possible that a chiral condensate forms before this
>fixed point is reached. If so, the fixed point is avoided, the theory is
>confining, and chiral symmetry breaks spontaneously. However, there is a
>region where the coupling constant evolves very slowly. These are the
>properties required for a walking technicolor theory. Simulations we have
>performed so far at finite temperature suggest that this theory might be
>conformal. So far, however, the results are inconclusive.
>
>For 3 flavours we know that the theory should be conformal. Simulations at
>N_t=6 (12^3 X 6 lattice) and N_t=8 (12^3 X 8 lattice), some of which used
>Fusion, did not yet show evidence of conformality. We are now simulating at
>N_t=12 (24^3 X 12 lattice). We started these simulations on Blues, and intend
>to continue them in FY2016 on Blues to enable continuity during the period
>when Edison at NERSC is unavailable due to a site move.
>
>Our sextet quark codes are based on our earlier triplet quark codes and use
>the RHMC simulation method. The Rational Hybrid Monte Carlo (RHMC) is a
>stochastic molecular dynamics algorithm. The functional integral of QCD is
>written as a partition function of a classical field theory evolving in a
>fictitious time. The determinant of the Dirac operator raised to a fractional
>power is calculated by introducing bosonic fields (pseudofermions), and
>sandwiching this Dirac operator raised to minus said fractional power between
>them. This fractional power of the Dirac operator is approximated to machine
>accuracy by a rational approximation. After defining this theory on a discrete
>space-time lattice, the inversions required by the partial-fraction expansion
>of the rational approximation are performed using Krylov space methods, in
>particular a multi-shift extension of the conjugate gradient algorithm. A
>global Metropolis Monte-Carlo accept/reject step applied at the end of each
>trajectory removes discretization errors introduced by the numerical
>integration of these stochastic equations of motion. We parallelize the code
>by assigning a fixed number of adjacent lattice sites to each MPI task.
>Network bandwidth ultimately limits how small a chunk of the lattice can be
>assigned to each task.
>
>Our 24^3 X 12 runs will be performed on 288 cores (18 nodes) of Blues.
>
>A short benchmark run of the 24^3 X 12 codes yielded 667 Gflops or 2.3
>Gflops/core. Since we have seen performances of > 3 Gflops/core on machines
>with similar processors, and > 4 Gflops/node on Edison, we suspect that at
>least 3 Gflops/core should be achievable on this machine. Since we have nearly
>exhausted our allocation on Blues, we cannot perform more extensive tests.
>Scaling tests on Fusion and on Edison at NERSC for numbers of cores from 24 up
>to 288 cores indicate that the per core performance increases up to 288 cores,
>which we believe to be related to cache usage.
>
>Our new project simulating QCD at finite mu using Complex Langevin with Gauge
>Cooling is currently running as serial code on several platforms, including
>Blues, where the little allocation we have remaining does not allow us to run
>anything else. The Langevin equation method is essentially the limiting case of
>the above algorithm in its older form (without the rational approximation),
>when each trajectory involves a single update of the fields. Its extension to
>the complex manifold merely involves extending the gauge-field manifold from
>SU(3) to SL(3,C). The main reason that it is serial code is that this project
>is still in its exploratory phase, meaning that modifications of the algorithm
>are still possible. Converting to parallel MPI code will not be difficult,
>since the compute-intensive kernel is almost identical to that of our older
>QCD codes, as are many other parts of the code. We do, however, expect some
>performance degradation, since we have been running this code at 64-bit
>precision, rather than the 32-bit precision of our QCD codes. When we have
>converted to parallel code, we will cease our serial runs on Blues. While we
>perform zero-temperature simulations at NERSC, TACC and SDSC, we plan to start
>finite temperature simulations on Blues, probably starting on 8^3*4 lattices
>using 16 or 32 cores, and then moving to 12^3*4 and 12^3*6 lattices on 48 and
>72 cores respectively. Note that these runs with low-level parallelism are
>discouraged on the flagship machines at NERSC.
>
>Our requested allocation is based on running 2 288-core jobs half of the time
>for the first 4-months and jobs using a total of 144 cores half the time for
>the last 8 months. Expected percentage of allocation for serial jobs -- 5%.
>      Current: undetermined amount
>Justification: We have not yet performed a detailed scaling analysis on Blues, however, we
>have such analyses performed on Fusion as well as Edison at NERSC. For Fusion,
>for our QCD with sextet quarks code on a 24^3 X 12 lattice running on Fusion
>we observed the following performances:
> 24 cores =   45 Gflops = 1.9 Gflops/core
> 48 cores =   95 Gflops = 2.0 Gflops/core
> 72 cores =  148 Gflops = 2.1 Gflops/core
> 96 cores =  220 Gflops = 2.3 Gflops/core
>144 cores =  369 Gflops = 2.6 Gflops/core
>288 cores =  784 Gflops = 2.7 Gflops/core
>For the same code and lattice size running on Edison at NERSC we observed the
>following performances.
> 24 cores =   62 Gflops = 2.6 Gflops/core
> 48 cores =  129 Gflops = 2.7 Gflops/core
> 72 cores =  214 Gflops = 3.0 Gflops/core
> 96 cores =  330 Gflops = 3.4 Gflops/core
>144 cores =  561 Gflops = 3.9 Gflops/core
>288 cores = 1219 Gflops = 4.2 Gflops/core
>We use a custom assignment of tasks to nodes in order to minimize
>communications. Earlier recoding reduced the number of global reductions in
>the routines, which use most of the CPU time, by a factor of 2.
>
>The new Complex Langevin codes are still serial, but they will be parallelized
>in the same manner as the RHMC codes. Algorithm changes are under consideration.
>There is also the possibility that we will prepare an OpenMP version in
>preparation for running on Cori II at NERSC.
>
>    Requested: 20000
>
>A specific reason has been given:
>Needed to finish current set of runs.
>
>This needs to be approved and the final allocation amount decided upon.
>
>Thank You,
>The LCRC Accounts System
>_______________________________________________
>allocations-admins mailing list
>allocations-admins@lcrc.anl.gov
>https://lists.lcrc.anl.gov/mailman/listinfo/allocations-admins