Hello,
A change in allocation has been requested:
Requester: sinclair (Donald Sinclair)
Project: Lattice-QCD
Title: Lattice simulations of Conformal and Walking
Technicolor.
Description: We perform simulations to evaluate the functional integrals of QCD-like
theories formulated on a discrete space-time lattice, to enable determination
of the non-perturbative aspects these theories. These include the properties
of these theories at non-zero temperature, including the scales of confinement
and chiral symmetry breaking, and such zero temperature properties as spectra
(including the Higgs mass), decay constants and the running of the gauge
coupling constant. We are particularly interested in those theories where the
coupling constant evolves very slowly, since these are candidate 'Walking-
Technicolor' theories. Related to these are theories with an infrared fixed
point (conformal field theories). Our first goal is to differentiate between
these two different types of behaviour for candidate theories.
We are performing simulations of QCD-like theories which are models for
Walking or Conformal Technicolor. We have been studying theories which are
essentially QCD but with colour-sextet rather than colour-triplet quarks. We
hope to measure the running of the QCD coupling constant. For 2 or 3 flavours,
2-loop perturbation theory predicts an infrared fixed point.
For 2 flavours, it is possible that a chiral condensate forms before this
fixed point is reached. If so, the fixed point is avoided, the theory is
confining, and chiral symmetry breaks spontaneously. However, there is a
region where the coupling constant evolves very slowly. These are the
properties required for a walking technicolor theory. Simulations we have
performed so far at finite temperature suggest that this theory might be
conformal. So far, however, the results are inconclusive.
For 3 flavours we know that the theory should be conformal. Simulations at
N_t=6 (12^3 X 6 lattice) and N_t=8 (12^3 X 8 lattice), some of which used
Fusion, did not yet show evidence of conformality. We are now simulating at
N_t=12 (24^3 X 12 lattice). We started these simulations on Blues, and intend
to continue them in FY2016 on Blues to enable continuity during the period
when Edison at NERSC is unavailable due to a site move.
Our sextet quark codes are based on our earlier triplet quark codes and use
the RHMC simulation method. The Rational Hybrid Monte Carlo (RHMC) is a
stochastic molecular dynamics algorithm. The functional integral of QCD is
written as a partition function of a classical field theory evolving in a
fictitious time. The determinant of the Dirac operator raised to a fractional
power is calculated by introducing bosonic fields (pseudofermions), and
sandwiching this Dirac operator raised to minus said fractional power between
them. This fractional power of the Dirac operator is approximated to machine
accuracy by a rational approximation. After defining this theory on a discrete
space-time lattice, the inversions required by the partial-fraction expansion
of the rational approximation are performed using Krylov space methods, in
particular a multi-shift extension of the conjugate gradient algorithm. A
global Metropolis Monte-Carlo accept/reject step applied at the end of each
trajectory removes discretization errors introduced by the numerical
integration of these stochastic equations of motion. We parallelize the code
by assigning a fixed number of adjacent lattice sites to each MPI task.
Network bandwidth ultimately limits how small a chunk of the lattice can be
assigned to each task.
Our 24^3 X 12 runs will be performed on 288 cores (18 nodes) of Blues.
A short benchmark run of the 24^3 X 12 codes yielded 667 Gflops or 2.3
Gflops/core. Since we have seen performances of > 3 Gflops/core on machines
with similar processors, and > 4 Gflops/node on Edison, we suspect that at
least 3 Gflops/core should be achievable on this machine. Since we have nearly
exhausted our allocation on Blues, we cannot perform more extensive tests.
Scaling tests on Fusion and on Edison at NERSC for numbers of cores from 24 up
to 288 cores indicate that the per core performance increases up to 288 cores,
which we believe to be related to cache usage.
Our new project simulating QCD at finite mu using Complex Langevin with Gauge
Cooling is currently running as serial code on several platforms, including
Blues, where the little allocation we have remaining does not allow us to run
anything else. The Langevin equation method is essentially the limiting case of
the above algorithm in its older form (without the rational approximation),
when each trajectory involves a single update of the fields. Its extension to
the complex manifold merely involves extending the gauge-field manifold from
SU(3) to SL(3,C). The main reason that it is serial code is that this project
is still in its exploratory phase, meaning that modifications of the algorithm
are still possible. Converting to parallel MPI code will not be difficult,
since the compute-intensive kernel is almost identical to that of our older
QCD codes, as are many other parts of the code. We do, however, expect some
performance degradation, since we have been running this code at 64-bit
precision, rather than the 32-bit precision of our QCD codes. When we have
converted to parallel code, we will cease our serial runs on Blues. While we
perform zero-temperature simulations at NERSC, TACC and SDSC, we plan to start
finite temperature simulations on Blues, probably starting on 8^3*4 lattices
using 16 or 32 cores, and then moving to 12^3*4 and 12^3*6 lattices on 48 and
72 cores respectively. Note that these runs with low-level parallelism are
discouraged on the flagship machines at NERSC.
Our requested allocation is based on running 2 288-core jobs half of the time
for the first 4-months and jobs using a total of 144 cores half the time for
the last 8 months. Expected percentage of allocation for serial jobs -- 5%.
Current: undetermined amount
Justification: We have not yet performed a detailed scaling analysis on Blues, however, we
have such analyses performed on Fusion as well as Edison at NERSC. For Fusion,
for our QCD with sextet quarks code on a 24^3 X 12 lattice running on Fusion
we observed the following performances:
24 cores = 45 Gflops = 1.9 Gflops/core
48 cores = 95 Gflops = 2.0 Gflops/core
72 cores = 148 Gflops = 2.1 Gflops/core
96 cores = 220 Gflops = 2.3 Gflops/core
144 cores = 369 Gflops = 2.6 Gflops/core
288 cores = 784 Gflops = 2.7 Gflops/core
For the same code and lattice size running on Edison at NERSC we observed the
following performances.
24 cores = 62 Gflops = 2.6 Gflops/core
48 cores = 129 Gflops = 2.7 Gflops/core
72 cores = 214 Gflops = 3.0 Gflops/core
96 cores = 330 Gflops = 3.4 Gflops/core
144 cores = 561 Gflops = 3.9 Gflops/core
288 cores = 1219 Gflops = 4.2 Gflops/core
We use a custom assignment of tasks to nodes in order to minimize
communications. Earlier recoding reduced the number of global reductions in
the routines, which use most of the CPU time, by a factor of 2.
The new Complex Langevin codes are still serial, but they will be parallelized
in the same manner as the RHMC codes. Algorithm changes are under consideration.
There is also the possibility that we will prepare an OpenMP version in
preparation for running on Cori II at NERSC.
Requested: 150000
A specific reason has been given:
Needed to finish current runs. Other resources are not available. Allocation on Stampede at TACC is exhausted. Allocation on Comet at SDSC is almost exhausted. NERSC is severely impacted by site move -- Edison is offline for 6 weeks, Hopper will be decommissioned 15th December, and Cori I is inadequate to take up the slack.
This needs to be approved and the final allocation amount decided upon.
Thank You,
The LCRC Accounts System
