Hello,
A yearly allocation for the LCRC cluster has been requested with the
following updated information:
Submitter/PI: John Blaas
Project Name: Lattice-QCD
Division: HEP
Project title: Lattice simulations of Conformal and Walking
Technicolor.
Associated funding: DOE
Other Systems: NERSC Cray XC30, "Edison"
NERSC Cray XT6, "Hopper"
NERSC Cluster, "Carver"
Current NERSC allocation: 24,000,000 MPP hours
TACC cluster, "Stampede"
Allocation 5,000,000 core-hours.
SDSC cluster, "Comet"
Allocation 1,000,000 core-hours.
Science: We are using our simulation methods developed for lattice QCD to study
extensions of the standard model of High Energy Physics, in which the Higgs
sector is strongly interacting, and the Higgs fields are composites. This has
become all the more relevant with the discovery of a Higgs-like particle at
the LHC at CERN. We are particularly interested in QCD-like theories whose
pions play the role of the Higgs field in giving masses to the W and Z weak
vector bosons. Such theories are called Technicolor theories. We are
particularly interested in Walking Technicolor theories, whose slowly evolving
couplings evade some of the technical difficulties associated with the
phenomenology of Technicolor and Extended Technicolor theories. Our initial
studies are of the thermodynamics of such theories, since these provide a
simple way of measuring the scales of confinement and of chiral symmetry
breaking. From such simulations, we can determine if confinement and
chiral-symmetry breaking survive the continuum limit. In this case, our chosen
theory is QCD-like. If not the theory is a conformal field theory.
We choose theories which 2-loop perturbation theory suggests have an infrared
fixed point and are thus conformal. If chiral symmetry breaks spontaneously
before this fixed point is reached, the fixed point is avoided and the theory
is ultimately QCD-like. However, the close proximity of the fixed point means
that there is a region where the running coupling constant evolves very
slowly, i.e. the theory walks.
The candidate Walking-Technicolor theory which we are studying is 'scaled up'
QCD with 2 colour-sextet quarks. QCD with 2 sextet quarks has just the right
number of Goldstone 'pions' (3) to give masses to the Ws and Z. It remains an
open question as to whether it has a light Higgs. Because it is difficult to
determine the nature of this theory, it is useful to study a closely related
theory with known behaviour. We study QCD with 3 sextet quarks which is almost
certainly conformal for comparison.
We started second project starting this year, namely simulations of Lattice QCD
at finite quark-number chemical potential mu, using Complex Langevin methods.
This is relevant to the understanding of the phases of nuclear-matter at zero
and finite temperature. Cold nuclear matter is found in neutron stars. Hot
nuclear matter (quark-gluon matter/plasma) is believed to be produced in
relativistic heavy-ion colliders, and was certainly present in the early
universe, Since the integrand of the partition function, which describes the
quantum dynamics of QCD at finite mu, is complex, standard simulation methods
which rely on importance sampling fail, and we turn to complex Langevin
simulations. Because these require using a non-compact extension of the gauge
group, runaway solutions are common, and these caused earlier use of this
method to fail. Recent advances indicate that at least some of these problems
were associated with large gauge excursions from well behaved gauge
configurations, and can be circumvented by judicious choice of gauge
(gauge-cooling) after each field update. To date the results look promising.
Project description: We perform simulations to evaluate the functional integrals of QCD-like
theories formulated on a discrete space-time lattice, to enable determination
of the non-perturbative aspects these theories. These include the properties
of these theories at non-zero temperature, including the scales of confinement
and chiral symmetry breaking, and such zero temperature properties as spectra
(including the Higgs mass), decay constants and the running of the gauge
coupling constant. We are particularly interested in those theories where the
coupling constant evolves very slowly, since these are candidate 'Walking-
Technicolor' theories. Related to these are theories with an infrared fixed
point (conformal field theories). Our first goal is to differentiate between
these two different types of behaviour for candidate theories.
We are performing simulations of QCD-like theories which are models for
Walking or Conformal Technicolor. We have been studying theories which are
essentially QCD but with colour-sextet rather than colour-triplet quarks. We
hope to measure the running of the QCD coupling constant. For 2 or 3 flavours,
2-loop perturbation theory predicts an infrared fixed point.
For 2 flavours, it is possible that a chiral condensate forms before this
fixed point is reached. If so, the fixed point is avoided, the theory is
confining, and chiral symmetry breaks spontaneously. However, there is a
region where the coupling constant evolves very slowly. These are the
properties required for a walking technicolor theory. Simulations we have
performed so far at finite temperature suggest that this theory might be
conformal. So far, however, the results are inconclusive.
For 3 flavours we know that the theory should be conformal. Simulations at
N_t=6 (12^3 X 6 lattice) and N_t=8 (12^3 X 8 lattice), some of which used
Fusion, did not yet show evidence of conformality. We are now simulating at
N_t=12 (24^3 X 12 lattice). We started these simulations on Blues, and intend
to continue them in FY2016 on Blues to enable continuity during the period
when Edison at NERSC is unavailable due to a site move.
Our sextet quark codes are based on our earlier triplet quark codes and use
the RHMC simulation method. The Rational Hybrid Monte Carlo (RHMC) is a
stochastic molecular dynamics algorithm. The functional integral of QCD is
written as a partition function of a classical field theory evolving in a
fictitious time. The determinant of the Dirac operator raised to a fractional
power is calculated by introducing bosonic fields (pseudofermions), and
sandwiching this Dirac operator raised to minus said fractional power between
them. This fractional power of the Dirac operator is approximated to machine
accuracy by a rational approximation. After defining this theory on a discrete
space-time lattice, the inversions required by the partial-fraction expansion
of the rational approximation are performed using Krylov space methods, in
particular a multi-shift extension of the conjugate gradient algorithm. A
global Metropolis Monte-Carlo accept/reject step applied at the end of each
trajectory removes discretization errors introduced by the numerical
integration of these stochastic equations of motion. We parallelize the code
by assigning a fixed number of adjacent lattice sites to each MPI task.
Network bandwidth ultimately limits how small a chunk of the lattice can be
assigned to each task.
Our 24^3 X 12 runs will be performed on 288 cores (18 nodes) of Blues.
A short benchmark run of the 24^3 X 12 codes yielded 667 Gflops or 2.3
Gflops/core. Since we have seen performances of > 3 Gflops/core on machines
with similar processors, and > 4 Gflops/node on Edison, we suspect that at
least 3 Gflops/core should be achievable on this machine. Since we have nearly
exhausted our allocation on Blues, we cannot perform more extensive tests.
Scaling tests on Fusion and on Edison at NERSC for numbers of cores from 24 up
to 288 cores indicate that the per core performance increases up to 288 cores,
which we believe to be related to cache usage.
Our new project simulating QCD at finite mu using Complex Langevin with Gauge
Cooling is currently running as serial code on several platforms, including
Blues, where the little allocation we have remaining does not allow us to run
anything else. The Langevin equation method is essentially the limiting case of
the above algorithm in its older form (without the rational approximation),
when each trajectory involves a single update of the fields. Its extension to
the complex manifold merely involves extending the gauge-field manifold from
SU(3) to SL(3,C). The main reason that it is serial code is that this project
is still in its exploratory phase, meaning that modifications of the algorithm
are still possible. Converting to parallel MPI code will not be difficult,
since the compute-intensive kernel is almost identical to that of our older
QCD codes, as are many other parts of the code. We do, however, expect some
performance degradation, since we have been running this code at 64-bit
precision, rather than the 32-bit precision of our QCD codes. When we have
converted to parallel code, we will cease our serial runs on Blues. While we
perform zero-temperature simulations at NERSC, TACC and SDSC, we plan to start
finite temperature simulations on Blues, probably starting on 8^3*4 lattices
using 16 or 32 cores, and then moving to 12^3*4 and 12^3*6 lattices on 48 and
72 cores respectively. Note that these runs with low-level parallelism are
discouraged on the flagship machines at NERSC.
Our requested allocation is based on running 2 288-core jobs half of the time
for the first 4-months and jobs using a total of 144 cores half the time for
the last 8 months. Expected percentage of allocation for serial jobs -- 5%.
Industry partnership:
Project URL:
Current FY Hours Used: undetermined amount
New FY Requested allocation: 1110000
Q1: 500000
Q2: 350000
Q3: 130000
Q4: 130000
Justification: We have not yet performed a detailed scaling analysis on Blues, however, we
have such analyses performed on Fusion as well as Edison at NERSC. For Fusion,
for our QCD with sextet quarks code on a 24^3 X 12 lattice running on Fusion
we observed the following performances:
24 cores = 45 Gflops = 1.9 Gflops/core
48 cores = 95 Gflops = 2.0 Gflops/core
72 cores = 148 Gflops = 2.1 Gflops/core
96 cores = 220 Gflops = 2.3 Gflops/core
144 cores = 369 Gflops = 2.6 Gflops/core
288 cores = 784 Gflops = 2.7 Gflops/core
For the same code and lattice size running on Edison at NERSC we observed the
following performances.
24 cores = 62 Gflops = 2.6 Gflops/core
48 cores = 129 Gflops = 2.7 Gflops/core
72 cores = 214 Gflops = 3.0 Gflops/core
96 cores = 330 Gflops = 3.4 Gflops/core
144 cores = 561 Gflops = 3.9 Gflops/core
288 cores = 1219 Gflops = 4.2 Gflops/core
We use a custom assignment of tasks to nodes in order to minimize
communications. Earlier recoding reduced the number of global reductions in
the routines, which use most of the CPU time, by a factor of 2.
The new Complex Langevin codes are still serial, but they will be parallelized
in the same manner as the RHMC codes. Algorithm changes are under consideration.
There is also the possibility that we will prepare an OpenMP version in
preparation for running on Cori II at NERSC.
Storage requirements: 1 TB
Thank You,
The LCRC Accounts System