The new SDK release is now live. The module files for this release are:
oneapi/release/2022.01.30.001
oneapi/eng-compiler/2022.01.30.002
--
Christopher Chan-Nui
christopher.chan-nui(a)intel.com
At 3pm CDT the new version of the Aurora SDK will be released. This will cause
a service disruption and running jobs will be affected.
The module files for this release are:
oneapi/release/2022.01.30.001
oneapi/eng-compiler/2022.01.30.002
Additionally as time progresses, the "latest" module files will track the most
recent, not fully tested, intel_compute_runtime driver.
oneapi/release/latest
oneapi/eng-compiler/latest
From this release forward the login nodes will move from using the public
intel_compute_runtime to the
agama prerelease driver.
MPI wrapper:
To run one MPI rank per tile, please use gpu_tile_compact.sh wrapper script.
This is a Argonne-made workaround until mpirun supports it natively, and it’s
not packaged by oneAPI SDK. The script is loaded by default by the oneapi
modulefile. It implements a compact algorithm to distribute tiles to the rank
(Rank 0, GPU 0, Tile 0. Rank 1, GPU 0, Tile 1. Rank 2, GPU 1, Tile 0, .etc) .
To use:
mpirun -n X gpu_tile_compact.sh a.out (edited)
OneAPI release notes are available the following link:
https://wiki.jlse.anl.gov/display/inteldga/DGA+SDK+Release+Notes
Please find the following changes that will propagate through the Arcticus nodes later today:
* Intel Compute Runtime (UMD) module structure
** Descriptive tagging of the runtime driver has been setup to ease switching to different versions
One can use `ml switch -f intel_compute_runtime intel_compute_runtime/release/<TAG>`
sdk-default - Refers to the default runtime loaded when one would use
oneapi module at the point where testset results are generated
latest - Refers to the newest runtime available to use but not
necessarily tested to work with all parts of the SDK
testing - Refers to the runtime currently being installed and/or
under testing
match-dgpu-kmd - Refers to the runtime version that was packaged together
with the KMD installed in the node image
* All the arcticus/arcticus_debug queue nodes would be updated to the latest
driver stack from agama-prerelease-316. This will bring about an user impact
change because the i915 KMD used in this release has known issues when used
with much older runtime as in the current SDK which is agama-prerelease-191.
Therefore once the image update process is completed today, results generated
from the older SDK may not be reproducible.
However moving to the newest runtime stack enables extensive set of feature
upgrades that brings about improvements in both stability and performance. In
addition enables several new features needed for the different components of
the SDK.
* arcticus_debug queue in addition has a DCD kernel module enabled for the time
being for evaluating the oneapi debugger.
Currently there are known issues affecting functionality and we are working
on traiging and fixing the issues. So expect an update at a later time when
this is available for testing by the users.
Since this is a known variation from the main arcticus queue, please be aware
that this may have an affect on workloads evaluated on this node.
* implicit scaling functionality:
agama-prerelease-303 driver stack onwards `EnableImplicitScaling` is set to
enabled and `EnableWalkerPartition` is deprecated. If you are setting either of
these manually or in your scripts please remove them.
Users shouldn’t necessarily need to deal with setting/unsetting this env for
running workloads on the GPU. The default behavior for workloads would be to
run on both the tiles implicitly.
If you would like to target only a single tile or a particular device, please
use `ZE_AFFINITY_MASK` accordingly. `EnableImplicitScaling` option is designed
in such a manner that if the driver recognizes the `ZE_AFFINITY_MASK` is
targeting a single tile, then internally implicit scaling is disabled. Should
your usecase contradict this behavior, please let us know and we shall file a
bug regarding this.
* Additional environment options:
If you were advised to use additional debug variables or environment options
to fine tune the driver stack it is recommended to remove them and check if
this is needed. There has been several cases were knowingly/unknowingly
setting such options affect the overall stability/performance of the code in
the GPU.
Please report to the mailing list or reach out to <servesh(a)anl.gov> should
there be conflicting info regarding this.
--
Christopher Chan-Nui
christopher.chan-nui(a)intel.com
