As a developer of network middleware, part of my interest in the KNL nodes at JLSE is the Omni-Path network.
I had been successfully using both pshm2 and librabric (psm2 provider) on the KNLs.
However, sometime between Oct 11 and Dec 3 things ceased to work as they had.
Those are the dates of the latest successful run and earliest fail run.
Sorry that I don't have anything between those dates.
Both psm2 and libfabric based programs are now failing with messages like:
knl08.ftm.alcf.anl.gov.223177PSM found 0 available contexts on opa device(s). (err=21)
Additionally, /usr/sbin/opainfo says:
{hargrove@knl04 ~}$ /usr/sbin/opainfo
oib_utils ERROR: [23478] oib_open_port: can't open UMAD port (hfi1_0:1)
opainfo: Unable to open hfi:port 0:1
(Though, in all honesty, I don't know if opainfo ever worked before).
-Paul
--
Computer Languages & Systems Software (CLaSS) Group
Computer Science Department Tel: +1-510-495-2352
Lawrence Berkeley National Laboratory Fax: +1-510-486-6900