As a developer of network middleware, part of my interest in the KNL nodes at JLSE is the Omni-Path network. I had been successfully using both pshm2 and librabric (psm2 provider) on the KNLs. However, sometime between Oct 11 and Dec 3 things ceased to work as they had. Those are the dates of the latest successful run and earliest fail run. Sorry that I don't have anything between those dates. Both psm2 and libfabric based programs are now failing with messages like: knl08.ftm.alcf.anl.gov.223177PSM found 0 available contexts on opa device(s). (err=21) Additionally, /usr/sbin/opainfo says: {hargrove@knl04 ~}$ /usr/sbin/opainfo oib_utils ERROR: [23478] oib_open_port: can't open UMAD port (hfi1_0:1) opainfo: Unable to open hfi:port 0:1 (Though, in all honesty, I don't know if opainfo ever worked before). -Paul -- Paul H. Hargrove [email protected] Computer Languages & Systems Software (CLaSS) Group Computer Science Department Tel: +1-510-495-2352 Lawrence Berkeley National Laboratory Fax: +1-510-486-6900