Yes, that did the trick. I tried that last night but hadn't gotten around to giving an update here yet. Thanks, Wyatt On Fri, Jul 30, 2021 at 8:58 AM Willmore, Frank Ted <[email protected]> wrote:

Hi Wyatt--

...

26 /usr/bin/ld: warning: libze_loader.so.1, needed by

/soft/restricted/CNDA/mpich/drop41.1/mpich-ofi-sockets-icc-default-ats-drop41/lib/libmpi.so, not found (try using -rpath or -rpath-link)

...

...

...

27 /usr/bin/ld: /soft/restricted/CNDA/mpich/drop41.1/mpich-ofi-sockets-icc-default-ats-drop41/lib/libmpi.so: undefined reference to `zeMemGetAllocProperties' ) I have the spack oneapi compiler definition set to load the oneapi module, but no others.

You might try adding the needed path to the 'environment' section of the compiler stanza.

...

Outside of spack I get undefined ze symbols from nm on libmpi.so: nm /soft/restricted/CNDA/mpich/drop41.1/mpich-ofi-sockets-icc-default-ats-drop41/lib/libmpi.so | grep zeDeviceGet U zeDeviceGet U zeDeviceGetCommandQueueGroupProperties U zeDeviceGetProperties U zeDeviceGetSubDevices

This is normal, it just means that these symbols are needed and expected to be defined elsewhere, in this case, as part of level zero.

...

These are defined in: /soft/libraries/intel-level-zero/api_+_loader/v1.2.3-Release-2021.06.01/lib64/libze_loader.so.1 But I'm not sure why the spack build environment would be unable to find that. The directory seems > to be loaded into the LD_LIBRARY_PATH when the oneapi module is loaded.

Yes, this appears to be the case. You might verify that this path is indeed present by examining the build environment (spack build-env) in which case there may be something deeper going on.

Adding explicitly to your LD_LIBRARY_PATH in the compiler stanza may be your best bet.

Cheers,

--Frank

Frank Willmore, Ph.D. ALCF Operations Argonne National Laboratory [email protected]

------------------------------ *From:* Ecp-aurora-sdk-discuss < [email protected]> on behalf of Wyatt Spear via Ecp-aurora-sdk-discuss < [email protected]> *Sent:* Wednesday, July 28, 2021 3:29 PM *To:* [email protected] < [email protected]> *Subject:* [Ecp-aurora-sdk-discuss] Undefined ze* symbols using oneapi's mpi with spack

Greetings,

I am not sure how to access the archive for this list so please forgive me if this is a known issue (or user error for that matter).

I'm seeing this with the default oneapi module on yarrow/arcticus (oneapi/2021.04.30.004), using spack to build upcxx (and similar errors for a few other packages) for E4S environment testing.

576 /usr/bin/ld: warning: libze_loader.so.1, needed by /soft/restricted/CNDA/mpich/drop41.1/mpich-ofi-sockets-icc-default-ats-drop41/lib/libmpicxx.so, not found (try using -rpath or -rpath-link)

...

...

577 /usr/bin/ld: /soft/restricted/CNDA/mpich/drop41.1/mpich-ofi-sockets-icc-default-ats-drop41/lib/libmpi.so: undefined reference to `zeMemGetAllocProperties' ...numerous other symbols...

(bonus example from hdf5:

26 /usr/bin/ld: warning: libze_loader.so.1, needed by /soft/restricted/CNDA/mpich/drop41.1/mpich-ofi-sockets-icc-default-ats-drop41/lib/libmpi.so, not found (try using -rpath or -rpath-link)

...

...

27 /usr/bin/ld: /soft/restricted/CNDA/mpich/drop41.1/mpich-ofi-sockets-icc-default-ats-drop41/lib/libmpi.so: undefined reference to `zeMemGetAllocProperties' )

I have the spack oneapi compiler definition set to load the oneapi module, but no others.

Outside of spack I get undefined ze symbols from nm on libmpi.so: nm /soft/restricted/CNDA/mpich/drop41.1/mpich-ofi-sockets-icc-default-ats-drop41/lib/libmpi.so | grep zeDeviceGet U zeDeviceGet U zeDeviceGetCommandQueueGroupProperties U zeDeviceGetProperties U zeDeviceGetSubDevices

But maybe that is expected?

These are defined in: /soft/libraries/intel-level-zero/api_+_loader/v1.2.3-Release-2021.06.01/lib64/libze_loader.so.1 But I'm not sure why the spack build environment would be unable to find that. The directory seems to be loaded into the LD_LIBRARY_PATH when the oneapi module is loaded.

Thanks for any advice.

Wyatt Spear