Hello This might be caused by hwloc. hwloc links with libXNVCtrl when available (because some people wants hwloc to expose the locality a NVIDIA X11 displays). OpenMPI disables some hwloc backends by passing things like enable_gl=no before invoking hwloc's configury: https://github.com/open-mpi/ompi/blob/master/opal/mca/hwloc/hwloc201/configu... enable_gl=no is the one that matters here, but others could be disabled in MPICH unless the corresponding objects are explicitly used in MPICH. Brice Le 12/02/2019 à 22:36, Martin Cuma via discuss a écrit :

Hi everyone,

I was wondering if someone could give me a feedback on a couple of quirks I found with MPICH 3.3, that I did not observe with 3.2.1.

1. If CUDA is found on a machine where MPICH is built, /lib64/libXNVCtrl.so.0 gets included in the MPICH library (libmpi.so). I tried the --without-x or --with-x=no options but it does not affect this. If I build on a machine that does not have CUDA, /lib64/libXNVCtrl.so.0 is not included and the library seems to work the same (on typical MPI applications).

This is annoying if we want to have a common MPICH for machines with and without CUDA installed, e.g. compute node with and without GPUs.

2. It seems that MPICH now requires the --with-slurm option to pick up the SLURM hostlist (= run w/o the -machinefile option). This was not the case with 3.2.1, where mpich picked up the SLURM hostlist without needing to be built with --with-slurm. The path to SLURM also gets encoded in the RPATH of mpirun and friends. I'd rather control that myself if possible.

This is an inconvenience for us as well since we have different clusters with SLURMs on different file systems/paths, and I'd like to also use the same MPICH build on desktops that don't have SLURM at all. I could put $ORIGIN to RPATH and stick that libslurm.so to the MPICH bin directory, since we tend to run the same SLURM version on all the clusters, but, still, it's clunky.

Again, appreciate if someone could put some light to this or give some good workarounds.

Thanks, MC