Hello,
I’m working on someone else’s cluster and the mpi is not well setup (i.e. not working).
I compiled my own mpich2 using
./configure --with-pmi=slurm --with-pm=no --prefix=/home/biddisco/apps/mpich2-1.5 --enable-shared --disable-static
And a simple hello world executed using srun, works as expected and I get the ranks from all nodes responding correctly.
When I used mpiexec previously, I ran an MPMD command like this
mpiexec -n 4 -env
DISPLAY :0.0 /home/biddisco/build/pv-38/bin/pvserver -rc -ch=128.178.187.20 -sp=11111 : -n 4 -env
DISPLAY :0.1 /home/biddisco/build/pv-38/bin/pvserver -rc -ch=128.178.187.20 -sp=11111 : -n 4 -env
DISPLAY :0.2 /home/biddisco/build/pv-38/bin/pvserver -rc -ch=128.178.187.20 -sp=11111
I need each of the 3 sets of mpi ranks to have a different DISPLAY environment variable to correctly utilise the GPUs.
Unfortunately, the mpiexec on the machine does not work with slurm and I’d like to use
srun –n XXX –env XXX /program : –n XXX –env YYY /program ...
etc etc
but srun doesn’t have an option to pass command line environment vars to the mpi processes. I just can’t find a way of launching an MPMD job using srun with different env vars.
Is there another way?
Thanks
JB
--
John Biddiscombe, email:biddisco @.at.@ cscs.ch
CSCS, Swiss National Supercomputing Centre | Tel: +41 (91) 610.82.07
Via Trevano 131, 6900 Lugano, Switzerland | Fax:
+41 (91) 610.82.82