KMP_AFFINITY only works for thread affinity though, no? If I have four MPI processes, each of the processes will assign threads to the same cores rather than cores in a different sub-NUMA cluster. I need to be able to specify process affinity and my problem is that 1) numactl can only control a single process and 2) the usual *mpirun -bind-to* and *-map-by* options don't really have enough levels of abstraction to match the KNL's hierarchy. For what it's worth, I tried *mpirun -bind-to numa -map-by numa ...* and that resulted in an error. It's not even clear, to me at least, what to expect if you say 'numa' -- one could interpret it to mean a tile (that shares L2 cache) or a cluster (that shares MCDRAM). On Fri, Apr 22, 2016 at 9:46 AM, Harms, Kevin <[email protected]> wrote:

KMP_AFFINITY=balanced is supposed to do that but if it's not working, it might be that the MPI library does not have proper tuning for KNL yet.

kevin

...

I'm trying to test a code using hybrid MPI/OpenMP on the KNL node but haven't been able to successfully set process affinity. I'd like to run four processes in SNC-4 mode with each process being bound to a cluster. Does anyone know if this is possible?

Thanks, Paul