Task-Core Affinity on Commodity Clusters
Categories: Running Jobs
Print this article
In general, the cluster may move MPI tasks between cores within a node.
To help prevent a job’s tasks from being moved between cores each idle cycle the
mpi_yield_when_idle OpenMPI option may be used. For example:
$ mpirun -n 8 -mca mpi_yield_when_idle 0 a.out
This will help prevent the core from being given to other waiting tasks. This only affects MPI processes when they are blocking in MPI library calls.
By default OpenMPI will set this variable based on whether it believes the node is over-allocated or under-allocated. If over-allocated,
mpi_yield_when_idle, will be set to a value other than (1), allowing the core to be given to other waiting tasks when idle. If under-allocated,
mpi_yield_when_idle, will be set to (0).
If more tasks are running on a node than are cores, the OS will swap all tasks between cores on the node. The
mpi_yield_when_idle option only helps to slow this down; it will not fully prevent the swaps.