[gridengine users] Core Binding with Intel Hyper-threaded Hosts
callum.lewis-smith at quoratetechnology.com
Tue Jan 27 17:46:21 UTC 2015
I have recently been trying to configure my cluster such that I may
restrict jobs to use a set number of compute cores.
>From reading the documentation it is my understanding the simplest way
of achieving this is through adding the -binding option when submitting
jobs using qsub.
Having played around with this option a bit I notice that binding is
done only at a core/socket level and doesn't seem to take into account
the multiple threads available to each core on a Intel hyper-threaded node.
As such if I submit a job in the following manner:
qsub -binding linear:1 job.sh
The script job.sh may use 100% CPU on each of the core's two threads.
I understand that the -binding option is implemented using the hwloc
package in sge_shepherd to bind the job based on the given configuration.
Using the hwloc package through the OS directly I seem to be able to get
processes on my hyper-threaded nodes to run on a single thread of a
single core but I don't see any way of achieving similar behaviour
through the grid.
I notice Univa Grid Engine supports the scheduler configuration
parameter COUNT_CORES_AS_THREADS=1 which allows it to treat threads as
discrete cores, when carrying out core binding on Intel hyper-threaded
machines, and was wondering if anyone knew of a way of achieving similar
behaviour through Open Grid Scheduler?
More information about the users