[gridengine users] Core Binding with Intel Hyper-threaded Hosts

Callum Lewis-Smith callum.lewis-smith at quoratetechnology.com
Tue Jan 27 17:46:21 UTC 2015


I have recently been trying to configure my cluster such that I may
restrict jobs to use a set number of compute cores.

>From reading the documentation it is my understanding the simplest way
of achieving this is through adding the -binding option when submitting
jobs using qsub.

Having played around with this option a bit I notice that binding is
done only at a core/socket level and doesn't seem to take into account
the multiple threads available to each core on a Intel hyper-threaded node.

As such if I submit a job in the following manner:

qsub -binding linear:1 job.sh

The script job.sh may use 100% CPU on each of the core's two threads.

I understand that the -binding option is implemented using the hwloc
package in sge_shepherd to bind the job based on the given configuration.

Using the hwloc package through the OS directly I seem to be able to get
processes on my hyper-threaded nodes to run on a single thread of a
single core but I don't see any way of achieving similar behaviour
through the grid.

I notice Univa Grid Engine supports the scheduler configuration
parameter COUNT_CORES_AS_THREADS=1 which allows it to treat threads as
discrete cores, when carrying out core binding on Intel hyper-threaded
machines, and was wondering if anyone knew of a way of achieving similar
behaviour through Open Grid Scheduler?

Thanks,
Callum



More information about the users mailing list