[gridengine users] sequential jobs on cluster

Reuti reuti at staff.uni-marburg.de
Fri May 23 21:47:58 UTC 2014


Please keep the list posted.

Am 23.05.2014 um 23:37 schrieb Feng Zhang:

> Thanks, Reuti!
> 
> Your method will work for sure. I am just wondering if SGE's scheduler
> policy can be set to do so. I know that for PE jobs, the PE env can be
> set to distribute processes in a round robin way.

In principle yes. Either:

- add an artificial load by setting "job_load_adjustments"/"load_adjustment_decay_time" in a schedule to increase "np_load_avg"

- Use the least used host: http://wiki.gridengine.info/wiki/index.php/StephansBlog

But matching your original intention is only correctly mapped by the consumable. The other two ways may fail.

-- Reuti


> On Fri, May 23, 2014 at 5:31 PM, Reuti <reuti at staff.uni-marburg.de> wrote:
>> Hi,
>> 
>> Am 23.05.2014 um 23:14 schrieb Feng Zhang:
>> 
>>> I am now running large disk IO jobs(sequential jobs NOT PE jobs) on my
>>> cluster. Is there any way I can submit my jobs evenly to all the
>>> nodes? The problem I have now is:
>>> 
>>> I have 10 jobs, and I have 10 nodes, and each node has 10 CPU cores.
>>> When I submit my jobs, all the 10 jobs are distributed to one node,
>>> and the other nodes are idling(almost). How can I run 1 job on 1 node
>>> instead?
>> 
>> By using a consumable complex like "bigio" and attach a "complex_value" of one to each exechost for it. Then request this also during job submission.
>> 
>> -- Reuti





More information about the users mailing list