[gridengine users] Consumable configuration best practices question for hundreds of resources for specific group of nodes

William Hay w.hay at ucl.ac.uk
Thu Mar 26 16:55:58 UTC 2015

On Thu, 26 Mar 2015 15:33:06 +0000
Yuri Burmachenko <yuribu at mellanox.com> wrote:

> William,
> thank you for your reply,
> These 4x nodes are connected to some kind of hardware that has 300 cells each can be in use or free.

Ok so far so good.  However from here on down you seem to be describing a partial solution rather than the problem you are trying to solve.  It isn't clear to me if the users care which cells they are using or whether this choice is being forced on them by your partial solution.  

If the cells are interchangable then representing them as a single integer consumable and allocating specific cells when the job starts would be a better approach.  Univa Grid Engine has specific support for this sort of thing(RSMAP).  SoGE doesn't (https://arc.liv.ac.uk/trac/SGE/ticket/1426) but there are ways to fake it.

> We thought that there should be 300 different (varieties) resource consumables in SGE for this GROUP of hosts each can be in use or free (in total 1 unit per resource consumable in total for this particular group of hosts).

Why did you think that?  Are the cells really so different that users have to know which ones they will be allocated when they submit jobs?  If there is any way to avoid treating the different cells as 300 distinct resource types I would do so otherwise the scheduler is likely to do a terrible job of scheduling them (as would almost any scheduler).

> Each user will be able to submit a job which can consume multiple resource consumables of different type.
> For example this group of hosts will have the following consumable resources: 1 res1 1 res2, 1 res3 … 1 res300.
> Any tips on the implementation, will be greatly appreciated.
> Thank You.


William Hay <w.hay at ucl.ac.uk>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 819 bytes
Desc: not available
URL: <http://gridengine.org/pipermail/users/attachments/20150326/5a1ba78c/attachment.sig>

More information about the users mailing list