[gridengine users] suggestions on setting up queues
dag at sonsorol.org
Fri Jan 16 20:51:23 UTC 2015
Queues are just a piece of the puzzle when it comes to handling resource
allocation on a multi user system, what (if any) scheduling policies
and resource quotas are you currently using?
That said you are using the queue methods in a good way. There are
certain things that can only be really done on a per-queue basis and top
of the list would be ACL protection and the ability to impose hard or
soft wallclock limits.
A fairshare-by-user policy with the queue structure you set up would be
a decent starting point from which you can gather more data and user
- resource quota would perfectly handle the "only N jobs per user can
run in the long-job.q cluster queue ..."
- I've had little success putting wallclock limits on interactive
queues; there are legit business/scientific reasons in many cases for a
long running interactive session. You might want to poll the users or
collect data on this. In a few different environments I've had decent
success by leaving interactive queue slots unrestricted but putting a
resource quota around how many slots a single user can consume. It's
also pretty easy to set up tools that would allow you to dynamically
adjust the size/count of the interactive slot pool to account for
changing demand - it's particularly easy when used with SGE hostgroup
> Stephen Spencer <mailto:spencer at cs.washington.edu>
> January 16, 2015 at 2:50 PM
> Good morning.
> With the number of users on our clusters growing, it's becoming less
> realistic to say "play fair 'cause you're not the only user of the
> I'm looking for suggestions on setting up queues, both the "why" and
> "how," that will allow more of our users access to the cluster.
> What I'm thinking of is a multi-queue approach:
> * some limited number of "interactive" slots (and they'd be
> * a queue for jobs with short time duration - the "express" queue
> * a queue for jobs that will run longer... but only so many of these
> per user
> Any and all suggestions are welcome.
> Thank you!
> Stephen Spencer
> spencer at cs.washington.edu <mailto:spencer at cs.washington.edu>
> users mailing list
> users at gridengine.org
More information about the users