[gridengine users] suggestions on setting up queues

Prentice Bisbal prentice.bisbal at rutgers.edu
Fri Jan 16 20:56:57 UTC 2015


I'd be careful about setting up too many queues. The more complicated 
you make it,the harder it is for your users to use. I'd start with the 
following. My apologies if you've already done some of these steps:

0. Find some way to monitor your scheduler's behavior, and figure out 
what you want to see happen. Without some kind of goals and metrics, how 
will you know if your changes are working as desired?

1. Require users specify a wallclock time when running jobs. This is 
required for step 2. Don't set a default wallclock time. Configure SGE 
to fail a job immediately if a wallclock time isn't specified. I did 
this a long time ago, but forgot how to this. I believe if you make '-w 
e' a default option for qsub (eg 'qsub -w e ......') jobs that do not 
specify h_rt will fail immediately. This will get your users to remember 
to always set h_rt.

2. Turn on backfill scheduling.

3. Look into fairshare scheduling.

Only after you've take these 3 steps, would I look into making 
additional queues.


On 01/16/2015 02:50 PM, Stephen Spencer wrote:
> Good morning.
> With the number of users on our clusters growing, it's becoming less 
> realistic to say "play fair 'cause you're not the only user of the 
> cluster."
> I'm looking for suggestions on setting up queues, both the "why" and 
> "how," that will allow more of our users access to the cluster.
> What I'm thinking of is a multi-queue approach:
>   * some limited number of "interactive" slots (and they'd be
>     time-limited)
>   * a queue for jobs with short time duration - the "express" queue
>   * a queue for jobs that will run longer... but only so many of these
>     per user
> Any and all suggestions are welcome.
> Thank you!
> Best,
> -- 
> Stephen Spencer
> spencer at cs.washington.edu <mailto:spencer at cs.washington.edu>
> _______________________________________________
> users mailing list
> users at gridengine.org
> https://gridengine.org/mailman/listinfo/users

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gridengine.org/pipermail/users/attachments/20150116/a2112f3d/attachment.html>

More information about the users mailing list