[gridengine users] Subordinate Queues
jfarran at uci.edu
Fri Jun 15 15:18:20 UTC 2012
I am playing with OGE subordinate Queues and I can't seem to get it right.
All my nodes are 64 cores and I set all my nodes to node pack jobs with:
qconf -rattr exechost complex_values "slots=64" node1 ( repeat for all other nodes )
The scheduler is then set with Load Formula to "slots". So up to 64 serial jobs get packed unto one node and after that, the 65th+ goes to the next node until it reaches 64 cores and so on. So each node will only run a max of 64 slots.
I have queue Q1 which points to node1 & node2, and queue Q2 which points to nodes3 & node4.
Q1 --> ( node1 node2 )
Q2 --> ( node3 node4 )
Now I like to have a subordinate Queue called SUB-Q which is subordinate to Q1 & Q2. So what I am trying to make it do is as follows:
If jobs ( serial or parallel ) are submitted to SUB-Q, it will use nodes from Q1 or Q2 ( node1 through node4 ). If jobs are submitted to Q1 or Q2, it will suspend any jobs that were submitted from SUB-Q.
Here is a scenario:
1 64-core parallel job is submitted to SUB-Q and the scheduler picks node1.
1 64-core parallel job is submitted to SUB-Q and the scheduler picks node3.
64 single-core jobs are submitted to SUB -Q and the scheduler picks node2.
64 single-core jobs are submitted to SUB -Q and the scheduler picks node4.
So now all nodes are full with the following jobs:
(job 1) Node1 running 64-cores mpi job.
(job 2) Node3 running 64-cores mpi job.
(job 3) Node2 running 64 serial jobs.
(job 4) Node4 running 64 serial jobs.
If a new 128 core mpi job jobs is submitted to Q1, have the scheduler suspend job #1 & #3 (node1 & node2) and then run the new 128 core job.
If 64 new serial jobs are submitted to Q2, have the scheduler suspend either jobs #2 or #4.
I have a few other questions, but this is a good start.
If possible please provide the OGE command lines to set this up as I am still a newbie with OGE.
More information about the users