[gridengine users] Docker optimizsed scheduling policy
reuti at staff.uni-marburg.de
Mon Sep 15 11:38:05 UTC 2014
Am 15.09.2014 um 11:37 schrieb Paolo Di Tommaso:
> Dear all,
> I'm wondering if it's possible to implement a scheduling policy in a SGE cluster that optimise the execution of jobs running Docker containers, in such a way that jobs are allocated in the nodes where a docker image, specified by the user, has already been downloaded.
> To clarify the usage scenario, take in consideration this example:
> * the docker daemon is installed in each cluster node;
> * the user can submit one (or more jobs) that execute a *docker run* command that requires a docker image X
> * when a job starts, docker pull automatically the image X if it has not yet been downloaded by a previously executed job.
> Since a Docker image requires some time (and bandwidth) to be downloaded I would like the scheduler tries to allocate that job where the image X has already been downloaded.
Time spend in a prolog/epilog does not count for the job execution time.
> If a node with that image does not exist, a low priority is associated to that job, so that it has a chance to be executed at some time on any node.
> Ideally, when a node which the image X become available, the job priority is updated so that it can be executed as soon as possible.
This sounds like a request for a BOOL complex with a set urgency. If set to true for a particular exechost, the image is available (some routine on the exechost has to scan what images are available and adjust the BOOL complexes for each image). You could phrase the availability as a soft request in the `qsub` command. A soft request does not take the urgency into account.
The effect would be, if requested as a hard request it's more on top of the waiting jobs, but needs an exechost where the image is already present. If requested as a soft request, it might run anywhere but has to wait longer.
But it also won't change the urgency once it could be used as a hard request. Sure something that might be worth to be implemented though. For now you can only use an external routine checking what images are available and adjust the -soft/-hard request for all pending jobs according to this.
More information about the users