[gridengine users] Wrong scheduling behaviour with parallel jobs

Dave Love d.love at liverpool.ac.uk
Mon Feb 28 16:20:58 UTC 2011

Andreas Haupt <andreas.haupt at desy.de> writes:

> What I can see is, the scheduler just considers one PE (pax6) although
> there are 8 PE available (pax1-pax8). I need to shortly explain our
> setup: There are 8 blade chassis containing 128 slots each. The jobs
> need to run on one single blade chassis only. That's why I created one
> PE (pax[1-8]) for each chassis and associated each PE to a single queue.

Why separate queues?  We have islands of fabric, and specify the PEs by
hostgroup in a single queue (except that we have a `long' queue too,
which I'm not sure is actually necessary).

> Our users submit their jobs with '-pe pax?' so that every chassis is
> considered for job execution. That works so far ...

[For what it's worth, we re-write a PE like `openmpi' to `openmpi-*' for
a bit of user convenience, using JSV.]

> Is anybody able to tell me why the scheduler just considers the PE pax6
> for reservation? All the waiting jobs seem to be serialized on it ...

No, but with the setup above, we do get reservations on different PEs,
and for both the normal and long parallel queues (I'm fairly sure, but
couldn't check currently).  It's just that they get lost if we leave the
qmaster running...  Actually, there was another possibly-relevant
problem:  I saw a reservation made cross-PEs on one occasion.

More information about the users mailing list