[gridengine users] Parallel Environment

Rayson Ho rayson at scalablelogic.com
Tue Jun 5 02:29:36 UTC 2012

On Mon, Jun 4, 2012 at 9:08 PM, Joseph Farran <jfarran at uci.edu> wrote:
> Telling OGE to use the "mpich" Parallel Environment requesting 64 slots
> (cores).
> Is there a way of having a generic/default PE environment so one can simple
> say:
>    #$ -pe 64

You can do something similar by defining a generic PE and you can then
use a generic name.

Keep in mind that a PE is a somewhat overloaded interface, and it
defines a lot of things:


So if for example the mpich PE and a Hadoop PE both are tight (ie.
OGS/GE can control the slave tasks, perform job control & accounting,
etc), then you can in theory use a common PE for both. And while we
are on this topic, if you want a tight PE, you will need to use qrsh
to invoke remote tasks.

Also, a generic "threaded" PE can be used for OpenMP applications,
Intel TBB programs, & user-threaded applications as the PE definition
are likely to be very similar.

> In our environment, there are *many* users with their own parallel type of
> programs and I like to have a generic PE being the default, but I don't know
> if you can get away with not specifying a PE name?

I haven't tried it myself, but looks like you can use a JSV or a qsub
wrapper for it if you think you really don't want to specify a PE:


But I would propably tell my users that it is a required parameter and
thus tell them not to be lazy!

> One other question.   We have a cluster here running sge6.2 and it has a PE
> for mpich with "Allocation Rule" set to  "Round Robin".   Should this not be
> set to "Fill Up" to fill up all cores on the first node before continuing
> with the next next node?     I am trying to understand if this was an
> oversight when it was setup or if there is a reason for this?

It depends on the MPI application - some applications perform better
when some MPI tasks can talk to their peers via shared memory, while
other MPI applications are better off when they can use more memory
per node. So you may want to talk to the person who set up the cluster
why he did it that way.

BTW, we also have an archive of scheduler related blog entries at:


> Thanks,
> Joseph
> _______________________________________________
> users mailing list
> users at gridengine.org
> https://gridengine.org/mailman/listinfo/users

More information about the users mailing list