[gridengine users] Hybrid PE
hans at MPA-Garching.MPG.DE
Fri Jul 22 13:10:34 UTC 2016
this is with SoGE 8.1.8 and OpenMPI 1.10.2. Our cluster has 24 nodes
with 20 cores each (40 with HT).
I am looking for information how to setup a PE for hybrid jobs (OpenMP
jobs running on more than one node). I created a PE with
"allocation_rule 20", and this seems to work fine with
OMP_NUM_THREADS=20. Then a user wanted to run a job with
OMP_NUM_THREADS=5 and used "mpirun -n $((NSLOTS/OMP_NUM_THREADS))", but
this does not work correctly, on some nodes there are running 20*5=100
threads, on some nodes there are no running threads. Creating a new PE
with "allocation_rule 5" has the disadvantage, that one cannot use more
than 24*5=120 slots.
-------------- next part --------------
A non-text attachment was scrubbed...
Size: 4927 bytes
Desc: S/MIME Cryptographic Signature
More information about the users