[gridengine users] usage of make -j

Christopher Heiny cheiny at synaptics.com
Tue Mar 5 20:40:03 UTC 2013


On 03/05/2013 10:35 AM, Reuti wrote:
> Hi,
>
> Am 05.03.2013 um 18:41 schrieb Vamsi Krishna:
>
>> >is there any way to run make -j <slots in node> when we submit to batch environment to use all the slots with -pe <slots in node>
> please have a look at the `qmake` tool which is installed with SGE in case you want to start the compilation on the command line (`man qmake`).
>
> If you are submitting the complete `make` job to a batch queue already, you could use:
>
> make -j $NSLOTS
>
> in your jobscript and it will be replaced at runtime with the requested number of slots as you specified it as argument to `qsub -pe smp 4` or alike. If you want to use always all cores in a machine:
>
>   qsub -l exclusive -pe smp -999 make.sh
>
> will always use the maximum up to 999 cores in a machine ("exclusive" is a complex being set up with "relop EXCL", "smp" a PE setup with "allocation_rule $pe_slots") and set $NSLOTS accordingly at runtime.

Hmmm.  That's an interesting trick.

Our make invocations are already embedded in lengthy Bash scripts 
submitted to GE.  We use on slot per machine and one slot per job to 
ensure exclusive access (within that queue) to a given machine.  Then we 
invoke make from the script like this:

     cpus=`grep processor /proc/cpuinfo | wc -l`
     make -j ${cpus}  .......

						Chris



More information about the users mailing list