[gridengine users] usage of make -j

Vamsi Krishna vkpolisetty at gmail.com
Fri Mar 15 03:51:15 UTC 2013


used the following command to submit two jobs

cwd: /test1, qsub -l exclusive -l hostname=`uname -n` -pe smp 12
batch_test.bash

cwd:/test2, qsub -l exclusive  -l hostname=`uname -n` -pe smp 12
batch_test.bash

only first job is running and other one is in queue wait status. but if i
submit -p smp 24, successfully it is using 24slots in the host. what could
be the reason in waiting for the second job when i use 12 slots.

job-ID  prior   name       user         state submit/start at     queue
                     slots ja-task-ID
-----------------------------------------------------------------------------------------------------------------
1347219 10.35000 batch_test user1    r     03/15/2013 01:21:33
         batch.q at node1 12
1347220 1.60000 batch_test user1     qw    03/15/2013 01:20:36
                      12


On Fri, Mar 8, 2013 at 9:29 PM, Vamsi Krishna <vkpolisetty at gmail.com> wrote:

> Thanks Reuti, Progressing well.
>
>  - V
>
>
>
> On Wed, Mar 6, 2013 at 2:10 AM, Christopher Heiny <cheiny at synaptics.com>wrote:
>
>> On 03/05/2013 10:35 AM, Reuti wrote:
>>
>>> Hi,
>>>
>>> Am 05.03.2013 um 18:41 schrieb Vamsi Krishna:
>>>
>>>  >is there any way to run make -j <slots in node> when we submit to
>>>> batch environment to use all the slots with -pe <slots in node>
>>>>
>>> please have a look at the `qmake` tool which is installed with SGE in
>>> case you want to start the compilation on the command line (`man qmake`).
>>>
>>> If you are submitting the complete `make` job to a batch queue already,
>>> you could use:
>>>
>>> make -j $NSLOTS
>>>
>>> in your jobscript and it will be replaced at runtime with the requested
>>> number of slots as you specified it as argument to `qsub -pe smp 4` or
>>> alike. If you want to use always all cores in a machine:
>>>
>>>   qsub -l exclusive -pe smp -999 make.sh
>>>
>>> will always use the maximum up to 999 cores in a machine ("exclusive" is
>>> a complex being set up with "relop EXCL", "smp" a PE setup with
>>> "allocation_rule $pe_slots") and set $NSLOTS accordingly at runtime.
>>>
>>
>> Hmmm.  That's an interesting trick.
>>
>> Our make invocations are already embedded in lengthy Bash scripts
>> submitted to GE.  We use on slot per machine and one slot per job to ensure
>> exclusive access (within that queue) to a given machine.  Then we invoke
>> make from the script like this:
>>
>>     cpus=`grep processor /proc/cpuinfo | wc -l`
>>     make -j ${cpus}  .......
>>
>>                                                 Chris
>>
>> ______________________________**_________________
>> users mailing list
>> users at gridengine.org
>> https://gridengine.org/**mailman/listinfo/users<https://gridengine.org/mailman/listinfo/users>
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gridengine.org/pipermail/users/attachments/20130315/6f8b01d1/attachment.html>


More information about the users mailing list