[gridengine users] How to set a environment variable on slave nodes?

Reuti reuti at staff.uni-marburg.de
Wed Dec 4 12:51:57 UTC 2013


Am 04.12.2013 um 12:34 schrieb Yong Wu:

> Dear Reuti,
> 
> The part of submitted script:
> #----start----
> #$ -pe mpi4 8
> #$ -l h="compute-0-27|compute-0-30"
> #$ -q all.q
> #$ -v LD_PRELOAD=/home/wuy/bin/libtime.so.1
> #----end----
> 
> $ qconf -sp mpi4
> pe_name            mpi4
> slots              9999
> user_lists         NONE
> xuser_lists        NONE
> start_proc_args    /opt/gridengine/mpi/startmpi.sh $pe_hostfile
> stop_proc_args     /opt/gridengine/mpi/stopmpi.sh
> allocation_rule    4
> control_slaves     FALSE
> job_is_first_task  TRUE
> urgency_slots      min
> accounting_summary TRUE
> 
> The master node is compute-0-27, while the slave node is compute-0-30. The variable "LD_PRELOAD" can only take effect on master node (compute-0-27). That is, the library shouldn't be loaded at  slave node (compute-0-30).

Aha, you mean master and slave of a parallel job.

What MPI library do you use? The easiest might be: don't call your binary directly but a wrapper. Depending on the rank you can either define the LD_PRELOAD or not. Some MPI libraries set an environment variable already like OMPI_COMM_WORLD_RANK (Open MPI) which can be checked:

#!/bin/bash
if [ "$OMPI_COMM_WORLD_RANK" -eq 0 ]; then
    export LD_PRELOAD=/foo/bar/baz
fi
exec "${@}"

and you can execute:

mipexec wrapper.sh real-binary arg1 arg2 ...

in your jobscript.

HTH -- Reuti



> I know and I tested you said. Although the env_list: LD_PRELOAD=/home/wuy/bin/libtime.so.1, the library shouldn't be loaded at  slave node (compute-0-30).
> 
> Thanks!
> 
> Yong WU
> 
> 
> 2013/12/4 Reuti <reuti at staff.uni-marburg.de>
> Hi,
> 
> Am 04.12.2013 um 11:31 schrieb Yong Wu:
> 
> >   I want to set a environment variable on slave nodes, and I did this in my submitted script:
> > #$ -v LD_PRELOAD=/home/wuy/bin/libtime.so.1
> 
> The variable will bet set at execution time of the script, wherever it's executed. Due you mean:
> 
> - the location is different for slave nodes in the cluster?
> - the library shouldn't be loaded at all on slave nodes?
> 
> Did you test with an:
> 
> echo $LD_PRELOAD
> 
> in the jobscript whether it's set? It will also be listed in `qstat -j <job_id>`.
> 
> -- Reuti
> 
> 
> >   But this variable can only take effect on master node. How to set a environment variable on slave nodes?
> >
> >   Any help about this issue will be apreciated!
> >
> > Thanks!
> >
> > Yong WU
> >
> > _______________________________________________
> > users mailing list
> > users at gridengine.org
> > https://gridengine.org/mailman/listinfo/users
> 
> 





More information about the users mailing list