[gridengine users] Effects of login_shells settings

Cook, Malcolm MEC at stowers.org
Wed Oct 31 19:53:49 UTC 2018


I think the quickest resolution to the origin of the errors you were first experiencing is not to add bash to the list of login_shells

Rather, have your users submit jobs from a shell environment in which no functions have been exported.

They are probably submitting from a bash shell using qsub’s `-V` option to which strives to recreate the entire environment , and then experiencing  this old SGE issue

Error with "export -f" shell (bash) functions<https://users.gridengine.sunsource.narkive.com/VQnqjycW/error-with-export-f-shell-bash-functions>

That said, if they are really wanting to be able to submit qsubs from bash environments and have bash functions work on the far side, then they are probably out of luck unless you either

  1.  find that this bug is fixed in some later version of SGE
  2.  contrive a workaround to use some other means than `-V` to re-establish the functions in the environment.
  3.  Ensure that all your functions are one-liners

However, I don’t think 3) is even possible due to how bash function exports apparently work.  The following shows the creation of the muli-line variable which trips up the implementation of SGE  -V

$ function test () { echo "$@" ; }
$ export -f test
$ env | grep -A1 -i bash_func_test
BASH_FUNC_test()=() {  echo "$@"
}

So, instead, you need approach (2), which is to put the desired functions in some .profile and have the qsubbed job source that .profile in the qsubbed environment.



From: users-bounces at gridengine.org <users-bounces at gridengine.org> On Behalf Of Ilya M
Sent: Wednesday, October 31, 2018 1:48 PM
To: users at gridengine.org
Subject: [gridengine users] Effects of login_shells settings

**CAUTION: Non-Stowers email**
Hello,

I had an unexpected effect after adding bash to the login_shells list in SGE 6.2.u5. Some array tasks started to get scheduled twice on the same node (accounting info corroborates that) and 1-2 tasks in a job would not produce any output at all, as if they did not run. This quickly became a wide-spread problem in the cluster and I had to revert the change by removing bash from the list.

I am trying to understand the cause of this behavior. I did not find anything in the logs. The qmaster logs and accounting only showed that some tasks were scheduled twice, running for the same number of seconds and both times completing successfully. Was this an uncovered bug?

The rationale for adding bash to the list, were the errors some users reported when trying to launch their jobs in bash:

/bin/bash: SPEcho: line 1: syntax error: unexpected end of file
/bin/bash: error importing function definition for `BASH_FUNC_SPEcho'
/bin/bash: SPCheckFile: line 1: syntax error: unexpected end of file
/bin/bash: error importing function definition for `BASH_FUNC_SPCheckFile'
/bin/bash: SPNotify: line 1: syntax error: unexpected end of file
/bin/bash: error importing function definition for `BASH_FUNC_SPNotify'
/bin/bash: SPCheckFileNonEmpty: line 1: syntax error: unexpected end of file
/bin/bash: error importing function definition for `BASH_FUNC_SPCheckFileNonEmpty'
/bin/bash: SPCheckExec: line 1: syntax error: unexpected end of file
/bin/bash: error importing function definition for `BASH_FUNC_SPCheckExec'
/bin/bash: SPLogError: line 1: syntax error: unexpected end of file
/bin/bash: error importing function definition for `BASH_FUNC_SPLogError'
/bin/bash: SPCkptDone: line 1: syntax error: unexpected end of file
/bin/bash: error importing function definition for `BASH_FUNC_SPCkptDone'
/bin/bash: SPCheckFunc: line 1: syntax error: unexpected end of file
/bin/bash: error importing function definition for `BASH_FUNC_SPCheckFunc'
/bin/bash: SPCkptShouldRun: line 1: syntax error: unexpected end of file
/bin/bash: error importing function definition for `BASH_FUNC_SPCkptShouldRun'
/bin/bash: SPCheckVar: line 1: syntax error: unexpected end of file
/bin/bash: error importing function definition for `BASH_FUNC_SPCheckVar'
/bin/bash: SPCheckDir: line 1: syntax error: unexpected end of file
/bin/bash: error importing function definition for `BASH_FUNC_SPCheckDir'
/bin/bash: SPLogInfo: line 1: syntax error: unexpected end of file
/bin/bash: error importing function definition for `BASH_FUNC_SPLogInfo'
/bin/bash: SPCleanDir: line 1: syntax error: unexpected end of file
/bin/bash: error importing function definition for `BASH_FUNC_SPCleanDir'
/bin/bash: SPLogWarning: line 1: syntax error: unexpected end of file
/bin/bash: error importing function definition for `BASH_FUNC_SPLogWarning'

And the discussion here: https://sourceforge.net/p/modules/mailman/message/28650889/, suggesting that the cause might be absence of bash in the login_shells list.

Looking forward to some ideas and explanations.

Thank you,
Ilya.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gridengine.org/pipermail/users/attachments/20181031/e885d11a/attachment-0001.html>


More information about the users mailing list