[gridengine users] qlogin + X11 + pam_sge_authorize ?

Alex Chekholko chekh at stanford.edu
Mon Jun 8 18:38:06 UTC 2015

On 06/08/2015 12:49 AM, William Hay wrote:
> On Fri, 5 Jun 2015 22:16:12 +0000
> Alex Chekholko <chekh at stanford.edu> wrote:
>> Hi all,
>> I have a standard grid engine cluster (sge-8.1.8 tarball from Dave
>> Love's site) where users use qlogin to get interactive shells on compute
>> nodes, and we use a qlogin wrapper script to enable X11 forwarding, by
>> using sshd instead of builtin qlogin_daemon.
>> Next, we'd like to limit SSH access to the compute nodes, except if a
>> user has a job running there.  Right now, users can SSH to any node and
>> some are starting to abuse this.
>> However, adding pam_sge_authorize to the sshd pam stack breaks my qlogin
>> wrapper, as it doesn't let the user ssh in for the qlogin job.
>> Does anyone have something like this working?  Maybe I'm missing
>> something simple.
>> https://arc.liv.ac.uk/SGE/htmlman/htmlman8/pam_sge_authorize.html
>> https://arc.liv.ac.uk/trac/SGE/browser/sge/source/3rdparty/tacc_pam_sge/pam_sge_authorize.c?rev=4811
>> I also don't quite understand what
>> https://arc.liv.ac.uk/SGE/htmlman/htmlman8/pam_sge-qrsh-setup.html
>> is for, no matter how many times I re-read those man pages.  Do I need
>> both pam_sge-qrsh-setup and pam_sge_authorize?
> pam_sge-qrsh-setup enables tight integration when using ssh which means grid
> engine can track usage and kill the job when it is finished.
> pam_sge_authorize is what you should use to allow access outside grid engine control
> by users with jobs on the node.
> Their usage is largely orthogonal.
> We don't allow users to log in to the nodes outside GE control at all so not
> tried pam_sge_authorize.
> Check that the execd_spool_dir is set correctly.
> Enable debug and see what the syslog records.

Thanks for the suggestions.  On a test compute node, I added
*.* /var/log/temp.log
to the bottom of /etc/rsyslog.conf and restarted rsyslog.

I added the pam parameters to /etc/pam.d/sshd
execd_spool_dir=/path/to/our/spool debug

Tried to test log in. From temp.log, I saw that it was a typo in my 
variable name!  There goes an hour of my life.

So now pam_sge_authorize.so works as expected for me.

> One (untested) suggestion.  You could have the qlogin sshd exec'd with -m to give it a different name
> and therefore presumably a different PAM config file.  That would mean you could use pam_sge_authorize only
> on the regular sshd while using pam_sge-qrsh-setup for qlogin/qrsh.

Yeah, for pam_sge-qrsh-setup I'm still a bit confused.  Here's what I 
have set:
rlogin_command               builtin
rlogin_daemon                builtin
rsh_command                  builtin
rsh_daemon                   builtin

And then if I do a 'qrsh', it gets me a shell on a compute node, and in 
the output of 'id', I see the additional gid.  I want to do the same 
thing but also have X11-forwarding work.  So for qlogin I have


And that works, and that's been working fine for years, and that's what 
our users run, and it looks like CPU usage gets recorded correctly in 
accounting, but mem usage doesn't, and I can't figure out how to get the 
extra gid in for my invocation of 'sshd -i'.  And maybe occasionally 
some processes get orphaned and not cleaned up correctly.

Also the 'debug' option of pam_sge-qrsh-setup doesn't seem to print 
anything at all.

Alex Chekholko chekh at stanford.edu

More information about the users mailing list