[gridengine users] Unwanted character encoding change

Guchun Zhang astonzhang at gmail.com
Mon Oct 8 11:28:42 UTC 2012


Hi William and Orion,

I have solved this problem by explicitly setting LANG in my script, as
suggested by both of you. Previously, I thought SGE would use the shell
environment settings automatically, but obviously it's not the case.

By the way, if anyone sees "export: command not found" in their STDERR, try
using -S /path/to/shell for qsub when submitting jobs.

William, I have only a very small scale cluster so accessing to each slave
node directly is not much a pain to me.

Many thanks,

Guchun

On 7 October 2012 14:27, William Hay <w.hay at ucl.ac.uk> wrote:

> On 6 October 2012 13:00, Guchun Zhang <astonzhang at gmail.com> wrote:
> > Hi Orion,
> >
> > Many thanks for your reply.
> >
> > But, I am not quite sure that is the real cause of the problem I have.
> > I have already checked LANG on all the slave nodes, which is all set to
>
> How?  Submiting a job that does echo $LANG is the only way I'd trust.
> Environment variables such as LANG are not necessarily the same for
> all processes on a node.  You need to find out what it is set to for
> processes started by Grid Engine.
>
> Most likely your batch jobs are not run in a login shell so don't get
> LANG set and default to the 'C' locale.  I'd try explicitly
> setting(and exporting) LANG in your job script to fix it.  Possibly
> check the values of the various LC_* variables as well.
>
> > UTF8. I also tried some simple jobs which only contain basic shell
> > commands, like cat and cp, and the output is still in UTF8. Moreover, I
>
> cat and cp don't (normally) worry about character encoding at all they
> just move bytes around.  These are interpreted when you view them
> later.
>
> > ssh-ed to the slave nodes and ran the processing code in the terminal.
> > The output files are also still in UTF8.
> >
> > So I am really puzzled.
> >
> > Regards,
> >
> > Guchun
> >
> > Sent from my Windows Phone
> > From: Orion Poplawski
> > Sent: 05/10/2012 16:50
> > To: Guchun Zhang; users at gridengine.org
> > Subject: Re: [gridengine users] Unwanted character encoding change
> > On 10/05/2012 09:45 AM, Guchun Zhang wrote:
> >> Hi,
> >>
> >> I am processing some UTF-8 coded files. The file processing code runs
> fine
> >> locally on the slave nodes. However, if I submit a job to run the file
> >> processing, the output files will be automatically coded in US-ASCII,
> which,
> >> well, is unwanted.
> >>
> >> My question is what could be the problem? Any lead is appreciated.
> >
> >
> > My guess is that the LANG environment variable is getting set
> differently.
> >
> >
> > --
> > Orion Poplawski
> > Technical Manager                     303-415-9701 x222
> > NWRA, Boulder Office                  FAX: 303-415-9702
> > 3380 Mitchell Lane                       orion at nwra.com
> > Boulder, CO 80301                   http://www.nwra.com
> > _______________________________________________
> > users mailing list
> > users at gridengine.org
> > https://gridengine.org/mailman/listinfo/users
> >
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gridengine.org/pipermail/users/attachments/20121008/d04ece70/attachment.html>


More information about the users mailing list