[gridengine users] Unwanted character encoding change
astonzhang at gmail.com
Mon Oct 8 11:28:42 UTC 2012
Hi William and Orion,
I have solved this problem by explicitly setting LANG in my script, as
suggested by both of you. Previously, I thought SGE would use the shell
environment settings automatically, but obviously it's not the case.
By the way, if anyone sees "export: command not found" in their STDERR, try
using -S /path/to/shell for qsub when submitting jobs.
William, I have only a very small scale cluster so accessing to each slave
node directly is not much a pain to me.
On 7 October 2012 14:27, William Hay <w.hay at ucl.ac.uk> wrote:
> On 6 October 2012 13:00, Guchun Zhang <astonzhang at gmail.com> wrote:
> > Hi Orion,
> > Many thanks for your reply.
> > But, I am not quite sure that is the real cause of the problem I have.
> > I have already checked LANG on all the slave nodes, which is all set to
> How? Submiting a job that does echo $LANG is the only way I'd trust.
> Environment variables such as LANG are not necessarily the same for
> all processes on a node. You need to find out what it is set to for
> processes started by Grid Engine.
> Most likely your batch jobs are not run in a login shell so don't get
> LANG set and default to the 'C' locale. I'd try explicitly
> setting(and exporting) LANG in your job script to fix it. Possibly
> check the values of the various LC_* variables as well.
> > UTF8. I also tried some simple jobs which only contain basic shell
> > commands, like cat and cp, and the output is still in UTF8. Moreover, I
> cat and cp don't (normally) worry about character encoding at all they
> just move bytes around. These are interpreted when you view them
> > ssh-ed to the slave nodes and ran the processing code in the terminal.
> > The output files are also still in UTF8.
> > So I am really puzzled.
> > Regards,
> > Guchun
> > Sent from my Windows Phone
> > From: Orion Poplawski
> > Sent: 05/10/2012 16:50
> > To: Guchun Zhang; users at gridengine.org
> > Subject: Re: [gridengine users] Unwanted character encoding change
> > On 10/05/2012 09:45 AM, Guchun Zhang wrote:
> >> Hi,
> >> I am processing some UTF-8 coded files. The file processing code runs
> >> locally on the slave nodes. However, if I submit a job to run the file
> >> processing, the output files will be automatically coded in US-ASCII,
> >> well, is unwanted.
> >> My question is what could be the problem? Any lead is appreciated.
> > My guess is that the LANG environment variable is getting set
> > --
> > Orion Poplawski
> > Technical Manager 303-415-9701 x222
> > NWRA, Boulder Office FAX: 303-415-9702
> > 3380 Mitchell Lane orion at nwra.com
> > Boulder, CO 80301 http://www.nwra.com
> > _______________________________________________
> > users mailing list
> > users at gridengine.org
> > https://gridengine.org/mailman/listinfo/users
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the users