[gridengine users] reading arguments using awk with escaped variables

Sara Rolfe smrolfe at u.washington.edu
Tue Apr 17 16:59:57 UTC 2012


Removing the quotes and adding the escape before the first $ as you  
suggested fixed the error.  I see what was happening now.

> infile=\$(awk -v line=\$SGE_TASK_ID 'NR == line' /myPath/ 
> my_outfile_list.txt)

I really appreciate your help figuring this out!

Thanks,
Sara

On Apr 17, 2012, at 8:14 AM, Reuti wrote:

> Am 17.04.2012 um 16:58 schrieb Sara Rolfe:
>
>> Thanks, I did try single quotes too, but get the same problem.   
>> Maybe it would help if you could look at the test script I'm  
>> submitting?  I'd really like to know if there's something I'm doing  
>> which makes escaping the variables necessary.  I've copied it below.
>>
>> #!/bin/bash
>> #$ -cwd
>> #$ -S /bin/bash
>> #
>> start=1
>> end=3
>> step=1
>> email="smrolfe at cs.washington.edu"
>> jobname="SmoothingFilter_test"
>> #
>> cat <<EOF | qsub
>
> No cat necessary:
>
> qsub <<EOF
>
>> #!/bin/bash
>> #\$ -S /bin/bash
>> #\$ -M $email
>> #\$ -m be
>> #\$ -N $jobname
>> #\$ -t $start-$end:$step
>> #\$ -e /myPath
>> #\$ -o /myPath
>
> No \ necessary, as $ followed by a blank won't be expanded.
>
> Why not specifying all as arguments to the `qsub` command itself?
>
>> #
>> uname -a
>> #
>> infile=$(awk -v 'line=$SGE_TASK_ID' 'NR == line' /myPath/ 
>> my_outfile_list.txt)
>
> infile=\$(awk -v line=\$SGE_TASK_ID 'NR == line' /myPath/ 
> my_outfile_list.txt)
>
> It should be expanded at execution time, not while the script is  
> created. So the $ needs to be escaped in both locations, and later  
> on no ', as it would prohibit the expansion.
>
> The below mentioned:
>
>>>>> awk "NR==\$SGE_TASK_ID" /myPath/fileList.txt
>
> should work too
>
> -- Reuti
>
>
>> #
>> echo "The input file is: $infile"
>> echo "The escaped input file is: \$infile"
>> echo "The task ID is: $SGE_TASK_ID"
>> echo "The escaped task ID is: \$SGE_TASK_ID"
>> #
>> EOF
>> #
>> exit 0
>>
>> The output file for the third iteration is:
>>
>> The input file is:
>> The escaped input file is:
>> The task ID is: undefined
>> The escaped task ID is: 3
>>
>> On Apr 17, 2012, at 2:37 AM, Reuti wrote:
>>
>>> Am 17.04.2012 um 11:08 schrieb George Georgalis:
>>>
>>>> On Mon, Apr 16, 2012 at 10:37 PM, Sara Rolfe <smrolfe at u.washington.edu 
>>>> > wrote:
>>>>> I am new to both scripting and SGE, so I don't understand why,  
>>>>> but I need to
>>>>> escape all the variables in my script.  For example,
>>>>>
>>>>> awk "NR==$SGE_TASK_ID" /myPath/fileList.txt
>>>>>
>>>>> produces a blank output, but if I escape the env variable, like:
>>>>>
>>>>> awk "NR==\$SGE_TASK_ID" /myPath/fileList.txt
>>>>>
>>>>> then I get the correct line from the text file.  The problem is  
>>>>> when I try
>>>>> to assign this output to a variable.  I still need to use the  
>>>>> escape, but I
>>>>> think it's not being passed correctly.
>>>>
>>>>
>>>> I usually use single quotes for awk. It prevents shell expansion of
>>>> the symbols. (Shell expansion applies to double quoted arguments  
>>>> but
>>>> not single quoted ones.)
>>>
>>> I second this. Using -v to pass arguments to awk (they are known  
>>> in the BEGIN block already) or list them after the script (they  
>>> are only available outside of the BEGIN block) and using single  
>>> quotation marks for the script is the careful way to avoid the  
>>> mentioned side effects.
>>>
>>> But I would accept an exception here if the only rule is a test  
>>> for the NR field as it's still clear what is done.
>>>
>>> --
>>>
>>> Do you submit an array job? Otherwise $SGE_TASK_ID won't be set to  
>>> 1 but "undefined". Then:
>>>
>>> ${SGE_TASK_ID/undefinded/1}
>>>
>>> might help and give you a 1 for non-array jobs, hence:
>>>
>>> #!/bin/sh
>>> echo $(awk "NR==${SGE_TASK_ID/undefined/1}" file)
>>>
>>> is working fine for me.
>>>
>>> -- Reuti
>>>
>>>
>>>> .George
>>>>
>>>>
>>>> -- 
>>>> George Georgalis, (415) 894-2710, http://www.galis.org/
>>>>
>>>> _______________________________________________
>>>> users mailing list
>>>> users at gridengine.org
>>>> https://gridengine.org/mailman/listinfo/users
>>>>
>>>
>>
>




More information about the users mailing list