'task command' how to work with local variables

March 24, 2021 22:16
3 comments

I having trouble writing tasks. I think the issue is that you can not simply cut and past bash scripts. This like variable substitution do not work as expected. I wonder if there is some more detailed documentation somewhere

for example

if typically use ${myVar} instead of $myVar in shell scripts. this works around spaces in file name kinds of problems.

womtool/cromwell always seems to try and expand ${myVar} and throws and error myVar was not defined in the input section of my task. I can use $myVar with local variables that are defined with the shell script, how ever if I do something like

# echo ${myVar}

womtool/cromwell throws and error even though the statement was commented out

# echo \$\{myVar\}

will pass

# echo \$myVar

does not pass.

Here is the comment I wanted to create in my test.wdl

# how to local variables work?
# you can not use \$\{var\} you must use dollar var
count=`ls . | wc -l`
echo "the number of files in . is $count"

Comments

3 comments

Sushma Chaluvadi
- Edited March 25, 2021 13:54
I noticed this pattern while working on converting a bash script that I was using locally into a WDL task. I have very much simplified the task but what I noticed is that variables that are defined in the input block of the task are referenced with the `~{myVar}` and then any variables that are created within the task - so, I guess the local variables for the task's VM - have to be referenced as `"$myVar"`. So, in my example below, I defined `~{id_details}` was defined in the input block and then `"$job_id"` was a local variable. Not entirely sure if that helps answer your question but it was what ended up working for me as I transferred by bash code into WDL format.
```
task demo_variables {
  input {
    File id_details
    String docker
  }
  command <<<
  awk '{print $1}' OFS="\t" ~{id_details} > id_list.txt

  while IFS="\t" read -r job_id
    do
      bq wait "$job_id"
      echo "$job_id"
  done < job_id_list.txt >> final_details.txt
  >>>

  runtime {
    docker: docker
  }
  output {
    File final_details = "final_details.txt"
  }
}
```
0
Andrew Davidson
- March 25, 2021 21:34
Hi Sushma

thanks for the tip. I think there is a bug in Cromwell. if you comment out a line in your bash task, Cromwell will still try to expand it. this will cause an error if the variable is not defined

Andy

0
Samantha (she/her)
- April 02, 2021 18:08
Hi Andrew Davidson,

Sorry for the late response. Can you please paste your full test WDL? I was not able to replicate the error when I tested with each of the following:
```
# echo ${myVar} 
```
```
# echo \$myVar
```
```
# how to local variables work?
# you can not use \$\{var\} you must use dollar var
count=`ls . | wc -l`
echo "the number of files in . is $count"
```
Best,

Samantha
0

Please sign in to leave a comment.