Setting runtime variables in WDL script

January 13, 2021 16:39

Hi Terra team,

I have just started trying to develop my own workflows, and so have written simple samtools-based one. However, I suspect I have set the runtime variables incorrectly, as I cannot get the job to run. I get the following error:

Could not build the path "get_mt_coverage_2021-01-12T18-01-55". 
It may refer to a filesystem not supported by this instance of Cromwell. Supported filesystems are: Google Cloud Storage, DRS. 
Failures: Google Cloud Storage: Path "get_mt_coverage_2021-01-12T18-01-55" does not have a gcs scheme (IllegalArgumentException) DRS: get_mt_coverage_2021-01-12T18-01-55 does not have a drs scheme. 
(IllegalArgumentException) Please refer to the documentation for more information on how to configure filesystems: http://cromwell.readthedocs.io/en/develop/backends/HPC/#filesystems

My current WDL script is as follows:

task get_mt_coverage {

    File bam_file
    Int memory_gb

    command {
        set -euo pipefail
        echo $(date +"[%b %d %H:%M:%S] samtools: getting coverage over chrM")
        samtools depth -r chrM ${bam_file}
        echo $(date +"[%b %d %H:%M:%S] done.")
    }

    output {
        File coverage_over_chrMT = "chrM_depth.csv"
    }

    runtime {
    	docker: "broadinstitute/genomes-in-the-cloud:2.3.1-1500064817"
        memory: "${memory_gb}GB"
    }

}


workflow get_mt_coverage_workflow {
    call get_mt_coverage
}

Bottom line is that I'm unsure what to define in runtime.. I am not actually using a docker to run this workflow.

Would appreciate your help.

Thanks,

Aine

Comments

11 comments

Samantha Velasquez

January 13, 2021 17:06
Hi Aine Fairbrother,

Thanks for writing in. Can you share the workspace where you are seeing this issue with GROUP_FireCloud-Support@firecloud.org by clicking the Share button in your workspace? The Share option is in the three-dots menu at the top-right.
1. Add GROUP_FireCloud-Support@firecloud.org to the User email field and press enter on your keyboard.
2. Click Save.
Let us know the workspace name, as well as the relevant submission and workflow IDs. We’ll be happy to take a closer look as soon as we can.

Best,

Samantha
0

Comment actions Permalink
Aine Fairbrother

January 13, 2021 17:15
Hi Samantha,

Thanks for getting back to me. I have shared that with the specified email.

The details for the job that threw up the error above are below:

Workspace ID: aine-phd/amppd_pilot_aine

Submission ID: 8f08c574-b417-48ac-b003-fedaf35ead8a

Workflow ID: e93c860f-f0c4-45d7-906f-df976e629eea

Thanks,

Aine
1

Comment actions Permalink
Aine Fairbrother

January 13, 2021 17:19
So, just to be clear, when I try to run the workflow without specifying a docker in 'runtime', the error is that I haven't specified a docker. Is there a default one based on my environment that I can input, or do custom workflows have to run off dockers?
0

Comment actions Permalink
Samantha Velasquez

January 13, 2021 17:44
Hi Aine Fairbrother,

Looks like your workspace is protected by an authorization domain. Can you please add svelasqu@broadinstitute.org to the authorization domain so I can access the workspace?

Thanks,

Samantha
0

Comment actions Permalink
Aine Fairbrother

January 13, 2021 19:11
Hi Samantha,

I have shared the workspace with your BI email, but not sure I can add that email to the authorisation domain as it’s amp-pd-researchers, am I incorrect?
0

Comment actions Permalink
Samantha Velasquez

January 13, 2021 22:05
Hi Aine Fairbrother,

That's fine. Can you instead share a screenshot of your workflow configuration?

Thanks,
Samantha
0

Comment actions Permalink
Aine Fairbrother

January 14, 2021 10:14
0

Comment actions Permalink
Aine Fairbrother

January 14, 2021 10:15
0

Comment actions Permalink
Samantha Velasquez

January 14, 2021 15:23
Hi Aine Fairbrother,

Thanks for sharing those screenshots. It looks like the issue is that you are passing the sample set ID get_mt_coverage_2021-01-12T18-01-55 to the bam_file variable, which is expecting a file.

Please let me know if you have any questions.

Best,

Samantha
0

Comment actions Permalink
Aine Fairbrother

January 14, 2021 15:33
Hi Samantha,

Thank you for clearing that up, I think I misunderstood the 'this.' syntax. This is very new to me!

I do have one more question, apologies if it is an ignorant one - how do I populate the 'runtime' section of the WDL script? When I do not define a docker, it produces an error that no docker is supplied. But I'm not sure which dockers are available for me to use.

Thank you for your help,

Aine
0

Comment actions Permalink
Samantha Velasquez

January 14, 2021 18:37
Hi Aine Fairbrother,

No problem. If you haven't already, I would suggest reading through our Workflows Quickstart Tutorial to get a better idea of how to run your workflows on Terra.

Defining a Docker is required when running your workflow on Terra. However, we aren't able to tell users which Docker to use since it's highly variable based on what you are trying to do. You mentioned that your workflow is based off of samtools, so if samtools has an official Docker of sorts then you can definitely use that.

Please let me know if you have any other questions.

Best,

Samantha
0

Comment actions Permalink

Please sign in to leave a comment.

Need Help?

Search our documentation and community forum

Setting runtime variables in WDL script

Comments