cnv-somatic workflow not compatible with gatk latest versions (4.1.7.0 above)
Hi,
Has anyone had success running cnv-somatic workflow with latest versions of gatk?
fails at `PlotDenoisedCopyRatiosNormal` Task when run with docker gatk 4.1.8.1 , 4.1.8.0 & 4.1.7.0
Error; CNVSomaticPairWorkflow.PlotDenoisedCopyRatiosNormal:NA:1 failed. Job exited without an error, exit code 0. PAPI error code 9. Please check the log file for more details: gs://fc-ac4624cb-a8fc-49a2-b071-d3a0ae799418/579c10eb-20eb-4906-b1ff-eab3216581f1/CNVSomaticPairWorkflow/000bc70b-f5a4-414e-b715-6053aa97d933/call-PlotDenoisedCopyRatiosNormal/PlotDenoisedCopyRatiosNormal.log.
a bit unclear what is throwing the error (entire log file attached below)
but works with gatk docker 4.1.6.0
Would be grateful if anyone could point me to a workflow compatible with with latest gatk versions.
Kind regards
Sam
`
2020/07/27 17:35:12 Starting container setup.
2020/07/27 17:35:15 Done container setup.
2020/07/27 17:35:16 Starting localization.
2020/07/27 17:35:26 Localization script execution started...
2020/07/27 17:35:26 Localizing input gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.dict -> /cromwell_root/gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.dict
2020/07/27 17:35:27 Localizing input gs://fc-ac4624cb-a8fc-49a2-b071-d3a0ae799418/579c10eb-20eb-4906-b1ff-eab3216581f1/CNVSomaticPairWorkflow/000bc70b-f5a4-414e-b715-6053aa97d933/call-PlotDenoisedCopyRatiosNormal/script -> /cromwell_root/script
2020/07/27 17:35:30 Localizing input gs://fc-ac4624cb-a8fc-49a2-b071-d3a0ae799418/579c10eb-20eb-4906-b1ff-eab3216581f1/CNVSomaticPairWorkflow/000bc70b-f5a4-414e-b715-6053aa97d933/call-DenoiseReadCountsNormal/268.hg38.standardizedCR.tsv -> /cromwell_root/fc-ac4624cb-a8fc-49a2-b071-d3a0ae799418/579c10eb-20eb-4906-b1ff-eab3216581f1/CNVSomaticPairWorkflow/000bc70b-f5a4-414e-b715-6053aa97d933/call-DenoiseReadCountsNormal/268.hg38.standardizedCR.tsv
2020/07/27 17:35:31 Localizing input gs://fc-ac4624cb-a8fc-49a2-b071-d3a0ae799418/579c10eb-20eb-4906-b1ff-eab3216581f1/CNVSomaticPairWorkflow/000bc70b-f5a4-414e-b715-6053aa97d933/call-DenoiseReadCountsNormal/268.hg38.denoisedCR.tsv -> /cromwell_root/fc-ac4624cb-a8fc-49a2-b071-d3a0ae799418/579c10eb-20eb-4906-b1ff-eab3216581f1/CNVSomaticPairWorkflow/000bc70b-f5a4-414e-b715-6053aa97d933/call-DenoiseReadCountsNormal/268.hg38.denoisedCR.tsv
Copying gs://fc-ac4624cb-a8fc-49a2-b071-d3a0ae799418/579c10eb-20eb-4906-b1ff-eab3216581f1/CNVSomaticPairWorkflow/000bc70b-f5a4-414e-b715-6053aa97d933/call-DenoiseReadCountsNormal/268.hg38.denoisedCR.tsv...
/ [0/1 files][ 0.0 B/ 9.2 MiB] 0% Done
/ [1/1 files][ 9.2 MiB/ 9.2 MiB] 100% Done
Operation completed over 1 objects/9.2 MiB.
2020/07/27 17:35:33 Localization script execution complete.
2020/07/27 17:35:38 Done localization.
2020/07/27 17:35:39 Running user action: docker run -v /mnt/local-disk:/cromwell_root --entrypoint= us.gcr.io/broad-gatk/gatk@sha256:8051adab0ff725e7e9c2af5997680346f3c3799b2df3785dd51d4abdd3da747b /bin/bash /cromwell_root/script
Picked up _JAVA_OPTIONS: -Djava.io.tmpdir=/cromwell_root/tmp.829f9894
17:35:44.167 INFO NativeLibraryLoader - Loading libgkl_compression.so from jar:file:/gatk/gatk-package-4.1.8.1-local.jar!/com/intel/gkl/native/libgkl_compression.so
17:35:44.495 INFO PlotDenoisedCopyRatios - ------------------------------------------------------------
17:35:44.495 INFO PlotDenoisedCopyRatios - The Genome Analysis Toolkit (GATK) v4.1.8.1
17:35:44.495 INFO PlotDenoisedCopyRatios - For support and documentation go to https://software.broadinstitute.org/gatk/
17:35:44.496 INFO PlotDenoisedCopyRatios - Executing as root@15ef1540b7e1 on Linux v4.19.112+ amd64
17:35:44.496 INFO PlotDenoisedCopyRatios - Java runtime: OpenJDK 64-Bit Server VM v1.8.0_242-8u242-b08-0ubuntu3~18.04-b08
17:35:44.497 INFO PlotDenoisedCopyRatios - Start Date/Time: July 27, 2020 5:35:44 PM GMT
17:35:44.497 INFO PlotDenoisedCopyRatios - ------------------------------------------------------------
17:35:44.497 INFO PlotDenoisedCopyRatios - ------------------------------------------------------------
17:35:44.498 INFO PlotDenoisedCopyRatios - HTSJDK Version: 2.23.0
17:35:44.498 INFO PlotDenoisedCopyRatios - Picard Version: 2.22.8
17:35:44.498 INFO PlotDenoisedCopyRatios - HTSJDK Defaults.COMPRESSION_LEVEL : 2
17:35:44.498 INFO PlotDenoisedCopyRatios - HTSJDK Defaults.USE_ASYNC_IO_READ_FOR_SAMTOOLS : false
17:35:44.498 INFO PlotDenoisedCopyRatios - HTSJDK Defaults.USE_ASYNC_IO_WRITE_FOR_SAMTOOLS : true
17:35:44.498 INFO PlotDenoisedCopyRatios - HTSJDK Defaults.USE_ASYNC_IO_WRITE_FOR_TRIBBLE : false
17:35:44.498 INFO PlotDenoisedCopyRatios - Deflater: IntelDeflater
17:35:44.498 INFO PlotDenoisedCopyRatios - Inflater: IntelInflater
17:35:44.499 INFO PlotDenoisedCopyRatios - GCS max retries/reopens: 20
17:35:44.499 INFO PlotDenoisedCopyRatios - Requester pays: disabled
17:35:44.499 INFO PlotDenoisedCopyRatios - Initializing engine
17:35:44.499 INFO PlotDenoisedCopyRatios - Done initializing engine
17:35:44.524 INFO PlotDenoisedCopyRatios - Reading and validating input files...
17:35:46.803 INFO PlotDenoisedCopyRatios - Contigs above length threshold: {chr1=248956422, chr2=242193529, chr3=198295559, chr4=190214555, chr5=181538259, chr6=170805979, chr7=159345973, chr8=145138636, chr9=138394717, chr10=133797422, chr11=135086622, chr12=133275309, chr13=114364328, chr14=107043718, chr15=101991189, chr16=90338345, chr17=83257441, chr18=80373285, chr19=58617616, chr20=64444167, chr21=46709983, chr22=50818468, chrX=156040895, chrY=57227415, chr16_KI270728v1_random=1872759, chr5_GL339449v2_alt=1612928, chr6_GL000250v2_alt=4672374, chr7_KI270803v1_alt=1111570, chr14_KI270847v1_alt=1511111, chr14_KI270846v1_alt=1351393, chr16_KI270853v1_alt=2659700, chr17_KI270857v1_alt=2877074, chr17_GL000258v2_alt=1821992, chr5_KI270897v1_alt=1144418, chr6_GL000251v2_alt=4795265, chr15_KI270905v1_alt=5161414, chr17_KI270908v1_alt=1423190, chr6_GL000252v2_alt=4604811, chr19_GL949748v2_alt=1064304, chr6_GL000253v2_alt=4677643, chr19_GL949749v2_alt=1091841, chr6_GL000254v2_alt=4827813, chr19_GL949750v2_alt=1066390, chr6_GL000255v2_alt=4606388, chr19_GL949751v2_alt=1002683, chr6_GL000256v2_alt=4929269, chr19_KI270938v1_alt=1066800}
17:35:47.043 INFO PlotDenoisedCopyRatios - Writing plots to /cromwell_root/out...
17:35:49.091 INFO PlotDenoisedCopyRatios - PlotDenoisedCopyRatios complete.
17:35:49.091 INFO PlotDenoisedCopyRatios - Shutting down engine
[July 27, 2020 5:35:49 PM GMT] org.broadinstitute.hellbender.tools.copynumber.plotting.PlotDenoisedCopyRatios done. Elapsed time: 0.08 minutes.
Runtime.totalMemory()=367525888
Using GATK jar /root/gatk.jar defined in environment variable GATK_LOCAL_JAR
Running:
java -Dsamjdk.use_async_io_read_samtools=false -Dsamjdk.use_async_io_write_samtools=true -Dsamjdk.use_async_io_write_tribble=false -Dsamjdk.compression_level=2 -Xmx6000m -jar /root/gatk.jar PlotDenoisedCopyRatios --standardized-copy-ratios /cromwell_root/fc-ac4624cb-a8fc-49a2-b071-d3a0ae799418/579c10eb-20eb-4906-b1ff-eab3216581f1/CNVSomaticPairWorkflow/000bc70b-f5a4-414e-b715-6053aa97d933/call-DenoiseReadCountsNormal/268.hg38.standardizedCR.tsv --denoised-copy-ratios /cromwell_root/fc-ac4624cb-a8fc-49a2-b071-d3a0ae799418/579c10eb-20eb-4906-b1ff-eab3216581f1/CNVSomaticPairWorkflow/000bc70b-f5a4-414e-b715-6053aa97d933/call-DenoiseReadCountsNormal/268.hg38.denoisedCR.tsv --sequence-dictionary /cromwell_root/gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.dict --minimum-contig-length 1000000 --output out --output-prefix 268.hg38
2020/07/27 17:35:50 Starting delocalization.
2020/07/27 17:35:51 Delocalization script execution started...
2020/07/27 17:35:51 Delocalizing output /cromwell_root/memory_retry_rc -> gs://fc-ac4624cb-a8fc-49a2-b071-d3a0ae799418/579c10eb-20eb-4906-b1ff-eab3216581f1/CNVSomaticPairWorkflow/000bc70b-f5a4-414e-b715-6053aa97d933/call-PlotDenoisedCopyRatiosNormal/memory_retry_rc
2020/07/27 17:35:51 Delocalizing output /cromwell_root/rc -> gs://fc-ac4624cb-a8fc-49a2-b071-d3a0ae799418/579c10eb-20eb-4906-b1ff-eab3216581f1/CNVSomaticPairWorkflow/000bc70b-f5a4-414e-b715-6053aa97d933/call-PlotDenoisedCopyRatiosNormal/rc
2020/07/27 17:35:52 Delocalizing output /cromwell_root/stdout -> gs://fc-ac4624cb-a8fc-49a2-b071-d3a0ae799418/579c10eb-20eb-4906-b1ff-eab3216581f1/CNVSomaticPairWorkflow/000bc70b-f5a4-414e-b715-6053aa97d933/call-PlotDenoisedCopyRatiosNormal/stdout
2020/07/27 17:35:54 Delocalizing output /cromwell_root/stderr -> gs://fc-ac4624cb-a8fc-49a2-b071-d3a0ae799418/579c10eb-20eb-4906-b1ff-eab3216581f1/CNVSomaticPairWorkflow/000bc70b-f5a4-414e-b715-6053aa97d933/call-PlotDenoisedCopyRatiosNormal/stderr
2020/07/27 17:35:56 Delocalizing output /cromwell_root/out/268.hg38.standardizedMAD.txt -> gs://fc-ac4624cb-a8fc-49a2-b071-d3a0ae799418/579c10eb-20eb-4906-b1ff-eab3216581f1/CNVSomaticPairWorkflow/000bc70b-f5a4-414e-b715-6053aa97d933/call-PlotDenoisedCopyRatiosNormal/out/268.hg38.standardizedMAD.txt
2020/07/27 17:35:57 Delocalizing output /cromwell_root/out/268.hg38.denoised.png -> gs://fc-ac4624cb-a8fc-49a2-b071-d3a0ae799418/579c10eb-20eb-4906-b1ff-eab3216581f1/CNVSomaticPairWorkflow/000bc70b-f5a4-414e-b715-6053aa97d933/call-PlotDenoisedCopyRatiosNormal/out/268.hg38.denoised.png
2020/07/27 17:35:58 Delocalizing output /cromwell_root/out/268.hg38.scaledDeltaMAD.txt -> gs://fc-ac4624cb-a8fc-49a2-b071-d3a0ae799418/579c10eb-20eb-4906-b1ff-eab3216581f1/CNVSomaticPairWorkflow/000bc70b-f5a4-414e-b715-6053aa97d933/call-PlotDenoisedCopyRatiosNormal/out/268.hg38.scaledDeltaMAD.txt
2020/07/27 17:36:00 Delocalizing output /cromwell_root/out/268.hg38.denoisedMAD.txt -> gs://fc-ac4624cb-a8fc-49a2-b071-d3a0ae799418/579c10eb-20eb-4906-b1ff-eab3216581f1/CNVSomaticPairWorkflow/000bc70b-f5a4-414e-b715-6053aa97d933/call-PlotDenoisedCopyRatiosNormal/out/268.hg38.denoisedMAD.txt
2020/07/27 17:36:01 Delocalizing output /cromwell_root/out/268.hg38.deltaMAD.txt -> gs://fc-ac4624cb-a8fc-49a2-b071-d3a0ae799418/579c10eb-20eb-4906-b1ff-eab3216581f1/CNVSomaticPairWorkflow/000bc70b-f5a4-414e-b715-6053aa97d933/call-PlotDenoisedCopyRatiosNormal/out/268.hg38.deltaMAD.txt
2020/07/27 17:36:03 Delocalizing output /cromwell_root/out/268.hg38.denoisedLimit4.png -> gs://fc-ac4624cb-a8fc-49a2-b071-d3a0ae799418/579c10eb-20eb-4906-b1ff-eab3216581f1/CNVSomaticPairWorkflow/000bc70b-f5a4-414e-b715-6053aa97d933/call-PlotDenoisedCopyRatiosNormal/out/268.hg38.denoisedLimit4.png
Required file output '/cromwell_root/out/268.hg38.denoisedLimit4.png' does not exist.
`
Comments
7 comments
Hi Samuel Terkper Ahuno,
Thank you for reporting this issue. Can you share the workspace where you are seeing this issue with GROUP_FireCloud-Support@firecloud.org by clicking the Share button in your workspace? The Share option is in the three-dots menu at the top-right.
Let us know the workspace name, as well as the relevant submission and workflow IDs. We’ll be happy to take a closer look as soon as we can.
Best,
Samantha
Hi Samantha (she/her)
I have shared workspace with GROUP_FireCloud-Support@firecloud.org email;
- failed - gatk 4.1.7.0; submission-id: b5b9e7a1-bbd6-4fda-a356-96ae196c0370
- failed - gatk:4.1.8.1; submission-id: 579c10eb-20eb-4906-b1ff-eab3216581f1
Hi Samuel Terkper Ahuno,
Can you confirm whether you are using the same inputs each time and only changing the GATK version?
Thanks,
Samantha
Hi Samantha (she/her)
just to be sure we not missing anything I repeated cnv-somatic-wokflow with same parameters;
Repeated subissions with "us.gcr.io/broad-gatk/gatk:4.1.8.0" ; Submission ID- 13a3e1a3-16b3-4762-ad58-d7dd904da9b5
sam
Hi Samuel Terkper Ahuno,
It looks like the issue here is that you are using an older version of the workflow with a newer version of GATK.
Version 4.1.6.0 of the workflow should be used with GATK 4.1.6.0, version 4.1.7.0 of the workflow should be used with GATK 4.1.7.0, and so on.
Try using version 4.1.7.0 of the workflow and the corresponding version of GATK.
Best,
Samantha
Hi Samantha (she/her)
i confirm cnv-somatic-pair workflow version 4.1.7.0 works with corresonponding gatk docker 4.1.7.0
thanks
S
Hi Samuel Terkper Ahuno,
Glad to hear. If you need any further assistance, please let us know.
Best,
Samantha
Please sign in to leave a comment.