Need Help?

Search our documentation and community forum

Terra is a cloud-native platform for biomedical researchers to access data, run analysis tools, and collaborate.
Terra powers important scientific projects like FireCloud, AnVIL, and BioData Catalyst. Learn more.

Error regarding cnv_somatic_pair_workflow

Comments

15 comments

  • Avatar
    Samantha (she/her)

    Hi Hanbing Song,

     

    Thanks for writing in. Can you share the workspace where you are seeing this issue with GROUP_FireCloud-Support@firecloud.org by clicking the Share button in your workspace? The Share option is in the three-dots menu at the top-right.

    1. Add GROUP_FireCloud-Support@firecloud.org to the User email field and press enter on your keyboard.
    2. Click Save.

    Let us know the workspace name, as well as the relevant submission and workflow IDs. We’ll be happy to take a closer look as soon as we can.

     

    Best,

    Samantha

    0
    Comment actions Permalink
  • Avatar
    Hanbing Song

    Hi Samantha, 

     

    Thanks for your reply and I just shared our workspace with you. The workspace name is fhuanglab/WES_pipeline. In there you might see the latest failed job running 2-CNV_Somatic_Pair which I need help with. 

     
    Thanks, 
     
    Hanbing
     
     
    0
    Comment actions Permalink
  • Avatar
    Samantha (she/her)

    Hi Hanbing Song,

     

    Can you confirm the submission ID that you would like us to look into? I also noticed you posted on the GATK forum with a similar question - is that regarding the same submission here?

     

    Best,

    Samantha

    0
    Comment actions Permalink
  • Avatar
    Hanbing Song

    Hi Samantha, 

    Yes, that was the same problem because I don't know which forum would be the proper one to post this. 

    The submission ID would be this one: 7878b8f7-2f4e-4c9e-9492-1052b0ce5c00

     

    Thank you

    Hanbing

    0
    Comment actions Permalink
  • Avatar
    Samantha (she/her)

    Hi Hanbing Song,

     

    Thanks for confirming. I'll address both posts here if that's okay with you.

    If you scroll further down through the backend log for the failed task, you'll see this error:

    You are receiving this error because the Funcotator datasource you are using (v1.7.20200521) is not supported by the GATK version you're running (v4.1.7.0). Since you are using an older version of GATK, you should use the older datasources. See this bug report for more information.

     

    Best,

    Samantha

    0
    Comment actions Permalink
  • Avatar
    Hanbing Song

    Hi Samantha, 

    Sure! I'll just reply to this thread then. 

    Yeah, that makes a lot of sense. Thank you for the insight! I just modified the funcotator datasource to the 1.6.20190124 version and restarted the job. However, I'm wondering if this would also be the cause of the PlotDenoisedCopyRatiosNormal step? 

     

    Hanbing

    0
    Comment actions Permalink
  • Avatar
    Hanbing Song

    Hi Samantha, 

    I restarted the workflow this morning and so far some of the samples failed again on the plotdenoised step and the funcotator step. I changed the funcotator datasource and didn't see the previous error anymore. Now the error msg for funcotator became this one: 

    A USER ERROR has occurred: Bad input: Is the input a file of segment variant contexts? Variant context does not represent a copy number segment: [VC null @ chr1:23134065-23134079 Q. of type=SYMBOLIC alleles=[G*, <DEL>] attr={END=23134079, Num_Probes=1, Segment_Call=-, Segment_Mean=-29.897353} GT=[] filters=

    And I noticed another strange thing in this run. Previously, some of the samples went through the plotdenoisedcopyrationormal steps and I was able to see the denoised CNV curve for the tumor samples. But for the other samples in the batch, this step failed and here is the log file.

    Could you please advise what could cause this error? 

    2021/03/24 17:00:44 Starting container setup.
    2021/03/24 17:00:47 Done container setup.
    2021/03/24 17:00:50 Starting localization.
    2021/03/24 17:00:58 Localization script execution started...
    2021/03/24 17:00:58 Localizing input gs://fc-9828bda3-33d6-40b9-ad27-544fb296a8e0/a2756ae7-b331-4ea3-8c0b-0843a18aea88/CNVSomaticPairWorkflow/3b3a80a3-218d-45e4-be09-c273522313ab/call-DenoiseReadCountsNormal/RESPOND_60100060_C1_Non-Malignant.standardizedCR.tsv -> /cromwell_root/fc-9828bda3-33d6-40b9-ad27-544fb296a8e0/a2756ae7-b331-4ea3-8c0b-0843a18aea88/CNVSomaticPairWorkflow/3b3a80a3-218d-45e4-be09-c273522313ab/call-DenoiseReadCountsNormal/RESPOND_60100060_C1_Non-Malignant.standardizedCR.tsv
    2021/03/24 17:01:01 Localizing input gs://fc-9828bda3-33d6-40b9-ad27-544fb296a8e0/a2756ae7-b331-4ea3-8c0b-0843a18aea88/CNVSomaticPairWorkflow/3b3a80a3-218d-45e4-be09-c273522313ab/call-DenoiseReadCountsNormal/RESPOND_60100060_C1_Non-Malignant.denoisedCR.tsv -> /cromwell_root/fc-9828bda3-33d6-40b9-ad27-544fb296a8e0/a2756ae7-b331-4ea3-8c0b-0843a18aea88/CNVSomaticPairWorkflow/3b3a80a3-218d-45e4-be09-c273522313ab/call-DenoiseReadCountsNormal/RESPOND_60100060_C1_Non-Malignant.denoisedCR.tsv
    Copying gs://fc-9828bda3-33d6-40b9-ad27-544fb296a8e0/a2756ae7-b331-4ea3-8c0b-0843a18aea88/CNVSomaticPairWorkflow/3b3a80a3-218d-45e4-be09-c273522313ab/call-DenoiseReadCountsNormal/RESPOND_60100060_C1_Non-Malignant.denoisedCR.tsv...
    / [0/1 files][    0.0 B/ 12.3 MiB]   0% Done                                    
    / [1/1 files][ 12.3 MiB/ 12.3 MiB] 100% Done                                    
    Operation completed over 1 objects/12.3 MiB.                                     
    2021/03/24 17:01:03 Localizing input gs://fc-9828bda3-33d6-40b9-ad27-544fb296a8e0/a2756ae7-b331-4ea3-8c0b-0843a18aea88/CNVSomaticPairWorkflow/3b3a80a3-218d-45e4-be09-c273522313ab/call-PlotDenoisedCopyRatiosNormal/script -> /cromwell_root/script
    2021/03/24 17:01:04 Localizing input gs://fc-9828bda3-33d6-40b9-ad27-544fb296a8e0/GRCh38.primary_assembly.genome.dict -> /cromwell_root/fc-9828bda3-33d6-40b9-ad27-544fb296a8e0/GRCh38.primary_assembly.genome.dict
    2021/03/24 17:01:05 Localization script execution complete.
    2021/03/24 17:01:11 Done localization.
    2021/03/24 17:01:12 Running user action: docker run -v /mnt/local-disk:/cromwell_root --entrypoint=/bin/bash us.gcr.io/broad-gatk/gatk@sha256:192fedf9b9d65809da4a2954030269e3f311d296e6b5e4c6c7deec12c7fe84b2 /cromwell_root/script
    Picked up _JAVA_OPTIONS: -Djava.io.tmpdir=/cromwell_root/tmp.42dbf577
    17:01:16.740 INFO  NativeLibraryLoader - Loading libgkl_compression.so from jar:file:/gatk/gatk-package-4.1.7.0-local.jar!/com/intel/gkl/native/libgkl_compression.so
    17:01:17.082 INFO  PlotDenoisedCopyRatios - ------------------------------------------------------------
    17:01:17.083 INFO  PlotDenoisedCopyRatios - The Genome Analysis Toolkit (GATK) v4.1.7.0
    17:01:17.083 INFO  PlotDenoisedCopyRatios - For support and documentation go to https://software.broadinstitute.org/gatk/
    17:01:17.084 INFO  PlotDenoisedCopyRatios - Executing as root@59c013a10564 on Linux v5.4.89+ amd64
    17:01:17.084 INFO  PlotDenoisedCopyRatios - Java runtime: OpenJDK 64-Bit Server VM v1.8.0_212-8u212-b03-0ubuntu1.16.04.1-b03
    17:01:17.085 INFO  PlotDenoisedCopyRatios - Start Date/Time: March 24, 2021 5:01:16 PM UTC
    17:01:17.085 INFO  PlotDenoisedCopyRatios - ------------------------------------------------------------
    17:01:17.085 INFO  PlotDenoisedCopyRatios - ------------------------------------------------------------
    17:01:17.087 INFO  PlotDenoisedCopyRatios - HTSJDK Version: 2.21.2
    17:01:17.087 INFO  PlotDenoisedCopyRatios - Picard Version: 2.21.9
    17:01:17.087 INFO  PlotDenoisedCopyRatios - HTSJDK Defaults.COMPRESSION_LEVEL : 2
    17:01:17.087 INFO  PlotDenoisedCopyRatios - HTSJDK Defaults.USE_ASYNC_IO_READ_FOR_SAMTOOLS : false
    17:01:17.089 INFO  PlotDenoisedCopyRatios - HTSJDK Defaults.USE_ASYNC_IO_WRITE_FOR_SAMTOOLS : true
    17:01:17.089 INFO  PlotDenoisedCopyRatios - HTSJDK Defaults.USE_ASYNC_IO_WRITE_FOR_TRIBBLE : false
    17:01:17.089 INFO  PlotDenoisedCopyRatios - Deflater: IntelDeflater
    17:01:17.089 INFO  PlotDenoisedCopyRatios - Inflater: IntelInflater
    17:01:17.090 INFO  PlotDenoisedCopyRatios - GCS max retries/reopens: 20
    17:01:17.090 INFO  PlotDenoisedCopyRatios - Requester pays: disabled
    17:01:17.091 INFO  PlotDenoisedCopyRatios - Initializing engine
    17:01:17.091 INFO  PlotDenoisedCopyRatios - Done initializing engine
    17:01:17.094 INFO  PlotDenoisedCopyRatios - Reading and validating input files...
    17:01:19.638 INFO  PlotDenoisedCopyRatios - Contigs above length threshold: {chr1=248956422, chr2=242193529, chr3=198295559, chr4=190214555, chr5=181538259, chr6=170805979, chr7=159345973, chr8=145138636, chr9=138394717, chr10=133797422, chr11=135086622, chr12=133275309, chr13=114364328, chr14=107043718, chr15=101991189, chr16=90338345, chr17=83257441, chr18=80373285, chr19=58617616, chr20=64444167, chr21=46709983, chr22=50818468, chrX=156040895, chrY=57227415, KI270728.1=1872759}
    17:01:19.723 WARN  PlotDenoisedCopyRatios - Contigs present in the file /cromwell_root/fc-9828bda3-33d6-40b9-ad27-544fb296a8e0/a2756ae7-b331-4ea3-8c0b-0843a18aea88/CNVSomaticPairWorkflow/3b3a80a3-218d-45e4-be09-c273522313ab/call-DenoiseReadCountsNormal/RESPOND_60100060_C1_Non-Malignant.standardizedCR.tsv are missing from the sequence dictionary and will not be plotted.
    17:01:19.882 WARN  PlotDenoisedCopyRatios - Contigs present in the file /cromwell_root/fc-9828bda3-33d6-40b9-ad27-544fb296a8e0/a2756ae7-b331-4ea3-8c0b-0843a18aea88/CNVSomaticPairWorkflow/3b3a80a3-218d-45e4-be09-c273522313ab/call-DenoiseReadCountsNormal/RESPOND_60100060_C1_Non-Malignant.denoisedCR.tsv are missing from the sequence dictionary and will not be plotted.
    17:01:19.967 INFO  PlotDenoisedCopyRatios - Writing plots to /cromwell_root/out...
    17:01:22.052 INFO  PlotDenoisedCopyRatios - PlotDenoisedCopyRatios complete.
    17:01:22.052 INFO  PlotDenoisedCopyRatios - Shutting down engine
    [March 24, 2021 5:01:22 PM UTC] org.broadinstitute.hellbender.tools.copynumber.plotting.PlotDenoisedCopyRatios done. Elapsed time: 0.09 minutes.
    Runtime.totalMemory()=389545984
    Using GATK jar /root/gatk.jar defined in environment variable GATK_LOCAL_JAR
    Running:
        java -Dsamjdk.use_async_io_read_samtools=false -Dsamjdk.use_async_io_write_samtools=true -Dsamjdk.use_async_io_write_tribble=false -Dsamjdk.compression_level=2 -Xmx6000m -jar /root/gatk.jar PlotDenoisedCopyRatios --standardized-copy-ratios /cromwell_root/fc-9828bda3-33d6-40b9-ad27-544fb296a8e0/a2756ae7-b331-4ea3-8c0b-0843a18aea88/CNVSomaticPairWorkflow/3b3a80a3-218d-45e4-be09-c273522313ab/call-DenoiseReadCountsNormal/RESPOND_60100060_C1_Non-Malignant.standardizedCR.tsv --denoised-copy-ratios /cromwell_root/fc-9828bda3-33d6-40b9-ad27-544fb296a8e0/a2756ae7-b331-4ea3-8c0b-0843a18aea88/CNVSomaticPairWorkflow/3b3a80a3-218d-45e4-be09-c273522313ab/call-DenoiseReadCountsNormal/RESPOND_60100060_C1_Non-Malignant.denoisedCR.tsv --sequence-dictionary /cromwell_root/fc-9828bda3-33d6-40b9-ad27-544fb296a8e0/GRCh38.primary_assembly.genome.dict --minimum-contig-length 1000000 --maximum-copy-ratio 4.0 --point-size-copy-ratio 0.2 --output out --output-prefix RESPOND_60100060_C1_Non-Malignant
    2021/03/24 17:01:23 Starting delocalization.
    2021/03/24 17:01:24 Delocalization script execution started...
    2021/03/24 17:01:24 Delocalizing output /cromwell_root/memory_retry_rc -> gs://fc-9828bda3-33d6-40b9-ad27-544fb296a8e0/a2756ae7-b331-4ea3-8c0b-0843a18aea88/CNVSomaticPairWorkflow/3b3a80a3-218d-45e4-be09-c273522313ab/call-PlotDenoisedCopyRatiosNormal/memory_retry_rc
    2021/03/24 17:01:24 Delocalizing output /cromwell_root/rc -> gs://fc-9828bda3-33d6-40b9-ad27-544fb296a8e0/a2756ae7-b331-4ea3-8c0b-0843a18aea88/CNVSomaticPairWorkflow/3b3a80a3-218d-45e4-be09-c273522313ab/call-PlotDenoisedCopyRatiosNormal/rc
    2021/03/24 17:01:26 Delocalizing output /cromwell_root/stdout -> gs://fc-9828bda3-33d6-40b9-ad27-544fb296a8e0/a2756ae7-b331-4ea3-8c0b-0843a18aea88/CNVSomaticPairWorkflow/3b3a80a3-218d-45e4-be09-c273522313ab/call-PlotDenoisedCopyRatiosNormal/stdout
    2021/03/24 17:01:27 Delocalizing output /cromwell_root/stderr -> gs://fc-9828bda3-33d6-40b9-ad27-544fb296a8e0/a2756ae7-b331-4ea3-8c0b-0843a18aea88/CNVSomaticPairWorkflow/3b3a80a3-218d-45e4-be09-c273522313ab/call-PlotDenoisedCopyRatiosNormal/stderr
    2021/03/24 17:01:29 Delocalizing output /cromwell_root/out/RESPOND_60100060_C1_Non-Malignant.denoised.png -> gs://fc-9828bda3-33d6-40b9-ad27-544fb296a8e0/a2756ae7-b331-4ea3-8c0b-0843a18aea88/CNVSomaticPairWorkflow/3b3a80a3-218d-45e4-be09-c273522313ab/call-PlotDenoisedCopyRatiosNormal/out/RESPOND_60100060_C1_Non-Malignant.denoised.png
    2021/03/24 17:01:30 Delocalizing output /cromwell_root/out/RESPOND_60100060_C1_Non-Malignant.standardizedMAD.txt -> gs://fc-9828bda3-33d6-40b9-ad27-544fb296a8e0/a2756ae7-b331-4ea3-8c0b-0843a18aea88/CNVSomaticPairWorkflow/3b3a80a3-218d-45e4-be09-c273522313ab/call-PlotDenoisedCopyRatiosNormal/out/RESPOND_60100060_C1_Non-Malignant.standardizedMAD.txt
    2021/03/24 17:01:31 Delocalizing output /cromwell_root/out/RESPOND_60100060_C1_Non-Malignant.denoisedMAD.txt -> gs://fc-9828bda3-33d6-40b9-ad27-544fb296a8e0/a2756ae7-b331-4ea3-8c0b-0843a18aea88/CNVSomaticPairWorkflow/3b3a80a3-218d-45e4-be09-c273522313ab/call-PlotDenoisedCopyRatiosNormal/out/RESPOND_60100060_C1_Non-Malignant.denoisedMAD.txt
    2021/03/24 17:01:33 Delocalizing output /cromwell_root/out/RESPOND_60100060_C1_Non-Malignant.scaledDeltaMAD.txt -> gs://fc-9828bda3-33d6-40b9-ad27-544fb296a8e0/a2756ae7-b331-4ea3-8c0b-0843a18aea88/CNVSomaticPairWorkflow/3b3a80a3-218d-45e4-be09-c273522313ab/call-PlotDenoisedCopyRatiosNormal/out/RESPOND_60100060_C1_Non-Malignant.scaledDeltaMAD.txt
    2021/03/24 17:01:34 Delocalizing output /cromwell_root/out/RESPOND_60100060_C1_Non-Malignant.deltaMAD.txt -> gs://fc-9828bda3-33d6-40b9-ad27-544fb296a8e0/a2756ae7-b331-4ea3-8c0b-0843a18aea88/CNVSomaticPairWorkflow/3b3a80a3-218d-45e4-be09-c273522313ab/call-PlotDenoisedCopyRatiosNormal/out/RESPOND_60100060_C1_Non-Malignant.deltaMAD.txt
    2021/03/24 17:01:36 Delocalization script execution complete.
    2021/03/24 17:01:37 Done delocalization.


     Thanks so much! 

    Hanbing

     

    0
    Comment actions Permalink
  • Avatar
    Samantha (she/her)

    Hi Hanbing Song,

    Can you confirm the submission ID for the job where you are seeing this new error? I'll take a look and get back to you as soon as I can.

    Best,

    Samantha

    0
    Comment actions Permalink
  • Avatar
    Hanbing Song

    Yes, sure! Sorry bout not including that. The job ID is a2756ae7-b331-4ea3-8c0b-0843a18aea88

    0
    Comment actions Permalink
  • Avatar
    Samantha (she/her)

    Hi Hanbing Song,

     

    The new error in your FuncotateSegments task is happening because the segment being annotated is too short.  There is an argument that specifies the minimum number of bases that a segment variant context must be in order to be annotated.  The default value is 75 and this variant is of length 14, so it's failing.  You will need to specify the --min-num-bases-for-segment-funcotation argument and provide a number that is smaller than the smallest segment in your input data.

     

    Best,

    Samantha

    0
    Comment actions Permalink
  • Avatar
    Hanbing Song

    Hi Samantha, 

     

    Thanks so much for spotting this. However, I still ran into this error in this step PlotDenoisedCopyRatiosNormal. The problem is that I have 24 samples in this batch and 11 seemed to pass this step and I could see a CNV denoised plot generated. And I checked the step locally and did not run into any problem. Could you please take a look at this error? Also for the job a2756ae7-b331-4ea3-8c0b-0843a18aea88

    Contigs present in the file /cromwell_root/fc-9828bda3-33d6-40b9-ad27-544fb296a8e0/a2756ae7-b331-4ea3-8c0b-0843a18aea88/CNVSomaticPairWorkflow/3b3a80a3-218d-45e4-be09-c273522313ab/call-DenoiseReadCountsNormal/RESPOND_60100060_C1_Non-Malignant.standardizedCR.tsv are missing from the sequence dictionary and will not be plotted.

    I'll restart another run reducing the --min-num-bases-for-segment-funcotation. 

     

    Thanks again, 

    Hanbing

    0
    Comment actions Permalink
  • Avatar
    Samantha (she/her)

    Hi Hanbing Song,

     

    Sorry for the delayed response. We are still looking into the error in the PlotDenoisedCopyRatiosNormal task as we think it may be a bug. We've reached out to GATK developers and will let you know once we hear back. 

    I also noticed the following error you're receiving in the FuncotateSegments task: A USER ERROR has occurred: min-num-bases-for-segment-funcotation is not a recognized option. You'll need to use at least version 4.1.8.0 because that is when the min-num-bases-for-segment-funcotation parameter was added. You'll need to use GATK version 4.1.8.0 as your GATK docker and use v4.1.8.0 of the WDL. 

    Please let me know if you have any questions.

     

    Best,

    Samantha

    0
    Comment actions Permalink
  • Avatar
    Hanbing Song

    Hi Samantha, 

     

    Thanks for following up on this issue and thanks for reaching out to GATKdevelopers for help! 

    Got it, I'll change the gatk version and test it out again soon. We just figured out which common site file to use for our CNV call so hopefully it'll smooth the outcome CNV curves a little bit. 

     

    Hanbing

    0
    Comment actions Permalink
  • Avatar
    Samantha (she/her)

    Hi Hanbing Song,

     

    Our developers noticed that you are using hg38_v0_exome_calling_regions.v1.interval_list as the input intervals, which is not correct. You should be using an interval list that corresponds to your target kit.

     

    Best,

    Samantha

    0
    Comment actions Permalink
  • Avatar
    Hanbing Song

    Hi Samantha, 

    Thanks for pointing it out! But I only have a .bed file for our target kit. Is there a way of converting it to the interval_list format or .bed file also works? 

    In the meantime, in this workflow I was using, there was also a "common site" input. If I put the target kit in the input interval then what should I use for the common site? 

    I tested the workflow out in my local clusters for the steps which were problematic but they all worked fine, which is very confusing to me. 

    Thanks! 

    Hanbing

     

    0
    Comment actions Permalink

Please sign in to leave a comment.

Powered by Zendesk