Canu Assembly failure for Long read pipeline with Mouse Genome reference
I am trying to run a long read pipeline with ONT long reads data for mouse models.
I was able to run https://github.com/broadinstitute/long-read-pipelines/blob/kvg_guppy_cpu/wdl/pipelines/ONT/Preprocessing/ONTBasecall.wdl successfully using my fast5 files.
When I am trying to run https://github.com/broadinstitute/long-read-pipelines/blob/3.0.1/wdl/ONTAssembleWithCanu.wdl , I am running into the below issue.
------------
2023/08/12 03:56:10 Starting container setup. 2023/08/12 03:56:12 Done container setup. 2023/08/12 03:56:13 Starting localization. 2023/08/12 03:56:19 Localization script execution started... 2023/08/12 03:56:19 Localizing input gs://fc-88614ae6-5245-4a6e-ab14-5c3fc9d007a2/submissions/87ee6a97-1be5-4429-9557-817c073de5ae/ONTAssembleWithCanu/f8d2b9d9-a770-45ef-a9b5-6b2a1d0b456f/call-MergeFastqs/cacheCopy/merged.fq.gz -> /cromwell_root/fc-88614ae6-5245-4a6e-ab14-5c3fc9d007a2/submissions/87ee6a97-1be5-4429-9557-817c073de5ae/ONTAssembleWithCanu/f8d2b9d9-a770-45ef-a9b5-6b2a1d0b456f/call-MergeFastqs/cacheCopy/merged.fq.gz 2023/08/12 03:56:25 Localizing input gs://fc-88614ae6-5245-4a6e-ab14-5c3fc9d007a2/submissions/87ee6a97-1be5-4429-9557-817c073de5ae/ONTAssembleWithCanu/f8d2b9d9-a770-45ef-a9b5-6b2a1d0b456f/call-Canu/Canu/61aef15e-aa55-498e-9e6f-e09331f445da/call-Correct/script -> /cromwell_root/script 2023/08/12 03:56:27 Localization script execution complete. 2023/08/12 03:56:31 Done localization. 2023/08/12 03:56:32 Running user action: docker run -v /mnt/local-disk:/cromwell_root --entrypoint=/bin/bash us.gcr.io/broad-dsp-lrma/lr-canu@sha256:b116e4c74fa74e384491457fb09b6729e40138d00d7611fea912ab130386d9eb /cromwell_root/script + canu -correct -p 65209 -d canu_correct_output genomeSize=2731m corMaxEvidenceErate=0.15 correctedErrorRate=0.15 -nanopore /cromwell_root/fc-88614ae6-5245-4a6e-ab14-5c3fc9d007a2/submissions/87ee6a97-1be5-4429-9557-817c073de5ae/ONTAssembleWithCanu/f8d2b9d9-a770-45ef-a9b5-6b2a1d0b456f/call-MergeFastqs/cacheCopy/merged.fq.gz -- Canu 2.0 -- -- Detected Java(TM) Runtime Environment '1.8.0_252' (from '/usr/local/openjdk-8/bin/java') with -d64 support. -- -- WARNING: -- WARNING: Failed to run gnuplot using command 'gnuplot'. -- WARNING: Plots will be disabled. -- WARNING: -- -- Detected 32 CPUs and 31 gigabytes of memory. -- No grid engine detected, grid and staging disabled. -- -- ERROR -- ERROR -- ERROR Found 1 machine configuration: -- ERROR class0 - 1 machines with 32 cores with 31 GB memory each. -- ERROR -- ERROR Task red can't run on any available machines. -- ERROR It is requesting: -- ERROR redMemory=32-48 memory (gigabytes) -- ERROR redThreads=4-8 threads -- ERROR -- ERROR No available machine configuration can run this task. -- ERROR -- ERROR Possible solutions: -- ERROR Change redMemory and/or redThreads -- ERROR ABORT: ABORT: Canu 2.0 ABORT: Don't panic, but a mostly harmless error occurred and Canu stopped. ABORT: Try restarting. If that doesn't work, ask for help. ABORT: ABORT: task red failed to find a configuration to run on. ABORT: 2023/08/12 03:56:34 Starting delocalization. 2023/08/12 03:56:35 Delocalization script execution started... 2023/08/12 03:56:35 Delocalizing output /cromwell_root/memory_retry_rc -> gs://fc-88614ae6-5245-4a6e-ab14-5c3fc9d007a2/submissions/87ee6a97-1be5-4429-9557-817c073de5ae/ONTAssembleWithCanu/f8d2b9d9-a770-45ef-a9b5-6b2a1d0b456f/call-Canu/Canu/61aef15e-aa55-498e-9e6f-e09331f445da/call-Correct/memory_retry_rc 2023/08/12 03:56:37 Delocalizing output /cromwell_root/rc -> gs://fc-88614ae6-5245-4a6e-ab14-5c3fc9d007a2/submissions/87ee6a97-1be5-4429-9557-817c073de5ae/ONTAssembleWithCanu/f8d2b9d9-a770-45ef-a9b5-6b2a1d0b456f/call-Canu/Canu/61aef15e-aa55-498e-9e6f-e09331f445da/call-Correct/rc 2023/08/12 03:56:39 Delocalizing output /cromwell_root/stdout -> gs://fc-88614ae6-5245-4a6e-ab14-5c3fc9d007a2/submissions/87ee6a97-1be5-4429-9557-817c073de5ae/ONTAssembleWithCanu/f8d2b9d9-a770-45ef-a9b5-6b2a1d0b456f/call-Canu/Canu/61aef15e-aa55-498e-9e6f-e09331f445da/call-Correct/stdout 2023/08/12 03:56:40 Delocalizing output /cromwell_root/stderr -> gs://fc-88614ae6-5245-4a6e-ab14-5c3fc9d007a2/submissions/87ee6a97-1be5-4429-9557-817c073de5ae/ONTAssembleWithCanu/f8d2b9d9-a770-45ef-a9b5-6b2a1d0b456f/call-Canu/Canu/61aef15e-aa55-498e-9e6f-e09331f445da/call-Correct/stderr 2023/08/12 03:56:42 Delocalizing output /cromwell_root/canu_correct_output/65209.correctedReads.fasta.gz -> gs://fc-88614ae6-5245-4a6e-ab14-5c3fc9d007a2/submissions/87ee6a97-1be5-4429-9557-817c073de5ae/ONTAssembleWithCanu/f8d2b9d9-a770-45ef-a9b5-6b2a1d0b456f/call-Canu/Canu/61aef15e-aa55-498e-9e6f-e09331f445da/call-Correct/canu_correct_output/65209.correctedReads.fasta.gz Required file output '/cromwell_root/canu_correct_output/65209.correctedReads.fasta.gz' does not exist.
Comments
5 comments
Hi Nikhil,
Thank you for writing in about this issue. Can you share the workspace where you are running this workflow with Terra Support by clicking the Share button in your workspace? The Share option is in the three-dots menu at the top-right.
Please provide us with
We’ll be happy to take a closer look as soon as we can!
Kind regards,
Pamela
Thanks for your response. I was able to clone the workflow on github and make changes to pass required resource configurations. With these changes, I was able to run the assembly successfully. Looks like resource parameters for long read pipelines for Canu are not configurable
Here are the details :
workspace-id: 88614ae6-5245-4a6e-ab14-5c3fc9d007a2
workflow-id: f8d2b9d9-a770-45ef-a9b5-6b2a1d0b456f
submission-id: 87ee6a97-1be5-4429-9557-817c073de5ae
Hi Nikhil,
Thank you for letting me know that you were now able to run the workflow successfully! I can let our development team know that this workflow may need to be updated to be successfully run on Terra. If you'd like to share the changes you made, this could be useful for other users running into similar issues in the meantime.
Kind regards,
Pamela
Hi Pamela, Sure. Changes are present in "test-long-read-canu-assembly" branch in my forked repository. https://github.com/nikhil777shingte/long-read-pipelines/tree/test-long-read-canu-assembly
Happy to assist if more questions.
Thank you for sharing that, Nikhil!
Please sign in to leave a comment.