Gather script fails in scatter gather.

Post author
smk_84

Hi,

I am trying to use WDL on my local cluster. I was trying to use WDL scatter gather parallelism. But everytime I run it I get a weird error that I have not gotten before. What makes it wierd is that it is something related with docker image which seems unlikely since both WDL tasks use the same docker image.

Attached are my WDL scripts. `scsorter_run.wdl` is the wdl script for task1. `scatter_gather_scsorter.wdl` is the second script which scatters the `run_scsorter` task across different marker gene lists and then gather task `merge_scsorter_results` merges the different output into a single object. 

The WDL pipeline runs fine till the scatter task and the necessary files are produced but then fails at the gather task and throws error in `scatter_gather_scsorter/53ff816a-8c88-40b9-be97-08470a2d5a0a/call-merge_scsorter_results/execution/stderr.submit` which looks like something related to docker image. There is no pull limit from dockerhub on my institution VPN so that seems unlikely.

`You must supply an 'application' with the -a or -app flag. See applications with 'bapp -l'
Request aborted by esub. Job not submitted.` 

I am wondering what is causing the failure? I would appreciate any insights into the same. All WDL scripts I have used so far have worked fine with my pipeline so its not the problem with cromwell config I think.

Here is the scatter gather script : scatter_gather_scsorter.wdl

version 1.0

import "../tasks/scsorter_run.wdl" as scsorter

workflow scatter_gather_scsorter{

  input {
    String docker_image
    String queue_name
    Int mem_gb
    File scsorter_script
    String Sample_name
    String rds_file_path
  File inputSamplesFile
  Array[Array[File]] inputSamples = read_tsv(inputSamplesFile)
  }

  scatter (sample in inputSamples) {
    call scsorter.run_scsorter as run_scsorter {
      input:
          docker_image=docker_image,
          queue_name=queue_name,
          mem_gb=mem_gb,
          Sample_name=Sample_name,
          rds_file_path=rds_file_path,
          scsorter_script=scsorter_script,
          marker_list_rds=sample[1],
          marker_list_name=sample[0],
  }

}

  call merge_scsorter_results {
      input: 
        docker_image=docker_image, 
        queue_name=queue_name,
        mem_gb=mem_gb,
        Sample_name=Sample_name,
        seurat_rds_path=rds_file_path,
        scsorter_runs=run_scsorter.scsorter_preds_rds
    }
}

task merge_scsorter_results {
  input {
    String docker_image
    String queue_name
    Int mem_gb
    File merge_scsorter_script
    String Sample_name
    String seurat_rds_path
    String type
    Array[File] scsorter_runs
  }
   command <<<
    Rscript ~{merge_scsorter_script} ~{seurat_rds_path} ~{Sample_name} ~{type} ~{sep=" " scsorter_runs}
    >>>
  output {
    File Seurat_merged_scsorter = glob("*.seurat_scsorter_mergedpreds.*.rds")[0]
  }
}

Here is the first task wdl : scsorter_run.wdl

version 1.0

task run_scsorter {

  input {
    String docker_image
    String queue_name
    Int mem_gb
    File scsorter_script
    String rds_file_path
    String Sample_name
    String marker_list_rds
    String marker_list_name
  }

   command <<<
    Rscript ~{scsorter_script} ~{rds_file_path} ~{Sample_name} ~{marker_list_rds} ~{marker_list_name}
    >>>

  runtime {
    docker : docker_image
    memory: mem_gb + " GB"
    queue: queue_name
  }

  output {
  File scsorter_preds_rds = glob("*_scsorter_preds_*.rds")[0]
  }
}

workflow scsorter_markerprediction {

  call run_scsorter
}

The Rscript called in `merge_scsorter_results` works fine on outside of WDL.

Comments

1 comment

  • Comment author
    Pamela Bretscher

    Hi smk_84,

    Thank you for writing in about this issue. Can you share the workspace where you are seeing this issue with Terra-Support@firecloud.org by clicking the Share button in your workspace? The Share option is in the three-dots menu at the top-right.

    1. Add Terra-Support@firecloud.org to the User email field and press enter on your keyboard.
    2. Click Save.

    Please provide us with

    1. A link to your workspace
    2. The relevant submission ID
    3. The relevant workflow ID

    We’ll be happy to take a closer look as soon as we can!

    Kind regards,

    ​Pamela

    0

Please sign in to leave a comment.