Need Help?

Search our documentation and community forum

Terra is a cloud-native platform for biomedical researchers to access data, run analysis tools, and collaborate.
Terra powers important scientific projects like FireCloud, AnVIL, and BioData Catalyst. Learn more.

Scatter: over chunks rather than for each

Comments

4 comments

  • Avatar
    Jason

    Hi Stephen,

    Thank you for your question. We'll take a look and get back to you as soon as we can!

    Kind regards,

    Jason

    0
    Comment actions Permalink
  • Avatar
    Jason

    Hi Stephen,

    We believe you should be able to do this in WDL, though you may need to take a look at the SPEC to find the WDL functions equivalent to the Python functions you mentioned. For example math.ceil(float) in Python would be ceil(float) in WDL.

    If you can find the WDL equivalents, you may be able to run the calculation right from your workflow block. Here's an example of how you can set up a calculation in a workflow block: https://github.com/broadinstitute/warp/blob/6a6cce619db2aa597927e625483fca6f83e90663/pipelines/broad/dna_seq/germline/joint_genotyping/JointGenotyping.wdl#L86

    If for whatever reason you can’t obtain a complete translation to WDL, you can indeed create a task and copy and paste your Python script into the command block, then call the task before you scatter to get Array[Array] files. This would result in a separate task, as you said. You could probably allocate a very small machine to this task if needed, though.

    If you have any further questions, please don't hesitate to let us know!

    Kind regards,

    Jason

    0
    Comment actions Permalink
  • Avatar
    Stephen Fleming

    Hi Jason,

    Okay, the example WDL you sent is very helpful.  I wasn't sure if any computation could be done in the workflow block, but the example shows that WDL standard library commands can be executed there.

    And I also realized that my particular use case actually requires a more complicated scatter anyway, so I will definitely have to use a separate task.

    Thanks for the information!
    Stephen

    0
    Comment actions Permalink
  • Avatar
    Jason

    Hi Stephen,

    Glad to hear it helped! If we can assist with anything else, please let us know. Best wishes!

    Kind regards,

    Jason

     

    0
    Comment actions Permalink

Please sign in to leave a comment.

Powered by Zendesk