multiple workflow configs to select among
Some of our workflows should have configurations for running with hg19 or hg38, and then each also has different sub-configurations with regard to input types available (fastqs or bams). It would be useful to have different named configurations available to users that could be selected among when running a workflow.
Currently, we appear to be restricted to just a single configuration that automatically loads when a workflow is selected.
Comments
7 comments
Hi Brian, you can actually duplicate and then edit a workflow configuration, to have different coexisting configs in the same workspace. See it in action here: https://app.terra.bio/#workspaces/help-gatk/Genomics-in-the-Cloud-v1/workflows. I use this to show how to run a workflow either with file paths or with the data model, but you could use this in the way you describe.
Thanks, Geraldine! This is a useful work-around for now. I think it would be better to have multiple configs available at some point so that you don't need to have two independent but identical workflows and having to maintain them separately (ie. separate snapshots to manage, having to change two workflows in terra any time you want to update to the next version, etc.). If we end up having 4 different configs (ie. 2 genomes supported x 2 input data types), we'd end up having to manage 4 separate workflow instances, which adds sufficient overhead in the SDLC.
Indeed, very fair points. FWIW some of our collaborators use Github Actions to maintain public workspaces that have complex content, which allows them to automate publishing updated versions, triggering test runs etc.
Anyway I'll let the team respond from the point of view of handling the feature request as such :)
oooh.... Github Actions used this way would be very useful to us. If it's possible for me to look through some example of how they do this, that would be hugely helpful. TIA!
I can only find the git actions workflow for publishing notebooks and dashboard markdown at the moment, but I could swear I heard this was in use for workflows too. Will keep digging but perhaps this can be a helpful starting point.
https://github.com/DataBiosphere/terra-example-notebooks/blob/master/.github/workflows/publish-to-terra.yml
awesome! thanks, Geraldine!! :-)
Thanks for writing in with this request, Brian. And thank you, Geraldine, for the great advice for possible workarounds in the meantime! We can definitely see the benefit of having multiple configurations available per workflow. I'll flag this up with our product team and reach out to you if the feature gets implemented in Terra!
Jason
Please sign in to leave a comment.