Create, edit, and share a new workflow

Updated November 23, 2021 20:26
1 comment

Creating or editing a new Workflow in Terra currently requires working in the "Broad Methods Repository." This article includes step-by step instructions for how to create a new workflow, edit a workflow, and share or clone a workflow, as well as a Glossary of Terms.

1. Create a new workflow

1.1. Navigate to the Workflow tab in your workspace and click on the "Find a Workflow" card:

Screen_Shot_2019-07-29_at_10.21.32_AM.png

1.2. From the popup, click on the Broad Methods Repository at the bottom, under "Find Additional Workflows":

1.3. You will be redirected to the Broad Methods Repository, where you can click on the blue button at the top right corner to "Create New Method. +":

Screen_Shot_2019-07-29_at_10.26.53_AM.png

1.4. Clicking the 'Create New Method.. +" will open up a modal to set up your new Workflow
You will enter your Namespace* and the Name of your new Workflow followed by your WDL (enter manually or with "Load from file..."). Workflow names can include letters, numbers, underscores, dashes, and periods. Press "Upload" once you've filled in all the information.

Screen_Shot_2019-07-29_at_10.32.40_AM.png

Glossary of terms (expand)

	Glossary
	Namespace* acts like a folder containing workflow and workflow configs. For example, if you add the workflow "ExampleNewWorkflow" to the Broad Methods Repository under the namespace "SupportDocs", its identifier would be: SupportDocs/ExampleNewWorkflow Many analysts choose to put their name and/or institution in the namespace field. You can label your namespaces however you see fit, provided the chosen namespace does not already exist. Administrators may verify namespaces, which like verified twitter accounts, establish the authenticity of namespaces attached to specific organizations (e.g., Broad Institute). You can permit others to use your namespace. This is great for labs who have multiple contributors but want people to find all of their methods under the "Miller-lab", for example. In this case, we may add the namespace "SupportDocs" to any workflows that we create for the purpose of documents. Name is the name of the specific workflow you are creating. This can be any name as long as it does not already exist. In this example, the workflow is called "ExampleNewWorkflow" Documentation is where you can add information about the WDL and what it does. We encourage developers to add descriptive information such that if you were to share this with another user, they are able to recognize the inputs and outputs as well as any other required information they may need to run the workflow successfully. Snapshot is a numerical value assigned to the WDL similar to a versioning system. From the WDL section, you can make edits directly to the script and each time you press Upload, you create a new Snapshot.

Glossary

Namespace* acts like a folder containing workflow and workflow configs. For example, if
you add the workflow "ExampleNewWorkflow" to the Broad Methods Repository under the
namespace "SupportDocs", its identifier would be: SupportDocs/ExampleNewWorkflow

Many analysts choose to put their name and/or institution in the namespace field. You can
label your namespaces however you see fit, provided the chosen namespace does not
already exist. Administrators may verify namespaces, which like verified twitter accounts,
establish the authenticity of namespaces attached to specific organizations (e.g., Broad
Institute).

You can permit others to use your namespace. This is great for labs who have multiple
contributors but want people to find all of their methods under the "Miller-lab", for
example.

In this case, we may add the namespace "SupportDocs" to any workflows that we create
for the purpose of documents.

Name is the name of the specific workflow you are creating. This can be any name as
long as it does not already exist. In this example, the workflow is called
"ExampleNewWorkflow"

Documentation is where you can add information about the WDL and what it does. We
encourage developers to add descriptive information such that if you were to share this
with another user, they are able to recognize the inputs and outputs as well as any other
required information they may need to run the workflow successfully.

Snapshot is a numerical value assigned to the WDL similar to a versioning system. From
the WDL section, you can make edits directly to the script and each time you press
Upload, you create a new Snapshot.

1.5. Once you've finished uploading, you'll be directed to your new workflow
You can "Export to Workspace..." from the top right-hand corner:

Screen_Shot_2019-07-29_at_10.39.48_AM.png

1.6. The configuration lets you set inputs and save them to use again and again
You can select a configuration if one already exists or use a blank configuration:

Screen_Shot_2019-07-29_at_10.40.11_AM.png

1.7. Choose the destination workspace
This is the Terra workspace you were working in and would like to export your Workflow to. You should be prompted to go to the destination workspace, which will redirect you back to your Terra Workflows tab where you should see your new workflow:

Screen_Shot_2019-07-29_at_10.40.44_AM.png

Use FISS to upload to the Methods Repository (step-by-step)

The(FI)reCloud (S)ervice (S)elector is another way to upload workflows to the Method Repository using scripting rather than the UI. You can use it from your terminal, following the instructions below.

1. Install FISS
You'll find instructions to do this in the readme of the FISS repo https://github.com/broadinstitute/fiss

2. To see the command line format and optional variables
Run fissfc meth_new

3. To put a WDL into a certain namespace with a method_name
Run the following

fissfc meth_new -m method_name -n method_namespace -d path/to/local/wdl/file [-s desc of method] [-doc optional documentation] [-c optional snapshot comment]

Note that the -m -n and -d inputs are required.

2. Edit a workflow in the Broad Methods Repository

2.1. Open your workflow in the Broad Methods Repository (Summary tab):

Screen_Shot_2019-07-29_at_10.44.29_AM.png

2.2. Press "Edit..." from the left-hand bar
This should bring up the same modal used to create your workflow:

Screen_Shot_2019-07-29_at_10.45.22_AM.png

2.3. You can edit directly in the WDL section window
The program will automatically recognize syntax errors when you press Upload. You can also change the snapshot comment depending on the changes that you made.

2.4. Press "Save as New Snapshot"
You can see that there are now 2 Snapshots. You will not need to export this workflow again to Terra. Just go to the Workflow tab to toggle between the snapshots when you click into the workflow.
Screen_Shot_2019-07-29_at_10.46.59_AM.png

3. Share or clone a workflow

You can duplicate (clone) a workflow within your workspace or export to a new workspace following these steps:

3.1. Go to the Workflows tab of your Terra Workspace

3.2. Click the three button icon for the workflow you want to share

3.3. Select "Duplicate" to clone the workflow within the workspace or select "Copy to Another Workspace" to copy (clone) to a different workspace

Was this article helpful?

1 out of 2 found this helpful

Have more questions? Submit a request

Comments

1 comment

Roger Dettloff
- June 04, 2020 00:32
Hi Sushma Chaluvadi
I found this article helpful, but it left me wondering how to upload a more complex WDL script that imports relative dependencies. The Cromwell server is said to allow us to submit a ZIP file containing workflow source files that are used to resolve local imports. This zip bundle will be unpacked in a sandbox accessible to this workflow. See here: https://cromwell.readthedocs.io/en/stable/api/RESTAPI/#submit-a-workflow-for-execution

It it possible to submit a zip bundle to describe a workflow to Terra?

2

Please sign in to leave a comment.