How to set up a WDL on Github/Dockstore

Yashasvika Duggal
  • Updated

Learn how to upload your WDL workflow script to GitHub/Dockstore. Dockstore is a popular platform for sharing workflows. Integrating your Dockstore account with GitHub and Terra makes it easier to share and use your workflows and keep them up-to-date as they evolve. 

Step 1. Upload your WDL to GitHub

1.1. Log into github.com

1.2. Create a new public Github repository. Include a README file when creating the new repository.
Screenshot of the menu used to create a new repository on GitHub. Orange rectangles highlight the options to make the repository public and include a readme file, which are both selected.

  • Instead of creating a new repository from scratch, you can make your own version of an existing repository by forking it. This is a useful alternative if you want to adapt an existing workflow that already exists on GitHub. 

    To fork a repository, locate it on GitHub and then click the fork button at the top right of the repository's screen.

    Screenshot of an example GitHub repository. An orange box and arrow highlight the 'fork' button used to create a copy of the repository.

1.3. Upload your workflow's WDL file(s) (and optional JSON configuration files) to the repository.

1.4. Add a file named .dockstore.yml to your GitHub repository, if it doesn't already have one.

.yml files must include the following

  • version: 1.2
  • name is required if your repository contains multiple WDL files (but optional if there's only one WDL)
  • subclass: WDL is required
  • primaryDescriptorPath is required
  • testParameterFiles are optional

Example .dockstore.yml file

  version: 1.2
  workflows:
  name: example_1
    subclass: WDL
    primaryDescriptorPath:/assoc-aggregate/assoc-aggregate.wdl
    testParameterFiles:
      - /assoc-aggregate/assoc-aggregate-terra-allele.json
  name: example_2
    subclass: WDL
    primaryDescriptorPath: /assoc-aggregate/assoc-aggregate.wdl
    testParameterFiles:
      - /assoc-aggregate/assoc-aggregate-terra-position.json

Learn more about .dockstore.yml files by reading Dockstore's overview of automatic syncing between Dockstore and GitHub.

Step 2. Sync your Dockstore and GitHub repositories

2.1. If you haven't already done so, register for Dockstore at https://dockstore.org/register.

Register for Dockstore with your GitHub account Registering for Dockstore with your GitHub account makes it easier to sync up repositories across these services. Do this by selecting "Register with GitHub" when you register for Dockstore at https://dockstore.org/register.

2.2. Log into Dockstore.

2.3. Click on My Dockstore (at the upper right of any Dockstore page).

2.4. Click the plus sign to register a workflow.

2.5. Select Register using GitHub Apps, then click Next.

Screenshot showing the process used to register a new workflow from the Dockstore dashboard. An orange rectangle and the number 1 highlight the button used to register a new workflow. Another orange rectangle and the number 2 highlight the option to Register using GitHub Apps in the resulting popup window.

2.6. On the screen, click Manage Dockstore installations on Github. You'll be taken to a GitHub page to install the Dockstore app in a specific repository. 

Screenshot of a popup window with a button labeled 'Manage Dockstore installations on GitHub'.

2.7. If prompted, select your GitHub account and log in.

2.8. In the Repository access section, select Only select repositories and use the select repositories drop-down menu to choose the repository with your workflow files.

What to expect

It may take a few minutes for Dockstore to sync with your GitHub repository. After a few minutes, check on the sync progress by navigating to the Workflows section of your Dockstore dashboard (select the Workflows icon in the left-hand panel):

Screenshot of the Dockstore dashboard for an example user. The 'Workflows' button on the left-hand panel is highlighted in orange.

Under the "GitHub" section on the left-hand side of the screen, select your organization (e.g., your GitHub account handle). Your workflow should be listed in this section:

Screenshot of the workflows section on an example user's Dockstore dashboard. An orange box highlights the user's GitHub handle, which contains several workflows synced from this GitHub account. Another orange box highlights one of the workflows listed in this section, which has been successfully synced from the user's GitHub repository.

What if you don't see your workflow? If you don't see your workflow listed where you expect to see it, don't panic! It may just be taking longer than you expect to sync, or you might need to dig around to uncover possible sync errors. Follow these tips to track down your workflow:

1. Make sure that you've selected the correct organization (or GitHub account) in the GitHub section.
2. If you don't see your workflow in the Published tab, look under the Unpublished tab.
3. If you still don't see your workflow, click on Refresh Organization
Screenshot of the GitHub section of the workflows tab of an example user's Dockstore dashboard. Orange rectangles highlight the 'refresh organization' and 'app logs' buttons.
4. If you still don't see your workflow, click on Apps Logs. Here you should see a row for your sync job. If it says "Failed", click to expand the log and see an error message. For example, the screenshot below shows an example where the sync job failed because the .dockstore.yml file did not specify a primaryDescriptorPath. 
Screenshot of an example log for Dockstore-GitHub synchronization jobs. An orange rectangle highlights the log for a failed push to Dockstore, which has been expanded to reveal an error message.

Step 3. Publish your Dockstore repository

When you create a new repository on Dockstore it won’t be published by default. Follow these steps to publish your workflow, making it discoverable by any Dockstore user.

3.1. Select the workflow from the Workflows section of your Dockstore dashboard.

3.2. Go to your workflow's Versions tab and use the Actions button to select a Default version.Screenshot showing how to set the default version for an example workflow. Orange boxes highlight the workflow's 'Versions' tab, the 'Actions' drop-down menu, and the 'Set Default Version' option

3.3. Click the blue Publish button at the upper right of the screen to make it publicly viewable and exportable to Terra.

Step 4. Export your workflow to Terra

4.1. Select the workflow from the Workflows section of your Dockstore dashboard.

4.2. Select View public page from the top right of the screen.

4.3. Click the Terra button from the Launch with section at the right-hand side of the screen.

Screenshot of the page used to export a workflow from Dockstore to Terra.

You will be taken to a Terra page, where you can select the workspace to which your workflow will be exported.

Was this article helpful?

1 out of 1 found this helpful

Comments

0 comments

Please sign in to leave a comment.