Overview: Automate workflows with the FISS API

Lin Betancourt
  • Updated

Learn how to build automated workflows, saving time and effort using the FISS API.

Use case: Automating workflows with the FISS API

Manually setting up and running 5 workflows is trivial, but if you had to run that same workflow on 100,000 whole genome samples, setting up manually is impossible. Scripting the process makes the task tractable, saving you time and effort, especially when you want to repeat the same analysis over and over again. You can use this approach to build command line scripts written in Python, or even Jupyter Notebooks, that help you with your day-to-day work.

With the FISS software library you can build automation - set up a workspace, move and manage data, and perform an analysis - but through Python scripts that you run on your computer or in a Jupyter Notebook. Leveraging the library lets you perform many of the same actions in the graphical Terra interface, but interacting more directly using scripting enables automating and scaling. 

FISS API tutorial workspace

Click here to go to the FISS tutorial workspace.

Want to know when the tutorial will be updated?  Click the "Follow" button at the top right of the article.

The tutorial workspace will include use cases with sample code for times when you might want to go beyond Terra's graphic interface. Notebooks within the workspace will highlight different examples of when you might find scripting particularly helpful. 

FISS tutorial notebook 1: Create a workspace and add data in a Terra workspace

Python script in a notebook will help
- Create the workspace
- Upload data to your workspace Google bucket
- Upload data pointers and metadata to the workspace
- Use the data model in a Python script or notebook

FISS tutorial notebook 2: Upload data and generate a sample data table

Python script in a notebook will help
- Upload genomic data files (CRAMs and CRAIs) to the workspace bucket
- Generate a sample table in the Data tab programatically
- Avoid errors that can occur when manually creating a TSV file 

FISS tutorial notebook 3 (coming soon): Automate and scale workflows 

Python script in a notebook will help
- Import and configure a workflow
- Launch and monitor the workflow analysis
- Debug your analysis 

Resources: Using the FISS library

See additional example notebooks (from the Biodata Catalyst Collection workspace). Links below are to the original Biodata Catalyst Collection. 
- Intro to FISS API in Python
- Intro to Fiss API in R
- terra-data-table-util

You must use the`model = flexible parameter` in any FISS commands for Gen3 data in Terra! Otherwise you are constrained to `sample` and `participant` data tables.

Example code (flexible model)

fapi.upload_entities_tsv(billing_project, workspace_name, 
'FISS-tutorial_subject.tsv', model='flexible')

Was this article helpful?

0 out of 1 found this helpful

Comments

0 comments

Please sign in to leave a comment.