Working with workspaces

Anton Kovalsky
  • Updated

Workspaces are the building blocks of Terra - a dedicated space where you and your collaborators can access and organize the same data and tools and run analyses together.

This article outlines project components and how to set up and use them to collaborate in a Terra workspace.

Workspaces: All the study components you need in one place

You can use a Terra workspace to keep everything for your study together in one place - data, metadata and analysis tools, as well as documentation and a record of all workflow submissions. Each distinct component has its own page (see screenshot below) which you can access by clicking the tab in the top of any page. Expand the sections below for more details about how to access the resources you need. 

Workspace-Dashboard_Screen_shot.png

  • The landing page (i.e. Dashboard) is your project overview - the questions you’re trying to answer, the data and analysis tools you'll use, etc. Good documentation makes your analysis easy to share (including with your future self) and reproduce.

    Editing the dashboard (in the markdown language)

    Click the pencil icon to the right of the "About the Workspace" header at the top to edit. The dashboard uses the markdown language, which lets you organize with headers and include links and additional references. 

    To learn more about Best Practices for documenting in a dashboard, see Documentation best practices.

    Workspace-Dashbord_How-to-edit_Screen_shot.png

    Useful workspace details are populated automatically in the right column of the Dashboard (scroll down for screenshots). Expandable sections include:

    Workspace information

    The workspace creation date, date last updated, workflow submissions, and what access level you have.

    Cloud information

    The cloud infrastructure, location of workspace storage, Google Project ID, workspace storage ID, estimated storage cost and size. Here is where you can open the workspace storage file system (Google bucket structure in GCP console).

    Workspace owners

    Who to ask if you need to gain access to the workspace. 

    Workspace tags

    Only visible to owners, tages are useful for searching and indexing. 

    Dashboard_Workspace-Information_Sreen_shot.png

    Dashboard_Tags_Screen_shot.png

    Dashboard_SCloud-Information_Screen_shot.png

    Dashboard_Owners-section-Screen_shot.png

  • Each workspace comes with its own storage (Google bucket) where data generated by a workflow analysis as well as interactive analysis files (i.e. notebook.ipynb and RStudio.Rmd files) are stored by default.

    Additional workspace storage options (advanced)Storage classes
    All Terra buckets are Standard storage class buckets. We may support Nearline Coldline, and Archive storage classes in the future, but they are not available at this time.

    Storage region (location)

    You can choose a specific region for your workspace storage when you create the workspace. See Working with non-US data in Terra for more information

    To access the dedicated workspace storage

    From the Dashboard

    Select the Open in browser link in the Cloud Information section.

    or

    From the Data page

    Click on the Files icon at the bottom of the left column. 

    To upload your own data (small numbers, small files)

    Option 1: In GCP console

    Clicking the Open bucket in browser link in the Cloud Information section of the Dashboard will take you to the Google Cloud Platform (GCP) console, where you can upload smaller files from your local machine by clicking or dragging. 
    Workspaces_Upload-data-via-GCP-console_Screen_shot.png

    Option 2: In Terra

    Clicking the Files icon from the Data page will display the Workspace bucket file structure in the UI. You can upload by 1) going to the Data tab, 2) clicking the Files icon (bottom of left column) and 3) selecting the "+" icon at the bottom right.
    Start-up-script_How-to-upload-to-workspace-storage_Screen_shot.png

    Large numbers and/or large data files

    You can also use gsutil in a terminal to copy data from a local machine or other cloud storage. To learn more, see Using the terminal and interactive analysis shell.

  • Like spreadsheets built right into the workspace, data tables help keep track of all project data no matter where in the cloud the files are stored. This becomes especially useful as the number of participants or samples in your study grows.

    Genomic data

    Screenshot of genomic data in a sample table

    Phenotypic data

    Screenshot of phenotypic data in a subjeect table

    Tables keep you from copying/storing input data for a workflows analysis

    In Terra, you can analyze data stored in the cloud without copying files to the workspace bucket. Workflows can input data using links to the data's actual location in the cloud from the table. And you can even write links to the generated files to the input table to associate it with the original. 

    Learn how to combine data from different studies or across datasets in a single table in this video.

  • Interactive analysis - built into your workspace

    Interrogate and visualize your data in real time using Galaxy, Jupyter Notebooks, or RStudio, Terra's integrated interactive analysis apps. All three apps run on virtual machines or clusters of machines in a workspace Cloud Environment.
    Screenshot of the Analyses tab

    Interactive app resources

  • Collect, configure (set up) and run workflows for bulk analyses from the Workflows page. Workflows are the sorts of repetitive analyses that can be automated, such as aligning sequencer reads or calling variants. You can set up and run a workflow by clicking on the workflow name in the card. Many options for saving costs - such as using call caching, checkpointing, or preemptibles - are available in Terra.

    Workflows-components_Workflows-page_Screen_shot.png

    Finding the workflow you need

    Not a coding expert? Browse and import published workflows in Dockstore or the Broad Methods Repository by selecting the "Find a Workflow" card from the Workspaces page.

    S52j_Workspaces_suggested_workflows_Screen_Shot.png

  • The Job History page is where you can check on the status of all current and past workflow submissions. 

    Troubleshooting

    You can troubleshoot failed flows by selecting the workflow name in the "Submission" column at the left.

    Read more in Troubleshooting Workflows: Tips and Tricks.

    Job-History-page_Failed-submission_Screen_shot.png

    Error logs

    See error messages (by hovering over the failed icon) and access further information (including error and log files) by clicking on the icons at right in the Submissions details page.Submissions-details-inJob-History_Screen_shot.png

  • To collaborate, you can "share" the project workspace with all the data, tools and generated data. Workspace owners control how much access collaborators have to resources, including funding, by assigning roles with different permission levels.

    Share-workspace_screen-capture.gif

    Learn more in Managing shared resources with groups and permissions.

Building workspaces using the Terra Library

Terra has three libraries that can help when you are building a project workspace. To access the libraries, click the main menu icon (three horizontal lines) at the top left of any page and open the "Library" submenu.  

Screenshot of Terra Libraries in the main menu

To learn more about using the Terra Data Library to build your workspace, see Build a workspace using data, showcase, and tools Library resources.

Was this article helpful?

2 out of 4 found this helpful

Have more questions? Submit a request

Comments

0 comments

Please sign in to leave a comment.