Working with workspaces

Anton Kovalsky

Workspaces are the building blocks of Terra - a dedicated space where you and your collaborators can access and organize the same data and tools and run analyses together.

Learn how to set up and use everything you need to collaborate in a Terra workspace.

Workspaces: All your analysis needs in one place

Your Terra workspace can contain data, metadata, and analysis tools, as well as documentation and a record of all workflow submissions. Each distinct component has its own page (see screenshot below), which you can access by clicking the tab at the top of any page. Expand the sections below for more details about how to access the resources you need. 

Screenshot showing an example workspace's tabs, which each allow different functions: Dashboard, Data, Analyses, Workflows, and Job History.

  • The landing page (i.e., Dashboard) is your project overview - the questions you’re trying to answer, the data and analysis tools you'll use, etc. Good documentation makes your analysis easy to share (with others, as well as with your future self) and reproduce.

    Editing the dashboard (in the Markdown language)

    Click the pencil icon to the right of the "About the Workspace" header at the top to edit. The dashboard uses the Markdown language, which lets you organize with headers and include links and additional references. 

    To learn more about Best Practices for documenting in a dashboard, see Documentation best practices.

    Screenshot of an example workspace's Dashboard tab. An orange arrow highlights the pencil icon, used to edit the description displayed on the dashboard.

    Useful workspace details are populated automatically in the right column of the Dashboard (scroll down for screenshots). Expandable sections include:

    Workspace information

    The workspace creation date, date last updated, workflow submissions, and your access level.

    Cloud information

    The cloud infrastructure, location of workspace storage, Google Project ID, workspace storage ID, estimated storage cost and size. Here is where you can open the workspace storage file system (Google bucket structure in Google Cloud console).

    Workspace owners

    If you need access to the workspace, ask the workspace owners. 

    Workspace tags

    Only visible to owners, tags are useful for searching and indexing. 

    Screenshot showing the Workspace Information section displayed on an example workspace's dashboard.

    Screenshot showing the tags section displayed on an example workspace's dashboard.

    Screenshot showing the Cloud Information section displayed on an example workspace's dashboard.

    Screenshot showing the Owners section displayed on an example workspace's dashboard.

  • Each workspace comes with its own storage (Google bucket) where data generated by a workflow analysis as well as interactive analysis files (i.e., notebook.ipynb and RStudio.Rmd files) are stored by default.

    Additional workspace storage options (advanced)Storage classes
    All newly created GCP workspace buckets will have Autoclass turned on by default. Autoclass automatically moves data to colder storage classes to reduce storage costs using a predefined lifecycle policy. There are no early deletion charges, no retrieval charges, and no charges for storage class transitions. For more information, see Google's documentation on Autoclass.

    Storage region (location)

    You can choose a specific region for your workspace storage when you create the workspace. See Working with non-US data in Terra for more information

    To access the dedicated workspace storage

    From the Dashboard

    Select the Open in browser link in the Cloud Information section.

    or

    From the Data page

    Click on the Files icon at the bottom of the left panel. 

    Manual upload (best for a small number of small files)

    Option 1: In Google Cloud console

    Clicking the Open bucket in browser link in the Cloud Information section of the Dashboard will take you to the Google Cloud console, where you can upload smaller files from your local machine by clicking or dragging. 
    alt

    Option 2: In Terra

    Clicking the Files icon from the Data page will display the Workspace bucket file structure in the UI. You can upload by 1) going to the Data tab, 2) clicking the Files icon (bottom of left column) and 3) selecting the "upload" button at the top right.
    Screenshot illustrating how to upload files to a Terra workspace through the workspace's 'Data' tab. Orange boxes, arrows, and numbers highlight the Data tab (1), the Files section (2), and the Upload button (3).

    gsutil (best for large numbers of files and/or large files)

    You can also use gsutil in a terminal to copy data from a local machine or other cloud storage. To learn more, see Using the terminal and interactive analysis shell.

  • Like spreadsheets built right into the workspace, data tables help keep track of all project data, no matter where files are stored in the cloud. This becomes especially useful as the number of participants or samples in your study grows.

    Genomic data

    Screenshot of genomic data in a sample table

    Phenotypic data

    Screenshot of phenotypic data in a subjeect table

    Tables spare you from copying/storing input data for a workflows analysis

    In Terra, you can analyze data stored in the cloud without copying files to the workspace bucket. Workflows can input data using links to the data's actual location in the cloud from the table. And you can even write links to the generated files to the input table to associate it with the original. 

    Learn how to combine data from different studies or across datasets in a single table in this video.

  • Interactive analysis - built into your workspace

    Interrogate and visualize your data in real time using Galaxy, Jupyter Notebooks, or RStudio, Terra's integrated interactive analysis apps. All three apps run on virtual machines or clusters of machines in a workspace Cloud Environment.
    Screenshot of the Analyses tab

    Interactive app resources

  • Collect, configure (set up) and run workflows for bulk analyses from the Workflows page. Workflows are the sorts of repetitive analyses that can be automated, such as aligning sequencer reads or calling variants. You can set up and run a workflow by clicking on the workflow name in the card. Many options for saving costs - such as using call caching, checkpointing, or preemptibles - are available in Terra.

    Screenshot showing the workflows available in an example workspace's Workflows tab.

    Finding the workflow you need

    Not a coding expert? Browse and import published workflows in Dockstore or the Broad Methods Repository by selecting the "Find a Workflow" card from the Workspaces page.

    Screenshot showing links to Dockstore and the Broad Methods Repository, from which you can import pre-written workflows into your Terra workspace.

  • The Job History page is where you can check on the status of all current and past workflow submissions. 

    Troubleshooting

    You can troubleshoot failed flows by selecting the workflow name in the "Submission" column at the left.

    Read more in How to troubleshoot failed workflows.

    Screenshot showing the contents of an example workspace's Job History tab, including a failed workflow. An orange box and orange arrow highlight the failed workflow's row, which is shaded red.

    Error logs

    See error messages (by hovering over the failed icon) and access further information (including error and log files) by clicking on the icons at right in the Submissions details page.Screenshot showing the submissions details page for an example workflow. An orange box and arrow highlight the icons listed in the 'links' column for one submission in the workflow.

  • To collaborate, you can "share" the project workspace with all the data, tools and generated data. Workspace owners control how much access collaborators have to resources, including funding, by assigning roles with different permission levels.

    Share-workspace_screen-capture.gif

    Learn more in Managing shared resources with groups and permissions.

Building workspaces using the Terra Library

Terra has three libraries that can help when building a workspace: data, showcase workspaces, and tools (workflows). To access the libraries, click the main menu icon (three horizontal lines) at the top left of any page and open the "Library" submenu.  

Screenshot of Terra Libraries in the main menu

To learn more about using the Terra Data Library to build your workspace, see Build a workspace using data, showcase, and tools libraries.

Was this article helpful?

Comments

1 comment

Please sign in to leave a comment.