This article outlines what operations in Terra have a cloud cost and how you will pay for Azure cloud charges using a linked Azure subscription. It explains how Terra and Azure billing are related (i.e., Terra Billing projects act as pass-throughs to cover cloud costs of your work in Terra), how cloud costs grow as you scale your work in Terra, and how Azure calculates and bills Terra users for charges (broadly).
What has a cost (Azure Cloud fee) in Terra
The Terra platform itself is a cloud-native system that does not charge administration or support fees. You pay only for the resources you use, including infrastructure costs for resources needed to power your group's Terra environmentT and cloud costs associated with operations in Terra.
Cloud costs passed through via a Terra Billing project
| Terra Environment | Your organization’s isolated resources that power workspaces | Starts to accrue when you create a Terra Billing project (currently starts at $10/day) and increases when more workspaces are created. |
| Data storage | Workspace storage (blob storage container) |
Created when you create a workspace. Cost based on usage (see Azure blob storage pricing). |
| Analysis (compute and disk) |
Running analysis apps (JupyterLab) | Accrues when VM is running. Cost based on usage (see Azure Cloud compute and Azure Cloud disk rates). |
| Running workflows | Accrues when workflow is running (see Azure Cloud compute and Azure Cloud disk rates). | |
| Egress | Copying data from other cloud vendors (Google or AWS) or from other regions within Azure | Cost based on usage (see Azure Cloud bandwidth pricing). |
Terra passes along Azure cloud charges with no markup (the links above will take you directly to the Azure pricing pages).
How costs scale
Creating a Terra Environment incurs a base cost to cover infrastructure resources. Base Terra Environment infrastructure shared across all workspaces in the environment includes fixed resources for logging, a database server, and a storage account. The Terra Environment also includes scalable resources to support workspaces.
Workspace resources scale in tiers
How many workspaces you can add until bumping up to the next tier depends on the workspace size - how many services, engines, and tools each workspace accommodates. You can fill your available workspace resource per tier with several smaller workspaces or fewer larger workspaces.
Terra Environment cost tiers
The Terra Environment powers all of the infrastructure needed to create and use Workspaces. Workspace resources include services and engines. Workspace services power data tables and workflow management, and engines execute specific workflow languages (such as Cromwell).
Each cost tier will accommodate a greater number of "smaller" workspaces (with fewer services running) - or fewer "larger" workspaces (with many services running).
Workspace "size" increases with
- Number of services running (data tables, workflow management, workflow engines)
- Number of engines (two users each running Cromwell is larger than one, for example)
How to reduce cost
You can reduce costs by keeping workspace resources in a lower-cost tier.
- Use fewer workspaces
- Delete unnecessary workspaces
- Delete unnecessary files from storage
Note that currently, deleting workspace services such as Cromwell and Workspace Data Services requires deleting the workspace.
How you are billed for work in Terra
There are two categories of cloud costs in Terra: infrastructure costs and working costs (cloud resources you use for data storage, egress, and analysis).
Below is more information about Terra/Azure billing structure to guide you in controlling costs. It is a top priority for us to reduce these costs - especially fixed infrastructure costs - in the near future.
Terra Environment infrastructure cost (per Billing project)
When you create a Terra Billing project, you create your own Terra Environment. Behind the scenes, Terra sets up your organization’s isolated infrastructure resources for powering workspaces, which are shared across all workspaces under a single Billing Project umbrella. This cloud framework is needed to build workspaces and run analysis apps.
You pay a fixed price for these infrastructure resources, which increases in increments (currently, the first tier is about $10/day per Terra Billing Project).
Fixed infrastructure costs accrue as long as you have a Terra Billing project
To remove Terra Environment costs - delete a Terra Billing Project
From the Billing page, click the trash can icon to the right of the Billing project. Note that you must be an owner to delete the Billing Project, so it will be listed under the Owned by You section at the left.
Deleting the Terra Billing project is the only way to eliminate the shared infrastructure costs. It is a top priority for us to reduce this cost in the near future.
Data storage, compute and egress costs (workspace-level)
You pay for cloud resources you use, such as data storage (unstructured and tabular data), analysis compute and disk (i.e., virtual machines or VMs), and egress (copying between cloud providers or between regions within a single cloud provider).
Costs for the cloud resources you use are calculated per workspace (not per user)
All work with a cost in Terra is done in these computational sandboxes. The costs for the cloud resources you use (storage, analysis compute and disk, and data egress) are covered by a linked Azure subscription by way of the Terra Billing project associated with the workspace.
You can think of paying for work in Terra much like paying for electricity in your house: the owner (or whoever's name is on the electric bill) is responsible for all the electricity used, not who uses it. As with your electric bill, it's not even possible in Terra to directly identify who is responsible for the cloud charges. For this reason, you will want to be careful when sharing a workspace, as any colleague with permission to spend can accrue cloud costs in the workspace.
Why is Terra’s Azure cost model different from Google's?
If you're familiar with Terra on Google, you may notice that the cost model for Terra on Azure is different. One significant difference is that some Terra-owned infrastructure has been shifted to user-owned in Terra on Azure. This change allows Terra on Azure to be more scalable and flexible, and gives billing project owners and workspace owners more control to maintain the isolation and security of your data. It does come with some added costs.
What to expect
When you create a billing project or workspace, Terra creates fixed infrastructure resources to run Terra on Azure. The billing project resources are shared across workspaces and applications within a workspace. The workspace infrastructure powers the workspace data tables. In Google, Terra owns and manages some of these resources for you, but in Azure, everything is managed within your Azure subscription, giving you maximum control over your data and resources.
We are working to provide more cost-effective infrastructure options in the near future.
Cost monitoring (Terra on Azure)
Terra Billing project spend reporting
Billing project owners can see the total spend, total compute, and total storage costs for their billing projects on the Billing page (https://app.terra.bio/#billing).
JupyterLab costs
Terra estimates the costs of running JupyterLab in a workspace cloud environment (see screenshots below). Note that these are estimates only of the costs of the VM compute (billed by the hour) and Persistent Disk disk (billed by the month).
In the environment details pane
|
In the sidebar
|
These cost estimates are also available on the Cloud environments page (app.terra.bio/#clusters).
The VM compute cost (per hour) is indicated with an arrow at the top (screenshot below). The disk cost (per month) is at the bottom.
Azure cost reporting
Currently only those with access to the subscription in the Azure portal can monitor detailed and exact costs as reported by Azure. Cost reports are based on the application deployed within the subscription.
For more details, see the Microsoft documentation Get started in cost analysis.