Terra expenses and breaking down a Google bill

Anika Das
  • Updated

All Google Cloud Platform (GCP) costs in Terra are calculated per workspace and paid via the Terra Billing project assigned to the workspace when it's created. Find out how to interpret your Google bill and optimize your spend by aligning the costs you see (in GCP console) with actions performed in Terra. This document includes information on the actions you can perform in Terra, which ones cost money, and which are free of charge.

Please reference Understanding and controlling Cloud costs as a companion document to this one.

 

Contents

Workspace infrastructure costs
Data storage costs
  - Workspace bucket
  - Cloud Environment Persistent Disk
Interactive Analysis costs
Workflows costs
How to view Google Cloud costs (on GCP console)
Use case examples
- Joint Genotyping workflow
- Creating a cloud environment
Pricing breakdown by SKU


Workspace infrastructure costs

This section describes workspace costs separate from data storage and analysis costs (see costs for  data storage, and running an interactive analysis or workflow, below).

Viewing Workspaces

Viewing workspaces is free of charge, and you can browse Showcase workspaces without creating a Terra account. Viewing other public (non-Showcase) workspaces or the data library requires you to register for a free Terra account. See instructions for setting up an account on Terra here

Cloning Workspaces

Cloning or creating workspaces in Terra is free of charge, although you will need to have access to a Terra Billing project to assign to the new workspace. When you do work in a workspace, all GCP costs will be paid through the assigned billing project.

You will pay for .ipynb  files
Note that if you are cloning a workspace with notebooks, the notebook files (i.e. ipynb files) are automatically copied along with the workspace and stored in the new Workspace bucket. You are charged for the storage of these copied files (See the section on the Workspace Bucket below). Other bucket contents, such as workflow outputs and uploaded data, are not copied to a cloned workspace.

Storing and maintaining Data Tables

What is free to use
Data tables are spreadsheet-like files integrated into the workspace and workflows. Looking at the details of a file in a data table is free. Copying data tables to a different workspace is also free. In most cases, downloading a file from Terra is free and the costs for download are covered by the billing project assigned to the workspace where the data files are stored (in the workspace bucket). 

What has a cost
Downloading files can incur costs for you if the Workspace bucket is a requester pays bucket. Workspaces with requester pays buckets will prompt you to select a workspace to charge for accessing the data (the egress charges will be covered by the workspace Terra Billing project).

Requester-pays_Choose-workspace-to-bill-dropdown_Screen_shot.png


Data storage costs

Workspace bucket

Storage in a Workspace bucket has a cost that varies depending on how much of the bucket you are using. Charges accrue daily, but Cloud Storage bills you only at the end of the billing period. You can view unbilled usage in your workspace project's billing page in the Google Cloud Console. For more detailed information, see Google Cloud Documentation.

What has an indirect cost (i.e. data egress charges)
Additionally, if someone views or downloads the contents of files in your workspace bucket, the workspace Billing project may accrue egress charges if the data moves outside of the bucket Cloud Storage region(s). For more details, see https://cloud.google.com/storage/pricing

What is free
Uploading
data to a workspace bucket (for example through the "Files" section of a workspace) does not incur upload costs. However the new data in the bucket will start to accrue storage fees immediately.
 

If you upload a workspace data table (i.e. TSV or TXT file), you will not pay for anything related to the TSV or any of the file paths in your table. You are only responsible for the storage costs of files in your Workspace bucket. You can reference files from an external bucket in your data table without paying, unless you copy files to your Workspace bucket or Cloud Environment persistent disk, or use them as input in a workflow analysis.

Default bucket  regions and cost
The default for Terra buckets is multi-region US standard storage, but other options are available. You can find the region(s) as well as an estimated monthly storage cost (Est. $/Month) in the Workspace Information in the right column of the Dashboard.

Screen_Shot_2021-10-06_at_2.41.00_PM.png

Closeup of Workspace Information
mceclip1.png

For more information on Google Cloud Storage pricing, see  https://cloud.google.com/storage/pricing.

Cloud Environment Persistent Disk

Cloud Environments come with an (optional) detachable persistent disk. When you create a Cloud Environment using the default options, you automatically get 50G of VM storage (the persistent disk) attached. You can choose a different persistent disk size in the Cloud Environment setup form. You pay a fixed amount per month for the disk you choose, until you delete it (i.e. whether or not the Cloud Environment VM is running). The cost for the default 50G disk is $2.00/month. 

 

Interactive Analysis costs

Interactive analysis apps such as Jupyter notebooks, Galaxy, RStudio or the workspace terminal all run on the Cloud Environment VM.  You can start a Cloud Environment by clicking the Cloud Environment widget at the top right of any workspace page.  You'll see a  configuration form where you can choose the environment (installed packages and software) and compute profile.

mceclip0.png

What has a cost

A Cloud Environment virtual machine (VM) has 3 separate billable components: a VM, a boot disk, and a detachable persistent disk. When creating a Cloud Environment, you can see the cost per hour based on the CPUs, Memory(GB), and disk size you choose. As you increase CPU, memory, or storage, the cost of the Cloud Environment will increase.

You will pay the VM cost/hour as long as the VM is running, whether or not it is performing calculations! Terra has a built-in autopause feature to prevent runaway costs if you go away from the Cloud Environment and it is no longer running an analysis. See Preventing runaway costs with Cloud Environment autopause to learn more. 

You can delete the VM and keep the detachable persistent disk. When you do that, you continue to pay for the detachable persistent disk. 

 

For more information on how to find the optimal compute and disk size, see Understanding and adjusting your Cloud Environment.

 

Pausing a cloud environment

If you pause your cloud environment, you will pay a nominal fee (<$0.01/hr) for the VM. You continue to pay for the detachable persistent disk at the normal rate ($0.04 per GB per month). For more information regarding what you pay for when you pause a VM, see the Google Cloud Documentation.

 

Workflows costs

Running workflows costs money for GCP services including VMs and associated disks. Service prices vary, and you will be charged according to what services you use and for how long. Terra has several built-in cost-savings features to reduce your workflow costs, including using preemptible VMs, deleting intermediate files, and using call- caching or checkpointing.

To learn more about  controlling compute costs, see the "Compute" section in Understanding and controlling cloud costs

 

G0_tip-icon.png


Note about workflow cost variability

 

The cost of running a specific workflow may be slightly different each time due to variations in the performance of the compute resources. 

If you notice a significant difference in run-times or costs, however, you might be up against a resource quota limit. See Google Cloud quotas - what are they and how do you request more?


How to view detailed Google Cloud charges (on GCP console)

Owners and admins of Google Cloud Billing accounts can access detailed billing reports (at the Google Cloud Billing account and Google project level) on GCP console. For workspaces created after September 27, 2021, you can look at how much of each service (i.e. data storage, VM) you used per month in the workspace.

1. Navigate to https://console.cloud.google.com/ and select the workspace Google project name from the dropdown at the top. The project name will have the format  <Terra-billing-project-name>--<workspace-name>
Detailed-cost-breakdown-on-GCP_Select-workspace-project_Screen_shot.png

HINT: To make sure you are seeing the right workspace project, you can cross-reference with the Google ID, which you can find on the workspace dashboard under Workspace Information. 
PPW-Google-project-in-Dashboard_Screen_shot_cropped.png

WARNING: You need to have owner, admin or viewer permissions on the Cloud Billing account to see cost reports on GCP console. If what you see doesn't match the descriptions or screenshots below, that is likely because you do not have the right permission. 

2. In the Billing card on the right, select the View detailed charges link.
Detailed-cost-breakdown-on-GCP_View-detailed-charges-link_Screen_shot.png

3. In the Reports page, you can filter by time range, SKUs, services, projects, and more.
Detailed-cost-vreakdown_Google-project-report-by-SKU_Screen_shot.png

4. To see how much a specific task costs, select the date range (on the right) and filter out storage costs by selecting Compute Engine (under Service). You can make a custom date range, too!

5. Running workflows and Interactive Analysis tasks (such as creating cloud environments) both use compute engine. If you perform multiple of these tasks on the same day, you will see the costs all combined on your Google bill. 

 

Use case examples

Below are a few examples with typical costs for running a workflow and a Cloud Environment analysis, along with the corresponding Google cost report. 

1. Joint Genotyping workflow in the Exome Analysis Pipeline

Exome Analysis Pipeline

Cost: $3.50

Breakdown of the Google bill 

Screen_Shot_2021-09-03_at_2.33.28_PM.png

 

2. Running a Cloud Environment w/ default 50GB Persistent Disk

Cloud Environment components
unnamed-4.png

Breakdown of the Google bill (Cloud Environment was active for 1 hour)Screen_Shot_2021-09-20_at_2.11.53_PM.png

 

Pricing breakdown by SKU

To better predict the charges for a given configuration, use the GCP pricing calculator.

Below is a breakdown of SKUs on your Google bill, what they represent, their typical price, and actions they correspond to in Terra. See the Google Cloud Platform SKUs for the complete list.

SKU

Meaning

Price

Action in Terra

Preemptible N1 Predefined Instance Core running in Americas

 

 

Preemptible N1 Predefined Instance Ram running in Americas

 

 

Preemptible Custom Instance Core running in Americas

 

 

Preemptible Custom Instance Ram running in Americas

 

When you use an Instance (a VM) on Google Cloud, you are charged based on how long you use the instance, the amount of virtual CPUs (Cores), and the amount of memory (Ram). 

The amount of cores and ram on a machine is often referred to as the "shape" of the machine. 

Google Cloud also offers different machine families, each with a curated set of processor and hardware configurations. 

N1 Predefined is one of these families

You can also create an instance with your own custom machine type. See vm-instance-pricing for more details. 

You also have the option to make your instance preemptible. 

A preemptible VM is an instance you can create and run at a much lower price than normal instances. However, Compute Engine might stop (preempt) these instances if it requires access to those resources for other tasks.

Instances are run on physical machines in different data centers around the world, so location availability (like one running in Americas) is considered in pricing.

0.031611 USD per hour

 

 

 

 

0.004237 USD per gibibyte hour

 

 

 

 

0.00698 USD per hour

 

 

 

 

0.00698 USD per hour

 

 

Run a workflow

Launch a Cloud Environment

Multi-Region Standard Class A Operations

Each time you perform one of the Class A Operations listed here in operations-by-class, it is one count. 

Ex. If you look at files in data section of workspace, it is performing an API call to show you what is in Google Bucket, which incurs a charge

0.005 USD per 1,000 count

Browsing files section in data tab

Run a workflow

Run a notebook

View or download data from the browser

Storage PD Capacity

Standard persistent disk storage 

0.00 USD (Free) per gibibyte month, for 0 to 30 gibibyte month, per month per account


0.04 USD per gibibyte month, for 30 gibibyte month and above, per month per account

Launch a cloud environment with persistent disk 

Run a workflow
*since workflows run for a limited amount of time, the cost is typically negligible compared to runtime cost of the VM

External IP Charge on a Preemptible VM

For a VM to have external network access, it needs to have an external IP address. There is an additional charge for this. You could create a VM with no external network access to avoid this charge.  See network-pricing for more details. 

0.002 USD per hour

Run a workflow

Launch a Spark cluster

Standard Storage US Multi-region

Storing your data in a Google bucket will accrue charges based on your region.

0.026 USD per gibibyte month

Storing files in workspace bucket

Network Egress via Carrier Peering Network - Americas Based

 

 

You can use a non-Google service provider (CenturyLink, Verizon, etc) to access Google Workspace applications. This is not a part of Google Cloud technically, but there is a Google Cloud charge to transfer data out of Google, through the service provider and onto your infrastructure. This charge is known as "Network Egress".

0.04 USD per gibibyte

Downloading files to local machine

 

Copying files from GCP to another cloud provider

Multi-Region Standard Class B Operations

Each time you perform one of the Class B Operations listed here in operations-by-class, it is one count. 

0.004 USD per 10,000 count

Browsing files section in data tab

Run a workflow

Run a notebook

View or download data from the browser

Network Google Ingress from Americas to Americas 

Network Internet Ingress from APAC to Americas

Network Internet Ingress from Americas to Americas

Network Internet Ingress from China to Americas

Ingress is when data is transferred through the network and into Google. network ingress is free.

0.00 USD (Free) per gibibyte

Uploading files 

Network Internet Egress from Americas to Americas

Data stored in Americas in Google Cloud Platform, was downloaded from Americas

0.00 USD (Free) per gibibyte, for 0 to 1 gibibyte, per month per account

0.12 USD per gibibyte, for 1 to 1,024 gibibyte, per month per account

0.11 USD per gibibyte, for 1,024 to 10,240 gibibyte, per month per account

0.08 USD per gibibyte, for 10,240 gibibyte and above, per month per account

Downloading files to local machine

Copying files from GCP to another cloud provider

Running a workflow or cloud environment VM if the bucket is in a specific US region (like us-east), and downloaded to a workflow/ cloud environment  VM in a different US region (like us-west) 

*This is very unlikely however as nearly 100% of all terra buckets are US multi-regional (which means egress is free to VMs in any US region).

 




Was this article helpful?

0 out of 0 found this helpful

Have more questions? Submit a request

Comments

0 comments

Please sign in to leave a comment.