Terra expenses and breaking down a Google bill

Anika Das
  • Updated

Find out how to interpret your Google bill and optimize your spend by aligning the costs you see (in GCP console) with actions performed in Terra. This document also includes information about actions you can perform in Terra, which ones cost money, and which are free of charge. All Google Cloud costs in Terra are calculated per workspace and paid via the Terra Billing project assigned to the workspace when it's created. 

Please reference Overview: Controlling Cloud costs on Terra as a companion to this article.

Workspace infrastructure costs

This section provides information on workspace costs that are distinct from data storage and analysis costs. Scroll down to see the costs for data storage, and running an interactive analysis or workflow, below.

Viewing Workspaces

Viewing workspaces is free of charge, and you can browse preview versions of Showcase workspaces without creating a Terra account. Viewing other public (non-Showcase) workspaces or the data library requires you to register for a free Terra account. See instructions for setting up an account on Terra here

Cloning Workspaces

Cloning or creating workspaces in Terra is free of charge. However, you will need to have access to a Terra Billing project to assign to the new workspace. All GCP costs for actions in a workspace (data storage, data transfer out, and VM compute) will be paid through the assigned billing project. Data files in workspace storage (Google Bucket or Azure container) are not copied and have no associated charge.

You will pay to store .ipynb  files

Note that if you are cloning a workspace with notebooks, the notebook files (i.e. ipynb files) are automatically copied along with the workspace and stored in the new Workspace Bucket. You are charged for the storage of these copied files (See the section on Workspace storage (i.e., Google Bucket below). Other bucket contents, such as workflow outputs and uploaded data, are not copied to a cloned workspace.

Storing and maintaining Data Tables

Data tables are spreadsheet-like files integrated into the workspace and workflows. The tables' infrastructure cost is included in the free Terra infrastructure.

What is free to use

  • Looking at data or the details of a file in a data table
  • Copying data tables to a different workspace
  • Downloading a file from Terra (in most cases, the costs for download are covered by the billing project of the workspace where the data files are stored. See Using Requester Pays workspaces/buckets for the exception).

What has a cost

Downloading files can incur costs for you if the Workspace bucket is a Requester Pays bucket.

To learn more about how to reduce data transfer costs for data storage, see Configuring GCS buckets to prevent data transfer charges.

Data storage costs

Each workspace has two dedicated cloud storage resources.

  • Workspace storage (Google Bucket)
  • Cloud Environment persistent disks 

Workspace storage (i.e., Google Bucket)

The cost for storage in a Workspace bucket depends on how much is in your bucket. Charges accrue daily, but Google Cloud Storage only bills at the end of the billing period. You can view unbilled usage on your workspace project's billing page in the Google Cloud Console. For more detailed information, see Google Cloud Documentation.

What has an indirect cost (i.e., data transfer out charges)

If someone views or downloads the contents of files in your workspace bucket, the workspace Billing project may accrue data transfer charges if the data moves outside of the bucket Cloud Storage region(s).

For more details, see https://cloud.google.com/storage/pricing

What is free?

Uploading data to a workspace bucket (for example, through the "Files" section of a workspace) does not incur upload costs. However, the new data in the bucket will start to accrue storage fees immediately.

You are only responsible for the storage costs of files in your Workspace bucket or Cloud Environment PD. If you upload a workspace data table (i.e., TSV or TXT file), you will not pay for anything related to the TSV or any of the file paths in your table. You can reference files from an external bucket in your data table without paying, unless use files stored in a different region as input in a workflow analysis, or copy them to your Workspace bucket or Cloud Environment persistent disk.

Default bucket regions and cost

The default for Terra buckets is Autoclass storage in us-central-1, but other options are available. You can find the region(s) as well as an estimated monthly storage cost (Est. $/Month) in the Workspace Information in the right column of the Dashboard.

Screenshot of Cloud Information section in right column of workspace dashboard

For more information on Google Cloud Storage pricing, see https://cloud.google.com/storage/pricing.

Cloud Environment Persistent Disk

Cloud Environments come with an (optional) Overview: Clou dEnvironment storage (detachable persistent disks). When you create a Cloud Environment using the default options, you automatically get 50G of VM persistent disk storage. You can choose a different persistent disk size in the Cloud Environment setup form.

You pay a fixed amount per month for the disk you choose, until you delete it (i.e., whether or not the Cloud Environment VM is running). The cost for the default 50G disk is $2.00/month

Interactive Analysis costs

Interactive analysis apps such as Jupyter notebooks, Galaxy, RStudio, or the workspace terminal all run on the Cloud Environment VM, which has a cloud cost while the cloud environment is running

Finding the Cloud Environment costs

You can see the current interactive analysis costs at any time by clicking the Jupyter, RStudio, or Galaxy logo in the right sidebar and clicking the Settings button (gear icon). 

Screenshot of cloud environment configuration pane with the estimated cost of a running and paused VM and customization options for conpute profile - VM type and size - and persistent disk size and type

What has a cost

A Cloud Environment virtual machine (VM) has three separate billable components: a VM, a boot disk, and a detachable persistent disk. When creating a Cloud Environment, you can see the cost per hour based on the CPUs, Memory(GB), and disk size you choose. As you increase CPU, memory, or storage, the cost of the Cloud Environment will increase.

You will pay the VM cost/hour as long as the VM is running, whether or not it is performing calculations. Terra has a built-in autopause feature to prevent runaway costs if you go away from the Cloud Environment and it is no longer running an analysis. See Preventing runaway costs with Cloud Environment autopause to learn more.

You can delete the VM and keep the detachable persistent disk. When you do that, you continue to pay for the detachable persistent disk.

For more information on how to find the optimal compute and disk size, see Your interactive analysis VM (Cloud Environment).

Pause a cloud environment to reduce costs

If you pause your cloud environment, you will pay a nominal fee (<$0.01/hr) for the VM. Terra has a built-in autopause feature to prevent runaway costs if you go away from the Cloud Environment and it is no longer running an analysis. See Preventing runaway costs with Cloud Environment autopause to learn more.

You continue to pay for the detachable persistent disk at the normal rate ($0.04 per GB per month for the default PD). For more information about what you pay for when you pause a VM, see the Google Cloud Documentation.

Workflows costs

Running workflows costs money for Google Cloud resources, including VMs and associated disks. Service prices vary, and you will be charged according to what services you use and for how long.

Options to reduce workflow storage and compute costs

Built-in cost-savings features to reduce your workflow costs

To learn more about controlling compute costs, see the Compute section in Overview: Controlling Cloud costs on Terra.

Note about workflow cost variability The cost of running a specific workflow may be slightly different each time due to variations in the performance of the compute resources. 

If you notice a significant difference in run-times or costs, however, you might be up against a resource quota limit. See How to troubleshoot and fix stalled workflows?

How to view detailed Google Cloud charges

Owners and admins of Google Cloud Billing accounts can access detailed billing reports (at the Cloud Billing account and Google project level) on Google Cloud console. For workspaces created after September 27, 2021, you can look at how much of each service (i.e., data storage, VM) used per month in the workspace.

1. Navigate to https://console.cloud.google.com/ and select the workspace Google project name from the dropdown at the top. The project name will have the format  <Terra-billing-project-name>--<workspace-name>

Screenshot of billing page in GCP console with arrow pointing to the Select workspace project field at the top

HINT: Finding the workspace projectTo make sure you are seeing the Google project associated with the workspace you're interested in, you can cross-reference with the Google ID, which you can find on the workspace dashboard under Workspace Information. 
Screenshot of workspace dashboard with the workspace Google Project ID at right under Cloud Information circled

2. In the Billing card on the right, select the View detailed charges link.

Screenshot of project details on GCP console with arrow pointing to the View detailed charges link at right in the Billing card.

Don't see any cost reporting info on GCP console? You need to have owner, admin or viewer permissions on the Cloud Billing account to see cost reports on GCP console. If what you see doesn't match the descriptions or screenshots below, that is likely because you do not have the right permission. 

3. In the Reports page, you can filter by time range, SKUs, services, projects, and more.
Screenshot of detailed cost breakdown on GCP View detailed charges link with options for filtering at right and SKU breakdowns below.

4. To view costs for a specific task, select the date range (on the right) and filter out storage costs by selecting Compute Engine (under Service). You can make a custom date range, too!

5. Running workflows and Interactive Analysis tasks (such as creating cloud environments) both use compute engine. If you perform multiple of these tasks on the same day, you will see all costs combined on your Google bill.  

Examples by use-case

Below are a few examples of typical costs for running a workflow and a Cloud Environment analysis, along with the corresponding Google cost report.

1. Joint Genotyping workflow in the Exome Analysis Pipeline

Exome Analysis Pipeline

Cost: $3.50

Breakdown of the Google bill

Screen_Shot_2021-09-03_at_2.33.28_PM.png

2. Running a Cloud Environment w/ default 50GB Persistent Disk

Cloud Environment components
Screenshot of Cloud environment configuration pane with 1 CPU of compute, 3.75GB of RAM, and a 50 GB persistent disk

Breakdown of the Google bill (Cloud Environment was active for 1 hour)

Screenshot of detailed billing for cloud environment use-case showing three SKUs with cost: the N1 core compute instance, the N1 RAM running in Americas, and PD storage

Pricing breakdown by SKU

To better predict the charges for a given configuration, use the GCP pricing calculator.

Below is a breakdown of SKUs on your Google bill, what they represent, their typical price, and the actions they correspond to in Terra. See the Google Cloud Platform SKUs for the complete list.

SKU

Meaning

Price

Action in Terra

Preemptible N1 Predefined Instance Core running in Americas

 

 

Preemptible N1 Predefined Instance Ram running in Americas

 

 

Preemptible Custom Instance Core running in Americas

 

 

Preemptible Custom Instance Ram running in Americas

 

When you use an Instance (a VM) on Google Cloud, you are charged based on how long you use the instance, the amount of virtual CPUs (Cores), and the amount of memoryRam).

The amount of cores and ram on a machine is often referred to as the "shape" of the machine.

Google Cloud also offers different machine families, each with a curated set of processor and hardware configurations.

N1 Predefined is one of these families

You can also create an instance with your own custom machine type. See vm-instance-pricing for more details. 

You also have the option to make your instance preemptible.

A preemptible VM is an instance you can create and run at a much lower price than normal instances. However, Compute Engine might stop (preempt) these instances if it requires access to those resources for other tasks.

Instances are run on physical machines in different data centers around the world, so location availability (like one running in Americas) is considered in pricing.

0.031611 USD per hour

 

 

 

 

0.004237 USD per gibibyte hour

 

 

 

 

0.00698 USD per hour

 

 

 

 

0.00698 USD per hour

 

 

Running a workflow

Launching a Cloud Environment

Multi-Region Standard Class A Operations

Each time you perform one of the Class A Operations listed here in operations-by-class, it is one count.

Ex. If you look at files in the data section of the workspace, it is performing an API call to show you what is in Google Bucket, which incurs a charge

0.005 USD per 1,000 count

Browsing files section in data tab

Running a workflow

Running a notebook

Viewing or downloading data from the browser

Storage PD Capacity

Standard Persistent Disk storage

0.00 USD (Free) per gibibyte month, for 0 to 30 gibibyte month, per month per account


0.04 USD per gibibyte month, for 30 gibibyte month and above, per month per account

Launching a Cloud Environment with persistent disk

Running a workflow
*since workflows run for a limited amount of time, the cost is typically negligible compared to runtime cost of the VM

External IP Charge on a Preemptible VM

For a VM to have external network access, it needs to have an external IP address. There is an additional charge for this. You could create a VM with no external network access to avoid this charge.  See network pricing for more details.

0.002 USD per hour

Running a workflow

Launching a Spark cluster

Standard Storage US Multi-region

Storing your data in a Google bucket will accrue charges based on your region.

0.026 USD per gibibyte month<

Storing files in workspace bucket

Network data transfer via Carrier Peering Network - Americas Based

 

 

You can use a non-Google service provider (CenturyLink, Verizon, etc) to access Google Workspace applications. This is not a part of Google Cloud technically, but there is a Google Cloud charge to transfer data out of Google, through the service provider and onto your infrastructure. This charge is known as "Network Data Transfer".

0.04 USD per gibibyte

Downloading files to local machine

Copying files from GCP to another cloud provider

Multi-Region Standard Class B Operations

Each time you perform one of the Class B Operations listed here in operations-by-class, it is one count.

0.004 USD per 10,000 count

Browsing files section in data tab

Running a workflow

Running a notebook

Viewing or downloading data from the browser

Network Google Ingress from Americas to Americas

Network Internet Ingress from APAC to Americas

Network Internet Ingress from Americas to Americas

Network Internet Ingress from China to Americas

Ingress is when data is transferred through the network and into Google. network ingress is free.

0.00 USD (Free) per gibibyte

Uploading files

Network Internet data transfer from Americas to Americas

Data stored in Americas in Google Cloud Platform, was downloaded from Americas

0.00 USD (Free) per gibibyte, for 0 to 1 gibibyte, per month per account

0.12 USD per gibibyte, for 1 to 1,024 gibibyte, per month per account

0.11 USD per gibibyte, for 1,024 to 10,240 gibibyte, per month per account

0.08 USD per gibibyte, for 10,240 gibibyte and above, per month per account

Downloading files to local machine

Copying files from GCP to another cloud provider

Running a workflow or cloud environment VM if the bucket is in a specific US region (like us-east), and downloaded to a workflow/ cloud environment  VM in a different US region (like us-west)

*Note that the default for Terra buckets is us-central-1 regional by default (data transfer is free to VMs in the default region).

Was this article helpful?

Comments

0 comments

Please sign in to leave a comment.