For peace of mind that you're not going over a project budget, read on for an overview of how BILLING PROJECT OWNERS to see the actual Google cloud costs of running workflows and the storage and compute costs of your top ten workspaces.
Two ways for project owners to surface storage and analysis costs
- View the costs for the top ten workspaces (in the Billing page)
- View the cost of individual workflow submissions (in the Job History page)
Prerequisite (Billing project owners must set up)Note that billing project owners/admins must set up this functionality. For step-by-step instructions on how to do this, see How to set up spend reporting.
Workspace compute and storage costs (top 10)
Billing project owners can see the total spend, total compute, and total storage costs of the top ten workspaces in the Billing page.
Workflow submission costs
1. To view the cost of an individual workflow submission, navigate to the Job History page of the associated workspace.
This page includes all workflow submissions for the workspace.
2. Click the submission of interest in the far left column.
3. If your spend reporting has been correctly set up, you will find the Total Run Cost at the top right corner. Note that the spend report can take up to 24 hours to appear in Terra.
If your submission included more than one execution, each will be listed separately under Run Cost.
Cost reporting caveats
Because Terra derives cost information from Google's billing export services, there are a few things to keep in mind.
Cost reports take a few hours to show up
Cost information from Google isn't available until some hours after the analysis has completed. It's best to allow a day to pass before checking in on the cost.
Costs are only reported for completed workflows
The cost is a true cost, not an estimate. It is only available after the workflow is complete and is not available for failed workflows.
Reports cover the total workflow cost
It is not possible to access task-level or instance-level costs (i.e., the cost of running a particular task within a workflow) with built-in cost reporting.
- For details about finding task-level costs, see Monitoring GCP cloud resources used in a workflow.
- For more granular costs or cost for workflows in progress, you could try running a cost-estimate notebook. See Cost-estimate notebook.
How accurate is the cost report? Terra's built-in cost reports come directly from Google. This is the actual cost for the given workflow (i.e., not an estimate). They are generated by accessing Google's billing repository, so they can take several hours to log.
How to set up cost reporting in Terra
For most people, there are two steps to setting up cost reporting. Once you complete these, the spend reporting will appear automatically. For step-by-step instructions, see How to set up spend reporting.
Step 1: Set up billing export on Google
- Most users will need to set up Google Cloud so that Terra can access Google's cost reports.
- This must be done once per Cloud Billing account.
- Users with a Broad Google sub-account can skip this step.
Step 2: Configure workflow spend reporting in Terra
- Once you have set up billing export, you will need to specify the Project ID and BigQuery dataset name in Terra.
- You will need to do this once for each Terra Billing project.