Learn how to create and list Google-backed TDR Billing Profiles, and share them with others. Creating a Billing Profile is the first step for data custodians/stewards in the Terra Data Repo. To learn how to set up an Azure-backed TDR billing profile, see How to create a TDR Billing Profile (Azure).
Overview
Before you can begin doing anything billable in the Terra Data Repository - adding datasets, ingesting data, or creating snapshots - you will need to create a Billing Profile. The Billing profile UUID links actions with a cost in TDR to a Google Cloud billing account.
A single Billing Profile can be reused across datasets, and a single dataset can use multiple profiles to bill for different snapshots of data.
Step-by-step instructions to create a billing profile
1.1. Set up a Google Billing Account
Before creating a billing profile, you will need access to a Google Cloud Billing account. For details, see Instructions to create a Google Cloud Billing Account.
Prerequisite
Note that you need to have the Billing Account User role on that Cloud Billing Account.
1.2. Add the Data Repo service account as a user on your account
To give the Data Repo access to your billing, go to your GCP Billing page to grant the Data Repository Service account (datarepo-jade-api@terra-datarepo-production.iam.gserviceaccount.com
) the Billing Account User role on the account as shown below.
Screenshot of billing management page in GCP console
Screenshot of Add Principal popup in GCP console
If you are working in the alpha environment, you’ll need to grant the following Data Repository Service account access as well: datarepo-jade-api@terra-datarepo-alpha.iam.gserviceaccount.com
.
1.3. Create a Billing Profile via Swagger API
Once the service account has access to a billing source, you’ll be able to use the createProfile API endpoint to create a new Billing profile. Be sure to Authorize Swagger first - otherwise, you won't be able to execute the API job.
Remember to authorize Swagger every time you use itSee How to authenticate/troubleshoot Swagger for TDR for step-by-step instructions.
What to expect
Once you're properly authenticated, click Try it out in the createProfile section. You'll be able to edit any input fields, such as the main request body, and you'll see an execute button somewhere below that field.
Modify the request body with the appropriate code, as shown below, and then click Execute.
Do not use the default response body in SwaggerTo execute successfully, make sure to copy/paste the response body below!
createProfile request body (copy/paste this code)
{
"biller": "direct",
"billingAccountId": "/*The Google Billing Account ID you can find on your GCP Billing page*/",
"cloudPlatform": "gcp",
"description": "string",
"profileName": "string"
}
Request body parameters
- The "biller" field should generally be set to "direct"
- The "billingAccountId" field should be set to the Google Billing Account ID that you can find on your GCP Billing page.
- Wherever the above example code says "string", you can put whatever arbitrary set of characters you like to help you keep track of things. The ProfileName can be useful if you use multiple billing accounts. As you'll see below, when you're looking up Billing Profile information, having a clearly named Billing Profile saves you a lot of trouble if you're trying to remember which Billing Profile ID goes with which Billing Account ID.
Screen capture for Try it out
What to expect
The createProfile API endpoint can take a while to execute. Once it has executed, you will see a series of responses from the server, including error or success codes. It can be challenging to decipher these responses to know whether the job was successful, or if it is still running.
You can check whether you successfully created a billing profile in two ways:
Check on the billing profile with Swagger
1. Wait until you see messages in the Response section of the Swagger UI (below the Execute button).
2. Go to the enumerateProfiles API endpoint, and click Execute. Note that you may need to re-authenticate Swagger first, if your authentication has timed out.
3. If the profile you just requested is listed in the response body, it was successfully created.
Check on the billing profile through the TDR website
1. Log into https://data.terra.bio/.
2. Click on the Activity tab.
3. Locate the job you just ran (the class should be "ProfileCreateFlight")
4. If the job has succeeded, the status will be completed.
Troubleshooting
If your job is taking awhile to run, you can monitor your job's progress with the jobs endpoints:
1. Copy your job's id from the "id" field in the createProfile job's response body:
2. Go to the retrieveJob API endpoint, enter the job id in the "id" field, and click Execute.
3. If the job_status is "succeeded," your billing profile was successfully created.
For more details and step-by-step instructions, see the troubleshooting section in How to authenticate/troubleshoot in Swagger.
How to look up billing profile information
If you lose track of your billing profile ID, you can always find it using the enumerateProfiles API endpoint to list out all of the profiles to which you have access.
What to expect
If you've properly authenticated using the Authorize button near the top of the page, clicking Execute on this endpoint will list all Billing Profiles associated with the credentials you used to log in. Info for each Billing Profile will appear in the response body that appears beneath the request body after you've executed. The relevant information in most cases is the number corresponding to the "id" field:
Next Steps
Once you've set up your TDR billing profile, you can add collaborators to your profile or start adding data to TDR by creating a TDR dataset through the TDR website or Swagger APIs. See How to use the TDR support articles to locate instructions for these and other actions in TDR.