Learn how to set up a service account to allow Terra to access resources on your behalf such as ingesting data into the Terra Data Repository, using API calls, or automating some aspects of bulk (workflow) analysis.
Service accounts and when to use them in Terra
A service account is a special kind of non-human account used to execute applications and run automated services, virtual machine instances, and other processes. Terra uses a service account -rather than your user ID - to do things on your behalf (e.g., running Application Programming Interface calls, running workflows, creating Terra workspaces, making data repo datasets/snapshots, etc.).
How service accounts differ from your user account (Google ID)
Service accounts do not have passwords and cannot be logged in via browsers or cookies. Instead, they are associated with public and private key pairs used for authentication to Google. Service accounts are not associated with your Google account. Assets created by a service account are not created in your Google account's domain (Google Workplace Domain).
Service accounts are tied to a Google project. You will need to have a separate service accounts for different actions in Terra such as calling APIs or running workflows. After you create a service account, you cannot move it to a different project. By default each project can have up to 100 service accounts that control access to resources. Since each Terra workspace has its own Google project, this means you will need to set up a separate service account for each workspace separately.
When to use a service account in Terra
When building any automation in Terra, it is best practice to use service accounts rather than private credentials as an individual.
Read on for step-by-step instructions to register a service account to use in Terra.
Step 1: Create a service account
1.1. Create or be added to a Google Cloud project.
1.2. Follow the step-by-step instructions in the Google Cloud support doc (external link) to set up your service account.
1.3. Click manage roles to select which roles to assign to this service account.
Choosing the service account roleHopefully you already know which role you need for the task you want the service account to do (often in the instructions for the action). There are hundreds of roles you can grant to a service account so take some time to see which scopes and permissions you may want to assign to the service account (note that roles can be updated later).
Most common services and permissions on Terra
- Google Cloud Storage (viewer, reader, writer, owner)
- Google BigQuery (viewer, reader, writer, owner)
- Google Compute Engine (can-compute)
These are the most common roles and permissions. There is no one size fits all solution. If you think you may require a less-used role, see this doc (Google reference) which outlines all the different types of roles you can assign a service account.
1.4. After you've finished the initial account set up save the email associated with the service account:
1.5. Before moving to step 2 (registration), go to the "keys" tab to create and download a key for your service account onto your local machine.
Note: Use the recommended .JSON format to download your key
Step 2: Register a service account
You will need the key file and service account email from part 1 (above) to register the service account. Registration allows the service account to call Terra APIs shared with the service account.
2.1. Run the script to register a service account (you can find the script on the Terra FireCloud tools page).
To run the script you will need
- -j <file path on local machine to the json file containing the key created for service account>
- -e <email address for the service account>
2.2. In your local terminal, run "git clone https://github.com/broadinstitute/firecloud-tools" with the following code. Cloning the FireCloud repository will allow you to run FireCloud tools used by Terra.
"git clone https://github.com/broadinstitute/firecloud-tools"
To learn how to install and use a terminal on your local machine, see How to move data between local and workspace storage (Step 1: Install gsutil locally).
2.3. Change the directory to "firecloud-tools" using the below code.
2.4. Run the following script in the firecloud-tools directory to register your service account.
./run.sh scripts/register_service_account/register_service_account.py -j <path to your service account credentials json file> -e <email address for owner of this service account, it's where notifications will go>
Replace the bolded sections with the file path to the key you downloaded in step 1 and the email associated with the service account.
Note: You will need to run this code in the Firecloud-tools directory on your local machine.
Example registration code
./run.sh scripts/register_service_account/register_service_account.py -j /Users/yduggal/downloads/setting-up-service-account-890a643c6881.json -e firstname.lastname@example.org
The service account is now registered with FireCloud! You are now able to share workspaces with this address or use it to call APIs in Terra.