Best practices for using service accounts in Terra

Allie Cliffe

Service accounts allow Terra to access resources on your behalf, such as ingesting data into the Terra Data Repository, using API calls, or automating some aspects of bulk (workflow) analysis. 

Overview: What is a service account?

Working with controlled-access data in the Cloud requires a high level of security. To maintain this security, Terra cannot use your user ID when interacting with cloud resources external to Terra. Instead, Terra creates and uses additional Google accounts (specific to you) called pet service accounts.

Pet service account details

A service account is a special type of Google account that lets Terra interface directly with Google Cloud on your behalf. For example, Terra uses a service account - rather than your user ID - to let you run workflows, create workspaces, make Data Repo datasets and snapshots; anything that uses Application Programming Interface calls. 

Your pet service account has the format: 
pet-7293562825402802834639@.iam.gserviceaccount.com

How service accounts differ from your user account (Google ID)

  • Service accounts do not have passwords and cannot be logged in via browsers or cookies.
    Instead, they are associated with public and private key pairs used for authentication to Google.  Service accounts are not associated with your Google account. Assets created by a service account are not created in your Google account's domain (Google Workplace Domain).
  • A service account is tied to a Google project.
    You will need to have separate service accounts for different actions in Terra, such as calling APIs or running workflows.  After you create a service account, you cannot move it to a different project. By default, each project can have up to 100 service accounts that control access to resources. 

When to use a service account in Terra

When building any automation in Terra, it is best practice to use service accounts rather than private credentials. Here are three concrete examples: 

1. You're running a Terra-supported application that needs to access data stored in a Terra workspace. Examples include analyzing Terra data with workflow or creating a dataset in the Terra Data Repository.

2. You're running a Terra-supported application that needs to access data or resources that are external to  Terra. For example, you may be using Terra to analyze data stored in an external Google bucket. Or, you may want to use a Terra notebook to call tools that are not yet available in Terra, such as the Machine Learning tools available through Vertex AI.

3. You're running a third-party application that needs to access data stored in a Terra workspace. For example, you might be using GitHub Actions to upload data on a fixed schedule, or updating a dashboard that pulls from Terra data to monitor population health metrics.

Best practices to manage service accounts

1. Register service accounts with Terra. In many situations, Terra creates a service account for you. However, in cases where you have to create your own service account, registering it with Terra allows the service account to call Terra APIs shared with the service account. See Step 2. Register your service account for step-by-step instructions.

2. Grant minimal privileges to your service accounts. To ensure that you only grant external resources access to the data that you want to share, be careful about the privileges that you grant to service accounts. For example, if an application only needs to read your data, there's no need to grant it "writer" access to your Terra workspace.

3. Store service accounts in Terra Groups (not Proxy Groups). Terra collects your service accounts into a Proxy Group. However, Proxy Groups have long, convoluted names which are hard to navigate when sharing data or resources with the Group. Therefore, in most cases the best practice is to create a Terra Group that acts as a proxy for your Proxy Group.

Whereas Proxy Group names are automatically generated, you can name Terra Groups something short and intuitive. As a result, Terra Groups are less error-prone when managing service accounts' privileges. You can also use Terra Groups to share data and resources with groups of other users. See How to use a service account in Terra for more information.

Next steps

To learn more about how to integrate Terra with data and resources outside of Terra, read Best practices for accessing external buckets, GCP VMs, and machine-learning tools  and How to access external Google Cloud resources.

For step-by-step instructions showing you how to create and manage a service account with Terra, read How to use a service account in Terra.

Was this article helpful?

Comments

5 comments

  • Comment author
    Kyle Vernest

    Thanks, Allie Hajian! Great article, I'll make sure to share it when we have requests from folks looking to use external buckets

    0
  • Comment author
    Matt Bookman
    • Edited

    I'd like to suggest a related "best practice" for Terra users.

    The Proxy Group identifier is not very human-friendly. If I am looking at a list of grants on a GCS bucket, seeing that there's a grant to PROXY_11564<etc>@firecloud.org is not helpful unless I happen to have a place to look up that Proxy Group.

    Instead, I suggest if your registered Terra account is j_doe@someplace.org, create a Terra Group named j_doe_at_someplace_org. Don't add anyone else to this group. You can then make grants to j_doe_at_someplace_org@firecloud.org. This group contains one member, namely the proxy group for j_doe@someplace.org. This is much easier for a human to reason over.

    Note that this approach extends to when you actually do want to make grants to groups of Terra users. It is better to add them all to a Terra group and then grant access to that group's firecloud.org Google Group, rather than directly granting to proxy groups.

    0
  • Comment author
    Felix Mbuga


     

    I don't have a proxy group listed under my profile (see above).

    Where can I find my proxy group please?

    0
  • Comment author
    Allie Cliffe

    Felix Mbuga - Hmmm, that's strange. Please submit a support ticket by going to Main menu (three horizontal lines at the top left of any page in Terra) > Support > Contact us. Someone in Frontline will be in touch with you to help track down the problem.

    1
  • Comment author
    Felix Mbuga

    Not sure what you did on your side but now I have a proxy group.

    Thanks!

    0

Please sign in to leave a comment.