Pet service accounts and proxy groups
This article explains a bit of the "under the hood" workings that enable you to interact with GCP Storage and GCP Compute without directly interfacing with the Google Cloud Platform console. Most users don't need this level of detail for working in Terra, but if you need to grant access to data that is on Google Cloud but outside of a Terra workspace, read on...
Contents
- Overview: How Terra interfaces with GCP
- Learn more about "service" (aka "pet") accounts and proxy groups
- Find your Terra proxy group
- Best Practices - Use a human-friendly Terra group instead of proxies
- Create an easy to manage Terra group in four steps
- Pro-tip: Granting permission to groups of Terra users
- Additional Resources
Overview: How Terra interfaces with GCP
In order to maintain the high security required for working with controlled-access data in the cloud, Terra cannot use your user ID when interacting with cloud resources external to Terra. Instead, Terra creates and uses additional Google accounts (specific to you) called "pet service accounts." Terra also puts all of these "pet service accounts" into a convenient Google Group (specific to you) called a "Proxy Group."
Click to learn more about service accounts (aka "pet" service accounts) and proxy groups
Pet service accounts
In addition to the user account you have registered with Terra, you also have one or more service accounts that Terra creates for you. A service account is a special type of Google account that lets Terra act directly with GCP on your behalf - so your workflows and notebooks can access your data in Google Cloud.Your pet service account has the format: pet-7293562825402802834639@<billing-project>.iam.gserviceaccount.com
Proxy groups
To manage these accounts and simplify access control, everyone registered in Terra has a unique group, called a "proxy group" that contains your registered user account and your service accounts. Your proxy group includes everything needed to grant access to cloud resources outside of Terra (such as a GCS bucket or BigQuery dataset) inside GCP console.
Your Proxy group has the format:PROXY_115642708824051152393@firecloud.org
Find your Terra proxy group
You'll see your proxy group listed near the bottom:
Best Practices: Use a human-friendly Terra group instead of proxies
|
|
|---|---|
| While it's necessary for accessing resources that exist outside of Terra, the pre-defined Proxy Group identifier is not very human-friendly.
If you're looking at a list of users with access to an external GCS bucket, seeing that there's a grant to Instead, you can create a Terra group (with a sensible name) as a proxy for your proxy. Note that this approach extends to when you actually do want to make grants to groups of Terra users. It is better to add them all to a Terra group and then grant access to that group's firecloud.org Google Group. |
Create your personal Terra group in four steps
- Go to your Groups page ("Main menu" --> "Groups" from the top left of any page in Terra)
- In the "Create a New Group" card, click on the blue "+" icon
- Enter your human-friendly user-ID (can be the same as your Terra login) and click the "Create Group" button
Terra creates a mirrored Google group that includes your Proxy (which already includes your user ID). When you (or the owner/admin) grant access on a GCP resource to the <terra-group>@firecloud.org group, both your end-user credentials and your pet service accounts have access to the GCP resource. - To access external resources, owners can now grant permission to your Terra group (i.e. j_doe_at_someplace_org@firecloud.org) in GCP console
Using "j_doe_at_someplace_org@firecloud.org," makes it easier to see who has access permissions!
|
|
|---|---|
| You can extend this to groups of Terra users, by first setting up a group for each person's Terra proxy (following the steps above).
Then create a managed group (i.e. To give permission to everyone in the group, you would grant access to |
Additional resources
|
|
|---|---|
| Want to learn more about how to access advanced GCP features (things not yet available in the Terra UI)? See this article. - WRITE to BigQuery - Interact with Cloud Storage buckets other than the workspace bucket - Run dsub jobs - Run Cloud Dataflow jobs - Run Cloud ML engine jobs For recommended best practices for accessing external resources (external buckets), see this article. |
Comments
2 comments
Thanks, Allie Hajian! Great article, I'll make sure to share it when we have requests from folks looking to use external buckets
I'd like to suggest a related "best practice" for Terra users.
The Proxy Group identifier is not very human-friendly. If I am looking at a list of grants on a GCS bucket, seeing that there's a grant to PROXY_11564<etc>@firecloud.org is not helpful unless I happen to have a place to look up that Proxy Group.
Instead, I suggest if your registered Terra account is j_doe@someplace.org, create a Terra Group named j_doe_at_someplace_org. Don't add anyone else to this group. You can then make grants to j_doe_at_someplace_org@firecloud.org. This group contains one member, namely the proxy group for j_doe@someplace.org. This is much easier for a human to reason over.
Note that this approach extends to when you actually do want to make grants to groups of Terra users. It is better to add them all to a Terra group and then grant access to that group's firecloud.org Google Group, rather than directly granting to proxy groups.
Please sign in to leave a comment.