Learn best practices for accessing external GCP resources from within Terra; for example, using data stored in external buckets, or running Google Cloud virtual machines or machine-learning tools in a notebook. Working in Terra lets you take advantage of Terra's built-in FedRAMP and FISMA-moderate security perimeter, even while accessing these external resources.
If you are looking for step-by-step directions, see How to access external Google Cloud resources.
If you are running an automation or service outside of Terra that needs to call Terra APIs, see How to use a service account in Terra.
If you are having trouble accessing files with a service account, see Error message: AccessDeniedException.
Overview - Accessing external resources
Terra is designed to remove some of the barriers of moving to the cloud, letting you focus on your work instead of spending time wrangling with Google. Behind the scenes, Terra interfaces with Google on your behalf using a special kind of Google account - called a service account - to perform tasks like accessing data in external Google buckets, or requesting and requisitioning other Google Cloud resources such as the Virtual Machines (VMs) that power Cloud Environments and workflows. This lets you work with data stored in Google buckets and run tools on Google VMs right in your workspace without having to worry about many of the technical details.
Service accounts
Every Terra user has one or more of these "pet" service accounts (one for each Billing project) for interfacing with the cloud outside Terra. Service accounts follow this format:
PROXY_<long-number>@firecloud.org
When Terra uses a service account (examples)
- When accessing a non-Terra GCS bucket, BQ dataset, GCR Docker image, etc.
- When running workflows or interactive analyses on virtual machines (VMs)
Service accounts protect your identity
Instead of using your user ID and credentials to access external resources, Terra uses the identity of the service account. Using an anonymous service account is required for data and workspace security, but it means there are a lot of non-human-friendly details (like the PROXY service account names) in the backend.
Best practices for individuals accessing external resources
In theory, you could use these predefined service/proxy groups if you need to interface with Google resources directly. We don't recommend that, because the long string of numbers in the service account names is hard for most humans to manage. To make this more concrete, imagine being a resource owner trying to identify who has access to the data in your external bucket. It's tough to do that when the list contains accounts with names likePROXY_18340hruhgouhb1foy34g<long-number>@firecloud.org
.
Best practice: Use a human-friendly group to represent a user (and their service account)
An alternative that maintains Terra's built-in data and workplace security is to create a Terra user group with a human-friendly name for interfacing with Google Cloud (i.e., granting access to external buckets). Terra-managed groups can include you and any other users who need access to the same workspace or billing project. Groups are designed to streamline resource management by making it easy to share resources with one entity (the group), rather than with several individuals.
Since Terra-managed groups include each user's non-human-friendly proxy by default, they can be used as a proxy to the service account, an easy way to put a human-friendly label on this back-end tool. Therefore, the best practice is to always use Terra groups for accessing external resources, even for one user.
To learn more about Terra groups, see Managing access to shared resources.
Example: Terra group for a single user Let's say you want to create a Terra group that contains a single user (user ID: j_doe@someplace.org
). Here's what you'd do:
1. Create a Terra Group: j_doe_at_someplace_org
2. Don't add anyone else to this group. It will only include the user's Terra ID and their service account proxy.
3. When accessing external resources, manage permissions using the group's id: j_doe_at_someplace_org@firecloud.org
Best practices for groups to access external resources
Managed groups are the best way to share resources within a group of individuals, such as a lab. This applies when sharing internal resources - like Terra workspaces and billing projects - and external resources. Sharing with a managed group, instead of a long list of individuals, saves time and avoids errors. The groups can be updated in Terra when people are added to - or leave - the lab or project. Updating a single group is far easier than updating all of the workspaces, billing projects, and external resources that an individual needs to access or leave.
Recommendations
Create a Terra user group with a human-friendly name that includes all team members and collaborators who need access to the external resource. Use it for interfacing with Google Cloud (i.e., granting access to external buckets) as well as managing access to shared team resources.
Example: Terra group for lab Let's say you want to create a Terra group for a whole lab. Here's what you'd do:
1. Set up a Terra Group for all collaborators, and name it something like my_lab
2. Add the Terra user ID of everyone in the lab to the group.
3. Add the group's ID (my_lab_@firecloud.org
) to Terra resources (like workspaces and billing projects) and external resources (like Google cloud buckets).
4. Whenever someone leaves the lab and no longer needs to access the lab's resources, remove their Terra ID from the group.