Best practices for managing shared resources (funding)

Allie Hajian
  • Updated

Understanding how to manage and control costs is an important consideration for a team with a shared funding source. Read on for two ways to share funding resources, and guidance of which to use (depending on your collaboration needs). 

To learn more about sharing data resources, see Best practices for sharing and protecting data resources.

Overview: How costs are calculated in Terra

Working in Terra incurs GCP fees based on how much storage, egress and compute resources you use. These costs are accrued at the workspace level (bottom), billed to a Terra Billing project (middle), and ultimately paid for by the Google Cloud Billing account (top). Terra Billing projects can support many different workspaces, such as one for each member of a team with shared funding.

Billing-heierarchy-in-cloud.png
Note: All Terra components are gray and all GCP components are blue.

Consider the case of several collaborators with the same funding source. The funding will be dispersed through a Google Cloud Billing account. Collaborators can access the shared funds at any level of the billing hierarchy:

  • Shared Cloud Billing account 
  • Shared Terra Billing project
  • Shared workspace

Roles at the top the hierarchy are able to create more resources. Roles at the bottom (in a workspace) are limited to operations in a workspace. How you share resources depends on your group's needs. For more details on Terra's billing and resources structure, see Managing shared resources with groups and permissions

Below are two options for sharing resources in a team: working in a shared workspace (PIs have the most control) or working in separate workspaces with shared billing (collaborators have more control and more flexibility). 

Working in shared team workspaces (PIs have most control)

Workspace owners (i.e. lab PIs) can share a workspace with an individual or a managed group. The workspace owner controls exactly what each collaborator can do in the workspace (read, write, execute). Collaborators can only accrue costs (run an analysis or store data) if the workspace owner gives sufficient workspace permission. 

For a detailed description of shared workspace roles, see Sharing data and tools with workspace access controls.

To learn more about enabling colleagues to create their own Terra Billing projects or access detailed cost breakdown and reports, scroll down to working in separate workspaces with team billing 

Workspace-permissions_Screen_shot.png
Note that workspace owners can change collaborator roles
at any time in the "Share workspace" modal (above).

How to set up a shared team workspace (step-by-step instructions)

1. Create the team workspace (this can only be done if you are a user on a Terra Billing account).

2. Make a managed group for the team by going to Profile > Groups from the main navigation.

3. Return to the team workspace and click the workspace operations icon (three vertical dots).Share-workspace-vertical-dots_Screen_shot.png

4. Open the Share workspace form by selecting Share from the dropdown.

5. Share with the group (start typing in the group name in the User email field).

6. Select enter or return after entering the group name.

7. Select the group's permissions

8. Don't forget to save!

Group versus shared permissions Note that everyone in the group shares the same workspace permissions. To assign more granular permissions, add individual collaborators one at a time. 

To learn more about the trade-offs of sharing with a group versus sharing with an individual, see Managing shared resources with groups and permissions.

Controlling spend in a shared workspace 

Workspace permissions are the primary way to control spend. The workspace owner controls what collaborators can accrue cloud costs by sharing a workspace and assigning workspace roles/permissions. Owners can change or remove an individual or group's permission at any time, and it takes effect immediately. 

All GCP costs are billed to the workspace billing project Note that it is not possible to get cost breakdowns per user for work done in a shared workspace. The most granular cost breakdown of each GCP resource (storage, compute and egress) is per workspace (for workspaces created after September 27, 2021) or per Terra Billing project (for workspaces created after September 27, 2021). 

Billing in Terra works much the same as billing for electricity in a building. The person on the electric bill (Terra billing owner) pays for all the electricity (GCP costs) used over the month by all roommates (collaborators). If one roommate turns up the thermostat and opens the windows (runs a huge analysis), it's the owner who pays for the extra electricity (GCP compute and storage costs). 

For additional information on controlling spend in a workspace, see How to set up and use GCP budget alerts or Terra expenses and breaking down a GCP bill.

How to add/remove individuals from a shared workspace

1. Click on the three vertical dots at the top right to access the workspace share form and change the user's role.
Controlling_lab_costs_Share_workspace_Screen_Shot.png

2. If adding someone, you need to hit enter (or return) after entering their user email.

3. Choose their role, or delete the person from the collaborator list.

4. Don't forget to save after changing!
Controlling_lab_costs_sharing_workspace_modal_Screen_Shot.png

How to add/remove a user from a workspace shared with a Group

To do this, you will add/remove the individual from the managed group the workspace is shared with. You do not need to adjust the workspace permissions. 

1. Go to Your Name > Groups from the main navigation menu
Controlling_costs_modifying_Groups_Screen_Shot.png

2. Edit the person's role in the group (you must have the right permissions, of course!)
Control_lab_costs_edit_user_role_Screen_Shot.png

Working in separate workspaces with team billing

Sharing billing resources (Billing projects) lets each person on your team create their own workspaces, which adds flexibility (collaborators don't have to ask for permission to work in your workspace) and avoids overwriting data and analysis tools in a shared workspace. 

Owners and administrators of Google Cloud Billing accounts can set up one Terra Billing project for the entire team or separate one for different collaborators or different work in Terra. It is easy to cut off spend by Billing project (see How to disable billing for step-by-step instructions).

A warning about controlling costs when using shared billing A Terra billing project user can create their own workspaces. As the workspace owner, they will be able to store and analyze data (i.e. accrue cost) in these workspaces and the GCP costs will be billed to the shared Terra Billing project

Billing project users (including the billing owner) don't automatically have access to workspaces created by others on the same Terra billing. That requires the workspace owner to grant the appropriate workspace permission to the user (see Working in a shared workspace, above).

To learn more about sharing billing versus sharing workspaces, see Managing shared resources with groups and permissions.

Step 1: Create a Terra Billing project

1. Expand the User Profile drop-down from the main navigation menu and select Billing.
S6a_Feb22_2019.png Screen Shot of Billing in main navigation menu

2. Click on Create a New Project.

3. If prompted, click to enable billing permissions, select the Google ID of the Google Billing account owner, and click Allow

S6b_Feb22_2019.png Screen shot - create a new Billing Project

4. Enter a unique name for your Terra Billing Project. Names must follow these rules:
     - Must be between 6 and 30 characters in length
     - May only contain alphanumeric characters, underscores, and dashes
  S6c_Feb22_2019.png Screen shot - Name new billing project

4. Select a linked Google Cloud Billing account to associate with the Terra billing project.
Note:
You may see multiple GCP Billing accounts. If you need to locate a GCP Billing account ID, navigate to the Google Developers Console and click on Billing. Look for the number below Billing account ID.

Step 2: Protect your group from getting locked out of billing

If your Terra Billing project will be used by a group (such as members in a research lab, or scientists with a common funding source), make sure to have more than one individual with "owner" permission. That way you will not be locked out of the Billing project if the sole owner leaves the group. 

Option 1: Create a Terra Managed Group and give the group "owner" permission 

See step-by-step instructions in Managing shared resources with groups and permissions

Option 2: Add a second owner in the UI

1. Go to your billing page
You'll find it under the main navigation (top left of any page) in the drop down under your name.
Add-billing-project-user_Go-to-billing-page_Screen_shot.png

2. Click on the Billing project link from the list. You'll be directed to the Billing project management page (below).
Billing-project_Add-user-add-owner_Screen_shot.png

3. Under the Users tab (right side), select the blue Add Users button. 

4. Update tan existing user's role to owner by selecting Edit role (pencil icon)

Add/remove collaborators on a Terra Billing project (step-by-step instructions)

Note that Cloud Billing Account owners and administrators (only) can add/remove collaborators from Terra billing. 

1. Navigate to Billing from the main navigation menu (Main menu > user name > Biling). 

2. Select the Billing Project from the list at the left.  
Controlling_lab_costs_Add_project_user_Screen_Shot.png

3. To add a collaborator, click on the Add a User card.

4. To change the collaborator role, click the Edit Role or Remove link at the right of their user ID.

Controlling spend/Disabling billing 

Teams change, and it's important to be able to disable billing by members no longer on a shared funding source. 

Important steps to control spend when using shared Billing projects Removing someone from a billing project does not keep them from spending money in an existing workspace that has been shared with them! It removes their ability to create new workspaces.

Collaborators can accrue costs even if you remove their billing permission!! Once you remove someone's billing permission, they can no longer create new workspaces. However, they can still work in existing workspaces, and that work will be billed to the Terra Billing project. To avoid accruing additional charges in existing workspaces, you must eliminate all possible sources of GCP costs following the checklist below.

Step 1. Remove workspace access
Choose this option if you still want to keep these workspace. You will need to remove the person from the managed group (for workspaces shared with a group, as recommended above) or remove them from each workspace (if workspaces are shared individually).

Step 2. Disable/transfer billing
Delete all workspaces created by the user
 (if you don't mind losing the data and other resources in the workspace) 
     - or -
Disable billing on the workspace (workspaces created after September 24, 2021) if you want to keep the workflows and data tables and documentation. See step-by-step instructions here
     - or -
Assign a different owner and remove the user from the workspace (if you want to keep the workspace).

If you are no longer using a shared Terra Billing project, you can disable billing for any projects (and all the associated workspaces) using the Swagger API. This will remove the GCP Billing account immediately from a Terra Billing project (i.e. you will not be able to clone a workspace). This will also disable billing on all workspaces created prior to September 24, 2021 immediately.

See step-by-step directions for disabling billing on a Terra Billing project using Swagger here

Before you start - make sure you have the right permission! In order to disable billing, you need to have owner or admin permission on the GCP Billing account. If you are using STRIDES or third party resellers such as Onix, you may not be able to follow the steps unless you request admin privileges from the third-party reseller (who is the Owner).

Workspaces created after September 24, 2021: Terra will then asynchronously remove the billing account from all the Google projects of all workspaces created under the Terra billing project. 

Scope: This will prevent additional costs accruing in all workspaces under this Terra billing project. Once you disable billing, no one will be able to start a workflow or notebook in any workspace associated with this project (if you disable the Terra billing project) - effective immediately.

Why doesn't removing someone's billing permissions control their spend? GCP costs are billed per billing project (i.e. by Terra workspace), not per user. This means, if the workspace still has a valid billing project, someone with "can-compute" permissions will still be able to run workflows or notebooks.

What can someone do without billing permissions?
When you remove users from the billing account or project, they will not be able to create new workspaces, but they can still accrue GCP costs in a workspace shared with them!

Was this article helpful?

0 out of 0 found this helpful

Have more questions? Submit a request

Comments

0 comments

Please sign in to leave a comment.