Best practices for managing shared team resources

Allie Hajian
  • Updated

(09-01-2021) This article has been edited with changes for the upcoming Project per Workspace functionality. You can update with additional changes, but please do not publish! Once the PPW feature is out, all the articles will be released at one time!

Slack Allie Cliffe (@acliffe) with any questions. 

Understanding how to manage and control costs is an important consideration for a team with a shared funding source. Read on for two ways to share funding resources and which one to use - depending on your collaboration needs. 

For information on best practices for sharing data resources, see this article.

To learn about sharing GCP Billing accounts (if colleagues need to be able to create their own Terra Billing projects, or look at detailed cost breakdown and reports), see this article 

Overview: How costs are calculated in Terra

Working in Terra incurs GCP fees that are passed along to users without any markup via a Terra billing project. GCP costs are based on how much storage, egress and compute is done in the project workspace (bottom level). These workspace costs are covered via a Terra billing project (gray box, middle level), funded by a GCP Billing Account (top level). Terra billing projects can include many different workspaces, such as one for each member of a team with shared funding.

Workspaces created prior to September 24, 2021
The Terra billing project  maps 1:1 to a Google project. Since all GCP costs are calculated per Google project, costs are summed across all the workspaces in the Terra Billing project. 

Workspaces created after September 24, 2021
Terra creates a GCP project for each workspace, so GCP costs are calculated per workspace, and summed across all workspaces associated with the Terra billing. The workspace Google project is mostly invisible to the Terra user, unless you are interfacing directly with GCP console (when setting up workflow spend reporting or Google budget alerts, for example). 

Billing-heierarchy-in-cloud.png
Note: All Terra components in the diagram are gray and all GCP components are blue.

Consider the case of several collaborators with the same funding source. The funding will be dispersed through a GCP Billing account (top level above). Collaborators can access the shared funds at any level of the billing hierarchy, depending on whether they will be using their own or a shared workspace. For a detailed description of the structure of costs and permissions in Terra, see this article

Below are two options for managing shared team costs. 

Working in shared team workspaces (PIs have most control)

Workspace owners (i.e. lab PIs) can share a workspace with an individual or a managed group. The workspace owner controls what each collaborator can do in the workspace. Collaborators can only accrue costs if they have the right workspace permission (to add data, for example, or compute an analysis).

TIP: Note that workspace owners can change collaborator roles at any time. 

Workspace-permissions_Screen_shot.png

How to set up a shared team workspace

1. Create the workspace.

2. Make a managed group for the team by going to "Profile" -> "Groups" from the main navigation.

3. Return to the team workspace and click the three vertical dots.

4. Open the "Share workspace" form by selecting "Share" in the dropdown.
Share-workspace-vertical-dots_Screen_shot.png

5. Add the group (start typing in the group name in the "User email" field).

6. Select "enter" or "return" after entering the group name.

7. Select the group's permissions. 

Don't forget to save!

G0_icon-tip.png


Group versus shared permissions

  Note that everyone in the group shares the same workspace permissions. 

To assign more granular permissions, add individual collaborators one at a time. 

To learn more about managing shared resources with groups, click here.

Controlling spending in a shared workspace 

The workspace owner controls what collaborators can do by assigning workspace roles/permissions when sharing a workspace. Workspace permissions give the most control over spend, as owners can change or remove an individual or group's permission at any time, and it takes effect immediately. 

Operations that have a cost

  • Move, copy and store data
  • Run an interactive analysis (Jupyter notebook)
  • Run a workflow (note that this will generate data, including intermediate files, that will be stored by default in the workspace bucket, which also has a cost)
G0_warning-icon.png


All GCP costs are billed to the workspace billing project

  Note that it is not possible to get cost breakdowns by user for different users working in the same workspace

How to add/remove individuals from a shared workspace

1. Click on the three vertical dots at the top right to access the workspace "share" form and change the user's role:
Controlling_lab_costs_Share_workspace_Screen_Shot.png

2. If adding someone, you need to hit "enter" (or "return") after entering their user email.

3. Choose their role, or delete the person from the collaborator list. Controlling_lab_costs_sharing_workspace_modal_Screen_Shot.png

4. Don't forget to save after changing!

How to add/remove a user from a workspace shared with a Group

To do this, you will add/remove the individual from the managed group the workspace is shared with.

1. Go to "Your Name" > "Groups" from the main navigation menu
Controlling_costs_modifying_Groups_Screen_Shot.png

2. Edit the person's role in the group (you must have the right permissions, of course!)
Control_lab_costs_edit_user_role_Screen_Shot.png

Working in separate workspaces with team billing

Owners and administrators of the GCP Billing account can create one or more Terra billing projects for each GCP billing account - one for the entire team or separate one for different collaborators or different work in Terra. One advantage of this is it is easy to cut off spend by Billing project.

A warning about controlling costs
A Terra billing project user can create their own workspaces. The workspace creator is the "owner" by default, and will be able to store and analyze data in any workspaces they create. The GCP costs will be billed to the shared Terra Billing project. 

TIP: Being on a team Billing project does not allow the collaborator to work in any workspaces created by others in the team - even under the shared billing project! That requires the workspace owner to grant the appropriate workspace permission to the user (option 1, above).

G0_warning-icon.png


Protect your group from getting locked out of billing!

  If your Terra Billing project will be used by a group (such as members in a research lab, or scientists with a common funding source), make sure to have more than one individual with "owner" permission. That way you will not be locked out of the Billing project if the sole owner leaves the group.

Add a second owner in the UI
- or - 
Create a Terra Managed Group and give the group "owner" permission

Learn more about creating and using a managed group here.

Step-by-step instructions to create a Terra Billing project

1. Expand the User Profile drop-down from the main navigation menu and select "Billing".
S6a_Feb22_2019.png Screen Shot of Billing in main navigation menu

2. Click on "Create a New Project."

3. If prompted, click to enable billing permissions, select the Google ID of the Google Billing account owner, and click "Allow"

S6b_Feb22_2019.png Screen shot - create a new Billing Project

4. Enter a unique name for your Terra Billing Project. Names must follow these rules:
    - Must be between 6 and 30 characters in length
    - May only contain alphanumeric characters, underscores, and dashes
  S6c_Feb22_2019.png Screen shot - Name new billing project

4. Select a GCP Billing account to associate with the Terra billing project.
Note:
You may see multiple GCP Billing accounts. If you need to locate a GCP Billing account ID, navigate to the Google Developers Console and click on Billing. Look for the number below "Billing account ID".

Add a second billing project owner (click for step-by-step instructions)

1. Go to your billing page. 
You'll find it under the main navigation (top left of any page) in the drop down under your name:
Add-billing-project-user_Go-to-billing-page_Screen_shot.png

2. Click on the billing project.

3. Under the "Users" tab, select the blue "Add Users" button. 

4. Update their role to owner by selecting the "Edit role" icon
Billing-project_Add-user-add-owner_Screen_shot.png

How to add/remove collaborators on a Terra Billing project

Billing Account owners and administrators (only) can add/remove collaborators.

1. Navigate to "Billing" from the main navigation menu (Main menu > user name > "Biling"). 

2. Select the Billing Project from the list at the left.  
Controlling_lab_costs_Add_project_user_Screen_Shot.png

3. To add a collaborator, click on the "+" icon on the "Add a User" card.

4. To remove or edit the role of a collaborator, click the link at the right of their user ID.

G0_warning-icon.png


Controlling spend when using shared Billing projects

  Removing a colleague from billing
Remember that removing someone from a billing project does not keep them from spending money in an existing workspace that has been shared with them! It removes their ability to create new workspaces.

We repeat - Users can still run workflows and notebooks in a workspace even after you remove them from the billing project!! 

After removing billing project permissions, to avoid accruing additional charges, you must eliminate all possible sources of continuing costs (at the workspace and billing project level), following the checklist below.

  1. Remove user from shared workspaces owned by other team members (if you still want to keep these workspace) or remove them from the group (if you've shared with a managed group, as recommended above). 

  2. Delete all workspaces created by the user (if you don't mind losing the data and other resources in the workspace) 
         - or -
    Disable billing on the workspace (workspaces created after September 24, 2021) if you want to keep the workflows and data tables and documentation. See step-by-step instructions here
         - or -
    Assign a different owner and remove the user from the workspace (if you want to keep the workspace)

Disabling billing (at the project level)

If you are no longer using a shared Terra Billing project, you can disable billing for any projects (and all the associated workspaces) using the Swagger API. This will remove the GCP Billing account immediately from a Terra Billing project (i.e. you will not be able to clone a workspace). This will also disable billing on all workspaces created prior to September 24, 2021 immediately.

Workspaces created after September 24, 2021: Terra will then asynchronously remove the billing account from all the Google projects of all workspaces created under the Terra billing project. 

Scope: This will prevent additional costs accruing in all workspaces under this Terra billing project. Once you disable billing, no one will be able to start a workflow or notebook in any workspace associated with this project (if you disable the Terra billing project) - effective immediately.

See step-by-step directions for disabling billing on a Terra Billing project using Swagger here

G0_warning-icon.png


Before you start - make sure you have the right permission!

 

In order to disable billing, you need to have owner or admin permission on the GCP Billing account. If you are using STRIDES or third party resellers such as Onix, you may not be able to follow the steps unless you request admin privileges from the third-party reseller (who is the Owner).

What should I do next?
You will need to ask the Billing account owner for admin privileges to disable billing.

 

G0_icon-tip.png


Why doesn't removing someone's billing permissions control their spend?

 

GCP costs are billed by billing project (i.e. by Terra workspace), not by user.

This means, if the workspace still has a valid billing project, any user with "can-compute" permissions will still be able to run workflows or notebooks in workspaces.

What can someone do without billing permissions?
When you remove users from the billing account or project, they will not be able to create new workspaces, but they can still accrue GCP costs in a workspace shared with them!

Was this article helpful?

0 out of 0 found this helpful

Have more questions? Submit a request

Comments

0 comments

Please sign in to leave a comment.