How to set up Terra on Azure (billing admins)

Allie Cliffe
  • Updated

Step-by-step instructions for team finance admins (with access to the Azure subscription that will cover cloud costs) to set up cloud billing for Terra on Azure. Once you go through these steps, you will be able to set up workspaces to store and analyze data in Terra.

Steps to set up billing (admins)

Terra-on-Azure_Billing-setup_Diagram.png

Terra on Azure supports several regions (with more coming soon!)Terra is available in the global Azure marketplace and all your resources will be created in the region you specify when deploying your Terra Environment. The Terra AMA (Azure Marketplace Application) can be deployed to the following regions.

      Central US | East US | South Africa North | South Central US | West US 2 | UAE North

The default (preferred) region is South Central US. NOTE: You will need to  select from the dropdown (there is no default region). If you need to deploy Terra on Azure to a different region, please contact support (support@terra.bio).

Step 1: Set Azure subscription prerequisites

Before you can create the Terra Azure Managed Application, you must be a member of at least one organization’s subscription in the Microsoft Azure Portal.

If your organization already uses Azure

You will need to request access to an Azure Subscription you can use for working in Terra from your IT admin. 

If you have trouble registering on Terra (app.terra.bio) because your work email is already associated with an Azure tenant, you can try using a personal email. You may need to ask your IT admin for further assistance. 

If your organization is new to Azure

To get started using Azure, you can set up a pay-as-you-go account on the Azure Portal here.  

Free credit accounts aren't compatible with TerraAzure's free credit offering creates a subscription with limitations set on quotas and resources that are needed to power Terra. Transitioning the free subscription to paid does not resolve these issues. For these reasons, we recommend not using free credits to try the Terra platform.

To limit the amount you spend while on a pay-as-you-go account, see How to set up an Azure Cloud cost budget

Overview: Subscription prerequisites 

Your Terra environment relies on several Azure services, listed below, for tasks like authentication, setting up storage and compute, and accessing cost information. The Terra managed application is empty when you deploy it in the Azure portal.

Terra and your Azure resource group

For Terra to work, your Azure subscription must have a number of resource providers enabled. Resources get deployed in the Managed Resource Group by the Terra Control Plane (which runs externally and is operated by the Broad Institute) when you perform certain actions in Terra UI. Thus you can think of Terra as a SaaS offering that happens to run some infrastructure in users’ Azure subscriptions.

Compute resources

  • AKS for running core analysis services and user-defined applications
  • Compute Data Science VMs for running tools like JupyterLab
  • Batch for running batch jobs (e.g. via Cromwell or Nextflow)
  • Azure Relay for proxying inbound http traffic

Storage resources

  • Storage Account + Blob storage
  • Postgres Flexible

Additional infrastructure

  • Log Analytics
  • App Insight
  • VNet and dedicated subnets for each compute resource

Resource providers that must be enabled

For additional information about these resources, see Additional Resources below.

  • Microsoft.Storage
    Terra leverages Storage Accounts and Blob Containers for holding unstructured data in Terra workspaces.
  • Microsoft.Compute
    Terra allocates Linux Virtual Machines to run JupyterLab and other analysis applications in Terra workspaces.
  • Microsoft.Authorization
    Registered by default. Terra authenticates through Azure Resource Manager (ARM) for managing cloud resources in your subscription.
  • Microsoft.Batch
    Terra leverages Azure Batch to run batch workflows using Cromwell in Terra workspaces.
  • Microsoft.OperationalInsights
    Terra provisions a Log Analytics Workspace to collect system logs, audit logs, and user application logs.
  • Microsoft.OperationsManagement
    Terra deploys a Data Collection Rule to connect Azure Kubernetes Service to the Log Analytics Workspace.
  • Microsoft.Insights
    Terra leverages Application Insights to monitor system health of applications in Terra workspaces.
  • Microsoft.Network
    Terra allocates a Virtual Network (VNet) and subnets to host applications running on compute resources.
  • Microsoft.DBforPostgreSQL
    Terra allocates a PostgreSQL Flexible Server for hosting SQL Databases for applications in Terra workspaces.
  • Microsoft.ContainerService
    Terra leverages Azure Kubernetes Service (AKS) to run applications in Terra workspaces, such as Cromwell and Workspace Data Service.
  • Microsoft.Relay
    Terra leverages Azure Relay to allow end users to securely access Terra workspace applications via https.
  • Microsoft.ManagedIdentity
    Terra creates User-Assigned Managed Identities and assigns them to compute resources, to allow for authentication and data access.

How to enable resource providers

You must have permission on your subscription to register a resource provider. This permission is included in Contributor and Owner roles.

1.1. Go to the Subscription page.

1.2. Go to the Resource providers section.

1.3. Search for each resource from the list above. If it isn’t registered, select it, then select Register. Screenshot 2023-12-07 at 12.04.32 PM.png

1.4. Complete registering all resources.

If you still get a resource error when creating a billing project, the error will identify what other resources still need to be registered.

Additional resources

Step 2: Create the Azure Managed Application

2.1. Go to the Terra application in the Managed Application Marketplace in the Azure portal under your subscription.

Can't find the Terra application?

The application can also be found under the Analytics or Compute categories or by searching for The Broad Institute publisher name in the marketplace.

ToA-Billing-1.2_Terra-in-Azure_Marketplace_Screenshot.png

Troubleshooting: If you cannot find the Terra ApplicationYou may need to explicitly add the Terra B2C tenant to your Azure tenant as an Enterprise Application.

What to do
Go to Home > Enterprise Applications > Consent and Permissions and select the Allow user consent for apps radio button.
ToA-EnterpriseApplications_Consent-and-permissions_Screenshot.png

2.2. Click on the Terra application and then click Create to surface the Terra application in Azure portal (screenshot below).

ToA-Billing-1.3_Creating-the-Terra-Managed-app_Screenshot.png

2.3. Enter the required information in the creation wizard.

Screenshot of the creation wizard
ToA-Billing-2.4_Creation-Wizard_Screenshot.png

Required information explanation

  • Subscription: The Azure subscription to bill to.
  • Resource Group: The parent collection of resources that the Terra application will belong to. This can be a new group or an existing one within the subscription. If you are creating a new one, it is useful to give a name that helps you easily identify the resource group. For example, if you have different resource groups in different locations, include the location in the name. 
  • Location: The region where the Terra application and its resources will be located.

    Location caveats: If you have data residency requirementsNote that this is the region where all data - including data in tables and in workspace blob storage - will be stored.

    The preferred region is South Central US (note that you will have to select from an alphabetical list in the dropdown). Terra also supports Central US, West US 2, East US, UAE North, and South Africa North. If you need to deploy Terra on Azure to a different region, please contact support (support@terra.bio).

  • Authorized Terra User: The email of the Terra user who will link a Terra billing project to this managed application. This could be the IT/Finance admin setting up billing, or you may designate a different person to set up on the Terra side (team lead or PI). You can use a comma-separated list of multiple email addresses.

    Microsoft and Google Authorized Terra UsersAn Authorized Terra User can be any active Terra account user, regardless of whether they use Microsoft or Google to sign into Terra. 

  • Application Name and Managed Resource Group: These are user-configurable values that refer back to the Terra application within the Azure portal.

    Names are limited to 40 charactersNote that this cannot be validated on the marketplace side. In order to avoid problems, you will need to keep track.

2.4. Click Review and Create, accept the license terms, and then click Create.

ToA-Billing-1.5_Create-Managed-app-final-step_Screenshot.PNG

It can take an hour or two to deploy the application We recommend waiting at least an hour before proceeding to the next step. If you run into problems following the next steps, please wait another hour or two before contacting support.

Step 3: Connect Terra to the Azure Managed Application

Note that if you haven’t yet registered for a Terra account, you will need to register first.

3.1. Wait until the managed app has been deployed in the Azure portal.

ToA-Billing-1.6_Deployment-complete_Screenshot.PNG

3.2. Go to the Terra Billing page (https://app.terra.bio/#billing) and sign in with the credentials for the Authorized Terra User (from 2.3 above).

Terra-on-Azure_Login_Screenshot.png

3.3. Click the Create button, and (3) select Azure Billing Project.

ToA-Billing-2.2_Create-Terra-on-Azure-Billing-Project_Screenshot.png

3.4. Follow the steps to link your Azure Subscription to Terra and create a new Terra Billing project (your own Terra Environment instance). Enter an Azure subscription ID, add additional users (optional), and enter a Billing Project name in the form.

ToA_Screenshot-of-UI-to-link-an-Azure-subscription-to-Terra.png

Finding the subscription ID

You'll find the subscription on the Azure Managed Application homepage. To get to that from the deployment page displayed in Step 3.1, click Go to Resource. Then you will see the Subscription ID like this:

ToA-Billing-2.3_Subscription-ID-inManaged-Application_-homepage_Screenshot.png

If you don’t see the Azure Managed Application Make sure your email address matches the “Authorized Terra User” entered in Step 1.4.

Colleagues don’t need to be on the Billing Project to collaborate It’s always best to start by giving colleagues minimal access and grant additional permissions as needed. The admin or PI who sets up billing can create workspaces and add collaborators as co-owners or writers. How you share funding, data, and analysis tools ultimately depends on your group's needs.

3.5. Click the Create button. You should now see the Azure Terra Billing Project in the list to the left.

ToA-Billing-2.4_Terra-on-Azure-Billing-Project-on-billing-page_Screenshot.png

This operation takes approximately 15 minutes to completeYou’ll see a loading spinner next to the project to indicate progress.

This is when you start to accrue the fixed workbench infrastructure cost ($10/day).

Troubleshooting

If you get an error message that includes the phrase “Missing required providers,” you will need to go back and make sure the indicated providers are enabled. These may be disabled by default for your organization. If so, you will need to ask to have them enabled.

What to do

See Step 1: Set up Azure subscription prerequisites for additional guidance.

Next steps: Set up workspaces/team access

Terra Billing project users can now create workspaces where your team can collaborate under your Terra Billing project (Terra Environment instance). 

Working in a shared workspace versus creating workspaces

  • All workspace costs are paid by the linked Azure Marketplace subscription via the associated Terra Billing project.
  • The workspace owner (creator) controls exactly what each collaborator can do in the workspace (i.e., reader, writer, owner roles). For a detailed description of shared workspace roles, see Sharing data and tools (workspace permissions.
  • Collaborators can only accrue costs (run an analysis or store or egress data) if the workspace owner gives sufficient workspace permission (writer or owner).
  • Sharing workspaces is a way to share a funding source that gives less control to individual team members. Colleagues in a shared workspace can perform actions with a cost (store data, run analyses) but cannot create workspaces (which have automatic infrastructure costs). 
  • Colleagues cannot create workspaces of their own unless they are on a Terra Billing project.

Controlling cost and access

Owners have fine-grained control of what collaborators in a shared workspace are able to do. When you share the workspace, you give each person or managed group reader, writer, or owner access.

For more details, see Sharing data and tools (workspace permissions).

To try out working in Terra

 

Was this article helpful?

Comments

0 comments

Please sign in to leave a comment.