Getting started with GPUs in a Cloud Environment

A graphics processing unit (GPU) is optimized for linear algebra computations, such as matrix multiplication. Terra supports the use of GPUs when using Jupyter Notebook Cloud Environments. This feature is currently in beta; this article outlines some general limitations and gives a breakdown of cost estimates for available configurations.

To learn more about GPUs on Terra, check out Speed up your machine learning work with GPUs.

Background

In Terra, interactive cloud environment images extend from a base Docker image dependent on Google's Deep Learning platform. This platform includes packages for using the CUDA parallel processing platform, the TensorFlow machine learning platform, and the PyTorch machine learning framework. This image also installs the NVidia drivers necessary for GPU support. All of Terra's interactive cloud environment docker images extend from Terra's base docker image, and so all are intended to support GPUs.

Warning There have been widely reported issues with the Deep Learning base image having internal version conflicts involving CUDA and TensorFlow's tendency to automatically update to an unsupported version. For this reason, we want to point out that the GATK base image supports TensorFlow 2.4.2 and CUDA 11.0. Images that automatically install TensorFlow 2.5.0 and CUDA 11.2 are prone to compatibility issues.

terra-jupyter-base 1.0.+
terra-jupyter-bioconductor 2.0.+
terra-jupyter-gatk 2.0.+
terra-jupyter-hail 1.0.+
terra-jupyter-r 2.0.+
terra-jupyter-python 1.0.+

How to add GPUs to your cloud environment

To add GPUs to your cloud environment's compute configuration, navigate to a workspace where you have Compute access. Click on the "Cloud Environment" button at the top right of your screen, then click the "Enable GPUs" check box in the "Cloud compute profile" section.

If you already have an existing environment, you'll see that the checkbox is unavailable, and you have to delete the environment manually. You can do this either by clicking the "delete" button at the bottom of the Cloud Environment widget or from the section of your profile that lists your cloud environments. This will make the GPU checkbox available for creating a new environment. This is also true if you want to modify the GPU configuration of an existing environment. For instance, if you want to increase GPU power or change the GPU type, you need to delete the existing environment and re-create it with the desired configuration.

How to check that you successfully enabled GPUs

You can check that you successfully installed relevant libraries using some of the code snippets below (note that for these commands to succeed, you must have clicked the Enable GPUs checkbox):

PyTorch

torch.cuda.is_available()

print(torch.version.cuda)

torch.cuda.current_device()

torch.cuda.get_device_name(#)

TensorFlow

tf.config.list_physical_devices('GPU')

tensorflow.python.client import device_lib
print(device_lib.list_local_devices())

GPU Limitations

• GPUs can be used on Terra with Jupyter Notebooks (e.g., for TensorFlow), but not with Galaxy or RStudio.
• As with other interactive analysis compute resources in Terra, only the n1 family of machines is supported.
• Terra does not support updating an existing machine's GPU configuration. If you need to modify your GPU-enabled machine, you need to delete and recreate the Cloud Environment.
• You may experience a runtime creation failure in one of the following circumstances:
  • You run up against your quota limitation. To find out how to check and change your quotas to fix the issue, see Google Cloud quotas: What are they and how do you request more?.
  • You see a ZONE_RESOURCE_POOL_EXHAUSTED error; you can either wait a day or two and try again, or use Swagger to create a GPU-enabled virtual machine in another zone within the us-central1 region via the API. The default zone is "us-central1-a", so you need to change the "zone" parameter to one of the other available zones.
• Terra only supports GPU use with the standard VM, so make sure you don't select a Spark compute type (or a Hail image).

Cost estimates for available configurations

Each GPU type has limitations as to how many GPUs are available for a given quantity of CPUs and memory. Below are a set of tables - one table for each GPU type - outlining cost estimates for launching cloud environments with all available GPU configurations. You can read more about GPUs on the Google Cloud Engine and GPU pricing in the Google Cloud Compute Engine documentation.