Getting started with GPUs in a Cloud Environment

A graphics processing unit (GPU) is optimized for linear algebra computations, such as matrix multiplication. Terra supports the use of GPUs when using Jupyter Notebook Cloud Environments. This feature is currently in beta; this article outlines some general limitations and gives a breakdown of cost estimates for available configurations.

To learn more about GPUs on Terra, check out Speed up your machine learning work with GPUs.

Background

In Terra, interactive cloud environment images extend from a base Docker image dependent on Google's Deep Learning platform. This platform includes packages for using the CUDA parallel processing platform, the TensorFlow machine learning platform, and the PyTorch machine learning framework. This image also installs the NVidia drivers necessary for GPU support. All of Terra's interactive cloud environment docker images extend from Terra's base docker image, and so all are intended to support GPUs.

How to add GPUs to your cloud environment

To add GPUs to your cloud environment's compute configuration, navigate to a workspace where you have Compute access. Click on the "Cloud Environment" button at the top right of your screen, then click the "Enable GPUs" check box in the "Cloud compute profile" section.

If you already have an existing environment, you'll see that the checkbox is unavailable, and you have to delete the environment manually. You can do this either by clicking the "delete" button at the bottom of the Cloud Environment widget or from the section of your profile that lists your cloud environments. This will make the GPU checkbox available for creating a new environment.

This is also true if you want to modify the GPU configuration of an existing environment. For instance, if you want to increase GPU power or change the GPU type, you need to delete the existing environment and re-create it with the desired configuration.

How to check that you successfully enabled GPUs

You can check that you successfully enabled GPUs and installed the libraries that are relevant to many Machine Learning analyses by running the code snippets below in a Jupyter notebook. Note that for these commands to succeed, you must have clicked the Enable GPUs checkbox.

• Note: this code will only work if you've set up your Cloud Environment using an Application Configuration that includes PyTorch (e.g., Pegasus). Check whether PyTorch is included in your Application Configuration by clicking What's installed on this environment? under the Application Configuration drop-down menu in the Cloud Environment setup menu.

  import torch
  print(torch.cuda.is_available())
  print(torch.version.cuda)
  print(torch.cuda.current_device())
  print(torch.cuda.get_device_name())

  If PyTorch is installed, you should see something like this:
  

• Note: this code will only work if you've set up your Cloud Environment using an Application Configuration that includes TensorFlow. Check whether TensorFlow is included in your Application Configuration by clicking What's installed on this environment? under the Application Configuration drop-down menu in the Cloud Environment setup menu.

  import tensorflow as tf
  print(tf.config.list_physical_devices('GPU'))
  from tensorflow.python.client import device_lib
  print(device_lib.list_local_devices())

  If TensorFlow is installed, you should see something like this:
  

GPU Limitations

• GPUs can be used on Terra with Jupyter Notebooks (e.g., for TensorFlow), but not with Galaxy or RStudio.
• As with other interactive analysis compute resources in Terra, only the n1 family of machines is supported.
• Terra does not support updating an existing machine's GPU configuration. If you need to modify your GPU-enabled machine, you need to delete and recreate the Cloud Environment.
• You may experience a runtime creation failure in one of the following circumstances:
  • You run up against your quota limitation. To find out how to check and change your quotas to fix the issue, see How to troubleshoot and fix stalled workflows.
  • You see a ZONE_RESOURCE_POOL_EXHAUSTED error; you can either wait a day or two and try again, or use Swagger to create a GPU-enabled virtual machine in another zone within the us-central1 region via the API. The default zone is "us-central1-a", so you need to change the "zone" parameter to one of the other available zones.
• Terra only supports GPU use with the standard VM, so make sure you don't select a Spark compute type (or a Hail image).

Cost estimates for available configurations

Each GPU type has limitations as to how many GPUs are available for a given quantity of CPUs and memory. Below are a set of tables - one table for each GPU type - outlining cost estimates for launching cloud environments with all available GPU configurations. You can read more about GPUs on the Google Cloud Engine and GPU pricing in the Google Cloud Compute Engine documentation.