Getting started with GPUs in a Cloud Environment

Anton Kovalsky
  • Updated

A graphics processing unit (GPU) is optimized for linear algebra computations, such as matrix multiplication. Terra supports the use of GPUs when using Jupyter Notebook Cloud Environments. This feature is currently in beta; this article outlines some general limitations and gives a breakdown of cost estimates for available configurations.

To learn more about GPUs on Terra, check out Speed up your machine learning work with GPUs.

Background

In Terra, interactive cloud environment images extend from a base Docker image dependent on Google's Deep Learning platform. This platform includes packages for using the CUDA parallel processing platform, the TensorFlow machine learning platform, and the PyTorch machine learning framework. This image also installs the NVidia drivers necessary for GPU support. All of Terra's interactive cloud environment docker images extend from Terra's base docker image, and so all are intended to support GPUs.

How to add GPUs to your cloud environment

To add GPUs to your cloud environment's compute configuration, navigate to a workspace where you have Compute access. Click on the "Cloud Environment" button at the top right of your screen, then click the "Enable GPUs" check box in the "Cloud compute profile" section.

 

Moving gif showinging a user clicking the 'Enable GPUs' checkbox for their Cloud Compute Profile.

If you already have an existing environment, you'll see that the checkbox is unavailable, and you have to delete the environment manually. You can do this either by clicking the "delete" button at the bottom of the Cloud Environment widget or from the section of your profile that lists your cloud environments. This will make the GPU checkbox available for creating a new environment.

This is also true if you want to modify the GPU configuration of an existing environment. For instance, if you want to increase GPU power or change the GPU type, you need to delete the existing environment and re-create it with the desired configuration.

How to check that you successfully enabled GPUs

You can check that you successfully enabled GPUs and installed the libraries that are relevant to many Machine Learning analyses by running the code snippets below in a Jupyter notebook. Note that for these commands to succeed, you must have clicked the Enable GPUs checkbox.

  • Note: this code will only work if you've set up your Cloud Environment using an Application Configuration that includes PyTorch (e.g., Pegasus). Check whether PyTorch is included in your Application Configuration by clicking What's installed on this environment? under the Application Configuration drop-down menu in the Cloud Environment setup menu.

    import torch
    print(torch.cuda.is_available())
    print(torch.version.cuda)
    print(torch.cuda.current_device())
    print(torch.cuda.get_device_name())

    If PyTorch is installed, you should see something like this:
    Screenshow showing an example of the output you can expect from running the code to check whether PyTorch has been successfully installed on your Cloud Environment.

  • Note: this code will only work if you've set up your Cloud Environment using an Application Configuration that includes TensorFlow. Check whether TensorFlow is included in your Application Configuration by clicking What's installed on this environment? under the Application Configuration drop-down menu in the Cloud Environment setup menu.
    import tensorflow as tf
    print(tf.config.list_physical_devices('GPU'))
    from tensorflow.python.client import device_lib
    print(device_lib.list_local_devices())

    If TensorFlow is installed, you should see something like this:
    Screenshot showing an example of the output you can expect from running the code to check whether TensorFlow has been successfully installed in your Cloud Environment.

GPU Limitations

  • GPUs can be used on Terra with Jupyter Notebooks (e.g., for TensorFlow), but not with Galaxy or RStudio.
  • As with other interactive analysis compute resources in Terra, only the n1 family of machines is supported.
  • Terra does not support updating an existing machine's GPU configuration. If you need to modify your GPU-enabled machine, you need to delete and recreate the Cloud Environment.
  • You may experience a runtime creation failure in one of the following circumstances:
  • Terra only supports GPU use with the standard VM, so make sure you don't select a Spark compute type (or a Hail image).

Screenshot of the drop-down menu used to select a VM type. An orange rectangle highlights the Standard VM option, which is the only type of VM that is compatible with GPUs.

Cost estimates for available configurations

Each GPU type has limitations as to how many GPUs are available for a given quantity of CPUs and memory. Below are a set of tables - one table for each GPU type - outlining cost estimates for launching cloud environments with all available GPU configurations. You can read more about GPUs on the Google Cloud Engine and GPU pricing in the Google Cloud Compute Engine documentation.

NVIDIA Tesla T4

# of CPUs Memory size (GB) # of GPUs Cost estimate (USD per hour)
1 3.75 GB 1 $0.41
1 3.75 GB 2 $0.76
1 3.75 GB 4 $1.46
2 7.5 GB 1 $0.46
2 7.5 GB 2 $0.81
2 7.5 GB 4 $1.51
2 13 GB 1 $0.48
2 13 GB 2 $0.83
2 13 GB 4 $1.53
4 15 GB 1 $0.55
4 15 GB 2 $0.90
4 15 GB 4 $1.60
4 26 GB 1 $0.60
4 26 GB 2 $0.95
4 26 GB 4 $1.65
8 7.2 GB 1 $0.65
8 7.2 GB 2 $1.00
8 7.2 GB 4 $1.70
8 30 GB 1 $0.74
8 30 GB 2 $1.09
8 30 GB 4 $1.79
8 52 GB 1 $0.84
8 52 GB 2 $1.19
8 52 GB 4 $1.89
16 14.4 GB 1 $0.93
16  14.4 GB 2 $1.28
16 14.4 GB 4 $1.98
16 60 GB 1 $1.12
16 60 GB 2 $1.47
16 60 GB 4 $2.17
16 104 GB 1 $1.31
16 104 GB 2 $1.66
16 104 GB 4 $2.36
32 28.8 GB 2 $1.85
32 28.8 GB 4 $2.55
32 120 GB 2 $2.23
32 120 GB 4 $2.93
32 208 GB 2 $2.61
32 208 GB 4 $3.31
64 57.6 GB 4 $3.68
64 240 GB 4 $4.45
64 416 GB 4

$5.20

NVIDIA Tesla K80

# of CPUs Memory size (GB) # of GPUs Cost estimate (USD per hour)
1 3.75 GB 1 $0.51
1 3.75 GB 2 $0.96
1 3.75 GB 4 $1.86
1 3.75 GB 8 $3.66
2 7.5 GB 1 $0.56
2 7.5 GB 2 $1.01
2 7.5 GB 4 $1.91
2 7.5 GB 8 $3.71
2 13 GB 1 $0.58
2 13 GB 2 $1.03
2 13 GB 4 $1.93
2 13 GB 8 $3.73
4 15 GB 1 $0.65
4 15 GB 2 $1.10
4 15 GB 4 $2.00
4 15 GB 8 $3.80
4 26 GB 1 $0.70
4 26 GB 2 $1.15
4 26 GB 4 $2.05
4 26 GB 8 $3.85
8 7.2 GB 1 $0.75
8 7.2 GB 2 $1.20
8 7.2 GB 4 $2.10
8 7.2 GB 8 $3.90
8 30 GB 1 $0.84
8 30 GB 2 $1.29
8 30 GB 4 $2.19
8 30 GB 8 $3.99
8 52 GB 1 $0.94
8 52 GB 2 $1.39
8 52 GB 4 $2.29
8 52 GB 8 $4.09
16 14.4 GB 2 $1.48
16  14.4 GB 4 $2.38
16 14.4 GB 8 $4.18
16 60 GB 2 $1.67
16 60 GB 4 $2.57
16 60 GB 8 $4.37
16 104 GB 2 $1.86
16 104 GB 4 $2.76
16 104 GB 8 $4.56
32 28.8 GB 4 $2.95
32 28.8 GB 8 $4.75
32 120 GB 4 $3.33
32 120 GB 8 $5.13
32 208 GB 4 $3.71
32 208 GB 8 $5.51

NVIDIA Tesla P4

# of CPUs Memory size (GB) # of GPUs Cost estimate (USD per hour)
1 3.75 GB 1 $0.66
1 3.75 GB 2 $1.26
1 3.75 GB 4 $2.46
2 7.5 GB 1 $0.71
2 7.5 GB 2 $1.31
2 7.5 GB 4 $2.51
2 13 GB 1 $0.73
2 13 GB 2 $1.33
2 13 GB 4 $2.53
4 15 GB 1 $0.80
4 15 GB 2 $1.40
4 15 GB 4 $2.60
4 26 GB 1 $0.85
4 26 GB 2 $1.45
4 26 GB 4 $2.65
8 7.2 GB 1 $0.90
8 7.2 GB 2 $1.50
8 7.2 GB 4 $2.70
8 30 GB 1 $0.99
8 30 GB 2 $1.59
8 30 GB 4 $2.79
8 52 GB 1 $1.09
8 52 GB 2 $1.69
8 52 GB 4 $2.89
16 14.4 GB 1 $1.18
16  14.4 GB 2 $1.78
16 14.4 GB 4 $2.98
16 60 GB 1 $1.37
16 60 GB 2 $1.97
16 60 GB 4 $3.17
16 104 GB 1 $1.56
16 104 GB 2 $2.16
16 104 GB 4 $3.36
32 28.8 GB 2 $2.35
32 28.8 GB 4 $3.55
32 120 GB 2 $2.73
32 120 GB 4 $3.93
32 208 GB 2 $3.11
32 208 GB 4 $4.31
64 57.6 GB 4 $4.68
64 240 GB 4 $5.45
64 416 GB 4 $6.20

NVIDIA Tesla V100

# of CPUs Memory size (GB) # of GPUs Cost estimate (USD per hour)
1 3.75 GB 1 $2.54
1 3.75 GB 2 $5.02
1 3.75 GB 4 $9.98
1 3.75 GB 8 $19.90
2 7.5 GB 1 $2.59
2 7.5 GB 2 $5.07
2 7.5 GB 4 $10.03
2 7.5 GB 8 $19.95
2 13 GB 1 $2.61
2 13 GB 2 $5.09
2 13 GB 4 $10.05
2 13 GB 8 $19.97
4 15 GB 1 $2.68
4 15 GB 2 $5.16
4 15 GB 4 $10.12
4 15 GB 8 $20.04
4 26 GB 1 $2.73
4 26 GB 2 $5.21
4 26 GB 4 $10.17
4 26 GB 8 $20.09
8 7.2 GB 1 $2.78
8 7.2 GB 2 $5.26
8 7.2 GB 4 $10.22
8 7.2 GB 8 $20.14
8 30 GB 1 $2.87
8 30 GB 2 $5.35
8 30 GB 4 $10.31
8 30 GB 8 $20.23
8 52 GB 1 $2.97
8 52 GB 2 $5.45
8 52 GB 4 $10.41
8 52 GB 8 $20.33
16 14.4 GB 2 $5.54
16  14.4 GB 4 $10.50
16 14.4 GB 8 $20.42
16 60 GB 2 $5.73
16 60 GB 4 $10.69
16 60 GB 8 $20.61
16 104 GB 2 $5.92
16 104 GB 4 $10.88
16 104 GB 8 $20.80
32 28.8 GB 4 $11.07
32 28.8 GB 8 $20.99
32 120 GB 4 $11.45
32 120 GB 8 $21.37
32 208 GB 4 $11.83
32 208 GB 8 $21.75
64 57.6 GB 8 $22.12
64 240 GB 8 $22.89
64 416 GB 8 $23.64

Was this article helpful?

Comments

0 comments

Please sign in to leave a comment.