Publish a Docker container image to Google Artifact Registry (GAR)

Learn how to create and use a public or private Google Artifact Registry (GAR) Docker image from inside Terra. This guide is useful for migrating or mirroring existing images from other repositories such as DockerHub. For a primer on Docker containers and related terminology, see this Dictionary entry.

Some of these instructions were adapted from Google Artifact Registry documentation. 

See How to configure Google Artifact Registry to prevent data transfer charges.

Step 1. Set up prerequisites

Before you begin, there are a few setup steps to complete. Note: Do these steps only once, the first time you publish a Docker container image. 

• Install Docker (see this tutorial for guidance).
• Install Google Cloud SDK.
• Set up a Google Cloud Billing account and Project.

  Setting up in Google first You need a Google Billing account to set up a new Google Project to store your images. This project must be different than your Terra Billing Project.

• Once you have a Google Billing account, follow the "Before you begin" section of the Google Artifact Registry Quickstart tutorial to create a Google project and a Google Docker repository.

Step 2. Create or obtain a container image

To create a new image, follow the first section of this Docker image tutorial to create and tag an image.

If you already have an image and local copy to use, continue to the next step (step 2. Tag image).

If your image is stored in DockerHub and you want to migrate it to Google Artifact Repository, pull it locally with this command:

docker pull <REPOSITORY-NAME>/<IMAGE>:<TAG>

where <REPOSITORY-NAME> is the name of the repository where the Docker is stored, <IMAGE> is the name of the image you are pulling, and <TAG> is a keyword or version number that you want to attach to identify a specific image.

Why not use the default tag? Note: The default tag is latest. We don't recommend using the "latest" tag when pipelining commands in real work, as the version running might change without warning. It's better practice to use explicit tags for pulling and pushing images.

Step 3. Tag image with registry name

Before pushing the Docker image to Artifact Registry, you need to tag it with its new repository name. This configures the docker push command to push the image to a specific location within your repository.

The new registry name format is: [LOCATION]-docker.pkg.dev/[PROJECT-ID]/[REPOSITORY]/[IMAGE].

• [LOCATION] is the regional or multi-regional location of your repository (e.g., us-central1, europe-west1).

• [PROJECT-ID] is your Google Cloud project ID.

• [REPOSITORY] is the name of your Artifact Registry repository.

• [IMAGE] is your image's name.

To tag your Docker image for Artifact Registry, run this command:

docker tag [IMAGE] [LOCATION]-docker.pkg.dev/[PROJECT-ID]/[REPOSITORY]/[IMAGE]

For example:

docker tag quickstart-image us-central1-docker.pkg.dev/my-project/my-repo/quickstart-image

Now, you're ready to push your image to Artifact Registry!

Step 4. Push your image to Google Artifact Registry (private)

To push your Docker image to Artifact Registry, run this command:

docker push [LOCATION]-docker.pkg.dev/[PROJECT-ID]/[REPOSITORY]/[IMAGE]

For example:

docker push us-central1-docker.pkg.dev/my-project/my-repo/quickstart-image

You can view your freshly pushed image in Artifact Registry by visiting the Google Cloud Console. Go to Artifact Registry and select the repository where you pushed the image.

By default, Google stores your images privately. Only authorized users with appropriate permissions have access to them. You have the option of sharing your images publicly as well; see the next step, which is optional.

Step 5. Optional: make your image public

You can also make your Artifact Registry repository public, which allows anyone to pull images from it without authentication. This is useful for public projects or if you want to use the images in environments like Terra or Cromwell without additional access steps.

To do this through the Google Cloud Console, you need to configure the IAM policy for the repository.

5.1. Navigate to the Artifact Registry service in the Google Cloud Console.

5.2. Select the repository you want to make public.

5.3. Click on the PERMISSIONS tab at the top.

5.4. Click GRANT ACCESS.

5.5. In the "New principals" field, type allUsers.

5.6. In the "Select a role" drop-down menu, choose the Artifact Registry Reader role.

5.7. Click SAVE.

This will grant pull (read-only) permissions to all users for all images in this repository.

Choosing public or private bucket permissions Note: Each Artifact Registry repository is a distinct entity. You can make one repository public while keeping others private. If you have specific images you want to make public while keeping others private, create a separate public repository for them.

If you later change an image from public to private, be aware that any pipelines or services using that formerly public image will break for anyone without explicit access to the private image.

Step 6. Share your repository with a Terra group

You can share your Artifact Registry repository with a new or existing group within Terra. We strongly recommend sharing with a group, as you can easily add new members later, and they will automatically gain access to all images shared with that group.

6.1. Go to Terra and view your groups.

6.2. To create a new group, click "Create New Group..." and follow the instructions.

6.3. Copy the email address under “Group Email”, for example, <YOUR-GROUP-NAME>@firecloud.org.

6.4. Navigate back to the Artifact Registry service in the Google Cloud Console.

6.5. Select the repository you want to share.

6.6. Go to the PERMISSIONS tab.

6.7. Click GRANT ACCESS.

6.8. Paste the group email address in the "New principals" field.

6.9. In the "Select a role" drop-down menu, choose Artifact Registry Reader.

6.10. Click SAVE.

This will give pull (read-only) permissions to Terra users in the group.

Step 7. Use the image in a workflow or interactive analysis

You can use a custom Docker image when setting up the virtual machine (VM) that runs an analysis in Terra.

• You can use a custom Docker image from Artifact Registry when setting up the virtual machine (VM) that runs an analysis in Terra.

  The path to your image in Artifact Registry is [LOCATION]-docker.pkg.dev/[PROJECT-ID]/[REPOSITORY]/[IMAGE].

  For example:

  us-central1-docker.pkg.dev/my-project/my-repo/quickstart-image

  Use this image path in your WDL workflow.

• To use your Docker image in an interactive analysis (Galaxy, Jupyter Notebook or RStudio), follow the steps below.

  7.1. Select "Custom" from the Environment drop-down menu.

  7.2.Input the container image path using the format<registry>/<repository>/<image>:<tag>.

  For example:

  us-central1-docker.pkg.dev/my-project/my-repo/quickstart-image:latest

  
  For more details, see Understanding and adjusting your Cloud Environment.