This article explains what to expect in Terra when interacting with data in a "Requester Pays" bucket. Requester Pays is an optional Google setting for data stored on the Google Cloud Platform. It allows dataset owners to make data accessible to everyone without having to pay egress fees when someone reads or copies the data from a different region.
NOTE: If you are interested in having Requester Pays enabled on your workspace bucket, please write to email@example.com with a link to your workspace and the name of the workspace bucket. You will not be able to run compute in the workspace if Requester Pays is enabled on the workspace bucket. Workspaces with Requester Pays enabled are typically read-only workspaces.
- What is a requester pays bucket?
- How to access requester pays data
- With an active billing project
- With a new Billing Project (including $300 GCP "getting started" credits
- With gsutil
- Charges for which you are responsible
- What happens when you leave a workspace with requester pays data enabled
What is a Requester Pays bucket/workspace?
In general, accessing data in a bucket in Google Cloud will generate an associated charge. These actions include:
- Performing a request
- Reading data
- Retrieving data
- Storing data
In the default setting (without requester pays enabled), these costs are charged to the Google Billing account used to create the bucket (the owner of the data, in other words). However, in some cases, data custodians will want to host data without being on the hook for every costly action taken in the bucket they are publicly hosting.
When Requester Pays is enabled, the host of the data only pays for data storage. Other costs are charged to the user accessing the data.
How to access requester pays data
When you try to access data in a bucket or workspace that has Requester Pays enabled, Terra will alert you with one of the three following pop-up windows. The alert you will see depends on which options are available to you.
1. Using an active Billing Project
If you have a Terra Billing Project to which you can charge any costs you incur, you will see a pop-up with a drop-down menu allowing you to select the Billing Project to charge:
2. Using a new Billing Project
If you have have not yet set up your own Billing Project, the pop-up will direct you to the Billing section, where you can set up a GCP Billing account and fund it with $300 in GCP "getting started" credits:
3. using gsutil
So long as you can provide a project to bill, you can run a gsutil cp command on an object in a requester pays bucket from a notebook, in a workflow, or on the command line:
gsutil -u <project_to_bill> cp <gs://path/to/file> <destination>
A note of caution: If you are mistaken about the bucket in question being a "Requester Pays" bucket and use this command, you may inadvertently charge the bucket owner for the egress.
Charges for which you are responsible
The following operations within a Requester Pays workspace generate charges for which the you, the requester, are responsible:
- Listing files in the “Files” section of the Data tab
- Previewing files in the Data tab (either from the “Files” section or an entity table)
- Downloading files in the Data tab (either from the “Files” section or an entity table)
- Listing notebooks in the “Notebooks” tab
- Getting a read-only preview of a notebook
- Launching a notebook for editing (if you have can-compute permission on the workspace)
- Copying a notebook to another workspace
- Copying notebooks when cloning a workspace
What happens when you leave a requester pays workspace
Please note that when you leave a Requester Pays workspace, the Billing Project selection you made will be cleared, and the next time you enter that workspace, you will be prompted to select a Billing Project again.