Learn how to install and run the Python package gcloud storage for managing and moving files in Google Cloud Storage locally.
Note that gsutil has been deprecated and replaced with gcloud storageCloud SDK includes a new command line interface (CLI), gcloud storage, that can be considerably faster than gsutil when performing uploads and downloads with less parameter tweaking. This new CLI has a syntax and command structure that is familiar to gsutil users but is fundamentally different in many important ways. To ease transition to this new CLI, gsutil provides a shim that translates your gsutil commands to gcloud storage commands if an equivalent exists, and falls back to gsutil's usual behavior if an equivalent does not exist. See Shim for running gcloud storage (Google documentation) for more details.
When to use a local instance of gcloud storage
- Moving files from local storage to Google Cloud Storage (i.e., from your own computer to a Terra workspace bucket or a non-Terra Google bucket)
- Downloading data from Google Cloud Storage (i.e., Terra workspace bucket or a non-Terra Google bucket) to your personal computer
- Copying data between two Google Cloud Storage locations
Step-by-step instructions to install gcloud storage locally
First, follow the latest instructions for your computer's operating system using Google's documentation about installing gcloud CLI.
Next, be sure to initialize the software using gcloud init. Here is an example of how to run that command on a Unix-based operating system like Mac OS or Linux:
./google-cloud-sdk/bin/gcloud init
gcloud init not working?If running the gcloud init command isn't working for you, try opening a new terminal window or restarting your terminal to ensure that the installation changes take effect.
Finally, if you have not yet been prompted to authenticate, you can do so using the following command:
gcloud auth login
You will be prompted to choose an account to continue to Google Cloud SDK.
Make sure the Google account you use is the same account associated with Terra. Otherwise, you will not have authorization to use some of the commands or access the data that lives in Terra.
Using gcloud storage
Once Google Cloud SDK is installed, you have initialized the package, and authenticated with your Terra account, you are ready to use gcloud storage. See Google's documentation about Google Cloud CLI to learn more about the tool. To copy data between locations, you will want to use the gcloud storage cp command.
Verify gcloud CLI installationBefore downloading or copying data using gcloud storage cp, you can use the gcloud storage ls command to look at the buckets you can access.
Run gcloud storage ls -p [project name] to list buckets for a specific workspace project ID.
Run gcloud storage ls [workspace bucket] to list the contents of a bucket workspace bucket.
Other gsutil resources
For more practice, see the gcloud storage tutorial.