Issues with notebooks and/or cloud environments can happen for a variety of reasons. Review some common issues users experience on Terra and find ways to troubleshoot.
Cloud Environment failed to create or is stuck loading
A common issue in creating a Cloud Environment is when creation fails or is stuck loading when starting or pausing for long periods of time.
Example screenshots
|
|
What to do
If you run into this issue, first try manually deleting your Cloud Environment.
1. Navigate to your list of Cloud Environments: app.terra.bio/#clusters.
2. Click on the trash bin icon to the right of the Cloud Environment to delete.
3. When deleting the cloud environment, you have the option to also delete the attached persistent disk. Try deleting just the cloud environment first, then creating a new one. If you still have trouble creating a new cloud environment, it's possible your persistent disk is in an error state. If this happens, you can try deleting the cloud environment and the persistent disk.
Best practices: Re-create cloud environments periodically
In general, it's good practice to delete and re-create your cloud environment periodically, since new software updates may impact compatibility with older cloud environments.
Don't lose work when deleting the persistent diskDeleting the persistent disk will delete all data and files saved to the disk (although notebooks saved to a workspace bucket will not be deleted). If you can't delete the persistent disk due to important data and/or files being lost, please reach out to Terra Support and we will assist you.
Galaxy users: Delete previous Galaxy instance first!If you need to delete and create a new Galaxy app, make sure the previous app is deleted first before creating a new app to restore files from the same disk.
Errors installing packages
Sometimes pip install --upgrade <pkg>` doesn't work successfully and you need to troubleshoot.
Delete and recreate the PD
Since package installs are written to the Terra detachable persistent disk, one troubleshooting approach is to delete and re-create that disk. Caveat: if you delete it during a troubleshooting session, you will lose any files (like generated data) on the disk.
Alternative: Start from an empty `packages/` directory
1. Open a terminal
2. Run the following commands to move all currently installed packages to another directory so they are no longer visible to pip, Python, and Jupyter:
cd $HOME
export PKG_STASH_DIR=packages-as-of-$(date +"%Y%m%d")
mkdir $PKG_STASH_DIR
# Move all the currently installed packages out of the existing
# destination directory for package installations.
mv packages/* $PKG_STASH_DIR
3. Now you can retry the pip commands to install the packages again!
Other troubleshooting tips
- Review error messages.
- Check package documentation for dependency information.
Custom cloud environments
Terra maintains preconfigured environments for you to use. If these environments don’t fit your needs, use a custom Docker Image or include a startup script. Or you can combine both - specify a custom Docker image and a startup script to modify your cloud environment.
Setting up a custom docker image
To learn more about developing and using custom Docker images in Terra, see these articles.
Supported container registries
- Dockerhub
- Github Container Registry
- Google Container Registry
Images hosted on Dockerhub and Github Container Registry must be public for you to use them in your cloud environment. Images hosted on Google Container Registry can be private as long as your Terra account can access them.
Using a startup script
A startup script is a bash shell script that is run once when a new cloud environment is created. It’s useful for one-time system configuration changes that must be performed as the root user, e.g.,
- Installing/updating commands/packages
- Modifying system configuration files
When to use a startup scriptStartup scripts are a great option for users who don’t want to set up and maintain a custom Docker image. They are a good balance of power and ease of use. It’s not difficult to make your own, and you can readily distribute the script for others who are using the same kernel in Terra.
Startup script example
#!/usr/bin/env bash
pip install nbconvert
apt-get update
apt-get install -yq pandoc texlive-xetex
R -e "install.packages('devtools')"
pip install scanpy[louvain]==1.4.4.post1 anndata==0.6.22rc1 h5py==2.9.0
R -e "BiocManager::install('multtest')"
Steps to use startup script
1. Create a startup script to perform the desired actions.
2. Upload/copy the startup script to a (workspace) bucket.
3. Copy the gs:// URI for the startup script.
4. Set the startup script location in the Cloud environment configuration pane, under Cloud compute profile.
5. Update the cloud environment.
Additional support
If you didn’t find an answer to your question above, feel free to reach out to support@terra.bio, keeping in mind our Terra Support Policy.
In your email, please include a description of your issue and as many of the following details as possible:
- Terra account email address
- Google Project ID for your workspace (found in the workspace dashboard on the right-hand side)
- Cluster ID (if possible, found in terminal, looks like saturn-<long character string>)
- Screenshot of your cloud environment configuration
- Approximate time issue first occurred
- Name of notebook
- Screenshot(s) of any error(s) you are receiving
If you can, please share your workspace with Terra-Support@firecloud.org and provide a link to your workspace with your inquiry.
Additional (external) resources
While our goal is to help you with all your Terra platform questions, some technology or language questions may be out of scope. For help regarding bioinformatics questions, we recommend asking the expert community in forums (see Bioinformatics StackExchange). For questions specific to the functionality of RStudio, Bioconductor, or Galaxy, please go to each of the application’s support forums: