Protecting data from a Cloud Environment created prior to August, 2020

Allie Hajian
  • Updated

Cloud Environments (Galaxy, Jupyter or RStudio) created prior to August 1, 2020 are incompatible with current features, and must be recreated. This article outlines how to see if you have an incompatible Cloud Environment as well as step-by-step instructions for protecting generated data and recreating the Cloud Environment. 

1. Identify notebooks in old clusters

To see what Cloud Environments you created under each billing project, and when you created them, go to https://app.terra.bio/#clusters.

See a virtual machine or cluster created before August 1st, 2020? Note the Billing Project name and keep reading. 

Screen_Shot_2020-10-09_at_1.39.55_PM.png

2. Check content of affected notebooks

To check what notebooks are affected, go to Your Workspaces and filter by the Billing Project (from step one above). Within the workspace, check in the Notebooks/Analyses tab of the filtered workspaces to see what analysis you ran. 
Screen_Shot_2019-10-08_at_1.10.38_PM.png

3. Save any data you want to keep

If you don’t save your output data to workspace storage (i.e. Google bucket), or if you delete your Persistent Disk, the data will be lost when you delete your Cloud Environment.

If there’s output data from a notebook you want to keep, you need to explicitly save it to the workspace bucket. Copying notebook output to a Google bucket explains exactly how. 

4. Create a new Cloud Environment

4.1. When you are ready to create a new Cloud Environment (e.g. you’ve copied your files into workspace or other storage), click on the Cloud Environment widget at the top right corner of your screen (Notebooks tab display) or Cloud icons in the sidebar (Analyses tab display).

4.2. In the Jupyter Cloud Environment pane, click Delete Environment Options near the bottom.
Screen_Shot_2021-04-09_at_6.27.45_PM.png

4.3. If you do not need to customize your Cloud Environment, you can create a new one now.

If you want to select the number of your compute instance CPUs , or enter a start-up script, click on the "Customize" option for different application configurations and other custom settings. 
Screen_Shot_2020-10-09_at_1.37.56_PM.png

To learn more about your PD deletion options, see Detachable persistent disks.

5. What if you used startup scripts?

Once you re-create the Jupyter Cloud Environment, you will need to re-run the startup script to install custom software, libraries and dependencies. Put the URI for the script in the Startup script field of the Custom compute profile before recreating the Jupyter environment. 
Jupyter-Cloud-Environment_Choose-startup-script_Screen_shot.png

Using a GATK custom startup script (workshops)

If you used the GATK custom startup script in a workshop, the URL to use when running the notebook again should be in the notebook itself.

Can't find your personal startup script? 

If you used your own custom startup script but can’t remember which one you used, we can help you find that information. Email us at dsp-education@broadinstitute.org or slack us at #dsp-comms-user-ed for help. 

Was this article helpful?

0 out of 0 found this helpful

Have more questions? Submit a request

Comments

0 comments

Please sign in to leave a comment.