How to set up persistent disk storage for your analysis app

Allie Cliffe
  • Updated

Learn how to set up detachable persistent disk storage when running an interactive analysis app (Jupyter notebook, RStudio, or galaxy). 

To learn more about detachable persistent disks, see the overview article

Step-by-step instructions

When you click on the Cloud Environment button, you should see the configuration options for your environment in this popup. At the bottom is a box for entering the type and size of your persistent disk. 

Detachable-persistent-disk_Create-in-Jupyter-Cloud-Environment-setup-pane_Screen_shot.png

If you modify the configuration of an existing Cloud Environment, you'll see the "Update" button turn blue (active). Clicking this button will reveal this message, letting you know that your work will be preserved through deletion and recreation.

Screen_Shot_2020-09-16_at_2.44.18_PM.png

Warning when decreasing your PD size

Decreasing your persistent disk will remove active code and any files on the PD. You could lose work if you choose to decrease the PD size in the middle of the analysis. Updating the PD with a smaller disk size will trigger a warning message to this effect:

Screen_Shot_2021-09-02_at_4.58.10_PM.png

You can click "Delete Environment Options" to see the options shown below.

Delete-Cloud-Environment_warning-pane_Screen_shot.png

If you don't want to save the contents of your detachable Persistent Disk, select the "Delete everything, including persistent disk" option. Just make sure you've moved anything you wish to keep from the Cloud Environment VM to another location, such as your workspace bucket.

See How to transfer data between your Cloud Environment PD and workspace storage.  

Default: Keep persistent disk

Selecting the default option, "Keep persistent disk, delete application and compute profile", will delete the VM after detaching the Persistent Disk. This disk will automatically be reattached the next time you spin up a cloud environment, assuming you select the standard VM.

When you click “Delete” here, you should see the popup below, where you can select a configuration before creating a new VM.

Screen_Shot_2020-09-16_at_3.49.15_PM.png

If you choose the standard VM, it will automatically reattach the saved disk. If you choose a Spark mode (clicking the “Customize” button shown below will show additional options), this storage will NOT reattach to that cloud environment because Spark and hail application configurations don't support the persistent disk feature.

The PD will, however, be saved until the next time you choose the standard VM option and click “Create”.

You can also click “Delete Persistent Disk”, if you no longer need to save files and data stored there. You’ll see a similar menu as before, but with only the option of deleting the persistent disk.

Delete the cloud environment, then the PD

Note that you can't delete a persistent disk that's attached to a cloud environment without first deleting that environment. If you go to your Cloud Environments page (main menu navigation > your name > "Cloud Environment" from the top left corner of any page in Terra), you'll see separate items for the Cloud Environment application and the detachable Persistent Disk. You can delete either of these here, but the option to delete the detachable disk will be only activated after you've detached the disk by deleting the Environment first.

PD details in the Cloud Environment page

For persistent disk information (name in Google Cloud console, billing ID, workspace ID), click on the "View" link in the "Details" column. 

Persistent-disk-details-in-Cloud-environment-page_Screen_shot.png

To keep from losing work while deleting or modifying your persistent disk, you may want to copy data from your Cloud Environment to another location. For detailed instructions, see Saving data from an interactive environment to your workspace bucket.

A note about auto-syncing behavior

Auto-synching -when Terra frequently auto-save your notebook back to workspace storage - may affect files stored on the VM's persistent disk. When you use a notebook in a Terra workspace, the VM creates subdirectories named after the workspace in the directory where the PD is mounted, and Terra's auto-syncing feature regularly interacts with the notebooks in these subdirectories.

If you're storing anything on the VM's persistent disk that you don't want to be affected by the auto-syncing behavior - for example, notebooks that you would like to keep private - we recommend keeping these in a specifically named subdirectory that is not named after a workspace (such as /home/jupyter/no-sync/).

Was this article helpful?

0 out of 0 found this helpful

Have more questions? Submit a request

Comments

0 comments

Please sign in to leave a comment.