You can create a Cloud Environment runtime with non-default settings that aren't exposed through Terra. In such cases, use the Swagger UI. Swagger lets you directly access the Leonardo API (the interactive analysis API on Terra responsible for essentials like Jupyter Notebooks and Galaxy).
Other things you can do with Swagger
- Delete a Terra billing project
- Adjust Autopause (interactive analysis)
- Work with scripts using the Firecloud Service Selector
Step-by-step instructions
Step 1. Authenticate your credentials on Swagger
1.1. Go to the Runtimes Swagger page https://notebooks.firecloud.org/#/runtimes, and click Authorize.
1.2. Check off all the boxes at the bottom of the pop-up and click Authorize again.
1.3. In the SSO popup box, select the Google ID (email) linked with the billing project you will use to create the Cloud Environment.
1.4. You'll see the popup below After you click Authorize" and select your login credentials (you may be asked for a password). Make sure to click Make sure to click Close (not Logout) to stay authenticated as you execute the commands below.
Step 2. Execute API functions
Now, you can use the API functions listed on the page, assuming the email you authenticated with has access to Terra resources.
2.1. Select createRuntime (or another function of interest) to expand the details.
2.2. Click Try it out to activate the fields.
2.3. Fill in the fields (details below).
In Swagger, "googleProject" refers to Terra Billing projects, so to use the functions, you need to input a Terra Billing project in the googleProject field.
The runtime creation function in the clip above allows you to create Cloud Environments via the API. Once you've put in a billing project and given your Cloud Environment runtime a name (remember, only lowercase alphanumeric symbols and dashes are allowed), you can click "execute."
Note: If you want this environment to be accessible through Terra, you MUST add a label to the request body, as shown below (don't forget the comma separating the label from the runtime configuration!).
Request body example
{
"runtimeConfig": {
"cloudService": "gce",
"machineType": "n1-standard-4",
"diskSize": 100
},
"labels": {"saturnAutoCreated": "true"}
}
Warning This is the exact label you must use for your Cloud Environment to appear in either the Terra or the AnVIL-powered-by-Terra UI.
2.4. Click the Execute button at the bottom.
What to expect
If you successfully created the runtime, when you scroll down, you should see a server response message (code 202) that says the request has been accepted but has yet to be completed.
If you included the label, you would see the new environment listed in your Terra profile's "Cloud Environments" section. You can see this is the one you just created on Swagger by clicking Details to see the runtime name, which should match the one you put into the "name" field in Swagger.
The entry will appear immediately upon execution, with the status "Creating"
Eventually, the status will change to Running (this is why the successful response code says the request is not fully completed - it takes the creation time into account).
Next steps
Now, you can launch an application (Galaxy, Jupyter, or RStudio) in a workspace.
Beware of ghost environments (if you don't include the label)If you execute the default sample request without modifying the request body, you can still get a 202 message - you have created an environment runtime, but you won't have access to this runtime through Terra.
This is only useful when someone is testing out different creation settings. In all likelihood, this is not what you want, as you'll have little interaction with this environment while still being charged for it. See below for how to delete such an environment.
Step 3. Delete Cloud Environments inaccessible in Terra
If you've inadvertently created such ghostly environments, you can use the "Delete" API function in the same section to eliminate them.
Finding the runtime name
If you don't remember the name of the runtime that is now invisible to you (or if you've created a bunch), you can use the Get functions (there are two: one for all the runtimes associated with a billing project and one for all the runtimes in the same section to list the existing runtimes.
Avoid creating duplicate Cloud Environments
Since you can only have one Cloud Environment active per workspace per user, you may run into trouble if you already have another in the workspace you used here. You can delete any older environments in the Cloud Environments section of your Terra profile.
GPU-enabled environments in different regions
A common reason to use Swagger to create Cloud Environments is to enable GPUs. Swagger lets you set the region where your virtual machine (VM) is created rather than having it be created by default in the "us-central1-a" zone (see available zones here). This is a workaround if you encounter runtime creation failure with the error ZONE_RESOURCE_POOL_EXHAUSTED; a common issue with GPU enabling since GPUs are still a relatively limited resource.
The code below can be pasted into the same API request body (https://notebooks.firecloud.org/#/runtimes/createRuntime) as shown above (notice that the "label" is already present in this sample code, and the zone is set to the next non-default region).
{
"runtimeConfig": {
"cloudService": "GCE",
"machineType": "n1-standard-4",
"diskSize": 100,
"zone": "us-central1-b",
"gpuConfig": {
"gpuType": "nvidia-tesla-t4",
"numOfGpus": "2"
}
},
"toolDockerImage": "us.gcr.io/broad-dsp-gcr-public/terra-jupyter-python:1.0.0",
"labels": {
"saturnAutoCreated": "true"
}
}
What to expect
Once you execute an API call using the request body above, you can access this newly created Cloud Environment through Terra. So long as the "saturnAutocreated" label is included, you'll have access to this environment in Terra (you'll see it in your Cloud Environments page, for example). You can delete this runtime from the Cloud Environments page in your Terra profile or using the deleteRuntime API function, as described in step 3 of the step-by-step instructions above.