Creating a custom cloud environment runtime with the Swagger API

Anton Kovalsky
  • Updated

Sometimes you may want to create a cloud environment runtime with non-default settings that aren't exposed through Terra. In such cases, use the Swagger UI. Swagger lets you directly access the Leonardo API (the interactive analysis API on Terra responsible for essentials like Jupyter Notebooks and Galaxy). 

Things you can do with Swagger include:
- Delete a Terra billing project
- Adjust Autopause (interactive analysis)
- Work with scripts using the Firecloud Service Selector 

Step-by-step instructions for creating a Cloud Environment with Swagger

 Step 1. Authenticate your credentials on Swagger

Go to the Swagger page here, and click "Authorize" to begin your authorization.2021-08-09_09-54-10.png

Check off all the boxes at the bottom of the pop-up and click "Authorize" again. Now, select the email linked with the billing project that you will use to create the Cloud Environment.

2021-08-09_09-55-00.png

After you click "Authorize" and select your login credentials (you may be asked for a password), you'll see the popup below. Make sure to click "Close", not "Logout" to stay authenticated as you execute the API functions.

2021-08-09_09-55-35.png

 Step 2. Execute API functions

Now, you can use the API functions listed on the page, assuming the email you authenticated with has access to Terra resources. Select the function of interest to expand the details, click "Try it out" to activate the fields, fill in the fields, then click "Execute."  

Aug-11-2021_13-07-01.gif

In Swagger, "googleProject" refers to Terra Billing projects, so to use the functions you need to input a Terra Billing project in the "googleProject" field.

The runtime creation function shown in the clip above allows you to create Cloud Environments via the API. Once you've put in a billing project and given your Cloud Environment runtime a name (remember, only lowercase alphanumeric symbols and dashes allowed), you can click "execute".

Note: If you want this environment to be accessible through Terra, you MUST add a label to the request body, as shown below (don't forget the comma separating the label from the runtime configuration!).

,
"labels": {"saturnAutoCreated": "true"}

Warning This is the exact label you must use for your Cloud Environment to appear in either the Terra or the AnVIL-powered-by-Terra UI.

Aug-11-2021_13-51-34.gif

If you successfully created the runtime, when you scroll down you should see a server response message (code 202) that says the request has been accepted but not yet completed. If you included the label, you will see the new environment listed in the "Cloud Environments" section of your Terra profile. You can see this is the one you just created on Swagger by clicking "Details" to see the runtime name, which should match the one you put into the "name" field in Swagger. The entry will appear immediately upon execution, with the status "Creating". Eventually, the status will change to "Running" (this is why the successful response code says the request is not fully completed - it takes the creation time into account). Now, you can head to a workspace to launch an application.

Screen_Shot_2021-08-11_at_2.00.23_PM.png

Note: If you don't include the label (e.g., if you just execute the default sample request without modifying the request body) you can still get a 202 message - you have created an environment runtime, but you won't have access to this runtime through Terra. This is only useful when someone is testing out different creation settings. In all likelihood, this is not what you want, as you'll have little interaction with this environment while still being charged for it.

 Step 3. Deleting your Cloud Environments

If you've inadvertently created such ghostly environments, you can use the "Delete" API function in the same section to get rid of them. If you don't remember the name of the runtime that is now invisible to you (or if you've created a bunch), you can use the "Get" functions (there are two: one for all the runtimes associated with a billing project, and one for all the runtimes in the same section to list the existing runtimes.

2021-08-11_14-18-25.png

Note: Remember that since you can only have one Cloud Environment active per Billing project, you may run into trouble if you already have another Cloud Environment on the Billing project you used here. You may want to delete any older environment that shows up in the Cloud Environments section of your Terra profile.

GPU-enabled environments in different regions

One common reason for using Swagger to create Cloud Environments is that it can be helpful for those who want to enable GPUs. Swagger lets you set the region where your virtual machine (VM) is created, rather than having it be created by default in the "us-central1-a" zone (see available zones here). This is a workaround if you encounter runtime creation failure with the error ZONE_RESOURCE_POOL_EXHAUSTED; a common issue with GPU enabling, since GPUs are still a rather limited resource.

The code below can be pasted into the same API request body (https://notebooks.firecloud.org/#/runtimes/createRuntime) as shown above (notice that the "label" is already present in this sample code, and the zone is set to the next non-default region).

{
"runtimeConfig": {
"cloudService": "GCE",
"machineType": "n1-standard-4",
"diskSize": 100,
"zone": "us-central1-b",
"gpuConfig": {
"gpuType": "nvidia-tesla-t4",
"numOfGpus": "2"
}
},
"toolDockerImage": "us.gcr.io/broad-dsp-gcr-public/terra-jupyter-python:1.0.0",
"labels": {
"saturnAutoCreated": "true"
}
}

Once you execute an API call using the request body shown above, you can access this newly created Cloud Environment through Terra. So long as the "saturnAutocreated" label is included, you'll have access to this environment in Terra. You can delete this runtime either in the Cloud Environments page in your Terra profile, or by using the deleteRuntime API function, as described in step 3 of the step-by-step instructions above.

Was this article helpful?

0 out of 0 found this helpful

Have more questions? Submit a request

Comments

0 comments

Please sign in to leave a comment.