Creating a custom cloud environment runtime with the Swagger API

Anton Kovalsky
  • Updated

To create a cloud environment runtime with non-default settings that aren't exposed through the Terra UI. In such cases, use the Swagger UI. Swagger lets you directly access the Leonardo API (Terra's interactive analysis API responsible for things like Jupyter Notebooks and Galaxy). 

Things you can do with Swagger include:
- Delete a Terra billing project
- Adjust Autopause (interactive analysis)
- Work with scripts using the Firecloud Service Selector 

Step-by-step instructions for creating a cloud environment with Swagger:

1. Authenticate your credentials on Swagger

Go to the Swagger UI page here, and begin by completing your authorization. Click "Authorize"

2021-08-09_09-54-10.png

check off all of the boxes at the bottom of the pop-up and click "Authorize" again, and then select the email that is associated with the billing project that you'll be using to create the cloud environment

2021-08-09_09-55-00.png

After you've clicked "Authorize" and selected your login credentials (you may be asked for a password), you'll see the pop up below. Make sure to click "close" and not "log off" to stay authenticated as you execute the API functions.

2021-08-09_09-55-35.png

 

2. Execute API functions

You can now use the API functions listed on the page, assuming the email you authenticated with has access to Terra resources. You do this by selecting the function of interest to de-collapse the details, click "Try it out" in order to activate the fields, fill in the fields, then click "Execute."  

Aug-11-2021_13-07-01.gif

In the Swagger UI, "googleProject" refers to Terra Billing Projects, so to use any of the functions you'll need to input a Terra Billing Project to which you have access in the "googleProject" field.

The runtime creation function shown in the clip above allows you to create cloud environments via the API. Once you've put in a billing project and given your cloud environment runtime a name (just remember, only lowercase alphanumeric symbols and dashes allowed), you can click "execute".

Note that if you want this environment to be accessible through the Terra UI, you MUST add a label to the request body, as shown below (don't forget the comma separating the label from the runtime configuration!):

,
"labels": {"saturnAutoCreated": "true"}

Note: This is the exact label you must use in order for your cloud environment to appear in either the Terra or the AnVIL-powered-by-Terra UI.

Aug-11-2021_13-51-34.gif

If you successfully create the runtime, when you scroll down you should see a server response message (code 202) that says the request has been accepted but not yet completed. If you've included the label, you will immediately be able to see the new environment listed in the "Cloud Environments" section of your Terra profile. You can see this is the one you just created on swagger by clicking "Details" to see the runtime name, which should match the one you put into the "name" field in the swagger UI. The entry will appear immediately upon execution, with the status "Creating"; eventually the status will change to "Running" (this is why the successful response code says the request is not fully completed - it takes the creation time into account), and then you can head to a workspace to launch an application.

Screen_Shot_2021-08-11_at_2.00.23_PM.png

Note: If you don't include the label (e.g. if you just execute the default sample request without modifying the request body) you can still get a 202 message - you will still have created an environment runtime, but you won't have access to this runtime through the Terra UI. This is generally only useful when someone is testing out different creation settings, so in all likelihood this is not what you want, since you'll have little interaction with this environment while still being charged for it.

3. Deleting your cloud environments

If you've inadvertently created such ghostly environments, you can use the "Delete" API function in the same section to get rid of them, and if you don't remember the name of the runtime that is now invisible to you (or if you've created bunch), you can use the "Get" functions (there are two: one for all the runtimes associated with a billing project, and one for all the runtimes associated with  in the same section to list the existing runtimes:

2021-08-11_14-18-25.png

As a final note, remember that since you can only have one cloud environment active per billing project, you may run into trouble if you already have another cloud environment on the billing project you used here, so you might want to delete any older environment you that shows up in the Cloud Environments section of your Terra profile.

GPU-enabled environments in different regions

One common reason for using swagger to create cloud environments is that it can be helpful for those who want to enable GPUs. Swagger allows you to set the region where your virtual machine is created, rather than just having it be created by default in the "us-central1-a" zone (see available zones here). This is a workaround if you encounter runtime creation failure with the error ZONE_RESOURCE_POOL_EXHAUSTED, which a common issue with GPU enabling, since GPUs are still a rather limited resource.

The code below can be pasted into the same API request body (https://notebooks.firecloud.org/#/runtimes/createRuntime) as shown above (notice that the "label" is already present in this sample code, and the zone is set to the next non-default region).

{
"runtimeConfig": {
"cloudService": "GCE",
"machineType": "n1-standard-4",
"diskSize": 100,
"zone": "us-central1-b",
"gpuConfig": {
"gpuType": "nvidia-tesla-t4",
"numOfGpus": "2"
}
},
"toolDockerImage": "us.gcr.io/broad-dsp-gcr-public/terra-jupyter-python:1.0.0",
"labels": {
"saturnAutoCreated": "true"
}
}

Once you've executed an API call using the request body shown above, you'll able to access this newly created cloud environment through the Terra UI. So long as the "saturnAutocreated" label is included, you'll have access to this environment in Terra, and you can delete this runtime either in the Cloud Environments page in your Terra profile, or by using the deleteRuntime API function, as described in step 3 of the step-by-step instructions above.

Was this article helpful?

0 out of 0 found this helpful

Have more questions? Submit a request

Comments

0 comments

Please sign in to leave a comment.