The issue was found at 9:32 AM ET on January 29, 2020 and impacts users attempting to stop or delete runtime/notebook environments in the Terra/FireCloud platform:
- Users may not be able to pause their environments; they will appear to be in
- Users may not be able to delete their environments. Status will spin for a long period of time and eventually provide an error similar to
Deletion of instance 'callisto-dev/us-central1-a/saturn-b5b69bf5-7a8a-4f4d-b0c9-7be67f04f72a-m' timed out.
- Runtime environments can be manually deleted in Terra/AoU, but the underlying resource may not be deleted in Google.
- Running runtime environments should still work, until they happen to auto-pause.
See the Timeline section for the latest troubleshooting and resolution updates and the Impact section to understand how this could impact your use of the system.
January 29, 2020 4:06 PM ET - Issue resolution - Terra/FireCloud runtime environments should be fully functional. If you experience further issues pausing or deleting your runtime environment, please let us know at firstname.lastname@example.org.
January 29, 2020 12:32 PM ET - Issue potentially resolved - Our internal team's experimentation suggests that this issue may be resolved. We will provide another update once the behavior has shown to be stable for the duration of the work day, or when Google confirms that the issue is shown as resolved on their end, whichever comes first.
January 29, 2020 9:32 AM ET - Issue discovered - Our internal team observed strange behaviors when attempting to stop and delete runtime environments. We've opened a support case with Google, and it appears that Google may already be aware of the issue.
Users may not be able to stop or delete runtime environments until the issue has been resolved.
Please follow this article to get the most up to date information on this incident. If you would like to be notified of all service incidents or upcoming scheduled maintenance, click Follow on this page.