Summary
We've identified an issue that is impacting workflow submission in Terra on GCP. Our engineers are currently investigating and working on a fix.
Timeline
October 16, 2023 3:22 PM ET - Issue Remediation - Our engineers restarted the Cromwell Runners and are starting to see workflows getting picked up and running again.
October 16, 2023 2:50 PM ET - Issue Escalated - After some initial investigation, our engineers declared this as service incident given the impact. They are currently working on resolving the issue.
October 16, 2023 12:45 PM ET - Issue Discovered - Our engineers noticed that Terra stopped submitting workflows to GCP at this time.
Impact
Users submitting workflows on Terra (GCP) may find that their jobs are stuck in "Submitting" status.
For more information
Please follow this article to get the most up to date information on this incident. If you would like to be notified of all service incidents or upcoming scheduled maintenance, click Follow on this page.