Need Help?

Search our documentation and community forum

Terra is a cloud-native platform for biomedical researchers to access data, run analysis tools, and collaborate.
Terra powers important scientific projects like FireCloud, AnVIL, and BioData Catalyst. Learn more.

Find out about service incidents and scheduled maintenance

  • Scheduled Maintenance - November 12, 2020 12:00PM - 2:00PM ET (NCI CRDC Framework Services) plus-icon minus-icon

    Summary Terra team will be releasing external authentication enhancements which may impact workflows and data access at the time of release on Thursday, November 12, 2020. Start: 12:00 PM ET End: 2:00 PM ET Users Impacted Users who utilize NCI CRD...
    See more

  • Google Service Incident - October 27, 2020 (Google Cloud Storage) plus-icon minus-icon

    Summary Google has reported issues for its Google Cloud Storage service, which may impact functionality of the Terra platform. Refer to Google's Status Dashboard for the latest on this issue directly from Google. See this article for the latest on...
    See more

  • Service Incident - October 26, 2020 (Cromwell) plus-icon minus-icon

    Summary We've discovered issues related to the aborting, submitting, and completion of workflows in Terra.  There may be delays updating the status of aborted workflows. Newly submitted workflows may take longer than normal to launch. These submi...
    See more

  • Service Incident - October 15, 2020 plus-icon minus-icon

    Summary Engineers discovered a Bond outage Thursday (10/15) evening causing intermittent problems with external account linking. Timeline October 15, 2020 11:20 PM - Issue remediated - Issue resolved itself October 15, 2020 6:40 PM - Issue discove...
    See more

  • Service Incident - October 6, 2020 plus-icon minus-icon

    Summary Engineers have identified an outage that is causing Terra to produce "Failed to fetch" messages for link status in the Terra profile. See the Timeline section for the latest troubleshooting and resolution updates and the Impact section to...
    See more

  • Service Incident - September 24, 2020 plus-icon minus-icon

    Summary Cromwell engineers have identified an issue with one of the workflow execution runner instances which caused potential slowness for some user workflows. See the Timeline section for the latest troubleshooting and resolution updates and the...
    See more

  • Service Incident - September 23, 2020 plus-icon minus-icon

    Summary The workflow execution cluster slowed down over the course of a couple hours and eventually stopped processing workflows altogether for around half an hour Wednesday night (9/23). The cluster was restarted and most workflows have continued...
    See more

  • Service Incident - September 8, 2020 plus-icon minus-icon

    Summary We've discovered an issue where the terminal in the Terra UI is currently not loading for users.  See the Timeline section for the latest troubleshooting and resolution updates and the Impact section to understand how this could impact you...
    See more

  • Scheduled Maintenance - August 27, 2020 8:00PM - 9:00PM EDT plus-icon minus-icon

    Summary Terra team will be releasing security enhancements which will have some impact for workflows and Methods Repository users at the time of release on Thursday, August 27, 2020. Start: 8:00 PM EDT End: 9:00 PM EDT Users Impacted Workflows use...
    See more

  • Service Incident - August 12, 2020 plus-icon minus-icon

    Summary We've identified an issue that is causing runtimes to be stuck in Updating status when updating Dataproc runtimes. See the Timeline section for the latest troubleshooting and resolution updates and the Impact section to understand how this...
    See more

  • Scheduled Maintenance - July 27, 2020 plus-icon minus-icon

    Summary Terra team will be releasing a hotfix for a Notebooks issue which will have some impact for Notebooks users at the time of release on Monday, July 27, 2020. Start: 9:00 PM EDT End: 9:30 PM EDT Users Impacted Notebook users Expected Behavio...
    See more

  • Service Incident - July 17, 2020 (Workflows Issue) plus-icon minus-icon

    Summary We've identified an issue that is disallowing users from running new workflows on Terra. See the Timeline section for the latest troubleshooting and resolution updates and the Impact section to understand how this could impact your use of ...
    See more

  • Service Incident - July 7, 2020 plus-icon minus-icon

    Summary The issue was found at 12:46 PM on July 7 and impacts some users launching workflows. See the Timeline section for the latest troubleshooting and resolution updates and the Impact section to understand how this could impact your use of the...
    See more

  • Service Notification - July 2, 2020 (Free Credits Projects) plus-icon minus-icon

    Summary Due to activity that is against the Terra Terms of Service, we have suspended the free credits program as of July 2, 2020. The Terra team is looking into the matter, and this article will be updated with any developments. Please follow thi...
    See more

  • Service Notification - June 29, 2020 (Free Credits Projects) plus-icon minus-icon

    Summary The service to sign up for free credits is currently unavailable. Service is scheduled to be restored tomorrow, June 30. Users who try to create a new free credits project will see an error similar to the following: Trial has no available...
    See more

  • Service Incident - June 29, 2020 plus-icon minus-icon

    Summary   We are seeing issues associated with linking to Identity & External Servers across a number of Terra domains. This issue is due to certain URLs not being whitelisted on Gen3, and missing aliases on our whitelist. Known issues: Error (re...
    See more

  • Service Incident - June 16, 2020 plus-icon minus-icon

    Summary An external configuration change has caused the import of cohort data feature in the Data Explorer to fail. Trying to export a cohort will result in an Import Error. See the Timeline section for the latest troubleshooting and resolution u...
    See more

  • Service Incident - June 11, 2020 plus-icon minus-icon

    Summary Our external ID linking service is currently returning errors on request with Bearer Tokens. This is impactful for users who link to DCF and DCP Frameworks. A Terra UI bug is resulting in existing or newly linked DCP or DCF Framework links...
    See more

  • Service Incident - June 9, 2020 plus-icon minus-icon

    Summary A configuration error led to Cromwell being down for approximately 25 minutes. The issue was swiftly resolved and Cromwell is operating as it normally would.   See the Timeline section for the latest troubleshooting and resolution updates ...
    See more

  • Service Incident - June 8, 2020 plus-icon minus-icon

    Summary On the evening of June 5, a member of our notebooks team identified a bug related to dataproc multi-node spark clusters. When users who have created a multi-node spark cluster stop their runtime in the Terra UI, it's possible that only som...
    See more

  • Scheduled Maintenance - June 4, 2020 9:00-10:00 PM ET plus-icon minus-icon

    Summary A second series of important updates are scheduled for Thursday, June 4. Changes include updates to the SSL certificate and wildcard certificates. These updates will be highly impactful for users of the platform during the maintenance wind...
    See more

  • Scheduled Maintenance - May 28, 2020 9:00-10:00 PM ET plus-icon minus-icon

    Summary A series of important updates are scheduled for Thursday, May 28. Changes include updates to the load balancer, openDJ, and SSL certificate. These updates will be highly impactful for users of the platform during the maintenance window. Th...
    See more

  • Service Incident - May 6, 2020 plus-icon minus-icon

    Summary On the morning of May 6, 2020, we reported massive spikes of jobs receiving PAPI error code 2 to Google. They have concluded this to be an error on their end, and are working to put out a fix within the week. See the Timeline section for t...
    See more

  • Service Incident - April 30, 2020 plus-icon minus-icon

    Summary On the morning of April 30, 2020, we are seeing job submissions and results retrieval queued for much longer than expected. We are currently investigating.  See the Timeline section for the latest troubleshooting and resolution updates and...
    See more

  • Service Incident - April 15, 2020 plus-icon minus-icon

    Summary On the morning of April 15, 2020, we are seeing two of our Cromwell instances are down. This will affect users' ability to view workflows. See the Timeline section for the latest troubleshooting and resolution updates and the Impact sectio...
    See more

  • eRA Commons Scheduled Maintenance - April 17-20, 2020 plus-icon minus-icon

    Summary The NIH eRA Commons system will be moving to the cloud during a maintenance window taking place April 17-20, 2020. Users will be unable to link or relink their eRA Commons connection during this window. We recommend all users relink their ...
    See more

  • Service Incident - April 8, 2020 plus-icon minus-icon

    Summary On the morning of April 8, 2020, we are seeing all Terra services down due to Google cloud infrastructure issues. See the Timeline section for the latest troubleshooting and resolution updates and the Impact section to understand how this ...
    See more

  • Service Incident - March 27, 2020 plus-icon minus-icon

    Summary On the morning of March 27, 2020, we noticed a number of Google Cloud service outages.  See the Timeline section for the latest troubleshooting and resolution updates and the Impact section to understand how this could impact your use of t...
    See more

  • Scheduled Maintenance - March 30, 2020 5:00-5:30 PM ET plus-icon minus-icon

    Summary An important Dockstore release is scheduled for Monday March 30. This will be highly impactful for users who submit workflows using docker images hosted in Dockstore. Monday March 30, 2020 Start: 5:00 PM ET End: 5:30 PM ET Users Impacted A...
    See more

  • Service Incident - March 10, 2020 plus-icon minus-icon

    Summary The issue was found at approximately 10:45 AM ET on March 9, 2020 and impacts users trying to submit jobs through Cromwell. Users will see jobs queued for a longer-than-expected period of time, with the Queued or QueuedInCromwell status. O...
    See more

Powered by Zendesk