Summary
We've identified an issue that will manifest in Terra being slow to update permissions. The issue is currently expected to self-resolve within the next 24 hours.
See the Timeline section for the latest troubleshooting and resolution updates and the Impact section to understand how this could impact your use of the system.
Timeline
January 9, 2025 7:30 AM ET - Incident Remediated - The issue has self-resolved
January 8, 2025 2:45 PM ET - Incident rediscovered - Our engineers have become aware that the permissions slowness has returned. We are investigating the issue, but expect this to self-resolve in 12 hours.
January 8, 2025 1:00 AM ET - Incident Remediated - Issue has self-resolved.
January 7, 2025 5:00 PM ET - Incident Declared - Our engineers have declared an official incident for this issue once it was confirmed that permissions propagation may take much longer than expected.
January 7, 2025 3:15 PM ET - Issue Discovered - Our engineers received a notification from Google about Terra Google Groups which triggered an investigation for the impact.
Impact
Terra users may see that it takes longer than usual to successfully:
-
Add users to and remove users from groups
-
Create a new user account
-
Get permission to workspace resources (bucket, notebooks, etc.) to workspaces they visit for the first time
- Get permission to workspace resources (bucket, notebooks, etc.) to newly created workspaces
Some updates may take on the order of hours to complete, while other updates may happen as quickly as expected.
The issue is currently estimated to self-resolve within the next 24 hours.
For more information
Please follow this article to get the most up-to-date information on this incident. If you would like to be notified of all service incidents or upcoming scheduled maintenance, click Follow on this page.