Need Help?

Search our documentation and community forum

Terra is a cloud-native platform for biomedical researchers to access data, run analysis tools, and collaborate.
Terra powers important scientific projects like FireCloud, AnVIL, and BioData Catalyst. Learn more.

Terra workspace deleted itself?!

Comments

5 comments

  • Avatar
    Jason Cerrato

    Hi Aisling,

    That seems very unexpected, though I'm not so familiar with how FISS works (it isn't built by or maintained by the Terra team). I will flag this up with our engineers for investigation and get back to you as soon as possible.

    Kind regards,

    Jason

    0
    Comment actions Permalink
  • Avatar
    Jason Cerrato

    Hi Aisling,

    One of our engineers was able to identify the moment the workspace was deleted.

    Mar 26 18:04:14 gce-rawls-prod803 rawls-proxy[1604]: 35.191.3.232 - 117475951917658785644 "SNIP" "Saturn" [2020-03-26T18:04:12.677-0400] 2161114 + "DELETE /api/workspaces/fccredits-xenon-denim-7890/TOPMed%20Alignment%20and%20Freeze8%20Variant%20Calling HTTP/1.1" 202 86 "https://app.terra.bio/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.149 Safari/537.36"

    The log indicates that the workspace was deleted using the Terra UI at Mar 26 18:04:14 Eastern time. The log specifically shows it was deleted through the UI, rather than through FISS or API call.

    Kind regards,

    Jason

    0
    Comment actions Permalink
  • Avatar
    Aisling O'Farrell

    At the risk of sounding like that one patient on Doctor House who insisted she knows how her inhaler works and then proceeded to spray it on her neck, it's doesn't seem possible for that to be the case. I have deleted workspaces via the UI before; it's very clear when you're deleting a workspace. It's not something that one can accidentally do. Is it possible to see if someone else deleted it by mistake? The workspace was shared with multiple people all at different IP addresses.

    0
    Comment actions Permalink
  • Avatar
    Jason Cerrato

    Hi Aisling,

    I deeply sympathize with what you must be feeling, hearing this report—it's certainly unbelievable that one would be capable of deleting a workspace in the UI by accident. I can confirm that the email address tied to the action is the same one you are using to contact us here.

    One of our engineers investigated whether there was a way to produce the very same error log using the command line, and found that you can create an identical log if you copied the network request from the UI (which copies all of the headers) from another deletion, and changed the workspace name. Using this, it's feasible that the workspace was deleted using the command line if you had copy-pasted any previously used commands that were used to delete a workspace. Based on your initial report, this doesn't sound likely, but it's just about the only way this could show up in the logs looking like this, where it shows the deletion having come from https://app.terra.bio on a browser without actually having been done in a browser directly. 

    The only other way this could have happened was if you were somehow logged in (on the UI or in gcloud) somewhere else and someone else deleted it with your account.

    If you have any idea at all of how this may have happened, please let us know.

    Jason

    0
    Comment actions Permalink
  • Avatar
    Aisling O'Farrell

    I managed to grab the Jupyter notebook I was working on when the workspace was deleted (it was still able to execute commands even as the workspace itself was deleted, strangely enough) plus another one from about two hours prior. Would that be of any use to your engineers? If so, where should I forward that? I'm on the BioData Catalyst slack; that might be easiest if your engineers are on there.

    0
    Comment actions Permalink

Please sign in to leave a comment.

Powered by Zendesk