mount my own Google bucket
- 5 comments
Hi! Can I mount my own Google bucket to workspace?
More detail : I'm using RStudio in Terra and my data is stored in my own Google bucket. I'd like to mount that bucket (e.g. using gcsfuse) and access data like a conventional file system. For now, it seems like I can 1) copy my data to the Terra-created/managed persistent disk using gsutil or 2) use data model to access the data and store output in the Terra-created/managed bucket assigned to each workspace (not quite even sure exactly where the outputs will be saved if the analysis is done in RStudio). However, the best option for me will be just mounting my own bucket directly.
Thanks for writing in. I'll take a look at this request and get you a response as soon as I can!
gcsfuse has been tricky for some users in the past, but we can see if it's possible to get it working for your case. What issue are you running into using it presently?
You will need to make sure your Terra account and its service accounts have access to your external resource. We recommend using a group to set up these permissions. See this article for more details: https://support.terra.bio/hc/en-us/articles/360050202631
After your Terra account and its service accounts have sufficient permission to the external resources, you can try using gcsfuse. Let us know if you run into any specific issues.
I've learned from our Interactive Analysis team that we unfortunately don't offer a mechanism for mounting buckets at this time. The recommended approach is to use client libraries (AnVIL package, gsutil, etc) to move data from GCS. An engineer also mentioned that this is what bucket-mounting tools are doing under the hood anyway. I took a look at Google's Cloud Storage FUSE documentation, and this appears to be correct.
Even if you were able to install gcs-fuse on your image, they believe it likely wouldn't work because it requires containers to be run in privileged mode, which isn't currently an option.
At this time it seems like gcs-fuse is unlikely to work in the current Terra environment. That said I'm happy to file a feature request for the team to consider building in support for gcs-fuse in the future.
Thanks for the update. It's good to know. And yes, it'll be great if you can file a feature request on gcsfuse.
This post may be of interest: https://support.terra.bio/hc/en-us/community/posts/4404590323483-Mount-Google-bucket-to-Mac
Please sign in to leave a comment.