pet account permissions for shared workspace
I am running into an error regarding pet account permissions. I have a workspace that I shared with another user and gave them owner permissions. When they run our workflow we get the following error running GATK:
code: 403
message: pet-109359867313582786694@covid-wgs-analysis.iam.gserviceaccount.com does not have serviceusage.services.use access to the Google Cloud project.
reason: forbidden
location: null
retryable: false
com.google.cloud.storage.StorageException: pet-109359867313582786694@covid-wgs-analysis.iam.gserviceaccount.com does not have serviceusage.services.use access to the Google Cloud project
.
at com.google.cloud.storage.spi.v1.HttpStorageRpc.translate(HttpStorageRpc.java:229)
at com.google.cloud.storage.spi.v1.HttpStorageRpc.list(HttpStorageRpc.java:370)
at com.google.cloud.storage.StorageImpl$8.call(StorageImpl.java:374)
at com.google.cloud.storage.StorageImpl$8.call(StorageImpl.java:371)
at shaded.cloud_nio.com.google.api.gax.retrying.DirectRetryingExecutor.submit(DirectRetryingExecutor.java:105)
at shaded.cloud_nio.com.google.cloud.RetryHelper.run(RetryHelper.java:76)
at shaded.cloud_nio.com.google.cloud.RetryHelper.runWithRetries(RetryHelper.java:50)
at com.google.cloud.storage.StorageImpl.listBlobs(StorageImpl.java:370)
at com.google.cloud.storage.StorageImpl.list(StorageImpl.java:326)
at com.google.cloud.storage.contrib.nio.CloudStoragePath.seemsLikeADirectoryAndUsePseudoDirectories(CloudStoragePath.java:107)
at com.google.cloud.storage.contrib.nio.CloudStorageFileSystemProvider.readAttributes(CloudStorageFileSystemProvider.java:810)
at java.nio.file.Files.readAttributes(Files.java:1737)
at java.nio.file.Files.isRegularFile(Files.java:2229)
at htsjdk.samtools.SamFiles.lookForIndex(SamFiles.java:73)
...
However, when I launch the workflow it runs successfully. The BAM file that GATK is trying to access and causes the error is an output from a previous task and therefore located in the workspace bucket. The workspace is named gatk-sv-pipeline-covid-2.
Comments
8 comments
Hi Mark,
Thanks for writing in. Can you share the workspace where you are seeing this issue with GROUP_FireCloud-Support@firecloud.org by clicking the Share button in your workspace? The Share option is in the three-dots menu at the top-right.
Let me know the relevant submission ID and I'll be happy to take a closer look!
Kind regards,
Jason
Hi Jason,
I have just shared the workspace with GROUP_FireCloud-Support@firecloud.org.
The submission id that failed due to pet account permissions is a80f9aac-c1b7-4650-9a63-1be184a1eedc.
Thank you,
Alba
Hi Alba,
Thank you for that. Can either of you confirm whether asanchis@broadinstitute.org is a member of the covid-wgs-analysis billing project? I see the error is associated with the project rather than the bucket, which would manifest in an error like
AccessDeniedException: 403 pet-109359867313582786694@covid-wgs-analysis.iam.gserviceaccount.com does not have storage.objects.list access to the Google Cloud Storage bucket.
If asanchis@broadinstitute.org is not a member of the covid-wgs-analysis project, can you try adding asanchis@broadinstitute.org to the project and re-running the failed workflow? I recommend running with call caching enabled to save on time and costs.
If you have any questions, please let us know.
Kind regards,
Jason
Hey Jason
If the user asanchis@broadinstitute.org had read/write access on a Terra Workspace, why wouldn't that work? Users in this role have the ability to run WDLs, etc.
Why does it need to be at the project level?
Thanks
Hey Sam Bryant,
You're correct that it should work since asanchis@broadinstitute.org is an Owner on the workspace! But I'd like to see if something in that permission setting isn't working as expected. There was a previously known issue where Owners were not being given the right permissions to use the workspace billing project even though Writers did. If you don't want to add asanchis@broadinstitute.org to the BP we can do a test having asanchis@broadinstitute.org set as a Writer instead of an Owner.
Let me know if either option works for you for a test.
Kind regards,
Jason
Hey all,
Just wanted to check in here to see how things were going, and if there's anything else we can do to assist.
Kind regards,
Jason
Old thread, but just following up here. The issue was, as Jason suggested, that the account was not added to the billing account. This makes perfect sense, although it would be nice if Terra could automatically check whether this is case and, if so, give a clear error when trying to launch the workflow.
Hi Mark Walker,
Thanks for the suggestion. I've sent this request to our development team for consideration.
Best,
Samantha
Please sign in to leave a comment.