gsutil cp: AccessDeniedException: 403 and unable to edit Bucket permissisons

Post author
Terrance Shea

In trying a gsutil cp command to get a local copy of some output from Terra Workflow in a Google Bucket I run into the error "AccessDeniedException: 403 tshea@broadinstitute.org does not have storage.buckets.getIamPolicy access to the Google Cloud Storage bucket."  I thought that I could simply edit the Permissions on the Bucket by adding the appropriate service account but I am unable to do so ("You need permissions for this action.  Required permission(s): storage.buckets.setIamPolicy").  I am the owner of the workspace ( see image below ) and I have confirmed that when with gcloud auth list that it is this same tshea @ broadinstitute credentialed account that is active when trying the gsutil cp command.  I am stumped on how I could be the owner of this Workspace and the user that launched the Workflow creating this Google Bucket that I would then not be able to access output via gsutil .  I appreciate any guidance you may have on how I can get permissisons set to download some data from this Workflow output.  Thank you.

Comments

11 comments

  • Comment author
    Samantha (she/her)

    Hi Terrance Shea,

     

    Thanks for writing in. Are you able to navigate to the bucket/file and view it in the Google Cloud Storage Browser? 

    If you are able, can you share the workspace where you are seeing this issue with GROUP_FireCloud-Support@firecloud.org by clicking the Share button in your workspace, and also add my email (svelasqu@broadinstitute.org) to the broad-bacterial authorization domain so I can access the workspace?

     

    Best,
    Samantha

    0
  • Comment author
    Terrance Shea

    Hi-

    Thank you for the quick response.  I have shared the Workspace and added you as a user to the broad-bacterial group.  And yes I am able to navigate the Bucket and view the files in the Bucket, and even able to download using the button on far right of screen shot below: 

     

    Please let me know if there is any other info I can provide to assist.

     

    Thank you.

    Terrance

    0
  • Comment author
    Samantha (she/her)

    Hi Terrance Shea,

     

    Can you let us know the workspace name or share a link to the workspace?

    Also, just to confirm, is the error you are getting when running the gsutil cp command: AccessDeniedException: 403 tshea@broadinstitute.org does not have storage.buckets.getIamPolicy access to the Google Cloud Storage bucket.? Can you share the full command you are running when you receive this error? It seems to be in response to a different command. You shouldn't have access to change any permissions on your workspace's underlying GCS bucket so that error message is expected, but you should already have access to download any of it's contents using gsutil cp.

     

    Best,

    Samantha

    0
  • Comment author
    Terrance Shea

    Hi Samantha-

     

    Thank you for your help.  I am not sure exactly which change I did to get this to now work (as I have been doing some trial and error with setting to different accounts (gcloud config set account ) and changing projects ( gcloud config set project) but I am now able to successfully run gsutil cp (and gsutil rsync).  For example this command now proceeds without error gsutil cp gs://fc-secure-57e2b775-b88c-47a8-ab24-842cc0547a81/8e19cc0a-cfc1-4f6c-9982-36bc597d58cb/workflowAssembly/ac894dea-797b-4cae-ae58-63eaafdfebb8/call-asm_metrics/shard-0/112231_D0.asm_metrics.txt .

     

    Thank you again for looking into this. I think this may now be marked as resolved.

     

     

    0
  • Comment author
    Samantha (she/her)

    Hi Terrance Shea,

     

    Glad to hear you were able to resolve the issue. If you need assistance with anything else, please don't hesitate to reach out.

     

    Best,

    Samantha

    0
  • Comment author
    Binyamin Knisbacher

    Hi Samantha, 

    I'm having a related issue - I own a workspace but for some reason when I use gsutil to give READER access to another user ("gsutil acl ch -u other.user@gmail.com:READER gs://MY_BUCKET")  or when I try "gsutil iam get" on the bucket, I get AccessDenied due to needing OWNER access. I verified that my "gcloud config list" is using the correct account and project as listed in the Terra workspace.  

    The errors are: 

    1) "CommandException: Failed to set acl for ... Please ensure you have OWNER-role access to this resource."
    2) "AccessDeniedException: 403 bknisbac@broadinstitute.org does not have storage.buckets.getIamPolicy access to the Google Cloud Storage bucket."

    This relates to two workspaces I own: 
    https://app.terra.bio/#workspaces/broad-firecloud-wupo1/Wu_CLL_WES 
    https://app.terra.bio/#workspaces/broad-firecloud-wupo1/CLL1085_RNA

    I gave READER access to GROUP_FireCloud-Support@firecloud.org so you could take a look. 

    Thanks,
    Binyamin

    0
  • Comment author
    Samantha (she/her)

    Hi Binyamin Knisbacher,

    Thanks for reaching out. As mentioned in one of the comments above, you shouldn't have access to change any permissions on your workspace's underlying GCS bucket using gsutil so the error message you are seeing is expected. If you want to grant access to the bucket, you'll need to share the workspace with other users via the Terra UI.

    Best,

    Samantha

    0
  • Comment author
    Binyamin Knisbacher

    Hi Samantha, 

    Okay. The thing is that I previously already gave the user READER access and they still can't access it via gsutil. Thus, I'd appreciate if you could further look into this. 

    The user is Neil. I prefer not to post his email here, but you can view his permissions in the workspace.

    Thanks,
    Binyamin

    0
  • Comment author
    Samantha (she/her)

    Hi Binyamin,

    Our Terra Support email has changed so I am not able to access the workspace yet. Our email is Terra-Support@firecloud.org. But to make things easier, you can now easily share the workspace with us by toggling the "Share with support" button to "Yes" in the Share module.

    What is the exact command Neil is running and what error is he seeing? Can he share a screenshot of what he is seeing? If you'd prefer to move this to a private thread, feel free to send an email to us at support@terra.bio.

    Best,

    Samantha

    0
  • Comment author
    Binyamin Knisbacher

    Nice new feature with the toggle button! Done. 

    This is the command (accesses a file in the first bucket I listed):

    gsutil cp gs://fc-secure-325796c7-0e26-4d1f-964a-d3544843109d/2c532d49-a907-4758-bd44-c7a3d225d7f3/picardRealignment_indel/90fc0178-f9aa-4670-b761-6ef248989811/call-index_normal/CLL-MDAC-0008normal.cleaned.bam /Users/nruthen/Downloads/CLL-MDAC-0008normal.cleaned.bam

    Output - 

    AccessDeniedException: 403 neil.email@gmail.com does not have storage.objects.list access to the Google Cloud Storage bucket.

    Neil Ruthen

    0
  • Comment author
    Samantha (she/her)

    Hi Binyamin,

    Is Neil able to access the workspace in Terra? It looks like it's protected by an authorization domain, so I suspect that Neil has not been granted access to that group yet and therefore cannot access the workspace and the associated bucket's contents.

    Best,

    Samantha

    0

Please sign in to leave a comment.