WDL can't import a public-access file in Google bucket?
Hi! My workflow fails with the following error:
[UserAction] Unexpected exit status 2 while running "/cromwell_root/script": /cromwell_root/script: line 31: syntax error near unexpected token `(' /cromwell_root/script: line 31: `wsi = HESlide("/cromwell_root/sh_misc/CMU-1-Small-Region.svs")
Based on the stderr log, this error seems to be caused by the file access issue:
Anonymous caller does not have storage.objects.get access to the Google Cloud Storage object. Permission 'storage.objects.get' denied on resource (or it may not exist).
However, I set this file publically accessible and can access it through the analyses engine within the Terra workspace (R console attached to the workspace I'm trying to run the above workflow):
rstudio@7d8f3c117a54:~$ gsutil cp gs://sh_misc/CMU-1-Small-Region.svs .
Copying gs://sh_misc/CMU-1-Small-Region.svs...
/ [1 files][ 1.8 MiB/ 1.8 MiB]
Operation completed over 1 objects/1.8 MiB.
So I'm not sure how to solve this error. Could you help me with this? Thanks!
Comments
7 comments
Hi Sehyun,
Thanks for writing in! Have you tried giving access to your Terra Service accounts as outlined in this document? If not, I would suggest following the steps and then trying the workflow again.
If you still run into any issues, could you please share the workspace where you are seeing this issue with Terra Support by clicking the Share button in your workspace? The Share option is in the three-dots menu at the top-right.
Please provide us with
We’ll be happy to take a closer look as soon as we can!
Kind regards,
Josh
Hi Josh,
I tried the suggested access level for my data in the cloud bucket, but it didn't work. I just shared the workspace and here is the additional information:
1. https://app.terra.bio/#workspaces/waldronlab-terra-rstudio/PathML_Preprocessing
2. 7492e1f9-f036-497e-a902-6bbd3646219b
3. 18319d0e-dfa8-442b-9fa8-62eeb90410df
Thanks,
Sehyun
Hi Sehyun,
Thanks for getting back to me. I took a look at your workspace and was able to access the file, so I don't think the issue is with the permissions on the external bucket. Also, in the log file for the submission, it does look like the file is localized into the system.
This leads me to believe that the first error message might be correct: that there is a syntax error with the WDL code. I noticed in the code that the ${ } is used for placeholders in a command block that uses <<< >>>. According to the WDL spec, When a command block uses <<< >>> then the placeholders should used this syntax: ~{ }.
My suggestion would be to change the placeholder syntax. Alternatively, the command block could be changed to use { } instead.
Please give that a try and let me know how your testing goes.
Best,
Josh
Hi Josh,
Thanks for the feedback. I tried your suggestion, but it didn't fix the error. Do you have any other suggestions? Could you run this workflow by any chance? (this workspace has only one small test file)
Thanks,
Sehyun
Hi Sehyun,
Thanks for the reply! I took a look at your workspace and I noticed something else: it appears that the WDL is using python in the command blocks. I believe the default for the command block is shell, so it will need to be changed to python. This can be done for one line by putting python before the command or by using the python <<CODE .. CODE syntax is shown in the example code here for more than one line of python code.
I would give that a try and let me know if you have any questions. Also with this level of code troubleshooting, The OpenWDL Slack and BioInformatics StackExchange are good resources to check out.
Best,
Josh
Hi Josh,
Thanks for checking my WDL script. I just realized that issue as well and made the workflow work with python3 <<CODE .. CODE syntax. Yeah! Also, those resources seem pretty good. I'll try them in the future as well. Thanks for all your help.
Best,
Sehyun
Hi Sehyun,
That's great news! I'm glad you were able to get your code to work!
Please let me know if you have any questions.
Best,
Josh
Please sign in to leave a comment.