R studio initialization error
Hi folks,
I've tried setting up an R studio environment on Terra, but it is giving me issues initializing it. The environment appears to start/resume, but when I try and open it using the "rocket" icon, I get an error as can be seen in the second screenshot below.
The error is "error setting up analysis file syncing" and when clicking on the details, the following appears:
{
"errorMessage": "DecodingFailure(Missing required field, List(DownField(pattern)))",
"errorCode": null,
"traceId": null
}
After some time attempting to load, the initialization process throws the following error, as can be seen in the first screenshot attached.
I have set up the R studio environment in the following workspace.
Please let me know if any other information would be useful to troubleshoot this issue. Thank you!
Comments
19 comments
Hi Kevin,
Thanks for reaching out with this issue! Can you please your workspace with Terra Support by clicking the Share button in your workspace? The Share option is in the three-dots menu at the top-right.
Could you also provide the following details so we can take a closer look at your issue?
Best,
Emily
Hi Emily,
Thank you for your prompt reply! I shared the workspace as suggested. Here are the additional details you requested:
1) Terra account email address: kmeli@broadinstitute.org
2) Google project ID: vanallen-firecloud-nih
3) Cluster ID: saturn-9ec0f7cf-234e-4bed-8817-d33b8eebcb18
4) Cloud environment configuration screenshot: see attached.
5) Time issue first occurred: Approximately 9/29.
Thank you for your help!
Kevin
Hi Kevin,
Thanks for providing that information! Our engineers are currently investigating this error. I will update you as we learn more!
Best,
Emily
Hi Kevin,
The "error setting up analysis file syncing" issue has been resolved. Could you try launching Rstudio again and let me know if you continue seeing issues?
Best,
Emily
Hi Emily,
Thank you for getting back to me. Unfortunately the issues still persists the same way. This time, however, when I open the workspace, I first noticed it was locked, and then I also get the following error, which I did not get before:
Unable to get bucket location:
{
"error": {
"code": 403,
"message": "pet-106573984799776840713@vanallen-firecloud-nih.iam.gserviceaccount.com does not have storage.buckets.get access to the Google Cloud Storage bucket.",
"errors": [
{
"message": "pet-106573984799776840713@vanallen-firecloud-nih.iam.gserviceaccount.com does not have storage.buckets.get access to the Google Cloud Storage bucket.",
"domain": "global",
"reason": "forbidden"
}
]
}
}
The workspace storage cost does not get displayed too (with an "Error loading storage cost data" pop up on the dashboard screen). When I click on the underlying google bucket to view the contents of my workspace, I see nothing but "Additional permissions required to list objects in this bucket. Ask a bucket owner to grant you 'storage.objects.list' permission".
Back to the R session, apart from the same issue repeating, now I also get the following error:
Error loading outdated analyses:
{
"error": {
"code": 403,
"message": "pet-106573984799776840713@vanallen-firecloud-nih.iam.gserviceaccount.com does not have storage.objects.list access to the Google Cloud Storage bucket.",
"errors": [
{
"message": "pet-106573984799776840713@vanallen-firecloud-nih.iam.gserviceaccount.com does not have storage.objects.list access to the Google Cloud Storage bucket.",
"domain": "global",
"reason": "forbidden"
}
]
}
}
To clarify, I do not see "error setting up analysis file syncing" error banner, but the end result is the same in that the Rstudio environment does not open up and eventually fails to open. I do see the above errors now, though.
Please let me know if you need any other information from me to address the issue. Thank you!
Hi Kevin,
I believe your workspace is currently being migrated (as part of the project-per-workspace model), which is why you are seeing those permission errors. This should wrap up at the end of the day and these errors should not persist tomorrow.
To clarify, you are still seeing the "Rstudio initialization error" message, correct?
Best,
Emily
I do not see the "error setting up analysis file syncing" error pop up, but the end result was the same in that Rstudio took a long time to load and then displayed the "Rstudio initialization error" again.
As a followup to the workspace migration note: I ended up unlocking my workspace thinking it was a one-off issue. Would that interfere with the migration?
Hi Kevin,
It looks like your migration was completed successfully! I'll continue to investigate the Rstudio initialization error and update you as soon as I am able.
Best,
Emily
Hi Emily,
Great, thank you. Is it weird that I now don't see my Rstudio environment anymore in the workspace? As in, there is no existing Rstudio env anymore in the workspace. I'm not sure if this is because of fixes on your end but wanted to bring it up if it's relevant.
Hi Kevin,
You won't be able to access your old cloud environments post-migration and must create a new environment in your workspace. Per this email, "during the migration process, any persistent disks still attached to an older workspace will become unattached. You will need to copy the unattached persistent disk to a newly migrated workspace and recreate your cloud environment to start using them again." Once you have copied your persistent disk, you may delete the old environment and disk on the environments page.
I was able to launch Rstudio in a copy of your workspace using a new cloud environment I spun up. Could you try creating a new cloud environment and see if that resolves the issue? I'll continue looking into the error in the meantime!
Best,
Emily
Hi Emily,
Unfortunately the issue persists. I was first able to create an Rstudio environment and launch it. Then I tried to create an Rproject within it with data from my copied persistent disk, and Rstudio started being glitchy. I tried to maybe pause and relaunch Rstudio, but this time I start getting the original error with Rstudio taking too long to initalize.
I noticed another peculiar feature that may be of interest to your team. I have another Rstudio environment in a workspace that was recently created according to the project-per-workspace model. That environment had been working fine for me for the time being. I tried to launch Rstudio there, as well as in the workspace we are talking about, at the same time. When Rstudio opened on both workspaces at the same time, both environments started refreshing constantly. I had to pause one environment or the other for the refreshing to stop.
Please let me know if I can try other things on my end or if you can look more into this issue. It's curious that the Rstudio environment I created in this workspace (which abided by the project-per-workspace model) has not given me much trouble, but the one I had created in the workspace in question, which was migrated yesterday, keeps suffering from the R initialization error.
Thank you,
Kevin
Hi Kevin,
For the new issue you experienced with being unable to work in Rstudio in 2 different workspaces at the same time, we have reported this bug to our engineers. For more information and a possible workaround see this known issue post.
Would you be able to share the name of your new cloud environment and your PD (visit https://app.terra.bio/#clusters, click Details, and see Name field)? Also, do you have an estimated time when you saw the error with the new cloud environment? I'm going to check the logs to see if anything there points to the root cause.
Best,
Emily
Hi Emily,
I saw the error with the new cloud environment in the am today (10/6).
Cloud environment name: saturn-2dd0c52e-bf23-414b-96a3-536d8e029ef4
PD name: saturn-pd-2bd0d1b6-789f-4701-b164-247cf4618ae1
Thank you for your help!
Kevin
Hi Kevin,
Thanks for the information! Please allow me to answer for Emily. Because you're able to create RStudio environments in other workspaces, this leads me to believe that part of the issue might have to do with the Persistent Disk itself. I have some follow up questions that might help us find a solution that works for you:
Please let me know if you have any questions.
Best,
Josh
Hi Josh,
Thank you for your reply! As requested, I am attaching a screenshot of my cloud environment page and the configuration of virtual env/persistent disk.
I have not tried to run the PD with a smaller size. I think I can also delete the PD and create a new one (need to double check this). In this case, do you think I should create one with a different disk size (smaller if anything)?
Thanks,
Kevin
Hi Kevin,
Thanks for providing the screenshots! This confirms the size we are also seeing for the Persistent Disk. If it's possible, I'd delete and re-create a new Persistent Disk. (Please let us know if you'd like help getting any data out of the Persistent Disk before you delete it.)
Please let me know how recreating the environment goes.
Let me know if you have any questions.
Best,
Josh
Hi folks,
Thank you for your help throughout! I did delete the disk and made a new environment/PD. I think I have noticed the real problem stems when I try to use an .Rdata file that I already have.
Essentially, after I created the new environment, Rstudio launches just fine. I created an Rproject and linked it to my .Rdata file so that it loads the workspace using that file every time the project opens. However, Rstudio does not successfully launch subsequent times, taking a long time then forcing me to launch it in "safe mode". I am trying to get around this by not having the project read the .Rdata file automatically, but load it every time I need it. It would of course be more convenient if it was successfully read in automatically. The .Rdata file in this case is ~30Gb, so I suspect the size may be part of why Rstudio has a hard time launching when it has to read it in. However in theory it should still be able to, even if it takes some time.
If your team has encountered similar issues when trying to use large .Rdata files, that would be good to know.
Thank you!
Kevin
Hi Kevin,
Sorry for the delayed response. I'm glad you were able to determine part of the cause of this issue. I believe that a file that large could cause some issues for the system. I have two suggestions that might be able to help you for now: You could try increasing the resources of the Cloud Environment that you use. (There will be a higher cost associated with that, however.)
Also, would it be possible to break the .Rdata file up into smaller files? That might be easier for the system to process, and could help us to confirm that the file size is the root cause of the issues.
Please let me know if that information was helpful or if you need anything else.
Best,
Josh
Please sign in to leave a comment.