I am running the Whole-Genome-Analysis-Pipeline, and the workflow keeps failing at the JointGenotyping.ImportGVCFs step (PAPI error code 10 or 9). I've ran this workflow before on 531 samples, and after I added 211 more (in the same bucket), I can't get joint genotyping to work successfully (note: all these gvcfs were preprocessed using GATK locally/outside of terra and I just uploaded the reblocked gvcfs to the cloud with intent of joint genotyping). Using the same workspace, I increased the disk space multiple times, but it has not resolved the problem. Unfortunately the log files aren't too informative (either just PAPI code 10 or code 9 with a truncated log output for ImportGVCFs). I also saw that there was a new version of the workspace released recently, so I cloned that workspace and then restarted the job, but still the problem reoccurs. Interestingly, the JointGenotyping.ImportGVCFs step seems to fail in shard 200 everytime.
Here are the relevant Submission IDs/Workflow IDs on workspace https://app.terra.bio/#workspaces/mgb-UO1_Ozelius/Whole-Genome-Analysis-Pipeline%20copy
b4849176-86c3-4a2a-bf30-19f80fbd0c7a Workflow ID90580e00-92f2-4537-85fc-b0032b2a2594
71081897-76de-4355-81c7-bd0674fc2a2b Workflow ID ef5779cf-8a71-4516-a8a9-ae4f4ab0a465
d512969d-a33c-457a-953c-f660c6e58d47 Workflow ID 52affaf4-4ba5-4ee6-aaf5-74173c0d5431
New workspace: https://app.terra.bio/#workspaces/mgb-UO1_Ozelius/Whole-Genome-Analysis-Pipeline%20copy%205-6-2023
Submission ID 8b17ac66-6ed9-44bb-acaf-0d9dcef7140e Workflow ID 69c94265-64fe-46e9-8057-462db612d32f
Thank you. Please let me know if you have insights on why this workflow keeps failing.