Workflow submissions not finishing
I have this workflow https://portal.firecloud.org/?return=firecloud#methods/neoantigen_prediction/run_prediction/27 in vanallen-firecloud-dfci/CR_NV_WES_IO_Meta and it seem like some of my submissions stop processing but when I abort the workflow and then try again they run. Can you please let me know why some of these workflows seem to just hang?
Here is a current submission: https://firecloud.terra.bio/#workspaces/vanallen-firecloud-dfci/CR_NV_WES_IO_Meta/job_history/c298a9fd-c7ec-47f4-a347-6e7c883d9f5a
and here is a prior submission https://firecloud.terra.bio/#workspaces/vanallen-firecloud-dfci/CR_NV_WES_IO_Meta/job_history/b9e3adce-f4aa-44fa-8ba0-b790b3460f0e
Note, for example, how for data entity P290_T_N, it didn't finish running after ~6 days in the prior submission but finished within a day in the current submission.
Please let me know if theres anything I can do to resolve this.
We'll have a look on our end and get back to you.
I asked the team and here is their suggestion:
According to the log files it looks like the neoantigenPredictionType1 task froze in the middle of the script execution, somewhere at the moment where it should have completed
Making binding predictionsand switch to
Parsing binding predictions. It doesn’t seem like the problem is on Terra side. Try ensuring the script or tool being used runs to completion, for instance check if there are any hidden infinite loops, waiting on conditions which never happen or anything similar.
Please sign in to leave a comment.