PAPI error code 2 failures
I'm getting errors with very little detail for a task I've recently run on much larger files:
Task Funcotator.Funcotate:NA:1 failed. The job was stopped before the command finished. PAPI error code 2. Execution failed: generic::unknown: action 3: waiting for container: starting container: running ["docker" "start" "-ai" "3319a676186064b1463bf519fa5282d0fad536127763fcae14382f9c4bcb8f24"]: exit status 125
The data is protected, so I can't give workspace access. Several failed workflow IDs are:
Logfiles simply stop in the middle of tool execution with no errors.
Hi David et. al.,
Google has confirmed this to be a problem on their end and they are looking to release a fix within the week. I will post all updates to the situation in our Known Issues thread here: https://support.terra.bio/hc/en-us/community/posts/360062450471-PAPI-Error-Code-2
Thanks for writing in. I'll take a look at these workflows as soon as I am able and get back to you!
I'm having exactly the same failure:
"Task UnmappedBamToAlignedBam.SortSampleBam:NA:1 failed. The job was stopped before the command finished. PAPI error code 2. Execution failed: generic::unknown: action 1: waiting for container: starting container: running ["docker" "start" "-ai" "5cf4a943e0048f5cd7089551759e246a8c0413a565eb97dce1c847188b6eeb3f"]: exit status 125"
I'm using my own cromwell-server, so now I think it is related to PAPIv2, as once described here.
I am having the same issue with PAPI error code 2
Thank you for the additional reports. We have reached out to Google to investigate as the number of reports for PAPI error code 2 have greatly spiked over the course of the past month, and especially within this past week. I will keep you updated as we hear more.
This really smells like PAPI misreporting preemption, especially as there are zero errors in the log, rather they seem to simple be cut off mid-execution.
Google has reported that the fix for this PAPI Error Code 2 has finished rolling out. It should no longer occur for new pipelines going forward but those in flight might still encounter it. If you submit a brand new workflow after receipt of this message and still run into the issue, please let us know.
Please sign in to leave a comment.