Trying to run joint-discovery-gatk4.wdl over 4000+ samples, the workflow errored out with:
2019-07-09 23:21:54,288 ERROR - WorkflowExecutionActor-54cf1196-53fa-48ab-8f96-b042abc85549 [UUID(54cf1196)]: Job BackendJobDescriptorKey_CommandCallNode_JointGenotyping.CollectMetricsSharded:671:1 failed to be created! Error: Root workflow tried creating 50043 jobs, which is more than 50000, the max cumulative jobs allowed per root workflow
I believe that this occurs because the intervals file contains 10,187 intervals and the workflow scatters 5 times over those 10,187 intervals.
I have worked around this problem by commenting out the collection and gathering of metrics (CollectMetricsSharded, GatherMetrics, and the *_metrics_files workflow outputs) such that the scatter over ApplyRecalibration can run. I can then re-run the workflow with call caching enabled, commenting out the ApplyRecalibration, in order to gather the metrics.
But it would be great if this maximum could be increased.
Please sign in to leave a comment.