"Invalid" WDL import fails
Perfectly good WDLs from 6-12 months ago are failing to load. They may still be on draft-3, but I don't know because I can't actually access them. I'm willing to update to WDL 1.0, but I need to access the newly invalid WDLs.
Full error json is as follows: { "causes": [], "exceptionClass": "org.broadinstitute.dsde.rawls.RawlsException", "message": "Failed to import 'https://api.firecloud.org/ga4gh/v1/tools/methodsDev:JointGenotypingTasks_WARP_applyVQSRModel/versions/4/plain-WDL/descriptor' (reason 1 of 1): Failed to process task definition 'GatherTranches' (reason 1 of 1): Failed to process expression 'write_lines(tranches)' (reason 1 of 1): Invalid parameter 'IdentifierLookup(tranches)'. Expected 'Array[String]' but got 'File'", "source": "rawls", "stackTrace": [ { "className": "org.broadinstitute.dsde.rawls.jobexec.MethodConfigResolver", "fileName": "MethodConfigResolver.scala", "lineNumber": 79, "methodName": "$anonfun$getMethodInputsOutputs$1" }, { "className": "scala.util.Success", "fileName": "Try.scala", "lineNumber": 262, "methodName": "map" }, { "className": "org.broadinstitute.dsde.rawls.jobexec.MethodConfigResolver", "fileName": "MethodConfigResolver.scala", "lineNumber": 70, "methodName": "getMethodInputsOutputs" }, { "className": "org.broadinstitute.dsde.rawls.workspace.WorkspaceService", "fileName": "WorkspaceService.scala", "lineNumber": 2022, "methodName": "$anonfun$getMethodInputsOutputs$2" }, { "className": "org.broadinstitute.dsde.rawls.util.MethodWiths", "fileName": "WithSupport.scala", "lineNumber": 53, "methodName": "$anonfun$withMethod$1" }, { "className": "slick.basic.BasicBackend$DatabaseDef", "fileName": "BasicBackend.scala", "lineNumber": 172, "methodName": "$anonfun$runInContextInline$1" }, { "className": "scala.concurrent.impl.Promise$Transformation", "fileName": "Promise.scala", "lineNumber": 470, "methodName": "run" }, { "className": "java.util.concurrent.ForkJoinTask$RunnableExecuteAction", "fileName": null, "lineNumber": -1, "methodName": "exec" }, { "className": "java.util.concurrent.ForkJoinTask", "fileName": null, "lineNumber": -1, "methodName": "doExec" }, { "className": "java.util.concurrent.ForkJoinPool$WorkQueue", "fileName": null, "lineNumber": -1, "methodName": "topLevelExec" }, { "className": "java.util.concurrent.ForkJoinPool", "fileName": null, "lineNumber": -1, "methodName": "scan" }, { "className": "java.util.concurrent.ForkJoinPool", "fileName": null, "lineNumber": -1, "methodName": "runWorker" }, { "className": "java.util.concurrent.ForkJoinWorkerThread", "fileName": null, "lineNumber": -1, "methodName": "run" } ], "statusCode": 400 }
Comments
8 comments
Clarification: these are version 1.0 WDLs (I was able to get them through the API), so apparently the type coercion that I have isn't valid (anymore?). Still, it was a pain to click on all the things in the API swagger page -- is there an easier way to retrieve my invalid WDLs?
Hi Laura,
Thanks for writing in with this issue! Do you receive this error when trying to import this workflow from Dockstore? If so can you provide the Dockstore link? If not could you provide where these WDLs are published/where you are importing them from?
Best,
Emily
These are WDLs I had already imported into my workspace and run, e.g. https://app.terra.bio/#workspaces/broad-firecloud-dsde-methods/1000G-high-coverage-2019%20Laura/workflows/methodsDev/JointGenotyping_WARP_applyVQSRModel_condition3 I modified them from the WARP repo.
I realized that the input and output variables for one task had the same name. This has been fixed in WARP since I imported, so probably also fixed in Dockstore. Once I made the fix in the methods repo I was able to import fine.
Do you know if the WDL validation changed recently? When I created these in the methods repo, there was validation, but it didn't catch this particular issue.
Hi Laura,
Apologies for the delay! I've reached out to our engineers to see if anything has changed recently with WDL validation on the methods repo.
I'll update you when I hear back.
Best,
Emily
Hi Laura,
Would you be able to share the newer version of your WDL that corrected the issue with us (barnes@broadinstitue and sshah@broadinstitute.org)? Could you also provide a link to the WDL in the methods repo?
Thanks,
Emily
Hi Emily,
This version works: https://portal.firecloud.org/?return=terra#methods/methodsDev/JointGenotyping_WARP_applyVQSRModel/5 I've also shared the config in the workspace. I updated the import to a newer version that doesn't have duplicate variable names and that fixed the issue.
Hi Laura,
Our engineers let me know that from time to time validation does change/tighten. This is likely what you experienced here with your workflows. Do you have any further questions about WDL validation?
Best,
Emily
That makes sense. I got everything working now. Thanks Emily!
Please sign in to leave a comment.