Passing Array of Arrays to workflow
I'm trying to test out a new workflow, and I've created a "sim" data table that has 3 columns (id, a string, and a list of files in the workspace's cloud storage bucket). I also have a wdl / workflow that takes as input an Array[Int], an Array[String], and an Array[Array[String]]. I also created a set table corresponding to my "sim" table that has a set that contains all (currently 5) rows in the "sim" table. I then attempt to run my workflow by selecting the sim_set table, selecting the appropriate set that contains all rows, and setting the input attributes to point to the appropriate values (e.g. "this.sims.sim_id", etc). However, the job is failing and I see the following error in the Job Manager:
Workflow input processing failed (Caused by [reason 1 of 1]: Failed to evaluate input 'inputfiles' (reason 1 of 1): No coercion defined from '"gs://fc-d6e33e80-32b4-4f1f-a9c6-957884ade6dd/Sim1/Assets/test.json"' of type 'spray.json.JsString' to 'Array[String]'.)
I then looked in the Job History and clicked the "Download TSV" link under the Workflows tab, and the "inputResolutions" column shows something unexpected. For the "inputfiles" argument to my workflow, the value is a single array of length 15, rather than what I would have expected, which is a length 5 array, with each entry being an array of length 3.
Am I misunderstanding how this should work? Is this not a supported scenario? (everything I've read elsewhere seems to indicate it is. e.g. https://support.terra.bio/hc/en-us/articles/6624394629147-How-to-include-arrays-in-a-data-table). If I remove the Array[Array[String]] parameter to the workflow and run, it succeeds, but I can't figure out how to get that array of arrays to work.
Thanks!
Comments
4 comments
Hi Jeff,
Thanks for writing in with this! A member of the Terra support team will follow up with you as soon as they are able.
If relevant, please let us know if there is any urgency around this request so that the team can prioritize it appropriately.
Kind regards,
Pamela
Hi Jeff,
Thank you for writing in about this issue. Can you share the workspace where you are seeing this issue with Terra Support by clicking the Share button in your workspace? The Share option is in the three-dots menu at the top-right.
Please provide us with
We’ll be happy to take a closer look as soon as we can!
Kind regards,
Shoaib
Okay, I think I've shared it properly with support.
Here's the workspace.
The submission id is 159ea100-5591-4e1c-9df1-f5698c3428d5.
The workspace id is d092e741-b95a-43ef-a0a8-4f33fefb5dcd.
Let me know if there's anything else you need.
Thanks!
Hi Jeff,
Normally, Terra does not really support nested arrays and it usually leads to errors, however as you stated earlier, there isn't anything in the WDL documentation that says this isn't allowed. We've gone ahead and filed a bug report so that the engineers can look into it soon and we'd be happy to give you a follow-up once this issue is resolved.
Kind regards,
Shoaib
Please sign in to leave a comment.