Using fofn files in docker
Hi,
I've got a task that runs in docker and creates a fofn file. This fofn is then to be used as input for another task (not in docker). However, the filepaths in the fofn are relative to the cromwell-root and as such not accessible to the new task.
How could I fix this?
NB, this is an overly simplified situation sketch, but I think it gets across the message.
To be more specific, I'm trying to use the WDL `size()` directive to decide what to do with each file in the fofn.
Thanks
M
Comments
5 comments
Hi Matthias,
Thanks for writing in. We'll take a look at your issue and get back to you as soon as we can!
Kind regards,
Jason
Hi Matthias,
Would you be able to create a .list file in your first task that gets fed into your next task as input? Here is a WDL that may give you an idea of how to write it into your WDL: https://portal.firecloud.org/#methods/bshifaw/make_fofn/3/wdl
Are you feeding the files from the fofn as input to the second task, or streaming those files in? If the solution above doesn't work for you I would love to get more details about the relationship between the two tasks, the fofn, and the actual files included in that list. If you would like me to take a closer look at your WDL, please share it with jcerrato@broadinstitute.org and I'll be happy to take a closer look.
Kind regards,
Jason
Hi Jason,
Thanks for the reply!
Might I refer to a github issue for more info? https://github.com/openwdl/wdl/issues/452
I'll also shoot you an e-mail with the workflow, perhaps it might be more clear then.
Thanks again
Matthias
FYI, fixed by letting the task that creates the fofn run on the host instead of in docker
Matthias
Hi Matthias,
I'm glad you were able to find a solution here! For anyone else who comes across this thread, the tasks were being run on an on-premise Slurm cluster rather than in Terra.
We've received your new inquiry and we'll be responding shortly. Thanks again for writing in.
Kind regards,
Jason
Please sign in to leave a comment.