Cromwell question: localization of reference files

Post author
Matthias De Smet

Hi,

First of all, sorry for asking a question thats already been asked 100's of times, but I can't seem to find an answer that applies.

I'm trying to run an alignment task in docker using bwa-mem2, but for some reason, the reference inputs aren't loaded into the same directory.

The references are imported into the task as a struct containing the fasta/fai/dict and bwa indexes. Bwa uses the fasta file as "reference base" to localise the rest of the indexes, but for some reason cromwell puts the fasta/fai/dict in a different dir than the bwa indexes.

According to every answer, doc or example workflow I found, my approach should work, except it doesn't and I'm at a loss as to why.

Any ideas?

Thanks

M

 

 

Comments

3 comments

  • Comment author
    Jason Cerrato

    Hi Matthias,

    I've typically found it to be the case that Cromwell constructs a similar path structure for the localized file based on where it originated. For example, if the file originates at location gs://example-bucket/path/to/file, Cromwell will localize the file at /cromwell_root/path/to/file. Does this sound like the situation you're running into?

    If so, many other users have created a workaround by including a command to copy or move the file from its localized location to a specific folder or the root, resulting in all of their files being in the same directory before continuing with the analysis in earnest. 

    Let us know what you think, and if your situation is actually different from what I've described above.

    Kind regards,

    Jason

    0
  • Comment author
    Matthias De Smet

    Hi Jason,

    That's exactly the case, the fasta/fai/dict are in another directory than the bwa indexes. Putting all the inputs in the same directory does fix the issue.

    I'd like to avoid putting many copy actions into our WDL's since it's just unneeded storage and IO overhead. Too bad cromwell doens't just put all files from a single struct in the same directory.

    Cheers

    Matthias

    0
  • Comment author
    Jason Cerrato

    Hi Matthias,

    I'm glad to see we were able to find you a working workaround. Here is another option you can consider: instead of adding "copy actions" to the WDL, you can add the creation of a symbolic link, e.g.

    ln /localization/path/file.txt /analysis/path/file.txt
    to the WDL, and then point the analysis to the latter path.

    I hope one of these works for you!

    Kind regards,

    Jason

    0

Please sign in to leave a comment.