Need Help?

Search our documentation and community forum

Terra is a cloud-native platform for biomedical researchers to access data, run analysis tools, and collaborate.
Terra powers important scientific projects like FireCloud, AnVIL, and BioData Catalyst. Learn more.

Cromwell question: localization of reference files

Comments

3 comments

  • Avatar
    Jason Cerrato

    Hi Matthias,

    I've typically found it to be the case that Cromwell constructs a similar path structure for the localized file based on where it originated. For example, if the file originates at location gs://example-bucket/path/to/file, Cromwell will localize the file at /cromwell_root/path/to/file. Does this sound like the situation you're running into?

    If so, many other users have created a workaround by including a command to copy or move the file from its localized location to a specific folder or the root, resulting in all of their files being in the same directory before continuing with the analysis in earnest. 

    Let us know what you think, and if your situation is actually different from what I've described above.

    Kind regards,

    Jason

    0
    Comment actions Permalink
  • Avatar
    Matthias De Smet

    Hi Jason,

    That's exactly the case, the fasta/fai/dict are in another directory than the bwa indexes. Putting all the inputs in the same directory does fix the issue.

    I'd like to avoid putting many copy actions into our WDL's since it's just unneeded storage and IO overhead. Too bad cromwell doens't just put all files from a single struct in the same directory.

    Cheers

    Matthias

    0
    Comment actions Permalink
  • Avatar
    Jason Cerrato

    Hi Matthias,

    I'm glad to see we were able to find you a working workaround. Here is another option you can consider: instead of adding "copy actions" to the WDL, you can add the creation of a symbolic link, e.g.

    ln /localization/path/file.txt /analysis/path/file.txt
    to the WDL, and then point the analysis to the latter path.

    I hope one of these works for you!

    Kind regards,

    Jason

    0
    Comment actions Permalink

Please sign in to leave a comment.

Powered by Zendesk