Write cromwell output to its own folder instead of the root directory Answered

Post author
Kamil Slowikowski

Problem

Each time a workflow runs, it creates log files in the root directory. After running hundreds of workflows, our root directory is now polluted with entries like this:

9166533f-5418-4769-a370-31ab81dba8f6

There are too many of these entries, so it is difficult to scroll and search for our neatly organized files and folders that are located in the same directory.

Please see the screenshot below for an example.

Solution

Stop writing the workflow outputs here:

/uniqueID

Start writing the workflow outputs here:

/workflow_output/uniqueID

 

Comments

2 comments

  • Comment author
    Jason Cerrato

    Hi Kamil,

    This message is to confirm that I've documented your request.

    Kind regards,

    Jason

    0
  • Comment author
    Kamil Slowikowski

    Here's a workaround for this issue.

    Step 1. Wait for all of the jobs in your workspace to finish running.

    Step 2. Run the bash script below to move the cromwell logs to a folder called workflow_logs

    #!/bin/bash
bucket=fc-abababab-abab-abab-abab-abababababab
while read x
do
  x=${x%/}
  slug=$(basename $x)
  gsutil -m mv $x gs://${bucket}/workflow_logs/${slug}
done < <(
  gsutil ls -d gs://${bucket}/????????-????-????-????-????????????
)
    1

Please sign in to leave a comment.