Workflows
Updated advice is available to understand and control workflow logging costs.
Terra uses GCP Cloud Logging to store and display WDL task logs. Cloud Logging applies a one-time charge for ingestion and a recurring charge for storage. Pricing information is available here.
Logs are generated by the command-line program invoked in a WDL command block, commonly referred to as a tool.
Control ingestion charges:
- Make sure the tool does not output repetitive logs, such as progress or timestamps
- If supported by the tool, use a command-line option to reduce verbosity
- Pipe tool output to
/dev/null - Pipe tool output to a file on disk and reference it in the WDL
outputssection. Cromwell will upload the log to GCS, where you can view and download it.
Control storage charges:
- Adjust log retention in workspace settings
- Reduce log volume with any mechanism mentioned in "Control ingestion charges"