Calculate memory usage of task
In order to choose an appropriate and cost-effective amount of memory for a task, it makes sense to run the task with a high level of memory, then assess how much memory the task actually used, and then use the estimated memory usage for future tasks.
However, Terra doesn't seem to report the actual memory used of any given task. It might be possible to run some memory profiling commands inside the task, but this requires actually modifying the workflow.
Does Terra provide any tools for memory profiling? Or is it perhaps possible to use a Google Cloud feature for this?
Comments
3 comments
Hey Migwell,
Thanks for writing in. I definitely understand the desire to assess how much memory your task actually used so you cab netter estimate memory usage for the future. We have an Active Feature Request to make Cromwell's monitoring options available which would help with this. I recommend upvoting the post and commenting your support if you haven't already. Our product team takes activity on this board into consideration when deciding what to prioritize.
For now, memory profiling commands are the best way to track utilization. If you have any questions about how to properly modify the workflow, we're happy to help as best we can.
Something else you may want to consider is making use of the "Retry with more memory" option. Configuring your WDL to work with this option may allow you to set lower memory values to start and have Terra automatically retry with more memory if the task fails due to insufficient memory.
Kind regards,
Jason
Thanks Jason. I've used `/usr/bin/time` in my workflow which seems to give me some reasonable results, and I have upvoted that post.
Hey Migwell,
Glad to hear! Thank you for upvoting - we'll be sure to pass your support along to the product team.
If we can help with anything else, please let us know!
Kind regards,
Jason
Please sign in to leave a comment.