WDL OutOfMemory Issue
Hello,
I have a script to fetch reads from basespace, and execute kraken2 on the reads. I have something like 300 reads (150 R1, 150 R2). The script fail at some point. I tried to set higher memory by still fail.
I'd glad to get help.
Thank you!
The part of code causing the issue:
scatter(sample_name in GetReadsList.samples_name) {
call FetchReads {
input:
basespace_sample_name = sample_name,
basespace_collection_id = basespace_collection_id,
api_server = api_server,
access_token = access_token
}
call Detect_Specie {
input:
read1 = FetchReads.read1,
read2 = FetchReads.read2,
sample_id = sample_name
}
}
The error :
"Task Specie_Detection.FetchReads:24:1 failed. The job was stopped before the command finished. PAPI error code 9. Execution failed: generic::failed_precondition: while running "-c grep -E -q 'OutOfMemory|Killed' /cromwell_root/stderr ; echo $? > /cromwell_root/memory_retry_rc": unexpected exit status 1 was not ignored [CheckingForMemoryRetry] Unexpected exit status 1 while running "-c grep -E -q 'OutOfMemory|Killed' /cromwell_root/stderr ; echo $? > /cromwell_root/memory_retry_rc": sh: write error: No space left on device
Comments
3 comments
Hi david maimoun,
Thank you for writing in about this issue. Can you share the workspace where you are seeing this issue with Terra Support by clicking the Share button in your workspace? The Share option is in the three-dots menu at the top-right.
Please provide us with
We’ll be happy to take a closer look as soon as we can!
Kind regards,
Samantha
Hi Samatha,
Thanks for reaching out
I fixed the issue; i was some sample name different that the others, and my task didn't handle properly.
But more generally, is it a good practice to set "memory" for each task, or let cromwell handle that alone?
Thank you
Hi David,
It's generally good practice to define the memory yourself. If you do not define it, it will use the default value which is 2G memory.
You can read more about the default runtime attributes here: Default runtime attributes for workflow submissions
Kind regards,
Jason
Please sign in to leave a comment.