Docker: argument list too long
I have 2 large scatter (>3000 samples) - gather statements in my pipeline. I hacked my way around the first one that was failing with a different error (Request payload size exceeds the limit: 5242880 bytes).
Now the error seems to be during initiating Docker in the second gather:
2019/05/07 11:17:14 E: command start failed: (fork/exec /usr/bin/docker: argument list too long)
How to deal with this if I cannot even pass that long list of arguments to the task (VM)? The first error I fixed by writing Array to file and manually copying from the bucket, e.g.:
task make_matrix {
Array[File] vectors
command <<<
cat ${write_lines(vectors)} > vectors_2_download.txt
mkdir dir_vectors
cat vectors_2_download.txt | gsutil -m cp -I dir_vectors/
paste dir_vectors/* > combined_vectors.txt
>>>
Comments
2 comments
Update:
Changing File to String in `Array[String] vectors` definition fixed the problem.
Damian
Damian,
Thank you for posting your question and the solution. Apologies for the delay! Glad to see that you were able to resolve the issue.
Sushma
Please sign in to leave a comment.