docker run, exec format error
Hello!
I am attempting to use the GATK-SV pipeline to call structural variants, including mobile element insertions, in Illumina WGS data. Due to MELT's restrictive license, the pipeline authors cannot provide a public docker image. Instead, they provide great instructions for the user to agree to the license, download the software, build the image, and upload it us.gcr.io. I did this with MELT version 2.2.2, while making some edits to the dockerfile and image's scripts to account for this version upgrade from MELT 2.0.5 patch (see my fork and branch here).
Then, upon testing the image with the gatk-sv-single-sample workflow and the NA12878 sample data, I receive an error in the RunMELT task. I have shared my workspace "libr-wgs/LIBR-WGS-Pipeline" with support, linked to the full RunMELT log file, and copied the relevant lines below.
2022/05/02 06:02:04 Running user action: docker run -v /mnt/local-disk:/cromwell_root -v /mnt/d-1b2e5749db4b0a6439c4895809508e1e:/mnt/af7b5955462dc70f18fa6a82eae18e22:ro --entrypoint=/bin/bash us.gcr.io/gage-wgs-pipeline/melt@sha256:471540294890c1775fecf30e5658530e050b536c6c82d2995d35738cdb74c8e4 /cromwell_root/script
exec /bin/bash: exec format error
grep: /cromwell_root/stderr: No such file or directory
- Mike
Comments
11 comments
Hi Mike,
Thanks for writing in! We can certainly take a look at this for you. As it turns out, Support doesn't currently have access to your workspace as it's protected by an Authorization Domain. My suggestion would be to add my account to the group, and I will take a look as soon as I can. I'm also unable to view the log file that you've shared, but once I have access to the workspace, I should be able to read that.
Please let me know if you have any questions.
Best,
Josh
Hi Mike,
Since you've already shared the workspace with support, you'll only need to have my email address (evans@broadinstitute.org) added to the Authorization Domain group. Once that's done, let me know and I'll take a look at your workspace as soon as I can.
Best,
Josh
Hi Josh,
Thanks for your reply! I just added you as a member to the Authorization Domain group.
Mike
Hi Mike,
Thanks for adding me! I've checked and I can now access the workspace. Let me take a look and I'll let you know once I have any updates.
Best,
Josh
Hi Mike,
We're still looking into your issue. Would it be possible to add my colleague (svelasqu@broadinstitute.org) to the Authorization Domain group? Once that's been done, we will continue to investigate this.
Best,
Josh
Done!
Hi Mike,
Thanks again for giving us access! We looked at the workspace and your new version of the workflow. After doing some testing, we concluded that your updated docker seems to be configured correctly. We were able to run a simple "hello world" WDL through it without any errors.
This confirms that the issue is happening with its interaction with the RunMelt task. That task is interacting with the Cromwell directory in a way the system doesn't expect. (It's possible the MELT script is deleting the log file.)
One possible cause for this could be the fact the original workflow uses a patched version of the MELT script. Our suggestion would be to reach out to the pipeline authors and ask if the workflow requires their specific patch of the script or what versions are required.
Please let me know if that information is helpful to you.
Best,
Josh
I encountered the exact same error trying to run a different workflow. I built a docker image with ubuntu:18.04 and a plink2 binary and ran a simple workflow with a plink2 command. It resulted in the "exec format error" shown above. Then I tried substituting a different base docker image in the FROM: field of my Dockerfile, and running the exact same workflow with that image instead, and it succeeded.
Additional notes: 1) I could run the plink2 command interactively in the original docker image on my local machine, 2) the different base image was originally based on ubuntu:18.04, but a version from 2 years ago.
Hi Stephanie,
Thanks for letting us know of this, I'm glad that you were able to get your issues resolved! Do you have any additional questions or concerns? We'd be happy to look into them!
Kind regards,
Shoaib
Hi Stephanie,
We haven't heard from you in a while so we're going to close out this ticket. If you still require assistance, simply respond to this post and we'll be happy to pick up where we left off!
Kind regards,
Shoaib
I recently hit this exact issue, and I can explain what is happening. The problem is not with any scripts, but the execution of the bash binary itself. This occurs because the docker image was built on a different architecture from what is supported in the cloud. In my circumstance, I built my image on my M1 Mac, which generates a docker image compatible with linux/arm64. This will not successfully run on cloud instances which typically expect a linux/amd64 platform. Hence the message "exec format error".
You can still use your M1 Mac to build cross platform images (such as linux/amd64), but it requires docker's buildx functionality. You can learn more about how to use buildx from the web (for example: here or here).
Please sign in to leave a comment.