Java Runtime Environment crash in FilterAlignmentArtifacts
I am trying to run FilterAlignmentArtifacts on some Mutect2 output (after filtering etc), however, the Java Runtime Environment crashes. This is also the case for every other gatk docker version that I tested.
2022/01/13 15:59:08 Starting container setup.
2022/01/13 15:59:10 Done container setup.
2022/01/13 15:59:12 Starting localization.
2022/01/13 15:59:24 Localization script execution started...
2022/01/13 15:59:24 Localizing input gs://fc-secure-09db8546-2b82-483a-b9d6-9f9e7a5d2805/af3082b1-39e1-4c0d-987a-8f99265bd9d7/Mutect2/7d1176b6-b2cd-4df0-8d33-7605beb69cdd/call-FilterAlignmentArtifacts/shard-0/attempt-3/script -> /cromwell_root/script
2022/01/13 15:59:27 Localization script execution complete.
2022/01/13 15:59:33 Done localization.
2022/01/13 15:59:34 Running user action: docker run -v /mnt/local-disk:/cromwell_root -v /mnt/d-1b2e5749db4b0a6439c4895809508e1e:/mnt/af7b5955462dc70f18fa6a82eae18e22:ro --entrypoint=/bin/bash broadinstitute/gatk@sha256:421d2fb2cc869249cef3f4d7a77289256d295b04ba623096228e0e5fd42939e9 /cromwell_root/script
Picked up _JAVA_OPTIONS: -Djava.io.tmpdir=/cromwell_root/tmp.1df1b1a5
15:59:39.071 INFO NativeLibraryLoader - Loading libgkl_compression.so from jar:file:/gatk/gatk-package-4.2.4.1-local.jar!/com/intel/gkl/native/libgkl_compression.so
15:59:39.304 INFO FilterAlignmentArtifacts - ------------------------------------------------------------
15:59:39.305 INFO FilterAlignmentArtifacts - The Genome Analysis Toolkit (GATK) v4.2.4.1
15:59:39.305 INFO FilterAlignmentArtifacts - For support and documentation go to https://software.broadinstitute.org/gatk/
15:59:39.305 INFO FilterAlignmentArtifacts - Executing as root@e9a0a37a1107 on Linux v5.10.68+ amd64
15:59:39.305 INFO FilterAlignmentArtifacts - Java runtime: OpenJDK 64-Bit Server VM v1.8.0_242-8u242-b08-0ubuntu3~18.04-b08
15:59:39.306 INFO FilterAlignmentArtifacts - Start Date/Time: January 13, 2022 3:59:39 PM GMT
15:59:39.306 INFO FilterAlignmentArtifacts - ------------------------------------------------------------
15:59:39.306 INFO FilterAlignmentArtifacts - ------------------------------------------------------------
15:59:39.307 INFO FilterAlignmentArtifacts - HTSJDK Version: 2.24.1
15:59:39.307 INFO FilterAlignmentArtifacts - Picard Version: 2.25.4
15:59:39.307 INFO FilterAlignmentArtifacts - Built for Spark Version: 2.4.5
15:59:39.307 INFO FilterAlignmentArtifacts - HTSJDK Defaults.COMPRESSION_LEVEL : 2
15:59:39.307 INFO FilterAlignmentArtifacts - HTSJDK Defaults.USE_ASYNC_IO_READ_FOR_SAMTOOLS : false
15:59:39.308 INFO FilterAlignmentArtifacts - HTSJDK Defaults.USE_ASYNC_IO_WRITE_FOR_SAMTOOLS : true
15:59:39.308 INFO FilterAlignmentArtifacts - HTSJDK Defaults.USE_ASYNC_IO_WRITE_FOR_TRIBBLE : false
15:59:39.308 INFO FilterAlignmentArtifacts - Deflater: IntelDeflater
15:59:39.308 INFO FilterAlignmentArtifacts - Inflater: IntelInflater
15:59:39.308 INFO FilterAlignmentArtifacts - GCS max retries/reopens: 20
15:59:39.308 INFO FilterAlignmentArtifacts - Requester pays: disabled
15:59:39.309 WARN FilterAlignmentArtifacts
[1m[31m !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
Warning: FilterAlignmentArtifacts is an EXPERIMENTAL tool and should not be used for production
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!![0m
15:59:39.309 INFO FilterAlignmentArtifacts - Initializing engine
15:59:43.148 INFO FeatureManager - Using codec VCFCodec to read file gs://fc-secure-09db8546-2b82-483a-b9d6-9f9e7a5d2805/af3082b1-39e1-4c0d-987a-8f99265bd9d7/Mutect2/7d1176b6-b2cd-4df0-8d33-7605beb69cdd/call-SelectPassingVariants/shard-0/cacheCopy/10417_118_selected.vcf.gz
15:59:43.370 WARN IntelInflater - Zero Bytes Written : 0
15:59:43.683 WARN IntelInflater - Zero Bytes Written : 0
15:59:43.721 INFO FilterAlignmentArtifacts - Done initializing engine
15:59:43.724 INFO NativeLibraryLoader - Loading libgkl_utils.so from jar:file:/gatk/gatk-package-4.2.4.1-local.jar!/com/intel/gkl/native/libgkl_utils.so
15:59:43.725 INFO NativeLibraryLoader - Loading libgkl_smithwaterman.so from jar:file:/gatk/gatk-package-4.2.4.1-local.jar!/com/intel/gkl/native/libgkl_smithwaterman.so
15:59:43.727 INFO SmithWatermanAligner - Using AVX accelerated SmithWaterman implementation
#
# A fatal error has been detected by the Java Runtime Environment:
#
# SIGSEGV (0xb) at pc=0x00007f34adc082cc, pid=17, tid=0x00007f35206c6700
#
# JRE version: OpenJDK Runtime Environment (8.0_242-b08) (build 1.8.0_242-8u242-b08-0ubuntu3~18.04-b08)
# Java VM: OpenJDK 64-Bit Server VM (25.242-b08 mixed mode linux-amd64 compressed oops)
# Problematic frame:
# C [libbwa.5817426980852573163.jnilib+0x92cc] bwa_mem2idx+0xcc
#
# Core dump written. Default location: /cromwell_root/core or core.17
#
# An error report file with more information is saved as:
# /cromwell_root/hs_err_pid17.log
#
# If you would like to submit a bug report, please visit:
# http://bugreport.java.com/bugreport/crash.jsp
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#
Using GATK jar /root/gatk.jar defined in environment variable GATK_LOCAL_JAR
Running:
java -Dsamjdk.use_async_io_read_samtools=false -Dsamjdk.use_async_io_write_samtools=true -Dsamjdk.use_async_io_write_tribble=false -Dsamjdk.compression_level=2 -Xmx11g -jar /root/gatk.jar FilterAlignmentArtifacts --input gs://fc-secure-09db8546-2b82-483a-b9d6-9f9e7a5d2805/af3082b1-39e1-4c0d-987a-8f99265bd9d7/Mutect2/7d1176b6-b2cd-4df0-8d33-7605beb69cdd/call-VariantCall/shard-0/cacheCopy/10417_118_bamout.bam --variant gs://fc-secure-09db8546-2b82-483a-b9d6-9f9e7a5d2805/af3082b1-39e1-4c0d-987a-8f99265bd9d7/Mutect2/7d1176b6-b2cd-4df0-8d33-7605beb69cdd/call-SelectPassingVariants/shard-0/cacheCopy/10417_118_selected.vcf.gz --reference gs://gcp-public-data--broad-references/hg19/v0/Homo_sapiens_assembly19.fasta --bwa-mem-index-image /cromwell_root/gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.fasta.64.bwt --output 10417_118_realignmentfiltered.vcf.gz --max-reasonable-fragment-length 10000 --min-aligner-score-difference-per-base 0.5 --min-mismatch-difference-per-base 0.05
2022/01/13 15:59:46 Starting delocalization.
2022/01/13 15:59:47 Delocalization script execution started...
2022/01/13 15:59:47 Delocalizing output /cromwell_root/memory_retry_rc -> gs://fc-secure-09db8546-2b82-483a-b9d6-9f9e7a5d2805/af3082b1-39e1-4c0d-987a-8f99265bd9d7/Mutect2/7d1176b6-b2cd-4df0-8d33-7605beb69cdd/call-FilterAlignmentArtifacts/shard-0/attempt-3/memory_retry_rc
2022/01/13 15:59:50 Delocalizing output /cromwell_root/rc -> gs://fc-secure-09db8546-2b82-483a-b9d6-9f9e7a5d2805/af3082b1-39e1-4c0d-987a-8f99265bd9d7/Mutect2/7d1176b6-b2cd-4df0-8d33-7605beb69cdd/call-FilterAlignmentArtifacts/shard-0/attempt-3/rc
2022/01/13 15:59:52 Delocalizing output /cromwell_root/stdout -> gs://fc-secure-09db8546-2b82-483a-b9d6-9f9e7a5d2805/af3082b1-39e1-4c0d-987a-8f99265bd9d7/Mutect2/7d1176b6-b2cd-4df0-8d33-7605beb69cdd/call-FilterAlignmentArtifacts/shard-0/attempt-3/stdout
2022/01/13 15:59:54 Delocalizing output /cromwell_root/stderr -> gs://fc-secure-09db8546-2b82-483a-b9d6-9f9e7a5d2805/af3082b1-39e1-4c0d-987a-8f99265bd9d7/Mutect2/7d1176b6-b2cd-4df0-8d33-7605beb69cdd/call-FilterAlignmentArtifacts/shard-0/attempt-3/stderr
2022/01/13 15:59:56 Delocalizing output /cromwell_root/10417_118_realignmentfiltered.vcf.gz -> gs://fc-secure-09db8546-2b82-483a-b9d6-9f9e7a5d2805/af3082b1-39e1-4c0d-987a-8f99265bd9d7/Mutect2/7d1176b6-b2cd-4df0-8d33-7605beb69cdd/call-FilterAlignmentArtifacts/shard-0/attempt-3/10417_118_realignmentfiltered.vcf.gz
Required file output '/cromwell_root/10417_118_realignmentfiltered.vcf.gz' does not exist.
I don't even know how to approach debugging this ... does anyone have any pointers? Many thanks!
Best,
Philipp
Comments
5 comments
Hi Philipp,
Thank you for writing in about this issue. Can you share the workspace where you are seeing this issue with Terra-Support@firecloud.org by clicking the Share button in your workspace? The Share option is in the three-dots menu at the top-right.
Please provide us with
We’ll be happy to take a closer look as soon as we can!
Kind regards,
Samantha
Hi Samatha,
I added the support email to the workspace ( https://app.terra.bio/#workspaces/carterlabtest/PH%20playground ). I hope you still have access to the carterlab group (from previous support requests).
I also added the email as a reader to the WDL. It's in particular submission ID af3082b1-39e1-4c0d-987a-8f99265bd9d7 (but the ones above that too) and there is only one workflow in that workspace.
Let me know if I can help with anything else!
Philipp
Hi Philipp Hahnel,
Sorry for the delayed response. You mentioned you are running FilterAlignmentArtifacts on some Mutect2 output. As the FilterAlignmentArtifacts tool documentation states, "the input bam to this tool should be the same tumor bam that Mutect2 was run on," not the output. This could be why you are seeing those errors.
Here is a GATK forum thread where this issue where this was discussed: https://gatk.broadinstitute.org/hc/en-us/community/posts/360071839192/comments/360014167732. And here is another forum post regarding the same error with some suggestions for workarounds: https://gatk.broadinstitute.org/hc/en-us/community/posts/360062334692-FilterAlignmentArtifacts-error.
Best,
Samantha
Dear Samantha,
thank you very much! I've been updating some legacy code, so it must have slipped my attention that the tool was updated in its usage. Thanks as well for pointing me to the threads! They also contained some other useful information.
Best,
Philipp
Dear Samantha,
despite updating the input for the FilterAlignmentArtifacts call, the error unfortunately still persists.
Would the GATK community forum that you linked to a better place to ask for guidance concerning this issue?
Best,
Philipp
Please sign in to leave a comment.