Repository navigation
Amazon Corretto Java 8 crashes since trace agent v1.56.0, v1.55.0 was good #10017
Description
Tracer Version(s)
1.56.0
Java Version(s)
1.8.0_462
JVM Vendor
Amazon Corretto
Bug Report
We started experiencing errors like this in our apps since yesterday after we started pulling version 1.56.0. This was not happening in version 1.55.0
Some of our apps run under Tomcat 8.5.100 and Amazon Corretto 1.8.0_462
This problem usually happens when the application is under load, else it is rare. But under load it may happen one time after the other...
# A fatal error has been detected by the Java Runtime Environment:
#
# SIGSEGV (0xb) at pc=0x00007fb5102ce122, pid=1, tid=0x00007fb4e4760b38
#
# JRE version: OpenJDK Runtime Environment (8.0_462-b08) (build 1.8.0_462-b08)
# Java VM: OpenJDK 64-Bit Server VM (25.462-b08 mixed mode linux-amd64 compressed oops)
# Problematic frame:
# V [libjvm.so+0x2ce122] Annotations::make_java_array(Array<unsigned char>*, Thread*)+0x72
#
# Core dump written. Default location: //core or core.1
#
# An error report file with more information is saved as:
# //hs_err_pid1.log
#
# If you would like to submit a bug report, please visit:
# https://github.com/corretto/corretto-8/issues/
It is sometimes different but always a SIGSEV
# A fatal error has been detected by the Java Runtime Environment:
#
# SIGSEGV (0xb) at pc=0x00007f5ee81c2d70, pid=1, tid=0x00007f5eca1fcb38
#
# JRE version: OpenJDK Runtime Environment (8.0_462-b08) (build 1.8.0_462-b08)
# Java VM: OpenJDK 64-Bit Server VM (25.462-b08 mixed mode linux-amd64 compressed oops)
# Problematic frame:
# V [libjvm.so+0xa73d70] Symbol::increment_refcount()+0x0
#
# Core dump written. Default location: //core or core.1
#
# An error report file with more information is saved as:
# //hs_err_pid1.log
#
# Compiler replay data is saved as:
# //replay_pid1.log
#
# If you would like to submit a bug report, please visit:
# https://github.com/corretto/corretto-8/issues/
There was no code change (JSP) or any other change in the applications beside the datadog trace jar.
We re-deployed all our affected applications with 1.55.0 and the issue disappear.
Expected Behavior
Our application should not crash on version 1.56.0 of the trace java agent, this was stable in version 1.55.0, reverting to 1.55.0 fixes the issue.
Reproduction Code
No response
Activity
mcculls commentedon Nov 21, 2025
Hi @RedVortex - could you open a support ticket and attach the full hs_err file as well as the core dump - thanks!
mcculls commentedon Nov 21, 2025
@RedVortex also if you can recreate this in a staging environment, it would be useful to find out if any of these settings help:
Each toggles a feature back to its state in v1.55.0, so if any of them do have an effect that would help focus the investigation.
selatotal commentedon Nov 22, 2025
The same is happing with me, in the same scenario.
Creating a Docker image with lastest version of Correto Java 8 Alpine and running an application with dd-java-tracer in latest-version, the problem is occurring. Fixing the 1.55.0 version is OK
RedVortex commentedon Nov 22, 2025
I believe someone in my team did that today after I have them the hs_err. I couldn't get the core dump however.
RedVortex commentedon Nov 22, 2025
We cannot recreate easily this situation unless we do it in production and since this create outages we cannot retry this unfortunately for now.
mcculls commentedon Nov 22, 2025
Hi @RedVortex - understood, from the data available at the moment the suggested workarounds would be to either revert to v1.55.0 or turn off the code-origin feature in v1.56.0 with:
We haven't yet recreated this locally, but we have a potential solution to hand.
mcculls commentedon Nov 25, 2025
Hi @RedVortex we just released v1.56.1 which should address the reported issue.
amit-p-jt commentedon May 11, 2026
We are on version 1.61.1. We are still facing the same problem
I'm attaching the hs_err file for your reference.
hs_err_pid441017.log
jpbempel commentedon May 12, 2026
Hi @amit-p-jt ,
Thanks for reporting this. This is a known issue (see previous posts) that is unfortunately a very elusive JVM bug.
If you can have a reproducer of this issue, we are VERY interested to get it.
Otherwise, so far we have only see this issue on JDK8. I recommend to upgrade to a more recent JDK.
Thanks