JVM crash on startup due to Tomcat Native APR incompatibility with OpenSSL 3.x

[mbiuki]

Problem Description

dotCMS crashes on startup with a JVM segmentation fault when the Tomcat Native APR library attempts to initialize OpenSSL 3.x. This prevents dotCMS from starting successfully.

Error Details

SIGSEGV (0xb) at pc=0x00007fbce915d234
Problematic frame: C [libcrypto.so.3+0x1fd234] EVP_MD_get0_provider+0x4
Java frames:
  org.apache.tomcat.jni.SSL.fipsModeGet()I+0
  org.apache.catalina.core.AprLifecycleListener.initializeSSL()V+198

Root Cause

The Tomcat Native APR library (tcnative 1.2.35) is incompatible with OpenSSL 3.x. This causes a native library crash during SSL initialization when the APR Lifecycle Listener attempts to load OpenSSL.

System Details:

• OS: Ubuntu 24.04.3 LTS
• Java: OpenJDK 21.0.4+7-LTS
• Tomcat: 9.0.108
• tcnative: 1.2.35 (outdated - Tomcat recommends minimum 1.3.0)
• OpenSSL: 3.x (libcrypto.so.3)

Proposed Solution

Remove libtcnative-1.so from container image to disable Tomcat Native/OpenSSL entirely and let Tomcat use pure Java JSSE for SSL/TLS operations.

This approach:

• Eliminates the OpenSSL 3.x compatibility issue
• Removes the need to maintain native library versions
• Uses Java's built-in SSL implementation (JSSE), which is fully functional
• Avoids FIPS mode complications with native OpenSSL
• Simplifies the container image

Configuration Location

dotCMS/src/main/resources/container/tomcat9/conf/server.xml:5

<Listener className="org.apache.catalina.core.AprLifecycleListener" SSLEngine="${CMS_SSL_ENGINE:-on}" />

Alternative Workarounds

1. Temporary workaround: Set environment variable CMS_SSL_ENGINE=off to disable APR SSL
2. Upgrade tcnative: Update to version 1.3.0+ (but this still requires maintaining native library compatibility)

Recommendation

The cleanest solution is to remove libtcnative-1.so from the container build entirely, eliminating this entire class of compatibility issues and simplifying the deployment.

Impact

• Severity: High - prevents dotCMS startup
• Affected environments: Systems with OpenSSL 3.x (Ubuntu 24.04+, RHEL 9+, etc.)
• Workaround available: Yes (disable APR or remove native library)

[mbiuki]

Quickest Solution for Immediate Relief

If you're experiencing this crash right now and need dotCMS running immediately, the fastest workaround is to disable the APR SSL Engine without modifying the container image:

Option 1: Environment Variable (Fastest - 30 seconds)

Set the environment variable before starting dotCMS:

export CMS_SSL_ENGINE=off
# Then start dotCMS normally
./mvnw -pl :dotcms-core -Pdocker-start

Or if using Docker Compose, add to your docker-compose.yml:

services:
  dotcms:
    environment:
      - CMS_SSL_ENGINE=off

This tells Tomcat's APR Lifecycle Listener to skip SSL initialization, avoiding the OpenSSL crash entirely. Tomcat will automatically fall back to using Java's built-in JSSE for SSL/TLS operations.

Option 2: Direct Configuration Edit (2 minutes)

Edit dotCMS/src/main/resources/container/tomcat9/conf/server.xml line 5:

<!-- Change from: -->
<Listener className="org.apache.catalina.core.AprLifecycleListener" SSLEngine="${CMS_SSL_ENGINE:-on}" />

<!-- To: -->
<Listener className="org.apache.catalina.core.AprLifecycleListener" SSLEngine="off" />

Then rebuild: ./mvnw install -pl :dotcms-core -DskipTests

---

Both workarounds achieve the same result: dotCMS starts successfully using Java JSSE instead of native OpenSSL. There is no functionality loss - Java's SSL/TLS implementation is fully featured and production-ready.

The permanent solution (removing libtcnative-1.so from the container build) can be implemented separately without time pressure.

[github-actions]

PRs linked to this issue

• fix: Remove Tomcat Native APR library to resolve OpenSSL 3.x crash by @mbiuki