Skip to content

Load bundled HDF core libraries before the JNI wrappers initialize - #480

Open
mattjala wants to merge 1 commit into
HDFGroup:masterfrom
mattjala:native_lib_preload
Open

Load bundled HDF core libraries before the JNI wrappers initialize#480
mattjala wants to merge 1 commit into
HDFGroup:masterfrom
mattjala:native_lib_preload

Conversation

@mattjala

Copy link
Copy Markdown
Collaborator

HDFView sometimes fails to start when another HDF5 or HDF4 installation is visible to the operating system's dynamic loader. The workaround so far has been to require users to hand-edit their PATH to avoid detection of the pre-existing libraries.

The cause is that two native libraries load by two different mechanisms. The JNI wrapper (hdf5_java) is loaded by the JVM and honors java.library.path, so our packaging already pins it correctly. But the wrapper lists the core library (libhdf5.so.320 on Linux, hdf5.dll on Windows) as a NEEDED dependency, and that one is resolved by the OS loader rather than the JVM. On Linux the loader checks LD_LIBRARY_PATH before the RUNPATH we ship, and on Windows the search order includes PATH. Either way a foreign copy can be detected first.

This change loads the bundled core libraries by absolute path before anything touches the wrapper. Once a library is mapped into the process, the loader satisfies the wrapper's dependency from the already-mapped copy, matching on soname on Linux and on module base name on Windows, and never searches PATH or LD_LIBRARY_PATH at all.

NativeLibraryLoader looks for each library in -Dhdfview.nativedir, then -Dhdfview.root (jpackage already sets this to $APPDIR), then the directory named by -Dhdf.hdf5lib.H5.hdf5lib, then each entry of java.library.path. The call sits at the top of FileFormat's static initializer, which is the earliest point the wrapper can initialize.

@mattjala
mattjala requested a review from lrknox as a code owner August 11, 2026 21:22
@mattjala mattjala added the Component - Build Build system improvement label Aug 11, 2026
*
* @return the resolved file, or {@code null} if this directory has no match
*/
private static File resolveInDir(String dir, String baseName, String mappedName)
HDFView sometimes fails to start when another HDF5 or HDF4 installation is
visible to the operating system's dynamic loader. The workaround so far has been
to require users to hand-edit their PATH to avoid detection of the
pre-existing libraries.

The cause is that two native libraries load by two different mechanisms. The JNI
wrapper (hdf5_java) is loaded by the JVM and honors java.library.path, so our
packaging already pins it correctly. But the wrapper lists the core library
(libhdf5.so.320 on Linux, hdf5.dll on Windows) as a NEEDED dependency, and that
one is resolved by the OS loader rather than the JVM. On Linux the loader checks
LD_LIBRARY_PATH before the RUNPATH we ship, and on Windows the search order
includes PATH. Either way a foreign copy can be detected first.

This change loads the bundled core libraries by absolute path before anything
touches the wrapper. Once a library is mapped into the process, the loader
satisfies the wrapper's dependency from the already-mapped copy, matching on
soname on Linux and on module base name on Windows, and never searches PATH or
LD_LIBRARY_PATH at all.

NativeLibraryLoader looks for each library in -Dhdfview.nativedir, then
-Dhdfview.root (jpackage already sets this to $APPDIR), then the directory named
by -Dhdf.hdf5lib.H5.hdf5lib, then each entry of java.library.path. The call sits
at the top of FileFormat's static initializer, which is the earliest point the
wrapper can initialize.
@mattjala
mattjala force-pushed the native_lib_preload branch from 3a3312c to 04dc9a8 Compare August 11, 2026 21:27
// Best-effort pin to the bundled core libraries before any JNI wrapper initializes.
// Failures fallback to the default library loading path.
NativeLibraryLoader.preloadHDF5();
NativeLibraryLoader.preloadHDF4();

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems like a solution at the wrong level. HDFView shouldn't need to know anything about how to look for and load the native libraries, it should only be concerned with using the object library and, consequently, the JNI wrappers. The JNI wrappers already perform the loading of HDF5 and HDF4 and all library loading logic should stay there only, as they're the primary consumers of those libraries and directly coupled to them. It's possible there's an issue with the loading logic there, but that seems unlikely since the logic there is basically exactly the same as in this PR.

To me it seems like this is an obvious problem with how either jpackage or another mechanism of the bundling process chooses to setup the order in which libraries are looked for.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Component - Build Build system improvement

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants