Skip to content

Fix OverlappingFileLockException in Gradle composite builds#4507

Open
hczedik wants to merge 1 commit into
GoogleContainerTools:masterfrom
hczedik:fix-lockfile-classloader
Open

Fix OverlappingFileLockException in Gradle composite builds#4507
hczedik wants to merge 1 commit into
GoogleContainerTools:masterfrom
hczedik:fix-lockfile-classloader

Conversation

@hczedik
Copy link
Copy Markdown

@hczedik hczedik commented May 23, 2026

In Gradle composite builds where multiple included builds have independent buildSrc directories, each included build loads jib-core in its own classloader.
The static lockMap in LockFile becomes multiple separate instances (one per classloader) so the synchronization that prevents concurrent FileChannel.lock() calls on the same file doesn't work, causing OverlappingFileLockException.

My fix stores the shared lock map in System.getProperties(), which returns the same Hashtable instance regardless of which classloader calls it. This is admittedly a hack (there's no clean JVM API for cross-classloader shared state), but it's a pattern used by some other libraries facing classloader isolation (e.g. RESTHeart, also based on this SO answer). Only bootstrap-classloader types (Path, Lock) are stored, so cross-classloader casting is safe.

Reproduction:
A Gradle composite build with 8 included builds, each with its own buildSrc declaring the jib plugin, running jibBuildTar in parallel. With jib 3.5.3, this reliably triggers the exception (and builds hang). With this fix applied, all 8 tasks complete successfully across repeated runs.
jib-parallel-build-reproducer.zip

Fixes #3347

@google-cla
Copy link
Copy Markdown

google-cla Bot commented May 23, 2026

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request modifies LockFile.java to store the lockMap within System.getProperties() instead of a static field. This change ensures a single JVM-wide lock map across different classloaders in Gradle composite builds, addressing OverlappingFileLockException. The reviewer noted that using System.getProperties() for non-string values could cause ClassCastException if store() is called and highlighted potential thread-safety issues on older Java 8 versions, suggesting these risks be documented.

Comment on lines +42 to +46
private static final ConcurrentHashMap<Path, Lock> lockMap =
(ConcurrentHashMap<Path, Lock>)
System.getProperties()
.computeIfAbsent(
"jib.lockFile.lockMap", key -> new ConcurrentHashMap<Path, Lock>());
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Using System.getProperties() as a global registry is an effective workaround for cross-classloader state sharing in Gradle composite builds, but it introduces a few side effects:

  1. Properties.store() side effect: java.util.Properties is designed for String keys and values. Calling store() or save() on System.getProperties() will now throw a ClassCastException because it iterates over all entries and expects values to be strings. Any other plugin or tool in the same JVM that attempts to persist system properties (e.g., for build reporting or environment capturing) will crash.
  2. Thread Safety on older Java 8: Hashtable (the parent of Properties) did not override computeIfAbsent until Java 8u40. On older updates, it uses a non-atomic default implementation from the Map interface, which could theoretically lead to multiple map instances being created if multiple classloaders initialize this class simultaneously, defeating the purpose of the fix.

While these are likely acceptable risks given the constraints of Gradle's classloader isolation, it is worth noting these global side effects in the code comments.

@hczedik hczedik force-pushed the fix-lockfile-classloader branch from e4c9c4e to e4816a3 Compare May 23, 2026 10:56
@hczedik
Copy link
Copy Markdown
Author

hczedik commented May 23, 2026

I have signed the Contributor License Agreement (CLA).

@diegomarquezp I would appreciate if you or another of the maintainers could review this. Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

jibDockerBuild build randomly fails in multi-project setup with parrallel flag

1 participant