Fix OverlappingFileLockException in Gradle composite builds#4507
Fix OverlappingFileLockException in Gradle composite builds#4507hczedik wants to merge 1 commit into
Conversation
|
Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA). View this failed invocation of the CLA check for more information. For the most up to date status, view the checks section at the bottom of the pull request. |
There was a problem hiding this comment.
Code Review
This pull request modifies LockFile.java to store the lockMap within System.getProperties() instead of a static field. This change ensures a single JVM-wide lock map across different classloaders in Gradle composite builds, addressing OverlappingFileLockException. The reviewer noted that using System.getProperties() for non-string values could cause ClassCastException if store() is called and highlighted potential thread-safety issues on older Java 8 versions, suggesting these risks be documented.
| private static final ConcurrentHashMap<Path, Lock> lockMap = | ||
| (ConcurrentHashMap<Path, Lock>) | ||
| System.getProperties() | ||
| .computeIfAbsent( | ||
| "jib.lockFile.lockMap", key -> new ConcurrentHashMap<Path, Lock>()); |
There was a problem hiding this comment.
Using System.getProperties() as a global registry is an effective workaround for cross-classloader state sharing in Gradle composite builds, but it introduces a few side effects:
Properties.store()side effect:java.util.Propertiesis designed forStringkeys and values. Callingstore()orsave()onSystem.getProperties()will now throw aClassCastExceptionbecause it iterates over all entries and expects values to be strings. Any other plugin or tool in the same JVM that attempts to persist system properties (e.g., for build reporting or environment capturing) will crash.- Thread Safety on older Java 8:
Hashtable(the parent ofProperties) did not overridecomputeIfAbsentuntil Java 8u40. On older updates, it uses a non-atomic default implementation from theMapinterface, which could theoretically lead to multiple map instances being created if multiple classloaders initialize this class simultaneously, defeating the purpose of the fix.
While these are likely acceptable risks given the constraints of Gradle's classloader isolation, it is worth noting these global side effects in the code comments.
e4c9c4e to
e4816a3
Compare
|
I have signed the Contributor License Agreement (CLA). @diegomarquezp I would appreciate if you or another of the maintainers could review this. Thank you! |
In Gradle composite builds where multiple included builds have independent buildSrc directories, each included build loads jib-core in its own classloader.
The static lockMap in LockFile becomes multiple separate instances (one per classloader) so the synchronization that prevents concurrent FileChannel.lock() calls on the same file doesn't work, causing OverlappingFileLockException.
My fix stores the shared lock map in System.getProperties(), which returns the same Hashtable instance regardless of which classloader calls it. This is admittedly a hack (there's no clean JVM API for cross-classloader shared state), but it's a pattern used by some other libraries facing classloader isolation (e.g. RESTHeart, also based on this SO answer). Only bootstrap-classloader types (Path, Lock) are stored, so cross-classloader casting is safe.
Reproduction:
A Gradle composite build with 8 included builds, each with its own buildSrc declaring the jib plugin, running jibBuildTar in parallel. With jib 3.5.3, this reliably triggers the exception (and builds hang). With this fix applied, all 8 tasks complete successfully across repeated runs.
jib-parallel-build-reproducer.zip
Fixes #3347