Skip to content

Dependency lockfile, provided-BOM, and shadowing report - #6825

Merged
ThuF merged 4 commits into
masterfrom
dependency-lockfile-6780
Aug 18, 2026
Merged

Dependency lockfile, provided-BOM, and shadowing report#6825
ThuF merged 4 commits into
masterfrom
dependency-lockfile-6780

Conversation

@ThuF

@ThuF ThuF commented Aug 18, 2026

Copy link
Copy Markdown
Contributor

Overview

Part 4 of the Dynamic dependency management epic (#6776) - the trust and reproducibility layer over parts 1-3:

  • Lockfile - every fully clean union resolution writes project-lock.json (instance-level, inside the resolved-modules directory, so baking that one directory into an image carries the whole reproducibility bundle): each activated artifact with its SHA-256, the requestedBy projects (roots) or the via root (transitives) and its scope, plus the version mediations with chosen/rejected versions and per-project attribution. Deterministically sorted, so two locks diff line by line. Before anything activates, every resolved artifact is verified against the lock - a checksum mismatch is a hard per-artifact failure (reported, evicted from the launch seed, not activated) while the verified rest keeps serving, and a failed pass never rewrites the lock, so a tampered artifact can never launder itself into the trusted set.
  • DIRIGIBLE_DEPENDENCIES_FROZEN=true - the mode immutable production images should run: the lockfile IS the resolution. The locked set activates checksum-verified from the local repository, no remote repository is ever consulted, no version is re-mediated, and a declaration the lock does not carry is rejected as frozen-mismatch with a pointed error. (DIRIGIBLE_DEPENDENCIES_LOCKFILE overrides the lockfile location.)
  • Provided-BOM - the application build generates a standard dependencyManagement POM (org.eclipse.dirigible:dirigible-provided-bom) from the build's own resolved runtime dependencies (the Spring Boot repackage input - never by unzipping the artifact) and embeds it as META-INF/dirigible-provided-bom.xml, so an air-gapped instance knows what the platform ships without any remote fetch, and a distribution building its own fat jar embeds its own inventory. The resolver treats every listed coordinate as provided: never downloaded, pruned from the graph with its subtree.
  • Shadowing report - a project declaring a groupId:artifactId the platform provides at a different version gets a WARN with both versions and a shadowed status on the endpoint and in the IDE view - the sharpest footgun of the whole mechanism is now impossible to hit silently. Child-first loading as a "fix" was deliberately rejected; the ProvidedBom javadoc records why (a visible report at resolution time beats invisible LinkageErrors of the same FQN at arbitrary call sites).
  • Per-artifact status report - GET /services/core/dependencies now carries a report with one accurate status per artifact: active | pending-restart | shadowed | mediated | failed | frozen-mismatch, plus declaredBy attribution, the frozen flag and the lockfile path.
  • IDE Dependencies view (Operations perspective, next to Artefacts) - read-only rendering of the report (coordinate, scope, declaring projects, status badge, details) plus a minimal Add Dependency… dialog that validates the coordinate (exact versions only) and writes the maven entry into the selected workspace project's project.json - the descriptor stays the API.

Worked example - the full trust chain

Declare (invoice-app/project.json in the registry):

{
    "guid": "invoice-app",
    "dependencies": [
        { "type": "maven", "id": "org.apache.poi:poi:5.2.5" },
        { "type": "maven", "id": "com.google.code.gson:gson:2.8.9" }
    ]
}

Resolve (POST /services/core/dependencies/resolve) - gson is shadowed (the platform ships 2.13.2), poi activates, and poi's other transitives (commons-codec, commons-io, commons-collections4, commons-math3, log4j-api) are pruned as platform-provided - only SparseBitSet is actually new:

  shadowed  com.google.code.gson:gson:2.8.9   requested: 2.8.9, provided: 2.13.2 - parent-first delegation serves the platform's version; the declared version is inert
    active  org.apache.poi:poi:5.2.5          Declared
    active  com.zaxxer:SparseBitSet:1.3       Via org.apache.poi:poi:5.2.5

The lockfile (resolved-modules/project-lock.json) and its diff after upgrading poi to 5.4.0 - two lines of substance, reviewable at a glance:

   "artifacts": [
     {
       "id": "com.zaxxer:SparseBitSet:1.3",
       "sha256": "f76b85adb0c00721ae267b7cfde4da7f71d3121cc2160c9fc00c0c89f8c53c8a",
-      "via": "org.apache.poi:poi:5.2.5",
+      "via": "org.apache.poi:poi:5.4.0",
       "scope": "module"
     },
     {
-      "id": "org.apache.poi:poi:5.2.5",
-      "sha256": "352e1b44a5777af2df3d7dc408cda9f75f932d0e0125fa1a7d336a13c0a663a7",
+      "id": "org.apache.poi:poi:5.4.0",
+      "sha256": "ace71e79873059e273036674560b50c3d6b945b7ca168b0d4962ad7650ae1eec",
       "requestedBy": [
         "invoice-app"
       ],

Frozen-mode boot (DIRIGIBLE_DEPENDENCIES_FROZEN=true, with an extra org.commonmark:commonmark:0.24.0 declared that the lock does not carry) - the locked set activates checksum-verified, the shadowed declaration stays a report (the embedded BOM answers without any network), and the new coordinate is rejected without being downloaded:

[WARN ] FrozenResolution - Declared [com.google.code.gson:gson] is SHADOWED: requested [2.8.9], the platform provides [2.13.2] - parent-first delegation serves the platform's version
[ERROR] FrozenResolution - Frozen-mode mismatch: Declared as [org.commonmark:commonmark:0.24.0] but not part of [project-lock.json] - frozen mode (DIRIGIBLE_DEPENDENCIES_FROZEN=true) activates the locked set only. Unfreeze the instance or rebuild the lock with a dynamic resolution to add or change dependencies.
[INFO ] DependencySynchronizer - Dependency layer swapped to generation [1]: added [com.zaxxer:SparseBitSet:1.3, org.apache.poi:poi:5.4.0], removed []
[INFO ] DependenciesService - Maven dependency frozen activation completed: [3] declared, [2] jar(s) activated, [0] platform-scoped, [0] mediated, [1] failure(s), classloader generation [1]

The Dependencies view (Operations perspective) rendering the same state with the shadowed badge:

dependencies-view

Tests

  • Unit: LockfileRoundTripTest (round trip, deterministic serialization, a flipped byte fails verification naming the coordinate), ProvidedBomTest (provided satisfied without a download, different version shadowed with both versions, provided transitive pruned, generator/parser round trip), FrozenModeTest (locked set resolves repository-free, new coordinate rejected with a pointed error, provided declarations are never frozen mismatches).
  • Integration: DependencyLockfileIT - clean resolution writes the lock; a tampered local-repository jar fails verification while the rest keeps serving and the lock is not rewritten; frozen mode rejects an unlocked coordinate without downloading it; a platform-provided coordinate declared at another version is shadowed and the platform's own class provably serves (marker-resource probe). LauncherAgentDeliveryIT additionally pins the embedded BOM inside the executable jar.
  • All part 1-3 ITs (DependencyResolutionIT, DynamicDependenciesIT, PlatformScopeDependencyIT) stay green with the verification pipeline and the real BOM live.

Fixes: #6780.

if (lockfile.isEmpty()) {
String message = "DIRIGIBLE_DEPENDENCIES_FROZEN=true but there is no lockfile at [" + lockfileStore.path()
+ "] - nothing is activated in frozen mode without a lock. Run one dynamic resolution to produce it.";
LOGGER.error("Frozen-mode activation failed: {}", message);
if (!Files.isRegularFile(jar)) {
failures.put(artifact.id(),
"The locked artifact [" + artifact.id() + "] is missing from the local repository [" + jar + "] - not activated");
LOGGER.error("Frozen-mode activation failed: the locked artifact [{}] is missing from [{}]", artifact.id(), jar);
if (!actual.equals(artifact.sha256())) {
failures.put(artifact.id(), "Checksum mismatch for the locked artifact [" + artifact.id() + "]: expected ["
+ artifact.sha256() + "], found [" + actual + "] - not activated");
LOGGER.error("Frozen-mode activation failed: checksum mismatch for [{}] at [{}]", artifact.id(), jar);
}
} catch (IOException e) {
failures.put(artifact.id(), "The locked artifact [" + artifact.id() + "] is unreadable: " + e.getMessage());
LOGGER.error("Frozen-mode activation failed: the locked artifact [{}] at [{}] is unreadable", artifact.id(), jar, e);
Lockfile lockfile = GSON.fromJson(Files.readString(path, StandardCharsets.UTF_8), Lockfile.class);
return Optional.ofNullable(lockfile);
} catch (IOException | JsonParseException e) {
LOGGER.error("The lockfile [{}] is unreadable - it is ignored until the next clean resolution rewrites it", path, e);
Comment on lines +112 to +115
LOGGER.info("Lockfile [{}] written: [{}] artifact(s), [{}] mediation(s)", path, lockfile.artifacts()
.size(),
lockfile.mediated()
.size());
Path link = directory().resolve(linkName(localRepository, artifact));
try {
if (Files.deleteIfExists(link)) {
LOGGER.warn("Evicted [{}] from the resolved-modules directory - its integrity verification failed", link.getFileName());
LOGGER.warn("Evicted [{}] from the resolved-modules directory - its integrity verification failed", link.getFileName());
}
} catch (IOException e) {
LOGGER.warn("Could not evict the dependency jar [{}]", link, e);
@ThuF
ThuF merged commit 43785bf into master Aug 18, 2026
10 checks passed
@ThuF
ThuF deleted the dependency-lockfile-6780 branch August 18, 2026 17:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Dependency lockfile, provided-BOM, and shadowing report

2 participants