Dependency lockfile, provided-BOM, and shadowing report - #6825
Merged
Conversation
| if (lockfile.isEmpty()) { | ||
| String message = "DIRIGIBLE_DEPENDENCIES_FROZEN=true but there is no lockfile at [" + lockfileStore.path() | ||
| + "] - nothing is activated in frozen mode without a lock. Run one dynamic resolution to produce it."; | ||
| LOGGER.error("Frozen-mode activation failed: {}", message); |
| if (!Files.isRegularFile(jar)) { | ||
| failures.put(artifact.id(), | ||
| "The locked artifact [" + artifact.id() + "] is missing from the local repository [" + jar + "] - not activated"); | ||
| LOGGER.error("Frozen-mode activation failed: the locked artifact [{}] is missing from [{}]", artifact.id(), jar); |
| if (!actual.equals(artifact.sha256())) { | ||
| failures.put(artifact.id(), "Checksum mismatch for the locked artifact [" + artifact.id() + "]: expected [" | ||
| + artifact.sha256() + "], found [" + actual + "] - not activated"); | ||
| LOGGER.error("Frozen-mode activation failed: checksum mismatch for [{}] at [{}]", artifact.id(), jar); |
| } | ||
| } catch (IOException e) { | ||
| failures.put(artifact.id(), "The locked artifact [" + artifact.id() + "] is unreadable: " + e.getMessage()); | ||
| LOGGER.error("Frozen-mode activation failed: the locked artifact [{}] at [{}] is unreadable", artifact.id(), jar, e); |
| Lockfile lockfile = GSON.fromJson(Files.readString(path, StandardCharsets.UTF_8), Lockfile.class); | ||
| return Optional.ofNullable(lockfile); | ||
| } catch (IOException | JsonParseException e) { | ||
| LOGGER.error("The lockfile [{}] is unreadable - it is ignored until the next clean resolution rewrites it", path, e); |
Comment on lines
+112
to
+115
| LOGGER.info("Lockfile [{}] written: [{}] artifact(s), [{}] mediation(s)", path, lockfile.artifacts() | ||
| .size(), | ||
| lockfile.mediated() | ||
| .size()); |
| Path link = directory().resolve(linkName(localRepository, artifact)); | ||
| try { | ||
| if (Files.deleteIfExists(link)) { | ||
| LOGGER.warn("Evicted [{}] from the resolved-modules directory - its integrity verification failed", link.getFileName()); |
| LOGGER.warn("Evicted [{}] from the resolved-modules directory - its integrity verification failed", link.getFileName()); | ||
| } | ||
| } catch (IOException e) { | ||
| LOGGER.warn("Could not evict the dependency jar [{}]", link, e); |
…and-line limit is never hit
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Overview
Part 4 of the Dynamic dependency management epic (#6776) - the trust and reproducibility layer over parts 1-3:
project-lock.json(instance-level, inside the resolved-modules directory, so baking that one directory into an image carries the whole reproducibility bundle): each activated artifact with its SHA-256, therequestedByprojects (roots) or theviaroot (transitives) and its scope, plus the version mediations with chosen/rejected versions and per-project attribution. Deterministically sorted, so two locks diff line by line. Before anything activates, every resolved artifact is verified against the lock - a checksum mismatch is a hard per-artifact failure (reported, evicted from the launch seed, not activated) while the verified rest keeps serving, and a failed pass never rewrites the lock, so a tampered artifact can never launder itself into the trusted set.DIRIGIBLE_DEPENDENCIES_FROZEN=true- the mode immutable production images should run: the lockfile IS the resolution. The locked set activates checksum-verified from the local repository, no remote repository is ever consulted, no version is re-mediated, and a declaration the lock does not carry is rejected asfrozen-mismatchwith a pointed error. (DIRIGIBLE_DEPENDENCIES_LOCKFILEoverrides the lockfile location.)dependencyManagementPOM (org.eclipse.dirigible:dirigible-provided-bom) from the build's own resolved runtime dependencies (the Spring Boot repackage input - never by unzipping the artifact) and embeds it asMETA-INF/dirigible-provided-bom.xml, so an air-gapped instance knows what the platform ships without any remote fetch, and a distribution building its own fat jar embeds its own inventory. The resolver treats every listed coordinate as provided: never downloaded, pruned from the graph with its subtree.groupId:artifactIdthe platform provides at a different version gets a WARN with both versions and ashadowedstatus on the endpoint and in the IDE view - the sharpest footgun of the whole mechanism is now impossible to hit silently. Child-first loading as a "fix" was deliberately rejected; theProvidedBomjavadoc records why (a visible report at resolution time beats invisibleLinkageErrors of the same FQN at arbitrary call sites).GET /services/core/dependenciesnow carries areportwith one accurate status per artifact:active | pending-restart | shadowed | mediated | failed | frozen-mismatch, plusdeclaredByattribution, thefrozenflag and the lockfile path.mavenentry into the selected workspace project'sproject.json- the descriptor stays the API.Worked example - the full trust chain
Declare (
invoice-app/project.jsonin the registry):{ "guid": "invoice-app", "dependencies": [ { "type": "maven", "id": "org.apache.poi:poi:5.2.5" }, { "type": "maven", "id": "com.google.code.gson:gson:2.8.9" } ] }Resolve (
POST /services/core/dependencies/resolve) - gson is shadowed (the platform ships 2.13.2), poi activates, and poi's other transitives (commons-codec, commons-io, commons-collections4, commons-math3, log4j-api) are pruned as platform-provided - only SparseBitSet is actually new:The lockfile (
resolved-modules/project-lock.json) and its diff after upgrading poi to 5.4.0 - two lines of substance, reviewable at a glance:"artifacts": [ { "id": "com.zaxxer:SparseBitSet:1.3", "sha256": "f76b85adb0c00721ae267b7cfde4da7f71d3121cc2160c9fc00c0c89f8c53c8a", - "via": "org.apache.poi:poi:5.2.5", + "via": "org.apache.poi:poi:5.4.0", "scope": "module" }, { - "id": "org.apache.poi:poi:5.2.5", - "sha256": "352e1b44a5777af2df3d7dc408cda9f75f932d0e0125fa1a7d336a13c0a663a7", + "id": "org.apache.poi:poi:5.4.0", + "sha256": "ace71e79873059e273036674560b50c3d6b945b7ca168b0d4962ad7650ae1eec", "requestedBy": [ "invoice-app" ],Frozen-mode boot (
DIRIGIBLE_DEPENDENCIES_FROZEN=true, with an extraorg.commonmark:commonmark:0.24.0declared that the lock does not carry) - the locked set activates checksum-verified, the shadowed declaration stays a report (the embedded BOM answers without any network), and the new coordinate is rejected without being downloaded:The Dependencies view (Operations perspective) rendering the same state with the
shadowedbadge:Tests
LockfileRoundTripTest(round trip, deterministic serialization, a flipped byte fails verification naming the coordinate),ProvidedBomTest(provided satisfied without a download, different version shadowed with both versions, provided transitive pruned, generator/parser round trip),FrozenModeTest(locked set resolves repository-free, new coordinate rejected with a pointed error, provided declarations are never frozen mismatches).DependencyLockfileIT- clean resolution writes the lock; a tampered local-repository jar fails verification while the rest keeps serving and the lock is not rewritten; frozen mode rejects an unlocked coordinate without downloading it; a platform-provided coordinate declared at another version isshadowedand the platform's own class provably serves (marker-resource probe).LauncherAgentDeliveryITadditionally pins the embedded BOM inside the executable jar.DependencyResolutionIT,DynamicDependenciesIT,PlatformScopeDependencyIT) stay green with the verification pipeline and the real BOM live.Fixes: #6780.