perf(cli): resolve the dependency graph once per command - #743
Merged
Conversation
A command resolves at several points — local cache sync, lockfile write, requirements check, `mops sources` — and each re-walked the whole graph from scratch, re-reading mops.toml and mops.lock every time. On a cold or stale-lock walk that also re-downloads packages purely to read their manifests. The walk is now memoized in-process, keyed on the *content* of mops.toml and mops.lock plus the root dir, conflict policy and MOPS_ENV. Content rather than path matters because commands rewrite their own inputs mid-run: checkIntegrity writes mops.lock, add/remove/update write mops.toml. Local `path` dependency manifests are live directories and are not covered by either file, so every one the walk reads is recorded and re-hashed on a hit — including ones that did not exist, so creating a manifest invalidates too. Registry package manifests are deliberately outside the key: published versions are immutable, which is the same assumption the lockfile's `graph` carry-over already relies on. Also extracts the ~85-line cross-major conflict report out of the walk, which leaves resolveUncached doing one thing, and deep-copies the graph on the way out so a caller mutating an inner edge map cannot write into the memo. reportedConflicts survives, with a corrected comment. It no longer guards against repeated identical resolves — the memo does that — but against the narrower case where inputs legitimately change mid-command, producing a new key and a second walk. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Contributor
Cursor AI review👍 APPROVE — looks safe to merge
VerdictDecision: APPROVE Generated for commit 65b0439 |
automation-sa-sre
approved these changes
Aug 12, 2026
automation-sa-sre
left a comment
There was a problem hiding this comment.
Automated approval: the AI review verdict for 65b0439 is APPROVE. See the "Cursor AI review" comment for details.
This was referenced Aug 13, 2026
Kamirus
added a commit
that referenced
this pull request
Aug 13, 2026
…746) [#743](#743) shipped the resolve memo with one bounded caveat, stated in its PR body: a caller arriving while a walk is in flight shares the pending promise without re-checking local `path` dep manifests — the one input the content key cannot see. That was unreachable while nothing resolved concurrently, but [#745](#745) landed real install concurrency in the same codebase, so the assumption is now one refactor away from breaking silently. This closes it. Every memo hit now takes the same path: settle the cached walk first, verify the local manifests it read after. A mismatch — an edit that landed after the walk read a manifest — falls through to a fresh walk instead of serving the stale read. Sharing an in-flight walk is thereby safe by construction, and the `localInputs: Map | null` in-flight sentinel disappears: the map is complete exactly when the promise settles, which is exactly when anyone reads it. ## How the test pins the interleaving No timers, no tick counting. Two facts make it deterministic: - the walk reads the **first** local dep manifest synchronously, before its first suspension point, so an edit issued right after the un-awaited call is always post-read; - the second call is made in the same tick, so it always finds the walk still in flight and shares its promise. The first caller must resolve without the edit (`deps.c` undefined) and the sharer must detect it on settle and re-walk (`deps.c` present). Mutation-checked: against the pre-fix memo, exactly this test fails. ## What is unchanged Rejection semantics: a failed walk still clears the slot and is never served to a later caller, and a concurrent sharer still sees the rejection. The pre-existing single-walk TOCTOU — an edit landing mid-walk gives *that* walk a torn view — is inherent to any one-pass resolve and out of scope; what this fixes is the memo *re-serving* such a read to another caller. `resolve`, `resolve-memo`, `cache-resilience`, `locked`, `local-path-lock`, `local-dep-manifest-lock`, `sync-plan` and `requirements` suites: 83/83. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Kamirus
added a commit
that referenced
this pull request
Aug 13, 2026
Establishes the manual tracking convention for the install benchmark from [#744](#744): the most recent meaningful run lives in `perf/install-bench/RESULTS.md`, replaced wholesale on each re-run, with a one-line history of prior headline numbers. Deliberately not a CI gate — runs are only comparable within one host and network, so a committed run is a point of reference. ## The run itself v2.22.0 vs v3 at f8c619c — the first measurement with the whole performance stack landed (#741 sorted lock, #742 update alignment, #743 resolve memo, #745 parallel installs, #746 in-flight re-check). Medians of 3 against the live registry: | scenario | v2 | v3 | v3 vs v2 | | --- | --- | --- | --- | | install-cold-nolock | 139.6s | 107.4s | 0.77x | | install-cold-validlock | 106.6s | 61.1s | 0.57x | | install-cold-stalelock | 128.0s | 95.7s | 0.75x | | install-warm-nolock | 3.20s | 5.62s | 1.76x † | | install-warm-validlock | 6.20s | 2.99s | 0.48x † | | install-warm-stalelock | 3.31s | 2.45s | 0.74x | | add-two | 4.00s | 3.84s | 0.96x | | update-few | 8.02s | 5.42s | 0.68x | | update-all | 14.9s | 2.87s | 0.19x | † transient registry-latency window, not code: v2's own warm numbers tripled against every prior run of the same released binary. A warm-only re-check half an hour later returned to line — v3 warm-nolock 2.33s (0.75x), warm-validlock 1.08s (0.42x) — and is recorded in RESULTS.md as the representative warm numbers, with the main table kept unedited. **v3 now beats released v2 in every scenario.** The cold no-lock install — v3's one regression before #745 (1.23–1.25x in both pre-parallel baselines) — is 0.77x, and CI-shaped installs (cold + committed lock) are 43% faster. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A single command resolves the dependency graph at several points — local cache sync, lockfile write, requirements check,
mops sources— and each one re-walked the whole graph from scratch, re-readingmops.tomlandmops.lockevery time. On a cold or stale-lock walk that also means re-downloading packages purely to read their manifests.The walk is now memoized in-process.
Why the key is content, not paths
Commands rewrite their own inputs mid-run:
checkIntegritywritesmops.lock, andadd/remove/updatewritemops.toml. A path-keyed memo would happily serve the pre-write graph afterwards. So the key is the content of both files, plus the root dir, the conflict policy andMOPS_ENV.Local
pathdependencies are the interesting case: they are live directories, and their manifests are in neither file. Every manifest the walk reads is therefore recorded and re-hashed on a hit — including ones that did not exist, recorded as""before the existence check, so creating a manifest where there was none also invalidates.Registry package manifests are deliberately outside the key. Published versions are immutable, so their content cannot change under us — the same assumption the lockfile's
graphcarry-over already depends on.How the invalidation was actually verified
A stale memo hit means a silently wrong dependency graph, which would be far worse than the redundant work it removes. So the tests don't assert on timing or call counts — they use a canary: a cached registry package's manifest is outside the key by design, so arming it to declare an extra dependency is invisible to a memo hit and shows up in
depsonly if a walk genuinely reran."canary" in depsis a direct hit/miss probe.Every case below was then mutation-tested — the relevant line was deliberately broken to confirm the test fails, so nothing passes vacuously:
mops.tomleditedmops.lockwritten mid-command{}lockMOPS_ENVchanged./local-dep→./staging-depsetConflictPolicy("ignore")Two properties are structural rather than tested: when the lock short-circuits, the result is a pure function of
mops.lock, which the key hashes, so an emptylocalInputsis correct rather than a gap. AndcheckLockFileLight()— hence the key — already depends on local manifests throughlocalDepsHash, giving local edits a second, independent invalidation path.One bounded caveat: while a walk is in flight, a concurrent caller shares the promise without re-checking local manifests. Nothing in the CLI resolves concurrently today, so it is unreachable, but it is the one place an edit landing mid-walk would be missed. Worth knowing before anyone parallelises resolution.
Cleanup, since this file was open anyway
The ~85-line cross-major conflict report moved out of the walk into its own function, which leaves the walk doing one thing. It moved character-for-character — every message string, the dependent ordering, the de-duplication and the
--conflicts errorexit — andresolve.test.tspasses unchanged, including its committed stdout/stderr snapshot.copyResultnow deep-copiesgraph's inner edge maps. It was shallow, so a caller mutatinggraph[pkgId][dep]would have written into the memo. No current caller does; the copy is cheap and closes the class.reportedConflictssurvives with a corrected comment. It no longer guards against repeated identical resolves — the memo does that now — but against the narrower case where inputs legitimately change mid-command. Leaving its old comment, which described the 3–5 passes this PR eliminates, would have actively misled the next reader. Whether conflict-reporting semantics should change at all is item 3, deliberately untouched here.Verification
resolve,cache-resilience,local-path-lock,local-dep-manifest-lockand the newresolve-memosuites: 36/36, snapshot matched. Broader sweep acrosslocked,path-dep-cycle,requirements,legacy-lock-flag,sync-plan,remove,remove-dry-runandoutdatedalso passes.github-dep-lock.test.tsfails locally withsocket hang upfetchinggithub.com/.../archive/<sha>.zip. That is the outage tracked as item 39 —github.comredirects tocodeload.github.com, and today the redirect host is failing while codeload returns 200. This PR never touches GitHub archives.Item 6 in #723.
🤖 Generated with Claude Code