v3 CLI backlog: Cargo/pnpm parity

[Kamirus]

v3 CLI backlog: Cargo/pnpm parity

Open work from an audit of the v3 CLI against Cargo and pnpm — resolution, lockfile, cache/.mops
layout, integrity, and install latency.

Numbers are stable IDs: merged PRs cite them, so they are never reused. Gaps are shipped items,
listed at the bottom. Everything is CLI-side unless marked [registry].

Nothing here blocks 3.0.0 GA.

---

Resolution

2. 0.x minor skew is never reported

Conflict reporting buckets by semver major, so base 0.7.3 vs 0.16.0 both bucket to 0 — no
warning, and --conflicts error cannot fire. Cargo treats 0.MINOR as the compatibility unit, and
most of the Motoko ecosystem lives at 0.x. cli/resolve-packages.ts, the majors set.
Parked: a cross-version conflict cannot actually be resolved today without moc --override
(#283), so louder reporting gives noise without a proportional action.

3. Conflict reporting is inert once mops.lock is fresh

Resolution short-circuits to lock.deps before the reporting block runs, so warnings appear exactly
once — on the install that created the lock — and never again. For CI checking out a committed lock,
reporting is dead. Fix: re-derive the version sightings from lock.graph at the short-circuit.
Related to #683. Parked: same reason as 2.

5. Mixed-kind conflicts are silently type-biased

When a registry version meets a GitHub ref for the same name, compareVersions coerces the missing
version to 0.0.0, so one kind always beats the other by arrival order, with no diagnostic.

---

Lockfile and integrity

7. Cache hits are never re-verified

A package already in the global cache skips download, and therefore verification, so a cache
poisoned after the fact (restored CI cache, another tool, bit rot) reaches .mops/ unchecked.
mops verify audits .mops/, never the cache. Wanted: an offline mops cache verify (see 13), and
a decision on whether the cache → .mops/ copy should hash.

7b. {MOPS_ENV} lockfiles are not portable

The lock stores expanded paths, so it satisfies --locked only under the MOPS_ENV that generated
it; multi-environment CI needs one lock per environment. Storing the raw value needs expansion moved
into sources.ts, sync-local-cache.ts and install-local-dep.ts, which consume resolved paths.
Parked: the placeholder is barely used, and the lock is already correct per environment.

9. The lockfile does not record which registry it came from

No equivalent of Cargo's source = "registry+…", so a lock is valid against whatever
MOPS_REGISTRY_CANISTER_ID points at. Parked: only the main registry is supported, so the risk
is theoretical.

---

Store and cache

11. No content-addressable store or hardlinks

.mops/ is a full byte copy of the global cache (ncp), so every file is written twice and K
projects hold K+1 copies of every shared dependency. The registry's per-file sha256 is already the
natural CAS key. Biggest remaining structural win, and CLI-only. Interacts with #716.

12. No cache eviction, pruning or GC

mops cache is size / clean / show only — no LRU, no age sweep, no refcounting, no
pnpm store prune equivalent. Nothing prunes stale .mops/<pkg>@<ver> on a version change either.

13. No offline mops cache verify

mops verify covers .mops/ and needs the network. A pnpm store status equivalent that re-hashes
cache entries against the lockfile offline would close 7.

14. No cross-process locking

Staging + atomic rename makes concurrent installs safe, but two processes downloading the same cold
package duplicate all the work. Cargo takes a file lock on its package cache and makes the second
process wait. Two narrow races remain: isDepCached's repair rmSync can delete a directory another
process is copying, and the Windows fallback in writeLockFileAtomic is a plain non-atomic write.

34. GitHub cache entries are keyed by ref, not resolved commit

Two projects pinning different commits of the same #main thrash one cache directory. Content is
still correct and verified — this is wasted downloads. Keying by …#<ref>@<resolved> fixes it but
changes mops sources output paths, so it needs cache.ts and mops.ts together.

---

Install performance

16. No registry metadata cache

No Cargo-index or npm-_cacache equivalent: getHighestVersion has no memo and
getHighestSemverBatch results are never persisted, so a repeated mops update re-asks every time.
A small TTL cache under the global cache dir, keyed by network, with an --offline bypass.

15. No --offline / --prefer-offline / --frozen

Nothing anywhere, and even --locked makes a registry call. Cache hits already need zero network, so
this is mostly threading a flag to the fetch-on-miss sites so they fail loudly. --frozen =
--locked --offline follows.

---

Command surface

18. mops why and mops tree

Both are pure presentation now — lock.graph plus the resolver's {deps, graph} has the data, all
offline.

19. mops licenses

Every cached package's mops.toml already carries license, and mops info already prints it for
one package.

21. mops.toml writing destroys comments and reorders keys

writeConfig is a value-only parse → re-stringify, so every comment is lost on add/remove/
bump/toolchain use/init, and scalar keys get hoisted above sub-tables. Cargo uses toml_edit
for surgical edits.

33. No CLI way to create a pinned alias

mops add core@1.0.0 updates an alias that already exists but deliberately will not create one —
the rule that would ("full semver + base declared elsewhere → write the alias key") turns every
ordinary version bump into a second declaration. So the documented two-version layout must be
hand-written. Cargo-shaped fix: mops add core@1.0.0 --pin.

---

CI and infrastructure

38. CI caches package tarballs, not the binaries postinstall downloads

setup-node's cache: "npm" caches ~/.npm; npm ci still runs postinstall scripts, and
wasm-pack and @dfinity/pic each fetch a binary from GitHub releases every run. A GitHub blip
therefore fails every job on every PR before any repo code runs — observed twice.
Options: retry the install step (cheapest, no staleness risk); cache node_modules keyed on the
lockfile + node major + runner image, which removes both downloads since the binaries live there;
cache wasm-pack's own tool dir for the Binaryen/wasm-opt fetch, which happens at build time.
An implementation was opened as #740 and closed unmerged.

39. GitHub archives are fetched through a redirect that can fail on its own

install-from-github.ts builds https://github.com/<org>/<repo>/archive/<ref>.zip, but github.com
only redirects to codeload.github.com. During an outage the redirect host returned HTTP 000 while
codeload returned 200 — every GitHub-dependency test failed for a hop mops need not make. Fetching
codeload directly removes it. Check first whether the redirect carries anything needed (private-repo
auth, ref forms only github.com resolves), and pair with a download retry.

---

Registry / canister changes [registry]

23. A certified hash endpoint — not a plain query

getFileHashesByPackageIds is public shared, so it pays consensus. It is now reached only when
mops.lock cannot answer, plus --locked and mops verify. Do not simply make it a query: a
query reply is signed by one node, package bytes are already query-served, and that combination lets
a single malicious node forge both. An earlier revision did exactly this and produced three integrity
holes. The legitimate version is a certified-data tree over hashByFileId, verified client-side —
consensus-equivalent trust at query latency. Note getFileHashesQuery exists and is intentionally
unused; a test fails if anything calls it.

24. Batch install-metadata query

getPackageFilesInfo(vec PackageId) -> (storage principal, fileIds, chunkCounts) query would
collapse 2P + ΣF calls into one per install.

25. getPackageDetails is oversized for the install path

The install consumes one field (publication.storage) but the reply carries version history with
benchmarks, dependents, download trends, owners. Interim: getPackageStorage(name, version) query.

26. getHighestSemverBatch traps at 100+ direct dependencies

assert(list.size() < 100) takes down mops outdated and mops update. Paginate or return an
error instead of trapping.

27. Dependency-index query

Resolving an uncached transitive version currently downloads the whole package to read one
mops.toml. Even getPackageDependencies(vec PackageId) query removes that, and it is the
prerequisite for any real constraint solver. Tracked as #19.

28. Semver ranges

Needs backend validator changes, the publish-time dep-exists check, an apiVersion major bump and an
ecosystem forward-compat window, since older CLIs cannot parse a spec string in a published
mops.toml. Deferred past v3 by decision; phased plan in TODO.md.

29. Tarball-per-package storage

Collapses the per-file fan-out and gives one integrity hash per package. Subsumes much of 11, 24 and
#716.

30. Publisher signatures

Hashes are computed by the registry canister itself, so registry compromise is total — there is no
trust anchor independent of it. Sigstore-style signing needs new canister state and API.

---

Decisions already taken

• mops update keeps its name. Under exact pins only one update operation can exist, so there is
  no update-vs-upgrade ambiguity to resolve, and renaming is breaking. If ranges land (28), that
  is when mops upgrade gets added alongside a redefined mops update. The remedy — saying in
  --help and the docs that it rewrites mops.toml — shipped in fix(cli): align mops update with outdated, and say it rewrites mops.toml #742.
• No lockfile format bump is needed. Validation never inspects key order or unknown fields, so
  additive optional fields and sorted output both ship without touching CURRENT_LOCK_VERSION (3).
• mops.lock: one checksum per package instead of per-file hashes #716 (one checksum per package instead of per-file) stays low priority on its own; fold it into
  11 or 29.
• HTTP/2 for registry requests: measured and not adopted (the "fewer connections" strand of 31).
  The boundary nodes do negotiate h2, but undici only multiplexes when the connection pool is capped,
  and capping is what costs: 16 concurrent requests against icp-api.io took 65–130 ms over 16
  parallel h1 connections, 68–98 ms over an uncapped h2 pool (still 16 sockets — no gain), and
  184–228 ms multiplexed over connections: 1. The cold-pool TLS handshakes h2 was meant to
  eliminate run in parallel, so 16 of them cost ~1 RTT of wall time, not 16. The connection-pressure
  case h2 would help is the one the fd-aware budget and halved-budget retry (feat(cli): retry installs on transient network failures, fd-aware request budget #747) already cover, and
  adopting it would mean bundling undici into the CLI. Re-measure if boundary-node behavior changes;
  numbers and method in perf(cli): overlap the API compatibility check with the install #748.

---

Shipped

#	What	PR
1	Install the closure of winning versions only	#735
4	Local path dependency cycle no longer overflows the stack	#739
6	Resolve the dependency graph once per command	#743
8	mops.lock written with sorted keys	#741
10	GitHub dependency commits and content hashes recorded and verified	#737
17	mops outdated exit codes, and GitHub deps reported	#734
20	add/remove section handling, org/repo shorthand, pinned aliases	#736
32	jest no longer resolves .did.js to the Candid file	#738
35–37	mops update aligned with outdated, shared GitHub rule, --help	#742
31 (part)	Parallel package installs, --concurrency / MOPS_CONCURRENCY, CPU-derived budget	#745
31 (part)	Transient network failures retry with the budget halved; budget capped by the fd soft limit	#747
31 (done)	checkApiCompatibility() overlapped with the install; h2 measured and declined (see decisions)	#748
—	mops sync alias data loss, remove --dry-run, cache clean on Windows	#725
—	Agent time-sync, blocking install telemetry, chained per-file round trip	#726
—	Two P0 lock-staleness bugs; mops.lock as the verification reference	#727