feat(cli)!: adopt the cargo lock model — --locked, self-healing install, download-time verification

[Kamirus]

CI pipelines get a real frozen-lockfile flag, and everyone else stops paying for a check that was re-reading the whole dependency tree on every command.

mops had three lockfile modes behind --lock <check|update|ignore>, plus a fourth that appeared only when the CI environment variable happened to be set. This replaces all of it with the model cargo, pnpm and yarn converged on: two modes, one flag. Plain commands are the dev flow and always keep the lock correct. --locked is the CI flow — it requires an up-to-date lock and never writes one.

Integrity moves with it. Verification now happens on the bytes as they are downloaded, so a corrupted or tampered download can never enter the cache, and installs no longer re-hash .mops/. The on-disk audit becomes an explicit command, mops verify.

Closes #516. Closes #517.

Before / After

A missing lock in CI. Before, mops test would silently resolve, download and write a lockfile nobody reviewed. After:

$ mops test --locked
Error: mops.lock is missing, but --locked was passed.
Run `mops install` to generate it, then commit mops.lock.

It fails before downloading anything.

Someone edited mops.toml and forgot to commit the lock.

$ mops install --locked
Error: mops.toml has changed since mops.lock was generated, but --locked was passed.
  Locked dependencies hash: 90a342a4a0b150765297bd0d09221c6338d3cadc509cda66b944f37fca569f1a
  Actual dependencies hash: 075072533858da89ffa178f5ee95ae566d21fb166294658ca4be9777750029f6
Run `mops install` (without --locked) to update mops.lock, then commit it.

Same repo, without --locked — the lock is simply brought up to date:

$ mops install
Packages installed

A hand-edited lock that no longer matches the manifest.

$ mops install --locked
Error: mops.lock does not match mops.toml, but --locked was passed.
  dependency core: mops.toml declares 1.0.0, mops.lock has 2.0.0
Run `mops install` (without --locked) to update mops.lock, then commit it.

A lock whose recorded hashes disagree with the registry. Note the hint differs here: plain mops install does not rewrite this lock, so pointing at it would loop (see below).

$ mops install --locked
Error: mops.lock does not match the registry, but --locked was passed.
  core@1.0.0/LICENSE: locked bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb, registry 840e3d57a38a8061f55d04470fbd58b9345326fa04ea10ee42add5c6e3b2aa08
Restore mops.lock from version control, or delete it and run `mops install` to regenerate it.

A corrupt lock — the self-healing path. This used to require mops install --lock update, and #514 was about that flag not actually working:

$ mops install --locked
Error: mops.lock could not be parsed, but --locked was passed.
Restore mops.lock from version control, or delete it and run `mops install` to regenerate it.

$ mops install
Packages installed

No flag recovers a broken lock any more, because none is needed.

An up-to-date lock under --locked is left alone — byte-identical content and unchanged mtime, verified in cli/tests/locked.test.ts.

The guarantee change: install no longer verifies files already on disk

Neither mops install nor mops install --locked gates a modified .mops/ tree any more. mops verify is the replacement. Read this if any pipeline relies on the old behavior.

This is intentional and is the reason the item sits in a major. It is called out here, in cli/CHANGELOG.md, on the mops.lock docs page and on the new mops verify page, and it is pinned by a test in both cli/tests/locked.test.ts and cli/tests/build.test.ts.

Integrity is now checked once, at download time, against the hashes published in the registry, before the package is committed to the cache. The per-install re-hash of .mops/ is gone. Editing a dependency in place therefore no longer fails your next command:

$ echo "// oops" >> .mops/core@1.0.0/src/Array.mo
$ mops install
Packages installed

mops verify is the replacement, and it reports exactly what install used to:

$ mops verify
Integrity check failed
.mops/core@1.0.0/src/Array.mo does not match mops.lock
  Locked hash: 7cc453a2d20a3dae84c54f2c4a2db6db7b454f6fca0ba23e6440d324c2a68f3f
  Actual hash: 477f5f7cd24bf2a703faf22e42bc0ea3560428bcc84e8bd68ca08982fc88fcdd
  Delete the `.mops/core@1.0.0` directory and run `mops install` to restore it.

$ mops verify
Integrity verified 3 package(s), 130 file(s)

Two consequences worth stating plainly:

• Packages already sitting in the global cache from an older CLI were never download-verified. mops verify audits them; mops cache clean forces a verified re-download.
• Download-time verification is genuinely load-bearing, not decorative. Corrupting a single byte in transit is caught and the package never reaches the cache or .mops/:

$ mops install
Error: integrity check failed for core@1.0.0
  Hash mismatch for core@1.0.0/src/Array.mo
  Expected: 7cc453a2d20a3dae84c54f2c4a2db6db7b454f6fca0ba23e6440d324c2a68f3f
  Actual:   f414fe00390251acc52b7563aef3e2118afd893a9e0e97bb4e1a06f9d9c09a71

One case is deliberately not self-healed

A lockfile whose recorded file hash values are wrong — reachable only by hand-editing or a botched merge — is not repaired by mops install, because detecting it needs getFileHashesByPackageIds, which is an update call, not a query:

	median warm mops install
valid lock, no hash fetch	1.6 s
lock regenerated, one hash fetch	2.8 s

Paying ~1.2 s on every install to catch a hand-edit would cost roughly nine times the 136 ms this PR saves, and a warm cache fetches no hashes at download time either, so there is nothing already in hand to compare against. Since those values are read only by --locked and mops verify and never by the build, a wrong hash cannot produce a wrong build.

What I did instead: --locked and mops verify report it with the hint that actually recovers (restore from version control, or delete and reinstall) rather than "run mops install", which would have looped forever. Covered by a test that asserts the working hint and walks the recovery.

Two related inconsistencies that are detectable offline now self-heal, one of which was a real correctness hole: a deps entry disagreeing with mops.toml used to be installed as-is, silently giving you the wrong version. Plain mops install now re-resolves instead.

Migration

Old invocation	New invocation
mops install --lock check	mops install --locked
mops install --lock update	mops install
mops install --lock ignore	no successor — the lock is always maintained
mops add|remove|update|sync --lock update	mops add|remove|update|sync
mops add|remove|update|sync --lock ignore	no successor
CI=1 mops install (implicitly meant check)	mops install --locked
mops test in CI with no prior install	mops test --locked
relying on install to detect a modified .mops/	mops verify
mops.lock in a library's .gitignore	commit mops.lock

A typical CI job becomes:

- run: mops install --locked
- run: mops test --locked

--locked is on mops install and on every implicitly-installing command: build, check, check-candid, check-stable, test, bench, generate candid. mops sources deliberately has none — the dfx packtool invokes it in the middle of a build and machine-parses its stdout, so failing there is a poor place to report a stale lock. Put mops install --locked earlier in the pipeline. Its stdout and the lock's mtime are unchanged by this PR.

Dependency-mutating commands (add, remove, update, sync) have no --locked: their job is to change dependencies, so they always update the lock.

Lockfile commit guidance is now "everyone commits"

mops.lock created. used to print Libraries: add mops.lock to .gitignore. It now prints Commit this file., and the docs say the same. A library's lockfile cannot pin anything for its consumers — they resolve their own graph and write their own lock — while committing it makes the library's own CI reproducible. If you have mops.lock in .gitignore because of the old advice, remove it.

What --locked checks, and one thing it does not

--locked requires that the lock is present, parseable, the current format version, pins every dependency declared in mops.toml to the same value, has deps and hashes agreeing on the registry package set, and records a hash for every file that matches the registry. It never writes mops.lock.

It does not re-walk the dependency graph to byte-compare a freshly computed lock, which is what I originally implemented. That design does not work, and the reason is worth recording:

Installing from a lockfile passes ignoreTransitive: true, which is the whole point — it skips downloading the dependency versions that lost a version conflict. Those versions' mops.toml files are therefore never in the cache, so resolvePackages({skipLock: true}) throws on them. On a simulated fresh clone (committed lock, cold cache, a diamond where base@0.10.2 loses to base@0.14.9) the re-walk crashed:

Error: ENOENT: no such file or directory, open '.../packages/base@0.10.2/mops.toml'
    at readConfig (cli/mops.ts:265:17)
    at collectDeps (cli/resolve-packages.ts:144:24)

The only ways to make the re-walk sound are to install the losing versions too (giving up the lockfile's main performance benefit, in exactly the CI case --locked targets) or to fetch each losing candidate's manifest individually (a new feature, and a network call per candidate). Neither belongs here. The walk-free checks above cover the realistic drift instead, and published registry versions are immutable, so a transitive version cannot change underneath a lock. The residual gap is transitive changes reached through a local path dependency, which are live directories by design.

This is the same structural limitation the resolver-correctness work hit (#679): a valid lock short-circuits resolution, so a fresh clone never re-walks the graph and never sees the cross-major conflict report. It cannot be closed from the lockfile side either, for the reason above. Recording it here rather than leaving it to be rediscovered.

Performance

The removed re-hash was proportional to the whole dependency tree and paid on every install, build, check and test. Measured on an 842-file / 33.5 MB tree (20 packages):

	median warm mops install
before	1.75 s
after	1.59 s

Isolated, the re-hash itself was ~136 ms (5 runs, 134–149 ms) on a warm page cache and a local SSD. So this is an honest but modest constant-factor win at this tree size — Node startup dominates a warm install — and it scales linearly with tree size and degrades on cold page cache, container overlay filesystems and networked volumes. The primary motivation is the guarantee model; the speed-up is a secondary benefit, and I would not sell it as the headline.

Also in here

• Plain mops install now migrates a lock that still carries absolute local path entries written by a pre-2.19.2 CLI. That previously required an explicit mops install --lock update, which no longer exists, so checkLockFileLight() treats such a lock as stale.
• Fixed mops install running the dependency-conflict check after a failed install, which crashed with an unhandled ENOENT on manifests the failed install never wrote. Pre-existing, but the new download-verification failure path makes it easy to hit.
• Deleted cli/helpers/deprecate-ci-lock.ts (nothing else used it) and the dead checkRemote() export.
• A lockfile that is valid JSON but structurally wrong (missing or non-object deps/hashes) is treated as corrupt and self-healed, rather than crashing with an unhandled TypeError. Shape is now validated at the read boundary.
• checkLockFileLight (which decides whether to install from the lock) and the --locked gate are derived from one inspectLockFile function, so they cannot disagree. When they did, --locked accepted a lock that install then declined to install from, silently falling back to re-resolving mops.toml.
• Restored the mops check-candid --help description, dropped by accident when --locked was wired in.

Verification

Baseline on kamil-v3/hidden-state: 25 suites / 201 tests / 73 snapshots. Now 26 / 229 / 73, all green (the count includes tests arriving with the merged base). npm run check and npm run lint clean.

New tests in cli/tests/locked.test.ts (targeted assertions, no snapshots, per AGENTS.md) cover: missing lock, missing lock on each of build/check/check-stable/test/bench, up-to-date lock left byte- and mtime-identical, changed manifest, unparseable lock, legacy format, locked version disagreeing with mops.toml, locked hash disagreeing with the registry, and the install-tolerates / verify-rejects pair. mops verify gets happy path, missing lock and not-installed cases. They use their own fixtures because Jest runs test files in parallel and sharing install/success races on mops.lock.

Two existing tests changed intent deliberately:

• build.test.ts "fails on a locally modified .mops/ file" became "tolerates ... that mops verify rejects" — that is the guarantee change.
• local-path-lock.test.ts "--lock update rewrites absolute local paths" became "plain install rewrites ...".

Beyond the suite, I ran a 25-case matrix by hand against the live registry covering the same ground plus CI=1 no longer implying check, --lock being rejected on all five commands, --locked being accepted on all seven implicit commands, mops sources having none, and the fresh-clone cold-cache path that caught the design flaw above.

Stacked on #677, and synced onto its reworked head. Two conflicts, both resolved in favor of the newer decisions:

• cli/cli.ts — feat(cli)!: stricter flags and safer defaults for v3 #676 made unknown flags before -- strict by removing allowUnknownOption(true). Kept that; --locked is a declared option so strict parsing accepts it, and the -- <tool flags> passthrough still works (verified for check --nope, build --locked, test -- -Werror).
• cli/CHANGELOG.md — kept all twelve entries from feat(cli)!: eliminate hidden shared state (network commands, lock bypass, shell init, exit code) #677/feat(cli)!: stricter flags and safer defaults for v3 #676. Dropped this branch's network-relocation entry, superseded by feat(cli)!: eliminate hidden shared state (network commands, lock bypass, shell init, exit code) #677 deleting set-network/get-network outright. Kept this branch's version of the entry for the seven implicitly-resolving commands; feat(cli)!: eliminate hidden shared state (network commands, lock bypass, shell init, exit code) #677's still referenced the removed --lock update flag and the .mops/ tamper guarantee that download-time verification replaces.

cli/integrity.ts auto-merged to this branch's version unchanged — the defaultLock/CI-lock machinery it replaced left no residue.

Confirmed against the reworked cleanCache() (now plain whole-tree removal of both caches, with the PROJECT_STATE_FILES special case reverted): nothing in the lock or verify work depends on it. mops.lock lives at the project root, never inside .mops/, so a full mops cache clean leaves it byte-identical, after which install --locked re-downloads and re-verifies without writing the lock, and mops verify passes. .mops/ is pure derived state.

Retarget with gh pr edit 681 --base v3 once #677 merges.

[github-actions]

Cursor AI review

👀 HUMAN REVIEW REQUESTED — significant intended changes detected

Category	Assessment	Details
Summary	✅	Replaces --lock <check|update|ignore> and CI auto-check with cargo-style --locked / self-healing install, moves integrity to download-time verification, and adds mops verify for on-disk audits.
Code Quality	✅	Single inspectLockFile feeds both checkLockFileLight and checkLockedPrerequisites; dead deprecate-ci-lock / checkRemote removed; installAllOrExit reuses one failure path across implicit installers.
Consistency	✅	Sibling installers (build/check/check-stable/check-candid/test/bench/generate candid) all get --locked; sources correctly stays skip-only; docs, CHANGELOG ## Next, and mops-cli skill updated together.
Security	✅	Traced verifyDownloadedPackageFiles in install-mops-dep.ts before commitStagingDir; registry fileId ↔ fileMeta.path matches PackagePublisher.mo; cache-hit skip and removed .mops/ re-hash are intentional (see S1).
Tests	✅	locked.test.ts covers missing/stale/corrupt/hash-mismatch/verify paths; build.test.ts and local-path-lock.test.ts updated for the new guarantee; cli.test.ts asserts --lock rejection and CI=1 no longer checks.
Maintainability	✅	Lock policy enum, shared defect descriptors/hints, and NEXT-MAJOR strike-throughs make the v3 end-state explicit without speculative helpers.

Significant Changes Requiring Human Review

• S1: v3 lock + integrity model (breaking CLI / trust boundary)
  • References: cli/cli.ts (≈90–105, 178–230, 368–555, 785–830, 976–990); cli/integrity.ts (≈48–60, 165–254, 328–343, 421–605, 618–730); cli/commands/install/install-all.ts (≈34–99); cli/commands/install/install-mops-dep.ts (≈108–131); cli/CHANGELOG.md (## Next); NEXT-MAJOR.md
  • Base behavior: --lock check|update|ignore plus CI-env auto-check; every install/integrity path re-hashed .mops/ against the lock/registry and could fail on local edits
  • Diff proof: removes --lock / CI auto-detect, adds --locked + self-healing maintain, verifies bytes only at download (cache hits skip), and moves on-disk audit to new mops verify
  • Impact: Confirm CI migration (--locked everywhere that relied on CI=1 or --lock check), that pipelines no longer depend on install failing on a tampered .mops/, and that walk-free --locked checks plus unverified warm-cache packages are acceptable for release
  • Confidence: High

Verdict

Decision: REQUEST_HUMAN_REVIEW
Risk: High
Reason: Intended v3 redesign of the lockfile CLI surface and package-integrity guarantee on the install/resolution hot path; implementation looks consistent and well-tested, but blast radius warrants an explicit human sign-off before merge.

---

Generated for commit a8526d3