Skip to content

fetch: a request is answered only within its line; never older within it (#481) - #484

Merged
EricAndrechek merged 6 commits into
mainfrom
fix-wrongline-481
Oct 6, 2026
Merged

EricAndrechek merged 6 commits into
mainfrom
fix-wrongline-481

Conversation

@EricAndrechek

@EricAndrechek EricAndrechek commented Oct 6, 2026 •

Copy link
Copy Markdown
Member

Summary

A request is answered only within its own ClickHouse line, and the anti-rollback ("monotonic") keep is scoped to the request (#481). Concurrent installs into one cache no longer delete or replace each other's entries (#482). All four bindings follow one rule. There is no version bump.

#481: the scoped keep rule, in all four

  • The keep rule. When the registry offers a build older than one already installed, the installed build is kept only if its signed version lies within the request.
    • For a line request (26.3), that means any newer build of that line.
    • For an exact request (26.3.38.2), only a newer build of that exact version.
    • An install of another line never answers.
    • The newest qualifying install is returned. This holds even when the offered build is installed too.
  • What each binding did before.
  • The load-time assertion. After every open through a registry, the library's own build_info clickhouse_version must lie within the request. Otherwise the open fails as CHTYPES_ARTIFACT_CORRUPT, a loader refusal with reason build_info_mismatch:clickhouse_version, want the request and got the version.
    • Why this code: fetch-v1.md §4 already makes "the signed version does not lie within the request" CHTYPES_ARTIFACT_CORRUPT, and build_info_mismatch is the existing loader refusal for a build_info field that disagrees. No new code, class or reason is added.
    • A non-version tag is not checked.
    • open_unverified takes no request, so it is out of scope.

#482: race-safe installs, in all four

  • Rename first. An entry that holds an acceptable record is never removed or replaced. Only an entry without one is moved aside, and it is put back if it turns out to be good.
    • TypeScript ran rm -rf on the final entry and then renamed.
    • Go, Python and Rust checked for an entry and then moved aside whatever stood there.
    • Measured on the unfixed code: Go across processes on macOS, and Python across threads.
  • Listing. A listing reads only <manifest-hex> names, never an installer's temporary directory. Python already did this.
  • TypeScript errors. A filesystem failure in the fetch layer is now a UsageError in the registry, and the usage exit status in the CLI, as in Go.

Tests

  • Conformance. request-scope-* (6 cases × file/http), generated by genfixtures, run in every binding's v1-conformance leg.
  • v1-cache-interop, writer × reader × line. Each writer fetches 26.8. Each reader adds 26.3 online, then answers both lines offline.
  • v1-cache-interop, concurrency. 8 processes at a time, for each binding alone and for all four mixed, 3 rounds each. A watcher reports torn, vanished and replaced entries.
  • Unit tests for the race and the assertion in each binding.

Evidence

  • Unfixed head 936690f, run 37478628838.
    • New conformance cases: Go and TypeScript fail 6 pairs each, Python 2, Rust 0.
    • Interop line matrix: every Go and TypeScript reader answered 26.3 with 26.8.5.1, whatever binding wrote the cache.
    • ts x8: 23/24 installed, REPLACED ×6, UNRECORDED ×1, VANISHED ×1.
    • mixed x8: REPLACED ×4, TORN-LIBRARY ×1, VANISHED ×1.
  • Fixed head: d11e58c (merges main at 546489f): all 21 required contexts green, v1-parity and v1-cache-interop included (measured once every run on the commit had finished). The one failure is the report-only security scan, which fails on main too.

Test plan

  • the new cases fail on the unfixed bindings (CI run 37478628838; Go also locally) and pass after the fix
  • go build/vet/test, golangci-lint; uv run pytest -q, ruff; tsc --noEmit (both configs), biome; cargo check/clippy/fmt
  • genfixtures --selftest/--check, schema_check.py, gen-constants --check, abi-v1/gen.py --check, lint-public/spelling/cited-paths/actions, dprint, markdownlint, policy-merge-check --check-guide/--selftest
  • every required check green on the head (see above)

Related issues

Fixes #481
Fixes #482

🤖 Generated with Claude Code

EricAndrechek and others added 6 commits October 6, 2026 10:22
…rrent installs in cache-interop (#481, #482)

Tests only; the fixes follow in the next commits, so this head is expected
to fail where the bindings are wrong.

- genfixtures: four request-scope-* cases over a tree serving lines 26.3 and
  26.8, each online: a higher line installed must not answer a lower line or
  an exact lower build; a newer build of the same line is kept for a line
  request (monotonic warning) but never answers an exact request for another
  version.
- cache-interop: writer x reader x line (two lines in one cache, the lower
  added online), and N=8 concurrent installs into one fresh cache per
  binding and mixed, with a watcher for torn, vanished and replaced entries.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…ed to it, install race-safely (#481, #482)

#481, all four bindings, one rule (the lead's ruling):
- The monotonic check keeps a newer install only when its signed version
  lies WITHIN the request: for a line request, a newer build of that line;
  for an exact request, a newer build of that exact version only. An install
  of another line never answers. The newest qualifying install is returned.
  Go (newerAlreadyInstalled) and TypeScript (checkMonotonic) compared every
  install for the platform; Python compared the same four-part version only,
  so it installed an older build of a line over a newer one. Rust already
  filtered by the request; it is unchanged.
- A load-time assertion after every registry open: the library's own
  build_info clickhouse_version must be within the request, else
  CHTYPES_ARTIFACT_CORRUPT with reason build_info_mismatch:clickhouse_version
  (the code fetch-v1.md section 4 gives a signed version outside the
  request). A non-version tag is not checked.

#482, all four bindings:
- An unpacked entry is installed rename-first: an entry with an acceptable
  record is never removed or replaced; only one without is moved aside, and
  put back if it turns out good. TypeScript deleted the final entry and then
  renamed (raw ENOTEMPTY, vanished entries). Go, Python and Rust checked and
  then moved aside whatever stood there, which could be another process's
  complete install (measured for Go across processes on macOS, for Python
  across threads).
- Listing reads only <manifest-hex> names, never an installer's temporary
  directory (Go, TypeScript, Rust; Python already did).
- TypeScript: a filesystem failure in the fetch layer is a UsageError in the
  registry and the usage status in the CLI, as Go gives it.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…safe installs; CHANGELOGs (#481, #482)

- fetch-v1.md section 9: a newer install is kept only within the request,
  and the load-time assertion (CHTYPES_ARTIFACT_CORRUPT, reason
  build_info_mismatch:clickhouse_version); section 1: how an unpacked entry
  is installed and what the interop job proves; section 10: the
  request-scope-* cases.
- bindings-v1.md: the assertion in the artifact-corrupt row and as step 7
  of the open sequence; the misuse row names the fetch layer's filesystem
  failure in Go and TypeScript.
- One #481 and one #482 Fixed bullet, the same in all four CHANGELOGs.
- Go race test: plain unpack dirs, no MkdirTemp (golangci usetesting).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…ild is installed too (#481)

The lead's ruling is "never answer a line request with a build older than
one already installed within that line". All four bindings returned the
offered build as soon as its manifest was installed, without the keep
check, so with 26.3.4.1 and 26.3.9.1 both installed and the registry
offering 26.3.4.1, an online `26.3` answered 26.3.4.1. Now the keep check
runs on that path too (Go, TypeScript, Python: the already-installed
branch; Rust: no longer skipped when `already`).

Two conformance cases pin it: request-scope-newer-beside-offered (26.3 ->
26.3.9.1, monotonic warning) and request-scope-exact-beside-newer
(26.3.4.1 -> 26.3.4.1). The first fails on the previous commit's Go.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant