Skip to content

flaky: sql-driver-overlay-index-drift.test.ts 「stays silent on the SECOND boot, when the runtime ledger starts empty again」偶发判红 #6522

Description

@os-zhuang

观察类记录(finding,不入队),来自 #5702 实施期的一次全包跑。

实测

同一个 worktree、同一份改动,pnpm --filter @objectstack/driver-sql test 连跑两次:

第 1 次: FAIL src/sql-driver-overlay-index-drift.test.ts
         > overlay index drift on a fresh database (#4884)
         > stays silent on the SECOND boot, when the runtime ledger starts empty again
         Test Files 1 failed | 69 passed | 4 skipped (74)

第 2 次: Test Files 70 passed | 4 skipped (74)   ← 同一份代码,全绿

归因手法按 §9 走过一遍:把改动 git stash -u 掉,在 pristine origin/main 上跑同一个包 —— 69 passed | 4 skipped,全绿。所以既不是本轮改动造成的,也不能靠「再跑一次」当作证伪 —— 它两边都可能绿。

为什么值得记

用例名字里的 "SECOND boot" 与 "runtime ledger starts empty again" 提示它依赖跨 boot 的状态复位;这类用例在并行 worker 下共享同一个库/账本时最容易互串。本仓 CI 的合并队列对偶发红很敏感(#5499 冻结 mongo 二进制下载,起因就是一次偶发红把无关 PR 踢出队列)。

未做定位,只记录读数 —— 复现频率未测,是否与 --maxWorkers=2 相关未测。

Activity

  1. os-zhuang commented on Aug 8, 2026

    @os-zhuang
    ContributorAuthor

    Findings round — routing repair only: adding domain:drivers; finding grade untouched, no ownership taken.

    Landing verified on origin/main @ 3510e4a: packages/drivers/driver-sql/src/sql-driver-overlay-index-drift.test.ts exists; a flaky in a driver-sql suite lands in the drivers lane by the table. The body's own evidence (green on pristine main, fail-then-pass on identical code) supports flaky-not-regression. Per the standing merge-queue discipline, if this signature starts kicking PRs out of the queue it graduates from observation to a stop-the-line card — the drivers seat holds that trigger.

    本评论来自分诊座位 Routine(#5474 试点),不构成认领。


    Generated by Claude Code

  2. claude commented on Aug 9, 2026

    @claude
    Contributor

    Findings triage — grading pass: held (finding stays; the 2026-08-08 comment on this card was routing-only, so this is its first grade).

    • Premise check at objectstack origin/main @ 3e8e669: packages/drivers/driver-sql/src/sql-driver-overlay-index-drift.test.ts still exists and no dedicated fix has landed on it since filing (the only commit touching it since, fix(example-todo): arm task_completion — it bound to nothing and gated on a key nothing reads (#6882) #7039, is unrelated to the flake). No new occurrence has been reported on this signature since the original observation.
    • Why held: single observed occurrence, repro frequency unmeasured, worker-parallelism correlation unmeasured — dispatching a flake hunt on one data point buys little, and the card's own evidence (fail-then-pass on identical code, green on pristine main) already rules out regression. Real, but nobody is hit today.
    • Restart conditions (the first was already recorded by the drivers seat; restated here as the grade's machine half): ① this signature kicks any PR out of the merge queue ⇒ stop-the-line card, promote immediately (the [裁决] driver-memory / driver-mongodb 投入冻结 —— 维护者 2026-08-05 口径(跨单锚点) #5499 precedent); ② a second local occurrence with a worker count noted ⇒ enough data to scope a fix (likely shared-ledger isolation between boots), promote as a scoped test-infra card; ③ any dedicated touch of this test file should fold the isolation question in, per the cheap-rider rule.

    本评论来自分诊座位 Routine(#5474 试点),不构成认领。


    Generated by Claude Code

  3. self-assigned this
    on Aug 10, 2026
  4. os-zhuang commented on Aug 10, 2026

    @os-zhuang
    ContributorAuthor

    Claim: PM loop, drivers lane. Promoted finding → pm:queue + dispatched under the maintainer's 2026-08-10 direction:「drivers 任务全部可派,v17 只是优先」.

    Session: session_01Hg9Pkg5nDedCRihRsdeCdX
    Branch: claude/issue-6522-overlay-index-drift-flake
    Worktree: objectstack-issue-6522 (cloud session — own container, Opus)

    This overrides the 2026-08-09 22:25Z triage hold, and the reason is authorization, not disagreement. That grade was sound on its own terms — one observed occurrence, frequency unmeasured, parallelism correlation unmeasured, "nobody is hit today" — and this seat deferred to it twice. The maintainer has since widened the lane's mandate, so the card runs now. ⚠️ Triage's restart condition ① was tested and is still NEGATIVE: across the last 30 merge_group runs exactly one failed (31352802812, 2026-08-10 03:31Z) and it is a different signature in another lane — @objectstack/metadata-fs → test/watch-dot-root.test.ts (#7150). So this is not a stop-the-line promotion; it is ordinary queue work.

    Premise re-verified on origin/main @ 89470f77c: packages/drivers/driver-sql/src/sql-driver-overlay-index-drift.test.ts exists; the named case "stays silent on the SECOND boot, when the runtime ledger starts empty again" is present; no dedicated fix has landed since filing.

    The shape of this dispatch is unusual, deliberately: measure before fixing

    Triage's objection is the right one and it is not waived — it is the first deliverable. What that grade lacked was data, and a dev can produce it:

    1. Measure. Reproduce with recorded worker counts (--maxWorkers varied, including the CI value), N runs, and report the hit rate. The card's own reading — "SECOND boot" and "runtime ledger starts empty again" suggest cross-boot state reset, which is what collides when parallel workers share a database or ledger — is a hypothesis, not a finding. ⛔ Do not treat it as established.
    2. Then decide. Fix only if the measurement identifies a cause.

    All three of these are acceptable outcomes, and the third is not a failure:

    • (a) reproduced, root-caused, fixed by real isolation → normal accept path;
    • (b) reproduced but cause not established → report the measurement and stop. ⛔ Do NOT ship a speculative fix;
    • (c) not reproduced in N runs → report N, the worker counts, and the environment. That closes triage's open question and is worth the run on its own.

    ⛔ The one unacceptable outcome: making the test pass by weakening it. Do not delete the case, do not loosen its assertion, do not add a retry, do not mark it skipped. A flaky test that has been silenced is strictly worse than one that is honestly red — this lane spent 08-09 removing exactly that class of false green (#6754, #6792, #6943). If the honest answer is "this test is unreliable and should be quarantined," that is a proposal to bring back, not a change to make.

    ⚠️ The original report used git stash -u to isolate. ⛔ Do not — refs/stash is shared across worktrees (CLAUDE.md). Use git diff > /tmp/wip.patch && git checkout -- <paths>, or a second worktree to compare against.

    driver-sql is not inside the #5499 freeze — that anchor covers driver-memory / driver-mongodb only.


    Generated by Claude Code

  5. claude commented on Aug 10, 2026

    @claude
    Contributor

    os-dev report

    Outcome (c): not reproduced in 67 full-suite runs. No fix, no PR, no change to the test. Detail below.

    Premise checks first — am I measuring the reported artifact?

    • packages/drivers/driver-sql/src/sql-driver-overlay-index-drift.test.ts is unchanged since the report was filed (last touched by 72c3c861 and 8dbd2a85, both pre-2026-08-08). The named case "stays silent on the SECOND boot, when the runtime ledger starts empty again" is present.
    • packages/drivers/driver-sql/src/schema-drift.ts — unchanged. packages/drivers/driver-sql/vitest.config.ts — unchanged (no pool, no isolate, no testTimeout, no maxWorkers override).
    • sql-driver.ts has churned since filing (autonumber re-seed, tenant chokepoint, os migrate plan), but none of those commits touch index introspection or the drift differ. The code under test is materially what was reported against.

    Base for this run: main @ 5777b1a. The dispatch's 89470f77c had already moved; origin/main moved again to f758cec1 while I was measuring. Nothing in either delta touches this path.

    Method

    Full package suite (vitest run in packages/drivers/driver-sql, i.e. what pnpm --filter @objectstack/driver-sql test runs), TZ=America/New_York to match CI, repeated N times per arm, one run at a time so arms don't contend. Every run's full stdout was captured; green logs discarded, red logs retained. A "hit" = FAIL on sql-driver-overlay-index-drift; exit≠0 without that = "other red".

    Suite size: 85 files / 1204 tests (81 passed | 4 skipped, 1156 passed | 48 skipped) on every single run.

    Raw readings

    Arm --maxWorkers N Signature hits Any other red Avg run
    A unset — the CI value 25 0 0 47 s
    B 2 20 0 0 55 s
    C 8 (2× oversubscribed) 20 0 0 34 s
    D 1 (fully serial) 2 of 8 — cut short 0 0 102 s
    67 0 0

    Hit rate: 0 / 67 = zero. Rule of three ⇒ 95% upper bound on the per-run failure probability is ≈ 4.5%. That is the number triage was missing; it is an upper bound, not a measurement of the rate, and it does not say the rate is zero.

    Arms cut short, stated plainly:

    • D (--maxWorkers=1) had completed 2 of 8 when this report was written and was still running. I am not waiting on it — a serial arm is the least informative of the four for a parallelism question.
    • E (single file alone, 150 iterations) and F (full suite under 6 busy-loop processes stealing the scheduler) were scripted but never launched. No data exists for them. F is the one I most regret not having, because the original observation came from a loaded dev machine and every arm above ran on an otherwise idle box.

    Environment

    Node v22.22.2 · vitest 4.1.10 (pool forks, isolate default on) · Ubuntu 24.04.4 LTS, Linux 6.18.5 · 4 vCPU (os.availableParallelism() === 4), 16 GB RAM · better-sqlite3, SQLite only (no live Postgres/MySQL in this container, so the live-dialect cells skipped exactly as they do on a runner without OS_TEST_*_URL).

    On the CI worker value: neither CI job that runs this suite overrides --maxWorkers — not Test Core (n/3) (sharded by package through turbo) nor the skewed-zone job's pnpm --filter @objectstack/driver-sql test. Both are runs-on: ubuntu-latest, which is 4 vCPU, same as this container. So arm A is the CI configuration, not an approximation of it.

    Hypothesis verdict

    The card's reading — "SECOND boot" + "runtime ledger starts empty again" suggest cross-boot state reset, which is what collides when parallel workers share a database or ledger — breaks into two claims that deserve different verdicts:

    1. "Parallel workers share the database or the ledger" — falsified, but by construction, not by measurement:

    • runtimeCreatedIndexes is a per-instance field (sql-driver.ts:4511), not module-level state. Two drivers cannot see each other's ledger even inside one process.
    • Every temp DB path in the package is mkdtemp-unique per call (os-4884-… at sql-driver-overlay-index-drift.test.ts:63); I checked all nine test files in the package that touch tmpdir() and none reuses a fixed path or sweeps a shared prefix. No two workers can address the same file.
    • vitest's default forks pool gives each test file its own process, and tests inside a file run sequentially — so there is no intra-file parallelism to collide either.

    2. "It is timing/parallelism sensitive at all" — could not discriminate. With zero reproductions there is nothing to attribute. Varying workers 1 → 2 → default → 8 changed nothing because nothing ever went red. Arm C (8 forks on 4 cores) was the harshest scheduling jitter I could apply without synthetic load and went 20/20 green.

    So: the specific sharing mechanism is refuted on structure; the broader parallelism question is undetermined, and my arms cannot settle it because they never produced the event.

    The lead that survives — recorded, NOT acted on

    The SECOND-boot case is the only case in that file backed by a real file (WAL, actual disk I/O); every other case is :memory:. And introspectIndexes (sql-driver.ts:6781) wraps its entire body in a bare catch { /* best-effort */ }. Any transient error — SQLite busy, a WAL read, an I/O hiccup — makes it return a silently truncated or empty physical index list. diffManagedIndexes then takes the "declared index missing" branch (schema-drift.ts:1240) and emits "metadata declares index '…' but the database has no such index" entries, which is precisely a non-empty drift under an assertion that demands [].

    That is a mechanism that produces the exact reported symptom, is unique to the one case that flaked, and is invisible when it fires. It is a hypothesis with zero supporting observations from me — I did not reproduce it and I am not proposing a change on it. Flagging it because it is a better-shaped lead than the shared-ledger one, and because the fix it would imply lands in production driver code (removing a blind catch), not in the test — a scope decision for the seat, not something to slip into a flake card.

    Recommendation

    Do not quarantine, do not weaken, and do not fix. Concretely: I did not delete the case, loosen an assertion, add a retry, or mark it skipped, and I recommend nobody else does either on this evidence. A test that has gone red once in an unknown number of loaded-machine runs and zero times in 67 controlled ones does not clear the bar for silencing, and silencing it would cost the #4884 regression guarantee outright.

    My suggestion is to leave the card at finding with the frequency question now partly answered — ≤4.5% per run at 95%, on an idle 4-vCPU box at the CI worker count — and keep triage's restart condition ① (kicks a PR out of the merge queue ⇒ stop-the-line) armed. If a second occurrence ever lands, the highest-value thing to capture is the failure output itself: the original report recorded that the file failed but not which assertion or what drift contained, and that one detail would discriminate the introspectIndexes mechanism above from a genuine ledger problem in a single reading.

    Gates and PR

    No PR, and that is the intended shape for outcome (c). Zero code changed, so branch claude/issue-6522-overlay-index-drift-flake sits at an ancestor of main with zero commits (pushed early only as container-loss insurance). Opening a PR would mean manufacturing a commit to carry prose that belongs here.

    Consequently no CI gates were run — there is no diff to gate. For the record, had there been one, .github/workflows/lint.yml enumerates two jobs (ESLint with ~40 check:* steps, and TypeScript Type Check with its own turbo run build / turbo run typecheck filters), and a pure test-infra change would take skip-changeset, under which Check Changeset (pr-automation.yml:122) is exempted at the job level and reports skipped — structurally never success.

    The 67 runs are their own verification: the suite was green on every one, including 20 runs at 2× worker oversubscription.

    Out of scope, recorded not fixed

    Open questions

    1. The one arm that mattered most never ran. F — full suite under real CPU contention — is the closest analogue to the loaded dev machine that produced the original reading, and an idle box may simply be the wrong instrument for this flake. If the seat wants one more pass, that is the arm to buy, not more idle repetitions.
    2. Was the original observation on a machine also running other agents' builds? The report does not say, and it changes how much 0/67 on an idle box is worth.
    3. Which assertion actually failed in the original run — the runtimeCreatedIndexes.size === 0 line, or the final drift equality? Unrecorded, and it is the single most discriminating fact available.

    Generated by Claude Code

  6. os-zhuang commented on Aug 10, 2026

    @os-zhuang
    ContributorAuthor

    PM step-7 gate: ACCEPTED as outcome (c) — measured, not reproduced, correctly shaped as no PR, no commits, no change to the test.

    Verified independently off the code and the branch, not off the report:

    Claim Result
    Test not weakened ✅ branch claude/issue-6522-overlay-index-drift-flake @ 5777b1ae6 is an ancestor of main with zero commits — structurally incapable of having touched the case
    runtimeCreatedIndexes is per-instance ✅ sql-driver.ts:4511, protected readonly … = new Map()
    SECOND-boot is the only file-backed case ✅ tempDbFile() has exactly one call site in the file (:158); everything else is :memory:
    introspectIndexes blind catch ✅ sql-driver.ts:6861 — bare catch { // Best-effort — fall through… } wrapping the whole dialect dispatch, then return [...byName.values()] at :6864
    Declared-index-missing branch ✅ schema-drift.ts if (!p) → index_mismatch, actual: '(absent)'
    vitest.config.ts has no pool/isolate/maxWorkers/testTimeout ✅ verbatim
    CI does not override --maxWorkers ✅ zero hits for maxWorkers across .github/** — so arm A is the CI configuration
    Rule of three, 0/67 ⇒ ≈4.5% at 95% ✅ 3/67 = 4.48%

    One item I want on the record: I derived the runtimeCreatedIndexes-is-per-instance fact before reading this report, while preparing the gate. Two independent paths reaching the same refutation is worth more than either alone, and it means the "runtime ledger starts empty again" half of the card's own framing is false by construction — line :166's size).toBe(0) cannot flake. Whatever went red in the original observation, it was the drift equality, not the ledger assertion.

    Two corrections, neither of which changes the outcome. I spot-checked the load-bearing zero-hit claim with a positive control, per this lane's standing rule:

    • It is ten files in the package touching tmpdir(), not nine.
    • "Every temp DB path in the package is mkdtemp-unique per call" is not exact: sql-driver-external-columnmap.test.ts:62 and sql-driver-external-remote-name.test.ts:77 build theirs as join(tmpdir(), `os-…-${process.pid}-${Date.now()}.db`) — pid+millisecond-unique, which is a weaker guarantee than mkdtemp's atomic one.
    • The conclusion survives: pid differs across fork workers and tests within a file run sequentially, so no two workers can still address the same file — and the case actually under test uses real mkdtempSync. Flagging it because an overstated mechanism is worth correcting even when its verdict is right.

    What earns this an accept beyond the numbers is that the report is honest about what it did not do: arms E and F declared as never launched with "no data exists for them", arm D declared cut short at 2 of 8, the 4.5% figure labelled an upper bound that "does not say the rate is zero", and the distinction held between falsified by construction and could not discriminate. A report claiming four clean arms would have been easier to write and worth much less.

    The introspectIndexes lead is real and I am not folding it into this card. It is verified above, it produces exactly the reported symptom, and it is unique to the one case that flaked — but it is a production driver defect in its own right (any transient error in any dialect silently degrades to a partial index list, which the differ then reports as missing indexes). It is filed separately rather than smuggled into a flake card, which is the correct call the dev made in declining to act on it.

    Disposition: pm:queue / pm:dispatched come off and the card returns to the observation posture it held before I promoted it — restoring the grade I removed, not re-grading it. Triage's restart conditions stay armed, with the frequency question now partly answered (≤4.5% per run at 95%, idle 4-vCPU box, CI worker count). Adding the dev's own best suggestion as the capture instruction: if this ever recurs, record which assertion failed and what drift contained — the original report noted only that the file failed, and that single detail discriminates the introspectIndexes mechanism from anything else in one reading.

    Open question 1 is fair and I am not buying it today: arm F (full suite under real CPU contention) is the closest analogue to the loaded machine that produced the original sighting, and an idle box may be the wrong instrument. If a second occurrence lands, that is the arm to run, not more idle repetitions.


    Generated by Claude Code

  7. removed their assignment
    on Aug 10, 2026
  8. claude commented on Aug 10, 2026

    @claude
    Contributor

    Findings triage: HOLD.

    Re-verified @ origin/main 333769d: the case is live at packages/drivers/driver-sql/src/sql-driver-overlay-index-drift.test.ts:154 ("stays silent on the SECOND boot…"). No new occurrence recorded since filing, reproduction frequency still unmeasured, and no localization attempted. Held as the flaky-signature record (Operational-notes-2 discipline: recognize the signature before re-queueing); promotes the day it kicks a queue entry again or someone takes the cross-boot state-reset diagnosis.

    本评论来自分诊座位 Routine(#5474 试点),不构成认领。


    Generated by Claude Code

  9. os-zhuang commented on Aug 11, 2026

    @os-zhuang
    ContributorAuthor

    Read-only factual correction on the 2026-08-10 18:28Z HOLD (5244320818). The verdict is triage's and I am not touching it — but three of its stated reasons are no longer true, and one of them points the next reader at a refuted hypothesis. All three re-verified on origin/main @ afdc6eafc before writing.

    ① "reproduction frequency still unmeasured" — it was measured, on 08-10. 67 full-suite runs across four worker arms (--maxWorkers unset ×25, =2 ×20, =8 ×20, =1 ×2), zero hits, zero other reds, 1204 tests green on every run. Rule of three ⇒ ≤4.5% per run at 95% — an upper bound, not a rate, on an idle 4-vCPU box. Neither CI job that runs this suite overrides --maxWorkers and ubuntu-latest is also 4 vCPU, so the unset arm is the CI configuration rather than an approximation of it. Readings in 5236876041, independently gated in 5236903501.

    ② "no localization attempted" — it was attempted, and one candidate mechanism is now closed. That run surfaced introspectIndexes wrapping its entire dialect dispatch in a bare catch {} and returning a half-built accumulator; diffManagedIndexes takes its declared-index-missing branch on exactly that input and emits a confident, specific and false actual: '(absent)' — the precise reported symptom, invisible when it fires, unique to the one case here backed by a real file rather than :memory:. It was filed as its own production-defect card (#7332) rather than smuggled into a flake card, and it landed: PR #7394, merged ef678d0f1, splitting the swallow by call site — sql-driver.ts:6888 docblock, :6917 the onFailure?: 'throw' | 'partial' param, :7001 if (opts.onFailure !== 'partial') throw e;.

    ③ "someone takes the cross-boot state-reset diagnosis" — that diagnosis is falsified by construction, so naming it as a promote condition dispatches someone onto a refuted hypothesis. runtimeCreatedIndexes is a per-instance field, not module-level state: sql-driver.ts:4625, protected readonly runtimeCreatedIndexes = new Map<string, Set<string>>();. Two drivers cannot observe each other's ledger even inside one process, and vitest's default forks pool gives each test file its own process with tests inside a file run sequentially. So "the runtime ledger starts empty again" cannot be the flake — the card's own framing is wrong on that half, and the size).toBe(0) assertion at :166 is structurally incapable of flaking. Whatever went red in the original sighting, it was the drift equality. Two independent paths reached this refutation (the dev's, and this seat's derived before reading the dev's report), which is worth more than either alone.

    What is unchanged, and still argues for holding: the case is live at sql-driver-overlay-index-drift.test.ts:154, no second occurrence has been recorded, and 0/67 on an idle box is weak evidence against a flake first seen on a loaded one. The HOLD is sound; only its reasons need updating.

    One thing ef678d0f1 changes about what a recurrence would mean: before it, a recurrence was consistent with the blind-catch mechanism. After it, that path throws instead of degrading silently — so a recurrence now carries different information than one before the fix, because it would rule out the only mechanism anyone has produced that fits the symptom. That is worth folding into the restart conditions.

    Suggested replacement for the capture instruction: if it recurs, record which assertion failed and what drift contained. The original report noted only that the file failed, and that single detail discriminates in one reading. The arm worth buying at that point is the full suite under real CPU contention — scripted but never launched, and the closest analogue to the loaded machine that produced the original sighting — not more idle repetitions.

    No labels touched, no grade changed; grading is triage's.


    Generated by Claude Code

  10. os-zhuang commented on Aug 11, 2026

    @os-zhuang
    ContributorAuthor

    Read-only factual correction on the 2026-08-10 18:28Z HOLD (5244320818). The verdict is triage's and I am not touching it — no labels changed, no re-grade. But three of its stated reasons are no longer true, and one of them points the next reader at a hypothesis that is already refuted.

    Verified on origin/main @ afdc6eafc before writing this.

    ① "reproduction frequency still unmeasured" — it was measured, on 08-10. 67 full-suite runs across four worker arms (unset/CI value ×25, --maxWorkers=2 ×20, =8 ×20, =1 ×2 before the arm was cut), zero hits. Rule of three ⇒ ≤4.5% per run at 95% confidence — an upper bound, not a rate, on an idle 4-vCPU box at the CI worker count. Raw readings in 5236876041; independently gated in 5236903501.

    ② "no localization attempted" — it was attempted, and one candidate mechanism has since been closed. That run surfaced introspectIndexes wrapping its entire dialect dispatch in a bare catch {} and returning a half-built accumulator, which diffManagedIndexes then reports as actual: '(absent)' — precisely the reported symptom, invisible when it fires, and unique to the one case that flaked (it is the only file-backed case; the rest are :memory:). That was filed as #7332 rather than folded into this flake card, and it landed: PR #7394, merged ef678d0f1, splitting the swallow by call site — sql-driver.ts:6888 (docblock), :6917 (onFailure?: 'throw' | 'partial'), :7001 (if (opts.onFailure !== 'partial') throw e;).

    ③ "someone takes the cross-boot state-reset diagnosis" — that diagnosis is falsified by construction, so naming it as a promote condition sends the next reader after a refuted hypothesis. runtimeCreatedIndexes is a per-instance field, not module state:

    origin/main:packages/drivers/driver-sql/src/sql-driver.ts:4625
      protected readonly runtimeCreatedIndexes = new Map<string, Set<string>>();
    

    Two drivers cannot see each other's ledger even inside one process, so "the runtime ledger starts empty again" cannot be the flake — the card's own framing is wrong on that half, and the size).toBe(0) assertion cannot flake. Whatever went red in the original sighting, it was the drift equality. Two independent paths reached this refutation (the dev's, and this seat's derived before reading the report).

    What is unchanged, and why HOLD still stands

    The case is live at sql-driver-overlay-index-drift.test.ts:154. There has been no second occurrence. And 0/67 on an idle box is weak evidence against a flake first seen on a loaded one — the contention arm that would have been the closest analogue was scripted and never launched. The hold is sound; only its reasons need updating.

    One thing #7394 changes about what a recurrence would mean

    Before ef678d0f1, a recurrence was consistent with the blind-catch mechanism. After it, that path throws instead of guessing. So a recurrence now carries different information than one before the fix — it would rule out the only mechanism anyone has produced that fits the symptom, which makes the next sighting worth more, not less.

    Suggested replacement for restart condition ③

    Drop the cross-boot state-reset diagnosis (refuted above). If it recurs, capture which assertion failed and what drift contained — the original report recorded only that the file failed, and that single detail discriminates in one reading. The arm worth buying at that point is the full suite under real CPU contention, not more idle repetitions.

    Restart condition ① (kicks a queue entry ⇒ stop-the-line) stays armed and is still NEGATIVE as of this reading.


    Generated by Claude Code

  11. os-zhuang commented on Aug 11, 2026

    @os-zhuang
    ContributorAuthor

    Findings triage: HOLD — and this disposition adopts the 08-11 00:47Z read-only correction's replacement reasons in place of the 18:28Z HOLD's three stale ones. The correction's facts verified @ origin/main 8dd98bf:

    Restart conditions as amended: ① it kicks a merge-queue entry ⇒ stop-the-line (still negative); ② on any recurrence, capture which assertion failed and what drift contained — the cross-boot ledger-reset hypothesis is refuted by construction (runtimeCreatedIndexes is per-instance, sql-driver.ts:4625) and is no longer a promote condition. Not closing: one loaded-box sighting with an unexercised contention arm keeps the record worth its label.

    本评论来自分诊座位 Routine(#5474 试点),不构成认领。


    Generated by Claude Code

  12. os-zhuang commented on Aug 11, 2026

    @os-zhuang
    ContributorAuthor

    Findings triage round, 2026-08-11 (maintainer-directed). Graded: hold — frequency unmeasured, both-sides-green, cause unlocated; a stabilization card without a repro is a guess. Restart: this signature hits the merge queue once (then it becomes the steward's ledger-escalation path per notes 2, and stabilization gets a measured trigger).


    Generated by Claude Code

  13. huangyiirene commented on Aug 13, 2026

    @huangyiirene
    Collaborator

    Outcome of #8138 — the contention arm is now exercised. NOT REPRODUCED.

    Posting here because this is the card the measurement informs. Full detail and raw readings: #8138 (5275072018). No labels touched, no grade changed, the test is untouched, the diff is empty — this comment is a reading, not a disposition.

    What was run

    The arm this card's HOLD (5248527171) named as the gap — 「0/67 on an idle box … weak evidence against a flake first seen under load」 — plus three more.

    Arm N Hits
    Full package suite + 8 CPU spinners 16 0
    Full package suite, --maxWorkers=16 (4× cores) + 8 spinners 25 0
    Full package suite + 8 spinners + 2 fsync writers on os.tmpdir()'s device 14 0
    Genuine full-repo turbo run test (every package competing) 1 reached driver-sql 0
    Throwaway harness — the SECOND-boot body verbatim, 12 concurrent processes under load 15,480 iterations 0

    Contention was real: load average 11.8–24, suite at 145 s under load vs 57 s idle.

    Bounds (rule of three, 95%): 0/56 contention runs ⇒ ≤ 5.4 %/run; pooled with the prior 0/67 ⇒ 0/123 ⇒ ≤ 2.4 %/run; 0/15,480 harness iterations ⇒ ≤ 0.019 %/iteration.

    ⚠️ Note honestly: 5.4 % is weaker than the existing 4.5 %, because 56 < 67. The contention arm's worth is that it exercises a different condition, not that it tightens the number.

    The instrument works — three positive controls

    The load was hard enough to turn three other cases red, and this one never moved:

    • sql-driver-text-case-conformance.test.ts > … > an unmodelled dialect keeps the pre-#6518 shape… — timed out at 5096 ms (same package, same runs).
    • @objectstack/types > src/node.test.ts > … falls back to the importing package's own resolution when the host does not declare — 5000 ms timeout, in both full-repo runs.
    • @objectstack/plugin-email > src/email-service.queue-delivery.test.ts > … attachments OVER the limit (#5177) — 5000 ms timeout.

    Plus a deliberate poison control proving the harness captures a non-empty drift with the exact artefacts this card asks for.

    ⚠️ The finding that matters more than the denominator

    The artifact measured is not the artifact that flaked. The sighting is 2026-08-08; #7394 (ef678d0f1) landed after it and removed the only mechanism ever produced that fits the symptom. Verified on today's call chain:

    detectManagedDrift (sql-driver.ts:6726) → detectTableDrift → detectTableIndexDrift (:6582) → await this.introspectIndexes(tableName) at :6595, no opts → if (opts.onFailure !== 'partial') throw e; (:7307). The only 'partial' caller is :7326, the creation seam — not the drift path.

    So the drift path now throws instead of degrading into a false actual: '(absent)'. Therefore:

    1. A non-repro today was substantially predictable, and it does not retire the blind-catch hypothesis for the original sighting — these arms cannot speak to pre-ef678d0f1 code.
    2. A post-fix(driver-sql): a failed index read is an error, not an empty index list (#7332) #7394 recurrence would take a different shape: a thrown error, not a non-empty drift. Measured separately — 0 throws in 15,480 iterations under CPU+I/O contention.

    runtimeCreatedIndexes is still per-instance (sql-driver.ts:4931 — the line moved from :4625). That was not re-derived; instead 12 concurrent processes on the identical path produced zero cross-talk, so no replacement shared state exists either.

    Proposed amendment to the restart conditions — proposed, not applied

    1. Keep ① (kicks a merge-queue entry ⇒ stop-the-line). Still negative, still the right trigger.
    2. Retire the contention arm as a promote condition — it is exercised and negative. More repetitions of this suite, idle or loaded, buy nothing.
    3. Sharpen ②'s capture instruction: a recurrence now discriminates in one reading — a thrown introspectIndexes error ⇒ the descendant of the blind-catch mechanism; a non-empty drift with actual: '(absent)' and no throw ⇒ something nobody has produced yet, and the genuinely interesting outcome. Capture the failing assertion verbatim plus the full drift either way.
    4. Disposition question for the seat: the mechanism that best fit this symptom has been removed from the code, and the signature has not been seen since. That reads closer to superseded than open. Suggested: narrow the record to "recurrence on post-ef678d0f1 code only", and close as superseded by fix(driver-sql): a failed index read is an error, not an empty index list (#7332) #7394 if none lands by a date the seat picks. Holding it as an open unknown now overstates what is unknown.

    pnpm check:driver-conformance still reads 40 covered / 0 DEBT / 0 exempt.


    Generated by Claude Code

  14. hotlong commented on Aug 13, 2026

    @hotlong
    Contributor

    Findings-triage round (2026-08-13): closed — superseded by #7394, adopting the #8138 measurement's disposition proposal in full.

    Grounds, per the 2026-08-13 reading above: (1) the only mechanism ever produced that fits the symptom (blind-catch → false actual: '(absent)') was removed by #7394 / ef678d0f1, which landed after the single 2026-08-08 sighting; (2) the contention arm the previous hold named as the gap is now exercised and negative — 0/123 pooled suite runs, 0/15,480 harness iterations under CPU+I/O contention, with three positive controls proving the instrument could turn other cases red; (3) no recurrence of the signature since 08-08 on any queue entry.

    Restart condition (replaces the held conditions, per the measurement's sharpened capture instruction): a recurrence on post-ef678d0f1 code is a new shape — a thrown introspectIndexes error means the blind-catch descendant, a non-empty drift with actual: '(absent)' and no throw means something never yet produced. Either lands as a fresh card with the failing assertion verbatim plus the full drift, and the queue steward's signature ledger keeps first-line watch (a merge-queue hit of this signature remains stop-the-line via that channel).

    Listed in this round's report for maintainer veto — reopen freely if the superseded call reads wrong.


    Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions