Symptom
integrationTests/apiTests/blob.test.mjs → "Per-device-type LMDB database sharding" fails intermittently in CI on Integration Tests shard 3/6, and passes on re-run with no code change. Two occurrences on the same PR (#1106) on 2026-07-01/02 alone — a PR whose diff is a pure formatter + test files that never touch blob/sharding code:
- Run 28542021803, Node v22:
AssertionError: doorlock schema missing (in all three device-type schemas are visible in describe_all), then cascading SQL queries target each device database independently: expected 200 "OK", got 404 "Not Found", with HdbError: Table 'doorlock.DoorlockBlob' does not exist in the Harper log. Passed on re-run.
- Run 28558632042, Node v22 (re-run): same shape, different fixture —
AssertionError: sensor schema missing, then the same 200→404 cascade. Passed on re-run.
Both times the same shard passed simultaneously on Bun, Node v24, Node v26, and Windows — the failure is timing-dependent, not version-deterministic (observed twice on v22, but the mechanism looks like scheduling luck rather than a v22 behavior).
Shape of the race
The failing assertion is schema visibility: a device-type schema (doorlock, sensor — varies per occurrence) created during setup is missing from a subsequent describe_all, and SQL against that device database then 404s. The suite itself times out around ~30s (Per-device-type LMDB database sharding (29992ms / 35174ms)). This smells like create-schema/table completing on one worker thread before the metadata is visible to the thread serving the next request — same eventual-consistency family as #1497 (async ITC not awaited) and adjacent to #687 (create_table returns success before table is available). Possibly a condition-wait issue per the AGENTS.md guidance (prefer waitFor(condition) over fixed sleeps).
Context
Acceptance
🤖 Generated with Claude Code
Symptom
integrationTests/apiTests/blob.test.mjs→ "Per-device-type LMDB database sharding" fails intermittently in CI on Integration Tests shard 3/6, and passes on re-run with no code change. Two occurrences on the same PR (#1106) on 2026-07-01/02 alone — a PR whose diff is a pure formatter + test files that never touch blob/sharding code:AssertionError: doorlock schema missing(inall three device-type schemas are visible in describe_all), then cascadingSQL queries target each device database independently: expected 200 "OK", got 404 "Not Found", withHdbError: Table 'doorlock.DoorlockBlob' does not existin the Harper log. Passed on re-run.AssertionError: sensor schema missing, then the same 200→404 cascade. Passed on re-run.Both times the same shard passed simultaneously on Bun, Node v24, Node v26, and Windows — the failure is timing-dependent, not version-deterministic (observed twice on v22, but the mechanism looks like scheduling luck rather than a v22 behavior).
Shape of the race
The failing assertion is schema visibility: a device-type schema (
doorlock,sensor— varies per occurrence) created during setup is missing from a subsequentdescribe_all, and SQL against that device database then 404s. The suite itself times out around ~30s (Per-device-type LMDB database sharding (29992ms / 35174ms)). This smells like create-schema/table completing on one worker thread before the metadata is visible to the thread serving the next request — same eventual-consistency family as #1497 (async ITC not awaited) and adjacent to #687 (create_table returns success before table is available). Possibly a condition-wait issue per the AGENTS.md guidance (preferwaitFor(condition)over fixed sleeps).Context
Acceptance
waitFor-style condition ondescribe_allcontents) rather than assuming synchronous visibility.🤖 Generated with Claude Code