Repository navigation
Data loss: schema-less heterogeneous ingest triggers single-node #1163 'shared structure missing', records decode as null #1282
Description
Activity
Resolution: stale-dependency artifact, not a defect in harper main
Root-caused. The data loss reproduces only with a stale
structon(< 1.0.7) innode_modules(the QA-explorer environment, like my initial repro, hadstructon1.0.4). harper main pinsstructon ^1.0.7, and with the pinned version this does not reproduce.Evidence (heterogeneous open-table ingest):
config structon 1.0.4 (stale) structon 1.0.7 (pinned/shipping) real RecordEncoder, classic, 50k6 / 50,000 50,000 / 50,000 isolation classic + maxOwnStructures=256catastrophic 0 loss isolation typed + maxOwnStructures=256catastrophic 0 loss, and fast (typedStructs bounded) structon 1.0.7 contains the
maxOwnStructurescap (the CDI-OOM fix) and correctly handles the two-byte-record path thatRecordEncoder'smaxOwnStructures = 256engages — so the 256 cap is correct and necessary on ≥ 1.0.7. (Earlier in this investigation I briefly proposed removing the cap; that was based on the stale 1.0.4 and would have reopened the typedStructs OOM — not pursued.)Exposure: any checkout/deployment with
structon < 1.0.7actually installed (dev checkouts or non-clean installs), not published 5.1.0 (consumers resolve^1.0.7→ 1.0.7).npm ci(notnpm install) keepsnode_modulesin sync with the lockfile and avoids this class of phantom. Thedev/harperanddev/harper-probase checkouts have been refreshed to structon 1.0.7.Guard added: #1284 promotes the QA-explorer scenario to a regression test under
integrationTests/database/— inserts 2,000 distinct-shape records and asserts they all read back intact — so a structon regression or accidental downgrade can't silently reintroduce the loss.Closing as not-reproducible on the pinned structon. Investigated with Claude Opus 4.8.
- added a commit that references this issue
on Jun 16, 2026
Metadata
Metadata
Assignees
Type
Fields
Priority
Summary
Ingesting a large batch of records with highly heterogeneous shapes into an open (non-
@sealed) table causes the vast majority of records to become unreadable — they decode asnullwith error #1163 "Record references a shared structure missing on this node". Critically, #1163 is documented as a replica-only condition, but this reproduces on a single node.Severity
Potential data loss. In the repro, 6 of 50,000 records were readable back. A uniform-shape control of the same size read back 50,000/50,000.
Repro
Found via exploratory QA (qa-explorer skill), harper
001bf7b9c(v5.1.0, main).@sealed).insertAPI in batches, where each record has a randomly-generated distinct shape (varying key names drawn from a large pool, varying nesting depth, varying value types for the same key name across records).Observed: logs flood with #1163 (e.g. "decoded as null", citing a structure id); almost all records return null/missing.
Expected: all inserted records readable, as with uniform-shape ingest.
A scratch reproduction test exists (qa-scratch/heterogeneous-ingest) and can be promoted to
integrationTests/database/as a regression test.Where to look
resources/RecordEncoder.ts:425(the #1163 path). The msgpackr/structon shared-structure (typedStruct) resolution appears to lose/skip structures under high shape churn on a single node.Open question
Not yet disambiguated whether records fail at write time or are written-but-unreadable on the shared-structure resolve path. First triage step: check whether the on-disk record bytes are present but the structure dictionary entry is missing.
Filed from qa-explorer wave 1 (F-001). Medium confidence it's a real defect; the data-loss symptom is reproducible.