Skip to content

Research: does re-seeding unchanged content add rows, or resolve to the rows already there? #338

Open

Description

@DutchJaFO

Question

When a seed runs a second time over content that has not changed, does it add rows to the database, or
does it plan Add actions that resolve to the rows already present?

Background

Observed across four consecutive startup logs on 2026-08-20, not from a bug report.

A fresh database seeds and the per-file new counts sum exactly to the total: 13 (curated) + 723
(NikhilNamal17) + 63 (vilaboim) = 799, and [Database - Stats] reports 799 quotes. On that path,
new and "rows added" coincide.

A second run against that already-populated database produced the same counts —
Quote[new=723 modified=9 …], Quote[new=63 modified=36 …], Quote[new=13 …] — and
[Database - Stats] still reported 799 quotes. So on a re-seed, 799 rows' worth of Add actions were
planned against a database that already held those 799 rows, and the total did not move.

FileImportReport documents New as "an Add action that resolved cleanly (Decided or Applied)", so
the counter is behaving as specified — it counts actions, not net inserts. That makes the log ambiguous
rather than wrong on a re-seed, and it is not the question here. The question is what the write path
did with those 799 Add actions.

Two readings, not yet distinguished:

  • Benign. Every domain id in this project is a deterministic hash of normalised content
    (QuoteIdentity.StableId, EntityIdentity.*), so a re-add resolves to the same id and the write is
    idempotent. Nothing is duplicated, the counts simply describe what the file asserts. If this holds,
    the outcome is a coverage gap and possibly a clearer log line.
  • A defect. Duplicate detection is not matching on re-import and the writes are colliding or being
    silently swallowed. If this holds, it is a real bug in the seeding path that has never been caught.

The row totals not moving is consistent with both.

Why nothing caught it

Worth recording, because the gap is more interesting than the symptom:

  1. ImportActionReportBuilderTests asserts the tally against synthetic actions it constructs itself —
    Build_AddDecided_CountsAsNew feeds one Add and expects New == 1. True by construction; it cannot
    disagree with the planner because it is the planner's own arithmetic.
  2. No test anywhere compares an import report against the database. Nothing counts rows before a
    seed, seeds, counts after, and checks the report against the delta.
  3. The two numbers only diverge on a re-seed of unchanged content. Every test and every demo uses a
    fresh database, which is exactly the case where they agree.

What to investigate

  1. Seed a real SQLite database with the bundled content, record row counts per entity type, seed the
    identical content again, and compare: do row counts move, and do they move by the amount the second
    report's new counts imply?
  2. Establish what the second pass's Add actions actually did at the write layer — inserted and
    collided, were skipped, or were resolved to the existing row by deterministic id.
  3. Determine whether modified counts on the re-seed (9 and 36 for the two bundled sources) reflect
    genuine field changes from the refreshed downloads, or are an artefact of re-planning.
  4. Check whether the same holds for the entity types other than Quote — Source showed
    new=348 modified=60 on the same re-seed.
  5. Decide whether the log line should distinguish "actions planned" from "rows changed", given a reader
    sees it immediately above [Database - Stats].

How findings are recorded

All findings must be posted as a comment on this issue before closing. The comment must directly answer
the question and state a recommendation if applicable.

Possible outcomes

  • Benign, with a coverage gap — file an issue for the missing real-database round-trip test, and
    optionally one for the log wording. This is the expected outcome.
  • A defect in the seeding write path — file it as a bug, since silent duplicate-collision on
    re-import would affect anyone who restarts after a source refresh.
  • Not worth pursuing — document why and close.

Note that the test in outcome 1 is worth having whichever way the answer falls, and overlaps #328's
"bundled imports against a real database" scope — check whether it belongs there before filing
separately.

Definition of done

  • All items in the investigation list addressed
  • Outcome clearly stated (see Possible outcomes above)
  • Any new issues filed and linked in the closing comment
  • Any ADR written and linked in the closing comment
  • Findings and recommendation posted as a comment on this issue

Activity

  1. added this to the v1.9.0 milestone on Aug 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    researchInvestigation or design spike

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions