Skip to content

feat(edgesync): add StateExported to the spoke ledger (#569) - #580

Merged
xe-nvdk merged 1 commit into
mainfrom
feat/edge-sync-ledger-exported-state
Aug 7, 2026
Merged

xe-nvdk merged 1 commit into
mainfrom
feat/edge-sync-ledger-exported-state

Conversation

@xe-nvdk

@xe-nvdk xe-nvdk commented Aug 7, 2026

Copy link
Copy Markdown
Member

PR 9a of 4 for the air-gap bundle (item 9 of #569). The ledger state comes first because the format and endpoints depend on it. Next: 9b format/MAC/export, 9c import/dedup, 9d ack.

Why this is a separate PR

The original plan was one PR that left exported files in pending. An adversarial review of the plan caught that this does not converge, and I reproduced it:

without StateExported:  9 files, batch 3, 3 rounds -> 3 files, each exported 3x
with it:                9 files, batch 3, 3 rounds -> 9 files, exactly once each

Pending orders partition_time DESC (ledger.go:392), so a capped export re-takes the newest N every time and the oldest files never leave the box. A submarine on a 90-day cycle would ship the same recent data forever while its backlog silently starved. That is a treadmill, not eventual consistency.

Transitions

pending  -> exported   MarkExported, bundle ID required
exported -> synced     MarkSynced, widened for the ack path (9d)
exported -> pending    RevertExported, scoped to one bundle

MarkExported accepts only pending: an in_flight row belongs to a running network transfer, and exporting it concurrently would put one file on two paths with no way to reconcile which acknowledgment arrived.

MarkSynced previously accepted only pending/in_flight, so an ack for an exported file would have been rejected — an air-gap spoke could never reach synced, PruneSynced would never prune, and the ledger would grow forever on the box least able to receive a site visit. Defined now even though the ack ships in 9d.

Operator surface

  • Stats counts exported separately, but PendingBytes still includes exported bytes — a file on a drive in transit has not arrived, and excluding it would make the backlog appear to shrink at exactly the moment nothing was delivered.
  • Unfinished includes exported entries, so they stay visible in the troubleshooting view.
  • exported_at / exported_bundle_id are readable on LedgerEntry and on the ledger endpoint. RevertExported filters on the bundle ID in SQL, but an operator whose drive did not arrive needs to read it to know what to revert. (Raised in review — previously only reachable by opening the SQLite file.)

Schema

Two nullable columns, added with tolerated duplicate-column errors since SQLite has no ADD COLUMN IF NOT EXISTS. 26.09.1 is unreleased (no tag), so only development databases are affected.

Test plan

  • 7 new tests; the starvation test verified to fail pre-change (3 of 9 files, each 3×), and the bundle-ID test likewise
  • Both mutants confirmed to compile before trusting the failure — a false negative I have hit before
  • Migration verified on a real pre-9a spoke database: 20 rows and their states preserved, restart idempotent, no duplicate-column error
  • Live binary: exported counted separately from pending, ledger endpoint lists the exported file first rather than hiding it
  • go test -race, go vet, gofmt clean

🤖 Generated with Claude Code

First of four PRs for the air-gap bundle (item 9). Adds the ledger state
the export path needs, ahead of the format and endpoints, because
everything else depends on it.

Without it, exported files stay `pending`. Pending orders
partition_time DESC, so a capped export re-takes the newest N every time
and the oldest files never leave the box:

  before:  9 files, batch 3, 3 rounds -> 3 files, each exported 3x
  after:   9 files, batch 3, 3 rounds -> 9 files, exactly once each

That is a treadmill, not eventual consistency, and it was the shape of
the original single-PR plan. An adversarial review of the plan caught it.

Transitions:

  pending  -> exported   MarkExported, bundle ID required
  exported -> synced     MarkSynced, widened for the ack path (PR 9d)
  exported -> pending    RevertExported, scoped to one bundle

MarkExported accepts only `pending`: an in_flight row belongs to a running
network transfer, and exporting it concurrently would put one file on two
paths with no way to reconcile which acknowledgment arrived.

MarkSynced previously accepted only pending/in_flight, so an ack for an
exported file would have been rejected and an air-gap spoke could never
reach `synced` — PruneSynced would never prune and the ledger would grow
forever on the box least able to receive a site visit. The transition is
defined now even though the ack ships in 9d.

Operator surface:
- Stats counts `exported` separately, but PendingBytes still INCLUDES
  exported bytes: a file on a drive in transit has not arrived, and
  excluding it would make the backlog appear to shrink at exactly the
  moment nothing was delivered.
- Unfinished includes exported entries, so they stay visible.
- exported_at / exported_bundle_id are readable on LedgerEntry and on the
  ledger endpoint. RevertExported filters on the bundle ID in SQL, but an
  operator whose drive did not arrive needs to READ it to know what to
  revert. (Raised in review; previously only reachable via sqlite3.)

Schema: two nullable columns, added with tolerated duplicate-column
errors since SQLite has no ADD COLUMN IF NOT EXISTS. 26.09.1 is
unreleased, so only development databases are affected.

Test plan:
- [x] 7 new tests; the starvation test verified to fail pre-change
      (3 of 9 files, each 3x) and the bundle-ID test likewise
- [x] mutants confirmed to COMPILE before trusting either failure
- [x] migration verified on a real pre-9a spoke database: 20 rows and
      their states preserved, restart idempotent, no duplicate-column error
- [x] live binary: exported counted separately from pending, and the
      ledger endpoint lists the exported file first rather than hiding it
- [x] go test -race, go vet, gofmt clean

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant