Skip to content

aw-sync: rowid upgrade bootstrap does not recover backfills lost under the old timestamp cursor #793

Open

Description

@TimeToBuildBob

Follow-up from #789 (review thread on the upgrade bootstrap).

Gap. On the first pull after upgrading to the rowid cursor (stored_rowid == 0 and the destination already has events), #789 seeds the cursor at the source's current max rowid and runs one last timestamp-cursor pass. A backfilled event (old timestamp, new rowid) that landed in the source before that first rowid pull sits below both boundaries: the max-rowid seed marks it consumed, and the timestamp pass starts after the destination's newest event. It is never copied.

Scope. This is not a regression. The same events were already lost under the old destination-as-marker cursor (#696); #789 stops new losses but does not recover old ones. My earlier description on the review thread ("only if the timestamp pass fails") was too narrow: the miss happens even when that pass succeeds.

Possible fix. A one-time, opt-in reconcile on the bootstrap pass (or an explicit aw-sync --reconcile flag): scan source history, then insert events missing from the destination, matched by (timestamp, duration, data). Heartbeat-merged events complicate the matching, so this needs a design decision before anyone implements it. It is not needed to merge #789.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions