Skip to content

fix(sync): never re-sync buckets synced from another host - #648

Merged
ErikBjare merged 2 commits into
ActivityWatch:masterfrom
TimeToBuildBob:fix/aw-sync-no-reexport-synced
Aug 8, 2026
Merged

ErikBjare merged 2 commits into
ActivityWatch:masterfrom
TimeToBuildBob:fix/aw-sync-no-reexport-synced

Conversation

@TimeToBuildBob

Copy link
Copy Markdown
Contributor

Fixes #647. Reported by @nairboon in discussions#1373.

Problem

A host's own data could return to it through a peer, leaving it with a -synced-from-<itself> duplicate of its own bucket. /timeline doesn't dedup, so every event rendered twice.

The existing guard works but only covers the direct path: find_remotes_nonlocal() filters the local device_id out of the remote list, so a host never reads its own export. It cannot see data laundered through a peer:

HOSTA pushes → HOSTB pulls (..._HOSTA-synced-from-HOSTA, correct) → HOSTB re-exports it on its next push → HOSTA pulls it back from HOSTB's export, whose path contains did_B and so is legitimately not filtered.

The missing rule: a host must only ever offer data it collected itself.

Change

sync_datastores() skips buckets carrying the -synced-from- marker, in both directions.

Push-side alone would stop new pollution, but affected users have the laundered bucket persisted in the shared sync folder, so it would keep being re-imported until that folder was cleaned by hand. Filtering on pull as well lets existing deployments self-heal.

$aw.sync.origin would be the natural provenance flag but is unusable: get_or_create_sync_bucket() writes it on push-staging as well as on import (per the existing FIXME about -synced leaking into the staging area), so it is set on a host's own exports too and would make every bucket look second-hand. The ID suffix is what the code already builds and parses, so that is what's matched.

Tests

New aw-sync/tests/sync_roundtrip.rs drives four datastores through push → pull → push → pull:

  • test_push_does_not_reexport_synced_buckets — the fix location
  • test_own_data_does_not_return_via_peer — the reported symptom

Both fail on master, the second with exactly the reported bucket list:

left: ["aw-watcher-window_HOSTA", "aw-watcher-window_HOSTA-synced-from-HOSTA"]
right: ["aw-watcher-window_HOSTA"]

The existing aw-sync/tests/sync.rs (5 tests) still passes; cargo fmt --check and clippy --all-targets are clean.

Notes for review

  • Deliberate consequence: data is only ever exchanged first-hand — relaying HOSTA→HOSTB→HOSTC no longer works. With Syncthing sharing one folder across all hosts that path is unnecessary, and relayed copies would be stale/partial. Worth confirming nobody relies on it.
  • Not fixed here: buckets already imported are not deleted. Affected users need to remove their *-synced-from-<own hostname> buckets once; they won't come back.
  • The -synced-from-HOST ID suffix is carrying provenance that arguably belongs in bucket metadata — it duplicates the hostname already in the ID and forces string-matching for dedup. Left alone here; happy to open a separate discussion.

A host's own data could return to it through a peer. HOSTA pushes, HOSTB
pulls and stores the data as `<bucket>_HOSTA-synced-from-HOSTA`, then
HOSTB's next push re-exports that bucket, and HOSTA pulls it back — ending
up with a synced copy of its own bucket next to the local one, so
/timeline renders every event twice.

The existing guard (`find_remotes_nonlocal`, which filters the local
device_id out of the remote list) only prevents a host from importing its
own export directly. It cannot see data laundered through a peer.

Skip buckets carrying the `-synced-from-` marker in both directions, so a
host only ever offers data it collected itself. Applying it on pull as
well as push means already-polluted sync folders stop re-infecting hosts
without needing the shared folder to be cleaned first.

Reported by nairboon in ActivityWatch/discussions#1373.
@greptile-apps

greptile-apps Bot commented Aug 6, 2026 •

Copy link
Copy Markdown

Greptile Summary

The PR prevents buckets imported from another host from being synchronized again, stopping peer-mediated copies from returning to their origin.

  • Adds a shared provenance-marker check to both push and pull synchronization.
  • Adds round-trip regression coverage for re-export prevention and duplicate avoidance.
  • Documents and tests the existing reserved-token behavior of -synced-from-.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains.

Important Files Changed

Filename Overview
aw-sync/src/sync.rs Filters already-synchronized buckets from both transfer directions and documents the existing bucket-ID provenance grammar.
aw-sync/tests/sync_roundtrip.rs Adds datastore round-trip regressions covering peer re-export, return-to-origin duplication, and reserved-token handling.

Sequence Diagram

sequenceDiagram
    participant A as HOSTA local
    participant AE as HOSTA export
    participant B as HOSTB local
    participant BE as HOSTB export
    A->>AE: Push first-hand bucket
    AE->>B: "Pull as *-synced-from-HOSTA"
    B--xBE: Push skips imported bucket
    BE-->>A: No copied bucket to pull back
Loading

Reviews (2): Last reviewed commit: "test(sync): pin -synced-from- as a reser..." | Re-trigger Greptile

Comment thread aw-sync/src/sync.rs
Addresses review feedback that the substring check could misclassify a
first-hand bucket whose ID happens to contain the marker.

The token is already structurally reserved: get_or_create_sync_bucket
splits on it to recover the original ID, so such a bucket would have its
ID truncated on import regardless of this filter. Making the reservation
explicit and tested rather than implicit.
@TimeToBuildBob

Copy link
Copy Markdown
Contributor Author

Status update (project monitoring)

Greptile P1 resolved — no blocking findings remain.

The single P1 finding (substring misclassification of first-hand buckets) was addressed in dc7ee40:

  • is_synced_bucket() docstring now makes -synced-from- an explicitly reserved token, not an implicit convention
  • test_synced_from_is_a_reserved_id_token pins both sides of the contract (reserved token is filtered; a normally-named bucket alongside it still exports)

Greptile reviewed the change and confirmed: "the concern is resolved." Thread resolved.

CI not yet triggered. This is a fork PR — GitHub Actions may require maintainer approval to run the first workflow. build.yml and lint.yml target pull_request on master; they should run once a maintainer approves the workflow run. The PR body notes cargo fmt --check and clippy --all-targets were clean locally.

Converged after 1 round. Recommend merge once CI passes.

@codecov

codecov Bot commented Aug 7, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 93.82716% with 5 lines in your changes missing coverage. Please review.
✅ Project coverage is 76.49%. Comparing base (656f3c9) to head (dc7ee40).
⚠️ Report is 83 commits behind head on master.

Files with missing lines Patch % Lines
aw-sync/tests/sync_roundtrip.rs 93.24% 5 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master     #648      +/-   ##
==========================================
+ Coverage   70.81%   76.49%   +5.67%     
==========================================
  Files          51       64      +13     
  Lines        2916     5211    +2295     
==========================================
+ Hits         2065     3986    +1921     
- Misses        851     1225     +374     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@TimeToBuildBob

Copy link
Copy Markdown
Contributor Author

CI passed — ready to merge

All checks are green (Android, macOS, Ubuntu, Windows, clippy, format, Codecov). Greptile confirmed 5/5. No unresolved threads.

The PR is ready for a maintainer to merge at their discretion.

@TimeToBuildBob

Copy link
Copy Markdown
Contributor Author

CI-green and mergeable (Greptile 5/5) — waiting only on a maintainer click.

This PR is ready to merge, but the bot has pull-only access to this repo and can't self-merge — surfacing it here so it isn't lost. The monitoring loop will stop re-flagging it now that this note is posted.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

aw-sync: host re-imports its own data via a peer, creating duplicate -synced-from-<self> buckets

2 participants