Skip to content

feat(aw-sync): status --clean-legacy prunes re-exported buckets from own staging - #723

Open
0xbrayo wants to merge 3 commits into
ActivityWatch:masterfrom
0xbrayo:fix/aw-sync-prune-legacy-synced-buckets
Open

0xbrayo wants to merge 3 commits into
ActivityWatch:masterfrom
0xbrayo:fix/aw-sync-prune-legacy-synced-buckets

Conversation

@0xbrayo

@0xbrayo 0xbrayo commented Sep 24, 2026

Copy link
Copy Markdown
Member

Addresses item 2 of #689.

Draft: this is legacy-path cleanup, and it's parked until the v0.14.0 stable tag per ActivityWatch/activitywatch#1445.

What

aw-sync status --clean-legacy [--dry-run]

  • Lists the re-exported …-synced-from-… buckets (from before fix(sync): never re-sync buckets synced from another host #648) that are still in this device's own staging dbs, with per-bucket event counts.
  • Without --dry-run, deletes those buckets and their events, then runs VACUUM so the replicated file actually shrinks.

Safety

  • Only dbs that scan_sync_dir classifies as OwnStaging are opened for writing. That classification is by device_id, so the local server must be reachable; otherwise the command refuses. Peers' files are never touched.
  • prune_synced_buckets refuses any id without the -synced-from- marker (via the new is_synced_bucket_id, which is_synced_bucket now delegates to).
  • Opens the db without SQLITE_OPEN_CREATE, so a staging db that disappeared is not recreated.
  • 30 s busy timeout in case a daemon pass is pushing at the same moment.
  • Idempotent: ids already gone are skipped.
  • The status warning about these buckets now points at the flag for own staging, and says only the owning device can prune a peer's copy.

Not in this PR: #689 item 1 (the "orphaned" 2-level db)

The 2-level {device_id}/test.db isn't actually an orphan today. #685 was closed, so the default daemon still stages there, while aw-sync sync stages at {hostname}/{device_id}/test.db. Either one can be the live copy depending on how the user runs aw-sync, so deleting either would force a full re-export. Retiring legacy staging belongs with the v2 devices/{device_id}/ switch.

Tests

4 new unit tests on real sqlite staging dbs:

  • prunes own staging in both layouts and leaves the peer's db untouched;
  • dry run leaves the file byte-identical;
  • first-hand ids are refused;
  • a missing bucket is a no-op, and a missing db is not created.

cargo test -p aw-sync passes, as do cargo clippy --workspace -- -D warnings and cargo fmt.

…own staging

Before ActivityWatch#648 the push path re-exported buckets that had themselves been
imported from a peer. Nothing re-creates them any more and every reader
skips them, but they still sit in this device's staging dbs, which every
peer replicates (hundreds of thousands of events on affected setups).

`aw-sync status --clean-legacy [--dry-run]` lists the `-synced-from-`
buckets in databases scan_sync_dir classifies as own staging, and deletes
them followed by VACUUM so the file actually shrinks. Peers' files are
never opened for writing, first-hand bucket ids are refused, and a
missing db is not recreated. The status warning now points at the flag
for own staging and says only the owner can prune a peer's copy.

Staging dbs themselves are not deleted: the daemon still stages at
{device_id}/test.db and `aw-sync sync` at {hostname}/{device_id}/test.db,
so neither is an orphan yet. That retirement belongs with the v2 layout.
@greptile-apps

greptile-apps Bot commented Sep 24, 2026 •

Copy link
Copy Markdown

RetriggerConfidence Score: 5/5

The PR appears safe to merge based on this review; no new actionable issue was established.

Summary

The PR adds aw-sync status --clean-legacy to inspect and prune re-exported buckets from this device’s own staging databases. The latest revision records unfinished VACUUM work in the database so a later run can retry it without vacuuming unrelated databases.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A[Scan own staging databases] --> B{Marked buckets or pending VACUUM?}
  B -- No --> C[Leave database alone]
  B -- Yes --> D{Dry run?}
  D -- Yes --> E[Report targets]
  D -- No --> F[Delete buckets and record pending marker]
  F --> G[VACUUM]
  G -- Success --> H[Clear marker]
  G -- Failure --> I[Keep marker for retry]
Loading

Reviews (3) · Last reviewed commit: "fix(aw-sync): retry --clean-legacy VACUU..."

Comment thread aw-sync/src/legacy_clean.rs Outdated
Comment thread aw-sync/src/legacy_clean.rs
Comment thread aw-sync/src/legacy_clean.rs
@codecov

codecov Bot commented Sep 24, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 81.21212% with 31 lines in your changes missing coverage. Please review.
✅ Project coverage is 79.93%. Comparing base (656f3c9) to head (b8eb654).
⚠️ Report is 135 commits behind head on master.

Files with missing lines Patch % Lines
aw-sync/src/legacy_clean.rs 84.31% 24 Missing ⚠️
aw-sync/src/main.rs 0.00% 4 Missing ⚠️
aw-sync/src/status.rs 0.00% 3 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master     #723      +/-   ##
==========================================
+ Coverage   70.81%   79.93%   +9.11%     
==========================================
  Files          51       76      +25     
  Lines        2916     8566    +5650     
==========================================
+ Hits         2065     6847    +4782     
- Misses        851     1719     +868     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

… on skipped dbs

- An own staging db whose freelist is >= 1/10 of the file stays a target
  and VACUUM runs whenever the freelist is non-empty, so a prune whose
  VACUUM failed is finished by the next run instead of reporting nothing
  to do.
- Refuse any staging path that resolves outside the sync dir layout
  through a symlink: own vs peer is decided from the path, but sqlite
  opens whatever it points at.
- An own staging db that was skipped (uninspectable or symlinked) makes
  the command exit non-zero, in dry runs too.
- Test temp dirs get a counter so parallel tests cannot share one.
@0xbrayo

0xbrayo commented Sep 24, 2026

Copy link
Copy Markdown
Member Author

🤖 Claude, on behalf of @0xbrayo

@greptile review

Comment thread aw-sync/src/legacy_clean.rs Outdated
Replace the free-space heuristic with a key_value marker written in the
same transaction as the deletes and removed once VACUUM succeeds. A db
with free pages from ordinary deletes is no longer vacuumed by cleanup.
@0xbrayo

0xbrayo commented Sep 24, 2026

Copy link
Copy Markdown
Member Author

🤖 Claude, on behalf of @0xbrayo

@greptile review

@0xbrayo
0xbrayo marked this pull request as ready for review September 24, 2026 09:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant