aw-sync: daemon (the default subcommand) never pulls — two incompatible sync-folder layouts

@TimeToBuildBob

aw-sync ships two mutually incompatible sync-folder layouts, and the one used by the default daemon subcommand can never see remotes written by the other one.

Net effect: running bare aw-sync (which is what the bundled binary and aw-qt do) is push-only, into a directory nothing else reads. It silently never pulls.

The two layouts

code path	used by	writes	scans
`sync_wrapper::push` / `pull`	`aw-sync sync` (no advanced flags), aw-android (`syncPush`/`syncPullAll`/`syncBoth` via JNI)	`{sync_dir}/{hostname}/{device_id}/test.db`	`{hostname}/{device_id}/*.db` (3 levels)
`sync::sync_run` called directly	`aw-sync daemon` — i.e. the default subcommand	`{sync_dir}/{device_id}/test.db`	`{sync_dir}//.db` (2 levels)

find_remotes() in aw-sync/src/util.rs walks exactly two levels:

fs::read_dir(sync_directory)?
    .filter(|p| p.is_dir())                 // {sync_dir}/{hostname}
    .flat_map(|d| fs::read_dir(d).unwrap()) // {sync_dir}/{hostname}/{device_id}  <- a dir, not a .db
    .filter(|path| path.extension() == "db")

Real peer databases sit one level deeper, so the filter drops every one of them. find_remotes_nonlocal() returns an empty vec, and the pull loop in sync_run() iterates over nothing.

Meanwhile setup_local_remote(sync_spec.path, device_id) stages the local push at {sync_dir}/{device_id}/test.db — at the folder root, with no hostname level — where get_remotes() (which requires a subdirectory containing a .db) will not find it either. So a daemon-mode device is invisible to aw-sync sync peers and to Android, in both directions.

Reproduction / evidence

Setup: macOS desktop running the bundled aw-sync with no arguments (→ daemon), sync folder shared over Syncthing with an Android device on v0.14.2b1.

~/ActivityWatchSync/ after the Android device pushed successfully:

POCO F8 Ultra/41662faa-.../test.db      9.4 MB    <- 3-level, from aw-android
poco_f8_ultra/41662faa-.../test.db      273 MB    <- 3-level, from aw-android
erb-m2.localdomain/d7bc68e7-.../test.db 
d7bc68e7-.../test.db                    1.19 GB   <- 2-level, from the local daemon

102 consecutive daemon sync passes, ~/Library/Logs/activitywatch/aw-sync/aw-sync_2026-09-14T21-45-45+0200.log:

    102 Pulling...
    102 Pushing...
      0 Found N remote db files      <- never logged, not once

The Android buckets never appear locally. The only reason this is not more widely reported is that the failure is completely silent — Pulling... is logged unconditionally before the (empty) loop.

Two further consequences on the same machine:

The local host-layout staging copy at erb-m2.localdomain/d7bc68e7-.../test.db last received an event on 2025-01-22. Everything since has gone into the root-level 2-level db, so this desktop has not published anything its peers can read for ~8 months.
That root db is 1.19 GB and Syncthing replicates it to every device, where nothing reads it. It also still contains …-synced-from-… buckets (re-exported peer data) predating fix(sync): never re-sync buckets synced from another host #648 — that fix stopped new ones being created but nothing cleans up existing ones.

Suggested fix

Collapse to one layout. Cheapest correct change: have Commands::Daemon call sync_wrapper::pull_all + sync_wrapper::push per cycle, exactly as the Commands::Sync fallback branch does, instead of driving sync::sync_run against the sync root. That makes the daemon agree with aw-sync sync and with Android, and leaves sync_run as the per-directory primitive it already is.

If the 2-level {device_id}/ scheme is preferred long-term (it is arguably the better one — see the identity discussion in the linked issues), then it needs a migration for existing folders plus a matching change in aw-android, and get_remotes()/find_remotes() must be unified rather than left as two different walkers.

Either way find_remotes() and get_remotes() should share one implementation — the TODO: share logic with find_remotes and find_remotes_nonlocal already in the source is precisely this bug.

Related, same area

sync_wrapper::pull() calls sync_run with sync_spec.path = {sync_dir}/{host}, and sync_run unconditionally calls setup_local_remote(path, device_id) — including in SyncMode::Pull. So pulling from a peer creates an empty {sync_dir}/{peer_host}/{my_device_id}/test.db inside that peer's folder. On the machine above this produced a d7bc68e7-… directory inside erb-laptop2-arch/, tekla-air-m1/, erb-pn50/, steamdeck/ and others.

This breaks the "each device only writes to files in the sync folder they own" invariant the whole design rests on (stated in aw-sync/README.md), and is a plausible source of the test.sync-conflict-*.db files that show up in these folders. setup_local_remote should only run when the mode actually pushes.

cc @TimeToBuildBob

Related, from the same investigation:

aw-sync: duplicate folders for one device_id silently truncate history on pull #683 — duplicate folders for one device_id silently truncate history on pull
aw-sync: sync failures are silent — add aw-sync status, loud diagnostics, per-device manifest, and surface peers in Raw Data #684 — sync failures are silent; aw-sync status, loud diagnostics, per-device manifest
Sanitized device hostname forks the sync identity and never migrates bucket hostnames aw-android#272 — sanitized device hostname forks the sync identity

Opened #685 with the cheap fix from this issue:

default daemon (what aw-qt and the bundled binary run) now does sync_wrapper::pull_all + push per cycle — same 3-level {hostname}/{device_id}/ layout as aw-sync sync and Android
setup_local_remote only runs when the mode actually pushes, so a pull no longer creates {peer_host}/{our_device_id}/test.db
find_remotes / get_remotes share one walker that sees both 2-level and 3-level .db files

Not in that PR: migrating the existing 2-level root db (the 1.19GB file on erb-m2). After #685 lands, new daemon pushes go where peers can read them; the leftover root db still needs a one-time move/cleanup.

Advanced --buckets / --start-date / --sync-db still use sync_run against the root (same split as aw-sync sync). That path can now see 3-level remotes; it still writes 2-level.

feat(aw-sync): add `status` doctor command and fail-loud empty-pull w…

Looked into this again; no additional action from Bob right now.

The cheap fix is in #685 (CI-green, mergeable). Self-merge is not eligible here (cross-repo / pull-only access) — waiting on a maintainer click. Leftover 2-level root-db migration is still not in that PR.

fix(aw-sync): make default daemon use host-layout pull/push

docs(aw-sync): make 3-level-only pull walker explicit

fix(aw-sync): make default daemon use host-layout pull/push

feat(aw-sync): add `status` doctor on the RemoteDb pull walker

feat(aw-sync): add `status` doctor command and fail-loud empty-pull w…

fix(aw-sync): make default daemon use host-layout pull/push

feat(aw-sync): add `status` doctor command and fail-loud empty-pull w…

fix(aw-sync): skip duplicate device_id folders that truncate history …

fix(aw-sync): make default daemon use host-layout pull/push

feat(aw-sync): add `status` doctor command and fail-loud empty-pull w…

fix(aw-sync): make default daemon use host-layout pull/push

feat(aw-sync): persist discovery warnings on zero-peer SyncReport

fix(aw-sync): treat empty local device_id as unknown in discovery war…

feat(aw-sync): persist discovery warnings on zero-peer SyncReport

fix(aw-sync): treat empty local device_id as unknown in discovery war…

feat(aw-sync): return and persist a SyncReport from each pass (#699)

docs(aw-sync): document the working sync path; drop stale limitations (…

@TimeToBuildBob

@TimeToBuildBob So how can we fix this?

PR #685 is the fix — daemon now aligns pull/push to the same 3-level {hostname}/{device_id}/ layout that aw-sync sync and Android use.

It has a merge conflict now from the v0.14.0 sequence merges. I'll rebase and push the fix so it's ready to land.

The one-time migration of the existing 2-level root db (the 1.19GB file on erb-m2) is a separate follow-up — that's not blocking the protocol fix in #685.

@TimeToBuildBob

@TimeToBuildBob Hold that — #685 is parked by decision, not stalled by conflicts. Recorded on ActivityWatch/activitywatch#1445 (and acknowledged there): the daemon will switch straight to the v2 devices/{device_id}/ layout when the format work lands, never passing through the 3-level layout as canonical. Rebasing #685 now is throwaway work and would produce a green, mergeable PR that contradicts the plan.

What v0.14.0 shipped for this issue instead: #698 (pull-only never stages in peer folders), #700 (read-only peer opens), #703 (broken peers skipped, not fatal), #687 (aw-sync status makes the never-pulling daemon visible), #701 (README leads with aw-sync sync and marks the daemon path broken). The daemon\x27s pull remains intentionally unfixed until v2.

Please don\x27t push a rebase to #685; leave it as-is with its parking comment.

fix(aw-sync): make default daemon use host-layout pull/push

Understood — holding, and walking back my 12:21 comment.

#685 is parked by decision, not stalled by conflict; the rebase I said I'd push was throwaway against the plan and has been reverted on the branch (back to its parked head 96a5ca0). What v0.14.0 shipped for the daemon-pull gap is the sequence you named — #698 (pull-only never stages in peer folders), #700 (read-only peer opens), #703 (broken peers skipped instead of aborting the pass), plus #687's aw-sync status visibility. The daemon's pull stays unfixed until the v2 devices/{device_id}/ layout, and the daemon moves straight there.

So: answer to "how can we fix this" is "not in v0.14.0 — the v2 layout retires the 3-level path entirely." I'll pick it up against that format work, not against the current layout.

fix(aw-sync): use sanitize_hostname in doctor; add own-device root-db…

fix(aw-sync): isolate per-peer discovery errors; extract discover_pee…

Navigation Menu

Repository navigation

aw-sync: `daemon` (the default subcommand) never pulls — two incompatible sync-folder layouts #682

The two layouts

Reproduction / evidence

Suggested fix

Related, same area

1 remaining item

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

aw-sync: daemon (the default subcommand) never pulls — two incompatible sync-folder layouts #682

Description

The two layouts

Reproduction / evidence

Suggested fix

Related, same area

Activity

ErikBjare commented on Sep 15, 2026

TimeToBuildBob commented on Sep 15, 2026

TimeToBuildBob commented on Sep 16, 2026

1 remaining item

ErikBjare commented on Sep 17, 2026

TimeToBuildBob commented on Sep 17, 2026

ErikBjare commented on Sep 17, 2026

TimeToBuildBob commented on Sep 17, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

aw-sync: `daemon` (the default subcommand) never pulls — two incompatible sync-folder layouts #682