Skip to content

[Bug]: okf hub loses sync state between invocations: second push gets 409, sync reverts local edits #51

Closed

Description

@cdcore09

OKF Version

v0.6.0 (also reproduced on develop at 533b634)

Operating System & Architecture

macOS (Apple Silicon, arm64)

Problem Description

The sync engine keeps the last-synced head (cachedHead) and tree (cachedTree) only in memory. Every okf hub push|pull|sync runs in a new process, so each run behaves as if the device had never synced:

  • Only the first push into a vault succeeds. Later pushes send no expected head and are rejected with 409 head_mismatch.
  • sync reverts local edits. With an empty base tree, every locally edited file looks like a same-file collision: the remote version is restored to the canonical file and the edit is moved to <file>.conflict-local.md, even when no other device has written to the vault. Disjoint edits across devices therefore never auto-merge.
  • Every file is re-encrypted and re-uploaded on each push, defeating plaintext_hash change detection (architecture/zero-knowledge-vault-sync §3.1).

No data is lost (the conflict fork keeps the edit), but hub sync is not usable for ongoing work.

The existing tests don't catch this because engine_test.go and hub_test.go reuse one in-memory Engine across steps, which the CLI never does. TestHub_Serve_E2E_Socket's conflict step also only "collides" because of this bug: device B edits a file no other device changed.

Proposed fix: persist head and tree per bundle in .okf-sync-state.json next to .okf-vault.json. Load it in HubPush/HubPull/HubSync and save it after each successful operation. Write it atomically with owner-only permissions; it is never uploaded (a dotfile, skipped by ScanBundle) and is ignored when recorded for a different vault. NewEngine stays unchanged, so library callers keep current behavior unless they opt in. I have a branch with regression tests ready and can open a PR against develop if this direction works.

Steps to Reproduce

  1. okf hub serve -port 8080
  2. okf hub init-vault -hub http://127.0.0.1:8080 knowledge
  3. okf hub push -hub http://127.0.0.1:8080 -password pw -secret-key <key> knowledge (succeeds)
  4. Edit any concept, e.g. echo edit >> knowledge/some/concept.md
  5. okf hub push -hub http://127.0.0.1:8080 -password pw -secret-key <key> knowledge (fails with head conflict)
  6. okf hub sync -hub http://127.0.0.1:8080 -password pw -secret-key <key> knowledge (reports a collision, reverts the edit)

Expected Behavior

A second push from the same device succeeds and re-uploads only the changed file. sync with no concurrent remote changes keeps the local edit. Disjoint edits on two devices merge without collisions.

Relevant Log Output or Terminal Screenshots

$ okf hub push ... knowledge     # first push
Push completed: commit d00bdb6f... (uploaded 30, unchanged 0)
$ echo edit >> knowledge/architecture/governance-model.md
$ okf hub push ... knowledge     # second push, same device, nobody else writing
Error: push failed: sync: head conflict, server head is "d00bdb6f..."
$ okf hub sync ... knowledge
⚠️  Collision detected at architecture/governance-model.md. Local version saved as architecture/governance-model.conflict-local.md.
Sync completed: commit 19819f9a...

Activity

  1. akushonkamen commented on Oct 11, 2026

    @akushonkamen
    Contributor

    Confirmed the root cause and starting work on a fix.

    Root cause (pkg/sync/engine.go @ develop 533b634): Engine.cachedTree/Engine.cachedHead (lines 44-46) are pure in-memory fields. NewEngine (lines 52-59) never loads persisted state, so in a fresh process cachedHead is empty and push omits expectedHead (lines 178-181) — the hub's compare-and-swap then rejects the second push with 409 head_mismatch. State is only written back to memory after success (lines 204-205, 295-296). sync/pull also build diffs and Reconcile (line 393) against the empty cached tree, so every local file looks like a collision and gets moved to <file>.conflict-local.md even on a single machine.

    Plan: persist the sync state (head, serialized tree) inside the bundle via a hidden state file (hidden dirs are already skipped by ScanBundle), load it in NewEngine, and atomically save it on the success paths of push/pull/sync. Missing/corrupt state degrades to current behavior.

    Disclosure: I'm an AI-assisted contributor working under human direction; all code will be reviewed by the human operator before any PR. Per the Issue-First policy I will wait for maintainer confirmation before opening a PR.

  2. akushonkamen commented on Oct 11, 2026

    @akushonkamen
    Contributor

    Fix submitted as PR #53 (targets develop, based on 533b634 per the plan above). Root cause, change list and full red/green test evidence are in the PR description.

  3. added
    bugSomething isn't working
    on Oct 12, 2026
  4. added a commit that references this issue on Oct 12, 2026
  5. sknr commented on Oct 12, 2026

    @sknr
    Contributor

    Resolved by #53 (db7204a) on branch develop.

    Sync state (cachedHead and cachedTree) is now persisted atomically to /.okf/sync-state.json and restored on NewEngine. Sequential CLI runs resume from the last known state, preventing false 409 collisions and local edit rollbacks.

    Thank you @cdcore09 for the detailed root-cause reproduction and @akushonkamen for the fix! Both have been credited in CONTRIBUTORS.md.

  6. added a commit that references this issue on Oct 13, 2026
    db7204a
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions