Repository navigation
fix(sync): persist sync state across engine invocations - #53
Merged
Merged
Conversation
The hub sync engine kept cachedHead/cachedTree purely in memory, so every `okf hub push|pull|sync` invocation started from an empty state: - the second push sent no expected head and was rejected by the hub compare-and-swap with 409 head_mismatch; - a subsequent sync reconciled against an empty base tree, so local edits to previously pushed files looked like incompatible concurrent modifications and were moved to `*.conflict-local.md` even with no other writer; - every push re-encrypted and re-uploaded the whole bundle because change detection had no baseline. Persist the last synced head and its tree manifest in <bundle>/.okf/sync-state.json (hidden directories are skipped by ScanBundle, so the state is never pushed as content). NewEngine loads it on startup; push/pull/sync atomically rewrite it (temp file + rename) after each successful commit. Missing, corrupt, or foreign-vault state is ignored, degrading to the previous empty-state behavior. TestHub_Serve_E2E_Socket previously locked in a collision that was an artifact of the lost state; it now edits and pushes on device A first, so device B's sync hits a genuine concurrent modification. Fixes okf-memory#51
Contributor
|
Merged into develop via commit db7204a. Thank you @akushonkamen for the clean implementation, atomic state handling, and comprehensive regression test suite! You have been credited in CONTRIBUTORS.md. |
cdcore09
added a commit
to cdcore09/okf-agent-memory
that referenced
this pull request
Oct 14, 2026
Creating different concepts in the same folder on two devices changes that folder's index.md on both sides. Only log.md had a semantic merge, so the shared index collided: the canonical index dropped one side's listing and the other was forked to index.conflict-local.md. Add a 3-way merge keyed by listing link target, preserving hand-written index text and falling back to the collision failsafe only when that text changed on both sides. The base index comes from the last-synced tree that okf-memory#53 persists in .okf/sync-state.json. Fixes okf-memory#52 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
10 tasks done
cdcore09
added a commit
to cdcore09/okf-agent-memory
that referenced
this pull request
Oct 14, 2026
Creating different concepts in the same folder on two devices changes that folder's index.md on both sides. Only log.md had a semantic merge, so the shared index collided: the canonical index dropped one side's listing and the other was forked to index.conflict-local.md. Add a 3-way merge keyed by listing link target, preserving hand-written index text and falling back to the collision failsafe only when that text changed on both sides. The base index comes from the last-synced tree that okf-memory#53 persists in .okf/sync-state.json. Fixes okf-memory#52
sknr
pushed a commit
that referenced
this pull request
Oct 19, 2026
Creating different concepts in the same folder on two devices changes that folder's index.md on both sides. Only log.md had a semantic merge, so the shared index collided: the canonical index dropped one side's listing and the other was forked to index.conflict-local.md. Add a 3-way merge keyed by listing link target, preserving hand-written index text and falling back to the collision failsafe only when that text changed on both sides. The base index comes from the last-synced tree that #53 persists in .okf/sync-state.json. Fixes #52
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #51
Description
okf hub push|pull|synceach spawn a freshsync.Engineper CLI invocation, but the engine's sync position (cachedTree/cachedHead,pkg/sync/engine.go:45-46) lived only in memory. This PR persists that position to a hidden state file inside the bundle so consecutive invocations behave like one continuous session.pkg/sync/state.go:{version, vault_id, head, tree}is persisted to<bundle>/.okf/sync-state.json.ScanBundleskips hidden directories, so the state file is never mistaken for bundle content.NewEngineloads it at startup (engine.go:61); missing, corrupt, or foreign-vault state silently degrades to the previous empty-start behavior.saveState()performs an atomic write (temp file + rename inside the hidden dir,pkg/sync/state.go:107-125) and is called on the three success paths — pushpkg/sync/engine.go:212, pullpkg/sync/engine.go:307, syncpkg/sync/engine.go:523.TestHub_Serve_E2E_Socket(pkg/sync/hub_test.go:349-373) previously locked in a pseudo-conflict: device B's local edit was flagged as a collision with no remote change. The test now makes device A edit and pushnote_b.mdbefore device B syncs, so the collision is genuine; all original conflict-fork assertions are preserved.Motivation & Context
Root cause:
Engine.cachedHead/cachedTree(pkg/sync/engine.go:45-46) are pure in-memory fields;NewEngine(pkg/sync/engine.go:50) starts empty. In a new processexpectedHeadis always nil (pkg/sync/engine.go:182), so the hub's compare-and-swap (pkg/sync/server.go:129,expected != current) rejects the second push with409 head conflict. SimilarlyReconcile(pkg/sync/engine.go) diffs against the empty cached tree, so on a single machine every local file looks like a collision and is moved to<file>.conflict-local.md— sync appeared to "revert" local edits.See the triage in this comment.
Changes Made
pkg/sync/state.go— versioned sync-state schema, load (with corrupt/foreign-vault fallback) and atomic saveNewEngine(pkg/sync/engine.go:61)pkg/sync/engine.go:212,307,523)pkg/sync/hub_test.go)TestEngine_PushAcrossEngineInstances_SecondPushNotConflict,TestEngine_SyncAcrossEngineInstances_LocalEditsKept,TestEngine_SyncStateFile_*Verification & Pre-Submission Checklist
All commands below were executed on this exact tree (
db7204a,fix/51-persist-sync-state).make testpasses with 100% test success —timeout 600 make test(go test -v -race ./...) → exit 0, all greenmake validatepasses onknowledge/— 23 concept(s), 0 error(s), 0 warning(s), 0 orphan(s), Conformantmake validate-examplespasses on all sample corpora — exit 0, all governance/symlink checks passedmake fmtandmake vetshow clean code —make fmtleft the tree unchanged (git statusclean);make vetexit 0;gofmt -l .no output;go vet ./pkg/sync/cleanRed/green evidence for the fix
Red (before the fix, recorded during independent acceptance of this change):
timeout 300 go test ./pkg/sync/ -run 'TestEngine_PushAcrossEngineInstances|TestEngine_SyncAcrossEngineInstances|TestEngine_SyncStateFile' -v→ 3 FAIL (second push409 head conflict; single-machine sync pseudo-conflict moving edits into*.conflict-local.md; state file missing), 1 PASS (ForeignVaultIgnored, which passes even empty-to-empty). Green (after the fix, re-run on this tree): same command → 4/4 PASS. Note: a-run TestPush…pattern matches 0 tests in this package (there are noTestPush*functions), so the two cross-engine-instance regressions above serve as the red/green proof.Regression (re-run on this tree)
timeout 600 go test ./pkg/sync/ -count=1→ok … 1.130s, 26/26 top-level tests PASS (13 pre-existing engine/vault tests + 6 hub tests + new regression tests)go test ./...→ 8 packagesok, 0 FAIL (cmd/okf-benchmarkhas no test files)timeout 600 make test→ exit 0 (with-race)AI Disclosure
This change was produced by an AI-assisted contributor (see disclosure in the issue claim comment) working under human direction; the fix passed independent acceptance review before submission.