Skip to content

feat(api,ui): revision-checked knowledge edit, delete and restore - #1987

Open
devin-ai-integration[bot] wants to merge 15 commits into
mainfrom
devin/1791156105-mem-03-knowledge-edit
Open

devin-ai-integration[bot] wants to merge 15 commits into
mainfrom
devin/1791156105-mem-03-knowledge-edit

Conversation

@devin-ai-integration

@devin-ai-integration devin-ai-integration Bot commented Oct 5, 2026 •

Copy link
Copy Markdown
Contributor

Summary

MEM-03: knowledge entries can now be edited, deleted and restored from /ui/…/knowledge/:id. Every write checks the revision, and each confirmation and receipt shows its .lore.md, AGENTS.md and sync consequences. Closes #1805. Part of the P2 gate of #1824.

Stacked on #1986 (MEM-02). #1983 is merged and #1986 has been rebased onto main. Review only the commits after 558408f8. Once #1986 squash-merges, this branch gets rebased with git rebase --onto origin/main 558408f8.

Core: packages/core/src/knowledge-edit.ts (exported as knowledgeEdit)

  • editKnowledge(id, { expectedRevision, actor, title?, content?, category?, confidence?, scope? })
    • Runs in a single transaction against the exact current head.
    • Title, content, category and scope changes append one version. Scope is stored as the versioned cross_project, through a new optional crossProject override on ltm.appendVersion.
    • Confidence goes through the existing mutable register (ltm.update) and appends no version.
    • An edit that changes nothing does nothing and does not export.
    • A shared entry with no project can't be changed to project scope.
    • After commit it re-embeds on a new version, invalidates caches and regenerates .lore.md.
  • deleteKnowledgeChecked(id, { expectedRevision, actor }) writes a tombstone only if the head is unchanged.
  • restoreKnowledge(id, { expectedRevision, actor, versionId? }) appends the chosen live version, or the latest live one, as a new revision. It returns restored_from. Tombstone targets and the version that is already current are refused, and history is never rewritten.
  • knowledgeEffects(id) returns { scope, project_id, revision, is_deleted, lore_file{enabled,path,affected}, agents_file{enabled, mode: pointer|inline|off, immediate:false}, sync{enabled} }.
  • Errors are KnowledgeEditError, coded invalid_request | not_found | deleted | stale_revision | title_conflict. Stale and deleted errors carry expected_revision and current_revision. Edit and checked delete refuse a tombstone head as deleted before comparing revisions; restore accepts a tombstone head.
  • Title conflicts follow the shared-title rule below. Unlike ltm.update, an edit refuses a colliding title with 409 instead of silently dropping it.

Gateway: knowledge-edit-api.ts and routes/knowledge.ts

Route Notes
PATCH /api/v1/knowledge/:id body {expected_revision, title?, content?, category?, confidence?, scope?, actor?}, unknown keys → 400
POST /api/v1/knowledge/:id/restore body {expected_revision, version_id?, actor?}
GET /api/v1/knowledge/:id/effects read-only consequences
DELETE /api/v1/knowledge/:id?expected_revision=N checked delete. Without the parameter, the legacy DELETE stays byte-identical (tested)

Status codes:

  • 400 for malformed or invalid input
  • 403 for writes in hosted mode
  • 404 for a missing entry
  • 409 for stale_revision, deleted and title_conflict
  • an empty-body 404 off loopback (management plane)

The existing GET /knowledge/:id and /versions routes still resolve; the route-registry test covers this.

UI

  • KnowledgeEditor.tsx is an inline editor on the knowledge document, with title, content, category, confidence and a project/shared scope switch. A projectless entry is pinned to shared.
    • Drafts live only in IndexedDB drafts, with baseRevision, and are written on a debounce or on Cancel. Nothing reaches the gateway until Save. A banner reads "Unsaved draft from … (based on vN)" with Resume and Discard.
    • Conflicts: a resumed draft based on an older revision, or a 409 stale_revision on save, keeps the draft. It shows the server version and the draft side by side, and asks for an explicit "Continue editing on vM" rebase before saving.
    • title_conflict shows inline. A hosted 403 saves the draft and locks the editor.
    • Save sends only the fields that differ from the current server entry. Confidence changes (decay, curator) don't bump the revision, so sending every field would silently overwrite them.
    • On success it shows Saved as vN · <lore effect> · <AGENTS effect> · <sync>.
  • Delete fetches /effects first. If the revision moved, the entry reloads before the confirmation opens. The confirmation lists the consequences and sends the checked revision, and the entry then opens in the deleted-entry view.
  • Restore (RestoreKnowledgeAction.tsx) is available on the deleted-entry view from MEM-02 and on each superseded live version in History. It replaces the MEM-02 "Restoring arrives with knowledge editing (MEM-03: Safe knowledge edit/delete with versioned restoration and visible sync/export effects #1805)" placeholder. The confirmation, stale reload and receipt work the same way as delete.
  • After a write, caches are reconciled or evicted. Every project collection is invalidated when a shared entry is involved.
  • All user content is rendered as inert text.

Shared-title uniqueness (owner request)

Rule: among live entries visible as shared (project_id IS NULL OR cross_project = 1), titles must be unique after normalisation: ltm.normalizeTitleKey(t) = t.trim().toLowerCase(), or ltm.titleKeySql(col) in SQL. Internal whitespace is not collapsed, so X Y and X Y count as different titles. Core enforces the rule on every path that makes an entry shared-visible (ltm.findSharedTitleConflict / findTitleConflict). No unique index or migration was added, because existing data may already contain duplicates (see below).

User-facing paths reject with a 409 title_conflict. The error carries the additive conflicting_entry: { id, title, project_id, scope }, which names the existing entry:

  • PATCH /knowledge/:id and restore: editKnowledge and restoreKnowledge check the target state whenever the normalised title or the scope changes, so a project→shared flip is checked too. Two exceptions are allowed:

    • a cosmetic retitle such as X→x;
    • a content-only edit of an existing duplicate.

    Errors are checked in the order deleted → stale_revision → title_conflict.

  • Move: reassignKnowledge, POST …/move and lore data move knowledge check inside the transaction. A refused move leaves the project, the scope and the transfer rows unchanged. The API throws TitleConflictError, which becomes the same 409. The CLI names the existing entry and points to duplicate review.

  • UI: the editor and Restore show the existing title (as inert text) with a link to it (knowledgeHref or globalKnowledgeHref) and a Review duplicates link to duplicatesHref. If the details are absent or malformed, they fall back to the generic message.

Automated paths never throw or drop knowledge. They reuse the existing ltm.create/update dedup guard, now with normalised titles:

  • create / tryCreate: a title that matches a shared entry merges into that entry (content and metrics) instead of creating a duplicate, as the existing exact-title guard already did. Explicit-ID shared and projectless creates are covered too. This applies to the curator, to the entries pattern extraction hands to ltm.create, and to structured import.
  • update: a colliding automated retitle keeps the old title and still applies the other fields. This is the existing ltm.update behaviour.
  • promoteCrossProject: a candidate that would collide stays project-scoped with its content intact. It is listed in the additive conflicts result and in the curator log, and a dry run reports the same list, including collisions between candidates in the same run.
  • .lore.md import stays project-scoped (crossProject: false), so it never creates a shared entry. A hand-written entry whose title matches a shared entry merges into it without loss, and an unknown UUID is created project-only.

Existing duplicates: nothing is renamed or deleted. ltm.listSharedTitleDuplicates() reports them. The dedup preview gains an additive shared_title_conflicts field, and the duplicate-review page shows a read-only Shared title conflicts section that links each entry. Merging across projects is #1980.

While doing this I found that reassignKnowledge sets cross_project = 1 on project→project moves, which makes a project-only entry shared. Filed as #1989 and not changed here.

.lore.md copy follows the real exporter. buildSection exports every current entry whose project_id is this project, including project-owned shared (cross_project = 1) entries, even though its comment says otherwise. That mismatch is filed as #1988. So lore_file.affected is project_id !== null. Only entries without a project say ".lore.md files are not affected (entries without a project are not exported)".

AGENTS.md copy is honest: pointer mode says "AGENTS.md pointer unchanged". Inline mode says the section updates on the next idle export. Nothing claims an immediate AGENTS change.

Notes / open questions

  • The actor is the constant "lore-ui", the same as MEM-02.

Tests

Shared-title commits (9b20940b..df6b1eb8), full gate on df6b1eb8:

  • pnpm install --frozen-lockfile, pnpm --filter @loreai/core build, pnpm run typecheck, pnpm run lint (existing warnings only), pnpm run format:check: pass
  • pnpm test (full): 549 files passed, 2 failed, 17 skipped. Tests: 12,336 passed, 5 failed, 229 skipped. The 5 failures are the known sandbox ones that also fail on clean main: four git URL rewrites and X-Lore-Git-Remote.
  • Focused Vitest run (core shared-title, ltm, knowledge-edit, curator and import; gateway api, cli-data-contract and dedup): 25 files, 493 tests passed
  • pnpm --filter @loreai/ui test: 50 files, 849 tests passed
  • pnpm --filter @loreai/ui build, pnpm --filter @loreai/gateway run bundle, node scripts/ui-deep-link-smoke.mjs, pnpm run build: pass
  • pnpm --filter @loreai/ui test:e2e: 203 passed, 11 skipped (the existing viewport-specific skips)

New shared-title tests:

  • core test/shared-title.test.ts:
    • trim and case variants collide, but internal whitespace doesn't
    • two projects each have a project-only X: sharing one is allowed; sharing the other is refused and its row and revision stay unchanged
    • a changed title and scope is checked against the next scope
    • cosmetic retitles and content-only edits of legacy duplicates are allowed
    • a restore conflict is reported
    • shared and projectless create/tryCreate merge
    • an automated ltm.update retitle collision is handled
    • a curator crossProject create is merged
    • a refused move leaves rows and transfers unchanged
    • listSharedTitleDuplicates is deterministic and read-only
    • structured global import
    • a .lore.md title variant merges without loss, and an unknown UUID stays project-only
    • promotion collisions in dry and real runs
  • gateway test/api.test.ts and test/cli-data-contract.test.ts: the PATCH, restore and move 409 envelopes with conflicting_entry; the CLI move refusal
  • ui test/api-client.test.ts, test/knowledge-edit.test.tsx and test/duplicate-review.test.tsx:
    • conflict links for both project and projectless targets
    • a hostile title stays inert
    • malformed details fall back to the generic message
    • the shared-title-conflicts section is shown, or absent when there are none
  • e2e knowledge-edit.spec.ts and dedup-review.spec.ts (seeded in seed.mjs): a sharing conflict with its links, and the duplicate-review section

985a16e1 fixes a bug found during browser verification. After an edit, delete or restore, Browse now calls refreshAfterKnowledgeWrite(), which reloads the entry, the versions, the active knowledge-page loaders and ws.projects. Before this, a delete followed by browser Back left the entry in the sidebar with a stale count. KnowledgeEditor gains onSaved.

  • ui test/shell.test.tsx: after a delete, a restore and a retitle, the sidebar and counts refresh.
  • e2e knowledge-edit.spec.ts: after a delete, browser Back shows the entry gone from the list and the count down, without a reload.
  • Results: typecheck, lint and format pass; UI 851 passed; UI build and gateway bundle pass; e2e 203 passed, 11 skipped.

Earlier MEM-03 commits:
Full gate run on cd9367bc (base origin/main fde28620). The follow-ups were re-verified with the commands that cover them:

  • b308bf27 (only changed fields are sent): UI suite 799 passed; E2E 195 passed, 11 skipped.
  • 44fdc3ce (the autosave re-arms on every edit; export effects follow project_id): focused core knowledge-edit.test.ts 10 passed; knowledge-edit.spec.ts E2E 4 passed. The full UI suite had 800 passed and 1 failed: the in-session-search virtualization assertion in session-view.test.tsx, which this PR doesn't touch. That spec passes in isolation on this branch and on clean main (66/66 each); being tracked in CI.
  • 16df62ca (the editor handles deleted refusals, from the Seer review): UI suite 803 passed.
  • 138f2018 (edit and checked delete report deleted before stale_revision, from the Seer review): focused core and gateway tests 101 passed; UI 803 passed.
  • typecheck, lint, format, UI build and gateway bundle pass on every head.

Full gate on cd9367bc:

  • pnpm install, pnpm run typecheck, pnpm run lint (warnings only, exit 0), pnpm run format:check: pass
  • pnpm test (full): 545 passed, 17 skipped, 7 failures, all in the sandbox-noise classes. Five reproduce on clean origin/main: four git URL rewrites and the gateway X-Lore-Git-Remote header. The other two are agents-file mtime tests, the same flaky class documented on feat(ui): apply reviewed dedup decisions with receipts and recovery #1986 (repeated runs of that file fail a different subset each time). This PR doesn't touch agents-file or the export code.
  • pnpm run build, pnpm --filter @loreai/core build, pnpm --filter @loreai/ui build, pnpm --filter @loreai/gateway run bundle: pass
  • node scripts/ui-deep-link-smoke.mjs: pass
  • pnpm --filter @loreai/ui test: 48 files, 794 tests passed
  • pnpm --filter @loreai/ui test:e2e: 195 passed, 11 skipped (the existing viewport-specific skips)
  • packages/gateway/test/sync.property.test.ts ×10: all passed

New tests:

  • core test/knowledge-edit.test.ts:
    • a second tab's stale edit is refused
    • a dedup apply racing a PATCH makes the PATCH stale
    • a confidence-only edit appends no version
    • scope changes append a version and can be restored
    • a projectless entry can't be changed to project scope
    • a stale restore after a re-delete is refused
    • a restore whose title is now taken is refused
    • no-op and refused edits don't export
    • calls inside a caller-owned transaction are refused
  • gateway test/api.test.ts:
    • PATCH, restore, effects and checked-DELETE status codes
    • malformed and unknown-key bodies
    • hosted-mode 403
    • the legacy DELETE response stays byte-identical
  • gateway test/management-access.test.ts: an off-loopback request to each new route gets an empty-body 404 and mutates nothing
  • gateway test/route-registry.test.ts: the new routes don't shadow GET /knowledge/:id or /versions
  • ui test/knowledge-edit.test.tsx (28 cases):
    • drafts: debounce, resume, Cancel-persist, storage unavailable
    • a stale draft needs an explicit rebase
    • only changed fields are sent; a confidence change on the server isn't clobbered; a no-op sends nothing
    • title conflict; hosted lock that keeps the draft
    • a deleted refusal on save keeps the draft and reloads; on delete it counts as done
    • delete and restore effects, with the stale reload
    • shared-scope invalidation
    • content longer than 1200 characters is preserved
  • ui test/api-client.test.ts: the new client methods and how the error type is surfaced
  • e2e e2e/knowledge-edit.spec.ts, on disposable per-viewport entries:
    • nothing is sent before Save
    • the exact PATCH body
    • a stale draft is retained, followed by delete and restore
    • restore of a superseded version
    • hostile markup stays inert

Definition of done

  • Knowledge can be edited with a mandatory revision check. A stale UI gets a 409 and keeps its draft.
  • Checked delete with visible consequences; the legacy DELETE is unchanged.
  • Restore from history appends a new version; history is never rewritten.
  • .lore.md, AGENTS.md and sync effects are shown before and after every write.
  • Drafts are local only and revision-based; there is an explicit rebase on conflict.
  • Hosted-mode refusal, malformed input and off-loopback 404 are covered by contract tests.
  • Shared-title uniqueness is enforced in core: user paths get a 409 that names the existing entry, automated paths merge or keep the old title, and existing duplicates are reported without changes.
  • Adversarial races: two tabs, PATCH vs dedup apply, restore after a re-delete, a restore title that is now taken, shared→project, confidence-only.
  • Playwright coverage, plus screenshots in desktop and mobile, light and dark.

Screenshots

Desktop light Desktop dark Mobile light Mobile dark
Editor + draft banner
Stale conflict
Save receipt
Delete confirm
Restore confirm

Link to Devin session: https://app.devin.ai/sessions/36dfe06b1fff4f36932bc410ca42a2cc
Open in Devin Desktop: https://app.devin.ai/desktop/session/36dfe06b1fff4f36932bc410ca42a2cc?variant=devin
Requested by: @BYK

@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".

  • Disable automatic comment, CI, and merge conflict monitoring

@github-actions

github-actions Bot commented Oct 5, 2026 •

Copy link
Copy Markdown
Contributor

Coverage Results 📊

✅ Patch coverage is 89.85% (292 of 325 changed executable lines covered; target 80%).
Project statement coverage is 85.84% (up 0.84 percentage points from base (34a4d85) to head (6fc42ad)).

Changed files with executable lines (13)
File Patch coverage Changed executable lines
packages/core/src/knowledge-edit.ts 88.11% 126/143 covered; missed: 135, 137, 161, 240, 241, 266, 299, 301, 303, 310, 316, 393, 439, 467, 495, 497, 525; partial branches: 131, 136, 160, 255, 265, 298, 300, 302, 304, 311, 341, 392, 406, 438, 466, 494, 496, 524
packages/gateway/src/knowledge-edit-api.ts 93.85% 61/65 covered; missed: 53, 98, 123, 187; partial branches: 31, 52, 71, 107, 122, 186, 194
packages/core/src/ltm.ts 91.67% 55/60 covered; missed: 734, 735, 736, 737, 738; partial branches: 733, 5025
packages/gateway/src/api.ts 78.57% 11/14 covered; missed: 457, 463, 488; partial branches: 456, 465, 468
packages/gateway/src/cli/data.ts 60.00% 6/10 covered; missed: 2240, 2242, 2243, 2252; partial branches: 2246
packages/core/src/curator.ts 100.00% 1/1 covered; partial branches: 1055
packages/core/src/data.ts 100.00% 4/4 covered
packages/core/src/import/structured.ts 100.00% 1/1 covered
packages/gateway/src/dedup-api.ts 100.00% 2/2 covered
packages/gateway/src/routes/knowledge.ts 100.00% 12/12 covered
packages/ui/src/contracts/dedup.ts 100.00% 6/6 covered
packages/ui/src/contracts/error.ts 100.00% 3/3 covered
packages/ui/src/contracts/knowledge.ts 100.00% 4/4 covered
Coverage diff
@@            Coverage Diff             @@
##          main     #1987       +/-##
==========================================
+ Coverage    85.00%    85.84%    +0.84%
==========================================
  Files          318       334       +16
  Tracked lines     49482     51789     +2307
  Branches     40478     42780     +2302
==========================================
+ Hits         42061     44455     +2394
+ Misses        7421      7334       -87
- Partials      4285      4483      +198

Generated by Coverage Action

Comment thread packages/gateway/src/knowledge-edit-api.ts
@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

Manual browser verification. Run against isolated real gateways (local on 7995, and hosted with LORE_HOSTED_MODE=1 on 7994) in Chromium, at desktop width and at 390px, in light and dark themes.

Round 1 at b308bf27: two discrepancies, both fixed in 44fdc3ce

  1. Autosave dropped later edits. Steps: change the title, wait for the first autosave, change the content, reload, then Resume. The title came back but the content was lost. Cause: the debounce effect didn't track the form fields. It now tracks them, and a regression test covers it.
  2. Wrong copy for a project→Shared flip. The success line said the project .lore.md was regenerated, and the next save claimed shared entries aren't exported, yet the file still contained the entry. Cause: the exporter (buildSection → forProject(path, false)) exports every entry whose project_id is this project, including cross_project = 1 entries, despite its comment; filed as .lore.md export includes project-owned shared (cross_project=1) entries, contrary to buildSection comment #1988. The effects now follow the exporter (affected = project_id !== null).

Passed in round 1:

  • A changed-field save sends exactly one PATCH, containing only expected_revision plus the changed fields. It creates a new history version, and the real .lore.md on disk is updated.
  • Saving with nothing changed shows "No changes to save" and sends no PATCH.
  • Drafts stay in IndexedDB only, with no network write before Save. The banner appears after a reload, and Discard removes the draft.
  • With two real tabs, the second save gets a 409. It shows the server version (v3) next to the draft; an explicit "Continue editing on v4" then saves.
  • An out-of-band confidence-only PATCH keeps the revision unchanged. A later title-only UI save omits confidence, and the server keeps 0.37.
  • A duplicate title shows an inline error (409 title_conflict) and appends no version.
  • An entry without a project has the Project radio disabled.
  • Delete: Escape and Cancel send nothing, and the confirmation sends expected_revision. If the entry is edited while the confirmation is open, the delete gets a 409 and the confirmation reloads its effects before a reviewed delete.
  • Restoring from the deleted view and restoring a superseded version both append a new version, and the tombstone stays in history.
  • Hostile <img onerror> title and content stay literal through edit, delete and restore. No image is injected and window.__pwned stays undefined.
  • The hosted gateway returns 403. The UI shows "Editing unavailable" and keeps the whole draft in IndexedDB.
  • Tab, Enter and Escape work through the forms and dialogs.
  • No horizontal overflow; the mobile dialog is 388px wide in a 390px viewport. No page errors.

Round 2 at 44fdc3ce: targeted re-check, all passed

  • (a) The autosave sequence keeps both edits after reload and Resume, at desktop/light and 390px/dark. IndexedDB holds both values.
  • (b) A scope-only PATCH {"expected_revision":1,"scope":"shared"} and a later title-only save both say "Project .lore.md regenerated". Their responses have affected:true, regenerated:true, and the file still contains the entry.
  • (c) A save on an entry without a project shows ".lore.md files are not affected (entries without a project are not exported)", with affected:false.

Not covered: sync-enabled and inline-AGENTS effects (neither is configured in the fixture). Round 2 didn't repeat every flow in all four viewport/theme combinations.

Resumed draft (desktop, light) Resumed draft (mobile, dark)
Project-owned shared entry: save Entry without a project: save
Two-tab conflict (desktop, dark) Stale delete refused
Restored history, tombstone kept Hosted-mode refusal

Written by Devin

Comment thread packages/core/src/knowledge-edit.ts
Comment thread packages/ui/src/routes/Browse.tsx
BYK and others added 4 commits October 5, 2026 08:40
…1804)

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
…ly (#1804)

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The shell's Folk status loader (#1982) now reads sync status once per
workspace, so the apply test counts the confirmation's own read as a delta.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@devin-ai-integration
devin-ai-integration Bot force-pushed the devin/1791156105-mem-03-knowledge-edit branch from 8a3b398 to df6b1eb Compare October 5, 2026 08:46
@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

Manual browser check of shared-title uniqueness at df6b1eb8, against a real bundled gateway with isolated data (port 7993, temporary DB).

  • PASS: Two projects each have a project-only X. The first one becomes Shared. The second is refused with 409 title_conflict and is still Project v1 after a reload.
  • PASS: The error names the existing entry. Its link (project route, or /ui/knowledge/:id for a projectless entry) and the Review duplicates link both open the right page.
  • PASS: The trim/case variant " x " conflicts with X. "X Y" and "X Y" both become Shared, since internal whitespace isn't collapsed.
  • PASS: A cosmetic X→x on the shared entry saves (v3). A content-only edit saves (v4) and sends only expected_revision and content.
  • PASS: Delete the shared entry, share a replacement with the same title, then restore the deleted one: it is refused with the linked conflict, and the tombstone stays at v5.
  • PASS: The 409 body contains conflicting_entry {id, title, project_id, scope}.
  • PASS: A hostile title <img src=x onerror=alert(1)> renders as literal text in both the message and the link. No image was injected, no dialog opened and no page errors were captured.
  • PASS: The legacy Shared title conflicts section on the duplicates page links every entry and has no buttons, inputs or merge actions. Renaming one entry of the pair removes it after a Rescan, and the response has shared_title_conflicts: [].
  • PASS: Checked at desktop and 390px, in Light and Dark, with no horizontal overflow.
Editor conflict, desktop Light Editor conflict, mobile Dark
Editor conflict, desktop Dark Editor conflict, mobile Light
Duplicates section, desktop Light Duplicates section, mobile Dark
Duplicates section, desktop Dark Duplicates section, mobile Light
Restore conflict Hostile title stays inert

Found during the check, outside the title rule: after a delete, pressing browser Back still listed the deleted entry in the knowledge sidebar with the old count, until a full reload. The document itself correctly showed the deleted state. The entry-detail callbacks reload only the entry, not the list loaders or the projects. I'm fixing that in a follow-up commit on this PR, with unit and e2e regression tests.

Sidebar before reload

This was a targeted check of the new behaviour, not a full MEM-03 regression pass.

Written by Devin

BYK and others added 11 commits October 5, 2026 09:52
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
)

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
…bered

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
…store

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@devin-ai-integration
devin-ai-integration Bot force-pushed the devin/1791156105-mem-03-knowledge-edit branch from 985a16e to 6fc42ad Compare October 5, 2026 09:55
Comment on lines +392 to +395
setSaveError(
"This entry was deleted. Your draft is kept on this device; restore the entry from History to continue.",
);
props.reloadEntry();

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: Error messages from setSaveError() are not displayed because setEditing(false) is called first, hiding the UI element that would show the error.
Severity: MEDIUM

Suggested Fix

Refactor the component to display the saveError message outside of the conditional block that depends on editing(). This will ensure that even when the editor is closed, the user is still notified of the error that occurred during the save operation.

Prompt for AI Agent
Review the code at the location below. A potential bug has been identified by an AI
agent. Verify if this is a real issue. If it is, propose a fix; if not, explain why it's
not valid.

Location: packages/ui/src/components/lore/KnowledgeEditor.tsx#L392-L395

Potential issue: When a user attempts to save a knowledge entry and the save fails due
to concurrent deletion or because editing is forbidden, the code calls
`setEditing(false)` before `setSaveError()`. This hides the editing form, which is the
only place the error message is configured to be displayed. As a result, the editor
closes without providing any feedback to the user, leaving them unaware of why their
changes were not saved. The error message is never rendered because its container is
hidden or unmounted before the error state is set.

Also affects:

  • packages/ui/src/components/lore/KnowledgeEditor.tsx:415~417

setActiveApplyKey(record.key);
if (!(await submitApply(record, true))) break;
}
} finally {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: An unhandled error in applyAccepted occurs if a group's status changes while the confirmation dialog is open, breaking the UI.
Severity: MEDIUM

Suggested Fix

Wrap the call to applyPlans() within a try-catch block inside the applyAccepted function. The catch block should handle the error by displaying a notification to the user and preventing the application from entering a broken state.

Prompt for AI Agent
Review the code at the location below. A potential bug has been identified by an AI
agent. Verify if this is a real issue. If it is, propose a fix; if not, explain why it's
not valid.

Location: packages/ui/src/components/lore/DuplicateReview.tsx#L469

Potential issue: The `applyAccepted` function can fail with an unhandled error. This
occurs if a user opens the confirmation dialog and then interacts with the background
UI, changing a group's status from 'accepted' to something else (e.g., 'skipped').
Because the dialog does not block mouse clicks on background elements, this is a
realistic user action. When the user then confirms the action, `applyAccepted` calls
`applyPlans()`, which throws an error because the group's state has changed. The missing
`try-catch` block around this call results in an unhandled promise rejection, leaving
the UI in a broken state with the dialog open.

BYK added a commit that referenced this pull request Oct 5, 2026
## Summary

Promoted shared entries (`project_id = P AND cross_project = 1`) were
never dedup candidates: the project run took `forProject(P)` (which
*included* them, so the curator's automatic `deduplicate(..., { dryRun:
false })` could even remove a promoted row in favour of a private
survivor), and the shared run only took `project_id IS NULL`. Dedup now
works over three explicit pools:

| pool | rows compared | group `scope` (unchanged enum) | survivor |
|---|---|---|---|
| `project` | private(P) × private(P) — `project_id = P AND
cross_project = 0` | `project` | existing ranking |
| `project_shared` (new) | private(P) × shared-visible | `project` |
**always the shared entry** |
| `shared` | shared-visible × shared-visible — `project_id IS NULL OR
cross_project = 1` | `global` | existing ranking |

A project run never pairs two entries owned by other projects; those are
left to the shared run.

**Core (`ltm.ts`)**
- `deduplicate(projectPath)` / `deduplicateGlobal()` use their own pool
queries; `forProject()` is unchanged for other callers.
- New `deduplicateAgainstShared(projectPath, { exclude })`: always a dry
run. It scores every private×shared pair with the same two signals as
`_dedup` (title overlap ≥ 0.7 with ≥ 4 words, or cosine ≥ threshold).
Each private entry goes to its single best shared match; ties use the
survivor ranking. That gives one cluster per shared keeper, and the
shared entry always survives. `exclude` matches version id or logical
id.
- `_dedup` and the new function share `loadDedupEmbeddings` /
`scoreDedupPair`. Titles are tokenized once per run (`titleTermSet` /
`titleOverlapSets`); `titleOverlap()` delegates to them, so its results
are unchanged.

**Apply (`dedup-apply.ts`)**: the scope check is role-aware. Revision
checks, per-group transactions, tombstones, provenance, receipts,
idempotent replay and recovery are untouched.
```
projectId === null : keep + merges must all be shared-visible
projectId === P    : merges must be private(P); keep may be private(P) or shared-visible
otherwise          : scope_mismatch   (no new refusal reason)
```
So a private entry can be folded into a shared keeper. A shared entry is
never removed in favour of a private one, and the survivor never changes
visibility.
- **Keep the dropped entry's content (`contentFromId`, additive and
opt-in):** a decision may name one of its `mergeIds` as `contentFromId`.
- That entry's title and content become a new version of the survivor:
same logical id, `project_id`, `cross_project` and category/metadata.
The source is then tombstoned like any other merge.
- This is how a "Project + shared" group keeps the private copy. The
shared entry stays shared and keeps every reference, only its content
changes, and the private content becomes shared because the reviewer
chose it.
- Revision checks still cover both entries, and provenance records
source→survivor.
- The receipt gains `contentFrom: { id, revision, replacedKeepRevision,
keepVersionId }`. `keepRevision` is the survivor's revision after apply,
so it stays usable as the next expected revision.
  - The survivor's earlier version stays in history for recovery.
  - Replays return the stored receipt.
- A new title that collides with a third live entry in the survivor's
scope (`ltm.titleCollides`, now exported) refuses the whole group with a
new `title_conflict` reason and rolls it back. The group's own merges
never count as collisions.
- Requests without `contentFromId` behave, and hash
(`dedupApplyPayloadHash`), byte-identically to before.
- `.lore.md` export now runs once per touched project:
`request.projectId` plus the origin project of every merged row. A
shared-pool merge can tombstone a promoted Q row, and Q's `.lore.md`
lists it, because `buildSection` exports `forProject(Q)`, which includes
promoted rows. With `contentFromId`, a promoted survivor's origin
project is exported too. Replays and fully refused operations still
export nothing.

**Gateway (`dedup-api.ts`)**: additive fields only. Each group gets
`pool: "project" | "shared" | "project_shared"`, and the response gets
`project_shared: DedupResult`. Existing `scope`, `project`, `global` and
the group-id prefixes (`project:`, `global:`) are unchanged; new groups
use `project_shared:`. The preview runs project → shared →
project_shared. Entries already clustered by the first two runs are
passed as `exclude`, by both version id and `logicalIdOf(id)`. The runs
are async, so a clustered entry may already be on a newer version. A
logical id therefore appears in at most one group. Groups are ordered
project, project_shared, shared.

**UI (small label only)**: `contracts/dedup.ts` accepts `pool` /
`project_shared`. The group label in `DuplicateReview.tsx` is now
pool-based: `Project` / `Project + shared` / `Shared`. The old `Shared
(no project)` label became wrong once promoted entries joined the shared
pool. Apply grouping still keys off `scope`, so `project_shared` groups
apply under the project, which the new scope rule allows.

![desktop
light](https://app.devin.ai/api/presigned_proxy?token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ1c2VyX2lkIjpudWxsLCJidWNrZXRfbmFtZSI6ImRldmluYXR0YWNobWVudHMiLCJidWNrZXRfa2V5IjoiYXR0YWNobWVudHNfcHJpdmF0ZS9vcmctODQ3Y2E3ZDFjOTIzNDZiMzhmOGM4NTZmZmYzZmY5MDUvMjc5NDlhZTUtYTc1ZS00ZTczLWEyMmItZWNkOWI0NWI3YzkyIiwiaWF0IjoxNzkxMTk2Mzc4LCJleHAiOjE3OTE4MDExNzgsIm9yZ19pZCI6Im9yZy04NDdjYTdkMWM5MjM0NmIzOGY4Yzg1NmZmZjNmZjkwNSIsImZpbGVuYW1lIjoiZGVkdXAtZGVza3RvcC1saWdodC5wbmcifQ.H6rwH1VSUA5yLWwdPnowMFsiyxdRLQ5w_ynyfYTEzFY)
![desktop
dark](https://app.devin.ai/api/presigned_proxy?token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ1c2VyX2lkIjpudWxsLCJidWNrZXRfbmFtZSI6ImRldmluYXR0YWNobWVudHMiLCJidWNrZXRfa2V5IjoiYXR0YWNobWVudHNfcHJpdmF0ZS9vcmctODQ3Y2E3ZDFjOTIzNDZiMzhmOGM4NTZmZmYzZmY5MDUvNDEyYzY1YWUtMzJiYS00ODNlLTllMWItN2RjMmYzYzIwOGJlIiwiaWF0IjoxNzkxMTk2Mzc4LCJleHAiOjE3OTE4MDExNzgsIm9yZ19pZCI6Im9yZy04NDdjYTdkMWM5MjM0NmIzOGY4Yzg1NmZmZjNmZjkwNSIsImZpbGVuYW1lIjoiZGVkdXAtZGVza3RvcC1kYXJrLnBuZyJ9.Fv085zAxK259WppncfgigdLeg5WbkCy2M3K_3NqFGvo)
<img
src="https://app.devin.ai/api/presigned_proxy?token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ1c2VyX2lkIjpudWxsLCJidWNrZXRfbmFtZSI6ImRldmluYXR0YWNobWVudHMiLCJidWNrZXRfa2V5IjoiYXR0YWNobWVudHNfcHJpdmF0ZS9vcmctODQ3Y2E3ZDFjOTIzNDZiMzhmOGM4NTZmZmYzZmY5MDUvYjc5ODFhNTQtNmFkMC00MTIxLTkzOTgtNzE5YjYwNzBhYTBhIiwiaWF0IjoxNzkxMTk2Mzc4LCJleHAiOjE3OTE4MDExNzgsIm9yZ19pZCI6Im9yZy04NDdjYTdkMWM5MjM0NmIzOGY4Yzg1NmZmZjNmZjkwNSIsImZpbGVuYW1lIjoiZGVkdXAtbW9iaWxlLWxpZ2h0LnBuZyJ9.Uq35FGMR7mp3V848S_EOLE7E5GJXsFz-hq3RDEiAUeQ"
width="300"> <img
src="https://app.devin.ai/api/presigned_proxy?token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ1c2VyX2lkIjpudWxsLCJidWNrZXRfbmFtZSI6ImRldmluYXR0YWNobWVudHMiLCJidWNrZXRfa2V5IjoiYXR0YWNobWVudHNfcHJpdmF0ZS9vcmctODQ3Y2E3ZDFjOTIzNDZiMzhmOGM4NTZmZmYzZmY5MDUvMTc4OWE5ZDAtYjM5MC00NzRiLTkxYjktNWZjMTM2ZGU1YjAxIiwiaWF0IjoxNzkxMTk2Mzc4LCJleHAiOjE3OTE4MDExNzgsIm9yZ19pZCI6Im9yZy04NDdjYTdkMWM5MjM0NmIzOGY4Yzg1NmZmZjNmZjkwNSIsImZpbGVuYW1lIjoiZGVkdXAtbW9iaWxlLWRhcmsucG5nIn0.6gSJhmD7fEzEwjwwm3VVfR1_i0Z3f2uP2ma328mUF9I"
width="300">

## Product choices (please confirm)
1. **Keeping the private copy in `project_shared` groups** works as
"replace the shared entry's content" (`contentFromId`). The preview
still suggests the shared entry as keeper. A plain keep=private,
merge=shared decision is still `scope_mismatch`, because it would remove
a shared row for every project. The UI apply flow (#1986) needs to send
`{ keepId: <shared>, mergeIds: [<private>, ...], contentFromId:
<private> }` when the user picks the private copy, and accept the
additive `contentFrom` / `title_conflict` receipt fields.
- **UI hookup is a follow-up in this PR:** the "keep the private copy"
option in the apply UI (it sends `contentFromId`) lands here once #1986
merges, after a rebase, with unit/e2e tests and browser evidence. Until
then `contentFromId` is reachable only through the API.
- **Title collisions** are reachable today only for case-insensitive
equal titles (main's `titleCollides` uses `LOWER(title)`). Once #1987
merges, its trimmed title key and shared-title uniqueness rule also
catch whitespace variants. `titleCollides` is the shared seam, so no
change is needed here.
- **No migration:** the "content replaced" fact lives in the stored
receipt (`dedup_operations.receipt`) and in the survivor's version
history. `dedup_provenance` keeps its schema (source→survivor row with
the reviewed keep revision).
2. **Origin-project export after shared-pool merges.** See above.
Heads-up for MEM-02 (#1986): its confirm copy says shared groups don't
affect `.lore.md` ("shared entries are not exported"). That is no longer
true when the merged shared entry is a promoted row.
3. **Cost of the wider shared pool.** The shared run is still O(m²) and
runs synchronously on the preview request, as it did before. The pool is
now bigger. Numbers are below. If real shared pools reach the thousands,
moving the preview off the event loop would be a follow-up.

## Benchmarks
Script: `~/bench-dedup/bench.mts` (not committed). Synthetic normalized
768-d embeddings, median of 5 runs, wall-clock ms, same machine.

| | before | after |
|---|---:|---:|
| `deduplicateGlobal` m=200 | 96.0 | 56.9 |
| `deduplicateGlobal` m=1000 | 2228.2 | 1067.4 |
| `deduplicateGlobal` m=2000 | 9859.7 | 5896.9 |
| `deduplicateAgainstShared` 200×200 | 117.3¹ | 96.4 |
| `deduplicateAgainstShared` 200×1000 | 522.7¹ | 237.6 |
| `deduplicateAgainstShared` 200×2000 | 2108.7¹ | 465.4 |

"Before" for `deduplicateGlobal` is `main`. ¹ This function doesn't
exist on `main`; these numbers are from this branch before the
title-tokenization change.
Realistic preview (500 shared, 200 private): `deduplicate` 34.6 ms,
`deduplicateGlobal` 294.3 ms, `deduplicateAgainstShared` 132.0 ms. Total
439.7 ms; the slowest single call is 294.3 ms.

## Tests
New/extended:
- `core/test/dedup.test.ts`
  - Project run ignores promoted rows.
- Shared run includes promoted Q + NULL and promoted Q + promoted R, and
excludes private rows.
  - Against-shared:
- keeps the shared entry even when the private one has higher
confidence;
    - assigns each private entry to its best match;
    - never pairs P-private with Q-private, or shared with shared;
    - honours `exclude` by id and by logical id;
- makes no writes, and returns empty when there are no shared entries.
- `core/test/dedup-apply.test.ts`
- Shared pool: merges Q-promoted into a NULL keeper; refuses private
rows.
- Project pool: merges private(P) into a NULL keeper and into a
Q-promoted keeper. It refuses:
    - NULL→private keep;
    - promoted(P)→private keep;
    - NULL keep with a Q-promoted merge;
    - a Q-private keep.
  - Keeper identity and provenance are preserved.
- Export: Q is exported once; NULL↔NULL exports nothing; Q and R are
each exported once; a replay exports nothing.
- `core/test/dedup-apply.test.ts`, `contentFromId`:
- Success with a NULL survivor and with a Q-promoted survivor: content
replaced, scope, logical id and history kept, provenance and receipt
correct, exports P (plus Q).
  - Stale survivor and stale source are refused with nothing written.
  - Replay.
  - `title_conflict` rolls back the whole group.
  - A same-title source applies.
  - Private↔private.
  - Validation: must be in `mergeIds`.
- Payload hash unchanged without the field (literal pinned from the
previous code).
- `gateway/test/api.test.ts`
- Route-level `contentFromId` on a `project_shared` group returns 200
with `contentFrom`, and the survivor is still shared. A bad
`contentFromId` returns 400.
- A `project_shared` group has `scope: "project"`, the route's project
id, and the shared entry as suggested keep; it applies through
`/dedup/apply` and the shared keeper stays unchanged.
  - Q↔R promoted pairs appear only in the shared pool.
- When all entries are mutual duplicates, no logical id repeats across
groups.
  - Additive `project_shared` is in the response.
- An entry edited between runs, so the shared cluster holds a stale
version id, still appears only in the shared group. This test fails
without `a4658a9c`.
- `gateway/test/ui-contracts.test.ts`: the real gateway preview, which
now includes a `project_shared` group, parses against the UI contract.
The fixture was regenerated.
- UI: `duplicate-review.test.tsx` checks all three labels. The e2e
`dedup-review.spec.ts` and `seed.mjs` seed a private↔promoted pair and
assert `Project + shared`.

Commands, on `2ce492c2` with base `main` @ `34a4d852`. Later commits
were rechecked as follows, plus CI:
- `a4658a9c` (preview exclude by logical id): `api.test.ts`,
`ui-contracts.test.ts`, typecheck, lint and format:check.
- `468dbede` (`contentFromId`): `dedup-apply.test.ts`, `dedup.test.ts`,
`api.test.ts`, `ui-contracts.test.ts`, `data-dedup-policy.test.ts`
(251/251), typecheck, lint, format:check, build and the gateway bundle.
- `eb9eb3d7`: pins timestamps on the dedup seeds in
`ui-contracts.test.ts`. `/knowledge` sorts by `updated_at DESC, id
DESC`, and same-millisecond seeds with random v4 ids made the `limit: 1`
page nondeterministic. Checked with `ui-contracts.test.ts` ×10, `pnpm
--filter @loreai/ui test` (803), typecheck, lint and format:check.
`knowledge-all-page.json` is identical to `main` again.
- Full `pnpm test` at `468dbede`: 12,317 passed, 9 failed, all in known
sandbox classes (git URL rewrite ×4, gateway remote header ×1,
agents-file mtime ×3) plus the `ui-contracts` ordering flake that
`eb9eb3d7` fixes.

Full list for `2ce492c2`:
- `pnpm install`, `pnpm run typecheck`, `pnpm run lint` (pre-existing
warnings only), `pnpm run format:check`, `pnpm run build`: pass.
- `npx vitest run packages/core/test/dedup.test.ts
packages/core/test/dedup-apply.test.ts
packages/core/test/curator-changed-entries.test.ts
packages/core/test/ltm.test.ts packages/gateway/test/api.test.ts
packages/gateway/test/ui-contracts.test.ts
packages/gateway/test/data-dedup-policy.test.ts`: pass.
- `pnpm --filter @loreai/core build`, `pnpm --filter @loreai/ui test`
(803 passed), `pnpm --filter @loreai/ui build`, `pnpm --filter
@loreai/gateway run bundle`, `node scripts/ui-deep-link-smoke.mjs`:
pass.
- `pnpm --filter @loreai/ui test:e2e`: the dedup-review spec passes on
desktop and mobile; the rest of the suite passes.
- `pnpm test` (full run): 7 failures, all in known sandbox classes that
also fail on clean `main`: git URL rewrite ×4, agents-file mtime ×2,
gateway remote header ×1. A `ui-static` dev-marker failure came from a
stale staged bundle and passed after rebuilding.

## Definition of done
- [x] Shared run clusters every shared-visible entry.
- [x] Project run surfaces private(P)↔shared pairs and never pairs two
other-project entries.
- [x] Shared entries are never removed by a project-scoped merge, and
the survivor never changes visibility. Private content becomes shared
only through an explicit `contentFromId`.
- [x] Keeping the private copy (`contentFromId`) has revision checks on
both entries, receipts, provenance, replay, `title_conflict` and
`.lore.md` export.
- [x] Revision checks, receipts, provenance, recovery and idempotency
are unchanged; export covers origin projects.
- [x] Existing response shapes unchanged; only `pool` and
`project_shared` are added.
- [x] Regression tests in core, gateway, contracts, UI unit and e2e.
- [x] No version bump, CHANGELOG edit, proxy/pipeline change, or
migration.

Closes #1980

Link to Devin session:
https://app.devin.ai/sessions/ee193c0c438e4de1954a6867892eb986
Open in Devin Desktop:
https://app.devin.ai/desktop/session/ee193c0c438e4de1954a6867892eb986?variant=devin
Requested by: @BYK

---------

Co-authored-by: Burak Yigit Kaya <ben@byk.im>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

MEM-03: Safe knowledge edit/delete with versioned restoration and visible sync/export effects

1 participant