Skip to content

Deleting a document leaves its facts in the graph with no evidence #268

Closed

Description

@WaylandYang

Measured on an isolated database — one document, one chunk, two entities, one fact with evidence, then DELETE FROM documents:

before  docs=1 chunks=1 entities=2 facts=1 evidence=1
after   docs=0 chunks=0 entities=2 facts=1 evidence=0 invalidated=0

documents::delete (documents.rs:400) is a bare DELETE FROM documents; everything else is FK cascade. chunks.document_id and fact_evidence.document_id / fact_evidence.chunk_id are all ON DELETE CASCADE, so every evidence row goes. facts has no FK to documents and is append-only,永不 DELETE, so the fact stays — live, unflagged, and now unsourced.

Three things are wrong with that.

  1. The product says otherwise. The Privacy policy's "Retention and deletion" section (web/src/i18n/en.ts:314) and the delete confirmation dialog (en.ts:618) both promise "Facts already extracted into the graph remain, with their provenance". The provenance does not remain.
  2. It contradicts the house rule everywhere else. replace_chunks soft-deletes superseded chunks rather than removing them, for the stated reason that fact_evidence references must not break (documents.rs:415, migrations/0002_ingest.sql:103); fact_evidence is introduced as 溯源一等公民 (0003_graph.sql:298), and its chunk_id ... ON DELETE CASCADE is the one FK in that file with no justification comment. Rejecting a fact keeps its evidence because deleting it "would erase what we once believed" (graph.rs:1523). An account is deactivated, not deleted, so the ledger stays attributable. Document deletion is the only place that subtracts.
  3. The blob is never deleted. BlobStore (blob.rs:15) has no delete method and nothing in the repo calls one, while document_versions rows cascade away — so the original file stays on disk, unreachable and unreplayable. Nothing today can remove it.

A fourth follows from the first: after a hard delete, the next sync of a source-backed document finds no row for its external_key (documents.rs:189), creates a fresh document and extracts again, while the old facts were never invalidated — the same assertion lands twice.

Decision. Deleting a document is an event on the recording axis. Delete leaves a tombstone and retires what depended on the document; purge is the separate, explicit step that removes content.

Delete

  • documents.deleted_at; DELETE becomes UPDATE. Chunks take the existing superseded_at. Nothing is cleared — text, embedding and the blob all stay until purge. Retrieval already excludes superseded chunks (documents.rs:646), the full-text index is already cleared by the handler, and both clocks are on chunks too (created_at / superseded_at), so an as-of read of a deleted document costs nothing extra (0019).
  • Facts are retired only when every evidence row's document is tombstoned: one fact row carries evidence from many documents (insert_fact returns the existing row for the same assertion, graph.rs:207-217; evidence is keyed (fact_id, chunk_id)), and a fact that still has a live source is untouched. The condition reads documents.deleted_at, not chunks.superseded_at — a fact whose evidence sits on an older version of a live document is stale, and stale_facts already rules that 没再提 ≠ 不成立: it goes to Review, it is not retired.
  • Retired facts get invalidated_at = now(), and the reason is recorded: document_deletions (document_id, deleted_by, deleted_at, invalidated_facts uuid[], reverted_at), the shape of entity_merges, so "a person retracted this" and "its source was deleted" stay distinguishable and the undo has its list.
  • Undo revives the document and the facts in invalidated_facts, the shape of revert_merge. A source sync that meets a tombstone for its external_key revives it the same way — for a synced document, a local delete is temporary by nature; to be rid of it, remove it at the source or exclude the prefix. The confirmation dialog says so. The partial unique index on (source_id, external_key) gains AND deleted_at IS NULL.
  • Every read of documents (29 sites across six store files) filters deleted_at IS NULL; the predicate lives in the store's document functions, and a database-backed test pins both directions.

Purge

  • purge_document: the blob, the chunk rows, fact_evidence.quote; affected facts are marked unsourced and shown under Review. Explicit, irreversible, for privacy and legal requests.

Wording

  • Policy and dialog become true: "Deleting a document removes it from the base and retires the facts that had no other source; its content stays until you purge it."

Rewinding to see what the graph held before the deletion is 0019, kept separate on purpose.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions