Skip to content

perf(core): move vec0 orphan discovery off startup and the event loop #1681

Closed

Description

@BYK

Problem

Embedding startup always runs full gcVec0DanglingRows(db()) in vec0 mode, even after migration and rechunk state have converged. Temporal orphan discovery uses a whole-table anti-join on the writer connection and blocks startup.

Evidence

  • An equivalent temporal dangling-row anti-join took 2.87 seconds: 2.24 seconds user CPU and 0.42 seconds system CPU.
  • Durable temporal migration state was already done=1 with no remaining rows, so the scan was unrelated to useful backfill work.

Required fix

Discover orphans in bounded pages on a read worker during idle maintenance. Return exact chunk IDs and delete them on the writer connection. Never rerun a timed-out scan synchronously.

Acceptance criteria

  • Local embedding prewarm and startup never await a full vec0 scan.
  • Orphan discovery is bounded and off-thread.
  • Cleanup deletes exact chunk IDs only.
  • Timeout or worker failure degrades safely and never blocks startup.
  • Backed rows survive while dangling rows are eventually removed.

Activity

  1. added
    bugSomething isn't working
    on Aug 26, 2026
  2. BYK commented on Sep 8, 2026

    @BYK
    OwnerAuthor

    Selected as the next reliability fix after auditing all 51 open issues. Current main still invokes the synchronous whole-table gcVec0DanglingRows scan after startup backfill, including an already-converged database. I am replacing that startup scan with bounded, idle-gated read-worker discovery and exact-ID writer deletes, with writer-side liveness rechecks and no synchronous fallback on worker failure. This addresses a confirmed blocking operation within the broader #1686 incident; the unproven retained-memory investigation in #1683 remains separate.

  3. added a commit that references this issue on Sep 9, 2026
    a685d77
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions