You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
perf(core): move vec0 orphan discovery off startup and the event loop #1681
Embedding startup always runs full gcVec0DanglingRows(db()) in vec0 mode, even after migration and rechunk state have converged. Temporal orphan discovery uses a whole-table anti-join on the writer connection and blocks startup.
Evidence
An equivalent temporal dangling-row anti-join took 2.87 seconds: 2.24 seconds user CPU and 0.42 seconds system CPU.
Durable temporal migration state was already done=1 with no remaining rows, so the scan was unrelated to useful backfill work.
Required fix
Discover orphans in bounded pages on a read worker during idle maintenance. Return exact chunk IDs and delete them on the writer connection. Never rerun a timed-out scan synchronously.
Acceptance criteria
Local embedding prewarm and startup never await a full vec0 scan.
Orphan discovery is bounded and off-thread.
Cleanup deletes exact chunk IDs only.
Timeout or worker failure degrades safely and never blocks startup.
Backed rows survive while dangling rows are eventually removed.
Selected as the next reliability fix after auditing all 51 open issues. Current main still invokes the synchronous whole-table gcVec0DanglingRows scan after startup backfill, including an already-converged database. I am replacing that startup scan with bounded, idle-gated read-worker discovery and exact-ID writer deletes, with writer-side liveness rechecks and no synchronous fallback on worker failure. This addresses a confirmed blocking operation within the broader #1686 incident; the unproven retained-memory investigation in #1683 remains separate.
Activity
fix(core): avoid temporal vec0 deletion scans
BYK commentedon Sep 8, 2026
Selected as the next reliability fix after auditing all 51 open issues. Current main still invokes the synchronous whole-table
gcVec0DanglingRowsscan after startup backfill, including an already-converged database. I am replacing that startup scan with bounded, idle-gated read-worker discovery and exact-ID writer deletes, with writer-side liveness rechecks and no synchronous fallback on worker failure. This addresses a confirmed blocking operation within the broader #1686 incident; the unproven retained-memory investigation in #1683 remains separate.fix(core): move startup vector orphan discovery to idle read workers (B…