HNSW: graph-size resolution is a lifetime id high-water mark — churn-heavy tables permanently over-provision build/search ef #2182

Follow-up to #2180 / #2181, from the pre-push review's main finding.

Both HNSW auto-scales (search ef, and construction ef as of #2181) resolve graph size from the shared id counter — a lifetime high-water mark of allocated node ids. Deletes never decrement it, so a churn-heavy table (TTL eviction, delete-and-reinsert ingest) reads permanently high. On the search side that means a generous-but-capped ef. On the build side it means a table holding, say, 30K live vectors that has allocated 5M ids over its life resolves 5M → efConstruction pinned at the cap → every insert runs a 512-candidate connection search on a 30K-node graph, ~5x the base build CPU, inside the write transaction, forever — with zero recall benefit. Documented as an accepted caveat in DESIGN.md (bounded at AUTO_EFC_MAX, wastes CPU, never hurts recall), but the right fix is a count that tracks live nodes.

Sketch: a second slot in the shared next-id buffer maintained as a live count (increment on insert, decrement on remove). The hard parts are (a) re-seeding it O(1) after restart — the largest-id reverse seek that re-seeds the id counter has no live-count analogue, and an exact count is a full key scan; (b) mixed-version workers sharing the buffer during a rolling upgrade. A persisted periodically-checkpointed count (exact-on-rebuild, drift-bounded between checkpoints) may be the pragmatic middle.

Also worth considering alongside (from the same review): a "pinned build, auto search" configuration does not currently exist — setting efConstruction pins the search-side default too, so the escape hatch for the churn cost trades away search-ef auto-scaling.

Generated by Claude Fable 5.