fix(index): defer merges until rollback is no longer possible

[msokh]

Summary

Prevent route rollback from leaving merge tasks that write into an already-published lexical generation.

Tantivy 0.26.1 replaces its updater during IndexWriter::rollback(). The old updater's merges can keep writing. Waiting on the replacement writer does not wait for those tasks, and killed updaters reject merge completion before metadata registration and GC.

Change

• Keep merge dispatch disabled for each rollbackable updater epoch.
• Give each route/cohort rollback replacement a fresh disabled activation control. Abandoned epochs remain disabled.
• Enable only the terminal epoch immediately before its prepared commit, after revalidation, manifest persistence and fallible progress reporting.
• Keep the existing same-writer merge wait on success and error. Preserve explicit test-support merge disabling.

The production merge thresholds, history format, source ownership, and semantic behavior are unchanged.

Reproduction

A real filesystem/Tantivy fixture with 600 committed records reproduced the failure in four runs. After rollback and the replacement writer's wait returned, the old merge wrote six unreferenced segment files and changed .managed.json. The merged segment never entered meta.json; all committed records stayed searchable. The original-writer wait control passed in all four runs.

The dependency reproduction uses an explicit merge to obtain a completion edge. The ctx regression observes real automatic merge dispatch from actual route checkpoints; it does not simulate pointer activation or claim to identify the historical failed source route.

Verification

The new ctx route/cohort regressions failed on the base at real rollbackable merge dispatch. The explicit merge-disabled control passed. After the fix, route rollback, cohort rollback, merge-disabled publication, source revalidation abort and failed Merging progress tests passed. Terminal publication still coalesces eligible segments and discards failed-route records.

A separate epoch-isolation regression protects abandoned updater controls.

• Linux ctx-history-index --lib before the test-support simplification: 244 passed, 1 ignored, 0 failed. The final-head repeat was interrupted when the Docker daemon disconnected (unexpected EOF); it is not a full-suite PASS for the final HEAD.
• macOS ctx-history-index-query --all-targets before the test-support simplification: 9 unit and 142 integration tests passed. The final-head integration repeat timed out after 900 seconds in the existing 512-segment topology fixture; tests preceding that fixture passed, but the repeat is not a full-suite PASS.
• Real public-API consumer smoke: 17 published documents, 16 survivors, 1 post-rollback document, 0 discarded documents.
• cargo fmt --all -- --check passed.
• Final-head route/cohort rollback, explicit merge disabling, invalidation and progress-failure regressions: 5 passed.
• Final-head macOS ctx-history-index --lib: 232 passed, 1 ignored, 2 explicitly filtered baseline failures named below; 0 failures in the remaining suite.

The macOS writer suite reported two existing APFS-clone assumptions (incremental_append_needs_controls_and_writer_scratch_not_a_second_base and candidate_merge_unlinks_preserve_published_certifications). Both also fail on the unchanged base; both pass on Linux. This PR does not change those tests or the cloning implementation.

No live ~/.ctx data was rebuilt or changed, and no installed binary was replaced.