feat(kb): ingest has no partial-success state — a timeout discards every page already read #17986

Part of #17527. Blocked by #17830.

Problem

Knowledge-base ingest has no partial-success state. add_document (autobot-backend/knowledge/documents.py) performs a single wait_for and stores nothing when it expires, so a long PDF that runs out of time loses every page already read. The work was done; the result is discarded.

The two outcomes a caller can observe are "fully stored" and "nothing stored". There is no third, and the one that actually happens for large documents — "most of it was read and then the clock ran out" — reports as the second.

Why this is the expensive shape

It is not merely lossy, it is silently lossy in a way that makes retry worse:

• A retry starts from zero, so a document that cannot finish inside one budget can never finish, no matter how many times it is retried. Each attempt pays full cost for nothing.
• Nothing records how far it got, so there is no signal distinguishing "this document is too big for the budget" from "the extractor crashed". Both are an empty collection.
• The failure scales with document value: the longer and richer the source, the more certain it is to be lost.

Proposal

• A per-document time budget, explicit rather than implicit in a single wait_for.
• A PARTIAL status recorded per chunk, so what was read is kept and what was not is known.
• Resumption from the last stored chunk rather than from the beginning.

Sequencing

Blocked by #17830: add_document does not chunk at all today — a whole transcript or PDF becomes one unit. Per-chunk partial status has nothing to attach to until chunking exists, so this must land after it. Recording the dependency rather than duplicating #17830's work.

Acceptance criteria

• A document that exceeds its time budget stores the chunks already processed and is marked PARTIAL, not failed-and-empty.
• PARTIAL is distinguishable by a reader from both COMPLETE and FAILED. An empty result and a partial result must not render alike — the same requirement as bug(codebase-analytics): duplicate detection reports 'nothing scanned' as a completed scan — the one tool the dedup programme needs returns nothing for AutoBot's own codebase #17982 and bug(analytics): the duplicates fetchers render a 200 {status:"error"} payload as "0 duplicates" (frontend half of #17982) #17983, in the ingest layer.
• A retry resumes from the last stored chunk rather than restarting.
• A test ingests a document with a budget deliberately too small and asserts both that chunks were stored and that the status says so. A test asserting only "it did not crash" does not satisfy this.
• The budget is configurable through the SSOT rather than a literal.

Related

• Umbrella: knowledge base intake is built but two legs are severed — upload has no UI, transcriber output cannot land #17527 — parent
• bug(kb): add_document never chunks — a whole transcript or PDF becomes one embedding, against its own docstring #17830 — blocker; chunking must exist first
• bug(codebase-analytics): duplicate detection reports 'nothing scanned' as a completed scan — the one tool the dedup programme needs returns nothing for AutoBot's own codebase #17982, bug(analytics): the duplicates fetchers render a 200 {status:"error"} payload as "0 duplicates" (frontend half of #17982) #17983 — the same empty-vs-failed confusion in the analytics layer