hardening(gateway): define global resource budgets and graceful drain deadlines #1617

Motivation

Lore already has several stream-specific limits, but resource control should be auditable across the whole process. Agent Gateway bounds bodies, pools, caches, windows, handshakes, queues, and shutdown. Lore needs a single inventory and coherent process-wide budget model.

Scope

• Inventory all request/response byte limits, retained stream state, caches, sessions, concurrent distillations/curations, workers, queues, and timers.
• Add missing global caps and explicit admission behavior.
• Define graceful drain plus a hard termination deadline.
• Ensure limits compose across chained recall continuations.

Acceptance criteria

• Every potentially unbounded collection or retained byte stream has a documented owner and limit.
• Limit errors are protocol-correct and do not leak hidden recall data.
• Concurrent sessions cannot each independently exceed the intended process budget.
• Shutdown stops admission, drains active work to a deadline, flushes required state, then terminates.
• Stress tests cover slow clients, large bodies, recall chains, worker saturation, and cache pressure.
• Resource-limit metrics identify the exhausted budget.

Focused implementation slices for #1708

The September performance incident is split into independently shippable work. Keep this issue as the broader resource-budget inventory; avoid implementing the same fixes twice.

• fix(embeddings): decouple request cancellation from worker lifetime #1745 — separate embedding caller cancellation from worker lifetime, keep detached native work accounted for, and use independent worker failure/watchdog deadlines.
• fix(reads): keep heavy worker failures off the gateway thread #1736 — heavy-read failure outcomes never execute unbounded scans on the gateway thread.
• perf(workers): bound read-pool queues and prioritize foreground requests #1739 — bounded pending read jobs/bytes and foreground priority.
• perf(background): gate maintenance batches on foreground and runtime pressure #1740 — pressure-aware admission between maintenance batches.
• fix(gateway): enforce a shared memory-preparation deadline with safe fallback #1741 — one shared preparation budget and safe degraded response, after its cancellation/read prerequisites.

Use #1733 to verify the combined result. Other global limits and graceful-drain scope in this issue remain open; completing one slice does not close this whole issue.

Separate recall-budget workstream

• feat(recall): replace the fixed ten-call cutoff with explicit chain budgets #1747 — request-owned recall budgets, bounded batched detail reads and stalled-retrieval policy; preserve final-continuation accounting.

This belongs to the broader resource-budget inventory and proceeds independently of #1708's latency investigation and implementation plan.