You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
test(perf): add a reproducible long-session workload with background pressure #1733
The #1710 benchmark demonstrates semantic/window preparation improvements but excludes LTM/embedding retrieval and upstream inference, and its database reopen occurs in the same process. It cannot establish responsiveness for the reported 7,228-message workload while maintenance runs.
Implementation plan
Extend the existing benchmark fixtures with seeded tool-heavy Responses sessions around 5,580 and 7,228 messages and a roughly 180k–200k-token active window. Include large tool outputs, provenance, current-turn tool pairs, and configurable knowledge/vector/backlog sizes.
Exercise the real gateway and SQLite/read-worker path against a deterministic local upstream stub. Include both append continuation and a genuinely fresh serving-process restart.
Add independent scenarios for idle backlog, foreground arrival during a backlog, unavailable/hung embeddings, unavailable/slow read workers, cancellation, and concurrent sessions. Use provider stubs for deterministic CI and an opt-in native-provider run for diagnosis.
Report decode time separately from decoded-request-to-upstream overhead, health latency during load, process CPU, queue depth/age, retained memory, fallback counts and query execution counts. Reuse perf(observability): attribute memory preparation and vector worker latency #1732 when available without making it a build dependency.
Commit the generator, commands and machine-readable baseline in the repository. Permit a local production-database copy workflow, but never commit/upload user transcripts or a live database.
Acceptance criteria
A single command reproduces the baseline without paid APIs or a private dataset.
Restart tests start a new process, and workload parameters/seeds/runtime/build identity are recorded.
Timing comparisons use warmup/repeated samples and report hardware; CI does not depend on brittle universal millisecond thresholds.
The report tracks the proposed integration goal of p95 post-decode Lore overhead below 2 s and health p95 below 250 ms on a stated reference machine, including degraded-provider scenarios. These are targets, not current measurements.
Dependencies and boundary
Can start independently and reuse the existing #1710 fixtures. #1731 and #1732 improve diagnosis but are not hard prerequisites. This is a performance/responsiveness harness, not the behavioral memory-evaluation suite in #1211 or heap-owner investigation in #1683.
Hard prerequisites
None. This issue can be implemented independently.
Parent incident: #1708
Priority: First: baseline and shared verification
Problem
The #1710 benchmark demonstrates semantic/window preparation improvements but excludes LTM/embedding retrieval and upstream inference, and its database reopen occurs in the same process. It cannot establish responsiveness for the reported 7,228-message workload while maintenance runs.
Implementation plan
Acceptance criteria
Dependencies and boundary
Can start independently and reuse the existing #1710 fixtures. #1731 and #1732 improve diagnosis but are not hard prerequisites. This is a performance/responsiveness harness, not the behavioral memory-evaluation suite in #1211 or heap-owner investigation in #1683.
Hard prerequisites
None. This issue can be implemented independently.