Quality gate: CI regression checks with elf-eval trace compare

[aurexav]

Summary

Add CI regression gates using elf-eval trace compare.

Problem

Without hard quality gates, retrieval regressions are discovered too late.

In Scope

• Add CI stage running trace-compare against fixed trace IDs.
• Define thresholds for positional churn, set churn, and top-k retention.
• Fail CI when thresholds are exceeded.
• Publish machine-readable diff artifacts.

Out of Scope

• Replacing existing integration tests.
• Full benchmark infrastructure expansion.

Deliverables

• CI workflow updates.
• Baseline trace set.
• Gate configuration docs.

Acceptance Criteria

• CI runs compare mode reproducibly.
• Regressions surface actionable metrics.
• Local reproduction instructions are documented.

Dependencies

• Search: graph relation context and explain integration #50
• Memory Policy v2: explicit remember/update/ignore/reject decisions #51

Implementation Checklist

• Baseline trace snapshot committed.
• CI job added.
• Threshold config documented.
• Failure artifacts exported.

Done When

• Retrieval quality regressions block merges by default.

[aurexav]

Shipped CI trace regression gate on main.

Deliverables:

• CI workflow: .github/workflows/quality.yml loads a fixed Postgres trace fixture and runs the gate binary.
• Baseline trace set + fixture: .github/fixtures/trace_gate/fixture.sql seeds search_traces, search_trace_candidates, and baseline search_trace_items for fixed trace IDs.
• Gate config: .github/fixtures/trace_gate/gate.json (thresholds + trace IDs) and .github/fixtures/trace_gate/config.toml.
• Gate runner: cargo run -p elf-eval --bin trace_regression_gate -- -c .github/fixtures/trace_gate/config.toml -g .github/fixtures/trace_gate/gate.json --out <file>.
• Docs: added local reproduction notes in docs/guide/evaluation.md.

Commit: 7b2d9c5

[aurexav]

Follow-up shipped in commit 8f66cca on main: added a trace SQL fixture exporter (elf-eval: trace_gate_export), expanded the CI trace gate fixture/gate.json with additional traces, and added a scheduled workflow for nightly harness signals (context misranking + ranking stability).

[aurexav]

Follow-up: nightly harness signals is back to green after config schema drift in the ranking stability harness.

Changes shipped on main:

• 914da5d: add required ranking.deterministic tables to the stability harness config template.
• ece25c9: add required search.explain, ranking.diversity, ranking.retrieval_sources, and security auth fields.
• 4f7d139: enable ranking.blend with non-empty segments to satisfy elf-eval request validation.

Verification:

• Nightly Harness Signals workflow_dispatch run 22309946876 succeeded (context + stability harness).