You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
eval: the trajectory gate has never run on a behaviour change — 0 of the last 60 merged PRs would trigger it #16158
Two independent measurements of what that means in practice:
Measurement
Result
Of the last 60 merged PRs, how many would have triggered it
0
Its entire run history
12 runs, all on 2026-09-06, all from issue-15812-budget-scope plus one push
So it has only ever run on the branch that built it. Nothing that changes agent behaviour under autobot-backend/agents/ (97 files) or the wider autobot-backend/llc/ tree (379 files) brings the eval along with it.
Promoted to blocking today, it would gate 0% of merges. A gate whose reach is its own source tree measures itself.
Acceptance criteria
Trigger paths cover the code whose behaviour the goldens assert — the agent and tool-dispatch surface, not just the harness
The intended population is stated in the workflow, so a later narrowing is visibly a change of scope rather than a tidy-up
Verified by measurement, not by reading the glob: re-run the "how many of the last N merged PRs would trigger this" count and record the number
The paths are not widened to everything — a gate that runs on every PR is a cost with no more signal than one that runs on the right ones
That last criterion is a real tension rather than a formality. This workflow installs dependencies and replays a corpus; running it on all ~60 PRs a day is not free, and the fix is to name the surface that actually changes trajectories.
Ordering
Not blocked by #16157, deliberately — widening the trigger is independent work and the edge would be a preference rather than a dependency. It is worth doing after it, though: widening the trigger first means running a check that cannot fail, more often. Sequence by judgement, not by a blocked_by edge that would misdescribe the relationship.
Activity
fix(eval): give the trajectory gate a candidate that can disagree wit…