Skip to content

perf(claude-ops): Stop-event unsurfaced-failure detector times out 113x, ~44x the per-turn hook budget #3512

Closed
#3851
Closed
perf(claude-ops): Stop-event unsurfaced-failure detector times out 113x, ~44x the per-turn hook budget#3512
#3851
Labels
agent-readyFully specified and briefed; eligible for autonomous pickup from the frontier.priority: highSignificant impact, or blocks an imminent release; staff this cycle.work-class: scopedA briefed fix or small feature; blast radius bounded by the brief, tests exist.

Description

@kyle-sexton
kyle-sexton
opened on Aug 31, 2026

Summary

Child of #3508. The Stop-event hook that audits for unsurfaced hook failures times out repeatedly, at roughly 44x the marketplace per-turn hook budget.

This issue was originally filed against guardrails. That was wrong — guardrails registers no Stop hooks at all. Corrected below.

The hook
Plugin claude-ops
Hook plugins/claude-ops/hooks/hook-failure-audit.sh
Registration plugins/claude-ops/hooks/hooks.json, Stop event
Configured timeout 10 s
Timeouts in window 113
Average recorded duration 21.9 s
Worst observed 74.1 s

The bar

docs/conventions/hook-budget/README.md (adopted #1809, surfaced by .claude/rules/hook-budget.md) sets the per-turn ceiling for the whole always-on Stop-shaped set at <= 500 ms, measured as parallel wall time.

At 21.9 s average this hook alone is ~44x the entire per-turn budget, and it shares that budget with two siblings:

Issue Plugin Stop timeout Timeouts Avg
#3512 (this) claude-ops 10 s 113 21.9 s
#3515 autonomy 15 s 73 27.6 s
#3516 disk-hygiene 20 s 52 31.8 s

Three plugins, three separate processes, one 500 ms budget between them. Consolidation across the three is likely worth more than optimizing any one, and none of the three can meet the ceiling alone at current spawn cost.

Why it is slow

Per #3508, process creation on the affected host is bimodal at 180-2,841 ms (median 1,108 ms at 501 concurrent processes). Lead any fix with a spawn count, not a duration — durations on that host are not comparable across time.

This hook is also structurally last in line: it inspects other hooks' results, so it runs after the rest of the turn's fan-out and compounds whatever contention they created.

Proposed work

  1. Count spawns first, via a PATH shim of wrapper scripts that log their own name then exec the real tool. Deterministic, drift-immune, cheap for a reviewer to verify.
  2. Early-exit before any spawn when no hook failure was recorded this turn — the common case.
  3. Consider sampling or async rather than running on every Stop.
  4. Evaluate consolidating the three Stop hooks into one process, given the shared 500 ms ceiling.

Acceptance criteria

  • Spawn count on the common path reported before and after, measured by PATH shim.
  • A turn with no hook failures exits before any external process spawn.
  • The hook's measured share, by the convention's method (EPOCHREALTIME around direct invocation, >= 10 runs, concurrent launch), is stated in the claude-ops README per hook-budget Rule 1.
  • The always-on per-turn set stays within <= 500 ms parallel wall, reported alongside the spawn-cost baseline at measurement time.
  • Existing behavioural tests pass; genuine unsurfaced failures are still detected.
  • Validated with scripts/affected-tests.sh --run.

Notes

Prior art in #1403 / #1385 (hook::jq_fields, strip_quoted_spans, deferred git rev-parse; a 65-spawn-per-Bash-call census and a 437 ms unit price). hook-utils.sh exists in 17 synced copies — changes go through scripts/sync-hook-utils.sh, never one plugin's copy.

Activity

  1. kyle-sexton
    added
    priority: highSignificant impact, or blocks an imminent release; staff this cycle.
    agent-readyFully specified and briefed; eligible for autonomous pickup from the frontier.
    work-class: scopedA briefed fix or small feature; blast radius bounded by the brief, tests exist.
    on Aug 31, 2026
  2. kyle-sexton
    changed the title [-]perf(guardrails): Stop hook: unsurfaced hook failures averages 21.9s and timed out 113x - reduce process spawns[/-] [+]perf(claude-ops): Stop-event unsurfaced-failure detector times out 113x, ~44x the per-turn hook budget[/+] on Aug 31, 2026

  3. kyle-sexton commented on Aug 31, 2026

    @kyle-sexton
    kyle-sexton
    on Aug 31, 2026
    ContributorAuthor

    Misattributed on filing — this is claude-ops, not guardrails. Retitled.

    guardrails ships no Stop hook at all; its hooks/hooks.json declares only PreToolUse and PostToolUse entries. The unsurfaced-hook-failure detector is claude-ops, which declares Stop timeout=10 and StopFailure timeout=5. That matches the observed 10,000 ms timeoutMs on the Stop-event timeouts in the corpus.

    All three Stop-event children are now correctly attributed and each belongs to a different plugin:

    Issue Plugin Stop timeout Timeouts
    #3512 (this) claude-ops 10 s 113
    #3515 autonomy 15 s 73
    #3516 disk-hygiene 20 s 52

    The per-turn budget is the binding constraint here and it is much tighter than I stated: docs/conventions/hook-budget/README.md sets <= 500 ms for the whole always-on per-turn set, shared across all three of these hooks. At 113 timeouts averaging 21.9 s, this one alone is ~44x the entire per-turn budget.

    Shared correction (applies to every child of #3508)

    1. The "15 s" timeout in the table above is wrong for most of these hooks. Timeouts are declared per hook in each plugin hooks/hooks.json, and they vary widely. Verified against the installed manifests and cross-checked against the timeoutMs field the harness records in transcripts:

    source-control  PreToolUse Bash                    timeout=15
guardrails      PreToolUse Bash|PowerShell         timeout=60
guardrails      PreToolUse Write|Edit|NotebookEdit timeout=60
guardrails      PostToolUse Write|Edit             timeout=30
context-guard   PreToolUse / PostToolBatch         timeout=60
disk-hygiene    PreToolUse Bash|PowerShell         timeout=60 ; Stop timeout=20
claude-ops      Stop timeout=10 ; StopFailure timeout=5
autonomy        Stop timeout=15

    Observed timeoutMs distribution across the transcript corpus confirms it: 1,365 PreToolUse:Bash timeouts at 15,000 ms (source-control), 246 at 60,000 ms and 384 PreToolUse:PowerShell at 60,000 ms (guardrails), Stop at 10,000 / 15,000 / 20,000 ms.

    This makes the guardrails hooks worse than reported, not better — each one burns a full 60 seconds before being killed, not 15.

    2. The acceptance criteria should cite the repo's existing budget, not the ad-hoc ones I wrote. docs/conventions/hook-budget/README.md (adopted in #1809, surfaced by .claude/rules/hook-budget.md on plugins/*/hooks/**) already defines the binding ceiling:

    Surface Budget
    Per tool call (PreToolUse + PostToolUse for one matcher) <= 1 s typical, <= 2 s worst-case
    Per turn (Stop / notification-shaped) <= 500 ms

    Measured as parallel wall time, not the sum. Rule 1 of that doc requires a plugin adding or widening an always-on hook to state its measured share in its README; Rule 2 states the budget never relaxes to absorb an overage. Use those numbers as the bar. My "under 2 s on a Windows host" was an accidental near-restatement of the per-tool-call worst case; the per-turn 500 ms figure I did not have at all, and it binds every Stop-event child.

    3. Prior art exists and should not be re-derived. #1403 (closed, needs-human) captures the spawn-reduction work from the unmerged PR #1385, including:

    • hook::jq_fields batched field extraction in lib/hook-utils.sh, replacing per-field jq calls across eight guards
    • a strip_quoted_spans shell-builtin replacement for a printf | sed | sed pipeline
    • a deferred git rev-parse --is-inside-work-tree probe in hardcoded-path-check.sh
    • a census of 65 process spawns per Bash tool call, and a measured 437 ms per-spawn unit price

    It also records why #1385 did not merge: a fresh-context verification found four contract-test regressions, two of which failed open. Any revival must clear those first.

    4. hook-utils.sh is synced, not per-plugin. It exists in 17 copies across plugins/*/hooks/hook-utils.sh, delivered by scripts/sync-hook-utils.sh. A change there must go through that script with all copies verified identical — editing one plugin's copy will be reverted or will diverge the fleet.

    5. Validate with scripts/affected-tests.sh --run (add --explain to see suite selection), per the repo AGENTS.md. A changed file mapping to zero suites is an error, not "nothing to run".

    6. Reference baseline for the host. The budget doc's reference measurements (2026-07-31, Windows 11 + Git Bash) are bash -c : ~ 80 ms and python3 -c pass ~ 160 ms, with the per-Bash-call always-on set at **~ 5.9 s parallel wall**. The host in #3508 measures a bimodal no-op spawn at 180-2,841 ms (median 1,108 ms) at 501 concurrent processes — 2-35x worse than that reference. Report the spawn baseline alongside any timing so a result on a healthy box is not mistaken for a pass on a contended one.

  4. kyle-sexton
    assigned
    kyle-sexton
    and unassigned
    kyle-sexton
    on Sep 6, 2026
  5. kyle-sexton
    added 2 commits that reference this issue on Sep 7, 2026

    fix(guardrails): every hook says so when it could not run (#3528) (#3895

    bbbb273

    perf(claude-ops): halve hook-failure-audit spawns by moving redirects…

    fad23df
  6. kyle-sexton
    closed this as completedin #3851on Sep 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    agent-readyFully specified and briefed; eligible for autonomous pickup from the frontier.priority: highSignificant impact, or blocks an imminent release; staff this cycle.work-class: scopedA briefed fix or small feature; blast radius bounded by the brief, tests exist.

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions