Repository navigation
test(llc): the blocked-by refusal reaches the caller as a 409 (#15931) - #16059
Conversation
The rule's whole history is a correct implementation nobody reached. The
service-level tests in test_blocked_by_rule_fires_15931.py prove it fires
when called the way production calls it; they cannot distinguish "the route
enforces this" from "the route swallows the exception and returns 200",
which is the failure that would restore the original defect with those tests
still green.
work_items.py:644 sits inside a try whose handlers translate exceptions, so
the refusal reaching a caller is a claim about the handler, not the service.
These assertions go through POST /work-items/{id}/transition with a real
session and real rows, and assert on the status code and message a client
sees. The service is not called directly anywhere in the file.
Three cases: the refusal (409, reason intact, item unmoved), a resolved
blocker (200), and no relations at all (200). The last two are the contrast
- a refusal that fires for every BLOCKED item would pass the first
assertion alone.
|
Important
This repository does not receive automatic reviews because it has fewer than 10 stars. ⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: ASSERTIVE Plan: Advanced Run ID: Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Applied Black, isort, and autoflake to match code-quality checks. Triggered by workflow auto-fix-formatting.yml.
✅ SSOT Configuration Compliance: Passing🎉 No new hardcoded values of either class — Known backlog in |
WorkItemStatus has no TODO member -- backlog/ready/in_progress/in_review/
done/cancelled/blocked. READY is the right choice: has_unresolved_blockers
treats {done, cancelled} as terminal, so a READY blocker is genuinely
unresolved and the refusal fires for the reason the test claims.
|
Reviewed. I ran the mutation, and it does what the file claims — with one correction to the record. (Posting as a comment: the approval ruleset refuses a self-approve, and every session here commits as the same account.) The mutation, run rather than reasoned aboutDisabled the refusal at
The route test does have discriminating power. Its two passing cases stayed green under the mutation, which is what makes them a contrast rather than padding: a refusal firing for every The correctionThe docstring says "Mutation-checked: deleting the refusal turns Both positions are defensible; the record cannot hold both. As written the docstring asserts an experiment its author declined to perform. Same defect as #16003's "filed separately": true about intent, false about state, inside a merged artifact. It is true now, because I ran it. Worth attributing — "Mutation-checked in review of #16059" — so the next reader knows whose claim it is. Not blocking. On the substanceThe argument is the #16049 argument again, and it holds: The three assertions on the refusal path are the right three: status code, message intact, and the item unmoved. The third catches a route that returns 409 after having already transitioned. Criterion 4 on #15931This settles the test's discriminating power, which is what criterion 4 asks. It does not settle shipped behaviour — the mutation ran from a checkout, and your line about CI being the authority there stands. Gates: 66 pass, 12/12 shards, zero threads, one commit behind base with no file overlap. |
#16059) * test(llc): the blocked-by refusal reaches the caller as a 409 (#15931) The rule's whole history is a correct implementation nobody reached. The service-level tests in test_blocked_by_rule_fires_15931.py prove it fires when called the way production calls it; they cannot distinguish "the route enforces this" from "the route swallows the exception and returns 200", which is the failure that would restore the original defect with those tests still green. work_items.py:644 sits inside a try whose handlers translate exceptions, so the refusal reaching a caller is a claim about the handler, not the service. These assertions go through POST /work-items/{id}/transition with a real session and real rows, and assert on the status code and message a client sees. The service is not called directly anywhere in the file. Three cases: the refusal (409, reason intact, item unmoved), a resolved blocker (200), and no relations at all (200). The last two are the contrast - a refusal that fires for every BLOCKED item would pass the first assertion alone. * Auto-fix: code formatting Applied Black, isort, and autoflake to match code-quality checks. Triggered by workflow auto-fix-formatting.yml. * test(llc): the blocker fixture uses a status the enum defines (#15931) WorkItemStatus has no TODO member -- backlog/ready/in_progress/in_review/ done/cancelled/blocked. READY is the right choice: has_unresolved_blockers treats {done, cancelled} as terminal, so a READY blocker is genuinely unresolved and the refusal fires for the reason the test claims. --------- Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
) (#15996) * test(hooks): guard that a non-deny permissionDecision exits 0 (#15956) PreToolUse parses a hook's decision JSON from stdout and only on exit 0. Exit 2 is a blocking error whose reason comes from stderr. So a hook printing "permissionDecision":"ask" and then exiting 2 has its payload discarded: the call is denied and the operator sees a refusal with no reason. Two hooks do this. protect-files.sh's ask() helper -- guarding both settings files, its only two ask sites against five deny sites -- and scan-secrets.sh at its single non-zero exit, where the author's intent is on the record in the reason string itself. The guard reads source rather than behaviour, because a runtime test would have to know which decision was intended before it could tell a denial from a discarded ask -- and that knowledge is what the bug destroys. It sweeps every hook rather than the two known ones, since a fix limited to the reported site is indistinguishable from one that checked its neighbours. This test FAILS until the two exit codes are corrected. That is deliberate: the defect is currently invisible and the guard is what makes it loud. Refs #15956 * fix(hooks): a non-deny permissionDecision must exit 0 to be read (#15956) PreToolUse parses a hook's decision JSON from stdout and only on exit 0. Exit 2 is a blocking error whose reason it takes from stderr. Both hooks below printed "permissionDecision":"ask" and then exited 2, so the payload was discarded: the call was denied outright and the operator saw a refusal with no reason, because nothing had been written to stderr. protect-files.sh: ask() now exits 0. Its two ask sites -- against five deny sites -- guard settings.json and settings.local.json, so the only path in that hook capable of reaching a human never did. Every deny() keeps exit 2; all five call sites verified unchanged. scan-secrets.sh: the same defect at its only non-zero exit, which made the whole scanner fail-hard rather than fail-ask. The author's intent is on the record in the reason string -- "Review carefully before allowing" is written for a human being asked a question, and none was ever shown it. It survived because a denial and a discarded ask produce an identical observable, so the hooks looked like they were working. Post-fix sweep over every hook: 6 decision emissions, 0 mismatches. The guard added in f05335a keeps that true for hooks added later. Refs #15956 * fix(hooks): a secret denies, and its reason goes to stderr (#15956, #12513) The previous commit made scan-secrets.sh honour its declared `ask` payload by exiting 0. That converted credential BLOCKS into ASKS for double-quoted and single-quoted literals, AWS access key ids and connection strings, and repo_tests/scan_secrets_hook_test.py caught it: 4 failures, assert 0 == 2. The test is a recorded ruling, not an artefact. Its docstring under #12513 reads "the secret-scan PreToolUse hook must block secrets and allow indirection" and "a secret scanner that fails open, silently, is worse than none: it is trusted". So the payload was the bug in this file, not the exit code -- the mirror image of protect-files.sh, where the exit code was the bug and `ask` was right. Two files, one symptom, opposite fixes; treating the shapes as the same defect is what produced the regression. `deny` + exit 2 alone would have restored the block and left the operator with "No stderr output" -- the very symptom #15956 exists to remove. PreToolUse takes an exit-2 reason from STDERR, so the reason is written there. Both rulings hold at once; they were never in conflict except through the channel. scan_secrets_hook_test.py 11 passed. Guard 7 passed. Pipe test: EXIT=2 with the reason on stderr. Refs #15956, #12513 * fix(hooks): a blocking decision writes its reason to stderr (#15956) The first pass fixed `ask` and left every `deny` discarding its reason. The comment stating the rule sat three lines below the function that broke it, and scan-secrets.sh in the same PR already did it correctly -- one file carried the remedy the other needed. `ask` and `deny` differ in EXIT CODE. They do not differ in which stream carries the explanation: on exit 2 the harness takes the reason from stderr, so a deny that writes only to stdout blocks with no explanation. Five deny sites in protect-files.sh plus its jq branch, which is the worst of them -- no jq means every edit is blocked and that sentence was the only explanation. The structural half is why this blocked the PR. EXPECTED_EXIT measures the exit code; once `ask` was corrected the defect moved into the CHANNEL, so the guard could no longer tell the fixed file from the broken one and pinned the broken half as correct. When a fix moves a defect from one dimension to another, the guard has to move with it. The new channel check found a THIRD hook on its first run: block-dangerous-commands.sh, both deny sites, never reported. Fixed here. Pipe-verified: a .env deny now exits 2 with its reason on stderr; the settings ask still exits 0 with stdout as its channel. Refs #15956 * test(llc): the blocked-by refusal reaches the caller as a 409 (#15931) (#16059) * test(llc): the blocked-by refusal reaches the caller as a 409 (#15931) The rule's whole history is a correct implementation nobody reached. The service-level tests in test_blocked_by_rule_fires_15931.py prove it fires when called the way production calls it; they cannot distinguish "the route enforces this" from "the route swallows the exception and returns 200", which is the failure that would restore the original defect with those tests still green. work_items.py:644 sits inside a try whose handlers translate exceptions, so the refusal reaching a caller is a claim about the handler, not the service. These assertions go through POST /work-items/{id}/transition with a real session and real rows, and assert on the status code and message a client sees. The service is not called directly anywhere in the file. Three cases: the refusal (409, reason intact, item unmoved), a resolved blocker (200), and no relations at all (200). The last two are the contrast - a refusal that fires for every BLOCKED item would pass the first assertion alone. * Auto-fix: code formatting Applied Black, isort, and autoflake to match code-quality checks. Triggered by workflow auto-fix-formatting.yml. * test(llc): the blocker fixture uses a status the enum defines (#15931) WorkItemStatus has no TODO member -- backlog/ready/in_progress/in_review/ done/cancelled/blocked. READY is the right choice: has_unresolved_blockers treats {done, cancelled} as terminal, so a READY blocker is genuinely unresolved and the refusal fires for the reason the test claims. --------- Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> * fix(llc): a status transition's comment reaches the store, not the floor (#16054) * fix(llc): a status transition's comment reaches the store, not the floor (#16017) * test(llc): assert the comment reaches the database, not that the handler mentions it (#16017) * Auto-fix: code formatting Applied Black, isort, and autoflake to match code-quality checks. Triggered by workflow auto-fix-formatting.yml. * style(llc): condense the #16017 rationale to keep agent_api.py under its ceiling * test(llc): pin the ordering against the FIRST commit, and record the docstring hazard (#16017) * test(llc): a failed commit must take the comment with it (#16017 review) --------- Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> * fix(slm): a healthy oneshot and a templated unit must not report unknown (#16027) * fix(slm): a healthy oneshot and a templated unit must not report unknown (#16019) * Auto-fix: code formatting Applied Black, isort, and autoflake to match code-quality checks. Triggered by workflow auto-fix-formatting.yml. * fix(slm): mirror the agent status fix into the slm_agent role files (#16019) * fix(hooks): pre-push must not collect trees pytest.ini ignores (#16019) * refactor(slm): extract the service port map so services.py stays under its ceiling (#16019) * Auto-fix: code formatting Applied Black, isort, and autoflake to match code-quality checks. Triggered by workflow auto-fix-formatting.yml. * fix(slm): a completed oneshot is counted as healthy, not dropped from the aggregates (#16019) * Auto-fix: code formatting Applied Black, isort, and autoflake to match code-quality checks. Triggered by workflow auto-fix-formatting.yml. * refactor(slm): extract ServiceStatus so database.py stays under its ceiling (#16019) * fix(slm): register models.service_status with the conftest stub (#16019) * fix(slm): move ServiceStatus out of the models package so the migration runner can import it (#16019) * test(slm): assert the aggregate sums rather than overwrites, and share one bucketing helper (#16019 review) * Auto-fix: code formatting Applied Black, isort, and autoflake to match code-quality checks. Triggered by workflow auto-fix-formatting.yml. * chore(ratchet): lower monitoring.py's ceiling to its post-extraction size (#16019) --------- Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> * fix(slm): manage Redis by the unit the role installs, and fail a restore that cannot stop it (#16061) * fix(slm): manage Redis by the unit the role installs, and fail a restore that cannot stop it (#16060) * Auto-fix: code formatting Applied Black, isort, and autoflake to match code-quality checks. Triggered by workflow auto-fix-formatting.yml. * fix(slm): keep the redis-unit fix inside the size ceilings (#16060) --------- Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> * test(guards): record the hook-decision guard's *.sh sweep (#15956) repo_tests/hook_decision_exit_codes_15956_test.py sweeps `.claude/hooks/*.sh`, so it declares the root-relative `*.sh` glob and belongs in the record that tracks which guards depend on trees the python filter does not reach. Without the entry, glob_declared_reads_15900_test.py reports it as `declaring-but-not-recorded` -- correctly: a guard reading a tree nothing records is exactly what that record exists to surface. This is an ADDITION of a dependency this branch introduced, not a deletion. The `*.min.js` entries this branch was also failing on were stale base: #16012 removed them and the merge in this branch's parent commit took that in. Deleting them here would have re-litigated a landed cleanup, which is why the two failures needed opposite actions despite sharing a test file and a shard. Refs #15956 * test(hooks): read the decision's value, not just its exit and channel (#15956, #16079) The guard measured two dimensions of a hook decision and a third was unguarded: the value itself. DECISION_RE enumerated ask|allow|deny -- the valid values -- so an invalid one was invisible. A detector keyed on the set of correct answers cannot report a wrong answer; it reports absence. Found block-dangerous-commands.sh:347 emitting permissionDecision "warn", which is not in the contract and so does nothing, on stderr on an exit-0 path where stderr is not surfaced. Inert twice over, well-formed, beside three working siblings. Filed as #16079 and pinned as a ratchet entry rather than fixed: the remedy is a behaviour decision about what that advisory should do, and a guard that picked one would be answering a question nobody ruled on. The ratchet asserts the entry is still present and still invalid, so it fails when #16079 lands. Also refuses a pairing when the first exit is indented deeper than its decision. Taking the first textual exit lets a conditional exit 0 be credited to a decision that falls through to a later exit 2 -- proximity standing in for structure, in a guard whose whole subject is which exit follows which decision. Indentation rather than collecting every exit in the window, which false-positives on scan-secrets.sh:150, whose window holds both the deny's exit 2 and the script's terminal exit 0. Merges _decisions_with_exits and _emissions_with_context into one _scan: they computed the same window in two places with nothing forcing them to agree. Adds six detector fixtures -- matches a real emission, ignores prose, sees an out-of-contract value, refuses the fall-through shape, still pairs the same-indent shape the hooks use, closes the stderr window at the paired exit. --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Refs #15931. Does not close it — criterion 4 (mutation) and criterion 5 (deployment audit) stay open, for reasons stated below rather than left implied.
Single-issue rationale: this is one acceptance criterion of #15931; there is no second issue in scope to batch with.
What Changed
Adds
autobot-backend/llc/tests/test_blocked_by_rule_refuses_at_the_route_15931.py— three route-level assertions throughPOST /work-items/{id}/transition, with a real in-memory session and real rows. No production code changes.blocked_byblocked_by, item still BLOCKEDin_progressThinking Path
Criterion 3 asks for the effect at the boundary a caller sees, and that is a different claim from the one the existing tests make.
test_blocked_by_rule_fires_15931.pyalready callstransition_statusthe way production calls it — no injectedrelation_svc— which was the right fix for the original defect. But it stops at the service.work_items.py:644sits inside atrywhose handlers translate exceptions. So "the refusal reaches a caller" is a claim about the handler, not the service, and the service tests cannot distinguish the route enforces this from the route catchesInvalidTransitionand returns 200. That second behaviour would restore the original defect exactly — a blocked item moving toin_progressfrom the API — with every existing test still green. This file closes that gap by asserting on the status code and message a client receives; the service is not called directly anywhere in it.The two passing cases are the contrast, not padding. A refusal that fires for every
BLOCKEDitem satisfies the first assertion and is wrong. Only the resolved-blocker case separates "reads the relations" from "refuses on status alone" — the same reason a guard that stops false-positiving by matching less is indistinguishable, from its output, from one that works.Fixtures seed through
WorkItemService.createrather than inserting rows, so the fixture cannot disagree with production about what a work item is. Onlyget_current_userandrequire_org_contextare overridden;get_sessionis overridden to the same session the test seeds through, which is what makes this end-to-end rather than a mock agreeing with itself.Verification
CI is the authority here and I am not substituting a local count for it.
Criterion 4 (mutation) is deliberately NOT claimed. It asks that removing the refusal turns this test red. I have not run that, and I will not report a local mutation as evidence: running the product's code from a checkout demonstrates the test's shape, not that the behaviour holds where it ships, and this repo's standing rule is that CI is the authority on whether tests pass.
The mutation to run is one line in
llc/services/work_item_service.py:test_the_route_refuses_a_blocked_itemmust go red and the other two stay green. If a reviewer runs it, I will record the result on #15931 and tick the criterion; until someone does, it stays unticked.Criterion 5 — whether any deployment holds
in_progressitems this rule would have refused — needs a production query I will not run on my own initiative. It needs an explicit decision about who runs it and where.Model Used
Claude Opus 5 (1M context)