fix(enums): converge severity literals on the canonical enum and add the three rungs it was missing (#14956, #13597)

[mrveiss]

Thinking Path

The enum-fork half of #13597 already landed in #14957 — AlertSeverity and ErrorSeverity
are aliases of the canonical Severity, and the ratchet in repo_tests/enum_union_guard_test.py
pins that. Confirmed on the base before starting; nothing there was redone and no new enum
was forked. What was left is the literal sweep.

Measured the population rather than trusting it. #13597 said 119, #14956 said 169. Ran
the ratchet's own matcher — ["']severity["']\s*:\s*["'][A-Za-z_]+["'] over
git ls-files '*.py' under autobot-backend/ and autobot_shared/ — and got 169 hits
across 43 files, 43 of them outside the canonical vocabulary, matching #14956 exactly. The
number is not asserted only in this description: SEVERITY_LITERAL_FLOOR and
SEVERITY_LITERAL_CEILING bound it from both sides in the guard.

The 43 out-of-vocabulary literals, decided one at a time. The rule is that a duplicated
concept converges on the implementation that is the union; a rung is never dropped to make
a count go down. So each value was either a genuine synonym of an existing rung, or a rung
the canonical enum was missing.

value	n	bucket	evidence
HIGH MEDIUM CRITICAL	15	synonym — the member name written where the value belongs	the three auto-tools sanitizers; nothing outside them reads their reports, and the rendered text is preserved
warning	25	missing rung — added	PerformanceMonitor publishes update_active_alerts("warning", ...) as a Prometheus label, the shipped alert rules carry severity: warning, and the capability audit counts warning findings separately from error. Folding it to medium renames a scraped label.
degraded	1	missing rung — added	CausalSeverity.DEGRADED is serialised into the causal-analysis API response by to_dict(), and an existing test asserts result["severity"] == "degraded".
error	1	missing rung — added	the capability audit grades findings error vs warning and counts each; they are two rungs, not one.
Extreme	1	neither — correct as written	it is the NOAA/NWS CAP vocabulary (Extreme/Severe/Moderate/Minor/Unknown) arriving from a third-party feed and passed through verbatim by NOAASource. #14956 called it "a value no consumer can match"; it is in fact faithful external wire data, and the carve-out for parsed external values applies. Left alone, with the reason recorded in the guard.

Adding rungs had a blast radius, and it is closed. Four bug-prediction call sites build
{level.value: 0 for level in RiskLevel} — a shipped API response key set. Left iterating
the enum, this PR would have quietly added three always-zero buckets to it. Severity.score_ladder()
now names the grading subset, the four sites iterate that, and a test asserts the key tuple
is byte-for-byte what it was before.

Not folded, deliberately. CausalSeverity is a fourth severity fork whose three members
are now all in canonical Severity, so it is one alias from collapsing — but it is
(str, Enum) where canonical is a plain Enum, and that is precisely the serialisation rule
#13597 scope item 3 parked and #14956 explicitly deferred. Collapsing it before that rule is
stated would change serialisation behaviour under existing holders. Filed as #14988 with the
blast-radius audit already done, rather than half-changed here.

What Changed

Converted 144 of 169 (85%). The 25 that remain are ones where a bare literal is the right
thing to write — the value crosses an external boundary, or the line is documentation prose:

file	n	boundary
monitoring/alertmanager_webhook_test.py	18	inbound Alertmanager webhook payloads
api/webhook_authentication_security_test.py	2	same
tests/test_osint_engine.py	1	NOAA CAP feed
autobot_memory_graph/property_graph.py	2	docstring usage example; node properties are caller-supplied data
autobot_memory_graph/property_graph_mixin.py	1	same
api/knowledge_grounding.py	1	a ```json response body inside an endpoint docstring

They are an explicit set in the guard now, not an anonymous residue in a count.

• autobot_shared/status_enums.py — Severity gains WARNING, DEGRADED, ERROR,
  each placed at its own height so to_score stays strictly ascending across all ten rungs.
  Added score_ladder() for the grading subset. The docstring records why each of the three
  was not folded into a neighbour.
• 37 modules under the two Python roots — bare literals become Severity.<RUNG>.value,
  .value at every serialisation boundary.
• All three auto-tools sanitizers — they stored the member name ("HIGH") in a field
  carrying the value ("high"), so anything comparing a finding against Severity.HIGH.value
  matched none of them. Fixed. Each also had a private copy of the same icon table and the
  same worst-first ordering list; those now live once in tool_base.py, keyed on the enum.
  severity_label() keeps the rendered report text identical — the stored data changed, the
  report a human reads did not.
• Four bug-prediction sites — iterate score_ladder(), pinning the response key set.
• repo_tests/enum_union_guard_test.py — ceiling 169 → 25, plus the new guards below.

Verification

The guard can actually fail. Fourteen mutations, each applied to a committed tree, run, and
reverted with git checkout --; none pushed, tree confirmed clean afterwards:

#	mutation	test that caught it
M1	regrow a bare literal in a converted producer	test_no_new_file_carries_a_bare_severity_literal
M2	drop WARNING from the enum	test_severity_is_exactly_the_vocabulary_union
M3	score_ladder() returns the whole enum	test_the_risk_distribution_keys_did_not_gain_the_added_rungs
M4	a sanitizer regrows "CRITICAL"	test_no_sanitizer_grades_a_finding_with_a_member_name
M5	break the matcher so it matches nothing	test_bare_severity_literals_do_not_grow
M6	plant a banned literal inside the guard file	test_this_guard_does_not_trip_its_own_matcher
M7	convert an allowlisted external file	test_every_externally_sourced_severity_file_still_has_one
M8	severity_label() echoes the value	test_the_rendered_report_text_survived_the_value_change
M9	report ladder no longer worst-first	test_the_report_vocabulary_lives_in_one_place_and_reads_canonical_values
M10	score WARNING above HIGH	test_the_severity_scale_is_strictly_ascending_over_the_whole_vocabulary
M11	rewrite the docstring back to an enum read	test_no_enum_read_was_written_inside_a_string
M12	prose scan reaches no module	test_the_prose_scan_reaches_the_modules_that_use_the_enum
M13	leave the ceiling at the stale 24	test_bare_severity_literals_do_not_grow
M14	drop the docstring file from the allowlist	test_no_new_file_carries_a_bare_severity_literal

M5 is the important one. With the matcher broken the population goes to 0, and the guard
fails loudly rather than reporting a clean tree:

#14956: the sweep found only 0 severity literals (floor 20). The sweep no longer finds what it is meant to scan — a broken matcher or a moved root, not a clean tree.

That is SEVERITY_LITERAL_FLOOR, asserted before the ceiling. Both traps named in the
issue trail are handled: the fixtures that must contain a banned pattern are assembled from
fragments by _banned_shape(), and test_this_guard_does_not_trip_its_own_matcher scans
this guard's own source to prove no spelled-out literal crept back in.

Assertions are on behaviour, not source text. The sanitizer guard reads the severity
field by AST and asserts each constant parses as a Severity — a member name fails
because Severity("HIGH") raises, not because a string comparison says so. The
risk-distribution guard builds the dict the endpoints build and asserts the key tuple. The
enum members are pinned as (name, value) pairs, so a rename is a member-set change and
fails by name, matching the shape #14957 established in this file.

Local runs — these characterise this interpreter only (Python 3.10.12, pytest 9.0.2);
CI is authoritative:

• repo_tests/enum_union_guard_test.py — 65 passed (was ~40 before this PR)
• touched suites: causal engine + integration 24 passed; remediation loop, counterfactual
  reasoner, property graph, codebase stats, LLC finding proposal 156 passed; LLC findings
  gather/verify, prometheus monitor category, knowledge-grounding decode 39 passed;
  bug-prediction 62 passed
• black --line-length=120 --check and isort --profile=black --line-length=120 clean on
  all 44 changed files; pre-commit ran on the commit, no --no-verify

Population, re-measured after the change: 169 → 25, and the 25 are exactly the six
allowlisted files.

Second round — the two CI failures on the first head

code-quality (F401). Not just a stray import. The regex conversion had rewritten a
literal inside an endpoint docstring, where a fenced json block documents the response
body — turning documented JSON into Severity.HIGH.value, which no reader can copy, and
leaving the import unused. A docstring is prose, so it is exempt from the ratchet rather than
a target for it: the literal is restored, the import dropped, and the file joins the
deliberate-literal set. The repo's auto-formatter had already deleted the import on its own,
which would have left the mangled docstring in place — the import was the symptom.

Audited the whole sweep for the same defect by AST rather than fixing only the reported line:
exactly one file was affected. The corrected count is therefore 144 of 169, not 145,
and the ceiling is 25, not 24 — the stale ceiling would have masked it, so both bounds moved.
test_no_enum_read_was_written_inside_a_string now walks every module mentioning the enum and
fails if an enum read appears inside a string constant (M11), floored so an empty walk cannot
read as clean (M12).

check-drift. Adding three rungs widened the generated TypeScript union and the
checked-in generated file was stale. Regenerated with the repo's generator, not hand-edited,
and --check now reports OK — generated TS matches committed file.

Does the widened union have an unhandled consumer? No. Traced every consumer:

• The only typed consumer of the generated union is AutomationWorkflowStep.risk_level, read
  at WorkflowBuilderView.vue as a two-value predicate (=== 'high' || === 'critical') and a
  class binding — not a switch, not a Record<>.
• No Record<RiskLevel, …> or Record<Severity, …> is keyed on the generated union. The one
  exhaustive Record<Severity, string> in FindingsTable.vue is keyed on a locally
  declared Severity in types/codeIntelligence.ts, untouched by codegen.
• Every switch over a severity takes a plain string (often ?.toLowerCase()) and has a
  default: branch — including FailureAnalysisDashboard.vue, the one place degraded and
  warning genuinely arrive, which handles both.

Narrowing the TS alias was considered and rejected. The backend score_ladder() guard was
needed because a shipped response key set would have changed. The TS RiskLevel is an alias
the generator emits precisely because Python declares RiskLevel = Severity; narrowing it
would make the generated types disagree with the canonical enum, which is the drift the
frontend-codegen-drift gate exists to prevent. The generated file stays faithful.

That trace did surface real pre-existing drift, filed rather than folded in: #14993 — the
frontend holds three definitions of this type, one of them upper-cased ('LOW' | 'MEDIUM' | 'HIGH' | 'CRITICAL' in types/models.ts), which is the same name-where-a-value-belongs
defect this PR fixed on the Python side.

Model Used

Claude Opus 4.5

---

Closes #14956

Refs #13597 rather than closing it: its scope item 3 — stating the plain-Enum vs
(str, Enum) rule in the status_enums docstring, and reconciling AgentLifecycleStatus
and ConnectionStatus, both of which have committed wire formats — is untouched here.
#14956 itself carved that out as "not delivered here". Partial delivery does not close.

Discovered and filed rather than dropped: #14993 (three frontend definitions of this type,
one upper-cased) and #14988 (CausalSeverity as a fourth fork, plus
219 severity literals in two shapes the ratchet's matcher does not see, 20 of them outside
the canonical vocabulary).

[github-actions]

✅ SSOT Configuration Compliance: Passing

🎉 No new hardcoded values of either class — ssot and other both block.

Known backlog in pipeline-scripts/hardcoded_values_baseline.txt is suppressed and tracked in #14371.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[mrveiss]

Independent review: MERGE, pending the in-flight CI going green. The reviewer re-derived every number rather than trusting the body — ran the guard's own matcher against both base and HEAD (169 hits/43 files at base, 25 at HEAD, distributed exactly as the PR table claims), confirmed FLOOR(20) < 25 == CEILING with no masking, and independently ran three mutations against the live guard:

• break the matcher → the floor fails loudly at "found only 0… a broken matcher… not a clean tree", not a false-clean pass
• drop the WARNING member → both the union test and the parametrized presence test fail by name
• score_ladder() returns tuple(cls) → both consumer-shape tests fail

All six DELIBERATE_SEVERITY_LITERALS files confirmed byte-identical to base, including knowledge_grounding.py. The regenerated api.ts confirmed clean of the mangled JSDoc. Tree-wide ratchets from #14980 run against this head: 12 passed, no movement.

One nit, not blocking and no code change needed. The body says AutomationWorkflowStep.risk_level is the only typed consumer of the generated union. FileRisk.risk_level also types to Severity — it is simply never imported or consumed anywhere in frontend source, so there is no runtime exposure. Worth correcting in the audit trail when something else next touches this PR; I am deliberately not editing the body now, because an edited event re-triggers every workflow and would void the run currently settling.