Skip to content

canonical(enums): 169 bare severity literals remain, 43 of them outside the canonical vocabulary #14956

Closed

Description

@mrveiss

Split out of #13597, which had two halves. The enum half is done; this is the remainder,
with the population measured rather than estimated.

What is already landed

  • AlertSeverity and ErrorSeverity are aliases of the canonical
    autobot_shared.status_enums.Severity (landed before this change).
  • repo_tests/enum_union_guard_test.py now asserts both aliases still resolve to Severity,
    fails if either is re-declared as its own enum, and ratchets the literal count so the
    population cannot grow
    while this issue is open.

What is left — the measured number

#13597 said 119. Re-measured on the tree at the time of filing: 169 bare
"severity": "<literal>" entries across 43 files under autobot-backend/ and
autobot_shared/. It grew by 50 between #13597 being filed and being worked.

Matcher used (the same one the ratchet uses):

["']severity["']\s*:\s*["'][A-Za-z_]+["']

Distribution of the literal values:

value count in canonical Severity?
high 50 yes
critical 32 yes
warning 25 no
medium 24 yes
low 14 yes
HIGH 9 no (wrong case)
info 5 yes
CRITICAL 3 no (wrong case)
MEDIUM 3 no (wrong case)
unknown 1 yes
error 1 no
degraded 1 no
Extreme 1 no

43 of the 169 are already outside the canonical vocabulary. That is not a tidiness
finding — it is the typo class #13597 predicted, present and shipping. "Extreme"
(tests/test_osint_engine.py:366) is a value no consumer can match; the 15 uppercase ones
(code_analysis/auto-tools/security_deep_sanitizer.py, playwright_sanitizer.py) are the
same word in the wrong case, so any consumer comparing to Severity.HIGH.value silently
misses them.

Why the ratchet ships before the sweep

A guard that prevents regrowth is worth more than a partial sweep: the count moved
119 → 169 precisely because nothing stopped it. The ceiling in
repo_tests/enum_union_guard_test.py is pinned at 169 and is to be lowered as literals
convert, never raised.

Work

  • Convert the in-process producers to Severity members, using .value at every
    serialisation boundary (str() of a member is not its value)
  • Decide the fate of warning / error / degraded — these look like a different
    vocabulary (log level, probe state), not a mis-spelled severity. Either give them
    their own enum or map them; do not silently fold them into Severity
  • Fix the 15 wrong-case and 1 nonsense (Extreme) values first — they are live defects,
    not debt
  • Leave values parsed from external scanner output alone (wire data, not domain values)
  • Lower SEVERITY_LITERAL_CEILING with each batch

Also from #13597, not delivered here

Scope item 3: Severity, TaskStatus, Priority, LLMProvider, OperationOutcome,
HealthStatus, AgentStatus are plain Enum while AgentLifecycleStatus and
ConnectionStatus are (str, Enum). Two serialisation behaviours in one module. Needs a
stated rule in the module docstring, and both existing (str, Enum) members have committed
wire formats so neither can simply be changed.

Refs #13597

Activity

  1. mrveiss commented on Aug 24, 2026

    @mrveiss
    OwnerAuthor

    Discovered while collapsing the enum forks in #14957 (closes #13845, #13846, #13597, #13578). Not fixed there — see the PR's "Discovered and filed, not fixed here" section for why the blast radius differs.

  2. github-actions commented on Aug 24, 2026

    @github-actions
    Contributor

    PR #14990 (merged to Dev_new_gui) references this issue with a close keyword.

    fix(enums): converge severity literals on the canonical enum and add the three rungs it was missing (#14956, #13597)

    If this issue is fully resolved, close it manually. If work remains, no action is needed.

  3. mrveiss commented on Aug 24, 2026

    @mrveiss
    OwnerAuthor

    Delivered and verified in Dev_new_gui.

    PR #14990 (squashed)
    Merge commit e2daa78685989c92b5ec04d0460e67e947e0240b
    In base confirmed via git merge-base --is-ancestor
    CI at merge 10/10 required contexts green, 58 success, 0 failures, 0 behind base
    Review independent code-reviewer pass, verdict MERGE

    Measured, not claimed

    144 of 169 converted (85%). Verified in base: SEVERITY_LITERAL_CEILING = 25,
    floor 20, DELIBERATE_SEVERITY_LITERALS present, Severity.score_ladder()
    defined, and knowledge_grounding.py carries zero Severity.HIGH.value.

    The reviewer re-derived the population independently rather than trusting the PR
    body — ran the guard's own matcher against both base and HEAD and got 169 hits
    across 43 files at base, 25 at HEAD, distributed exactly as claimed.

    The 43 out-of-vocabulary literals, decided per-literal

    bucket n basis
    synonym — member name written where the value belongs 15 three auto-tools sanitizers, no external reader
    missing rung, added to the enum 27 see below
    correct as written, left alone 1 Extreme is NOAA/NWS CAP wire data passed through verbatim by NOAASource

    Three rungs were genuinely missing and were added rather than folded away:
    warning is a scraped Prometheus label (update_active_alerts("warning", ...),
    and shipped alert rules carry severity: warning); degraded is serialised into
    the causal-analysis API response and asserted by an existing test; error is
    graded separately from warning by the capability audit. Folding any of them to
    a neighbour would have renamed a scraped label or changed a wire value.

    Severity.score_ladder() names the grading subset so the four bug-prediction
    sites building {level.value: 0 for level in RiskLevel} — a shipped API key set —
    did not silently gain three always-zero buckets.

    The sweep had corrupted documentation

    A flake8 F401 turned out to be a symptom: the regex had rewritten a literal
    inside an endpoint docstring's fenced JSON block, turning documented output
    into "severity": Severity.HIGH.value — uncopyable by any reader. Worse, that
    docstring is embedded as JSDoc in the OpenAPI-derived
    src/types/generated/api.ts, so the mangled text had already propagated into
    shipped frontend type definitions. The repo's auto-formatter had independently
    deleted the orphaned import, which would have left the corruption shipping in
    both artifacts with no lint error to show for it.

    The literal was restored, the whole sweep was audited by AST for the same defect
    (exactly one file affected), and a new guard
    test_no_enum_read_was_written_inside_a_string now fails if an enum read ever
    lands inside a string constant. Prose is exempt from the ratchet by rule, which is
    why the allowlist now carries two qualifying reasons rather than one.

    Guard robustness verified by mutation, not by reading

    14 mutations, all caught. The reviewer independently ran three of them against the
    live guard: breaking the matcher makes the population floor fail loudly with
    "the sweep no longer finds what it is meant to scan — a broken matcher or a moved
    root, not a clean tree"
    rather than reporting a false clean; dropping the
    WARNING member fails the vocabulary-union test by name; making score_ladder()
    return the whole enum fails both consumer-shape tests.

    Not closed by this

    Closing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions