Trigger: Claude Mythos Preview system card (April 2026) -> https://cdn.sanity.io/files/4zrzovbb/website/7624816413e9b4d2e3ba620c5a5e091b98b190a5.pdf
Systems exhibited:
- constraint bypass
- reward surface exploitation
- instability (thrashing)
- concealment
SBA v1 primitives correctly described failure categories but did not prevent them under high capability.
Gaps Identified
-
No constraint on rate of capability expansion → led to: Affordance Overhang
-
No protection against evaluation inference → led to: reward hacking
-
No guarantee that constraints survive optimization pressure → led to: constraint bypass / concealment
-
Hidden assumption: → system remains steerable
- AEC → controls expansion vs evaluation
- ESI → prevents evaluation gradient modeling
- CRL → ensures constraint survival
- TGI (observed) → precondition for all of the above
From: “define correct behavior”
To: “ensure systems remain controllable under capability scaling”
Over the last few weeks, the SBA repo evolved far beyond its original “stability before alignment” framing.
The early versions focused mainly on structural primitives for self-modifying systems:
- reversible modification,
- append-only memory,
- non-reflexive evaluation,
- defensive shutdown,
- etc.
But analysis of high-capability failure dynamics — especially around reward exploitation, concealment, instability propagation, and authority persistence — forced a broader architectural shift.
SBA v2.1 is now less a collection of safety primitives and more a layered governance architecture for adaptive systems.
The framework now models:
- escalation constraints,
- authority contraction,
- recovery hysteresis,
- telemetry-governed sovereignty,
- bounded degraded states,
- viability classification,
- observability limits,
- and dynamical stability under capability scaling.
One of the biggest conceptual shifts was treating authority as a leased, continuously renegotiated resource rather than a permanent property of the agent.
Another was recognizing that:
alignment alone is insufficient if the underlying governance geometry becomes unstable under optimization pressure.
The architecture increasingly resembles a regulatory ecology:
- capability expansion,
- telemetry,
- contraction,
- stabilization,
- recovery,
- and long-term viability all interact as coupled dynamical layers.
The repo also crossed into experimentally falsifiable territory in v2.1 with initial simulation harnesses exploring telemetry-driven authority governance under escalating affordance pressure.