Investigate: STE-inspired controlled authoring standard for skill normative text #591

Summary

Investigate whether an STE-inspired controlled authoring standard for the normative portions of skills (phases, hard rules, gate conditions) measurably improves agent compliance, and if so, codify it into the skill-author v2 house style with CI lint enforcement. This is an investigation with an experiment gate, not a committed rewrite of 23 skills.

Explicit non-goal: literal ASD-STE100 adoption. The STE dictionary is aerospace-domain (no "commit", "lint", "subagent"); what would actually be built is a custom controlled vocabulary plus a subset of STE writing rules. Marketing/essay prose is out of scope.

Motivation

Instruction-following research says the failure modes controlled language attacks are exactly the ones a prompt-level governance layer suffers:

• Compliance degrades with instruction count and length: best frontier models followed ~68% of instructions at 500 simultaneous instructions, with primacy bias (IFScale, https://arxiv.org/abs/2507.11538); agentic-scenario compliance declines with instruction word count (AgentIF, https://arxiv.org/abs/2505.16944).
• Every resident token spends a finite attention budget; material mid-context effectively fades (Lost in the Middle, https://arxiv.org/abs/2307.03172; Anthropic context-engineering guidance, https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents).
• Instruction hierarchies are unreliable, and ambiguity is the raw material agents rationalize with (Control Illusion, https://arxiv.org/abs/2502.15851).

Shorter, unambiguous, imperative normative text is therefore a plausible compliance lever: fewer tokens per rule, one meaning per word, no modal wiggle room.

The house style is already halfway there: locked vocabulary ("gate", never "guardrail"/"checkpoint"), the invoked/routes/dispatches verb discipline, MUST/never phrasing. This issue asks whether finishing the job pays.

Full research capture (four dossiers + findings) lives in the arbiterforge-site repo under .codearbiter/research/ (2026-08-03 sweep).

Proposed candidate ruleset (draft, to be validated by the experiment)

Split every skill body into two marked tiers:

Normative tier (phases, hard rules, gate conditions) obeys:

1. Imperative mood, active voice. "Run the suite." not "The suite should be run."
2. One instruction per sentence. No chained clauses carrying two obligations.
3. Sentence cap ~20 words for instructions, ~25 for conditions.
4. Controlled vocabulary only: one term per concept, one concept per term, maintained as a word list (extends the existing gate/route/dispatch lexicon).
5. Banned modals and hedges in normative text: should, may, consider, typically, generally, ideally, where possible. Obligations are MUST / never; everything else is not normative.
6. Conditions state the observable check, not the intent: "If the suite exits non-zero, stop." not "If tests seem broken, be careful."
7. No forward references a model must hold in memory; each phase self-contains its gate condition.

Rationale tier (the "why") stays ordinary prose, capped short (guideline: <=2 sentences per rule), clearly marked so it is never parsed as obligation. Rationale is retained deliberately: brief why-text helps models generalize to situations the enumerated steps did not anticipate; stripping it trades edge-case judgment for happy-path compliance.

Enforcement (if adopted)

• Codify the ruleset into skill-author's Phase 4 self-review (v2 house style).
• Make it mechanical, not a discipline: a Vale ruleset (custom vocabulary + banned-modal + sentence-length checks scoped to normative-tier blocks) run in CI over core/surface/skills/**. Same medicine as the INDEX-drift finding from the same research sweep: checkable invariants over authoring discipline.

Experiment gate (do this before converting anything at scale)

There is no direct study showing STE-style text improves LLM compliance; the case above is inference. Test it cheaply first:

1. Pick one high-traffic skill (commit-gate is the obvious candidate).
2. Produce an STE-styled rewrite of its normative tier; hold rationale and semantics constant.
3. A/B against the current text: existing suite + a scripted set of gate-pressure scenarios (attempted wildcard add, commit-on-red, phase-skipping), across at least two hosts (ca, ca-codex) to control for model family.
4. Measure: gate-compliance rate, override/BLOCK event counts, token footprint of the loaded skill body, and any change in false-positive stops.

Acceptance criteria for this investigation

• Candidate ruleset reviewed and frozen for the experiment
• commit-gate normative-tier rewrite produced (semantics-preserving; diff reviewed)
• A/B results captured with the four measures above, two hosts minimum
• Decision recorded: adopt (ADR + house-style update + Vale CI), adapt, or reject with evidence
• If adopted: conversion plan for remaining skills ordered by routing frequency

Origin

2026-08-03 research sweep for the arbiterforge.com "Why hard gates" essay; surfaced alongside related findings (INDEX/body drift should be CI-checked; blocking vs. reminder hook tiers should be classified) which are candidates for separate issues.