Skip to content

umbrella(docs): map AutoBot's control plane — every layer, its doc, and the code that enforces it #16835

Closed
0 of 13 issues completed

Description

@mrveiss

What controls AutoBot? There is no one place that answers this

Every control layer below has at least one document. What does not exist is a map — a single
page that names each layer, points at the doc that defines it, and points at the code that enforces
it. Without that, "what stops an agent doing X" is answerable only by someone who already knows, and
a layer whose doc has drifted from its implementation looks identical to one that has not.

This umbrella is to produce that map and, in producing it, find the layers where the documentation
describes something the code no longer does.

Layers named in the request, with what exists today

Established by locating dedicated documentation per layer — this is a filename and path survey,
not a check that each doc is current:

Layer Dedicated doc found
Data access / tenancy docs/developer/AUTHENTICATION_RBAC.md, docs/llc/tenant-context.md
Authentication docs/developer/AUTHENTICATION_RBAC.md, docs/developer/ROLES.md
System instructions docs/developer/PROMPT_MIDDLEWARE_GUIDE.md
Tool permissions docs/security/FILE_PERMISSIONS_SECURITY_ARCHITECTURE.md — but see below
Logging docs/developer/LOGGING_STANDARDS.md
Escalation docs/developer/APPROVAL_STATUS_DESIGN_IMPROVEMENT.md, docs/architecture/TERMINAL_APPROVAL_WORKFLOW.md
Governance docs/architecture/EU_AI_GOVERNANCE.md
Backup / restore docs/operations/disaster-recovery.md
Scaling docs/operations/scaling-strategy.md, docs/architecture/Scaling_Roadmap_and_Architecture_Evolution.md

The one apparent gap in that list: the only doc matching "permission" covers file permissions.
Whether an agent's tool permissions — what it may invoke, and what forces an approval — are
documented anywhere is not established by this survey and is the first thing to check.

Layers that appear to be missing from the list

Each verified present in code, so these are real controls currently outside the map:

Layer Enforcing code
Secrets management autobot-backend/services/secrets_service.py
Input provenance / prompt-injection defence security/content_firewall.py, knowledge/ingest_sanitize.py (umbrella #16777)
Network egress policy research_browser_manager.py (the guarded-fetch rule, core rule 8)
Data retention & deletion api/admin_retention_policies.py
Emergency stop / kill switch api/advanced_control.py (emergency_system_stop)
Update & deployment control builtin updater / code-sync path — the rule that no other route may deploy
Model & provider routing docs/developer/TIERED_MODEL_ROUTING.md, llm_shared/providers/

And one that may genuinely not exist: rate limiting / cost control. A grep for rate_limit in
autobot-backend/ returns only conftest.py and a test module — no production call site surfaced.
That is a survey result, not a conclusion; it needs someone to confirm whether request or spend
limiting exists under another name before it is treated as absent.

Why a map rather than nine more documents

The documents mostly exist. Three things do not:

  1. No index. Nothing lists the control layers as a set, so "did we cover everything?" cannot be
    asked, which is exactly the question the request opens with — "might have left something out".
  2. No doc-to-enforcement link. A layer's doc and the code that enforces it are not connected, so
    drift is silent. This repository has spent significant effort on guards that distinguish
    nothing found from did not look (docs/developer/MEASUREMENT_DISCIPLINE.md); the control
    documentation currently cannot make that distinction about itself.
  3. No statement of what is deliberately absent. Rate limiting may be missing by decision or by
    oversight, and today nothing records which.

Acceptance criteria

  • A single control-plane document exists, listing every layer above, and for each: what it
    controls, the doc that defines it, and the file:symbol that enforces it.
  • Each layer's entry is verified against current code on main, with the evidence quoted — not
    inherited from the existing doc's own claims.
  • Tool permissions are resolved: either the doc that covers them is named, or the gap is filed.
  • Rate limiting / cost control is resolved: either the enforcing code is named, or its absence is
    recorded as deliberate or filed as a gap.
  • Every layer with no enforcement found, or whose doc contradicts the code, has a child issue.
  • The map states its own survey method, so the next person can re-run it rather than re-derive it.

Children to follow once the survey lands; filing this first so the scope is visible rather than
discovered one gap at a time.

Activity

  1. added sub-issues on Sep 16, 2026
  2. mrveiss commented on Sep 17, 2026

    @mrveiss
    OwnerAuthor

    Closing — criteria verified against the merged map on main

    PR #16852 merged as 27908606d. It did not auto-close this issue: the body carries no Closes
    keyword. That is a side effect of fixing the PR-body gates — switching the 13 child references to
    Refs to satisfy Check same-scope batching removed the only closing keyword along with them. Worth
    noting as a hazard of that gate, not a fault of the work.

    Verified against origin/main, not against the PR:

    13 of 17 layers carried real drift or no doc. Three were clean; scaling was reclassified from a
    control to non-enforced ops guidance, and that reclassification was independently re-checked by a
    second reviewer who re-ran the autoscaling grep rather than accepting it.

    Two findings from the survey were serious enough to become their own work rather than doc corrections:
    #16843 (emergency stop passes no affected_tasks, and nothing reads the paused set — clicking it
    stops nothing) and #16845 (cost tracking calling a method that does not exist, swallowed at debug
    level since April, so Budget Policy never fired for chat traffic). Both are in flight.

    The method is what makes this closable: five parallel investigations, each instructed to re-run any
    cited command and re-read any cited code rather than trust doc text
    , and a second reviewer who
    re-derived three rows independently — including reading all four retention-purge task files directly
    rather than re-running the same grep.

  3. mrveiss commented on Sep 21, 2026

    @mrveiss
    OwnerAuthor

    Reopened by owner ruling (2026-09-21).

    Closed 2026-09-17 while still holding 13 open children — #16839, #16840, #16841, #16842, #16843, #16844, #16845, #16846, #16847, #16848, #16849, #16850, #16851. Verified against the native sub-issue edges, not a label or title probe; a label-based sweep is what hid this umbrella and 13 of the 19 currently-orphaned children in the first place.

    An umbrella closed over open children stops aggregating them: each child stays findable on its own, but nothing reports the set, so the remaining work becomes invisible at the level it was being tracked at. Reopening restores that.

    This stays open until its children are genuinely delivered, not to tidy the board. The owner has also ruled that the general case gets a guard rather than repeated sweeps — an umbrella will be blocked from closing while it holds open children. Filing that separately; this reopen is the one-off correction.

  4. mrveiss commented on Oct 3, 2026

    @mrveiss
    OwnerAuthor

    Closure: 6 of 6 verified against origin/main (1edec3064c)

    Landed in 0e5d2f29ab: docs/architecture/CONTROL_PLANE_MAP.md (254 lines), linked from docs/architecture/_index.md.

    Criterion Evidence
    One document listing every layer: what it controls, its doc, the enforcing file:symbol Summary table :32-50, 17 rows, each with Controls / Doc / Enforcing code / Verdict / Issue. Per-layer detail sections ### 1. … ### 17. follow it.
    Each layer verified against code on main, evidence quoted rather than inherited from the doc "Survey method" (:6-28): "checked independently against current code on main — never by trusting the layer's own doc … A citation here is either a file:line/symbol … or an exact search command and its real output". Every layer resolves to one of four defined verdicts; none is "could not determine".
    Tool permissions resolved Row 5, Doc gap: enforcement is real at middleware/builtin/permission_enforcement.py PermissionEnforcementExtension, with no correct doc. Filed as #16841.
    Rate limiting / cost control resolved Row 17, Drifted: autobot_shared/rate_limiter.py::RateLimiter named as partial enforcement; the cost-tracker and uninvoked-limiter gaps filed as #16845.
    Every unenforced or contradicted layer has a child issue Every Drifted or Doc gap layer cites one: #16839, #16840, #16841, #16842, #16843, #16844, #16845, #16846, #16847, #16848, #16849, #16850 (+ #16851 for stale rows). All 13 are native sub-issues of this one, and all are open. The two layers without an issue are Enforced (7, network egress) and Not a control (15, scaling).
    The map states its own survey method :6-28, as quoted above, including how to re-run it

    Spot-check of the map's own citations: row 2 cites auth_middleware.py:864 get_current_user and :967 check_admin_permission. Both symbols still exist, now at :859 and :962: line drift of 5, the symbols intact. Row 5's permission_enforcement.py:60 PermissionEnforcementExtension is exact. That is the drift the survey method tells a re-runner to expect; it does not invalidate a verdict.

    This issue's deliverable was the map plus a child issue per gap, and both exist. The 13 children stay open and carry the fixes. Closing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    docsdocumentationImprovements or additions to documentation

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions