Repository navigation
docs: doc-wide under-claim audit — extends maturity review across all of docs/ - #371
Merged
Merged
Conversation
… of docs/ Companion to 2026-05-31-maturity-doc-vs-code.md (PR #370). Same methodology, broader surface: every .md file under docs/. Findings (8 files affected): - docs/roadmap.md: 4 entries already shipped or partially shipped - docs/security.md: 'not yet enforced' callout has 2 stale entries - docs/compliance.md: inherits maturity.md under-claims - docs/api/crd-reference.md: meshAuthBackend 'scaffolded' is stale (EntraAgentIdentity wired end-to-end, verified live on AKS) - docs/architecture.md: macOS Docker Desktop caveat missing, entra-auth-sidecar absent from architecture diagrams - docs/use-cases.md: TUI undersold (1042 LOC, security panel, etc.) - docs/use-cases/exec-brief-walkthrough.md: harness claim outdated (docker now 9/9 as of PR #367) - docs/maturity.md: see PR #370 Total under-claim or outdated entries identified: 8 status changes + 10 missing rows/sections + 1 missing CRD reference page. Three suggested remediation PRs sketched in §H: 1. 'docs: refresh what ships today' (~150 LOC, low risk) 2. 'docs: architecture diagrams + KarsAuthConfig CRD ref' (~300 LOC) 3. 'docs: maturity.md restructure' (~250 LOC, ties into PR #370) No fixes applied — analysis only, consistent with the validate-first, fix-later pattern. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: Pal Lakatos-Toth <pallakatos@github.com>
Dependency Review✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.Scanned FilesNone |
Pal Lakatos-Toth (pallakatos)
added a commit
that referenced
this pull request
May 31, 2026
Following PRs #370 (maturity.md review) and #371 (doc-wide audit), this extends the analysis specifically into the architecture/CRD/AGT-boundary doc surface. Methodology: trace every claim about a CRD field, component, deployment, provider variant, or guarantee to the actual code. Findings: A. CRDs (docs/api/crd-reference.md) - KarsSandbox spec table covers 8 fields; code has 17. 9 missing (agent, azureServices, resources, upstreamCompatibility, meshAuth, sigsAgentSandbox, upstreamSandboxRef, aiConformanceReference, customSecurityAttributes) — mostly Phase-6 Entra additions. - KarsSandbox.status missing agentIdentity (operators look this up via kubectl get karssandbox -o jsonpath='{.status.agentIdentity.appId}'). - EntraAgentIdentity status caveats (line 684,694) stale (covered in PR #371 §D.1). B. Architecture (docs/architecture.md) - §Components table lists 4 components; code has 6 first-class (missing mesh-plugin, conformance-runner, eval-corpus). - 'The nine CRDs' subheader should be 'nine workload CRDs' (text body correctly says 11 total). C. AGT boundary (docs/architecture/agt-boundary.md) — MOST OUTDATED: - §1 claims 'A2A AgentCard signing — until AGT ships'; but kars-a2a-core/src/card_signing.rs is 494 LOC of working Ed25519 JWS signer (sign_card + verify_card + TrustedKeys). - §2 provider variant 'AgtRustSdk' doesn't exist in code (it's 'Agt'). - §3 'KarsSandbox.spec.agt.outageMode' field doesn't exist. - §4 'kars never builds a standalone audit chain' — but inference-router/src/audit/merkle.rs exists (library-only Merkle anchoring). Needs footnote. - §5 'kars always builds' list omits 7 VAPs, AP2 trust/ledger, Phase 6 Entra Agent ID surface. D. Architecture diagrams (docs/architecture-diagrams.md) - §7 CRD diagram shows 9 workload CRDs; doc says 'the nine CRDs' but cluster has 11 (missing KarsAuthConfig + KarsPairing). - §8 cluster topology mermaid omits entra-auth-sidecar despite being live in every AKS cluster post-PR #360. - §8 shows a2a-gateway in kars-system but it's not deployed by default (needs gating or removal). - §6 control plane diagram represents controller as single box; code has 11 separate reconciler modules. E. A2A gateway (docs/architecture/a2a-gateway.md) - 8445 listener config-only — accurately disclosed ✅ - Known limitations should drop AP2 if listed (AP2 IS wired). Total: 16 actionable findings; 2 suggested doc-only PRs sketched in §G. No fixes applied — analysis only, per the established validate-first pattern. File: docs/internal/security-validations/2026-05-31-crd-arch-agtboundary-deep-audit.md (21227 chars, 250 lines). Signed-off-by: Pal Lakatos-Toth <pallakatos@github.com> Co-authored-by: Pal Lakatos-Toth <pallakatos@github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Companion to PR #370 (
docs/internal/security-validations/2026-05-31-maturity-doc-vs-code.md). Same methodology, broader surface: every.mdfile underdocs/.User asked: "so considering this review the whole doc and see if there are any underclaimed thing..."
Headline: 8 files have under-claims, all rooted in the same pattern
docs/maturity.mddocs/roadmap.mddocs/security.mddocs/compliance.mddocs/api/crd-reference.mdmeshAuthBackendmarked "scaffolded" — actually live on AKSdocs/architecture.mddocs/use-cases.mddocs/use-cases/exec-brief-walkthrough.mdMost material findings
docs/roadmap.mdline 29 — "aggregate token budgets not yet metered" is wrong.inference-router/src/routes/inference.rs:295(and the other two inference routes) callcheck_budget(policy.daily_tokens, policy.monthly_tokens). 641 LOC ofbudget.rswith UTC-calendar daily + monthly counters, on-disk persistence, HTTP 429 on overrun.docs/api/crd-reference.mdline 694 —EntraAgentIdentity (scaffolded; see roadmap)is stale. Verified live on AKS:MESH_AUTH_BACKEND=EntraAgentIdentity, 4 unique per-sandboxPINNED_AGENT_IDENTITY_APP_IDvalues, entra-auth-sidecar issuing tokens scopedhttps://ai.azure.com/.default. PR feat(entra-agent-id): per-sandbox Entra Agent ID foundation #360 (Phase 6) shipped this 2 days ago.docs/security.mdline 22-25 — drops 2 stale entries: "signed-OCI advisory" (actually authoritative when opted in), "attest scaffolded" (618 LOC of working CLI code).docs/architecture.mdline 25 footnote — single-container macOS Docker Desktop caveat missing. Yesterday's validation (PR docs(security-validation): cross-platform validation report — AKS, local-k8s, docker #368, Finding Bump jsonwebtoken from 9.3.1 to 10.3.0 #1) confirmed UID 1000 reads the API key file due to Docker Desktop VirtioFS UID virtualization.Total
KarsAuthConfig)Three suggested remediation PRs sketched in §H. No fixes applied — analysis only.
File added
docs/internal/security-validations/2026-05-31-docs-wide-underclaim-audit.md(239 lines)