docs: rebuild canonical product architecture on protected main

AGENTS.md

@@ -38,6 +38,26 @@ push or open a PR.
   code-scanning tools can't converge on one PR ref. Gating happens via the
   Security **job results**; do not add tools to the `code_scanning` rule.
 
+### Repository-writer lease and dependency authority
+
+- Enforce **one writer per repository branch**. Before every repository write,
+  refetch the **exact PR head and target blob SHA**. If either changed, inspect
+  the intervening work and reconcile once before editing; never overwrite an
+  independently moved branch from stale state.
+- Repositories outside `ContextualWisdomLab/contextual-orchestrator`, including
+  the central `ContextualWisdomLab/.github` control plane and repositories with
+  their own dedicated maintenance loops, are **read-only dependencies** unless
+  the task is explicitly assigned to that repository. Do not edit their
+  branches, dispatch **write-capable agents**, resolve their review threads, or
+  merge their PRs from this repository's loop.
+- Live GitHub state is authoritative. A predecessor-head, stale-head,
+  cancelled, absent, failed, queued, pending, skipped-required, or
+  synthetic-merge result is not current-head evidence and must never be reused
+  to approve or merge a later tree.
+- Do not create one-shot, self-modifying, encoded-patch, branch-local repair, or
+  temporary write-capable GitHub Actions workflows. Prefer direct reviewed
+  changes tied to the exact current head.
+
 ### Code exploration
 
 - This repo has **no `.codegraph/` index**, so use normal search
@@ -56,23 +76,26 @@ push or open a PR.
 - The reference implementation is xtrmLLMBatchPython's pgcrypto-encrypted
   Postgres credential registry (`get_credential(name)`); reuse that pattern (a
   DB-backed KV is fine) unless a dedicated KV is adopted.
-- **Known deviation to migrate:** this repo currently resolves provider API
-  keys from env — `ModelClient` reads `os.environ.get(agent.api_key_env)` in
-  `contextual_orchestrator/orchestrator.py` (and `CONTEXTUAL_ORCHESTRATOR_*`
-  tokens in `__main__.py`). Move these to KV-backed reads; keep env only as the
-  bootstrap path that seeds the KV.
+- Protected main resolves provider keys through `get_credential`; the legacy
+  `api_key_env` field is only a credential-name compatibility alias. Do not
+  reintroduce request-time environment fallback. Process/bind configuration
+  may still use explicit `CONTEXTUAL_ORCHESTRATOR_*` bootstrap inputs.
 
 ### This repo: the org LLM gateway
 
 - `contextual-orchestrator` is the org's **LLM-communication hub** — the
   OpenAI-compatible front door consumed by **gyeot** and **scopeweave**.
 - **Direction:** grow it toward a **LiteLLM-class multi-provider gateway**. The
   org is open to a **Rust/Python hybrid** to cut overhead.
-- Its `ModelClient` currently reads `os.environ.get(agent.api_key_env)` — this
-  is the KV-principle deviation above. Resolve the API key (including the org
-  `OPENAI_API_KEY`) from the **KV / credential registry**, not env.
-- The **OpenCode review pipeline is separate** and stays on **GitHub Models** —
-  do not change it.
+- Its `ModelClient` resolves the credential name through the **KV / credential
+  registry**, including `OPENAI_API_KEY`; do not add ambient environment
+  fallback at request time.
+- The **OpenCode review pipeline is separate and centrally governed** by
+  `ContextualWisdomLab/.github`. Do not hard-code or replace its provider pool,
+  reviewer identities, or credential chain from this repository. For live model
+  tests and autonomous development work owned by this repository, use
+  `NVIDIA_NIM_API_KEY`; never repurpose `COPILOT_GITHUB_TOKEN` as a model or
+  development-agent credential.
 
 ### This repo's role in the ecosystem
 
@@ -83,15 +106,15 @@ push or open a PR.
   email/PIM that DOM-decomposes emails/files into a persisted knowledge graph).
   Each component below is a **standalone program that must ALSO work as a git
   submodule**, grown separately and together:
-  - **waf-ids-ai-soc** — WAF / IDS / AI SOC / LB / APIM.
+  - **wardnet** — WAF / IDS / AI SOC / LB / APIM.
   - **clearfolio** — document viewer.
   - **pg-erd-cloud** — ERD tool.
   - **contextual-orchestrator** — this repo: LLM cost/perf/upstream-LB gateway
     (beyond LiteLLM).
   - **codec-carver** — STT / omni-modal speech-video codec.
   - **fast-mlsirm** — LLM-as-a-Judge calibration + evaluation-item quality
     (uses aFIPC FIPC + kaefa item-fit).
-  - **feelanet-adfs** — passwordless SSO (OIDC/SCIM/ADFS/LDAP/FIDO2/OAuth2.1,
+  - **keyverse** — passwordless SSO (OIDC/SCIM/ADFS/LDAP/FIDO2/OAuth2.1,
     eliminate passwords).
   - **newsdom-api** — PDF→DOM sidecar.
   - **semantic-data-portal** — upper ontology / catalog / governance plane with
@@ -109,3 +132,22 @@ push or open a PR.
   scheduling (e.g. LLM-cascade / model-routing and queueing/load-balancing
   papers).
 <!-- END cwl-agent-guidance -->
+
+## Canonical product documentation
+
+Start at [`docs/README.md`](docs/README.md). Root `ARCHITECTURE.md`, PRD, TRD,
+ERD, UML, ADRs, threat model, test strategy, operability, incident response,
+traceability, and references are one status-qualified graph. Behavior changes
+must update the affected authority and documentation contract test.
+
+## Execution continuity
+
+- Treat prompt edits, audits, status summaries, and documentation assessments
+  as intermediate work when the request also authorizes repository changes.
+- Continue the safe chain: verify live target state, repair the smallest
+  coherent authority set, run focused and full evidence, publish a reviewable
+  branch/PR, inspect its exact-head state, then take the next non-conflicting
+  authorized task while a control-plane check is pending.
+- Stop only for a real authority choice, destructive ambiguity, permission
+  boundary, or external dependency that blocks every safe continuation. Never
+  turn queued, absent, stale, synthetic, or status-only evidence into success.

ARCHITECTURE.md

@@ -0,0 +1,223 @@
+# Contextual Orchestrator architecture
+
+**Document state:** `accepted_architecture`<br>
+**Canonical role:** current component, trust-boundary, and deployment authority
+
+`docs/architecture.md` remains a research-to-product note. This document is the
+system architecture authority and links detailed runtime diagrams in
+`docs/UML.md` and data ownership in `docs/ERD.md`.
+
+## Architectural intent
+
+Contextual Orchestrator is one provider-neutral orchestration domain exposed as
+a Python library, CLI, and OpenAI-compatible HTTP service. It keeps policy and
+evidence inside one deployable boundary while allowing optional infrastructure
+adapters. Basic operation does not require the wider CWL ecosystem.
+
+```mermaid
+flowchart TB
+    caller["API consumer"] --> delivery["HTTP / CLI delivery"]
+    operator["Platform operator"] --> admin["Admin and evidence API"]
+    delivery --> coordinator["CostRoutingCoordinator: sync or batch"]
+    coordinator --> domain["TaskOrchestrator: route or conduct"]
+    coordinator --> ledger["Cost ledger"]
+    coordinator --> batch["Local or pg-llm-batch adapter"]
+    delivery -. passthrough / route stream .-> domain
+    admin --> domain
+    domain --> client["ModelClient provider adapter"]
+    domain --> state["Optional workflow / agent stores"]
+    client --> provider["OpenAI-compatible provider"]
+```
+
+The dotted path is a protected-main exception: raw compatible passthrough and
+route streaming bypass part of coordinator accounting. It is a documented gap,
+not the target evidence architecture.
+
+## Bounded contexts
+
+| Context | Responsibility | Does not own |
+|---|---|---|
+| Delivery | Authentication, input bounds, HTTP/CLI translation, compatible response framing. | Model policy or provider credentials. |
+| Orchestration domain | Route/conduct choice, workflow plan, access lists, agent selection, verification, synthesis, trace, budget. | Host identity, tenant directory, or provider network implementation. |
+| Provider adapter | KV credential lookup, compatible request, timeout/retry, usage capture, transport validation. | Workflow policy or review authority. |
+| Cost and batch hub | Token/count provenance, configured prices, attribution, sync/batch decision, backend lifecycle. | Route/conduct policy, fabricated prices, or external batch persistence. |
+| State and credential adapters | Optional SQLite state/agent overlay, PEP-249 ledger, in-memory or pgcrypto credentials. | Legal basis, tenant authorization, or enterprise backup policy. |
+| Operator evidence | Admin, trace, evaluation, access, audit, analytics, and readiness projections. | Certification, independent approval, or production SLO proof. |
+
+## Module map
+
+| Module | Role |
+|---|---|
+| `orchestrator.py` | `ModelAgent`, `WorkflowStep`, `OrchestrationPolicy`, `ModelClient`, `TaskOrchestrator`, state stores, cache, redaction, budgets, traces, and readiness reports. |
+| `server.py` | Threaded stdlib HTTP delivery, bearer scopes, validation, rate/concurrency controls, routing, SSE framing, and error translation. |
+| `admin.py` | Dependency-free operator console. |
+| `api_contract.py` | Machine-readable OpenAPI subset and operation identities. |
+| `credentials.py` | Credential protocol, in-memory backend, pgcrypto Postgres backend, and registry functions. |
+| `kv_config.py` | Intentional no-DSN in-memory configuration plus an authoritative fail-closed `pg-llm-batch` Postgres adapter on the active #96 stack. |
+| `cost_ledger.py` | Price book, prompt-safe usage records, telemetry, non-blocking export, SQL store, and rollups. |
+| `batch_routing.py` | Routing hints/policy, local and external chat/embedding batch contracts. |
+| `cost_router.py` | Coordinates token counting, sync/batch channel choice, ledger, and backend submission/retrieval. |
+| `token_counting.py` | Deterministic heuristic and optional Postgres `pg_tiktoken` adapter. |
+| `conventions.py` | Two-or-more-word snake_case validation. |
+| `__main__.py` | CLI completion, server, evaluation, and credential bootstrap. |
+
+## Control plane and data plane
+
+The control plane includes agent configuration, policy, credentials, prices,
+budgets, provider exclusions, evaluation, and operator evidence. The data plane
+includes validated request payloads, selected step context, provider requests,
+answers, usage signals, and optional batch payload references.
+
+Control-plane changes may affect later requests but cannot rewrite the evidence
+attached to a completed run. Data-plane payloads must not be copied into broad
+usage telemetry. Protected main has only admin and inference bearer scopes: no
+dedicated trace scope exists, and an inference-scoped caller may request
+`include_orchestration_trace: true`. Purpose- and tenant-specific trace authority
+is an accepted boundary that still needs host RBAC or a dedicated runtime scope.
+
+## Route and conduct
+
+`TaskOrchestrator.complete()` is the stable split:
+
+- `route` selects and calls one eligible worker. It is the only mode that can
+  honestly relay live provider SSE tokens on protected main.
+- `conduct` creates a bounded template or validated generated workflow. Each
+  `WorkflowStep.access` tuple names prior step outputs deliberately included in
+  that worker's context. Verification precedes synthesis when policy requires
+  it. Any HTTP stream is framed after the answer exists.
+
+The deterministic policy is the protected-main authority. Learned routing,
+recursive coordination, and role-specific reasoning controls require
+comparable-budget evidence before replacing it.
+
+Protected-main agent choice is deterministic tag/domain/priority scoring. It is
+not learned, price-aware, or load-balanced. `route_p95_seconds` is exposed but
+does not currently participate in dispatch, and `cheapest_upstream()` is not
+called by either routing layer.
+
+## Trust boundaries
+
+1. **Caller boundary:** bearer scope, bind policy, body/role/mode/rate/concurrency
+   validation precede orchestration.
+2. **Context boundary:** access lists limit cross-step visibility. Trace exposure
+   defaults off, but protected-main inference authority can opt in; dedicated
+   purpose/tenant trace RBAC remains `planned`.
+3. **Credential boundary:** provider secrets are names in model configuration
+   and values in KV; environment is bootstrap transport only.
+4. **Provider boundary:** protected main requires HTTPS and globally routable
+   destinations. The stronger DNS-pinned, redirect/proxy-safe, strictly bounded
+   response implementation is `active_pr` in #96.
+5. **Persistence boundary:** in-memory is default. Enabling a file or database
+   creates an operator obligation for access, encryption, retention, backup,
+   deletion, and recovery.
+6. **Evidence boundary:** a local report, check status, automated review, human
+   approval, and protected merge are different authorities.
+7. **Host boundary:** a CWL host retains identity, tenancy, legal basis,
+   business data, and deployment unless a versioned contract delegates them.
+
+## Data ownership
+
+- In-memory workflow, evaluation, audit, analytics, circuit, and cache state are
+  process-owned and ephemeral.
+- Optional SQLite stores provide standalone durability, not a normalized
+  enterprise data plane.
+- The cost ledger has an in-memory default and a portable PEP-249 SQL store.
+- The active `PriceBook` reads ConfigStore, not the SQL
+  `llm_price_entries` table. That table is created but dormant.
+- Provider credentials may be in-memory for development or pgcrypto-encrypted
+  in Postgres.
+- `docs/database_design.sql` is a normalized production target and must not be
+  confused with runtime-created SQLite schemas.
+- External batch/config/secret objects accessed through `pg-llm-batch` are
+  owned by that service or adapter.
+
+## Deployment forms
+
+### Standalone
+
+One process serves CLI or HTTP, mock or configured providers, in-memory state,
+and optional SQLite/SQL/KV adapters. Loopback binding is the safe default.
+
+### CWL composition
+
+An ingress or host authenticates the user and supplies a purpose-bound request.
+Contextual Orchestrator selects and executes models. `pg-llm-batch` may execute
+latency-tolerant work. naruon, inkspan, Clearfolio, and other systems consume
+explicit interfaces and retain their own data and authorization boundaries.
+
+## Failure domains and degraded behavior
+
+| Domain | Isolation and degraded behavior |
+|---|---|
+| One provider/model | Bounded transient client retry and eligible orchestration failover; permanent provider errors receive no same-client retry and fail when no eligible candidate remains. |
+| Credential registry | Non-mock execution fails closed; mock/offline operation remains available. |
+| Optional state store | Persistence evidence is unavailable; the service must not claim durable history. |
+| Cost export | Non-blocking store may degrade while prompt-safe health exposes the loss. |
+| External batch service | Interactive route remains independently usable; process-local job lookup is lost on restart even when an external job survives. |
+| Admin integration | Inference and library paths remain independently usable. |
+| Automated review/control plane | Protected merge waits; repository-local development and verification continue. |
+
+## Architecture invariants
+
+- Agent pools are data, not provider-specific branches in domain logic.
+- Access is explicit; a worker never receives all previous outputs by default.
+- Credentials are resolved by name at the provider boundary.
+- Estimates are labeled and unknown prices remain unknown.
+- Optional integrations do not break standalone behavior.
+- No repository-local result claims certification or independent approval.
+- New scientific arithmetic owned by this service is Rust-first with
+  parity-verified CPU/GPU paths; currently such arithmetic is `out_of_scope`.
+- Database identifiers use two-or-more-word snake_case unless an external
+  standard fixes the field name.
+
+## Known protected-main divergences
+
+- Workflow-derived spend/budget and the independent cost ledger are not
+  synchronized. Missing ledger price becomes zero while spend analytics labels
+  it unknown; this violates the accepted unknown-price invariant.
+- Raw passthrough records analytics but no workflow or ledger row. Route
+  streaming bypasses the coordinator and durable `_StateStore`; a mid-stream
+  failure can leave no retained run.
+- Coordinator batch handles are process-local. Restart loses lookup, and chat
+  result replay can duplicate usage; embedding idempotency is also process-local.
+- Static OpenAPI, runtime dispatch, scopes, and endpoint prose are separate
+  authorities and have drifted.
+- Protected `main` may still downgrade a configured Postgres configuration path
+  to process-local memory. The `active_pr` #96 stack used by this documentation
+  branch changes that behavior: An explicitly configured Postgres KV backend is
+  authoritative and fails closed with ConfigBackendUnavailableError. This is
+  not protected-main behavior until #96 merges.
+- Token counting may deliberately degrade to the documented heuristic when the
+  optional Postgres counter cannot be constructed; that result remains
+  estimated evidence and must be operator-visible.
+- Commercial/readiness responses are derived documents, not persisted domain
+  entities or external attestations.
+
+## Status-qualified evolution
+
+- PR #96: `active_pr` provider transport, response trust, and configured-KV
+  fail-closed boundary.
+- PR #109: `active_pr` local loopback MLX provider and audited model judgment.
+- PR #111: `active_pr` price-aware tie-breaking and administrator credential
+  workflow; valid security and evidence blockers remain.
+- PR #112: `active_pr` fail-closed commercial release authorization; it is not
+  protected authority until integration.
+- PR #114: `active_pr` equivalent-endpoint racing; issue acceptance remains
+  incomplete.
+- PR #66: `superseded` closed-unmerged synchronous-embeddings and KV-bootstrap
+  evidence; the requirement remains planned.
+- PR #82: `superseded` closed-unmerged dependency-bootstrap evidence; rebuild
+  only its unique intent after #96 protects main.
+- PR #90: `superseded` closed-unmerged NIM benchmark evidence; issue #86
+  remains planned.
+- PR #94: `superseded` closed-unmerged free-first fallback evidence; the
+  requirement remains planned.
+- PR #99: `superseded` closed-unmerged adaptive-reasoning evidence; the
+  requirement remains planned.
+- PR #121: `superseded` closed-unmerged partial liveness/readiness, request-
+  framing, and trace-authority evidence. Issues #117, #118, and #119 remain
+  planned and require a complete test-first rebuild on the accepted #96 line.
+
+No active pull request is architecture authority until its exact head passes
+repository policy and reaches protected main. See `docs/TRACEABILITY.md` for the
+dated repository snapshot and `docs/adr/README.md` for decision status.