Skip to content

refactor(config): the #18094 de-fork remainder — one key reader, one feature flag, one language detector, and image-gen keys vault-first through an origin-bound resolver (#18094) - #18371

Open
mrveiss wants to merge 27 commits into
mainfrom
issue-18094-defork-remainder
Open

mrveiss wants to merge 27 commits into
mainfrom
issue-18094-defork-remainder

Conversation

@mrveiss

@mrveiss mrveiss commented Oct 12, 2026 •

Copy link
Copy Markdown
Owner

Thinking Path

#18094 is the remainder of the de-fork work after #17419/#16177: forked env defaults, shadowed constants and same-name functions. It was finished from a local-only branch from 2026-10-10 and rebased onto current main, roughly 190 commits later. Commits main had superseded were dropped: the analytics embedding model is delivered by #18143's index-space table. It also carries #17856 rows 13a and 16, which were deferred to this branch, the #15722 digest entry it owns, and the owner's 2026-10-12 decision on credential precedence.

What Changed

  • Env and constants:

    • the un-prefixed NPU/SLM/provider vars are registered (autobot_shared/env_registry_npu.py);
    • NPU_WORKER_HOST is split into bind and connect hosts;
    • one global-user sentinel (__global__), whose readers accept both spellings;
    • the similarity constants are homed in analytics_infrastructure.py;
    • the browser-VM noVNC URL is in ssot_config and config.compat is removed;
    • the SLM agent fails clearly on a missing node id, and the key name comes from node_proxy.
  • One detect_language. The NPU agent maps the shared enum back to its emitted names.

  • inventory(canonical): 19 concepts with more than one definition and no existing issue — led by two secret stores whose docstrings each claim they cannot disagree about the key #17856 row 13a: one internal-API-key reader, node_proxy.internal_api_key(), used by auth_middleware, the SLM auth, the vault client, the agent and its mirror, and the migration precheck. Guard: one_internal_api_key_reader_17856_test.py, with 12 contrast fixtures and a reach floor.

  • inventory(canonical): 19 concepts with more than one definition and no existing issue — led by two secret stores whose docstrings each claim they cannot disagree about the key #17856 row 16: one is_feature_enabled (autobot_shared/feature_flags.py). Three copies with no production callers are retired, and 10 stale fork-baseline entries are dropped.

  • canonical(knowledge): two functions named content_hash return different values — full digest vs 16-char truncation #15722: the analytics dedup key goes through content_digest(width=16), byte-identical.

  • Security, image-generation credentials:

    • provider keys resolve through provider_key_vault, and a scoped resolver is injected only into the built-in plugin, bound to its origin, not its self-declared name;
    • review found the startup PluginManager in lifespan.py passed no resolvers at all. Fixed: one plugin_paths.py supplies search dirs and trusted roots to both hosts;
    • a CWD-relative plugins/core-plugins is searched but never trusted;
    • THREAT_MODEL records the origin-bound resolver.
  • Credential precedence (owner, 2026-10-12): the vault comes first. resolve_provider_key defines the order once, and resolve_image_provider_key delegates to it. The env var is only a fallback, logged once per name with no value. Hydration loads every vaulted name, even when its env var is set.

  • An unreadable vault key is refused (owner, 2026-10-12). When the vault holds a key but the read fails, _hydrate_one_key records the name as unreadable, which is distinct from "not in the vault". resolve_provider_key then raises VaultKeyUnavailableError ("vault unavailable: ... refusing the environment fallback") and never returns the env value, which may be rotated out. A later successful hydration, a capture, or removal of the vault row clears the refusal. The two log lines are distinct: "vault unreadable: refusing" at ERROR, and "vault has no key: using the environment" at WARNING. Only registry labels are logged, never a value. registration_refusal() contains a refusal at a registration site, so one refused key skips one provider.

  • A vault outage refuses every key (review of refactor(config): the #18094 de-fork remainder — one key reader, one feature flag, one language detector, and image-gen keys vault-first through an origin-bound resolver (#18094) #18371). A configured vault (AUTOBOT_SECRETS_ROOT_KEY set) whose startup hydration fails leaves "does it hold this key?" unknown. Failures that count are an unreachable DB, a failing session factory, and a query or decrypt error partway through. The vault state is recorded as not_hydrated, not_configured, reachable or unreachable. While it is unreachable, every vault-resolved name is refused ("could not be reached at startup") until a hydration succeeds. Env is allowed in one case only: the vault was reachable and has no row for that name.

    lifespan.py no longer swallows the failure at debug level as "non-critical". hydrate_at_startup() classifies the vault, logs at WARNING or ERROR, and never raises.

  • Second review (9a): env only for keys the vault truly lacks.

    • Root key unset with rows in the vault. Hydration no longer reads an unset AUTOBOT_SECRETS_ROOT_KEY as "no vault". Listing rows needs no root key, so every held name is listed. Without a usable root key (unset or malformed), each held name is recorded as unreadable and refused. Only names with no row fall back to env. VAULT_NOT_CONFIGURED is removed.

    • Every process hydrates, not only the lifespan.

      • resolve_provider_key hydrates on first resolution in any process that has not read the vault, root key or not, since listing rows needs none. A DB the process cannot reach is an outage, and keys are refused. This is services/provider_key_process_init.py: it runs on user_management.database.isolated_session_factory (unpooled, disposed), in a private event loop on its own thread. It is safe inside a running loop, never raises, and fails closed if that thread dies.
      • celery_app also hydrates in worker_process_init, before a worker's first task.
      • The test that pinned "never hydrated → env" has been replaced.
    • Stale and partial state. Hydration rebuilds the cache from the rows into a new map and swaps it in, so a deleted row is never served again and a reader never sees a half-filled cache.

    • Non-lifespan entry points and how each is covered:

      Entry point Resolves keys via Covered by
      Celery worker (celery_app, e.g. knowledge_tasks → knowledge/base.py) KB LLM/embedding, LLM providers worker_process_init hydration, plus the first-resolution hook
      Celery beat none (schedules only; beat_init reconciles the schedule) nothing to cover
      Operator scripts under autobot-infrastructure/shared/scripts that drive backend internals (infra: ten infrastructure scripts import backend internals and fail standalone — one guard for the class #18288 BACKEND_TOOLS/UNCLASSIFIED, e.g. verify_knowledge_consistency.py, analysis/test_llm_config.py) KnowledgeBase, LLM interfaces first-resolution hook when a root key is set
      Knowledge-base MCP server (shared/mcp/tools/knowledge-base-mcp/.../embedded.py) KnowledgeBase first-resolution hook
      Backend CLIs (autobot-backend/cli) and code_intelligence/shared/process_offload.py, knowledge/rag_benchmarks.py subprocesses any provider they reach first-resolution hook. A forked child inherits the parent's hydrated state; a spawned child hydrates on first use.
      Plugins (image, video) the injected resolver run inside the backend, which is hydrated by the lifespan. Standalone, the CredentialSlot reads env by design.
      SLM backend and SLM-invoked tools none. The SLM does not import provider_key_vault (system_secrets_vault.py:91,243 mention it only in prose) out of scope
    • Third review (9a):

      • The root-key gate on lazy hydration is removed, so there is no residual left. The previous "no root key → env" test is flipped into a contrast test.
      • Unit tests have no vault DB, so the harness supplies the state rather than a production gate: testkit/provider_key_vault_state.py, an autouse fixture imported by the backend conftest, starts every test against an explicit empty, reachable vault.
      • The hydration thread is a daemon. The wait is bounded by AUTOBOT_PROVIDER_KEY_HYDRATION_TIMEOUT_S (default 30). A timeout fails closed and releases the lock.
      • An unreachable vault is retried on a later resolution after AUTOBOT_PROVIDER_KEY_REHYDRATE_BACKOFF_S (default 30). The delay doubles per failed retry, is capped at 20x, and resets on success. Both variables are registered.
  • More credential readers go through the resolver.

    • Video: RUNWAY, SORA and KLING keys take the same vault-first, origin-bound path as image. That covers the plugin, the backend's file-loaded copy of its providers module (resolver bound to the plugin's origin), and GET /video-generation/providers through key_status(), which is shared with image.
    • Shared plugin plumbing: both plugins use one SDK CredentialSlot, replacing the image plugin's private copy, and the SDK's CredentialUnavailableError, which VaultKeyUnavailableError subclasses.
    • Tilde: TILDE_API_KEY is vault-resolved.
    • Guard: repo_tests/provider_key_env_reads_route_through_vault_18094_test.py fails on any env read of a registered or credential-shaped name that bypasses resolve_provider_key. It has 10 contrast fixtures, a reach floor of 2617, a registered scope, and a shrink-only allowlist.
    • Not routed: MODAL_TOKEN_* (the Modal SDK reads the environment itself) and PAPERCLIP_API_KEY (autobot_shared cannot import the backend resolver). Both are allowlisted under Modal and Paperclip credentials bypass the vault-first provider-key resolver #18383, a sub-issue of tech-debt: forked env defaults, shadowed constants and same-name functions — the remainder after #17419/#16177/#18093 #18094.
  • How each caller surfaces a refusal:

    • Registration sites skip that one provider, log the reason at ERROR, and let startup continue: the LLM bootstrap (every keyed provider, custom OpenAI, OpenRouter, HuggingFace, Nous), the lifespan.py adapters, the search registry (SearXNG as a whole, Brave), and the capability registry (Slack, Discord).
    • These report the refusal as a state rather than raising it:
      • provider health checkers return UNAVAILABLE with the reason;
      • GET /image-generation/providers marks that provider unavailable with the reason;
      • voice realtime is_configured returns False and logs;
      • the Slack hook turns into a logged, uncached null hook;
      • the image plugin load logs and continues.
    • At call time the call fails with the reason:
      • LLM providers return LLMResponse.error;
      • adapters raise to their caller;
      • KB init logs and returns False;
      • the claude-code backend fails its health check and execution;
      • email notification dispatch logs the failure;
      • threat-intelligence construction raises into the security API's error handler;
      • voice realtime negotiate() raises.

Behaviour changes

  • A deployment with a vault value and an env var for the same name now uses the vault value. Processes that never hydrate the cache (outside the backend lifespan) still resolve from env, and now log a fallback warning. In the multi-name chains (HF_TOKEN → HUGGINGFACE_API_TOKEN → NOUS_API_KEY), the order of names is unchanged; precedence changes per name.
  • A vault outage at startup now refuses every vault-resolved key instead of silently using env. Before, it was logged at debug as "non-critical".
  • An unset root key no longer means "no vault". Names the vault holds are refused; only names with no row use env.
  • Celery workers, and every other process that resolves a key, now read the vault before serving one, root key or not. Before, they served env without reading it. If such a process cannot reach the vault DB, it refuses keys and retries with backoff.
  • TILDE_API_KEY and the RUNWAY, SORA and KLING video keys now resolve vault-first, like the other provider keys. A video provider whose key is refused reports that reason in the listing and on /generate.
  • A vault row that cannot be read is now refused, not resolved from env. A provider whose key the vault holds but cannot read is not registered (logged at ERROR), or its calls fail with "vault unavailable", even when the env var is set. The env fallback applies only to keys the vault does not hold.
  • A per-instance settings["api_key"] (provider or adapter settings) still outranks resolve_provider_key. That is pre-existing and unchanged; the vault-first order applies only when no per-instance key is given.
  • detect_language now classifies .pyi, .pyw, .mjs, .cjs, .mts, .cts and .vue, which the NPU agent used to report as "unknown". This affects only newly indexed files; no stored data is rewritten.
  • The image plugin loaded at startup now resolves its keys through the vault.
  • The backend internal-key check reads the key per call instead of from cached settings; the value is the same in deployment.

Verification

  • Refusal and provider-key set, after merging main: 931 passed, 17 skipped. This covers the new provider_key_refusal_18094_test.py (12) and llm_shared/tests/test_provider_key_refusal_18094.py, plus the existing tests for every caller module and the credential-vault guard. Its cases:
    • a vault row plus a read failure plus an env var gives a refusal, never the env value, with no value in any log;
    • a key absent from the vault plus an env var gives env, with the "vault has no key" warning;
    • a failed read followed by a successful re-hydrate resolves again;
    • a capture or a removed row clears the refusal;
    • the bootstrap skips only the refused provider;
    • the image listing, Slack hook, health checker, search registry and realtime voice each contain the refusal.
  • After the review fixes: 999 passed, 17 skipped, 2 failed. Both failures are explained:
    • api/video_generation_test.py::test_generate_provider_disabled: its fake provider lacked the new unavailable_reason. Fixed in dfb06167c1, 6/6 now.
    • security/threat_intelligence_test.py::test_clear_expired: a fixed sleep(1.1) against a 1s TTL (tests: 32 more fixed sleeps are followed by an assertion, and some decide the outcome by the clock #16255). It fails only in the long mixed run under nice -n 19 and passes alone (33/33) and with its siblings (76/76). The cache is untouched here.
    • New provider_key_outage_18094_test.py covers:
      • an outage refuses even with env present, logging at ERROR with no value;
      • a failure partway through hydration is an outage;
      • no vault configured gives env;
      • reachable with the key absent gives env;
      • recovery after an outage;
      • a process that never hydrates keeps env;
      • video key scoping;
      • a Tilde refusal.
    • Plugin credential_slot_18094_test.py: video and image plugin suites pass, 22 and 7. The guard passes, 12.
  • After the second review: 1787 passed, 17 skipped, 1 failed. The failure is security/threat_intelligence_test.py::test_cache_expiration, the same fixed-sleep TTL flake class as test_clear_expired. It passes alone (33/33) and is filed as test(security): threat-intel cache TTL tests fail intermittently in long mixed runs (fixed 1.1s sleep vs 1s TTL) #18400. provider_key_outage_18094_test.py (17) now also covers:
    • root key unset refuses held names and only those;
    • a deleted row leaves no stale value;
    • the swap is atomic (a reader mid-hydration sees the previous complete cache);
    • an unhydrated process with a root key hydrates before resolving, and refuses on an outage;
    • a lazily hydrated vault value wins;
    • without a root key, an unhydrated process uses env and does not touch the DB;
    • the private-engine path classifies a DB failure as an outage, once per process, and is safe inside a running loop;
    • celery_app has the worker_process_init hydration hook.
  • Third review, run on a275bea6e2. The branch was 51 behind, so main was merged in first.
    • The hub conflicts (reach_scope_claim, the content-digest list, two reach floors) were resolved by taking main's copy and re-applying only this branch's lines.

    • After the merge:

      • provider-key, security and caller core: 513 passed, 17 skipped;
      • reach_scope_claim_17844 run alone: 10 passed;
      • ratchets plus guards: 708 passed, 2 failed. The two failures were reach floors (chunker-private-hook 4920→5059, deployment-mode-detector 4676→4737) that the merged population left in the red band. Both were re-pinned mid-window in a275bea6e2; reach_declarations plus those guards then gave 642 passed.
    • New outage tests, 21 in all, add:

      • without a root key, an unhydrated process still refuses held names (the contrast test);
      • a hung hydration fails closed after the timeout and releases the lock;
      • an unreachable vault is retried after the backoff, the delay doubles, and it resets on recovery.
    • Broad pre-merge run, 6,600 tests with every module that resolves keys: 4425 passed, 2159 skipped, 4 failed, 7 errors. None are in the code this PR changes:

      • chat_knowledge_system_e2e_test (7 errors): needs an authenticated live backend and gets HTTP 401;
      • file_upload_comprehensive_test::test_knowledge_base_integration: checks that the CWD-relative data/file_manager_root exists;
      • mcp_cache_test::test_cache_age_tracking and two test_fact_store_authority tests: pass when run alone.

      I could not compare against main, because the worktree ceiling (15 of 15) blocked a temporary main checkout.

    • The pre-push pytest step timed out on 127 files, so a275bea6e2 was pushed with AUTOBOT_PREPUSH_ALLOW_TIMEOUT=1 after the runs above.

  • Pre-push on 2326fde67d: [pre-push OK] pytest: all relevant tests pass. This is the first full pre-push run on this branch. The run before it caught the redaction census reading the plugins' key-name tuples as a private vocabulary, fixed in 2326fde67d (census 15/15). Ratchets (file size, ceiling parity, guard_reach_meta, symbol forks, reach_declarations): 626 passed. reach_scope_claim_17844 run alone: 10 passed.
  • Earlier set: 262 passed (16 skipped need Postgres). Plugin SDK, image key and install tests: 205 passed. Covered cases include:
    • a community plugin with the built-in name gets no resolver;
    • an empty core dir doesn't let an impostor in;
    • a symlink from core doesn't qualify;
    • a str subclass can't spoof membership;
    • no key value reaches the log;
    • the vault wins over env, and LLM and image keys share one order.
  • Backend touched areas: 1751 passed; SLM 661; shared, plugins and scripts 554; ratchets and reach 646; reach_scope_claim_17844 run alone after merging main: 10 passed.
  • The pre-push tests hit their 600s budget on 107 files, so the push used AUTOBOT_PREPUSH_ALLOW_TIMEOUT=1 after the runs above. Three Postgres-backed precedence tests in tests/migrations/test_provider_key_vault.py have not run locally; CI is their first run.

Not done, stated rather than implied

So this PR is Refs #18094.

Refs #18094
Refs #17856
Refs #15722

Model Used

Claude Opus 5.5 (master session: decisions, brief, PR). Finished by a Claude Opus subagent; the original commits came from a subagent on 2026-10-10.

🤖 Generated with Claude Code

…llers read ssot_config (#18094)

Dead-code review (dead-code-review skill), caller evidence, this commit's base:
- AST scan of every tracked .py for `from config|config.compat|src.config import
  {API_BASE_URL,BACKEND_HOST_IP,BACKEND_PORT,HTTP_PROTOCOL,OLLAMA_HOST_IP,
  OLLAMA_PORT,OLLAMA_URL,REDIS_HOST_IP,Config,legacy_config}` and `config.<name>`
  attribute reads: live callers were diagnose_backend.py, monitoring_system.py,
  automated_testing_procedure.py, profile_api_endpoints.py, test_configuration.py
  and validate_configuration.py (not in the issue's list; it also imported
  REDIS_PORT/REDIS_URL/WS_* that config never exported, so both of its checks
  already returned False). All now read ssot_config.backend_url / ollama_url /
  port.* / redis_url / websocket_url. After: the same scan prints nothing.
- `git grep config.compat`: only the two backward-compat tests (removed) and a
  docstring in service_config.py (fixed). get_config_value_from, *_section_from,
  get_llm_config/get_redis_config/reload_config/validate_config pass-throughs
  and the *_async wrappers had no importer; config/__init__.py keeps its own
  module-level versions, which are separate definitions.
- Legacy env names: AUTOBOT_VM_* / AUTOBOT_PORT_* appear nowhere in ansible,
  infrastructure, compose, systemd or slm (git grep; the only AUTOBOT_VM_ hit is
  AUTOBOT_VM_ARCHITECTURE, an unrelated flag), so no AliasChoices were added.
  ConfigRegistry also reads Redis keys vm.*/port.*; those were not inspected.

VNC is NOT migrated: SSOT vnc_url (vm.main) is the backend desktop VNC server
(roles/vnc runs on the backend host), while PLAYWRIGHT_VNC_URL (vm.browser)
is the browser VM's noVNC (browser-vnc-websockify.service, playwright-vnc
compose). They are two servers, so PLAYWRIGHT_* and get_vnc_direct_url stay in
a slimmed compat.py until SSOT gains a browser-VM property.

Also: docs/developer/04-configuration.md used the nonexistent `src.config` path.
…it NPU_WORKER_HOST into bind and connect (#18094)

B3: autobot_shared/env_registry_npu.py registers NPU_WORKER_PORT/HOST/BIND_HOST,
SLM_NODE_ID/SECRET_KEY/ENCRYPTION_KEY/USERS_DATABASE_URL and the three
image-provider keys under their live names (no renames). Wired by extending the
existing sibling import line (env_registry.py net +0 lines); ENV_VARS.md
regenerated by pipeline-scripts/generate_env_docs.py. The generated-table test
no longer assumes the last row is AUTOBOT_-prefixed.

B4: the NPU worker reads NPU_WORKER_BIND_HOST first and falls back to the legacy
NPU_WORKER_HOST, so existing deployments keep working. `autobot doctor` connects
via ssot (AUTOBOT_NPU_WORKER_HOST), still honours a legacy NPU_WORKER_HOST, and
maps a wildcard bind value to 127.0.0.1. ansible aiml.yml sets the bind address
under the new name; the other ansible/config sites (hosts.yml, local.yml,
hyperv-deployment.yaml, load_config.sh) carry the connect meaning and are
unchanged.
…the shared enum back to its emitted names (#18094)

Language gains the members the NPU extension map had (java, c, cpp, csharp, ruby, rust, php, swift, kotlin, scala, powershell, xml, html, css, sql, markdown); .sh stays SHELL and Language('bash'|'sh'|'zsh') resolves to SHELL. npu_code_search_patterns.detect_language delegates to base_analyzer and keeps emitting exactly the old strings (bash for .sh/.bash, zsh for .zsh), pinned by a 29-row table test. The new members are classification-only: MultiLanguageAnalyzer does not start reporting 'No analyzer registered' for .java/.md/... files. agents/conftest.py real-loads the leaf base_analyzer, which the top-level conftest stubs.
… comes from node_proxy (#18094)

slm-agent-standalone.py no longer falls back to the node id 'unknown' (it would register a bogus node); like the role agent it logs and exits 1. slm/agent/agent.py reads AUTOBOT_INTERNAL_API_KEY through node_proxy.INTERNAL_KEY_ENV (the agent already imports autobot_shared on the node); the ansible mirror is changed in the same commit and pipeline-scripts/detect-agent-code-drift.sh reports the pairs byte-identical.
…ompat removed (#18094)

ssot_config.browser_vnc_url is f'http://{vm.browser}:{port.vnc}/vnc.html', exactly what compat's PLAYWRIGHT_VNC_URL and get_vnc_direct_url built (same host key vm.browser and port key port.vnc; registry_defaults maps both to the ssot fields). vnc_url (backend desktop VNC) is unchanged. research_browser.py and test_configuration.py read the property; config/compat.py and its config/__init__ plumbing are deleted. AST scan of every tracked .py for imports of the compat/PLAYWRIGHT_*/get_vnc_direct_url names: zero hits. ssot_config.py stays at its ceiling: the vnc_server comment is condensed to make room.
- research_browser.py: the local is now `vnc_url`, not a name that mimicked the
  removed constant. api_endpoint_migrations_test.py:10436 is left alone (another
  branch owns it): that skipped class asserts on code moved to
  _get_docker_browser_info; tracked under #5359.
- npu_code_search_patterns.detect_language: .pyi .pyw .mts .cts .mjs .cjs .vue are
  now classified via the canonical map (previously 'unknown' extensions now
  classified via the canonical map; affects newly indexed files only, no stored
  data rewritten). All 7 are pinned in the table test.
- No new suppressions: the three `nosec B104` and the eval `noqa: S307` are gone.
  The bind-all default and doctor's wildcard mapping use the existing
  NetworkConstants.BIND_ALL_INTERFACES (which carries the one documented
  exemption); the bind test resolves the worker's `host = ...` chain from the AST.
- env_registry_npu SLM_NODE_ID: says slm/agent/agent.py accepts --node-id and
  slm-agent-standalone.py reads only SLM_NODE_ID. ENV_VARS.md regenerated.
- analytics_infrastructure: embedding_model defaults to None and resolves
  ssot_config.embedding_model at construction, so AUTOBOT_EMBEDDING_MODEL is
  honoured. The tautological constant test is dropped; override tests added.
- diagnose_backend, monitoring_system, profile_api_endpoints (plus
  automated_testing_procedure, test_configuration, validate_configuration, which
  also import autobot_shared) get the repo-root bootstrap; monitoring_system also
  gets autobot-backend for `constants`. test_infra_scripts_import_14129 imports
  each without PYTHONPATH.
- Docs that showed the removed names now show ssot_config: CONFIGURATION_GUIDE.md,
  VNC_PORT_CONFIGURATION.md, system-state.md, CORE_RULES.md.
- agents/conftest.py: docstring corrected (the package is no longer stubbed,
  #18070) and the base_analyzer swap is a per-test fixture that restores the
  previous sys.modules entry and parent attribute (the sys.modules leak guard
  rejects a module-scoped swap).
…, scoped resolver injected into the plugin (#18094)

SECURITY-SENSITIVE: credential read path; review on Opus.

Design: the image-generation keys (FLUX_API_KEY, STABILITY_API_KEY, and
OPENAI_API_KEY referenced from LLM_PROVIDER_KEY_NAMES, not repeated) join the
existing resolver services/provider_key_vault.py as IMAGE_PROVIDER_KEY_NAMES,
wired into VAULT_RESOLVED_CREDENTIAL_NAMES and the name->owner map (OPENAI
keeps its LLM owner). No new resolver, no secrets_service, no storage change,
no migration, no rotation: read path only.

Precedence is unchanged: env first, then the vault-hydrated cache
(resolve_provider_key). api/image_generation.py now reads through
resolve_image_provider_key; availability stays a bool and the reason string is
unchanged.

Scoping: resolve_image_provider_key(name) accepts only
IMAGE_PROVIDER_KEY_NAMES and raises ValueError (message carries no name or
value) for anything else. PluginLoader/PluginManager take an optional
credential_resolvers {plugin name: resolver} and set plugin.credential_resolver
on that plugin only, before initialize(). The backend's
get_plugin_loader() passes plugin_credential_resolvers(), which maps only
image-generation-plugin. The plugin installs the resolver into its tool module
and falls back to os.environ.get(NAME, "") when standalone, so the
standalone semantics are unchanged.

Checklist: no key value in any log, exception or repr (tests assert via
caplog on success, missing and error paths); resolver refuses non-image names;
precedence env-then-vault proven for image names; no new endpoint; egress
unchanged; the plugin reaches no other secret through the injection.

Lifespan: autobot-backend/initialization/lifespan.py builds the PluginManager
that loads plugins at startup (the only other construction site) and is owned
by issue-18141-rag-blend-forks, so it is NOT edited here. The one-line wiring,
PluginManager(plugin_dirs, plugin_credential_resolvers()), lands after #18141.
Until then startup loads the plugin standalone (env-only), as today.

loader.py shrinks 696 -> 692; both ceiling records lowered.
…t-in plugin origin, not its self-declared name (#18094)

SECURITY-SENSITIVE: credential read path; review on Opus.

manifest.name is self-declared, so the loader no longer injects a resolver on
a name match alone. PluginLoader/PluginManager take trusted_roots; a resolver
is injected only when the plugin's resolved real path (symlinks followed) lies
under one, else None plus a WARNING naming the plugin and reason (never a
value). No trusted roots means no injection. get_plugin_loader() trusts only
the core-plugins dirs. Logic lives in plugin_sdk/credential_binding.py.

plugin_install rejects (409) a name declared by any core-plugins manifest,
read from disk rather than a hardcoded list.

resolve_image_provider_key requires type(name) is str, so a str subclass with a
forged __hash__/__eq__ cannot pass the membership check. capture_provider_key
refreshes the in-process cache on a successful write when env does not hold the
name, so an admin rotation takes effect without restart; env stays first.

Plugins run in-process: the resolver is origin-bound scoping, not a sandbox
(stated in the resolver docstring). Deferred: THREAT_MODEL.md paragraph (owned
by issue-12771-searchresult-fork); lifespan.py wiring (owned by #18141).

loader.py 692 -> 690, backend plugin_manager.py 702 -> 700; ceilings lowered.
…_shared.content_digest, byte-identical (#15722)

_generate_content_hash now calls content_digest(content, width=16); the input
is encoded UTF-8 strict as before, so every stored key is unchanged (pinned by
analytics_content_hash_15722_test). The _QUEUED entry this branch owned in
one_content_digest_15722_test is removed. The mid-module import re moves to the
import block, so the file stays at its 805-line ceiling.
…mport the backend; leave the #18288 allowlist (#18094)

Both now read ssot_config instead of the backend config package, so the class
guard reports them as stale allowlist entries.
…key() (#18094)

#17856 row 13a, deferred to this branch by #18271. Five sites read the shared
internal-service key on their own: the backend auth gate through
ssot_config.misc, and the SLM auth gate, the SLM vault client, the agent (and
its ansible mirror, same commit) and the vault migration precheck straight from
the environment. All now call autobot_shared.node_proxy.internal_api_key().

Behaviour: none in a deployed process. The backend's systemd unit loads the
same .env through EnvironmentFile and main.py load_dotenv()s it, so os.environ
already carries the value ssot_config read. The backend gate now reads it per
call rather than from the cached settings object.

Guard: repo_tests/one_internal_api_key_reader_17856_test.py (AST, contrast
fixtures, reach scope registered); the credential-vault allowlist loses the
auth_middleware entry it no longer needs.
#18094)

#17856 row 16, deferred to this branch by #18271 (owner decision 2026-10-10:
autobot_shared/feature_flags is canonical, the copies retire).

Retired, each with zero callers outside tests (static imports, attribute
calls, patch targets, string keys in py/yml/j2/json/ts/vue checked; the only
other hit is a bash function in load_config.sh, a different language):
- config.is_feature_enabled (module wrapper) and _ConfigStub.is_feature_enabled
- config.validation.ValidationMixin.is_feature_enabled (a get_nested
  "<x>.enabled" read; config_manager_test now asserts those leaves via get_nested)
- user_management.config.is_feature_enabled

symbol_fork_baseline.json drops is_feature_enabled and the nine clusters this
branch's config.compat removal already dissolved (Config, get_llm_config,
get_redis_config, validate_config and five *_async wrappers); the ratchet is
bidirectional, so it now pins them gone.
…lvers too; trust is anchored, not CWD-relative (#18094)

Security review of this branch's credential commits found the resolver wired
into only one of the two plugin hosts. initialization/lifespan.py builds the
PluginManager that actually loads and runs plugins at startup, and it passed
no resolvers, so the image-generation plugin the backend runs never received
its vault-backed resolver (it fell back to the environment). plugin_manager.py's
PluginLoader, the only host that got them, restated the same search list.

- plugin_paths.py is the one definition of the search dirs and the trusted
  roots; both hosts call it and pass plugin_credential_resolvers().
- Trust is anchored: the deployed core root plus this checkout's own
  plugins/core-plugins (resolved from the source file). The CWD-relative dev
  fallback is still searched, in the same order, but is never trusted, so
  starting the backend from another directory cannot make that directory's
  plugins/core-plugins count as built-in. Fail closed either way.
- Ceilings lowered in both size baselines: plugin_manager.py 700 -> 695,
  lifespan.py 2340 -> 2331.
- THREAT_MODEL records the shared host lists.
…he bottom of their windows (#18094)

audio-extension-allowlist 5937 -> 6010 and redaction-concept-census 3247 ->
3369, both upward (the strict direction). reach_declarations_test failed on
both after the rebase: ordinary growth on main plus this branch's new modules
put each floor within the #17818 zero-tolerance band. The audio floor is not
the suggested 6107 because the sweep completes 6075 files; 6010 clears the
window's safe bottom (5947) with margin below that count.
…ider key, in one place (#18094)

Owner decision (2026-10-12): the secrets manager is the canonical credential
store; the env var is only a logged migration fallback. Applied once, in
services/provider_key_vault.resolve_provider_key, which every caller goes
through, so image (resolve_image_provider_key) and LLM/search/integration keys
share one resolution order with one definition:

- resolve_provider_key: the hydrated vault value wins; config_value (the env
  var as ssot_config read it) is used only when the vault holds nothing, and
  that fallback logs a WARNING once per name per process -- the registry label
  only, never a value.
- hydrate_provider_keys_from_vault no longer skips a name whose env var is
  set (that skip is what made env win), and a capture refreshes the cache
  whether or not the env var is set.

Behaviour change: a deployment with both a vault value and an env var for the
same name now uses the vault value. Processes that never hydrate (no backend
lifespan) still resolve from the env var, as before, now with the warning.

Callers (all through resolve_provider_key; none overrides the order itself):
agent_loop/search/registry.py, agent_loop/slack_hook.py,
initialization/lifespan.py, integrations/capability_registry.py,
knowledge/base.py, llm_shared/adapters/{anthropic,groq,openai}_adapter.py,
llm_shared/providers/{anthropic,bootstrap,custom_openai,groq,huggingface,
mistral,nous_portal,openai,openrouter}.py, security/threat_intelligence.py,
services/execution/claude_code_backend.py, services/notification_service.py,
services/provider_health/providers.py,
voice_processing/realtime/openai_provider.py, api/image_generation.py (via
resolve_image_provider_key). Only their comments asserted env-first; those
now state the new order. Name chains (huggingface/nous_portal: HF_TOKEN, then
HUGGINGFACE_API_TOKEN, then NOUS_API_KEY) keep their name order; precedence
changes per name.

Tests that pinned env-first now pin vault-first: search registry, slack hook,
capability registry, realtime OpenAI provider, and the Postgres-backed
tests/migrations/test_provider_key_vault.py (three tests; skipped locally, no
Postgres). New: vault beats a conflicting env var, env used only when the
vault lacks the key and logged once by name, LLM and image paths agree,
hydration loads a vaulted key whose env var is set. .secrets.baseline drops
the one registry_test entry whose line (the env-value assertion) is gone.
@coderabbitai

coderabbitai Bot commented Oct 12, 2026 •

Copy link
Copy Markdown
Contributor

Important

Review skipped

Too many files!

This PR contains 125 files, which is 25 over the limit of 100.

To get a review, reduce the PR to 100 files or fewer by splitting it into smaller PRs or changing its base branch.

Upgrade to a paid plan to raise the limit.

This review couldn't start because sufficient usage credits or metered capacity aren't available. Add credits or update usage-based reviews in the billing tab, then retry.

Check out review usage here.

⚙️ Run configuration
  • Configuration used: Repository: mrveiss/AutoBot-AI/.coderabbit.yaml
  • Review profile: ASSERTIVE
  • Plan: Advanced
  • Run ID: b740ff94-d73f-426c-a2b6-aa5ca97b49bd

📥 Commits

Reviewing files that changed from the base of the PR and between df211d3 and a275bea.


⛔ Files ignored due to path filters (2)
  • repo_tests/python_file_size_ratchet_baseline.py is excluded by !repo_tests/python_file_size_ratchet_baseline.py
  • scripts/python_file_size_known_large.py is excluded by !scripts/python_file_size_known_large.py

📒 Files selected for processing (125)
  • .secrets.baseline
  • autobot-backend/advanced_rag_optimizer.py
  • autobot-backend/agent_loop/search/registry.py
  • autobot-backend/agent_loop/search/registry_test.py
  • autobot-backend/agent_loop/slack_hook.py
  • autobot-backend/agent_loop/test_slack_hook.py
  • autobot-backend/agents/conftest.py
  • autobot-backend/agents/npu_code_search_patterns.py
  • autobot-backend/agents/npu_language_detection_18094_test.py
  • autobot-backend/api/_video_providers_loader.py
  • autobot-backend/api/image_generation.py
  • autobot-backend/api/research_browser.py
  • autobot-backend/api/simple_terminal_e2e_test.py
  • autobot-backend/api/video_generation.py
  • autobot-backend/api/video_generation_test.py
  • autobot-backend/auth_middleware.py
  • autobot-backend/celery_app.py
  • autobot-backend/cli/doctor.py
  • autobot-backend/code_intelligence/analytics_content_hash_15722_test.py
  • autobot-backend/code_intelligence/analytics_infrastructure.py
  • autobot-backend/code_intelligence/base_analyzer.py
  • autobot-backend/code_intelligence/cross_language_patterns/detector.py
  • autobot-backend/code_intelligence/similarity_thresholds_18094_test.py
  • autobot-backend/config/__init__.py
  • autobot-backend/config/browser_vnc_url_18094_test.py
  • autobot-backend/config/compat.py
  • autobot-backend/config/compat_removal_18094_test.py
  • autobot-backend/config/config_migration_integration_test.py
  • autobot-backend/config/service_config.py
  • autobot-backend/config/validation.py
  • autobot-backend/conftest.py
  • autobot-backend/constants/user_scope_constants.py
  • autobot-backend/initialization/lifespan.py
  • autobot-backend/integrations/capability_registry.py
  • autobot-backend/integrations/capability_registry_test.py
  • autobot-backend/knowledge/rag_benchmarks.py
  • autobot-backend/knowledge/search_components/retrieval_learner.py
  • autobot-backend/llm_shared/providers/bootstrap.py
  • autobot-backend/llm_shared/providers/openai.py
  • autobot-backend/llm_shared/tests/test_provider_key_refusal_18094.py
  • autobot-backend/plugin_install.py
  • autobot-backend/plugin_install_builtin_name_18094_test.py
  • autobot-backend/plugin_manager.py
  • autobot-backend/plugin_paths.py
  • autobot-backend/plugin_paths_18094_test.py
  • autobot-backend/services/feedback_aggregator.py
  • autobot-backend/services/image_provider_key_18094_test.py
  • autobot-backend/services/provider_health/providers.py
  • autobot-backend/services/provider_key_outage_18094_test.py
  • autobot-backend/services/provider_key_process_init.py
  • autobot-backend/services/provider_key_refusal_18094_test.py
  • autobot-backend/services/provider_key_vault.py
  • autobot-backend/testkit/provider_key_vault_state.py
  • autobot-backend/tests/cli/test_doctor.py
  • autobot-backend/tests/migrations/test_provider_key_vault.py
  • autobot-backend/tests/services/feedback_aggregator_test.py
  • autobot-backend/tests/test_defork_constants_18094_test.py
  • autobot-backend/user_management/config.py
  • autobot-backend/user_management/database.py
  • autobot-backend/utils/config_manager_test.py
  • autobot-backend/voice_processing/providers/lv/tilde_provider.py
  • autobot-backend/voice_processing/realtime/openai_provider.py
  • autobot-backend/voice_processing/realtime/openai_provider_test.py
  • autobot-infrastructure/autobot-npu-worker/docker/npu_inference_server.py
  • autobot-infrastructure/autobot-npu-worker/docker/npu_worker_main.py
  • autobot-infrastructure/shared/scripts/automated_testing_procedure.py
  • autobot-infrastructure/shared/scripts/diagnose_backend.py
  • autobot-infrastructure/shared/scripts/monitoring_system.py
  • autobot-infrastructure/shared/scripts/profile_api_endpoints.py
  • autobot-infrastructure/shared/scripts/slm-agent-standalone.py
  • autobot-infrastructure/shared/scripts/test_configuration.py
  • autobot-infrastructure/shared/scripts/validate_configuration.py
  • autobot-slm-backend/ansible/inventory/group_vars/aiml.yml
  • autobot-slm-backend/ansible/roles/slm_agent/files/slm/agent/agent.py
  • autobot-slm-backend/migrations/migrate_system_secrets_to_vault.py
  • autobot-slm-backend/monitoring/business_intelligence_dashboard.py
  • autobot-slm-backend/monitoring/performance_benchmark.py
  • autobot-slm-backend/monitoring/performance_monitor.py
  • autobot-slm-backend/services/auth.py
  • autobot-slm-backend/services/manifest_loader.py
  • autobot-slm-backend/slm/agent/agent.py
  • autobot-slm-backend/tests/services/base_dir_canonical_18094_test.py
  • autobot-slm-backend/user_management/services/vault_client.py
  • autobot_shared/env_registry.py
  • autobot_shared/env_registry_npu.py
  • autobot_shared/env_registry_security.py
  • autobot_shared/plugin_sdk/base.py
  • autobot_shared/plugin_sdk/credential_binding.py
  • autobot_shared/plugin_sdk/credential_origin_binding_18094_test.py
  • autobot_shared/plugin_sdk/credential_resolver_injection_18094_test.py
  • autobot_shared/plugin_sdk/loader.py
  • autobot_shared/plugin_sdk/plugin_manager.py
  • autobot_shared/redis_management/config.py
  • autobot_shared/ssot_config.py
  • autobot_shared/ssot_constants.py
  • docs/configuration/VNC_PORT_CONFIGURATION.md
  • docs/developer/04-configuration.md
  • docs/developer/ENV_VARS.md
  • docs/developer/THREAT_MODEL.md
  • docs/development/CORE_RULES.md
  • docs/guides/CONFIGURATION_GUIDE.md
  • docs/system-state.md
  • pipeline-scripts/check_env_var_registry_test.py
  • plugins/core-plugins/image-generation-plugin/main.py
  • plugins/core-plugins/image-generation-plugin/tools/__init__.py
  • plugins/core-plugins/image-generation-plugin/tools/credential_resolver_18094_test.py
  • plugins/core-plugins/image-generation-plugin/tools/generate_image.py
  • plugins/core-plugins/video-generation-plugin/main.py
  • plugins/core-plugins/video-generation-plugin/tools/credential_slot_18094_test.py
  • plugins/core-plugins/video-generation-plugin/tools/providers.py
  • repo_tests/_reach_policy.py
  • repo_tests/chunker_private_embedding_hooks_18185_test.py
  • repo_tests/credential_vault_resolution_allowlist.py
  • repo_tests/infra_backend_imports_allowlist_18288.py
  • repo_tests/npu_bind_connect_split_18094_test.py
  • repo_tests/one_content_digest_15722_test.py
  • repo_tests/one_deployment_mode_detector_12771_test.py
  • repo_tests/one_internal_api_key_reader_17856_test.py
  • repo_tests/provider_key_env_reads_route_through_vault_18094_test.py
  • repo_tests/reach_scope_claim_17844_test.py
  • repo_tests/slm_agent_identity_18094_test.py
  • repo_tests/symbol_fork_baseline.json
  • repo_tests/test_infra_scripts_import_14129.py
  • scripts/daily_health_check.py
  • scripts/daily_health_check_defaults_18094_test.py

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions

Copy link
Copy Markdown
Contributor

✅ SSOT Configuration Compliance: Passing

🎉 No new hardcoded values of either class — ssot and other both block.

Known backlog in pipeline-scripts/hardcoded_values_baseline.txt is suppressed and tracked in #14371.

@github-actions

Copy link
Copy Markdown
Contributor

Notice: 25 open PRs — past the runaway threshold (25)

This is not a request to defer this PR. The actual limit is the open-PR cap (AUTOBOT_OPEN_PR_CAP, default 40), enforced at pre-push against NEW branches only -- pushing more commits to this PR's own branch is never blocked by it.

This notice only means the count is high enough to be worth a glance for a runaway — something opening PRs in a loop, or a merge pipeline that has stalled so nothing is draining.

Currently open:

If the queue is draining normally, ignore this. Otherwise:

  1. Check whether CI is dispatching at all — see the ci-dispatch-watchdog status on these PRs
  2. Merge the ones whose CI has finished and review has passed: gh pr merge <number> --squash --delete-branch
  3. Look for a loop opening near-identical PRs

Warn-only runaway detector — .github/workflows/pr-queue-gate.yml. It never blocks a merge.

…lved from env (#18094)

Owner decision (2026-10-12): when the System vault holds a provider key but
the read fails, resolution refuses with a clear "vault unavailable" error on
that provider call instead of falling back to the env value, which may hold a
rotated-out key. The env fallback stays only for keys the vault does not hold.

provider_key_vault:
- _hydrate_one_key records a failed read in _unreadable_keys (and drops any
  stale cached value); a successful read, a capture, or the row's removal
  clears it.
- resolve_provider_key raises VaultKeyUnavailableError for those names. The
  two log texts are distinct: "vault unreadable: refusing the environment
  fallback" (ERROR) vs "vault has no key: using the environment (migration
  fallback)" (WARNING, once per name). Only registry labels are logged,
  never a value.
- registration_refusal(component): contains a refusal at a registration site
  so one refused key skips one provider, logged at ERROR, never its siblings.

Callers checked, and how each surfaces the refusal:
- Registration (skip that one, reason at ERROR, startup continues):
  llm_shared/providers/bootstrap.py (all keyed providers, custom OpenAI,
  OpenRouter, HuggingFace, Nous), initialization/lifespan.py adapters,
  agent_loop/search/registry.py (SearXNG whole, Brave),
  integrations/capability_registry.py (Slack, Discord).
- Reported state, not raised: services/provider_health/providers.py
  (UNAVAILABLE with the reason), api/image_generation.py /providers (that
  provider unavailable with the reason), voice realtime is_configured (False,
  logged), agent_loop/slack_hook.py (null hook, logged, not cached), the
  image-generation plugin load (logged; the tool returns the reason per call).
- Call time, fails with the reason: llm_shared providers' _resolve_api_key
  (LLMResponse.error), llm_shared adapters (raised to the router),
  knowledge/base.py (KB init logs and returns False), claude_code_backend
  (execution_manager logs health errors; execution raises), notification
  email (dispatch logs it), threat_intelligence (service construction raises
  into the security API's error handler), voice realtime negotiate().

Tests: services/provider_key_refusal_18094_test.py (12) and
llm_shared/tests/test_provider_key_refusal_18094.py (bootstrap skips only the
refused provider).
…mainder

# Conflicts:
#	repo_tests/reach_scope_claim_17844_test.py
@mrveiss

mrveiss commented Oct 12, 2026

Copy link
Copy Markdown
Owner Author

Security review (Opus) at ccc1cc3: blocked. There is one resolver and the vault wins over env, but the owner's ruling, "refuse if the vault holds it", is not implemented: a held key whose read fails falls back to the env var.

Severity Location Finding Fix
High (fail-open) services/provider_key_vault.py:341-343 _hydrate_one_key catches every read or decrypt exception and returns False, without recording that the key is held. resolve_provider_key (:171-185) then treats "held, read failed" as "not held" and returns the env value. Track failed reads in a set (or a tri-state cache). The resolver raises a refusal for a held+failed name and never returns env.
High (fail-open) initialization/lifespan.py:432-440 A vault outage (DB down, root key missing, an exception finding the system secret) aborts hydration at debug, labelled "non-critical", so every key resolves from env. Treat an outage as "held status unknown": refuse the names that have an env value, or fail startup. Log at ERROR.
Medium provider_key_vault.py:171-185 Only two of the three branches exist (vault, env). There is no refusal branch. Add it in this one resolver.
Medium tests No test covers read-failure → refuse, or outage during hydration. The migration tests skip without Postgres. Unit tests: read raises with the env var set → refusal; outage during hydration → refusal or startup failure.
Medium voice_processing/providers/lv/tilde_provider.py:45, autobot_shared/paperclip_client.py:123, services/execution/modal_backend.py:133 TILDE_API_KEY, PAPERCLIP_API_KEY and MODAL_TOKEN_* read env directly, outside VAULT_RESOLVED_CREDENTIAL_NAMES. Route them through resolve_provider_key, or record them as out of scope in #18094 with the reason.
Low provider_key_vault.py:342, lifespan.py:438 Key names are logged without the frozenset-label pattern used at :195/:310. No values are logged. Use the same label pattern.

Checked and fine: the plugin trust anchoring in 32de2e3 (core path anchored to the module file, symlinks resolved, core searched before community), and the floor re-pins in 522f014.

… a key-name tuple the redaction census reads as a vocabulary (#18094)
@mrveiss

mrveiss commented Oct 12, 2026

Copy link
Copy Markdown
Owner Author

Security re-review (Opus) at 2326fde: blocked on one remaining High. H1, M, L and the Celery hook all hold. Verified:

  • _hydrate_all (:500-512) refuses held names without a root key, with a contrast test at :83;
  • the cache is rebuilt from rows (:501/:517);
  • the swap is atomic (one tuple assignment, :517);
  • the Celery worker_process_init hook hydrates whether or not a root key is set;
  • a failed DB read refuses.
Severity Location Finding Fix
High services/provider_key_vault.py:247, pinned by provider_key_outage_18094_test.py:193 Lazy hydration runs only when _root_key_configured(). A process outside the lifespan (CLI, operator script, MCP embedded, spawned subprocess) with no root key never lists rows, so it serves env for held names, which is what the ruling forbids. The stated residual is avoidable: listing needs no root key (_hydrate_all already lists and refuses without one), and the Celery hook already hydrates ungated. The body's residual line contradicts the module docstring (:44). Drop the and _root_key_configured() gate, so every unhydrated process lists rows. An unreachable DB then refuses, which the ruling accepts. Turn :193 into a contrast: no root key and the row held → refused; no row → env.
Low provider_key_process_init.py:41-54 worker.join() has no timeout and holds the lock for the whole join, so a hung DB connect blocks every first caller, including the event-loop thread. join(timeout) from an env-backed constant; on timeout, set VAULT_UNREACHABLE.
Low provider_key_process_init.py:42, provider_key_vault.py:256 UNREACHABLE is permanent (only NOT_HYDRATED re-hydrates), so a transient blip at worker init refuses keys for the worker's whole lifetime. That fails closed, but the docstring says "until a hydration succeeds". Retry with backoff, or document that a restart is needed.
Low provider_key_outage_18094_test.py:223-233 The Celery hook test only checks the decorator via AST. Call the hook with hydrate_this_process stubbed.

Logs are clean: labels and exception type names only.

…ed wait; unreachable vault retried with backoff (#18094)

Third review of #18371 (9a). The HIGH: lazy hydration was gated on a
configured root key, so a non-lifespan process without one still served env
for names the vault holds. The gate is dropped: listing rows needs no root
key, so any process that has not read the vault reads it on its first key
resolution, and a DB it cannot reach is an outage that refuses keys (owner
ruling). The pinned residual test is flipped into a contrast test.

Unit tests have no vault DB, so the harness, not production, supplies the
state: testkit/provider_key_vault_state.py is an autouse fixture (imported by
the backend conftest) that starts every test against an explicit empty,
reachable vault. conftest.py ceiling 1504 -> 1502 (a docstring compacted to
make room for the import).

LOWs:
- The hydration thread is a daemon, waited for at most
  AUTOBOT_PROVIDER_KEY_HYDRATION_TIMEOUT_S (default 30); on timeout the vault
  is marked unreachable (fail closed) and the lock is released. A late finish
  still swaps its result in.
- UNREACHABLE is no longer permanent: a later resolution retries after
  AUTOBOT_PROVIDER_KEY_REHYDRATE_BACKOFF_S (default 30), doubling per failed
  retry up to 20x, reset on success. Keys stay refused until a retry succeeds.
Both variables are registered in env_registry_security.py.
…mainder

# Conflicts:
#	repo_tests/audio_extension_allowlist_test.py
#	repo_tests/one_content_digest_15722_test.py
#	repo_tests/reach_scope_claim_17844_test.py
#	repo_tests/redaction_concept_census_test.py
…e bottom of their windows (#18094)

chunker-private-hook-scan 4920 -> 5059 and deployment-mode-detector-scan
4676 -> 4737, each to the mid-window value reach_declarations_test measured
after merging main (populations 5209 and 4812, skips 0, so the sweep
completes the whole population). Upward, the strict direction.
@mrveiss

mrveiss commented Oct 12, 2026

Copy link
Copy Markdown
Owner Author

Security delta review (Opus) at a275bea: approve. The owner's "refuse if the vault holds it" ruling now holds on every path:

  • Gate removed: provider_key_vault.py:240 hydrates whenever the state is NOT_HYDRATED/UNREACHABLE, and _root_key_configured is deleted.
  • Real contrast test: provider_key_outage_18094_test.py:~198 runs the real hydration with the root key unset. A held name raises "could not be read"; a no-row name gets env.
  • Testkit fixture: test-only (the one importer is autobot-backend/conftest.py:45), and the outage tests still drive the real hydration.
  • Join timeout: releases the lock via with and fails closed, tested.
  • Backoff: refuses while UNREACHABLE (:250), retries lazily on resolution, and is capped at 20×.
  • Logs: type names and registry labels only.

Follow-ups are filed as #18426. None fails open:

  • Medium: the synchronous retry joins on the request path, so a vault outage can stall the event loop for up to 30s per retry window.
  • Lows: a late stale-snapshot swap, needing a generation check; unclamped env reads; and registry wording that describes the race as intended.

Queued for the next merge train.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant