Repository navigation
refactor(config): the #18094 de-fork remainder — one key reader, one feature flag, one language detector, and image-gen keys vault-first through an origin-bound resolver (#18094) - #18371
Conversation
…ead ssot_config (#18094)
…sult defaults and the global-user sentinel (#18094)
…llers read ssot_config (#18094) Dead-code review (dead-code-review skill), caller evidence, this commit's base: - AST scan of every tracked .py for `from config|config.compat|src.config import {API_BASE_URL,BACKEND_HOST_IP,BACKEND_PORT,HTTP_PROTOCOL,OLLAMA_HOST_IP, OLLAMA_PORT,OLLAMA_URL,REDIS_HOST_IP,Config,legacy_config}` and `config.<name>` attribute reads: live callers were diagnose_backend.py, monitoring_system.py, automated_testing_procedure.py, profile_api_endpoints.py, test_configuration.py and validate_configuration.py (not in the issue's list; it also imported REDIS_PORT/REDIS_URL/WS_* that config never exported, so both of its checks already returned False). All now read ssot_config.backend_url / ollama_url / port.* / redis_url / websocket_url. After: the same scan prints nothing. - `git grep config.compat`: only the two backward-compat tests (removed) and a docstring in service_config.py (fixed). get_config_value_from, *_section_from, get_llm_config/get_redis_config/reload_config/validate_config pass-throughs and the *_async wrappers had no importer; config/__init__.py keeps its own module-level versions, which are separate definitions. - Legacy env names: AUTOBOT_VM_* / AUTOBOT_PORT_* appear nowhere in ansible, infrastructure, compose, systemd or slm (git grep; the only AUTOBOT_VM_ hit is AUTOBOT_VM_ARCHITECTURE, an unrelated flag), so no AliasChoices were added. ConfigRegistry also reads Redis keys vm.*/port.*; those were not inspected. VNC is NOT migrated: SSOT vnc_url (vm.main) is the backend desktop VNC server (roles/vnc runs on the backend host), while PLAYWRIGHT_VNC_URL (vm.browser) is the browser VM's noVNC (browser-vnc-websockify.service, playwright-vnc compose). They are two servers, so PLAYWRIGHT_* and get_vnc_direct_url stay in a slimmed compat.py until SSOT gains a browser-VM property. Also: docs/developer/04-configuration.md used the nonexistent `src.config` path.
…it NPU_WORKER_HOST into bind and connect (#18094) B3: autobot_shared/env_registry_npu.py registers NPU_WORKER_PORT/HOST/BIND_HOST, SLM_NODE_ID/SECRET_KEY/ENCRYPTION_KEY/USERS_DATABASE_URL and the three image-provider keys under their live names (no renames). Wired by extending the existing sibling import line (env_registry.py net +0 lines); ENV_VARS.md regenerated by pipeline-scripts/generate_env_docs.py. The generated-table test no longer assumes the last row is AUTOBOT_-prefixed. B4: the NPU worker reads NPU_WORKER_BIND_HOST first and falls back to the legacy NPU_WORKER_HOST, so existing deployments keep working. `autobot doctor` connects via ssot (AUTOBOT_NPU_WORKER_HOST), still honours a legacy NPU_WORKER_HOST, and maps a wildcard bind value to 127.0.0.1. ansible aiml.yml sets the bind address under the new name; the other ansible/config sites (hosts.yml, local.yml, hyperv-deployment.yaml, load_config.sh) carry the connect meaning and are unchanged.
…the shared enum back to its emitted names (#18094) Language gains the members the NPU extension map had (java, c, cpp, csharp, ruby, rust, php, swift, kotlin, scala, powershell, xml, html, css, sql, markdown); .sh stays SHELL and Language('bash'|'sh'|'zsh') resolves to SHELL. npu_code_search_patterns.detect_language delegates to base_analyzer and keeps emitting exactly the old strings (bash for .sh/.bash, zsh for .zsh), pinned by a 29-row table test. The new members are classification-only: MultiLanguageAnalyzer does not start reporting 'No analyzer registered' for .java/.md/... files. agents/conftest.py real-loads the leaf base_analyzer, which the top-level conftest stubs.
… comes from node_proxy (#18094) slm-agent-standalone.py no longer falls back to the node id 'unknown' (it would register a bogus node); like the role agent it logs and exits 1. slm/agent/agent.py reads AUTOBOT_INTERNAL_API_KEY through node_proxy.INTERNAL_KEY_ENV (the agent already imports autobot_shared on the node); the ansible mirror is changed in the same commit and pipeline-scripts/detect-agent-code-drift.sh reports the pairs byte-identical.
…ompat removed (#18094) ssot_config.browser_vnc_url is f'http://{vm.browser}:{port.vnc}/vnc.html', exactly what compat's PLAYWRIGHT_VNC_URL and get_vnc_direct_url built (same host key vm.browser and port key port.vnc; registry_defaults maps both to the ssot fields). vnc_url (backend desktop VNC) is unchanged. research_browser.py and test_configuration.py read the property; config/compat.py and its config/__init__ plumbing are deleted. AST scan of every tracked .py for imports of the compat/PLAYWRIGHT_*/get_vnc_direct_url names: zero hits. ssot_config.py stays at its ceiling: the vnc_server comment is condensed to make room.
- research_browser.py: the local is now `vnc_url`, not a name that mimicked the removed constant. api_endpoint_migrations_test.py:10436 is left alone (another branch owns it): that skipped class asserts on code moved to _get_docker_browser_info; tracked under #5359. - npu_code_search_patterns.detect_language: .pyi .pyw .mts .cts .mjs .cjs .vue are now classified via the canonical map (previously 'unknown' extensions now classified via the canonical map; affects newly indexed files only, no stored data rewritten). All 7 are pinned in the table test. - No new suppressions: the three `nosec B104` and the eval `noqa: S307` are gone. The bind-all default and doctor's wildcard mapping use the existing NetworkConstants.BIND_ALL_INTERFACES (which carries the one documented exemption); the bind test resolves the worker's `host = ...` chain from the AST. - env_registry_npu SLM_NODE_ID: says slm/agent/agent.py accepts --node-id and slm-agent-standalone.py reads only SLM_NODE_ID. ENV_VARS.md regenerated. - analytics_infrastructure: embedding_model defaults to None and resolves ssot_config.embedding_model at construction, so AUTOBOT_EMBEDDING_MODEL is honoured. The tautological constant test is dropped; override tests added. - diagnose_backend, monitoring_system, profile_api_endpoints (plus automated_testing_procedure, test_configuration, validate_configuration, which also import autobot_shared) get the repo-root bootstrap; monitoring_system also gets autobot-backend for `constants`. test_infra_scripts_import_14129 imports each without PYTHONPATH. - Docs that showed the removed names now show ssot_config: CONFIGURATION_GUIDE.md, VNC_PORT_CONFIGURATION.md, system-state.md, CORE_RULES.md. - agents/conftest.py: docstring corrected (the package is no longer stubbed, #18070) and the base_analyzer swap is a per-test fixture that restores the previous sys.modules entry and parent attribute (the sys.modules leak guard rejects a module-scoped swap).
…, scoped resolver injected into the plugin (#18094) SECURITY-SENSITIVE: credential read path; review on Opus. Design: the image-generation keys (FLUX_API_KEY, STABILITY_API_KEY, and OPENAI_API_KEY referenced from LLM_PROVIDER_KEY_NAMES, not repeated) join the existing resolver services/provider_key_vault.py as IMAGE_PROVIDER_KEY_NAMES, wired into VAULT_RESOLVED_CREDENTIAL_NAMES and the name->owner map (OPENAI keeps its LLM owner). No new resolver, no secrets_service, no storage change, no migration, no rotation: read path only. Precedence is unchanged: env first, then the vault-hydrated cache (resolve_provider_key). api/image_generation.py now reads through resolve_image_provider_key; availability stays a bool and the reason string is unchanged. Scoping: resolve_image_provider_key(name) accepts only IMAGE_PROVIDER_KEY_NAMES and raises ValueError (message carries no name or value) for anything else. PluginLoader/PluginManager take an optional credential_resolvers {plugin name: resolver} and set plugin.credential_resolver on that plugin only, before initialize(). The backend's get_plugin_loader() passes plugin_credential_resolvers(), which maps only image-generation-plugin. The plugin installs the resolver into its tool module and falls back to os.environ.get(NAME, "") when standalone, so the standalone semantics are unchanged. Checklist: no key value in any log, exception or repr (tests assert via caplog on success, missing and error paths); resolver refuses non-image names; precedence env-then-vault proven for image names; no new endpoint; egress unchanged; the plugin reaches no other secret through the injection. Lifespan: autobot-backend/initialization/lifespan.py builds the PluginManager that loads plugins at startup (the only other construction site) and is owned by issue-18141-rag-blend-forks, so it is NOT edited here. The one-line wiring, PluginManager(plugin_dirs, plugin_credential_resolvers()), lands after #18141. Until then startup loads the plugin standalone (env-only), as today. loader.py shrinks 696 -> 692; both ceiling records lowered.
…t-in plugin origin, not its self-declared name (#18094) SECURITY-SENSITIVE: credential read path; review on Opus. manifest.name is self-declared, so the loader no longer injects a resolver on a name match alone. PluginLoader/PluginManager take trusted_roots; a resolver is injected only when the plugin's resolved real path (symlinks followed) lies under one, else None plus a WARNING naming the plugin and reason (never a value). No trusted roots means no injection. get_plugin_loader() trusts only the core-plugins dirs. Logic lives in plugin_sdk/credential_binding.py. plugin_install rejects (409) a name declared by any core-plugins manifest, read from disk rather than a hardcoded list. resolve_image_provider_key requires type(name) is str, so a str subclass with a forged __hash__/__eq__ cannot pass the membership check. capture_provider_key refreshes the in-process cache on a successful write when env does not hold the name, so an admin rotation takes effect without restart; env stays first. Plugins run in-process: the resolver is origin-bound scoping, not a sandbox (stated in the resolver docstring). Deferred: THREAT_MODEL.md paragraph (owned by issue-12771-searchresult-fork); lifespan.py wiring (owned by #18141). loader.py 692 -> 690, backend plugin_manager.py 702 -> 700; ceilings lowered.
…_shared.content_digest, byte-identical (#15722) _generate_content_hash now calls content_digest(content, width=16); the input is encoded UTF-8 strict as before, so every stored key is unchanged (pinned by analytics_content_hash_15722_test). The _QUEUED entry this branch owned in one_content_digest_15722_test is removed. The mid-module import re moves to the import block, so the file stays at its 805-line ceiling.
…key() (#18094) #17856 row 13a, deferred to this branch by #18271. Five sites read the shared internal-service key on their own: the backend auth gate through ssot_config.misc, and the SLM auth gate, the SLM vault client, the agent (and its ansible mirror, same commit) and the vault migration precheck straight from the environment. All now call autobot_shared.node_proxy.internal_api_key(). Behaviour: none in a deployed process. The backend's systemd unit loads the same .env through EnvironmentFile and main.py load_dotenv()s it, so os.environ already carries the value ssot_config read. The backend gate now reads it per call rather than from the cached settings object. Guard: repo_tests/one_internal_api_key_reader_17856_test.py (AST, contrast fixtures, reach scope registered); the credential-vault allowlist loses the auth_middleware entry it no longer needs.
#18094) #17856 row 16, deferred to this branch by #18271 (owner decision 2026-10-10: autobot_shared/feature_flags is canonical, the copies retire). Retired, each with zero callers outside tests (static imports, attribute calls, patch targets, string keys in py/yml/j2/json/ts/vue checked; the only other hit is a bash function in load_config.sh, a different language): - config.is_feature_enabled (module wrapper) and _ConfigStub.is_feature_enabled - config.validation.ValidationMixin.is_feature_enabled (a get_nested "<x>.enabled" read; config_manager_test now asserts those leaves via get_nested) - user_management.config.is_feature_enabled symbol_fork_baseline.json drops is_feature_enabled and the nine clusters this branch's config.compat removal already dissolved (Config, get_llm_config, get_redis_config, validate_config and five *_async wrappers); the ratchet is bidirectional, so it now pins them gone.
…lvers too; trust is anchored, not CWD-relative (#18094) Security review of this branch's credential commits found the resolver wired into only one of the two plugin hosts. initialization/lifespan.py builds the PluginManager that actually loads and runs plugins at startup, and it passed no resolvers, so the image-generation plugin the backend runs never received its vault-backed resolver (it fell back to the environment). plugin_manager.py's PluginLoader, the only host that got them, restated the same search list. - plugin_paths.py is the one definition of the search dirs and the trusted roots; both hosts call it and pass plugin_credential_resolvers(). - Trust is anchored: the deployed core root plus this checkout's own plugins/core-plugins (resolved from the source file). The CWD-relative dev fallback is still searched, in the same order, but is never trusted, so starting the backend from another directory cannot make that directory's plugins/core-plugins count as built-in. Fail closed either way. - Ceilings lowered in both size baselines: plugin_manager.py 700 -> 695, lifespan.py 2340 -> 2331. - THREAT_MODEL records the shared host lists.
…he bottom of their windows (#18094) audio-extension-allowlist 5937 -> 6010 and redaction-concept-census 3247 -> 3369, both upward (the strict direction). reach_declarations_test failed on both after the rebase: ordinary growth on main plus this branch's new modules put each floor within the #17818 zero-tolerance band. The audio floor is not the suggested 6107 because the sweep completes 6075 files; 6010 clears the window's safe bottom (5947) with margin below that count.
…ider key, in one place (#18094) Owner decision (2026-10-12): the secrets manager is the canonical credential store; the env var is only a logged migration fallback. Applied once, in services/provider_key_vault.resolve_provider_key, which every caller goes through, so image (resolve_image_provider_key) and LLM/search/integration keys share one resolution order with one definition: - resolve_provider_key: the hydrated vault value wins; config_value (the env var as ssot_config read it) is used only when the vault holds nothing, and that fallback logs a WARNING once per name per process -- the registry label only, never a value. - hydrate_provider_keys_from_vault no longer skips a name whose env var is set (that skip is what made env win), and a capture refreshes the cache whether or not the env var is set. Behaviour change: a deployment with both a vault value and an env var for the same name now uses the vault value. Processes that never hydrate (no backend lifespan) still resolve from the env var, as before, now with the warning. Callers (all through resolve_provider_key; none overrides the order itself): agent_loop/search/registry.py, agent_loop/slack_hook.py, initialization/lifespan.py, integrations/capability_registry.py, knowledge/base.py, llm_shared/adapters/{anthropic,groq,openai}_adapter.py, llm_shared/providers/{anthropic,bootstrap,custom_openai,groq,huggingface, mistral,nous_portal,openai,openrouter}.py, security/threat_intelligence.py, services/execution/claude_code_backend.py, services/notification_service.py, services/provider_health/providers.py, voice_processing/realtime/openai_provider.py, api/image_generation.py (via resolve_image_provider_key). Only their comments asserted env-first; those now state the new order. Name chains (huggingface/nous_portal: HF_TOKEN, then HUGGINGFACE_API_TOKEN, then NOUS_API_KEY) keep their name order; precedence changes per name. Tests that pinned env-first now pin vault-first: search registry, slack hook, capability registry, realtime OpenAI provider, and the Postgres-backed tests/migrations/test_provider_key_vault.py (three tests; skipped locally, no Postgres). New: vault beats a conflicting env var, env used only when the vault lacks the key and logged once by name, LLM and image paths agree, hydration loads a vaulted key whose env var is set. .secrets.baseline drops the one registry_test entry whose line (the env-value assertion) is gone.
|
Important Review skippedToo many files! This PR contains 125 files, which is 25 over the limit of 100. To get a review, reduce the PR to 100 files or fewer by splitting it into smaller PRs or changing its base branch. Upgrade to a paid plan to raise the limit. This review couldn't start because sufficient usage credits or metered capacity aren't available. Add credits or update usage-based reviews in the billing tab, then retry. ⚙️ Run configuration
⛔ Files ignored due to path filters (2)
📒 Files selected for processing (125)
You can disable this status message by setting the
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
✅ SSOT Configuration Compliance: Passing🎉 No new hardcoded values of either class — Known backlog in |
…lved from env (#18094) Owner decision (2026-10-12): when the System vault holds a provider key but the read fails, resolution refuses with a clear "vault unavailable" error on that provider call instead of falling back to the env value, which may hold a rotated-out key. The env fallback stays only for keys the vault does not hold. provider_key_vault: - _hydrate_one_key records a failed read in _unreadable_keys (and drops any stale cached value); a successful read, a capture, or the row's removal clears it. - resolve_provider_key raises VaultKeyUnavailableError for those names. The two log texts are distinct: "vault unreadable: refusing the environment fallback" (ERROR) vs "vault has no key: using the environment (migration fallback)" (WARNING, once per name). Only registry labels are logged, never a value. - registration_refusal(component): contains a refusal at a registration site so one refused key skips one provider, logged at ERROR, never its siblings. Callers checked, and how each surfaces the refusal: - Registration (skip that one, reason at ERROR, startup continues): llm_shared/providers/bootstrap.py (all keyed providers, custom OpenAI, OpenRouter, HuggingFace, Nous), initialization/lifespan.py adapters, agent_loop/search/registry.py (SearXNG whole, Brave), integrations/capability_registry.py (Slack, Discord). - Reported state, not raised: services/provider_health/providers.py (UNAVAILABLE with the reason), api/image_generation.py /providers (that provider unavailable with the reason), voice realtime is_configured (False, logged), agent_loop/slack_hook.py (null hook, logged, not cached), the image-generation plugin load (logged; the tool returns the reason per call). - Call time, fails with the reason: llm_shared providers' _resolve_api_key (LLMResponse.error), llm_shared adapters (raised to the router), knowledge/base.py (KB init logs and returns False), claude_code_backend (execution_manager logs health errors; execution raises), notification email (dispatch logs it), threat_intelligence (service construction raises into the security API's error handler), voice realtime negotiate(). Tests: services/provider_key_refusal_18094_test.py (12) and llm_shared/tests/test_provider_key_refusal_18094.py (bootstrap skips only the refused provider).
…mainder # Conflicts: # repo_tests/reach_scope_claim_17844_test.py
|
Security review (Opus) at ccc1cc3: blocked. There is one resolver and the vault wins over env, but the owner's ruling, "refuse if the vault holds it", is not implemented: a held key whose read fails falls back to the env var.
Checked and fine: the plugin trust anchoring in 32de2e3 (core path anchored to the module file, symlinks resolved, core searched before community), and the floor re-pins in 522f014. |
… a key-name tuple the redaction census reads as a vocabulary (#18094)
|
Security re-review (Opus) at 2326fde: blocked on one remaining High. H1, M, L and the Celery hook all hold. Verified:
Logs are clean: labels and exception type names only. |
…ed wait; unreachable vault retried with backoff (#18094) Third review of #18371 (9a). The HIGH: lazy hydration was gated on a configured root key, so a non-lifespan process without one still served env for names the vault holds. The gate is dropped: listing rows needs no root key, so any process that has not read the vault reads it on its first key resolution, and a DB it cannot reach is an outage that refuses keys (owner ruling). The pinned residual test is flipped into a contrast test. Unit tests have no vault DB, so the harness, not production, supplies the state: testkit/provider_key_vault_state.py is an autouse fixture (imported by the backend conftest) that starts every test against an explicit empty, reachable vault. conftest.py ceiling 1504 -> 1502 (a docstring compacted to make room for the import). LOWs: - The hydration thread is a daemon, waited for at most AUTOBOT_PROVIDER_KEY_HYDRATION_TIMEOUT_S (default 30); on timeout the vault is marked unreachable (fail closed) and the lock is released. A late finish still swaps its result in. - UNREACHABLE is no longer permanent: a later resolution retries after AUTOBOT_PROVIDER_KEY_REHYDRATE_BACKOFF_S (default 30), doubling per failed retry up to 20x, reset on success. Keys stay refused until a retry succeeds. Both variables are registered in env_registry_security.py.
…mainder # Conflicts: # repo_tests/audio_extension_allowlist_test.py # repo_tests/one_content_digest_15722_test.py # repo_tests/reach_scope_claim_17844_test.py # repo_tests/redaction_concept_census_test.py
…e bottom of their windows (#18094) chunker-private-hook-scan 4920 -> 5059 and deployment-mode-detector-scan 4676 -> 4737, each to the mid-window value reach_declarations_test measured after merging main (populations 5209 and 4812, skips 0, so the sweep completes the whole population). Upward, the strict direction.
|
Security delta review (Opus) at a275bea: approve. The owner's "refuse if the vault holds it" ruling now holds on every path:
Follow-ups are filed as #18426. None fails open:
Queued for the next merge train. |
Thinking Path
#18094 is the remainder of the de-fork work after #17419/#16177: forked env defaults, shadowed constants and same-name functions. It was finished from a local-only branch from 2026-10-10 and rebased onto current
main, roughly 190 commits later. Commitsmainhad superseded were dropped: the analytics embedding model is delivered by #18143's index-space table. It also carries #17856 rows 13a and 16, which were deferred to this branch, the #15722 digest entry it owns, and the owner's 2026-10-12 decision on credential precedence.What Changed
Env and constants:
autobot_shared/env_registry_npu.py);NPU_WORKER_HOSTis split into bind and connect hosts;__global__), whose readers accept both spellings;analytics_infrastructure.py;ssot_configandconfig.compatis removed;node_proxy.One
detect_language. The NPU agent maps the shared enum back to its emitted names.inventory(canonical): 19 concepts with more than one definition and no existing issue — led by two secret stores whose docstrings each claim they cannot disagree about the key #17856 row 13a: one internal-API-key reader,
node_proxy.internal_api_key(), used byauth_middleware, the SLM auth, the vault client, the agent and its mirror, and the migration precheck. Guard:one_internal_api_key_reader_17856_test.py, with 12 contrast fixtures and a reach floor.inventory(canonical): 19 concepts with more than one definition and no existing issue — led by two secret stores whose docstrings each claim they cannot disagree about the key #17856 row 16: one
is_feature_enabled(autobot_shared/feature_flags.py). Three copies with no production callers are retired, and 10 stale fork-baseline entries are dropped.canonical(knowledge): two functions named content_hash return different values — full digest vs 16-char truncation #15722: the analytics dedup key goes through
content_digest(width=16), byte-identical.Security, image-generation credentials:
provider_key_vault, and a scoped resolver is injected only into the built-in plugin, bound to its origin, not its self-declared name;PluginManagerinlifespan.pypassed no resolvers at all. Fixed: oneplugin_paths.pysupplies search dirs and trusted roots to both hosts;plugins/core-pluginsis searched but never trusted;Credential precedence (owner, 2026-10-12): the vault comes first.
resolve_provider_keydefines the order once, andresolve_image_provider_keydelegates to it. The env var is only a fallback, logged once per name with no value. Hydration loads every vaulted name, even when its env var is set.An unreadable vault key is refused (owner, 2026-10-12). When the vault holds a key but the read fails,
_hydrate_one_keyrecords the name as unreadable, which is distinct from "not in the vault".resolve_provider_keythen raisesVaultKeyUnavailableError("vault unavailable: ... refusing the environment fallback") and never returns the env value, which may be rotated out. A later successful hydration, a capture, or removal of the vault row clears the refusal. The two log lines are distinct: "vault unreadable: refusing" at ERROR, and "vault has no key: using the environment" at WARNING. Only registry labels are logged, never a value.registration_refusal()contains a refusal at a registration site, so one refused key skips one provider.A vault outage refuses every key (review of refactor(config): the #18094 de-fork remainder — one key reader, one feature flag, one language detector, and image-gen keys vault-first through an origin-bound resolver (#18094) #18371). A configured vault (
AUTOBOT_SECRETS_ROOT_KEYset) whose startup hydration fails leaves "does it hold this key?" unknown. Failures that count are an unreachable DB, a failing session factory, and a query or decrypt error partway through. The vault state is recorded asnot_hydrated,not_configured,reachableorunreachable. While it isunreachable, every vault-resolved name is refused ("could not be reached at startup") until a hydration succeeds. Env is allowed in one case only: the vault was reachable and has no row for that name.lifespan.pyno longer swallows the failure at debug level as "non-critical".hydrate_at_startup()classifies the vault, logs at WARNING or ERROR, and never raises.Second review (9a): env only for keys the vault truly lacks.
Root key unset with rows in the vault. Hydration no longer reads an unset
AUTOBOT_SECRETS_ROOT_KEYas "no vault". Listing rows needs no root key, so every held name is listed. Without a usable root key (unset or malformed), each held name is recorded as unreadable and refused. Only names with no row fall back to env.VAULT_NOT_CONFIGUREDis removed.Every process hydrates, not only the lifespan.
resolve_provider_keyhydrates on first resolution in any process that has not read the vault, root key or not, since listing rows needs none. A DB the process cannot reach is an outage, and keys are refused. This isservices/provider_key_process_init.py: it runs onuser_management.database.isolated_session_factory(unpooled, disposed), in a private event loop on its own thread. It is safe inside a running loop, never raises, and fails closed if that thread dies.celery_appalso hydrates inworker_process_init, before a worker's first task.Stale and partial state. Hydration rebuilds the cache from the rows into a new map and swaps it in, so a deleted row is never served again and a reader never sees a half-filled cache.
Non-lifespan entry points and how each is covered:
celery_app, e.g.knowledge_tasks→knowledge/base.py)worker_process_inithydration, plus the first-resolution hookbeat_initreconciles the schedule)autobot-infrastructure/shared/scriptsthat drive backend internals (infra: ten infrastructure scripts import backend internals and fail standalone — one guard for the class #18288BACKEND_TOOLS/UNCLASSIFIED, e.g.verify_knowledge_consistency.py,analysis/test_llm_config.py)shared/mcp/tools/knowledge-base-mcp/.../embedded.py)autobot-backend/cli) andcode_intelligence/shared/process_offload.py,knowledge/rag_benchmarks.pysubprocessesCredentialSlotreads env by design.provider_key_vault(system_secrets_vault.py:91,243mention it only in prose)Third review (9a):
testkit/provider_key_vault_state.py, an autouse fixture imported by the backend conftest, starts every test against an explicit empty, reachable vault.AUTOBOT_PROVIDER_KEY_HYDRATION_TIMEOUT_S(default 30). A timeout fails closed and releases the lock.AUTOBOT_PROVIDER_KEY_REHYDRATE_BACKOFF_S(default 30). The delay doubles per failed retry, is capped at 20x, and resets on success. Both variables are registered.More credential readers go through the resolver.
GET /video-generation/providersthroughkey_status(), which is shared with image.CredentialSlot, replacing the image plugin's private copy, and the SDK'sCredentialUnavailableError, whichVaultKeyUnavailableErrorsubclasses.TILDE_API_KEYis vault-resolved.repo_tests/provider_key_env_reads_route_through_vault_18094_test.pyfails on any env read of a registered or credential-shaped name that bypassesresolve_provider_key. It has 10 contrast fixtures, a reach floor of 2617, a registered scope, and a shrink-only allowlist.MODAL_TOKEN_*(the Modal SDK reads the environment itself) andPAPERCLIP_API_KEY(autobot_sharedcannot import the backend resolver). Both are allowlisted under Modal and Paperclip credentials bypass the vault-first provider-key resolver #18383, a sub-issue of tech-debt: forked env defaults, shadowed constants and same-name functions — the remainder after #17419/#16177/#18093 #18094.How each caller surfaces a refusal:
lifespan.pyadapters, the search registry (SearXNG as a whole, Brave), and the capability registry (Slack, Discord).GET /image-generation/providersmarks that provider unavailable with the reason;is_configuredreturns False and logs;LLMResponse.error;negotiate()raises.Behaviour changes
TILDE_API_KEYand the RUNWAY, SORA and KLING video keys now resolve vault-first, like the other provider keys. A video provider whose key is refused reports that reason in the listing and on/generate.settings["api_key"](provider or adapter settings) still outranksresolve_provider_key. That is pre-existing and unchanged; the vault-first order applies only when no per-instance key is given.detect_languagenow classifies.pyi,.pyw,.mjs,.cjs,.mts,.ctsand.vue, which the NPU agent used to report as"unknown". This affects only newly indexed files; no stored data is rewritten.Verification
main: 931 passed, 17 skipped. This covers the newprovider_key_refusal_18094_test.py(12) andllm_shared/tests/test_provider_key_refusal_18094.py, plus the existing tests for every caller module and the credential-vault guard. Its cases:api/video_generation_test.py::test_generate_provider_disabled: its fake provider lacked the newunavailable_reason. Fixed indfb06167c1, 6/6 now.security/threat_intelligence_test.py::test_clear_expired: a fixedsleep(1.1)against a 1s TTL (tests: 32 more fixed sleeps are followed by an assertion, and some decide the outcome by the clock #16255). It fails only in the long mixed run undernice -n 19and passes alone (33/33) and with its siblings (76/76). The cache is untouched here.provider_key_outage_18094_test.pycovers:credential_slot_18094_test.py: video and image plugin suites pass, 22 and 7. The guard passes, 12.security/threat_intelligence_test.py::test_cache_expiration, the same fixed-sleep TTL flake class astest_clear_expired. It passes alone (33/33) and is filed as test(security): threat-intel cache TTL tests fail intermittently in long mixed runs (fixed 1.1s sleep vs 1s TTL) #18400.provider_key_outage_18094_test.py(17) now also covers:celery_apphas theworker_process_inithydration hook.a275bea6e2. The branch was 51 behind, somainwas merged in first.The hub conflicts (
reach_scope_claim, the content-digest list, two reach floors) were resolved by taking main's copy and re-applying only this branch's lines.After the merge:
reach_scope_claim_17844run alone: 10 passed;a275bea6e2; reach_declarations plus those guards then gave 642 passed.New outage tests, 21 in all, add:
Broad pre-merge run, 6,600 tests with every module that resolves keys: 4425 passed, 2159 skipped, 4 failed, 7 errors. None are in the code this PR changes:
chat_knowledge_system_e2e_test(7 errors): needs an authenticated live backend and gets HTTP 401;file_upload_comprehensive_test::test_knowledge_base_integration: checks that the CWD-relativedata/file_manager_rootexists;mcp_cache_test::test_cache_age_trackingand twotest_fact_store_authoritytests: pass when run alone.I could not compare against
main, because the worktree ceiling (15 of 15) blocked a temporary main checkout.The pre-push pytest step timed out on 127 files, so
a275bea6e2was pushed withAUTOBOT_PREPUSH_ALLOW_TIMEOUT=1after the runs above.2326fde67d:[pre-push OK] pytest: all relevant tests pass. This is the first full pre-push run on this branch. The run before it caught the redaction census reading the plugins' key-name tuples as a private vocabulary, fixed in2326fde67d(census 15/15). Ratchets (file size, ceiling parity, guard_reach_meta, symbol forks, reach_declarations): 626 passed.reach_scope_claim_17844run alone: 10 passed.strsubclass can't spoof membership;reach_scope_claim_17844run alone after mergingmain: 10 passed.AUTOBOT_PREPUSH_ALLOW_TIMEOUT=1after the runs above. Three Postgres-backed precedence tests intests/migrations/test_provider_key_vault.pyhave not run locally; CI is their first run.Not done, stated rather than implied
git ls-files, 5 of those unanchored, and 33 report clean on an empty scan #14896, which is open.MODAL_TOKEN_ID/MODAL_TOKEN_SECRETandPAPERCLIP_API_KEYstill bypass the resolver, tracked in Modal and Paperclip credentials bypass the vault-first provider-key resolver #18383 and allowlisted in the new guard.main), and a bug(slm-tests): test_wizard_inventory_parity_14286.py's ad-hoc loader breaks on services.role_manifest_shape import #17164 sighting (three SLMtests/servicesfiles fail to collect).AUTOBOT_BASE_DIRfallbacks stay on purpose:role_registry.py:23is the remote node install root, a different concept, andcode_sync.py:2588is the bootstrap fallback when SSOT fails to load.__global_user__stored spelling needs a Umbrella: SLM Data hygiene page — detect and safely clean up orphaned data across services #17038 key rewrite; for now it is read-only.ssot_config.misc.internal_api_keyis still declared but has no production reader.DEFAULT_HEARTBEAT_INTERVALstill has 3 definitions:slm/agent/agent.py:99, its mandated ansible mirror, and the Windows NPU worker'sbackend_telemetry.py:345.detect_language, this PR consolidates none of them.So this PR is Refs #18094.
Refs #18094
Refs #17856
Refs #15722
Model Used
Claude Opus 5.5 (master session: decisions, brief, PR). Finished by a Claude Opus subagent; the original commits came from a subagent on 2026-10-10.
🤖 Generated with Claude Code