Repository navigation
bug(slm-secrets): saving LLM settings overwrites every provider API key in the vault with its masked display string #17826
Description
Activity
- added a commit that references this issue
on Oct 2, 2026 Fix in #17846, plus one correction to the chain and a recovery note
Correction to hop 6. The round-trip never reached
vault_rotate.LLMProviderConfighas noapi_key_vault_idfield, so the vault reference is dropped when_load_llm_configbuilds the model, andmodel_dump()on save never carries it. Every save took thevault_createbranch instead.EnvelopeSecretsService.createinserts unconditionally, with no(owner_vault, name)uniqueness, so each damaging save created a new system-vault secret holding the mask and repointed the setting at it.What that means for recovery. Proposal for a human under #17038; no agent restores or rewrites credentials:
Storage path What a damaging save did Where the real key is Vault (vault credential configured on the SLM) new secret with the mask; setting repointed to it still in the vault: an older row named llm:provider:<name>:api_key. Every save that carried a key (mask or genuine) minted a new row, so the real key is the newest row of that name whose value is not mask-shaped.Inline fallback (vault not configured) mask encrypted over the ciphertext in the settings row only in a database backup from before the first damaging save Not determined: whether any deployment has actually been hit. Checking which path the live SLM is on means reading its vault-credential configuration, which was refused as credential materialization, and I did not pursue it another way. A human can answer it read-only, but a row count alone does not: before #17846 a genuine key change also took
vault_create, so several rows for one name prove saves happened, not damage. The deciding check is whether the row the setting currently points to (api_key_vault_id) holds a value of the mask shapexxxx...xxxx/****. That means reading a secret value, which is a human's call.- added a commit that references this issue
on Oct 2, 2026 - added a commit that references this issue
on Oct 2, 2026 Closure evidence: #17846 merged as
e446aed3d7; every AC verified againstorigin/mainAC Evidence in origin/mainA submitted api_keyequal to the mask leaves the stored secret untouched, proven by a GET→PUT testautobot-slm-backend/tests/api/test_llm_config_round_trip_17826.py:test_vault_path_round_trip_leaves_the_stored_reference_untouched(GETs through a real TestClient, PUTs the response back, asserts the stored reference is byte-identical and the vault had no create, rotate or read) andtest_a_display_mask_put_back_is_refused_and_nothing_is_written(a mask gets 422, stored entry unchanged)Stop returning the field (omission, the sso_secrets.pypattern)api/llm_config.py:210GET and:224PUT both declareresponse_model_exclude=_NO_PROVIDER_KEYS.user_management/services/llm_secrets.pypublic_provider_viewdropsapi_key,api_key_vault_idandapi_key_ref._mask_api_keyand theretrieve_provider_api_keyimport are absent fromllm_config.py(grep: 0 hits).Regression test covers the vault path and the inline-encryption fallback Same file: test_vault_path_round_trip_…andtest_inline_encryption_path_round_trip_leaves_the_ciphertext_byte_identical, plus unit coverage intests/services/test_llm_secrets.py(TestRoundTripLeavesTheStoredKeyUntouched)CI on the merged head
3f56d23205: green.migration-testsskipped by its path gate (no migration or model path in the diff), withmigration-matrixSUCCESS.startup-import-smokereal check SUCCESS; the placeholder skipped by design.Still open, deliberately not closed by this fix:
- Recovery on installations already hit. The proposal for a human, under Umbrella: SLM Data hygiene page — detect and safely clean up orphaned data across services #17038, is the earlier comment on this issue. Whether any host was affected is not determined.
- bug(slm-llm): the provider Test button cannot test a saved cloud provider, and the naive fix would exfiltrate its key #17845: the Test button cannot test a saved provider.
- bug(secrets): the envelope store allows duplicate (owner_vault, name), so first-time secret creation is not idempotent #17850: duplicate
(owner_vault, name), blocked by this issue's recovery. - bug(slm-llm): renaming or removing an LLM provider orphans its stored API key #17858: rename or removal orphans the key.
- investigate(slm-llm): stored LLM provider API keys have no production reader #17859: no production reader of stored keys.
What's broken
Saving any change on the SLM admin LLM settings page overwrites every configured provider's real API key in the vault with its 11-character masked display string. The key is destroyed; there is no undo and no confirmation.
The chain (every hop verified)
autobot-slm-backend/api/llm_config.py:127-142—_load_llm_config()resolves the real key for each provider:p["api_key"] = await retrieve_provider_api_key(p.get("name", ""), p).autobot-slm-backend/api/llm_config.py:194-196—get_llm_config()masks it for display:provider.api_key = _mask_api_key(provider.api_key), producingsk-a...b3f2(_mask_api_key,:113-117).autobot-slm-frontend/src/views/settings/admin/LLMSettings.vue:70-82—fetchConfig()doesObject.assign(config, resp.config). The mask is now the value ofconfig.providers[i].api_keyin browser state.autobot-slm-frontend/src/views/settings/admin/LLMSettings.vue:85-96—saveConfig()PUTs the whole object:api.saveConfig({ ...config }). Any unrelated edit (toggling a provider'senabled, changing the Ollama port, adding a model) enables the save button and sends the masked strings back.autobot-slm-backend/api/llm_config.py:212-216—save_llm_config()doesif d.get("api_key"): d = await store_provider_api_key(p.name, d). No comparison against the mask.autobot-slm-backend/user_management/services/llm_secrets.py:96—await vault_rotate(uuid.UUID(existing_vault_id), api_key)rotates the vault entry tosk-a...b3f2.The only shape check on this path is
if not api_key.startswith("gAAA")in the non-vault fallback (llm_secrets.py:88), which exists to avoid double-encryption and does not catch a mask.Failure scenario
An admin opens LLM settings, flips one provider's
enabledtoggle, clicks Save. Every other provider's API key is now the stringsk-a...b3f2in the vault. The next LLM call to those providers fails with an auth error, and the original keys are unrecoverable from AutoBot.Acceptance criteria
api_keyequal to the mask leaves the stored secret untouched — verified by a test that GETs the config, PUTs it back unchanged, and asserts the vault value is unchanged.autobot-slm-backend/user_management/services/sso_secrets.py:144-189, whichpops the sensitive field from the sanitized config so a naive round-trip is a safe no-op. Omission is structurally safer than a mask because it cannot be mistaken for a value.Related
autobot-backend/knowledge/connectors/credential_store.py:285-332rotate()merges only supplied fields and is the other in-repo pattern that avoids this trap.docs/research/private-tenant-chat-reference-app.md(finding F1).