Repository navigation
security(secrets): credential redaction is name-keyed and cannot see credentials in free text — add content scanning before mail is indexed #13708
Description
Activity
- added 8 commits that reference this issue
on Sep 17, 2026 Verified against merged code (
origin/main@928573e3bb, PR #17077)The fix landed on branch
issue-13708-content-redaction, merged intomainvia the
vehicle-v090-2026-09-19-security-kbtrain (PR #17077, member PR #16895). All 9 commits
tagged#13708are in the merge range9b21b99a78..928573e3bb. Issue only referenced
(not auto-closed) because #17077'sClosesline listed multiple issues and only the first
fired — reopening/closing this manually now that the fix is verified.AC Status Evidence Content-scanning redactor in autobot_shared/,redact_valueleft intact for current callers✅ Met autobot_shared/secret_redaction.py:303scan_content_for_credentials(),:362redact_content().redact_valueunchanged at:172, still used byautobot_shared/ssot_config.py,autobot-backend/models/settings.py,autobot-backend/pki/config.py,autobot-backend/llm_shared/observability/tracing_config.py.Quarantine path: a body with a detected credential is stored redacted, never reaches the indexer/embedder in the clear ✅ Met autobot-backend/knowledge/ingest_sanitize.py:122—sanitize_fact_content()runsredact_content()beforestore_fact/update_factreach_store_and_vectorize_fact(Redis projection + ChromaDB vectorization). Chokepoint closed further in commits126c186ab7(true KB-write chokepoint, all 4 API write routes + connector sync),f99e076738(version-rollback gap),c25aea6124(ECL pipeline, backup restore, URL-shaped metadata fields, dedup-hash consistency) — enforced by an AST sweep guard,repo_tests/kb_content_redaction_chokepoint_guard_test.py.Wired into llc/api/replay.py, closing the "not redacted... would require regex patterns not currently present" gap at the old lines 261-263✅ Met autobot-backend/llm_shared/credential_redaction.py:76—redact_string()now callsredact_content()(the new content scanner) after its own patterns; consumed byllc/services/replay_service.py:349-369(redact_dict/redact_string), which backsGET /agents/{id}/runs/{id}/replay-log?redact_pii=trueinllc/api/replay.py. The old caveat text no longer exists anywhere in the repo (git grepfor it returns nothing). End-to-end test:autobot-backend/llc/tests/test_replay_credential_redaction.py— drives a real onboarding-email body with a password through_log_to_dict(..., redact_pii=True)and asserts the password is gone.False-positive behaviour tested — code snippet / base64 image must not quarantine a whole thread ✅ Met autobot_shared/secret_redaction_test.py_FALSE_POSITIVES(lines ~100-114):code_snippet(api_key = os.environ.get(...),getKey()),base64_image(adata:image/png;base64,...<img>tag), plus plain text, UUID, commit SHA, constant name, GitHub URL, function signature — all asserted to produce zero matches and be left untouched byredact_content.Tested against the reproduction: a real credential shape is caught, an ordinary body is not ✅ Met Same file, _TRUE_POSITIVES(lines ~52-61): "your temporary password is X" phrasing, an OpenAI-shaped key, a JWT, a PEM private-key block, a basic-auth URL, and AWS/GitHub/Slack key shapes — each asserted caught byscan_content_for_credentialsand masked byredact_content, with the original span asserted gone from the output. Idempotency also covered (redact_content(redact_content(x)) == redact_content(x)).No hardcoded UI strings if an operator-facing message was added ✅ N/A / met No frontend/ .vuefiles appear in any#13708-tagged commit (git log --all --grep="#13708" --name-onlyhas zero.vuehits) — this landed as backend redaction logic only, no operator-facing string was added.All 6 acceptance criteria are met in the code currently on
main. Closing.Closing: all 6 ACs verified against merged code on main (see evidence comment above).
Part of umbrella #13707.
Design doc: docs/research/email-reading-capability-audit.md
Wave 1 — foundation. Depends on: nothing. Can start immediately.
This blocks the mail connector. No mail may reach the knowledge base until this ships.
Problem
autobot_shared/secret_redaction.pyredacts by field name, not by content. It cannotsee a credential sitting in free text.
is_credential_field(name)(line 84) matches a field name againstCREDENTIAL_SUFFIXES.redact_value(name, value)(line 121) only masks when handed a credential-shaped name.redact_url_userinfo(line 100) handles connection strings, and fails closed onunparseable input — correct, but still URL-shaped values only.
Every caller is a config/model context:
autobot_shared/ssot_config.py,models/settings.py,pki/config.py,services/config_revision_service.py,services/workflow_secret_service.py,llm_shared/observability/tracing_config.py.Nothing scans a message body.
Why this matters now
Signup and onboarding emails routinely carry the credential in the body — "your temporary
password is X", "here is your API key: Y". Under umbrella #13707 those bodies get parsed by
knowledge/connectors/content_extraction.py, persisted, indexed into Chroma, and embedded.A credential copied into embeddings cannot be revoked. Deleting the vault row does
nothing about the vector-store copy, the logs, or the replay record.
This is already a known hole, independently of mail:
llc/api/replay.py:261-263states thatemails are not redacted in replay, noting it "would require regex patterns not"
currently present.
Scope
Add a content-scanning redactor alongside the existing name-keyed one. This is a
companion capability, not a rewrite —
redact_valuestays as-is for config paths.bearer/JWT shapes, private-key PEM blocks,
password is <x>phrasings, basic-auth URLs.callers can quarantine rather than silently mangle.
Acceptance criteria
autobot_shared/, with the name-keyedredact_valueleft intact and still used by its current callerspassed to the indexer/embedder in the clear
llc/api/replay.py, closing the gap documented at lines 261-263trigger quarantine of an entire thread
credential shape is caught, and an ordinary body is not
Files
autobot_shared/secret_redaction.py— companion content scannerautobot-backend/llc/api/replay.py— consume it; remove the "not redacted" caveatautobot-backend/knowledge/connectors/content_extraction.py— redact before persistenceEvidence
Full audit: docs/research/email-reading-capability-audit.md section 4.3.