Repository navigation
Conversation
…le regexes CodeQL flagged 3 high-severity alerts introduced by the #17077 merge: - py/clear-text-logging-sensitive-data (npu_client.py:471): the error log on embedding-generation failure logged a raw prefix of caller-supplied text. generate_embedding_with_fallback() is a generic low-level client called from several places (work_intent_similarity, semantic_search, vector_search_engine, code_intelligence) that don't all redact before embedding, so a failed call could log real secret material. Now redacts via autobot_shared.secret_redaction.redact_content() before truncating. - py/polynomial-redos x2 (secret_redaction.py:325, :347): the PEM-block and data-URI matchers used unbounded quantifiers ([\s\S]*? and [^,\s]+) whose lazy/greedy scan restarts at every occurrence of the anchor literal in the input, giving O(n^2) worst case on adversarial content (many "-----BEGIN ... PRIVATE KEY-----" or "data:" occurrences with no closing marker) -- ironic since this module itself scans arbitrary KB/fact content. Bounded both quantifiers to generous but finite limits that comfortably cover real PEM key bodies (up to ~8192-bit RSA) and real MIME types, verified linear (not quadratic) scaling and full behavioral equivalence against every true-positive/false-positive fixture in secret_redaction_test.py.
Contributor
|
Warning Review limit reachedNext included review available in 42 minutes. View limit detailsLimit details: You’ve used the included review currently available. You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. Review configuration: ⚙️ Run configurationConfiguration used: Repository: mrveiss/AutoBot-AI/.coderabbit.yaml Review profile: ASSERTIVE Plan: Advanced Run ID: 📒 Files selected for processing (2)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
1 task
…og path (#17104 review) redact_content(text)[:80] ran every redaction regex over the full caller-supplied text (up to ~100 KB) on an unconditional logger.error in the embedding-failure path. Slice to 512 chars before redacting -- well over the final 80-char prefix, so a credential crossing that boundary is still masked, but the regex cost is now bounded.
Contributor
This was referenced Sep 19, 2026
| model_name, | ||
| last_reason, | ||
| text[:80], | ||
| redact_content(text[:512])[:80], |
mrveiss
added a commit
that referenced
this pull request
Sep 19, 2026
Owner
Author
|
Carried by #17116 ( |
mrveiss
added a commit
that referenced
this pull request
Sep 19, 2026
mrveiss
added a commit
that referenced
this pull request
Sep 19, 2026
…ason - role_registry.py: 727 -> 715 lines. Collapses the slm-backend #16889 comment from 9 lines to 3 and merges it with the adjacent pre-existing #14275 comment, and tightens the post_sync_cmd string layout -- no behavior change, back under the grandfathered 715-line ceiling (#14236). - secrets_baseline_reasons.py: the Basic Auth Credentials entry for autobot_shared/secret_redaction.py cited a bare "line 242", which drifts whenever a regex rewrite upstream (like this vehicle's own #17104 member) moves the comment -- reworded to cite the comment above `_BASIC_AUTH_URL_RE` by name instead. - glob_declared_reads_15900_test.py: the main merge below brought in #15317's chromadb_bind_not_hardcoded_15317_test.py, whose *.service, *.service.j2 and docker/*.yml glob declarations were undetected by that guard's own coverage sweep -- recorded all three (plus adding the guard to the pre-existing *.sh entry's set) so the pre-push hook's glob-declared-reads check passes. Also merges origin/main (143 commits) to bring this code-touching PR to exactly 0-behind, per the owner's merge-safety rule.
mrveiss
added a commit
that referenced
this pull request
Sep 22, 2026
…dependabot uv) (#17116) * fix(ci): give the SLM backend's tests their own venv in CI (#16394) python-shard shared one venv between autobot-backend and autobot-slm-backend's pytest invocations, built from requirements-ci.txt -- autobot-backend/requirements.txt's langgraph-sdk caps websockets<16, while autobot-slm-backend/requirements.txt declares websockets>=17.1,<18 (SLM's real production version). SLM's tests always ran against 15.0.1 as a result, and #16391 carried a named exemption in check_dependency_floors.py's KNOWN_CROSS_VENV_EXEMPTIONS pending a real fix. setup-python-suite/action.yml now builds a second, parallel venv from autobot-slm-backend/requirements.txt + requirements-ci-test.txt, exposed via a new `slm-venv` output (not activated onto $GITHUB_PATH, reached explicitly so it coexists with the backend venv). ci.yml points the slm-backend pytest invocation at that venv's own interpreter, and splits the strict dependency-floor check into two --roots-scoped calls (one per venv, mirroring scripts/setup-ci-parity-env.sh's existing pattern) so each service's floors are judged against what its own venv actually installs. The KNOWN_CROSS_VENV_EXEMPTIONS entry is removed and pinned empty by a test. * fix(ci): python-paths covers the shell wrappers python-suite tests (#16249) `pipeline-scripts/check-pre-commit-hook-pr.sh` is run by two python-suite tests, and no pattern in `.github/filters/python-paths.yml` matched it. A change confined to the wrapper computed `python != 'true'`, the required-context shim reported `python-suite` green, and the tests that exercise the change never ran — a guard bypassable by touching the one file it exists to watch. Six paths, not the one reported: four wrappers with sibling tests (check-pre-commit-hook-pr.sh, check_baseline_no_growth.sh, detect-hardcoded-values.sh, pr-queue-open-list.sh) and the two libraries they source (scripts/lib/git-scope.sh, scripts/lib/hardcoded-value-rules.sh). Both extra findings came from the new guard failing against my own enumeration, before review rather than after: * I paired scripts to tests by NAME, which crosses a hyphen/underscore boundary silently — pr-queue-open-list.sh is tested by pr_queue_open_list_test.py. The guard pairs them by CONTENT and named it. * my source-line pattern matched the quoted argument as `[^"]+`, which stops at the first INNER quote. The portable spelling nests them — `source "$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/../x.sh"` — so it yielded `$(cd `, ended in no `.sh`, and was dropped as if the line sourced nothing. Only the simple `"${REPO_ROOT}/..."` spelling parsed, which is how hardcoded-value-rules.sh — sourced by two of the four — surfaced at all, and how git-scope.sh then read as absent. The extractor now matches the `.sh` argument itself, last one on the line. Both variables are assigned `$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)`, checked in each script rather than inferred from the name. The transitive closure was checked too, per the issue's "not verified" note: neither library sources anything, though both look like they might — git-scope.sh contains the word `source` four times and hardcoded-value-rules.sh six, and every occurrence is prose in a comment. An anchored grep returned zero for the first; counting the unanchored word is what made that zero trustworthy. Why the existing coverage guard cannot catch this, which the issue did not record. `python_filter_covers_its_guards_test.py` exists for exactly this class and two independent things stop it: it sweeps `repo_tests/` only, and `pipeline-scripts`, `scripts` and `tools` sit in its `_NOT_A_READ` set — whose stated premise, "trees whose contents no guard reads directly", is false, since check-pre-commit-hook-pr_test.py reads its own `.sh`. Sweeping those directories would still discard the finding. Widening it was measured, not guessed. The sweep was reimplemented independently and validated against the recorded baseline (38 uncovered reads, equal to MAX_UNCOVERED_READS and to the 38 recorded entries) before any option was judged: today 38 also sweep pipeline-scripts/ 58 ...and drop pipeline-scripts from _NOT_A_READ 63 ...and drop scripts too 77 Every option raises a count whose own rule is that it may only fall, and the 25 newly surfaced reads are mostly workflow and config files — a different population from this defect. So the narrow invariant is asserted directly in `repo_tests/python_filter_covers_tested_shell_wrappers_test.py`: a wrapper that python-suite tests must itself be able to trigger python-suite. Exact, and it costs the ratchet nothing — re-measured after the filter change, still 38, so the equality rule holds in both directions. That guard imports the matcher from the existing one rather than restating it. Two matchers drifting apart would disagree silently and certify coverage the real gate does not grant; a rename breaks the import loudly instead. Four of its six tests are controls: the sweep must find at least four tested wrappers and name the reported one; the extractor is pinned to exact sourced sets for all four real wrappers, including the empty one, which is what would catch an extractor that started matching prose; an argument that does not resolve to a repository file is REPORTED rather than skipped, so an unreadable source line cannot look like a script that sources nothing; and the imported matcher is asserted to say no to an absent path, without which both findings tests pass vacuously. Closes #16249 * test(guards): record the new wrapper guard's `*.sh` glob (#16249) `python-suite shard 8/12` failed on `glob_declared_reads_15900_test.py::test_the_record_names_exactly_the_guards_that_declare_each_glob`: declaring-but-not-recorded: ['repo_tests/python_filter_covers_tested_shell_wrappers_test.py'] The guard added by this PR sweeps `pipeline-scripts/*.sh`, and #15900's record names every guard that declares each tracked glob so a sweep cannot quietly appear or vanish. A new declarer has to be added in the same change — which is the record doing its job, not an obstacle. Only `*.sh` needed recording. The guard also calls `glob("*_test.py")`, and that pattern is not tracked: the record carries `*_test.sh` but no `*_test.py`. Checked the two other guards this session added for the same trap — `command_position_scan_test.py` and `hook_declarations_are_reachable_test.py` — and neither globs at all; they read a module's `dir()` and the router's exported routes respectively. So this is the only entry needed. * fix(ci): declare numpy/jinja2/aiosqlite the SLM venv now needs for real (#16394) Giving autobot-slm-backend's tests their own venv (this same PR) surfaced 3 ModuleNotFoundErrors across every shard's SLM test run -- numpy, jinja2, and aiosqlite were never declared in autobot-slm-backend/requirements.txt, only ever present because CI's OLD shared venv installed them transitively for the backend's own needs (aiosqlite>=0.22.1 is autobot-backend/requirements.txt's own pin; numpy/jinja2 arrived from some other backend dependency's own closure, never declared directly there either). tests/test_bi_dashboard.py needs real numpy (documented in its own comment, no stub) and falls back to a jinja2 stub only when the real package is genuinely absent; migrations/migrate_system_secrets_to_vault_test.py and user_management/services/sso_e2e_test.py need the real `sqlite+aiosqlite://` SQLAlchemy driver for in-memory async DB tests. numpy pinned via the existing `-c ../constraints/shared.txt` convention (autobot-backend/requirements.txt:9's same mechanism), not a local version -- this dev box's Python 3.10.12 cannot resolve constraints/shared.txt's numpy>=2.5.3 floor (needs 3.11+), so that one is unverifiable locally; CI runs the declared 3.14.7. jinja2 and aiosqlite verified installable in an isolated venv on this box (jinja2 3.1.6, aiosqlite 0.22.1). * fix(ci): mirror SLM's jinja2 into CI and fix its Docker constraints copy (#16394) * fix(tests): use the canonical repo_root() helper, not a hand-derived one (#16249) * chore(deps): bump the uv group across 1 directory with 2 updates Bumps the uv group with 2 updates in the /autobot-infrastructure/shared/mcp/tools/knowledge-base-mcp directory: [anyio](https://github.com/agronholm/anyio) and [httpx2](https://github.com/pydantic/httpx2). Updates `anyio` from 4.12.1 to 4.14.2 - [Release notes](https://github.com/agronholm/anyio/releases) - [Commits](agronholm/anyio@4.12.1...4.14.2) Updates `httpx2` from 2.10.0 to 2.12.0 - [Release notes](https://github.com/pydantic/httpx2/releases) - [Changelog](https://github.com/pydantic/httpx2/blob/main/src/httpx2/CHANGELOG.md) - [Commits](pydantic/httpx2@v2.10.0...v2.12.0) --- updated-dependencies: - dependency-name: anyio dependency-version: 4.14.2 dependency-type: indirect dependency-group: uv - dependency-name: httpx2 dependency-version: 2.12.0 dependency-type: indirect dependency-group: uv ... Signed-off-by: dependabot[bot] <support@github.com> * fix(tests): import Path in python_filter_covers_tested_shell_wrappers_test.py (#16249) pyflakes F821: Path is used in 3 type annotations (dict[Path, list[str]], def _sourced_paths(script: Path)) but never imported. from __future__ import annotations defers evaluation at runtime, but pyflakes still resolves annotation strings statically. Pure import addition, no logic change. * fix(security): redact secrets in npu_client log, bound ReDoS-vulnerable regexes CodeQL flagged 3 high-severity alerts introduced by the #17077 merge: - py/clear-text-logging-sensitive-data (npu_client.py:471): the error log on embedding-generation failure logged a raw prefix of caller-supplied text. generate_embedding_with_fallback() is a generic low-level client called from several places (work_intent_similarity, semantic_search, vector_search_engine, code_intelligence) that don't all redact before embedding, so a failed call could log real secret material. Now redacts via autobot_shared.secret_redaction.redact_content() before truncating. - py/polynomial-redos x2 (secret_redaction.py:325, :347): the PEM-block and data-URI matchers used unbounded quantifiers ([\s\S]*? and [^,\s]+) whose lazy/greedy scan restarts at every occurrence of the anchor literal in the input, giving O(n^2) worst case on adversarial content (many "-----BEGIN ... PRIVATE KEY-----" or "data:" occurrences with no closing marker) -- ironic since this module itself scans arbitrary KB/fact content. Bounded both quantifiers to generous but finite limits that comfortably cover real PEM key bodies (up to ~8192-bit RSA) and real MIME types, verified linear (not quadratic) scaling and full behavioral equivalence against every true-positive/false-positive fixture in secret_redaction_test.py. * fix(perf): bound redact_content's input on npu_client's hot failure-log path (#17104 review) redact_content(text)[:80] ran every redaction regex over the full caller-supplied text (up to ~100 KB) on an unconditional logger.error in the embedding-failure path. Slice to 512 chars before redacting -- well over the final 80-char prefix, so a credential crossing that boundary is still masked, but the regex cost is now bounded. * fix(security): stop logging redacted-but-content-derived text on the embedding-failure path CodeQL py/clear-text-logging-sensitive-data does not model redact_content() as a sanitizer, so a redacted excerpt still trips the query. Log no input-derived content at all: length plus a truncated sha256 hash is enough to correlate failures without ever reproducing what was sent. Drops the now-unused redact_content import. * fix(ci): route slm-backend's post_sync_cmd through build-filtered-requirements.sh (#16889 review) #16394 gave autobot-slm-backend/requirements.txt its own sibling-relative -c ../constraints/shared.txt include, which the deployed directory can't resolve. A bare pip install -r aborted the sync, and the && silently skipped alembic upgrade head (#11069/#14272) -- the same failure mode already fixed for the backend and ai-stack roles. Routes through the same canonical script so all three deploy paths share one implementation. * fix(ci): give slm-backend's filtered-requirements temp file its own name (#16889 review) The previous fix copied the backend role's post_sync_cmd pattern verbatim, including its /tmp/requirements-filtered-slm.txt output path -- already used by the backend role itself. code_sync runs sync jobs as concurrent tasks, so on a node hosting both roles the two jobs can interleave and one filter's output gets pip-installed into the other role's venv: a silent wrong-deps install, not a visible failure. Adds a regression test asserting every role's filtered-output /tmp path is unique, verified to catch the exact collision by reverting locally and confirming it fails, then restoring. * fix(vehicle): trim role_registry.py to ceiling, reword line-number reason - role_registry.py: 727 -> 715 lines. Collapses the slm-backend #16889 comment from 9 lines to 3 and merges it with the adjacent pre-existing #14275 comment, and tightens the post_sync_cmd string layout -- no behavior change, back under the grandfathered 715-line ceiling (#14236). - secrets_baseline_reasons.py: the Basic Auth Credentials entry for autobot_shared/secret_redaction.py cited a bare "line 242", which drifts whenever a regex rewrite upstream (like this vehicle's own #17104 member) moves the comment -- reworded to cite the comment above `_BASIC_AUTH_URL_RE` by name instead. - glob_declared_reads_15900_test.py: the main merge below brought in #15317's chromadb_bind_not_hardcoded_15317_test.py, whose *.service, *.service.j2 and docker/*.yml glob declarations were undetected by that guard's own coverage sweep -- recorded all three (plus adding the guard to the pre-existing *.sh entry's set) so the pre-push hook's glob-declared-reads check passes. Also merges origin/main (143 commits) to bring this code-touching PR to exactly 0-behind, per the owner's merge-safety rule. --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Thinking Path
Fetched
origin/mainat the merge commit for #17077 (928573e3bb), then pulled the actual CodeQL SARIF for that PR's Python analysis (refs/pull/17077/merge, analysis id1803753251, 3 results) rather than trusting only the alert summary — the org-wide/code-scanning/alertsendpoint hadn't indexed these yet (main's CodeQL scan for this exact commit hasn't run), so the SARIF was the only way to get the precise CodeQL data-flow trace and confirm these are real, not fabricated line/rule mappings.The SARIF flow for Finding 1 showed
generate_embedding_with_fallback'stextparameter is reachable, atnpu_client.py:471, both from a redacted fact-content path and (confirmed separately by grepping callers) from several unredacted callers (work_intent_similarity.py,semantic_search.py,vector_search_engine.py,code_intelligence/*) — so this isn't CodeQL missing an existing sanitizer, it's a genuine gap for callers that don't pre-redact.For Findings 2/3, read both flagged regexes (
_PEM_BLOCK_RE,_DATA_URI_RE) and reasoned through the backtracking: both have an unbounded quantifier ([\s\S]*?,[^,\s]+) that gets retried from every occurrence of the pattern's own anchor literal in adversarial input with no closing marker — O(n^2). Fixed by bounding both quantifiers to generous-but-finite limits, then verified empirically (see Verification) rather than assuming correctness.What Changed
autobot-backend/services/npu_client.py:generate_embedding_with_fallback's failure-path error log now redactstextviaautobot_shared.secret_redaction.redact_content()before truncating to 80 chars, instead of logging the raw caller-supplied prefix (CodeQLpy/clear-text-logging-sensitive-data). Reused the existing security(secrets): credential redaction is name-keyed and cannot see credentials in free text — add content scanning before mail is indexed #13708 redaction utility rather than writing new masking logic, per repo convention.autobot_shared/secret_redaction.py: bounded the two regex quantifiers CodeQL flagged as polynomial-ReDoS-vulnerable (py/polynomial-redos):_PEM_BLOCK_RE: header class capped at 40 chars (real PEM key-type headers are well under 12), body capped at 16384 chars (~5x a real 4096-bit RSA key body, ~2.5x an 8192-bit one)._DATA_URI_RE: media-type span capped at 255 chars (real MIME types, even long ones, are well under that).Verification
_TRUE_POSITIVES/_FALSE_POSITIVESfixture fromsecret_redaction_test.pythrough old vs. new: all 16 cases match (old == newmatch presence for every fixture, for both regexes) — no behavior change for real credentials or ordinary text."-----BEGIN PRIVATE KEY-----" * n, no closing marker) at n=2500/5000/10000/20000: time scales linearly with input size (roughly doubles when input doubles), confirming the fix changes the complexity class from O(n^2) to O(n) rather than just shrinking the constant.pytestonautobot_shared/secret_redaction_test.pypassed on push, independently confirming the equivalence analysis above.gh apithatgenerate_embedding_with_fallbackhas unredacted callers beyond the fact-content path, so Finding 1 is a real gap, not a CodeQL false positive against an already-sanitized value.Residual note (not fixed here, out of scope for this PR): the direct callers of
redact_content/scan_content_for_credentialsviaknowledge_graph_routes.py→pipeline/runner.pyhave no upstream content-length cap (unlike thebulk.pyfact-content path, which caps at 100000 chars) — so a very large adversarial document on that path would still take non-trivial (linear, not quadratic) time. Flagging for awareness; happy to file a follow-up issue if wanted.Single-issue rationale
Both fixes are the direct, minimal remediation for the 3 CodeQL alerts introduced by the same merge (#17077) and land together because they're the same security gate blocking
main— no unrelated changes bundled in.Model Used
Claude Sonnet 5