Skip to content

fix(guards): mypy resolves autobot_shared so enum aliases stay types; the embedding-literal guard covers namespaced ids and the infra builders (#18345) - #18393

Merged
mrveiss merged 3 commits into
mainfrom
issue-18345-guard-enum-batch
Oct 12, 2026
Merged

mrveiss merged 3 commits into
mainfrom
issue-18345-guard-enum-batch

Conversation

@mrveiss

@mrveiss mrveiss commented Oct 12, 2026 •

Copy link
Copy Markdown
Owner

Thinking Path

Two guard and enum issues in one batch (same scope, no shared files with anything in flight):

What Changed

  • types: an enum alias of the canonical Severity used as an annotation is 'not valid as a type' in the backend mypy run (6 instances) #18345, the cause (established, not guessed):
    • Reproduction: run from autobot-backend/, mypy cannot resolve autobot_shared. ignore_missing_imports = true then turns Severity into Any, so a module-level alias reads as a variable. Run from the repo root, the same files are clean.
    • Fix, once: mypy_path = "$MYPY_CONFIG_FILE_DIR" in pyproject.toml [tool.mypy]. No per-site changes.
    • The X: TypeAlias = Severity form the issue suggested hides the bug: it reports success while X is silently Any. So it was rejected.
  • test(guard): the embedding-model literal guard misses namespaced model ids and other families, and skips two infra roots #18328:
    • One prefix list: KNOWN_EMBEDDING_MODEL_PREFIXES and is_embedding_model_id() in autobot_shared/embedding_model.py, now the only embedding-family list (the guard's old regex is deleted). It allows an optional org/ namespace and covers the e5/gte/jina, multi-qa- and codebert families. A reranker (bge-reranker-*) shares the bge- prefix but is excluded.
    • Roots gain autobot-infrastructure/shared/scripts/populate_kb_chromadb.py and autobot-infrastructure/shared/tools/index_documentation.py. The issue named a wrapper with no literal; the real collection builder is the shared/tools copy.
    • New hits, classified per bug(kb): embedding cache key and generator resolve the embedding model from different config fields #18143:
      • Both builders → active, so they now call active_embedding_model() instead of "nomic-embed-text". They write autobot_docs, which is not in knowledge/index_embedding_space.py and is read by doc_searcher.py with active_embedding_model() plus a stored-dimension check. The old literal is what could mismatch.
      • code_embedding_generator.py microsoft/codebert-base → index-bound (CODE_INDEX).
      • pinned_model_registry.py → not-a-choice (a revision/digest pin).
    • reach_scope_claim_17844 is updated for the new roots.

Verification

  • types: an enum alias of the canonical Severity used as an annotation is 'not valid as a type' in the backend mypy run (6 instances) #18345 pin: repo_tests/mypy_enum_alias_resolution_18345_test.py (3 tests, about 11 s) runs mypy on a fixture from a non-root cwd with the repo config and asserts no valid-type. Its contrast (a config without mypy_path) asserts valid-type appears.
  • Strict autobot_shared/ mypy gate unchanged: over all 391 files, origin/main and HEAD both give "86 errors in 17 files", with identical sorted error lists.
  • test(guard): the embedding-model literal guard misses namespaced model ids and other families, and skips two infra roots #18328 contrasts: detected are sentence-transformers/all-MiniLM-L6-v2, BAAI/bge-small-en-v1.5, microsoft/codebert-base, intfloat/e5-large-v2, thenlper/gte-base and jinaai/jina-embeddings-v2-base-en. Not matched are meta-llama/…, mistralai/…, microsoft/phi-3-mini, BAAI/bge-reranker-v2-m3, llama3:8b and a bare gte. Mutation (removing the reranker exclusion) fails that case.
  • Floor: embedding-model-literal-sites is re-pinned mid-window, 2595 → 2604 (population 2754, window 2454..2754).
  • Results:
    • reach_declarations + reach_scope_claim + both guards: 559 passed.
    • After the reranker change, the embedding guard alone: 28 tests passed.
    • code_intelligence tests for the six modules: 207 passed.
    • Sonnet gate: APPROVE.

Risks

Model Used

Anthropic Claude Opus 5.5 (claude-opus-5-5), coordinating. Anthropic Claude Sonnet 5.5 (claude-sonnet-5-5) subagents for implementation and the gate.

Issue Link

Closes #18345
Closes #18328

Follow-up: #18382, the remaining infra nomic-embed-text literals outside the guard's roots.

Changelog fragment

  • N/A (type-checker configuration and a repo guard; no runtime change apart from the two infra builders reading the configured embedding model)

Summary by CodeRabbit

  • Bug Fixes
    • Knowledge base and documentation indexing now use the active embedding model instead of a fixed default. This keeps indexing aligned with the model configured for the application and avoids inconsistencies when a different supported embedding model is selected.
    • Supported embedding model identifiers are now recognised more consistently, including namespaced identifiers, while chat and malformed identifiers are excluded.

… stay types (#18345)

The backend mypy run is rooted in autobot-backend/, where autobot_shared did not
resolve; ignore_missing_imports made Severity Any and every X = Severity alias a
variable (valid-type). mypy_path = $MYPY_CONFIG_FILE_DIR fixes all six at once.
Pinned by a fixture run with a contrast config lacking the path.
…gte/jina and the infra builders (#18328)

The matcher reads one list, KNOWN_EMBEDDING_MODEL_PREFIXES in autobot_shared/embedding_model.py.
Roots gain populate_kb_chromadb.py and shared/tools/index_documentation.py; both now take
active_embedding_model(). Reach floor re-pinned 2595 -> 2604 (window 2454..2754, population 2754).
@github-actions

Copy link
Copy Markdown
Contributor

Notice: 28 open PRs — past the runaway threshold (25)

This is not a request to defer this PR. The actual limit is the open-PR cap (AUTOBOT_OPEN_PR_CAP, default 40), enforced at pre-push against NEW branches only -- pushing more commits to this PR's own branch is never blocked by it.

This notice only means the count is high enough to be worth a glance for a runaway — something opening PRs in a loop, or a merge pipeline that has stalled so nothing is draining.

Currently open:

If the queue is draining normally, ignore this. Otherwise:

  1. Check whether CI is dispatching at all — see the ci-dispatch-watchdog status on these PRs
  2. Merge the ones whose CI has finished and review has passed: gh pr merge <number> --squash --delete-branch
  3. Look for a loop opening near-identical PRs

Warn-only runaway detector — .github/workflows/pr-queue-gate.yml. It never blocks a merge.

@coderabbitai

coderabbitai Bot commented Oct 12, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

📝 Walkthrough

Walkthrough

The changes update embedding-model ID detection and Ollama model selection in two infrastructure scripts. They also set mypy’s import path relative to its configuration file and add regression tests for enum aliases.

Changes

Embedding model coverage and selection

Layer / File(s) Summary
Model-ID validation and guard coverage
autobot_shared/embedding_model.py, repo_tests/embedding_model_literal_guard_18143_test.py, repo_tests/reach_scope_claim_17844_test.py
The literal guard uses a shared predicate to identify embedding-model IDs. It scans the two infrastructure scripts, tests namespaced IDs and additional model families, and rejects chat, reranker, incomplete and malformed IDs.
Dynamic Ollama model selection
autobot-infrastructure/shared/scripts/populate_kb_chromadb.py, autobot-infrastructure/shared/tools/index_documentation.py
Both scripts use the model name returned by active_embedding_model() when configuring Ollama embeddings. The base URLs remain unchanged.

Mypy enum alias resolution

Layer / File(s) Summary
Configuration and alias regression test
pyproject.toml, repo_tests/mypy_enum_alias_resolution_18345_test.py
The mypy configuration sets mypy_path to $MYPY_CONFIG_FILE_DIR. Regression tests check enum-alias diagnostics when mypy runs from a temporary directory, including a configuration without mypy_path.

Priority: ➖ Normal

Severity of issue fixed: Medium

Merge Risk: 🟡 Moderate · up to 1b6ec

The documentation indexing scripts now follow the configured embedding model but do not check the model used to build their existing collections. Changing the model without rebuilding can break indexing or degrade search results. Resolve this before merging, or confirm that a rebuild is enforced when the model changes. The new mypy test also needs a timeout so a stalled run cannot hang the test suite.

Pre-merge checks | Passed 4 | Failed 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage Warning Docstring coverage is 36.36% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 11 functions across 6 files. (1 skipped: … Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title accurately summarises both main changes: mypy resolution for enum aliases and expanded embedding-model literal guard coverage. It is specific and clear despite being longer than preferred.
Linked Issues check Passed The PR meets the coding requirements for [#18345] and [#18328]. For [#18345], mypy_path = "$MYPY_CONFIG_FILE_DIR" fixes the unresolved autobot_shared import from a backend-rooted run. The regressi…
Out of Scope Changes check Passed The changes stay within the linked issue scope. The mypy configuration and regression test support [#18345]. The shared matcher, guard-root updates, scope record, embedding-model changes in both bui…

Full details: Docstring Coverage

Explanation

Docstring coverage is 36.36% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 11 functions across 6 files. (1 skipped: 1 unsupported.)


  • Fix all pre-merge checks with AI
✨ Finishing Touches 💡 1
📝 Generate docstrings 💡
  • Commit to this branch
  • Create a new PR



🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR

  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@mrveiss

mrveiss commented Oct 12, 2026

Copy link
Copy Markdown
Owner Author

Merge-gate review at 1b6ecea: approve, on top of the author's Sonnet gate. KNOWN_EMBEDDING_MODEL_PREFIXES is one list read by both is_embedding_model_id and the guard, with rerankers excluded. The #18345 fix is the single mypy_path line, pinned by a no-mypy_path contrast test. The floor is re-pinned mid-window. Queued for the next merge train.

@github-actions

Copy link
Copy Markdown
Contributor

✅ SSOT Configuration Compliance: Passing

🎉 No new hardcoded values of either class — ssot and other both block.

Known backlog in pipeline-scripts/hardcoded_values_baseline.txt is suppressed and tracked in #14371.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2


ℹ️ Review info
⚙️ Run configuration
  • Configuration used: Repository: mrveiss/AutoBot-AI/.coderabbit.yaml
  • Review profile: ASSERTIVE
  • Plan: Advanced
  • Run ID: 3deb7662-0a4f-4f6c-95b3-97b0abd8d639
📥 Commits

Reviewing files that changed from the base of the PR and between 13be1b4 and 1b6ecea.

📒 Files selected for processing (7)
  • autobot-infrastructure/shared/scripts/populate_kb_chromadb.py
  • autobot-infrastructure/shared/tools/index_documentation.py
  • autobot_shared/embedding_model.py
  • pyproject.toml
  • repo_tests/embedding_model_literal_guard_18143_test.py
  • repo_tests/mypy_enum_alias_resolution_18345_test.py
  • repo_tests/reach_scope_claim_17844_test.py

Included review availability: This review used your included allowance. Your plan provides up to 1 included review per hour; 0 remain after this review.

llm = LlamaIndexOllamaLLM(model=default_model, base_url=ollama_base_url)

embed_model = OllamaEmbedding(model_name="nomic-embed-text", base_url=ollama_base_url)
embed_model = OllamaEmbedding(model_name=active_embedding_model(), base_url=ollama_base_url)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🗄️ Data Integrity & Integration | 🟠 Major | 🏗️ Heavy lift

Check the embedding model recorded for each persistent collection. If the configured model changes after a collection has been populated, both scripts can write or query vectors using an incompatible embedding space. A dimension mismatch can reject writes; equal dimensions can conceal incorrect search results.

  • autobot-infrastructure/shared/scripts/populate_kb_chromadb.py#L148-L148: check the autobot_docs collection’s recorded model before selecting it for indexing, and rebuild when the model differs.
  • autobot-infrastructure/shared/tools/index_documentation.py#L375-L375: apply the same check to ChromaDBIndexer before upserts or searches.
📍 Affects 2 files
  • autobot-infrastructure/shared/scripts/populate_kb_chromadb.py#L148-L148 (this comment)
  • autobot-infrastructure/shared/tools/index_documentation.py#L375-L375

Comment on lines +45 to +50
cwd=cwd,
capture_output=True,
text=True,
encoding="utf-8",
check=False,
)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🩺 Stability & Availability | 🟠 Major | ⚡ Quick win

Set a timeout for the mypy subprocess.

If mypy stalls, this test waits indefinitely and can block the test run. Set a timeout on subprocess.run and handle subprocess.TimeoutExpired with a useful failure message. Based on learnings, process-spawning calls must have a timeout.

🧰 Tools
🪛 ast-grep (0.45.3)

[error] 35-50: Command coming from incoming request
Context: subprocess.run(
[
sys.executable,
"-m",
"mypy",
f"--config-file={config}",
"--follow-imports=silent",
f"--cache-dir={cwd / 'cache'}",
"alias.py",
],
cwd=cwd,
capture_output=True,
text=True,
encoding="utf-8",
check=False,
)
Note: [CWE-78] Improper Neutralization of Special Elements used in an OS Command ('OS Command Injection').

(subprocess-from-request)

Source: Learnings

This was referenced Oct 12, 2026
@mrveiss
mrveiss merged commit 03c742a into main Oct 12, 2026
86 checks passed
@mrveiss
mrveiss deleted the issue-18345-guard-enum-batch branch October 12, 2026 19:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant