Switch default LLM provider from Anthropic to Gemini

[manana2520]

Summary

• Default LLM_PROVIDER changed from claude to gemini
• Two new separate model settings: GEMINI_INTAKE_MODEL (Graphiti entity extraction) and GEMINI_CONVERSATION_MODEL (Slack bot RAG), both default to gemini-2.5-flash
• LLM factory no longer silently falls back between providers — if the configured provider is unavailable, it fails with a clear error
• Removed ANTHROPIC_API_KEY injection from all Cloud Run services/jobs (Terraform)
• Removed dead GEMINI_MODEL_ID env var from Terraform and CI
• Cleaned up Anthropic secret IAM bindings from bot/jobs service accounts
• Added GOOGLE_GENAI_USE_VERTEXAI=true to Slack bot services
• Backward compat: existing VERTEX_AI_LLM_MODEL env var maps to new settings via model validator
• All provider code (Claude, Vertex Claude, Ollama) remains in codebase — user can switch providers via .env

Test plan

• 9 new config tests (test_config_gemini.py) — defaults, backward compat, overrides
• Updated LLM factory tests — fail-fast behavior instead of fallback
• 325 unit tests pass, 0 failures
• CI pipeline: security review, builds, staging deploy, E2E tests
• Manual test: DM staging bot after deploy, verify Gemini answers work

[github-actions]

🔒 Security Review

Security Score: A | 2 INFO

Security Review: Switch Default LLM Provider from Anthropic to Gemini

Decision: APPROVE ✓

Summary

This PR successfully switches the default LLM provider from Anthropic Claude to Google Gemini with comprehensive security controls. The migration is well-architected and preserves all existing security patterns.

Key Security Strengths

1. Credential Management (A02, A05)

   • All credentials remain in Secret Manager (Slack tokens, Neo4j password, Confluence API token)
   • ANTHROPIC_API_KEY removed from Cloud Run service/job IAM bindings
   • Configuration uses pydantic-settings with empty string defaults
   • Backward compatibility maintained via migrate_vertex_ai_llm_model() validator
   • No hardcoded credentials introduced

2. LLM Provider Factory (A07, A04)

   • Fail-fast behavior: No silent fallbacks between providers
   • If configured provider is unavailable, raises clear LLMProviderNotConfiguredError
   • Explicit provider validation in factory and Graphiti client
   • Provider availability checked via is_available() method before use

3. Vertex AI Authentication (A02)

   • Service account credentials auto-detected in Cloud Run environment
   • Gemini client properly configured with vertexai=True for Vertex AI mode
   • Custom client object pattern avoids embedding API keys in configuration
   • GCP_PROJECT_ID and VERTEX_AI_LOCATION properly validated

4. Terraform/IaC Security (A01, A05)

   • IAM bindings updated: Removed Anthropic key access, added Vertex AI permissions
   • Neo4j password passed via environment variable (not hardcoded)
   • Secrets properly mounted via Secret Manager
   • VPC connector enforces private routing to Neo4j
   • No public exposure of Bolt port (7687)

5. Configuration Validation (A04, A05)

   • Runtime warning for default ADMIN_PASSWORD in non-debug mode
   • Model validators ensure backward compatibility without ambiguity
   • Environment variable precedence clearly defined (explicit > VERTEX_AI_LLM_MODEL > default)
   • Comprehensive test coverage (9 new config tests validating all scenarios)

6. Logging & Secrets (A09)

   • No credentials logged at any level
   • Factory logs provider names only, not credentials
   • Health checks run in executor to avoid blocking event loops
   • Error handling converts API errors without exposing tokens

Test Coverage

• 325 unit tests pass (0 failures)
• 9 new configuration tests validate:
  • Default values (LLM_PROVIDER = "gemini")
  • Backward compatibility (VERTEX_AI_LLM_MODEL migration)
  • Override precedence (explicit settings override defaults)
  • Provider flexibility (all providers still configurable)
• CI pipeline includes security review gate before deployment

Pre-Scan Findings Resolution

• Line 298 (placeholder comment): Correctly implements Vertex AI authentication pattern; no security issue
• Line 157 (error message): Appropriate fail-fast validation that helps operators without exposing secrets

Deployment Safety

• Staging bot can be tested immediately with Gemini
• Production deployment to Cloud Run provides automatic credential injection
• Fallback to Anthropic/Ollama remains available via LLM_PROVIDER setting
• No breaking changes to existing deployments

Compliance

• ✓ No hardcoded credentials
• ✓ No credential injection in logs
• ✓ SLACK_SIGNING_SECRET verification intact
• ✓ Neo4j Bolt port secured (VPC only)
• ✓ GCS FUSE checkpoint operations preserved
• ✓ SQLite WAL checkpoint handling unchanged
• ✓ Backward compatibility with existing deployments
• ✓ Follows pydantic-settings best practice
• ✓ Accepted ADRs respected (default ADMIN_PASSWORD validation, credential defaults, AI reviewer scripts)

Recommendation

Approve. This PR represents a secure, well-tested migration to Gemini with robust error handling, comprehensive configuration options, and zero credential exposure risks.

Findings

🔵 [INFO] Placeholder comment 'api_key="vertex-ai-mode", # Placeholder - not used with custom client' in Vertex AI Gemini initialization

File: src/knowledge_base/graph/graphiti_client.py:line 298
Category: A05_misconfiguration
Impact: Code clarity only; the placeholder string is intentional and documented. The actual client authentication uses service account credentials injected by the Vertex AI environment.
Recommendation: No action required. This is a documented pattern for Vertex AI authentication via pre-configured client object. The comment explains why the api_key is a placeholder.

🔵 [INFO] Error message 'NEO4J_PASSWORD is required for Neo4j backend' exposes configuration requirement

File: src/knowledge_base/graph/graphiti_client.py:line 157
Category: A05_misconfiguration
Impact: Low impact. This is a fail-fast validation error that helps operators quickly identify missing required credentials. The error does not expose the actual password or sensitive data.
Recommendation: No action required. This is appropriate error handling that aids operational debugging without exposing secrets.

OWASP Top 10 Checklist

Category	Status
A01 Access Control	✅ PASS
A02 Crypto Failures	✅ PASS
A03 Injection	✅ PASS
A04 Insecure Design	✅ PASS
A05 Misconfiguration	✅ PASS
A06 Vulnerable Components	✅ PASS
A07 Auth Failures	✅ PASS
A08 Integrity Failures	✅ PASS
A09 Logging Monitoring	✅ PASS
A10 Ssrf	✅ PASS

---

🤖 Security review powered by Claude

[github-actions]

LGTM! This Pull Request clearly and comprehensively addresses the stated intent of switching the default LLM provider to Gemini.

The changes are well-implemented across the codebase, configuration, and infrastructure:

• Intent Fulfilled: The LLM_PROVIDER default is updated to gemini, new GEMINI_INTAKE_MODEL and GEMINI_CONVERSATION_MODEL settings are introduced, and GOOGLE_GENAI_USE_VERTEXAI=true is correctly applied in Cloud Run services.
• Robustness: The LLM factory's new fail-fast behavior instead of silent fallback is a significant improvement for debugging and operational clarity, and it's well-covered by unit tests.
• Backward Compatibility: The VERTEX_AI_LLM_MODEL migration logic in config.py correctly handles existing deployments while allowing explicit override with the new Gemini-specific variables.
• Security & Clean-up: Crucially, the removal of ANTHROPIC_API_KEY injection and associated IAM bindings from Cloud Run services/jobs in Terraform aligns with the provider switch and enhances security by removing unused access. The removal of the GEMINI_MODEL_ID environment variable is also a good clean-up.
• Architecture Alignment: Configuration is consistently managed via pydantic-settings. The Graphiti client is updated to correctly integrate with the Gemini provider using Vertex AI authentication (service account credentials).
• Test Coverage: The dedicated test_config_gemini.py adds excellent coverage for the new configuration settings, and existing LLM factory tests confirm the fail-fast behavior.
• Terraform Review: The Terraform changes are precise, setting the new Gemini-specific environment variables for the Slack bot and Confluence sync jobs in both staging and production, and correctly managing IAM roles for Vertex AI access.

The overall quality of the changes is high, demonstrating careful consideration for maintainability, clarity, and operational robustness.