Repository navigation
bug(ansible): autobot-ai-stack crash-loops — systemd PYTHONPATH omits /opt/autobot, so 'No module named autobot_shared' #10121
Description
Activity
Verified the fix is sufficient (live install)
Simulated the fix by adding
/opt/autobotto the ai-stack PYTHONPATH and importing the actual uvicorn entrypoint as theautobotuser against the deployed code:```
PYTHONPATH=/opt/autobot:/opt/autobot/autobot-ai-stack:/opt/autobot/autobot-ai-stack/src:/opt/autobot/autobot-backend
.../autobot-backend/venv/bin/python -c "import ai_api_server"
→ IMPORT OK — app = FastAPI (LocalLLM/Ollama + agent router initialize cleanly)
```So the missing
/opt/autoboton PYTHONPATH is the only blocker — there is no second-order import/init failure behind it. Adding the standalone shared path to the unit template (or installingautobot_sharedinto the venv) clears the crash loop and the AI Stack starts. Note: takes effect on a node only after redeploy/reinstall (the co-located self-node redeploy path is itself blocked by #10095/#10109).Fix implemented + verified — PR held on queue limit
Committed on branch `issue-10121` (`a4861a82c`): in `roles/ai-stack/templates/autobot-ai-stack.service.j2`, appended the standalone shared path to PYTHONPATH, mirroring the backend unit (#10020):
```
PYTHONPATH={{ ai_install_dir }}:{{ ai_install_dir }}/src:{{ backend_install_dir }}:{{ ai_install_dir | dirname }}/autobot_shared:{{ ai_install_dir | dirname }}
```
renders to…:/opt/autobot/autobot_shared:/opt/autobot. Verified: template renders cleanly, and a clean-env import with that exact PYTHONPATH loadsai_api_server(FastAPI app + LLM/agent init) with no error.PR not yet opened — the Dev_new_gui PR queue is at the project limit (5 open). Will push + open the PR as soon as a slot frees (or on request). Branch is committed so the work is safe.
- added a commit that references this issue
on Jun 14, 2026 Closed — merged to Dev_new_gui via PR #10130. Commit:
99c06f884. ai-stack unitPYTHONPATHnow includes/opt/autobot/autobot_shared:/opt/autobot(mirrors the backend unit, #10020), resolving theModuleNotFoundError: autobot_sharedcrash loop (NRestarts 2968). Verified the entrypoint imports cleanly with this path; smoke-test green. Takes effect on redeploy/reinstall (self-node redeploy path tracked separately in #10095/#10109). Part of #9919.
Summary
autobot-ai-stack.service(the AI Processing Stack / agent router on :8080) crash-loops indefinitely — on the live installNRestarts=2968,status=1/FAILUREevery ~5–10s since the 06-11 install. This is the real cause of the Code Sync / dashboard panel showing "AI Stack (ChromaDB): Connection refused" and "chromadb: Connection refused" (the panel probes the down :8080 server). ChromaDB itself (:8100) is healthy.Root cause
The unit imports
autobot_shared:Environment="PYTHONPATH=/opt/autobot/autobot-ai-stack:/opt/autobot/autobot-ai-stack/src:/opt/autobot/autobot-backend"— it omits/opt/autobot, where the standaloneautobot_sharedpackage lives.autobot_sharedis not installed into the backend venv's site-packages either:env -i .../venv/bin/python -c "import autobot_shared"→ ModuleNotFoundError (it only resolves when/opt/autobothappens to be on PYTHONPATH).autobot-backend.serviceunit works only because its PYTHONPATH explicitly includes anautobot_sharedpath; the ai-stack unit template was never given the same.Impact
AI Stack server permanently down → agent-router / AI-stack-backed features unavailable; dashboard shows ChromaDB-related services as "Connection refused" even though ChromaDB is fine; ~3000 restarts burning CPU.
Fix direction (repo)
In the ai-stack role's systemd unit template, add the standalone shared path to
PYTHONPATH(…:/opt/autobotor the explicit/opt/autobot/autobot_sharedparent) — mirroring the backend unit — OR installautobot_sharedinto the shared venv's site-packages so PYTHONPATH isn't load-bearing. Reconcile with #10020 (single authoritativeautobot_shared).Repro
Co-located install →
systemctl status autobot-ai-stackshows auto-restart loop;sudo -u autobot env -i PYTHONPATH=<unit value> .../venv/bin/python -c "import autobot_shared"→ ModuleNotFoundError.Part of #9919 (deployment-install); related to #10020.