Repository navigation
setup-wizard: LLM-provider/adapter selection doesn't install required software or provision keys — claude_code agents skip with 'claude CLI not found' on a stock install (generalises to Ollama model pull, API keys) #12681
Description
Activity
Assessment — the title is a misnomer, and half of this is already done
Scoped before implementing. Correcting the framing so the remaining work is
visible.No wizard selects an LLM provider
- SLM wizard steps (
api/setup_wizard.py:44-54): welcome → add_nodes →
test_connections → enroll_agents → assign_roles → configure_secrets →
provision_fleet → verify_health.configure_secretscollects exactly one
value:hf_token. OnboardingWizard.vue: doctor → preset → apply. It recommends an
llm_tier; it installs nothing.- The adapter is chosen at hire time —
HireAgentModal.vue→
AdapterTypeSelect.vue→GET /api/llc/adapters.
So this is not a gap in provider selection; it is a gap between hiring an agent
and that agent being able to run.Already solved — recommendations 3 and 4
llc/api/adapters.py:21-50computesavailableby calling
adapter.is_cli_available()inside the backend process — i.e. as the
autobotservice account, which is exactly the check that was missing — via
resolve_cli_binary(configured override → PATH →~/.local/bin,
/usr/local/bin, npm global bin).AdapterTypeSelect.vuedisables unavailable
adapters and labels them "(CLI not installed)".Ollama also already has both a runtime pull API
(api/adapters.py:108-137, streaming) and an Ansible pull loop
(roles/llm/tasks/main.yml:162-171).What is actually left
Independent of secrets entirely:
- Nothing installs the
claudeCLI onto the service PATH. The only
@anthropic-ai/claude-codeinstall in the tree is in
.github/workflows/ai-security-review.yml— CI only. This is the literal
thing the reporter had to do by hand. - Hire accepts an unavailable adapter.
llc/api/agent_hires.py:365-372
validates only that the type is registered, never that it is available.
The UI greys it out; the API does not — this is exactly how a hired agent
ends up unable to run. - The skip reason is one field from visible. It is persisted
(heartbeat_scheduler.py:522-547writesstatus=SKIPPED, error=exc.reason)
and returned (llc/api/runs.py:47), butHeartbeatMonitor.vuenever renders
it. Cheapest fix here by a wide margin. - Ollama's pull list is disconnected from any UI selection — it comes from
role defaults.
Depends on secrets, but on much less than #12684:
-
No provider credential ever reaches an adapter subprocess.
subprocess_support.py:265-278injects onlyAUTOBOT_LLC_API_KEYand
AUTOBOT_LLC_API_BASE. A key entered in any of the six secrets UIs never
reaches an agent run — it works only if already in the backend service's own
environment.This does not need store consolidation:
provider_key_vault.resolve_provider_key
already does env-first/vault-fallback and is already used by the in-process
LLM path. The fix is calling that same resolver at the adapter env-injection
seam. It does need AUTOBOT_SECRETS_ROOT_KEY is generated by nothing — the canonical secret store is dark on every provisioned deployment #14758 (the root key), or the vault half is
permanently empty.
Sequencing that matters
Do not land item 2 before item 1. Today you can hire and hand-install the
CLI. Rejecting the hire first makes deployments strictly worse — you cannot
hire at all until someone installs it. Land the install, or ship the rejection
with an explicit override.Decide the claude auth model before item 5. The
claudeCLI may
authenticate by subscription login rather than API key; injecting
ANTHROPIC_API_KEYchanges billing and behaviour. That is a decision, not an
implementation detail.Item 1 carries supply-chain surface — pulling an npm global package onto
nodes at provision time. Pin the version, gate on the adapter being wanted, and
report rather than fail when offline.Also worth splitting out
The "secondary observation" in this issue (
output_text: Noneon a completed
run,replay_service.record_runerroring per run) is a distinct evidence-loss
defect. Part of it is already fixed — see #13614 / PR #14755, where the
transcript path was resolved from a placeholder run id that never matched.- SLM wizard steps (
Item 3 delivered — #14761 merged. Issue stays open.
Verified on
origin/Dev_new_gui, not from the PR branch:Check Result AgentRun.errordeclaredpresent Reason row rendered ( .run-reason)present .status-skippedstylingpresent HeartbeatMonitor.skipReason.test.tspresent Locales carrying both new keys 11/11 frontend-testspassed on the merged head — the component tests are verified by
CI, since I could not run the frontend suite locally.What was actually wrong
Nothing was missing from the pipeline.
HeartbeatDispatchSkippedis already
handled deliberately — the scheduler recordsSKIPPEDand storesexc.reason—
and_run_to_dictalready returned it. The reason reached the client and was then
dropped one line short of the screen, because the component'sAgentRuninterface
never declared the field.So every skip looked identical: a missing adapter binary, an active cooldown, and
an agent with no work were indistinguishable without querying the database. That is
the "fails invisibly" symptom this issue reports.SKIPPEDalso had no status colour defined at all, whilequeued,running,
failed,timed_outandunknownall did — the one status whose entire meaning is
"this did not run" rendered as unstyled text.A review pass caught one more thing worth recording: in Persian I had used
رد شد,
which is the phrasing this codebase already uses for denied
(common.permissionDenied.title= "دسترسی رد شد"). A Persian operator would have
read a skipped run as a rejected one — the exact confusion this change exists to
remove, defeating its own purpose in one locale. Corrected before merge.Still open — items 1 and 2
- Nothing installs the
claudeCLI onto the service PATH. llc/api/agent_hires.py:365-372accepts an unavailable adapter — it validates
only that the type is registered, never that it is available, which is how a
hired agent ends up unable to run.
The sequencing recorded earlier still holds: do not land item 2 before item 1.
Today an operator can hire and hand-install the CLI afterwards. Rejecting the hire
first, while nothing installs the binary, makes provisioning strictly worse.Item 3 was independent of that ordering, which is why it went first.
Related and now in flight: #14784 provisions
AUTOBOT_SECRETS_ROOT_KEY, without
which the canonical secret store is unreachable — the same silent-degradation
pattern, and a prerequisite for provider keys having one place to live.- Nothing installs the
- added 6 commits that reference this issue
on Aug 22, 2026 All three items delivered
item PR what it was 1 — nothing installs the agent CLIs #14790 ansible installed neither claudenorgh; searching the whole ansible tree for "claude" returned zero files2 — hire accepts an unavailable adapter #14797 the hire path never consulted the availability probe that already existed 3 — the skip reason is never rendered #14761 the reason was persisted and served, and died one line short of the screen Verified on
origin/Dev_new_gui, not from the PR branches:- ansible installs
claude-codeand proves it resolves on the host _adapter_unavailable_reasonis wired intohire_agent, including the
unimplemented-adapter branch- the heartbeat monitor renders the reason, with keys in all 11 locales
The sequencing mattered and was kept
Item 3 first, because it makes the failure legible while it still exists. Then item
1, because it installs the binaries. Then item 2 — rejecting a hire while nothing
installed the CLI would have made provisioning strictly worse, since an operator
could previously hire and hand-install afterwards.What each turned out to be
None of the three was a missing feature. Every one was something already built that
did not reach the next hop:- the CLI install existed in the container image and had no ansible equivalent
- the availability probe existed and powered
GET /adapters, and the hire path
never asked it - the skip reason existed in the database and in the API response, and the
component's interface never declared the field
That is the same defect shape three times, which is why the issue read as
"provisioning fails invisibly" rather than as three separate bugs.Not closed here
#14800 —
PortabilityService._import_agentcreatesagent_org_nodesrows with
anadapter_typefrom an imported template, with neither the registration nor the
availability check. So the item-2 gate currently covers one of the two paths
that create runnable agents. Pre-existing, and worth saying plainly rather than
letting the gate look complete.Closing this issue; #14800 tracks the remaining path.
- ansible installs
- added a commit that references this issue
on Aug 22, 2026
Summary
Selecting an agent LLM provider / execution adapter (e.g.
claude_code) in the Setup Wizard does not install the software that adapter requires or provision its credentials. The choice is recorded, but the runtime dependency is left unmet — so an agent configured with that provider silently fails to run. This is a setup-wizard / provider-selection gap, upstream of the adapter's own error handling (#12478).How it was found (concrete, live)
Exercising the LLC "run a hired agent" path (ABR marketing-agent dogfooding) on a stock deployment:
claude_code/ modelclaude-sonnet-4-6— all succeed.run_id(202) but the run is skipped:claudeCLI the adapter needs was never installed anywhere the service account (autobot) can execute. On this box it existed only in a human user's home (~/.local/bin, mode 0750 — unreadable byautobot).claudebinary to/usr/local/bin(on the service PATH) immediately fixed resolution — the next heartbeat ran with zero "not found" skips and reachedcompleted.So the only thing between "adapter selected" and "agent runs" was a piece of provisioning the wizard never performed.
The gap generalises beyond
claudeThis is not a claude-specific bug — it is a missing responsibility in the provider-selection step. Every provider choice has an unmet-dependency failure mode the wizard should close:
claude_codeclaudeCLI, on the service PATH (not a user home)ANTHROPIC_API_KEYToday the wizard lets you pick any of these and reports success, while the actual precondition (binary installed for the service user, model pulled, key present) is silently absent. The failure only surfaces later, at first agent run, as an opaque skip.
Why this matters
claudeinstalled (in their own home) hits this, because agents execute as theautobotsystemd service, not as the logged-in user. The wizard is the right place to encode "install it where the service can reach it."Recommendation
Make provider selection own its preconditions. In the wizard's LLM-provider step, for the chosen provider(s):
claude→/usr/local/bin, on the service PATH;ollama pull <model>for local models), or clearly report what the operator must install and verify it before completing the step.sudo -u autobot claude --version,sudo -u autobot ollama list— so a binary in a human home (0750) is correctly detected as not usable.install.shprovisioning cover the same ground for headless installs, so wizard and installer agree.This closes the gap that #12478 could only paper over: #12478 gave the adapter a good error message and an
AUTOBOT_CLAUDE_CLI_PATHescape hatch, but nothing installsclaudeor points that variable at a service-reachable copy. That is the wizard/provisioning step's job.Secondary observation (separate, lower confidence)
Once the CLI resolved, the run reached
final_status: completedbut withoutput_text: None, andllc.services.replay_service.record_runlogged an error ("record_run failed … skipping") for each run. Worth a look on its own: an agent run completing without captured output, and run-recording erroring, may be a distinct defect from this provisioning gap. Not filed pending confirmation.Environment
install.shdeploy at/opt/autobot; agents execute as theautobotsystemd service (PATH…/venv/bin:/usr/local/bin:/usr/bin:/bin). Adapterclaude_code,claudeCLI 2.1.219. Reproduced live; manual/usr/local/bininstall confirmed the fix. Related: #12478 (adapter message/escape-hatch), #12210 (trigger), #12309 (transitions).