Skip to content

LLC adapters: no read-only sign-in probe before dispatch — signed-out agents burn scheduler slots #13098

Open

Description

@mrveiss

Problem

Nothing checks whether an agent's CLI is signed in before the scheduler
dispatches a run to it. A signed-out agent is dispatched anyway, spawns, fails,
and is diagnosed only after the fact by keyword-scanning its output tail. The
next heartbeat re-dispatches it and the cycle repeats.

The result is a silent, self-repeating burn of scheduler slots and agent budget,
surfaced to the operator only as generic run failures with no actionable cause.

Evidence

The only pre-dispatch auth check in the entire adapter layer:

  • autobot-backend/llc/adapters/copilot_subscription_adapter.py:91-107 — and it
    works by spawning gh auth status and awaiting it, i.e. a process spawn on
    every single invoke.

No check at all in:

  • llc/adapters/claude_code_adapter.py
  • llc/adapters/claude_code_subscription_adapter.py
  • llc/adapters/codex_subscription_adapter.py
  • llc/adapters/copilot_local_adapter.py

Nothing gates dispatch in the scheduler:

$ grep -rn --include=*.py "should_skip\|can_dispatch\|precheck\|_eligible" autobot-backend/llc/scheduler/
(no output)

Failure is instead classified post-hoc by is_rate_limit_output()
(llc/adapters/subprocess_support.py:56-70), a keyword match over error text.

Proposed approach

Add a declarative, read-only, zero-spawn credential probe per adapter —
the counterpart to the existing resolve_cli_binary() 3-tier search
(llc/adapters/subprocess_base.py:66-105), which answers "is it installed".
This answers "is it usable".

Probe kinds, as data rather than per-adapter code:

Kind Check
file path exists
json file parses and a named key / non-empty object is present
fileMatch any line matches a regex
keychain OS keychain entry exists — exit code only, never read the secret
anyOf first matching sub-probe wins (per-OS variants)

Wire it into the heartbeat scheduler as a pre-dispatch gate.

Hard constraint — fail OPEN, not closed

A failed or inconclusive probe must warn and dispatch anyway. It must never
block a run.

These probes read third-party CLIs' private on-disk credential formats, which
change without notice. A probe that wrongly reports "signed out" and blocks
dispatch is strictly worse than having no probe at all, because the scheduler is
unattended — there is no human present to notice and override.

Acceptance criteria

  • Declarative probe definition per adapter (data, not bespoke code per agent)
  • Probe performs no process spawn and never reads secret material
  • Scheduler logs an actionable reason (e.g. "agent CLI not signed in — run
    <login command>") and records it on the work item
  • Probe failure never blocks dispatch (explicit test)
  • copilot_subscription_adapter's spawn-based check is replaced by the
    read-only probe
  • Tests cover: signed-in, signed-out, and probe-error (inconclusive) paths

Notes

Found during a capability audit of the LLC adapter layer.

Activity

  1. mrveiss commented on Jul 31, 2026

    @mrveiss
    OwnerAuthor

    Parent: #13096 (capability audit umbrella).

  2. mrveiss commented on Jul 31, 2026

    @mrveiss
    OwnerAuthor

    Consolidation note — reuse the existing SKIPPED semantics.

    LLCRunStatus.SKIPPED already means "heartbeat was not dispatched to any adapter (no adapter / CLI absent / no agent_class) — degraded, distinct from COMPLETED or FAILED" (GH#9951, llc/models/enums.py:165-167).

    A signed-out agent is the same shape of outcome: not dispatched, degraded, operator action needed. The probe should very likely resolve to SKIPPED with a specific reason rather than a new status value.

    This also reinforces the fail-open constraint already on this issue — SKIPPED is a reporting outcome, not a block, which is exactly the desired behaviour.

  3. modified the milestones: Backlog, v0.13.0 on Sep 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions