Skip to content

GET /v1/agents reports every live agent as status 'unknown' while the column holds 'active' — the filter and the body disagree #312

Description

@khaliqgant

Summary

GET /v1/agents serializes an agent whose stored status is 'active' as "status": "unknown". Every live agent in the workspace appears indeterminate. The filter parameter reads the real column, so the same request filtered and unfiltered disagrees with itself.

Reproduction

Against a live workspace of 864 agents on 2026-08-07:

GET /v1/agents?status=active   -> 329 rows, every row serialized as "status": "unknown"
GET /v1/agents?status=offline  -> 536 rows, serialized as "offline"
GET /v1/agents?status=online   -> 0 rows
GET /v1/agents?status=unknown  -> 0 rows

329 + 536 = 865, the full workspace at the time of the read.

The filter is applied to the stored column before serialization:

// packages/engine/src/engine/agent.ts:109
rows = await db
  .select()
  .from(agents)
  .where(
    and(eq(agents.workspaceId, workspaceId), eq(agents.status, status)),
  );

So ?status=active matching 329 rows proves the column holds 'active', and ?status=unknown matching zero proves nothing holds 'unknown'. Yet those 329 rows serialize as "unknown".

Two agents confirmed in the ?status=active set were exchanging messages at the moment of the read, so this is not a staleness artifact.

Where it is not

Nothing in the engine writes 'unknown' to agents.status — the only writers are 'active' (registerAgent :59, touchLastSeen :294) and 'offline' (sweepStaleAgents :303). listAgents returns status: a.status unmapped (:129). No mapping was found in packages/sdk-typescript either.

The divergence is therefore introduced somewhere in the deployed build that this checkout does not reflect. Worth confirming which build is deployed as part of triage — the gap between this tree and the running engine is itself useful to know.

One plausible reading, offered as a hypothesis rather than a finding: the response may be reporting presence (derived, "unknown" when no live presence signal is subscribed) using the same field name as the stored registration/staleness status. The event types in packages/sdk-typescript/src/types.ts:561-567 describe a richer presence model (active / idle / waiting / blocked / offline) than the two values the column holds, which would be consistent with two different concepts sharing one name.

Why this is worth more than a cosmetic fix

The stored value is load-bearing for an identity decision. registerAgentViaNode guards its tokenHash overwrite with:

setWhere: or(
  ne(agents.status, 'active'),
  and(eq(agents.locationType, 'via_node'), or(...)),
),

An engineer reasoning about that guard who checks the field through the API will conclude that no agent is ever 'active', that the first disjunct is always true, and therefore that the guard is unconditional and the identity boundary on the fleet registration path is wide open. That conclusion is false — the column does hold 'active' and the guard works — but it is the conclusion the API hands you, and it was very nearly escalated as a live security incident on 2026-08-07 before the column was probed directly.

A field that reads one way to SQL and another way to every API consumer is a trap for exactly the people trying to verify a security property.

Suggested direction

Either is fine; the current state is not:

  • serialize the stored value faithfully, or
  • keep presence in the response but give it a different name, and expose the stored value under its own key

If the two-concepts reading is right, the naming is the whole fix.

Related

Context

Filed from an incident investigation on 2026-08-07. Khaliq owns the merge gate — no agent merges.

Activity

  1. khaliqgant commented on Aug 7, 2026

    @khaliqgant
    MemberAuthor

    Additional context: the status column is not maintained at all, which compounds this.

    This issue reports that stored 'active' serializes as "unknown". Separately, nothing ever clears 'active': sweepStaleAgents (packages/engine/src/engine/agent.ts:298) has no caller — its only other references are .d.ts declarations under dist/ — because its driving loop was removed in the Fly.io → Cloudflare Workers migration. #306 tracks restoring it.

    Measured on the live workspace: of 329 records stored 'active', 305 have lastSeen older than five minutes, and the oldest has been 'active' for 23.9 days.

    So the field is wrong in two independent ways at once. An operator reading agent list sees "unknown" for every live agent; an engineer reading the column sees 'active' for agents that have been gone for weeks. Neither view answers "is this agent alive", and lastSeen is currently the only field that does.

    Whatever is done here should be decided alongside #306, since fixing the serialization without restoring the sweep would replace one misleading answer with a differently misleading one.

  2. khaliqgant commented on Aug 7, 2026

    @khaliqgant
    MemberAuthor

    Provenance note. Source references in this issue were read from main at 08ddec7. The deployed engine is not this checkout — it demonstrably carries code no local tree has: stored status = 'active' serializes as "unknown" through a mapping that exists nowhere in this source (#312). Treat every file:line here as provisional until checked against the deployed build; if you cannot find a reference, suspect a build difference before assuming the reference is wrong.

    Measurements taken against the live workspace — record shapes, status counts, staleness ages, and CLI probe results — do not depend on this and stand on their own.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions