Skip to content

feat(controller): count completed turns by the limit that ended them - #445

Closed
teemow wants to merge 3 commits into
giantswarmfrom
ku-443-turns-counter
Closed

teemow wants to merge 3 commits into
giantswarmfrom
ku-443-turns-counter

Conversation

@teemow

@teemow teemow commented Oct 12, 2026

Copy link
Copy Markdown
Member

Closes #443

Problem

A turn that ends at its Harness's budgetUSD or maxTurns is recorded only in the task's status metadata (stopped_by). Nothing exports it as a metric, so nothing can alert on an Agent whose turns keep ending at a limit: an agent looping, or a limit set too low for its work.

Proposed solution

The controller counts completed turns, kagent.agent.turns (Prometheus kagent_agent_turns_total), labelled:

Label (Prometheus) Value
gen_ai_agent_name the Agent's name
k8s_namespace_name the Agent's namespace
kagent_turn_stopped_by budget or turns for a turn that ended at that limit; absent for a turn that finished on its own
  • Counted in the runtime task store when the runtime settles the task as completed. The new SettleSessionTaskResult returns the task only to the call that published the boundary, so a retried settlement counts once and the count agrees with the stored tasks. Failed, canceled and waiting turns are not counted.
  • An Agent's three series are created at zero with its first task, so increase() sees the first completed turn.
  • Uses the controller's shared OpenTelemetry provider: on the metrics endpoint the chart's ServiceMonitor scrapes, and in the OTLP export.
  • stopped_by moves to go/api/a2a (StoppedByMetadataKey), read by both the controller and the harness runtime.
  • Telemetry contract: the metric and the kagent.turn.stopped_by attribute in telemetry/registry, regenerated; docs/architecture/runtime-and-lifecycle.md documents the metric and an alert expression.
  • Carried row fork/carried/controller-count-completed-turns-by-the-limit.md; upstream branch upstream/agent-turns-counter, on feat(api): Harness.spec.claude.limits — a budget and a turn cap on every turn of a Claude Code agent kagent-dev/kagent#3020's head.

Acceptance criteria

  • A turn ending at budgetUSD counts with stopped_by="budget", one at maxTurns with "turns", a normal turn without the label (unit test TestSettledTurnsAreCountedByStoppedBy, which also reads the Prometheus exposition).
  • A retried settlement counts once; failed and waiting turns are not counted.
  • Labels are the Agent's name and namespace and stopped_by, nothing per Session or person.
  • make semconv-check passes, the generated contract is current, make -C go lint reports 0 issues, the PostgreSQL store tests pass.

A turn that ends at its Harness's budgetUSD or maxTurns is recorded only
in the task's status metadata (stopped_by). Nothing exports it as a
metric, so nothing can alert on an Agent whose turns keep ending at a
limit: an agent looping, or a limit set too low for its work.

The controller now counts completed turns, kagent.agent.turns
(kagent_agent_turns_total in Prometheus), labelled with the Agent's name
(gen_ai.agent.name) and namespace (k8s.namespace.name) and, for a turn
that ended at one of its own limits, kagent.turn.stopped_by (budget or
turns). The runtime task store counts a turn when the runtime settles
its task as completed: SettleSessionTaskResult returns the task only to
the call that published the boundary, so a retried settlement counts
once and the count agrees with the stored tasks. Failed, canceled and
waiting turns are not counted. An Agent's series start at zero with its
first task, so increase() sees the first completed turn. The labels name
no Session, task or person.

The metric uses the controller's shared OpenTelemetry provider, so it is
on the metrics endpoint the chart's ServiceMonitor scrapes and in the
configured OTLP export. stopped_by moves to api/a2a, where the
controller and the harness runtime both read it. The telemetry contract
gains the metric and the attribute; the runtime lifecycle guide
documents both.

Signed-off-by: Timo Derstappen <teemow@gmail.com>
Signed-off-by: Timo Derstappen <teemow@gmail.com>
@teemow

teemow commented Oct 12, 2026

Copy link
Copy Markdown
Member Author

Written by an agent: replaced by #448, the same commits on the current giantswarm head (this line is rebase-merged, so a branch behind its base is replaced, not merged up).

@teemow teemow closed this Oct 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

harness: export a counter of completed turns by the limit that ended them

1 participant