Repository navigation
Conversation
A turn that ends at its Harness's budgetUSD or maxTurns is recorded only in the task's status metadata (stopped_by). Nothing exports it as a metric, so nothing can alert on an Agent whose turns keep ending at a limit: an agent looping, or a limit set too low for its work. The controller now counts completed turns, kagent.agent.turns (kagent_agent_turns_total in Prometheus), labelled with the Agent's name (gen_ai.agent.name) and namespace (k8s.namespace.name) and, for a turn that ended at one of its own limits, kagent.turn.stopped_by (budget or turns). The runtime task store counts a turn when the runtime settles its task as completed: SettleSessionTaskResult returns the task only to the call that published the boundary, so a retried settlement counts once and the count agrees with the stored tasks. Failed, canceled and waiting turns are not counted. An Agent's series start at zero with its first task, so increase() sees the first completed turn. The labels name no Session, task or person. The metric uses the controller's shared OpenTelemetry provider, so it is on the metrics endpoint the chart's ServiceMonitor scrapes and in the configured OTLP export. stopped_by moves to api/a2a, where the controller and the harness runtime both read it. The telemetry contract gains the metric and the attribute; the runtime lifecycle guide documents both. Signed-off-by: Timo Derstappen <teemow@gmail.com>
Signed-off-by: Timo Derstappen <teemow@gmail.com>
5 tasks
…dev#3020 Signed-off-by: Timo Derstappen <teemow@gmail.com>
Member
Author
|
Written by an agent: replaced by #448, the same commits on the current giantswarm head (this line is rebase-merged, so a branch behind its base is replaced, not merged up). |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #443
Problem
A turn that ends at its Harness's
budgetUSDormaxTurnsis recorded only in the task's status metadata (stopped_by). Nothing exports it as a metric, so nothing can alert on an Agent whose turns keep ending at a limit: an agent looping, or a limit set too low for its work.Proposed solution
The controller counts completed turns,
kagent.agent.turns(Prometheuskagent_agent_turns_total), labelled:gen_ai_agent_namek8s_namespace_namekagent_turn_stopped_bybudgetorturnsfor a turn that ended at that limit; absent for a turn that finished on its owncompleted. The newSettleSessionTaskResultreturns the task only to the call that published the boundary, so a retried settlement counts once and the count agrees with the stored tasks. Failed, canceled and waiting turns are not counted.increase()sees the first completed turn.stopped_bymoves togo/api/a2a(StoppedByMetadataKey), read by both the controller and the harness runtime.kagent.turn.stopped_byattribute intelemetry/registry, regenerated;docs/architecture/runtime-and-lifecycle.mddocuments the metric and an alert expression.fork/carried/controller-count-completed-turns-by-the-limit.md; upstream branchupstream/agent-turns-counter, on feat(api): Harness.spec.claude.limits — a budget and a turn cap on every turn of a Claude Code agent kagent-dev/kagent#3020's head.Acceptance criteria
budgetUSDcounts withstopped_by="budget", one atmaxTurnswith"turns", a normal turn without the label (unit testTestSettledTurnsAreCountedByStoppedBy, which also reads the Prometheus exposition).stopped_by, nothing per Session or person.make semconv-checkpasses, the generated contract is current,make -C go lintreports 0 issues, the PostgreSQL store tests pass.