feat(telemetry): emit turn_id and agent_id on turn and tool events#1675
Draft
7Sageer wants to merge 10 commits into
Draft
feat(telemetry): emit turn_id and agent_id on turn and tool events#16757Sageer wants to merge 10 commits into
7Sageer wants to merge 10 commits into
Conversation
The turn lifecycle telemetry events (turn_started, turn_ended, turn_interrupted) never carried the turn id, while tool_call and tool_call_dedup_detected did. Any analysis correlating a turn's start, end, or interruption back to its tool calls had nothing to join on. Add turn_id (already in scope) to all three track() calls, matching the existing key/value convention used by tool_call_dedup_detected. Update the strict turn_started/turn_interrupted assertion to cover it.
…lemetry Port the v1 fix to agent-core-v2: turn lifecycle telemetry events (turn_started, turn_ended, turn_interrupted) carried no turn id while tool_call did, leaving nothing to correlate a turn's start, end, or interruption back to its tool calls. Add turn_id to the three event interfaces, the telemetry registry property docs, and the three track2() calls in AgentLoopService, matching the existing ToolCallEvent key convention. Extend the turn telemetry assertions in loop.test.ts to cover it.
turn_id is a per-agent counter, so it collides across the main agent and subagents within a session. Emit each agent's scope id as agent_id on turn_*, tool_call, tool_call_dedup_detected, api_error and subagent_created (plus parent_agent_id) so events become attributable via (session_id, agent_id, turn_id).
🦋 Changeset detectedLatest commit: ccd67e5 The changes in this PR will be included in the next version bump. This PR includes changesets to release 5 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
- subagent_created: parent_tool_call_id, so a child run joins to the tool call that launched it
- permission_policy_decision / permission_approval_result: agent_id, turn_id, tool_call_id
- plan_submitted / plan_resolved / plan_enter_resolved, context_projection_repaired: agent_id
- compaction_finished / compaction_failed: agent_id + optional turn_id
- cron_scheduled / cron_deleted: optional agent_id of the scheduling agent
- api_error: turn_id + request_kind, so compaction request failures are distinguishable from turn requests
- tool_call_repeat: agent_id + optional turn_id; tool_call_dedup_detected stops fabricating turn_id: 0 outside a turn
- background_task_created/completed: task_id on both, unified kind vocabulary ('process' replaces the legacy 'bash' alias on created)
- agent lifecycle: auto-assigned agent-N ids now skip ids persisted by previous runs, so a resumed session cannot reissue agent-0 and collide with earlier telemetry
…telemetry-turn-id # Conflicts: # packages/agent-core-v2/src/agent/fullCompaction/fullCompactionService.ts # packages/agent-core-v2/src/agent/loop/loopService.ts # packages/agent-core-v2/src/app/telemetry/events.ts # packages/agent-core-v2/src/session/agentLifecycle/agentLifecycleService.ts # packages/agent-core-v2/test/agent/llmRequester/llmRequesterService.test.ts # packages/agent-core-v2/test/agent/loop/loop.test.ts # packages/agent-core-v2/test/agent/toolExecutor/toolExecutor.test.ts # packages/agent-core-v2/test/session/agentLifecycle/agentLifecycle.test.ts
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Related Issue
No linked issue — this is a telemetry data-quality fix; the problem is explained below.
Problem
Turn and tool telemetry events could not be reliably attributed to a specific agent within a session:
turn_idwas missing from most turn/tool events, and where present it is only a per-agent counter — the main agent and every subagent each restart from 0.turn_idcollides across agents and there is no way to tell whether aturn_*/tool_call/api_errorevent came from the main agent or a subagent. Background subagents interleave with the main agent on the timeline, so ordering cannot disambiguate them either.As a result, downstream analysis (per-turn cost, tool error rates, subagent effectiveness, dedup behavior) cannot attribute events to an agent.
What changed
Make turn and tool telemetry attributable via
(session_id, agent_id, turn_id):turn_idonturn_started/turn_interrupted/turn_ended(v1 and v2) and ontool_call(v1), and clarify in the registry thatturn_idis a per-agent index.agent_id(the main agent ismain; subagents use their scope id) onturn_started/turn_interrupted/turn_ended,tool_call,tool_call_dedup_detected,api_error, onsubagent_created(the spawned child'sagent_id, plusparent_agent_idso a spawn event joins to the child's later events), and on the agent-level settings eventsmodel_switch/thinking_toggle/skill_invoked/flow_invoked/yolo_toggle/afk_toggle.turn_idstays per-agent — it is persisted and referenced by the wire protocol, approvals, and the activity kernel — so this adds a dimension instead of renumbering turns globally, keeping the change additive and low-risk.How
agent_idis sourced:agent-core-v2): the Agent-scopedAgentTelemetryContextServiceinjects the existingIAgentScopeContextand seedsagent_idfromscopeContext.agentId. Turn events pick it up through the ambient context;tool_call/tool_call_dedup_detected/api_error, which are emitted through the App-scope telemetry singleton, passagent_idexplicitly fromIAgentScopeContext. The Agent-scoped profile / skill / rpc services do the same for the settings events (model_switch/thinking_toggle/skill_invoked/flow_invoked/yolo_toggle/afk_toggle).agent-core): each agent's telemetry client is wrapped once at construction withwithTelemetryProperties(..., { agent_id: id }), covering every event that agent emits.agent_idis registered on the affected event definitions inevents.ts(with owner/purpose metadata), so it is type-checked and documented like every other telemetry property. v1's wrapper covers every agent-level event and v2 now carriesagent_idon the same event names, so the two engines agree on every event they share.Checklist
gen-changesetsskill, or this PR needs no changeset.gen-docsskill, or this PR needs no doc update.