Skip to content

Conversation

@marklysze
Copy link
Collaborator

@marklysze marklysze commented Sep 8, 2025

Why are these changes needed?

Events are not emitted for handoffs and these are useful to know so you can monitor your orchestrations and diagnose flows.

Handoff events so far for AfterWorks, OnContextCondition, LLM-based OnCondition, and ReplyResult transition.

Events are simple one-liners, e.g.:
***** AfterWork handoff (agent_b): agent_c *****
***** OnContextCondition handoff (coordinator_agent): weather_specialist *****
***** LLM-based OnCondition handoff (coordinator_agent): traffic_specialist *****

Related issue number

N/A

Checks

@joggrbot

This comment has been minimized.

@marklysze marklysze added the enhancement New feature or request label Sep 8, 2025
@marklysze marklysze self-assigned this Sep 8, 2025
Copy link
Collaborator

@priyansh4320 priyansh4320 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should ***** StringLLM OnCondition handoff (coordinator_agent): traffic_specialist *****
be a more suitable indicator for this particular handoff event message ,
" LLM-based OnCondition handoff (coordinator_agent): traffic_specialist "

I have 1 Question:
Since, these are triggered events , should these event also be update the chat.completion response json.
this will also allow to have a robust key : value trace , along with a message trace.

@marklysze
Copy link
Collaborator Author

should ***** StringLLM OnCondition handoff (coordinator_agent): traffic_specialist ***** be a more suitable indicator for this particular handoff event message , " LLM-based OnCondition handoff (coordinator_agent): traffic_specialist "

Thanks @priyansh4320 - I'm afraid we don't know if it has been derived from a StringLLMCondition or ContextStrLLMCondition, unless we look to add this metadata somewhere. Ideally it would be good to have some kind of identifier that could trace back to the actual handoff - this will be useful for a UI to be able to depict it in action. We can consider that for a separate, broader, PR.

I have 1 Question: Since, these are triggered events , should these event also be update the chat.completion response json. this will also allow to have a robust key : value trace , along with a message trace.

Thanks, can you expand on this a bit further with an example of what it may look like. My initial thoughts are that the events system is separate to the ChatCompletions (e.g. there are a lot of events that aren't part of ChatCompletions).

@priyansh4320
Copy link
Collaborator

priyansh4320 commented Sep 8, 2025

should ***** StringLLM OnCondition handoff (coordinator_agent): traffic_specialist ***** be a more suitable indicator for this particular handoff event message , " LLM-based OnCondition handoff (coordinator_agent): traffic_specialist "

Thanks @priyansh4320 - I'm afraid we don't know if it has been derived from a StringLLMCondition or ContextStrLLMCondition, unless we look to add this metadata somewhere. Ideally it would be good to have some kind of identifier that could trace back to the actual handoff - this will be useful for a UI to be able to depict it in action. We can consider that for a separate, broader, PR.

I have 1 Question: Since, these are triggered events , should these event also be update the chat.completion response json. this will also allow to have a robust key : value trace , along with a message trace.

Thanks, can you expand on this a bit further with an example of what it may look like. My initial thoughts are that the events system is separate to the ChatCompletions (e.g. there are a lot of events that aren't part of ChatCompletions).

@marklysze , can we use chat.history meta to add events, here is an example,
So, here I have added some events ,
message_sent , tool_invoked, tool_completed, as an example, similarly Handoff events can be updated in some places,

[
  {
    "timestamp": "2025-09-09T09:00:00Z",
    "event": "message_sent",
    "role": "assistant",
    "agent": "assistant",
    "message": "Today, let's introduce our kids to the solar system."
  },
  {
    "timestamp": "2025-09-09T09:00:05Z",
    "event": "message_sent",
    "role": "user",
    "agent": "teacher_agent",
    "message": "Draft lesson plan: Intro to Solar System (Grades 3–5, 45-60 mins). Covers planets, facts, model or drawing activity, and a short quiz. Includes video, Q&A, and homework on favorite planet."
  },
  {
    "timestamp": "2025-09-09T09:00:07Z",
    "event": "tool_invoked",
    "role": "assistant",
    "agent": "reviewer_agent",
    "tool": "do_review",
    "inputs": {
      "lesson_plan": "Solar system overview, activities, video, review. Includes hands-on and follow-up."
    },
    "tool_call_id": "call_gVKeTHUXzK3wuG7lW1RoZFUv"
  },
  {
    "timestamp": "2025-09-09T09:00:10Z",
    "event": "tool_completed",
    "role": "tool",
    "agent": "_Swarm_Tool_Executor",
    "tool_call_id": "call_gVKeTHUXzK3wuG7lW1RoZFUv",
    "output": "Review complete"
  },
  {
    "timestamp": "2025-09-09T09:00:12Z",
    "event": "message_sent",
    "role": "assistant",
    "agent": "reviewer_agent",
    "message": "Review complete. Suggestions: align to 4th grade standards, add differentiated tasks, and integrate writing for cross-discipline support."
  }
]

@elCaptnCode
Copy link
Collaborator

elCaptnCode commented Sep 26, 2025

im dealing with this as we speak. i would love some sort of event that could use that can tell me if my handoff is working properly

@marklysze
Copy link
Collaborator Author

should ***** StringLLM OnCondition handoff (coordinator_agent): traffic_specialist ***** be a more suitable indicator for this particular handoff event message , " LLM-based OnCondition handoff (coordinator_agent): traffic_specialist "

Thanks @priyansh4320 - I'm afraid we don't know if it has been derived from a StringLLMCondition or ContextStrLLMCondition, unless we look to add this metadata somewhere. Ideally it would be good to have some kind of identifier that could trace back to the actual handoff - this will be useful for a UI to be able to depict it in action. We can consider that for a separate, broader, PR.

I have 1 Question: Since, these are triggered events , should these event also be update the chat.completion response json. this will also allow to have a robust key : value trace , along with a message trace.

Thanks, can you expand on this a bit further with an example of what it may look like. My initial thoughts are that the events system is separate to the ChatCompletions (e.g. there are a lot of events that aren't part of ChatCompletions).

@marklysze , can we use chat.history meta to add events, here is an example, So, here I have added some events , message_sent , tool_invoked, tool_completed, as an example, similarly Handoff events can be updated in some places,

[
  {
    "timestamp": "2025-09-09T09:00:00Z",
    "event": "message_sent",
    "role": "assistant",
    "agent": "assistant",
    "message": "Today, let's introduce our kids to the solar system."
  },
  {
    "timestamp": "2025-09-09T09:00:05Z",
    "event": "message_sent",
    "role": "user",
    "agent": "teacher_agent",
    "message": "Draft lesson plan: Intro to Solar System (Grades 3–5, 45-60 mins). Covers planets, facts, model or drawing activity, and a short quiz. Includes video, Q&A, and homework on favorite planet."
  },
  {
    "timestamp": "2025-09-09T09:00:07Z",
    "event": "tool_invoked",
    "role": "assistant",
    "agent": "reviewer_agent",
    "tool": "do_review",
    "inputs": {
      "lesson_plan": "Solar system overview, activities, video, review. Includes hands-on and follow-up."
    },
    "tool_call_id": "call_gVKeTHUXzK3wuG7lW1RoZFUv"
  },
  {
    "timestamp": "2025-09-09T09:00:10Z",
    "event": "tool_completed",
    "role": "tool",
    "agent": "_Swarm_Tool_Executor",
    "tool_call_id": "call_gVKeTHUXzK3wuG7lW1RoZFUv",
    "output": "Review complete"
  },
  {
    "timestamp": "2025-09-09T09:00:12Z",
    "event": "message_sent",
    "role": "assistant",
    "agent": "reviewer_agent",
    "message": "Review complete. Suggestions: align to 4th grade standards, add differentiated tasks, and integrate writing for cross-discipline support."
  }
]

@priyansh4320 thanks for noting these, the events system does include some of these events, if you'd like to check out agent events and see if there's anything missing we can add them in another PR.

@codecov
Copy link

codecov bot commented Oct 17, 2025

Codecov Report

❌ Patch coverage is 96.11650% with 4 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
autogen/agentchat/group/group_tool_executor.py 89.74% 3 Missing and 1 partial ⚠️
Files with missing lines Coverage Δ
autogen/agentchat/group/events/handoff_events.py 100.00% <100.00%> (ø)
...ogen/agentchat/group/events/reply_result_events.py 100.00% <100.00%> (ø)
autogen/agentchat/group/group_utils.py 76.54% <100.00%> (+0.74%) ⬆️
autogen/agentchat/group/group_tool_executor.py 88.88% <89.74%> (+0.34%) ⬆️

... and 41 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants