feat: Step through execution with runs

[marklysze]

Why are these changes needed?

Currently, AG2's run() and run_group_chat() methods execute the workflow in background threads, meaning that they run independently of when events are consumed. This is somewhat deceptive, as it feels like you're stepping through the orchestration but it is running separately and could be completed by the time you even move to the second event.

I'm introducing the ability to step through events and hold execution at those events with new iterator methods, run_iter on ConversableAgent, and run_group_chat_iter for group chats. This stops the background execution, essentially it is waiting for you to acknowledge the step before it continues to process.

This utilises the current events framework, with a step controller helping to stop execution on the events.

The following events always trigger a step of the iter_on filter on the new methods:

Event Type	Reason
InputRequestEvent	User must be able to respond to input requests
ErrorEvent	Errors are raised as exceptions
RunCompletionEvent	Signals completion (returns None)

iter_on
The iter_on parameter allows you to specify which events you want to stop on. This is useful if you don't need to stop on all events.

Using custom events
I've added a notebook that provides a guide on creating your own events that you can then step on.

Example of stepping through

import os
from autogen import ConversableAgent
from autogen.events.agent_events import InputRequestEvent
from dotenv import load_dotenv

load_dotenv()

llm_config = {
    "config_list": [
        {
            "model": "gpt-4o-mini",
            "api_key": os.environ.get("OPENAI_API_KEY"),
        }
    ]
}

# Create two agents for a simple conversation
jack = ConversableAgent(
    "Jack",
    system_message="Your name is Jack and you are a comedian. Tell one short joke and then say FINISH.",
    is_termination_msg=lambda x: "FINISH" in x.get("content", ""),
    llm_config=llm_config,
)

emma = ConversableAgent(
    "Emma",
    system_message="Your name is Emma. Laugh at Jack's joke and say FINISH.",
    llm_config=llm_config,
)

# Step through every event using run_iter
response = jack.run_iter(emma, message="Emma, tell me a joke!", max_turns=2)

# Or, only yield specific event types (e.g., text and termination events):
# response = jack.run_iter(emma, message="Emma, tell me a joke!", max_turns=2, yield_on=[TextEvent, TerminationEvent])

step_count = 0
for event in response:
    step_count += 1
    print(f"\n[Step {step_count}] Event type: {event.type}")

    # Handle input requests - prompt user for input
    if isinstance(event, InputRequestEvent):
        user_input = input("  Input requested: ")
        event.content.respond(user_input)
        continue

    # Access event content
    if hasattr(event, "content") and hasattr(event.content, "content"):
        content = str(event.content.content)
        preview = content[:100] + "..." if len(content) > 100 else content
        print(f"  Content: {preview}")

print(f"\n[Step {step_count}] Run completed!")
print(f"\nTotal steps: {step_count}")
print(f"Summary: {response.summary}")

Related issue number

N/A

Checks

• I've included any doc changes needed for https://docs.ag2.ai/. See https://docs.ag2.ai/latest/docs/contributor-guide/documentation/ to build and test documentation locally.
• I've added tests (if relevant) corresponding to the changes introduced in this PR.
• I've made sure all auto checks have passed.

[joggrbot]

📝 Documentation Analysis

All docs are up to date! 🎉

---

✅ Latest commit analyzed: 0edbd63 | Powered by Joggr

[priyansh4320]

Thank you @marklysze , this is a great feature,
event look very pretty in terminal, and the Custom Event feature and the event filter in StepController are Awesome.
the turn count shown at the end of chat, is very helpful feature in orchestration design.
good documentation.
great design.
Approving this Pull request.

the only change I would suggest, is to change
step_on: Sequence[type] | None = None, ---> step_on: Sequence[type[BaseEvent] | None = None , aiming for Custom event type check.
Merry Christmas :)

[Lancetnik]

Great job, @marklysze ! Really impressive
But, I have some doubts about suggested public API

Why you prefer current impl:

with agent.run(..., step_mode=True) as response:
    while True:
        event = response.step()
        if event is not None:
            ...

Instead of alternative API without changes in current behavior?

for msg in agent.run_iter(...):
    ... # msg always `BaseEvent` type instead of `BaseEvent | None`

[marklysze]

Great job, @marklysze ! Really impressive But, I have some doubts about suggested public API

Why you prefer current impl:

with agent.run(..., step_mode=True) as response:
    while True:
        event = response.step()
        if event is not None:
            ...

Instead of alternative API without changes in current behavior?

for msg in agent.run_iter(...):
    ... # msg always `BaseEvent` type instead of `BaseEvent | None`

Appreciate this thought and it's a better Pythonic approach with the iterator, and I think the clean-up that I used the context manager for, is better your suggested way. I am working through a conversion to run_iter :).

[marklysze]

Thanks @Lancetnik and @priyansh4320 for your feedback and suggestions - I have refactored to new iterators, run_iter and run_group_chat_iter. Would appreciate your review, again.

[codecov]

Codecov Report

❌ Patch coverage is 61.92053% with 115 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
autogen/agentchat/conversable_agent.py	6.15%	61 Missing ⚠️
autogen/agentchat/group/multi_agent_chat.py	18.75%	26 Missing ⚠️
autogen/io/run_response.py	83.11%	21 Missing and 5 partials ⚠️
autogen/io/thread_io_stream.py	50.00%	1 Missing and 1 partial ⚠️

Files with missing lines	Coverage Δ
autogen/agentchat/__init__.py	100.00% <100.00%> (ø)
autogen/io/step_controller.py	100.00% <100.00%> (ø)
autogen/io/thread_io_stream.py	84.61% <50.00%> (-4.28%)	⬇️
autogen/agentchat/group/multi_agent_chat.py	31.06% <18.75%> (-5.42%)	⬇️
autogen/io/run_response.py	74.36% <83.11%> (+9.69%)	⬆️
autogen/agentchat/conversable_agent.py	63.04% <6.15%> (-3.01%)	⬇️

... and 19 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.