Skip to content

Python: [BREAKING] Moved to a single get_response and run API#3379

Open
eavanvalkenburg wants to merge 98 commits intomicrosoft:mainfrom
eavanvalkenburg:python_single_response
Open

Python: [BREAKING] Moved to a single get_response and run API#3379
eavanvalkenburg wants to merge 98 commits intomicrosoft:mainfrom
eavanvalkenburg:python_single_response

Conversation

@eavanvalkenburg
Copy link
Member

@eavanvalkenburg eavanvalkenburg commented Jan 22, 2026

Motivation and Context

Summary

  • Migrate chat/agent telemetry to mixin-based usage and remove legacy decorators, with streaming telemetry now using finalizers/teardown hooks instead of consuming streams.
    • This makes understanding the code a lot simpler, because we can set attributes on the chat client in the init of those mixin (making them technically not a mixin)
    • Added those parameters to the constructors, making it easier to configure things like function calling
  • Replace function invocation decorators with FunctionInvokingChatClient/FunctionInvokingMixin across clients, tests, and samples; update docs/comments accordingly.
  • Introducing a ResponseStream object that can is created to unify the API's
    • It is generic over TUpdate and TFinal, which in our case is usually ChatResponseUpdate and ChatReponse or the agent equivalent.
    • It features a update_hook mechanism, to allow you to run code while the internal stream is being unpacked, this can mostly be leveraged by middleware
    • It features a teardown hook mechanism, this get's run when the stream is exhausted, it's used now by the telemtry to record the duration
    • It features a finalizer (one or more) mechanism, that runs after the end of the stream, which is used to turn the updates list into a final object, this can be used by middleware and is also used in function calling and telemetry
    • In principle the ResponseStream is created by the most lowlevel object, the actual chat client implementations, and ideally all the layers in between should only use the hooks to do something, FunctionCalling does not work that way, because there are multiple calls to the underlying chat client that all then have to be combined into a single stream at runtime. Agent also creates a new stream, because it goes from ResponseStream[ChatResponseUpdate, ChatResponse] to ResponseStream[AgentResponseUpdate, AgentResponse], but the object has a classmethod called wrap that is used to wrap the ResponseStream from the chat client into the new ResponseStream in the Agent.
  • Overall this change reduces the number of times we iterate the stream and return a new AsyncGenerator and the new hooks actually make it simpler to create middleware that alters the stream (as the sample shows), it should therefore also improve performance a bit.
  • Removed use_instrumentation/use_agent_instrumentation and use_function_invocation decorators; mixins are now the supported path.

Description

Contribution Checklist

  • The code builds clean without any errors or warnings
  • The PR follows the Contribution Guidelines
  • All unit tests pass, and I have added new tests where possible
  • Is this a breaking change? If yes, add [BREAKING] prefix to the title of the PR.

Fixes #3585
Fixes #3607
Fixes #3617
Fixes #3581
Fixes #3610

Copilot AI review requested due to automatic review settings January 22, 2026 17:34
@markwallace-microsoft markwallace-microsoft added documentation Improvements or additions to documentation python labels Jan 22, 2026
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR consolidates the Python Agent Framework's streaming and non-streaming APIs into a unified interface. The primary changes include:

Changes:

  • Unified run() and get_response() methods with stream parameter replacing separate run_stream() and get_streaming_response() methods
  • Migration from decorator-based (@use_instrumentation, @use_function_invocation) to mixin-based architecture for telemetry and function invocation
  • Introduction of ResponseStream class for unified stream handling with hooks, finalizers, and teardown support
  • Renamed AgentExecutionException to AgentRunException

Reviewed changes

Copilot reviewed 84 out of 85 changed files in this pull request and generated 28 comments.

Show a summary per file
File Description
_types.py Added ResponseStream class for unified streaming, updated prepare_messages to handle None
_clients.py Refactored BaseChatClient with unified get_response() method, introduced FunctionInvokingChatClient mixin
openai/_responses_client.py Consolidated streaming/non-streaming into single _inner_get_response() method
openai/_chat_client.py Similar consolidation for chat completions API
openai/_assistants_client.py Unified assistants API with stream parameter
_workflows/_workflow.py Consolidated run() and run_stream() into single run(stream=bool) method
_workflows/_agent.py Updated WorkflowAgent.run() to use stream parameter
Test files (multiple) Updated all tests to use run(stream=True) and get_response(stream=True)
Sample files (multiple) Updated samples to demonstrate new unified API
Provider clients Updated all provider implementations (Azure, Anthropic, Bedrock, Ollama, etc.) to use mixins

@eavanvalkenburg eavanvalkenburg force-pushed the python_single_response branch 3 times, most recently from 07afd46 to dd65afa Compare January 23, 2026 10:46
@markwallace-microsoft
Copy link
Member

markwallace-microsoft commented Jan 23, 2026

Python Test Coverage

Python Test Coverage Report •
FileStmtsMissCoverMissing
packages/a2a/agent_framework_a2a
   _agent.py148894%261, 399–400, 437–438, 467–469
packages/ag-ui/agent_framework_ag_ui
   _client.py1501788%83–84, 88–92, 96–100, 263, 295, 464–466
   _message_adapters.py4549579%89, 99–100, 109–112, 115–119, 121–126, 129, 138–144, 147, 151–153, 162–164, 184, 190–192, 222, 235–236, 246–247, 284, 287, 289, 292, 295, 311, 328, 350, 381, 386, 397–398, 449, 465–466, 532–535, 537, 543, 551–552, 554, 558–561, 574, 663–666, 668, 734, 769–771, 773–776, 779–780, 782, 788, 791, 793, 796, 798, 804–805, 807
   _run.py48112174%155–162, 305, 324–325, 340–341, 356, 384–386, 411, 414–417, 419–420, 423–429, 432–434, 437, 453–455, 462, 468–470, 474, 479–481, 483–484, 500–504, 515, 528, 530–531, 547, 568–569, 626–628, 640–642, 840, 851–852, 859, 877–879, 913–915, 932, 938, 946, 948, 984–990, 993–996, 998–1007, 1010, 1018–1021, 1028, 1031–1032, 1037, 1043–1045, 1049, 1054–1057, 1071–1073
   _types.py360100% 
   _utils.py101397%72, 257, 262
packages/ag-ui/agent_framework_ag_ui/_orchestration
   _tooling.py570100% 
packages/anthropic/agent_framework_anthropic
   _chat_client.py36215058%371, 403, 405, 420, 442–445, 454, 456, 487–491, 493, 495–496, 498, 503–504, 506, 539–540, 549, 551–552, 557, 574–575, 617, 632, 636–637, 653, 662, 664, 668–669, 712–714, 716, 729–730, 737–739, 743–745, 749–752, 763, 765, 787, 797, 819–825, 832–833, 841–842, 850–853, 860–861, 867–868, 874–875, 881, 889–891, 895, 902–903, 909–910, 916–917, 923, 931–934, 941–942, 961, 968–969, 988, 1010, 1012, 1021–1022, 1028, 1050–1051, 1057–1058, 1067–1077, 1084–1090, 1097–1103, 1110–1119, 1126–1129
packages/azure-ai/agent_framework_azure_ai
   _agent_provider.py115397%122–123, 251
   _chat_client.py4847584%382, 387–388, 390–391, 394, 397, 399, 404, 665–666, 668, 671, 674, 677–682, 685, 687, 695, 707–709, 713, 716–717, 725–728, 738, 746–749, 751–752, 754–755, 762, 770–771, 779–780, 785–786, 790–797, 802, 805, 813, 819, 827–829, 832, 854–855, 988, 1016, 1031, 1152, 1178, 1187, 1196, 1329
   _client.py1951393%360, 362, 411, 440–445, 488, 523, 525, 601
   _project_provider.py115694%132–133, 211, 309, 353, 386
packages/chatkit/agent_framework_chatkit
   _converter.py1334665%115, 120, 168, 170, 340, 393, 395, 414–416, 418, 436, 438, 440, 443, 455, 465, 483, 503–527, 529–531
packages/copilotstudio/agent_framework_copilotstudio
   _agent.py83593%154–155, 190, 198, 315
packages/core/agent_framework
   _agents.py3203589%473, 885, 921, 1020–1022, 1135, 1176, 1178, 1187–1192, 1198, 1200, 1210–1211, 1218, 1220–1221, 1229–1233, 1241–1242, 1244, 1249, 1251, 1285, 1325, 1345
   _clients.py52394%294, 495, 497
   _middleware.py3291695%80, 83, 88, 797, 799, 801, 922, 949, 951, 976, 1057, 1061, 1185, 1189, 1250, 1324
   _serialization.py105496%516, 532, 542, 610
   _threads.py136397%343, 474–475
   _tools.py7888589%232, 278, 329, 331, 359, 529, 564–565, 667, 669, 689, 707, 721, 733, 738, 740, 747, 780, 851–853, 894, 916–944, 979, 987, 1228, 1433, 1490, 1494, 1573–1577, 1595, 1597–1598, 1703, 1707, 1757, 1759, 1775, 1777, 1841, 1868, 1921, 1989, 2165–2166, 2193, 2201, 2214, 2224–2225, 2260, 2316, 2348
   _types.py10389690%81, 90–91, 145, 150, 169, 171, 175, 179, 181, 183, 185, 203, 207, 233, 255, 260, 265, 269, 295, 299, 645–646, 1017, 1079, 1096, 1114, 1119, 1137, 1147, 1164–1165, 1167, 1185–1186, 1188, 1195–1196, 1198, 1233, 1244–1245, 1247, 1285, 1542, 1595, 1602, 1624, 1630, 1678, 1721–1726, 1748, 1753, 1882, 1894, 2133, 2142, 2162, 2254, 2475, 2743, 2747, 2759, 2766, 2777, 2981–2983, 2986–2988, 2992, 2997, 3001, 3113–3115, 3143, 3197, 3201–3203, 3205, 3216–3217, 3220–3224, 3230
   observability.py6078286%334, 336–338, 341–343, 348–349, 355–356, 362–363, 370, 372–374, 377–379, 384–385, 391–392, 398–399, 406, 662, 665, 673–674, 677–680, 682, 685–687, 690–691, 719, 721, 732–734, 736–739, 743, 751, 852, 854, 1003, 1005, 1009–1014, 1016, 1019–1023, 1025, 1137–1138, 1140, 1327, 1375–1376, 1492–1494, 1553, 1723, 1877, 1879
packages/core/agent_framework/_workflows
   _agent.py3325483%59, 67–73, 107–108, 340–341, 350–351, 358, 360, 366–367, 453–454, 463, 470, 496, 547, 576, 635–638, 644, 650, 654–655, 658–664, 668–669, 675, 749–750, 761, 800, 821, 830, 834, 836–838, 845
   _agent_executor.py1602286%98, 146, 164–165, 216–217, 219–220, 252–254, 262–264, 274–276, 278, 282, 286, 290–291
   _base_group_chat_orchestrator.py1701292%135, 301, 316, 350–352, 356, 375, 436, 480–482
   _const.py60100% 
   _conversation_state.py36586%40, 45, 47, 49, 64
   _message_utils.py18383%22, 33, 37
   _orchestration_request_info.py540100% 
   _orchestrator_helpers.py21290%90–91
   _runner_context.py168696%84, 87, 383, 403, 491, 495
   _workflow.py2721993%89, 267–269, 271–272, 290, 294, 322, 424, 642, 684, 689, 692, 711–713, 726, 794
   _workflow_context.py157994%62–63, 71, 75, 89, 165, 190, 308, 427
packages/core/agent_framework/azure
   _chat_client.py79494%301, 303, 316–317
   _responses_client.py37683%146, 169, 198–201
packages/core/agent_framework/openai
   _assistant_provider.py1101190%156–157, 169, 294, 360, 475–480
   _assistants_client.py2753587%358, 360, 362, 365, 369–370, 373, 376, 381–382, 384, 387–389, 394, 405, 430, 432, 434, 436, 438, 443, 446, 449, 453, 464, 549, 634, 671, 708–711, 763, 780
   _chat_client.py2672192%179–180, 184, 298, 305, 386–393, 395–398, 408, 493, 530, 546
   _responses_client.py5626288%277–278, 283, 314, 322, 345, 407, 439, 464, 470, 488–489, 511, 516, 572, 586, 603, 616, 671, 750, 755, 759–761, 765–766, 789, 858, 880–881, 896–897, 915–916, 1047–1048, 1064, 1066, 1141–1149, 1197, 1252, 1267, 1303–1304, 1306–1308, 1322–1324, 1334–1335, 1341, 1356
   _shared.py1351688%63, 69–72, 151, 153, 155, 162, 164, 177, 253, 277, 341–342, 344
packages/mem0/agent_framework_mem0
   _provider.py85692%67–68, 90, 174–175, 178
packages/orchestrations/agent_framework_orchestrations
   _group_chat.py2913986%171, 332, 339, 366, 377–378, 384, 389, 405, 432–437, 439, 472–475, 477, 482–486, 650, 655, 669, 750, 756, 802, 822, 897–898, 932, 951, 970, 980
   _handoff.py3835785%109–110, 112, 141–142, 164–174, 176, 178, 180, 185, 285, 339, 364, 392, 400–401, 415, 464–465, 497, 544–546, 733, 740, 745, 832, 835, 844–847, 857, 862, 869, 875–878, 913, 918, 1115, 1118, 1126, 1144, 1151, 1226
   _magentic.py6209185%66–75, 80, 84–95, 260, 271, 275, 295, 356, 365, 367, 409, 426, 435–436, 438–440, 442, 453, 595, 597, 637, 685, 721–723, 725, 733–736, 740–743, 786, 813–816, 907, 913, 919, 958, 996, 1025, 1042, 1053, 1107–1108, 1112–1114, 1138, 1159–1160, 1173, 1189, 1211, 1259–1260, 1298–1299, 1458, 1467, 1470, 1475, 1871, 1926, 1941, 1970
packages/purview/agent_framework_purview
   _middleware.py950100% 
packages/redis/agent_framework_redis
   _chat_message_store.py1491490%199, 232, 322–323, 326, 329, 485, 575–579, 588, 592
   _provider.py188995%255, 257, 265, 270–271, 274, 329, 386, 398
TOTAL16510202187% 

Python Unit Test Overview

Tests Skipped Failures Errors Time
3909 225 💤 0 ❌ 0 🔥 1m 13s ⏱️

@eavanvalkenburg eavanvalkenburg changed the title Python: [BREAKING} Python single response Python: [BREAKING] Moved to a single get_response and run API Jan 23, 2026
@eavanvalkenburg eavanvalkenburg force-pushed the python_single_response branch 4 times, most recently from 32f0473 to 5c78d91 Compare January 30, 2026 05:03
@eavanvalkenburg eavanvalkenburg requested a review from a team as a code owner January 30, 2026 16:25
@markwallace-microsoft markwallace-microsoft added .NET workflows Related to Workflows in agent-framework lab Agent Framework Lab labels Feb 1, 2026
@github-actions github-actions bot changed the title Python: [BREAKING] Moved to a single get_response and run API .NET: Python: [BREAKING] Moved to a single get_response and run API Feb 1, 2026
@eavanvalkenburg eavanvalkenburg changed the title .NET: Python: [BREAKING] Moved to a single get_response and run API Python: [BREAKING] Moved to a single get_response and run API Feb 1, 2026
@eavanvalkenburg
Copy link
Member Author

Fixes lingering CI failures: import missing response types in streaming telemetry finalizers, move AG-UI tests to ag_ui_tests with config updates, and track service thread IDs in AG-UI test client.\n\nChecks: uv run poe fmt/lint/pyright/mypy; uv run poe all-tests.

- Fix a2a tests: Role comparisons and ChatMessage signatures
- Fix lab tau2 source: Role enum comparison in flip_messages, log_messages, sliding_window
- Fix lab tau2 tests: ChatMessage signatures and Role comparisons
After rebasing on upstream/main which merged PR microsoft#3647 (Types API Review
improvements), fix all packages to use the new API:

- ChatMessage: Use keyword args (role=, text=, contents=) instead of
  positional args
- Role: Compare using .value attribute since it's now an enum

Packages fixed:
- ag-ui: Fixed Role value extraction bugs in _message_adapters.py
- anthropic: Fixed ChatMessage and Role comparisons in tests
- azure-ai: Fixed Role comparison in _client.py
- azure-ai-search: Fixed ChatMessage and Role in source/tests
- bedrock: Fixed ChatMessage signatures in tests
- chatkit: Fixed ChatMessage and Role in source/tests
- copilotstudio: Fixed ChatMessage and Role in tests
- declarative: Fixed ChatMessage in _executors_agents.py
- mem0: Fixed ChatMessage and Role in source/tests
- purview: Fixed ChatMessage in source/tests
- durabletask: Use str() fallback in role value extraction
- core: Fix ChatMessage in _orchestrator_helpers.py to use keyword args
- core: Add type ignore for _conversation_state.py contents deserialization
- ag-ui: Fix type ignore comments (call-overload instead of arg-type)
- azure-ai-search: Fix get_role_value type hint to accept Any
- lab: Move get_role_value to module level with Any type hint
- Increase job timeout from 10 to 15 minutes
- Reduce per-test timeout to 60s (was 900s/300s)
- Add --timeout_method thread for better timeout handling
- Add --timeout-verbose to see which tests are slow
- Reduce retries from 3 to 2 and delay from 10s to 5s

This ensures individual test timeouts are shorter than the job
timeout, providing better visibility when tests hang.

With 60s timeout and 2 retries, worst case per test is ~180s.
- Fix ChatMessage positional args in docstrings: _serialization.py, _threads.py, _middleware.py
- Fix ChatMessage in tau2 runner.py
- Fix role comparison in _orchestrator_helpers.py to use .value
- Fix role comparison in _group_chat.py docstring example
- Fix role assertions in test_durable_entities.py to use .value
… no tools

OpenAI API requires tool_choice and parallel_tool_calls to only be
present when tools are specified. Restored the logic that removes
these options when there are no tools.

- Restored check in _chat_client.py to remove tool_choice and
  parallel_tool_calls when no tools present
- Restored same logic in _responses_client.py
- Reverted test to expect the correct behavior
- Refactor streaming function invocation to use get_final_response() on inner streams
- Fix MiddlewareTermination to accept result parameter for passing results
- Fix _AutoHandoffMiddleware to use MiddlewareTermination instead of context.terminate
- Fix AgentMiddlewareLayer.run() to properly forward function/chat middleware
- Remove duplicate middleware registration in AgentMiddlewareLayer.__init__
- Fix exception handling in _auto_invoke_function to properly capture termination
- Fix mypy errors in core package
- Update tests to use stream=True parameter for unified run API
- Merge testutils.py into conftest.py for azurefunctions integration tests
- Merge dt_testutils.py into conftest.py for durabletask integration tests
- Convert all integration tests to use fixtures instead of direct imports
  (fixes ModuleNotFoundError with --import-mode=importlib)
- Add sample_helper fixture for azurefunctions tests
- Add agent_client_factory and orchestration_helper fixtures for durabletask
- Integration tests now skip with descriptive messages when services unavailable
- Restructure devui tests into tests/devui/ with proper conftest.py
- Add test organization guidelines to CODING_STANDARD.md
- Remove __init__.py from test directories per pytest best practices
The hook was skipping all tests in the test session, not just
integration tests. Now it only skips items in the integration_tests
directory.
Use patch.object on the imported module instead of @patch with string
path to ensure the mock takes effect regardless of import timing.
@eavanvalkenburg eavanvalkenburg added this pull request to the merge queue Feb 5, 2026
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Feb 5, 2026
Increase from 2 to 8 seconds to allow time for:
- Python startup and module imports
- Azure OpenAI client creation
- Agent registration with DTS worker
- Worker connection to DTS

This helps prevent test failures in CI where the first tests may run
before the worker is fully ready to process requests.
The _consume_stream method now expects a ResponseStream that can provide
a final AgentResponse via get_final_response(). Update the test to use
ResponseStream with AgentResponse.from_updates as the finalizer.
@eavanvalkenburg eavanvalkenburg added this pull request to the merge queue Feb 5, 2026
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Feb 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

breaking change Introduces changes that are not backward compatible and may require updates to dependent code. documentation Improvements or additions to documentation lab Agent Framework Lab python workflows Related to Workflows in agent-framework

Projects

None yet

6 participants