Repository navigation
Yield final image when streaming with HostedImageGenerationTool - #7774
Open
iamAdarshh wants to merge 2 commits into
Open
iamAdarshh wants to merge 2 commits into
iamAdarshh wants to merge 2 commits into
Conversation
Contributor
There was a problem hiding this comment.
Copilot review overview
🟢 Approval recommended
The fix is covered by regression tests and there are no unresolved review comments.
Review effort: Lite
Findings: None
What changed in this PR
Fixes streaming HostedImageGenerationTool responses so the final generated image is surfaced correctly.
Changes:
- Handles completed image-generation results during streaming.
- Shares result construction between streaming and non-streaming paths.
- Adds regression tests and corrects response fixtures.
| File | Summary |
|---|---|
test/Libraries/Microsoft.Extensions.AI.OpenAI.Tests/OpenAIResponseClientTests.cs |
Adds streaming image-generation regression tests and updates fixtures. |
src/Libraries/Microsoft.Extensions.AI.OpenAI/OpenAIResponsesChatClient.cs |
Emits final streamed images and shares result creation logic. |
💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.
Author
|
@dotnet-policy-service agree |
When streaming a Responses API response that uses image generation, the completed ImageGenerationCallResponseItem was treated like items whose content had already been fully yielded via deltas, so an empty update was produced and the final image was dropped. Partial images are intermediate renders, not deltas of the final image, so consumers only ever saw the last partial image, or no image at all when StreamingCount was not set. Yield an ImageGenerationToolResultContent for the completed item, built the same way as in the non-streaming path. Coalescing replaces the partial results with the final one since they share the same CallId. Also fix the streaming test fixtures to use the "result" property that the OpenAI SDK reads for the final image bytes. Fixes dotnet#7758
iamAdarshh
force-pushed
the
fix/7758-streaming-final-image
branch
from
September 23, 2026 11:55
e9a9a47 to
38cdad5
Compare
Collaborator
🎉 Good job! The coverage increased 🎉
Full code coverage report: https://dev.azure.com/dnceng-public/public/_build/results?buildId=1608856&view=codecoverage-tab |
jozkee
reviewed
Oct 7, 2026
| // but any partial images yielded along the way are lower-quality intermediate renders rather than | ||
| // deltas of the final image. Yield the final image here; coalescing will replace the partial | ||
| // results with this one, since they share the same CallId. | ||
| case ImageGenerationCallResponseItem { ImageResultBytes: not null } imageGenItem: |
Member
There was a problem hiding this comment.
Should we remove the ImageResultBytes: not null guard here? The non-streaming path handles every ImageGenerationCallResponseItem unconditionally via AddImageGenerationContents, so the guard introduces a streaming/non-streaming asymmetry when ImageResultBytes is null.
We should also cover that with a test.
Author
There was a problem hiding this comment.
Good catch, removed the guard in 21edcd9 so streaming now produces a result for every ImageGenerationCallResponseItem, same as AddImageGenerationContents. I turned HostedImageGenerationTool_StreamingAndNonStreaming_ProduceSameImageContents into a theory that also covers a failed item with no result; that case fails with the guard in place and passes without it.
jozkee
added
waiting-author-feedback 📭
…eaming Drop the ImageResultBytes guard so the streaming path produces an ImageGenerationToolResultContent for every ImageGenerationCallResponseItem, matching the non-streaming path. Extend the streaming/non-streaming parity test to cover an item without a result.
jozkee
enabled auto-merge (squash)
October 9, 2026 16:52
Collaborator
🎉 Good job! The coverage increased 🎉
Full code coverage report: https://dev.azure.com/dnceng-public/public/_build/results?buildId=1630024&view=codecoverage-tab |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #7758. Also fixes microsoft/agent-framework#8674.
Problem
When streaming a Responses API response that uses
HostedImageGenerationTool, the final generated image was never surfaced. InOpenAIResponsesChatClient.GetStreamingResponseAsync, the completedImageGenerationCallResponseItem(fromresponse.output_item.done) was grouped withMessageResponseItem/ReasoningResponseItem, whose content has already been yielded via deltas, so an empty update was produced andImageResultByteswas dropped.Partial images are lower-quality intermediate renders, not deltas of the final image. As a result:
StreamingCountset, consumers only received the partial images, and the last one they saw was a partial render.StreamingCount, streaming produced no image at all.Fix
ImageGenerationCallResponseItemits own case in the streaming path that yields anImageGenerationToolResultContentwith the final image.CreateImageGenerationResultContent, shared by the streaming and non-streaming (AddImageGenerationContents) paths, so both build the result identically.ImageGenerationCallResponseItem, including one withoutImageResultBytes.Coalescing:
CoalesceImageResultContentreplaces earlier results with later ones for the sameCallId, soToChatResponse()ends up with just the final image.Roundtripping: the final result's
RawRepresentationis theImageGenerationCallResponseItem, the same as in the non-streaming path, so it is sent back once. Partial results carry a streaming update rather than aResponseItemand are not sent back, so there is no duplication.Tests
HostedImageGenerationTool_Streaming_YieldsFinalImageAfterPartialImages:StreamingCount = 3yields 3 partial images followed by the final image, and coalescing keeps only the final image.HostedImageGenerationTool_Streaming_WithoutPartialImages_YieldsFinalImage: withoutStreamingCount, exactly one image result (the final image) is yielded.HostedImageGenerationTool_StreamingAndNonStreaming_ProduceSameImageContents: streaming and non-streaming produce the same tool call and result contents, both for a completed item and for a failed item with noresult.All of them fail without the fix. I also updated the existing streaming fixtures from
image_result_b64toresult, which is the property the OpenAI SDK actually reads forImageResultBytes; the old name was silently ignored.Microsoft.Extensions.AI.OpenAI.Testspasses on net8.0, net9.0 and net10.0.Microsoft Reviewers: Open in CodeFlow