Repository navigation
Conversation
…unctuation heuristic MessageConverter.isPartialResponse classified streaming chunks by ASCII terminal punctuation (. ! ? \n). Turns ending with CJK terminal punctuation (。!?) were never marked final, so the response was silently not persisted to the session; conversely a '.' inside a longer answer fired a premature final mid-stream. Stream chunks are now always partial; StreamingResponseAggregator (already present, previously unused) produces the single final response on stream completion. The completion signal is authoritative regardless of locale, provider vocabulary (stop/end_turn/STOP/tool_calls) and gateways that drop finishReason. Persistence timing is unchanged: the final event was already the last chunk in the correct case. Mid-stream errors now produce no final, instead of persisting a truncated response. isPartialResponse is deprecated and no longer called. Fixes google#1608
|
Hi @caps-xia, thank you for your contribution and quick fix.It is currently under review by our team. We will update you if any additional details are needed. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Link to Issue or Description of Change
Closes: #1608
Problem:
MessageConverter#toLlmResponseclassifies each streaming chunk as partial/final viaisPartialResponse, an ASCII terminal-punctuation heuristic (.,!,?,\n). Two failure modes:.fires a premature final inside a longer answer.There is also a related defect: since
Runnerstreams partial events but never persists them (by design — see the comment inRunner), and the punctuation-"final" chunk only carries its own delta text, sessions today persist only the last punctuation-ending chunk fragment instead of the full answer.Solution:
Stop classifying chunks mid-stream. Streaming chunks are always partial;
StreamingResponseAggregator(already present but never wired) produces the single aggregated final response on stream completion:SpringAIstreaming path: feed each converted chunk to a per-request aggregator and forward it; on the Flux completion callback, emitaggregator.getFinalResponse()— exactly one final event carrying the full text and accumulated tool calls — beforeonComplete().MessageConverter: streaming chunks are always partial; the old punctuation heuristic is retained (unused) for rollback/reference.usageMetadata, which would otherwise be lost with the unpersisted partial events.Why the completion signal: it is the only end-of-turn signal guaranteed across providers —
finishReasonspelling/availability varies (stop/end_turn/STOP; tool-call turns usetool_calls; some gateways omit it entirely), and punctuation is locale-dependent. This also matches the framework's intended design:Runneralready expects "the final aggregated event" to carry the full content, and the Gemini bridge (Gemini.java) uses the same partials + aggregated-final pattern via its ownStreamingResponseAggregator.Behavioral notes:
Testing Plan
Unit Tests:
testGenerateContentStreamingCjkTerminalPunctuationPersistsFinal— a CJK-terminated turn produces exactly one final (persisted) event with the full text.testToLlmResponseStreamingAlwaysMarksChunksPartial— streaming chunks are always partial regardless of punctuation.testGenerateContentStreamingFinalCarriesUsageMetadata— usage metadata from the last chunk survives onto the aggregated final.testGenerateContentStreamingandtestGenerateContentStreamingBackpressurefor the final event (3 chunks + 1 final; 1000 + 1).mvn -pl contrib/spring-ai -am test→ 217 tests, 0 failures, BUILD SUCCESS.Manual End-to-End (E2E) Tests:
Reproduced with a real OpenAI-compatible gateway and a Chinese-language workflow:
Checklist
Additional context
Implements option 2 as discussed on the issue (root fix — stream-completion-based final detection; @hirematha confirmed the reproduction there).
isPartialResponseis kept unused as rollback/reference documentation of the previous behavior. Happy to also provide the interim CJK-punctuation patch as a separate PR if maintainers want a short-term mitigation.