fix(ai): attach user media to Gemini requests#1241
Open
ling-senpeng13 wants to merge 2 commits into
Open
Conversation
GeminiChatModel.convertMessage() built the USER content from userMsg.getText() only and silently dropped userMsg.getMedia(), so images passed via the media input path never reached Gemini — the model received a text-only message. Same bug (and same shape of fix) as conductor-oss#1238 for Anthropic. Convert user media into Gemini inline data parts (base64) alongside the text. Adds a GeminiApi.Part.inlineData(mimeType, base64Data) factory (the InlineData record already existed, just had no request-side factory). Adds GeminiChatModelMediaTest, which captures the contents passed to generateContent and asserts the user message carries an inline image part with the verbatim base64 payload. Verified it fails before the fix (inline part missing) and passes after. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Fixes spotlessJavaCheck CI failure by wrapping lines per Google Java Format. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Images passed as input via the
mediapath never reached Gemini models — the model received a text-only message. Same bug as #1238 (Anthropic), in the Gemini converter:LLMHelper.getMessage()builds a Spring AIUserMessagewith media (bytes), butGeminiChatModel.convertMessage()only forwardedgetText().Fix
GeminiChatModel.convertMessage(): when aUserMessagecarries media, build text + one inline-dataPartper media item (base64), mirroring the OpenAI path and the fix(ai): attach user media to Anthropic requests #1238 Anthropic fix. Text-only messages are unchanged.GeminiApi: add aPart.inlineData(mimeType, base64Data)factory (theInlineDatarecord already existed; it just had no request-side factory).Test
Adds
GeminiChatModelMediaTest, which mocks the API, captures thecontentspassed togenerateContent, and asserts the user content carries an inline image part (inlineData) withmimeType=image/pngand the verbatim base64 payload — plus the accompanying text part.Validated both directions:
Scope
Gemini provider only; other providers untouched.
convertMessageis the single message-building path for Gemini, so all Gemini requests are covered. Companion to #1238 (Anthropic) — the two custom converters that dropped media.🤖 Generated with Claude Code