Repository navigation
Conversation
A user turn created by LLMContext.create_image_message() carries its text and image in one Content (two parts), and an image-only turn carries a single part with no text. Neither matched the single-text-part shape the adapter's regular-message check required, so a context holding only such turns got the system instruction appended as an extra trailing user message, while it was also sent as the system_instruction parameter. A message now counts as regular when any of its parts is neither a function call nor a function response, so the injection still fires only for contexts holding nothing but function calls and responses.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Please describe the changes in your PR. If it is addressing an issue, please reference that as well.
Summary
The Gemini adapter appended the system instruction as an extra trailing user message when the context held only image-bearing user messages, while the same text was also sent through the
system_instructionparameter.Why
GeminiLLMAdapter._from_universal_context_messages()decides whether to re-append the system prompt as a user message using a check that only counted a message as "regular" when it had exactly one part carrying text (src/pipecat/adapters/services/gemini_adapter.py:400onmain):A user turn created by
LLMContext.create_image_message()(src/pipecat/processors/aggregators/llm_context.py:137) converts to aContentwith two parts (text + image), and an image-only turn converts to a single part withinline_dataand no text. Neither matched that single-text-part shape, so a context holding only such turns was treated like a function-messages-only context and the system instruction was injected a second time as the conversation's final user message (gemini_adapter.py:410onmain).Changes
src/pipecat/adapters/services/gemini_adapter.py:400). The system instruction is still appended as a user message for contexts holding nothing but function calls and responses; text, image, audio, and file parts all count as regular content now.changelog/5910.fixed.md.Testing
uv run pytest "tests/test_get_llm_invocation_params.py" -k "system_instruction_not_appended_for_image or system_instruction_appended_when_only_function"— 3 passed. Onmainthe two image tests fail withAssertionError: 2 != 1(the system instruction appended as a second message); the third test pins the preserved function-messages-only behavior.uv run pytest tests/test_get_llm_invocation_params.py— 179 passed, 18 subtests passed.uv run pytest tests/test_run_inference.py tests/test_function_calling_adapters.py tests/test_google_thinking_defaults.py tests/test_flows_context_strategies.py tests/test_google_stream_timeout.py tests/test_realtime_service_tool_sync.py— 152 passed, 2 subtests passed.uv run pytest tests/test_gemini_live_interaction_status.py tests/test_gemini_live_tracing.py tests/test_gemini_stt.py— 56 passed, 5 skipped.uv run pytest(full suite) — 4756 passed, 33 skipped, 99 subtests passed. The only 3 failures are the RNNoise tests, which fail identically onmainin this environment (pyrnnoiseraisesTypeError: Graph.__init__() got an unexpected keyword argument 'rate'), so they are pre-existing and unrelated.uv run ruff check— clean.uv run ruff format --check— clean.uv run pyright src/pipecat/adapters/services/gemini_adapter.py tests/test_get_llm_invocation_params.py— 0 errors.