fix(google): keep unspoken model text out of the realtime transcript

[anzemur]

What

In a Gemini realtime session running with audio output and output transcription enabled (the plugin default), RealtimeSession.handleServerContent no longer forwards modelTurn.parts[].text into the generation's text stream. Only outputTranscription — the transcription of the audio the model actually speaks — feeds the transcript.

Behaviour is unchanged when the session runs in text modality (modalities: [Modality.TEXT]) or when output transcription is disabled (outputAudioTranscription: null): there the model's text parts are the response and still flow through.

Why

Both writers currently land in the same gen.textChannel:

if (part.text) { gen.outputText += part.text; gen.textChannel.write(part.text); }   // modelTurn
…
gen.outputText += text; gen.textChannel.write(text);                                 // outputTranscription

That channel is the message's textStream, so whatever goes in ends up in the lk.transcription streams, in the assistant ChatMessage (ConversationItemAdded), and in the plugin's own _chatCtx replay on reconnect.

In an audio session a text part on the model turn is never spoken. We hit this in production on gemini-3.1-flash-live-preview (audio modality, outputAudioTranscription: {}, no thinkingConfig): the model occasionally writes out a function call as text instead of emitting a toolCall — the transcript then contains things like

call:assetGenerator{context:…,title:…,type:quiz

followed, in the same generation, by the words that were actually spoken. In one turn the text was cut off exactly where the real toolCall arrived (handleToolCall → markCurrentGenerationDone closes the channel), so the partial call was persisted as its own assistant message; in another the whole call came through as text, no toolCall fired, and the leaked text was the only trace. Nothing was said aloud — the leak was invisible in the audio and very visible in captions and persisted history.

The same failure class is acknowledged by Google for Gemini 3.x (function calls verbalized into the text stream in call:<name>{…} / <call:default_api:…/> shape): https://discuss.ai.google.dev/t/gemini-3-5-flash-lite-verbalizes-its-function-call-as-pseudo-xml-in-the-text-stream-instead-of-emitting-a-functioncall-part/176699 — and reported against the Python plugin in livekit/agents#5662. The plugin already skips part.thought for the same reason (reasoning is not spoken); an unflagged text part in an audio-with-transcription session is the remaining way unspoken text reaches the transcript.

Approach

One condition computed once per server message, next to the existing discardOutput:

const forwardModelText =
  !this._realtimeModel.capabilities.audioOutput ||
  this.options.outputAudioTranscription === undefined;

and the text-part branch becomes if (part.text && forwardModelText). inlineData handling, outputTranscription, tool calls, _firstTokenTimestamp and the turn-complete bookkeeping are untouched. Text-modality sessions and audio sessions without transcription behave exactly as before.

The Python plugin has the identical merge (realtime_api.py, _handle_server_content: current_gen.push_text(part.text) and current_gen.push_text(output_transcription.text)), so the same one-line gate applies there if you want parity — happy to open that too.

Tests

Wire replay — plugins/google/src/realtime/realtime_transcript.test.ts drives the real RealtimeSession through a mocked live.connect() (same harness as live_setup_wiring.test.ts), plays server frames in, and asserts on the public output streams (messageStream → textStream/audioStream, functionStream) the way AgentActivity reads them:

frames in	on main	with this PR
text part call:assetGenerator{context: → audio → outputTranscription: 'Tako je!' → complete	text 'call:assetGenerator{context:Tako je!', 1 audio frame	text 'Tako je!', 1 audio frame
text part call:getWeather{location: → toolCall getWeather	text 'call:getWeather{location:', tool call delivered	text '', tool call delivered
modalities: [TEXT], text part 'Hello there.'	text 'Hello there.'	unchanged
outputAudioTranscription: null, text part 'Hello there.'	text 'Hello there.'	unchanged

The first row is the production artifact byte for byte (leaked call glued to the spoken words in one assistant message).

Unit — Google Realtime model text parts in realtime_api.test.ts covers the same three configurations at the handleServerContent level (Object.create(RealtimeSession.prototype) pattern already used in that file); the audio+transcription case is red on main.

Live probe (local only, not committed) — real gemini-3.1-flash-live-preview through the patched plugin: a spoken reply arrives as modelTurn.inlineData (8 frames) + outputTranscription ('smoke ok'), zero model text parts, and the plugin's transcript equals the transcription; a tool-calling prompt yields the call on functionStream. So the frame shapes the replay test assumes are the ones the API sends, and speech/transcription/tool calls still flow.

Checks: pnpm exec vitest run plugins/google — 45 passed / 2 skipped (pre-existing); tsc --noEmit for the plugin — clean; eslint — no new findings (three pre-existing no-explicit-any warnings elsewhere in the file); prettier --check — clean; changeset added (@livekit/agents-plugin-google: patch).

🤖 Generated with Claude Code

[changeset-bot]

🦋 Changeset detected

Latest commit: 53905cb

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 39 packages

Name	Type
@livekit/agents-plugin-google	Patch
@livekit/agents	Patch
@livekit/agents-plugin-anam	Patch
@livekit/agents-plugin-anthropic	Patch
@livekit/agents-plugin-assemblyai	Patch
@livekit/agents-plugin-azure	Patch
@livekit/agents-plugin-baseten	Patch
@livekit/agents-plugin-bey	Patch
@livekit/agents-plugin-cartesia	Patch
@livekit/agents-plugin-cerebras	Patch
@livekit/agents-plugin-deepgram	Patch
@livekit/agents-plugin-did	Patch
@livekit/agents-plugin-elevenlabs	Patch
@livekit/agents-plugin-fishaudio	Patch
@livekit/agents-plugin-hedra	Patch
@livekit/agents-plugin-hume	Patch
@livekit/agents-plugin-inworld	Patch
@livekit/agents-plugin-krisp	Patch
@livekit/agents-plugin-lemonslice	Patch
@livekit/agents-plugin-liveavatar	Patch
@livekit/agents-plugin-livekit	Patch
@livekit/agents-plugin-minimax	Patch
@livekit/agents-plugin-mistral	Patch
@livekit/agents-plugin-mistralai	Patch
@livekit/agents-plugin-neuphonic	Patch
@livekit/agents-plugin-openai	Patch
@livekit/agents-plugin-perplexity	Patch
@livekit/agents-plugin-phonic	Patch
@livekit/agents-plugin-protoface	Patch
@livekit/agents-plugin-resemble	Patch
@livekit/agents-plugin-rime	Patch
@livekit/agents-plugin-runway	Patch
@livekit/agents-plugin-sarvam	Patch
@livekit/agents-plugin-silero	Patch
@livekit/agents-plugin-soniox	Patch
@livekit/agents-plugin-tavus	Patch
@livekit/agents-plugin-trugen	Patch
@livekit/agents-plugin-xai	Patch
@livekit/agents-plugins-test	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[devin-ai-integration]

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no bugs or issues to report.