Skip to content

fix(firebaseai): keep LiveSession.receive() open on voiceActivity frames - #18770

Open
SelaseKay wants to merge 1 commit into
mainfrom
fix/ai-live-voice-activity
Open

SelaseKay wants to merge 1 commit into
mainfrom
fix/ai-live-voice-activity

Conversation

@SelaseKay

Copy link
Copy Markdown
Contributor

Description

gemini-3.8-live sends a voiceActivity frame when the user starts or stops speaking. firebase_ai did not model that message, so parsing threw FirebaseAISdkException. LiveSession.receive() is an await for over the broadcast controller, so that error closed the Dart stream while the WebSocket stayed open. Apps that treat onDone as a closed session lost the rest of the spoken turn.

This parses voiceActivity into LiveServerVoiceActivity (type and audioOffset). An unrecognized activity type still yields the message, with type left null. A top-level key the SDK does not recognize is logged and skipped, so receive() stays open for the next frame. A server error payload or invalid JSON still ends the stream.

An exhaustive switch on LiveServerMessage needs a new case, the same way GoingAwayNotice did.

Related Issues

Fixes #18749

Checklist

Before you create this PR confirm that it meets all requirements listed below by checking the relevant checkboxes ([x]).
This will ensure a smooth and quick review process. Updating the pubspec.yaml and changelogs is not required.

  • I read the Contributor Guide and followed the process outlined there for submitting PRs.
  • My PR includes unit or integration tests for all changed/updated/fixed behaviors (See Contributor Guide).
  • All existing and new tests are passing.
  • I updated/added relevant documentation (doc comments with ///).
  • The analyzer (melos run analyze) does not report any problems on my PR.
  • I read and followed the Flutter Style Guide.
  • I signed the CLA.
  • I am willing to follow-up on review comments in a timely manner.

Breaking Change

Does your PR require plugin users to manually update their apps to accommodate your change?

  • Yes, this is a breaking change.
  • No, this is not a breaking change.

Test plan

  • flutter test test/live_test.dart test/live_session_test.dart in packages/firebase_ai/firebase_ai.
  • On device, gemini-3.8-live with microphone audio keeps LiveSession.receive() open after ACTIVITY_START, and later audio or turnComplete frames still arrive.
  • An unrecognized live frame is logged and does not close receive().

gemini-3.8-live sends a voiceActivity message the parser rejected, which ended the receive stream while the socket was still open.
@gemini-code-assist

Copy link
Copy Markdown
Contributor
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize the Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counterproductive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[firebase_ai] LiveSession.receive() ends on gemini-3.8-live's voiceActivity message (unhandled format) — every spoken turn kills the stream

1 participant