Skip to content

VinF Hybrid Inference: flatten request contents rather than plucking the first #8991

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: vaihi-exp-google-ai
Choose a base branch
from

Conversation

erikeldridge
Copy link

Context

While waiting for the Prompt API to support sequence input, we were ignoring all but the first item in the list of Content objects in the prompt request. We fall back to Cloud for Chat support for the same reason.

Note the fallback logic we use for chat support checks if a given request has multiple roles, and this logic runs for all requests as part of isAvailable.

Problem Statement

We had some feedback that we should log a warning when we ignore input.

Solution

Given we're already ensuring a request only has a single role before running it on device, it's safe to flatten all content objects, rather than just plucking the first.

@erikeldridge erikeldridge requested a review from a team as a code owner May 1, 2025 20:29
Copy link

changeset-bot bot commented May 1, 2025

⚠️ No Changeset found

Latest commit: 9968791

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

Copy link
Contributor

github-actions bot commented May 1, 2025

Vertex AI Mock Responses Check ⚠️

A newer major version of the mock responses for Vertex AI unit tests is available. update_vertexai_responses.sh should be updated to clone the latest version of the responses: v11.1

@google-oss-bot
Copy link
Contributor

Size Report 1

Affected Products

  • @firebase/vertexai

    TypeBase (31261ca)Merge (e8f6144)Diff
    browser46.2 kB46.2 kB+50 B (+0.1%)
    main47.3 kB47.4 kB+50 B (+0.1%)
    module46.2 kB46.2 kB+50 B (+0.1%)
  • firebase

    TypeBase (31261ca)Merge (e8f6144)Diff
    firebase-vertexai.js36.3 kB36.3 kB+26 B (+0.1%)

Test Logs

  1. https://storage.googleapis.com/firebase-sdk-metric-reports/FDVNY66HrF.html

@google-oss-bot
Copy link
Contributor

Size Analysis Report 1

Affected Products

  • @firebase/vertexai

    • getGenerativeModel

      Size

      TypeBase (31261ca)Merge (e8f6144)Diff
      size26.4 kB26.4 kB+26 B (+0.1%)
      size-with-ext-deps45.4 kB45.4 kB+26 B (+0.1%)

Test Logs

  1. https://storage.googleapis.com/firebase-sdk-metric-reports/o33mdT2nGM.html

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants