VinF Hybrid Inference: throw if only_on_device and model is unavailable by erikeldridge · Pull Request #8965 · firebase/firebase-js-sdk

erikeldridge · 2025-04-22T23:33:51Z

Problem: we're currently falling back to Cloud even if the mode is only_on_device.

Proposal:

isAvailable should return true unconditionally if mode is only_on_device
createSession should throw if the model is unavailable

Also: remove stale tests for systemInstruction; I added a backlog item for figuring out if we want to honor systemInstruction passed in the request vs onDeviceParams.

changeset-bot · 2025-04-22T23:33:55Z

⚠️ No Changeset found

Latest commit: 69f393e

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

github-actions · 2025-04-22T23:34:07Z

Vertex AI Mock Responses Check ⚠️

A newer major version of the mock responses for Vertex AI unit tests is available. update_vertexai_responses.sh should be updated to clone the latest version of the responses: v10.0

erikeldridge · 2025-04-22T23:37:43Z

packages/vertexai/src/methods/chrome-adapter.test.ts

        })
      ).to.be.false;
    });
-    it('returns false if AI API is undefined', async () => {


This was stale, since we no longer have an ai namespace

erikeldridge · 2025-04-22T23:38:19Z

packages/vertexai/src/methods/chrome-adapter.test.ts

      const adapter = new ChromeAdapter(
-        {} as LanguageModel,
+        {
+          availability: async () => Availability.available


We now check availability first, so this was added in a couple places

erikeldridge · 2025-04-22T23:39:28Z

packages/vertexai/src/methods/chrome-adapter.test.ts

        })
      ).to.be.false;
    });
-    it('returns false if request system instruction has function role', async () => {


We used to copy the systemInstruction out of the request, but removed that in favor of onDeviceParams.systemInstruction, which isn't currently validated. Added backlog item to revisit this.

google-oss-bot · 2025-04-22T23:43:23Z

Size Report ¹

Affected Products

@firebase/vertexai
Type Base (5fa83b3) Merge (cdb22ca) Diff
browser 39.3 kB 39.4 kB +130 B (+0.3%)
main 40.3 kB 40.4 kB +130 B (+0.3%)
module 39.3 kB 39.4 kB +130 B (+0.3%)
firebase
Type Base (5fa83b3) Merge (cdb22ca) Diff
firebase-vertexai.js 31.5 kB 31.6 kB +123 B (+0.4%)

Test Logs

Base (5fa83b3): https://github.com/firebase/firebase-js-sdk/actions/runs/14604863125
Merge (cdb22ca): https://github.com/firebase/firebase-js-sdk/actions/runs/14607770260

https://storage.googleapis.com/firebase-sdk-metric-reports/eMYXV93doM.html

google-oss-bot · 2025-04-22T23:54:57Z

Size Analysis Report ¹

Affected Products

@firebase/vertexai
- getGenerativeModel
  Size
  Type Base (5fa83b3) Merge (cdb22ca) Diff
  size 22.3 kB 22.4 kB +103 B (+0.5%)
  size-with-ext-deps 41.3 kB 41.4 kB +104 B (+0.3%)

Test Logs

Base (5fa83b3): https://github.com/firebase/firebase-js-sdk/actions/runs/14604863125
Merge (cdb22ca): https://github.com/firebase/firebase-js-sdk/actions/runs/14607770260

https://storage.googleapis.com/firebase-sdk-metric-reports/GrI0sktjYQ.html

Fix languageCode parameter in action_code_url (#8912) * Fix languageCode parameter in action_code_url * Add changeset Vaihi add langmodel types. (#8927) * Adding LanguageModel types. These are based off https://github.com/webmachinelearning/prompt-api?tab=readme-ov-file#full-api-surface-in-web-idl * Adding LanguageModel types. * Remove bunch of exports * yarn formatted * after lint Define HybridParams (#8935) Co-authored-by: Erik Eldridge <erikeldridge@google.com> Adding smoke test for new hybrid params (#8937) * Adding smoke test for new hybrid params * Use the existing name of the model params input --------- Co-authored-by: Erik Eldridge <erikeldridge@google.com> Moving to in-cloud naming (#8938) Co-authored-by: Erik Eldridge <erikeldridge@google.com> Moving to string type for the inference mode (#8941) Define ChromeAdapter class (#8942) Co-authored-by: Erik Eldridge <erikeldridge@google.com> VinF Hybrid Inference: Implement ChromeAdapter (rebased) (#8943) Adding count token impl (#8950) VinF Hybrid Inference #4: ChromeAdapter in stream methods (rebased) (#8949) Define values for Availability enum (#8951) VinF Hybrid Inference: narrow Chrome input type (#8953) Add image inference support (#8954) * Adding image based input for inference * adding image as input to create language model object disable count tokens api for on-device inference (#8962) VinF Hybrid Inference: throw if only_on_device and model is unavailable (#8965)

Fix languageCode parameter in action_code_url (#8912) * Fix languageCode parameter in action_code_url * Add changeset Vaihi add langmodel types. (#8927) * Adding LanguageModel types. These are based off https://github.com/webmachinelearning/prompt-api?tab=readme-ov-file#full-api-surface-in-web-idl * Adding LanguageModel types. * Remove bunch of exports * yarn formatted * after lint Define HybridParams (#8935) Co-authored-by: Erik Eldridge <erikeldridge@google.com> Adding smoke test for new hybrid params (#8937) * Adding smoke test for new hybrid params * Use the existing name of the model params input --------- Co-authored-by: Erik Eldridge <erikeldridge@google.com> Moving to in-cloud naming (#8938) Co-authored-by: Erik Eldridge <erikeldridge@google.com> Moving to string type for the inference mode (#8941) Define ChromeAdapter class (#8942) Co-authored-by: Erik Eldridge <erikeldridge@google.com> VinF Hybrid Inference: Implement ChromeAdapter (rebased) (#8943) Adding count token impl (#8950) VinF Hybrid Inference #4: ChromeAdapter in stream methods (rebased) (#8949) Define values for Availability enum (#8951) VinF Hybrid Inference: narrow Chrome input type (#8953) Add image inference support (#8954) * Adding image based input for inference * adding image as input to create language model object disable count tokens api for on-device inference (#8962) VinF Hybrid Inference: throw if only_on_device and model is unavailable (#8965) VinF Hybrid Inference: update docs (#8970) VinF Hybrid Inference: consolidate onDeviceParams initialization (#8969) VinF Hybrid Inference: disable multi-turn support (#8973) VinF Hybrid Inference: remove default expected input types (#8974) VinF Hybrid Inference: set image (and text) as default input type (#8984) VinF Hybrid Inference: log debug messages in conditional logic (#8992)

Return true unconditionally if mode is only_on_device

a685395

erikeldridge requested a review from a team as a code owner April 22, 2025 23:33

Remove stray log statement

8979482

erikeldridge commented Apr 22, 2025

View reviewed changes

Throw Vertex error

69f393e

erikeldridge merged commit e839743 into vaihi-exp Apr 23, 2025
33 of 45 checks passed

erikeldridge deleted the erikeldridge-vertex-modethrow branch April 23, 2025 01:24

gsiddh mentioned this pull request Apr 23, 2025

Complete Hybrid inference impl #8966

Open

firebase locked and limited conversation to collaborators May 24, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Comments

VinF Hybrid Inference: throw if only_on_device and model is unavailable#8965

VinF Hybrid Inference: throw if only_on_device and model is unavailable#8965
erikeldridge merged 3 commits intovaihi-expfrom
erikeldridge-vertex-modethrow

erikeldridge commented Apr 22, 2025 •

edited

Loading

Uh oh!

changeset-bot bot commented Apr 22, 2025 •

edited

Loading

Uh oh!

github-actions bot commented Apr 22, 2025

Uh oh!

erikeldridge Apr 22, 2025

Uh oh!

erikeldridge Apr 22, 2025

Uh oh!

erikeldridge Apr 22, 2025

Uh oh!

google-oss-bot commented Apr 22, 2025 •

edited

Loading

`@firebase/vertexai`

`firebase`

Uh oh!

google-oss-bot commented Apr 22, 2025 •

edited

Loading

`@firebase/vertexai`

`getGenerativeModel`

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Comments

Conversation

erikeldridge commented Apr 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

changeset-bot bot commented Apr 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

⚠️ No Changeset found

Uh oh!

github-actions bot commented Apr 22, 2025

Vertex AI Mock Responses Check ⚠️

Uh oh!

erikeldridge Apr 22, 2025

Choose a reason for hiding this comment

Uh oh!

erikeldridge Apr 22, 2025

Choose a reason for hiding this comment

Uh oh!

erikeldridge Apr 22, 2025

Choose a reason for hiding this comment

Uh oh!

google-oss-bot commented Apr 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Size Report 1

Affected Products

@firebase/vertexai

firebase

Test Logs

Uh oh!

google-oss-bot commented Apr 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Size Analysis Report 1

Affected Products

@firebase/vertexai

getGenerativeModel

Test Logs

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

erikeldridge commented Apr 22, 2025 •

edited

Loading

changeset-bot bot commented Apr 22, 2025 •

edited

Loading

google-oss-bot commented Apr 22, 2025 •

edited

Loading

Size Report ¹

`@firebase/vertexai`

`firebase`

google-oss-bot commented Apr 22, 2025 •

edited

Loading

Size Analysis Report ¹

`@firebase/vertexai`

`getGenerativeModel`