VinF Hybrid Inference: throw if only_on_device and model is unavailable #8965

erikeldridge · 2025-04-22T23:33:51Z

Problem: we're currently falling back to Cloud even if the mode is only_on_device.

Proposal:

isAvailable should return true unconditionally if mode is only_on_device
createSession should throw if the model is unavailable

Also: remove stale tests for systemInstruction; I added a backlog item for figuring out if we want to honor systemInstruction passed in the request vs onDeviceParams.

changeset-bot · 2025-04-22T23:33:55Z

⚠️ No Changeset found

Latest commit: 69f393e

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

github-actions · 2025-04-22T23:34:07Z

Vertex AI Mock Responses Check ⚠️

A newer major version of the mock responses for Vertex AI unit tests is available. update_vertexai_responses.sh should be updated to clone the latest version of the responses: v10.0

erikeldridge · 2025-04-22T23:37:43Z

packages/vertexai/src/methods/chrome-adapter.test.ts

@@ -61,19 +61,8 @@ describe('ChromeAdapter', () => {
        })
      ).to.be.false;
    });
-    it('returns false if AI API is undefined', async () => {


This was stale, since we no longer have an ai namespace

erikeldridge · 2025-04-22T23:38:19Z

packages/vertexai/src/methods/chrome-adapter.test.ts

@@ -82,7 +71,9 @@ describe('ChromeAdapter', () => {
    });
    it('returns false if request contents empty', async () => {
      const adapter = new ChromeAdapter(
-        {} as LanguageModel,
+        {
+          availability: async () => Availability.available


We now check availability first, so this was added in a couple places

erikeldridge · 2025-04-22T23:39:28Z

packages/vertexai/src/methods/chrome-adapter.test.ts

@@ -107,51 +100,6 @@ describe('ChromeAdapter', () => {
        })
      ).to.be.false;
    });
-    it('returns false if request system instruction has function role', async () => {


We used to copy the systemInstruction out of the request, but removed that in favor of onDeviceParams.systemInstruction, which isn't currently validated. Added backlog item to revisit this.

google-oss-bot · 2025-04-22T23:43:23Z

Size Report ¹

Affected Products

@firebase/vertexai
Type Base (5fa83b3) Merge (cdb22ca) Diff
browser 39.3 kB 39.4 kB +130 B (+0.3%)
main 40.3 kB 40.4 kB +130 B (+0.3%)
module 39.3 kB 39.4 kB +130 B (+0.3%)
firebase
Type Base (5fa83b3) Merge (cdb22ca) Diff
firebase-vertexai.js 31.5 kB 31.6 kB +123 B (+0.4%)

Test Logs

https://storage.googleapis.com/firebase-sdk-metric-reports/eMYXV93doM.html

google-oss-bot · 2025-04-22T23:54:57Z

Size Analysis Report ¹

Affected Products

@firebase/vertexai
- getGenerativeModel
  Size
  Type Base (5fa83b3) Merge (cdb22ca) Diff
  size 22.3 kB 22.4 kB +103 B (+0.5%)
  size-with-ext-deps 41.3 kB 41.4 kB +104 B (+0.3%)

Test Logs

https://storage.googleapis.com/firebase-sdk-metric-reports/GrI0sktjYQ.html

Fix languageCode parameter in action_code_url (#8912) * Fix languageCode parameter in action_code_url * Add changeset Vaihi add langmodel types. (#8927) * Adding LanguageModel types. These are based off https://github.com/webmachinelearning/prompt-api?tab=readme-ov-file#full-api-surface-in-web-idl * Adding LanguageModel types. * Remove bunch of exports * yarn formatted * after lint Define HybridParams (#8935) Co-authored-by: Erik Eldridge <erikeldridge@google.com> Adding smoke test for new hybrid params (#8937) * Adding smoke test for new hybrid params * Use the existing name of the model params input --------- Co-authored-by: Erik Eldridge <erikeldridge@google.com> Moving to in-cloud naming (#8938) Co-authored-by: Erik Eldridge <erikeldridge@google.com> Moving to string type for the inference mode (#8941) Define ChromeAdapter class (#8942) Co-authored-by: Erik Eldridge <erikeldridge@google.com> VinF Hybrid Inference: Implement ChromeAdapter (rebased) (#8943) Adding count token impl (#8950) VinF Hybrid Inference #4: ChromeAdapter in stream methods (rebased) (#8949) Define values for Availability enum (#8951) VinF Hybrid Inference: narrow Chrome input type (#8953) Add image inference support (#8954) * Adding image based input for inference * adding image as input to create language model object disable count tokens api for on-device inference (#8962) VinF Hybrid Inference: throw if only_on_device and model is unavailable (#8965)

Return true unconditionally if mode is only_on_device

a685395

erikeldridge requested a review from a team as a code owner April 22, 2025 23:33

Remove stray log statement

8979482

erikeldridge commented Apr 22, 2025

View reviewed changes

Throw Vertex error

69f393e

erikeldridge merged commit e839743 into vaihi-exp Apr 23, 2025
33 of 45 checks passed

erikeldridge deleted the erikeldridge-vertex-modethrow branch April 23, 2025 01:24

gsiddh mentioned this pull request Apr 23, 2025

Complete Hybrid inference impl #8966

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

VinF Hybrid Inference: throw if only_on_device and model is unavailable #8965

VinF Hybrid Inference: throw if only_on_device and model is unavailable #8965

erikeldridge commented Apr 22, 2025 •

edited

Loading

changeset-bot bot commented Apr 22, 2025 •

edited

Loading

github-actions bot commented Apr 22, 2025

erikeldridge Apr 22, 2025

erikeldridge Apr 22, 2025

erikeldridge Apr 22, 2025

google-oss-bot commented Apr 22, 2025 •

edited

Loading

`@firebase/vertexai`

`firebase`

google-oss-bot commented Apr 22, 2025 •

edited

Loading

`@firebase/vertexai`

`getGenerativeModel`

VinF Hybrid Inference: throw if only_on_device and model is unavailable #8965

VinF Hybrid Inference: throw if only_on_device and model is unavailable #8965

Conversation

erikeldridge commented Apr 22, 2025 • edited Loading

changeset-bot bot commented Apr 22, 2025 • edited Loading

⚠️ No Changeset found

github-actions bot commented Apr 22, 2025

Vertex AI Mock Responses Check ⚠️

erikeldridge Apr 22, 2025

Choose a reason for hiding this comment

erikeldridge Apr 22, 2025

Choose a reason for hiding this comment

erikeldridge Apr 22, 2025

Choose a reason for hiding this comment

google-oss-bot commented Apr 22, 2025 • edited Loading

Size Report 1

Affected Products

@firebase/vertexai

firebase

Test Logs

google-oss-bot commented Apr 22, 2025 • edited Loading

Size Analysis Report 1

Affected Products

@firebase/vertexai

getGenerativeModel

Test Logs

erikeldridge commented Apr 22, 2025 •

edited

Loading

changeset-bot bot commented Apr 22, 2025 •

edited

Loading

google-oss-bot commented Apr 22, 2025 •

edited

Loading

Size Report ¹

`@firebase/vertexai`

`firebase`

google-oss-bot commented Apr 22, 2025 •

edited

Loading

Size Analysis Report ¹

`@firebase/vertexai`

`getGenerativeModel`