Issue loading Phi 3.5 vision models

[andrzejwp]

Dear team,

I'm having an issue loading either of the Phi 3.5 vision models.

Tested models:

• Phi-3.5-vision-instruct q0f16
• Phi-3.5-vision-instruct q3f16_1
• Phi-3.5-vision-instruct q4f16_1
• Phi-3.5-vision-instruct q4f32_1

Tested platforms:

• Mac OS Sequoia 15.3, Macbook Pro M2 Max
  • Edge Version 132.0.2957.115 (Official build) (arm64)
  • Arc Version 1.78.1 (57736
• Windows 11 24H2
  • Chrome 131.0.6778.265

Tested code

• Version deployed on https://chat.webllm.ai
• locally executed code from web-llm/examples/vision-model/ (commit sha 632d347 - current main branch)

The error

In all cases I get the same error when loading the Phi 3.5 vision models. After downloading the model this error appears in the browser console log:

sw.js:64 [FATAL] /Users/cfruan/Documents/tvm/web/../src/runtime/relax_vm/ndarray_cache_support.cc:333: ValueError: Cannot find parameter in cache: vision_embed_tokens.img_processor.vision_model.embeddings.position_embedding.q_weight

Can you please suggest how to address this?

[adem-jaz]

I had the exact same issue.

[andrzejwp]

Hi Web-llm team! Any updates on this?

[marciopuga]

same here, any update?

[DerFichtl]

Same here! Looks like this is a duplicate of #640 ... since November not working. I wonder if this is the only way to have vision model in browser? Any alternatives?

[InfTkm]

Same issue here.

[mewdev]

Same here.

[grantalope]

Same issue.

[vimalkodoth]

any fix yet? :-( any other alternatives for image analysis?

[diogo-strube]

I suppose is not fixed yet, right folks? Facing same issue.