[Image Classification] Failed to execute 'dispatch' on 'MLContext': Invalid inputs

[ibelem]

Failed to execute 'dispatch' on 'MLContext' after landing #45

Repro Steps A

1. Enable WebNN in browser
2. Visit https://microsoft.github.io/webnn-developer-preview/demos/image-classification/?provider=webnn&devicetype=gpu&model=mobilenet-v2&run=1
3. Click "Classify" button
4. It will show performance results, click "Classify" button again

Repro Steps B

1. Enable WebNN in browser
2. Visit https://microsoft.github.io/webnn-developer-preview/demos/image-classification/?provider=webnn&devicetype=gpu&model=mobilenet-v2&run=1
3. Click "Classify" button
4. It will show performance results, click "ResNet50" button
5. Click "Classify" button

Actual Result

Failed to execute 'dispatch' on 'MLContext': Invalid inputs: The context of MLGraph doesn't match the context of the MLTensor with name "pixel_values".

@egalli do you have any clue to fix this issue, thanks!

CC @Honry @fdwr @huningxin

Note

This issue didn't occur by using compute() in ORT Web before.

How to build the Transformers.js dists and apply for developer preview image classification demo

1. Clone https://github.com/xenova/transformers.js/tree/v3 // "onnxruntime-web": "1.20.0-dev.20241016-2b8fc5529b",
2. npm install
3. Apply the patch https://github.com/ibelem/transformers.js/blob/v3-webnn-2/0001-v3-perf-code.patch or follow the code change in ibelem/transformers.js@7cb760f
4. npm run build
5. Copy the dists from /transformers.js/dist to webnn-developer-preview/assets/dist_transformers/dynamic-runs-1.19-dev/

[egalli]

It seems to be related to how we created a new MLContext per sessions. I have created a PR that adds a cache for MLContexts so that InferencesSessions created with the same parameters will use the same MLContext.

FYI, while testing change to the ORT, I noticed that transformers.js defaults to downloading the .wasm file from a CDN. If we want to use the version this repo, we'll need to set that variable to

transformers.env.backends.onnx.wasm.wasmPaths = "../../assets/dist_transformers/dynamic-runs-1.20-dev/";

[ibelem]

Thank you @egalli , I will re-build transformers.js dist once merging the microsoft/onnxruntime#22510 , also set transformers.env.backends.onnx.wasm.wasmPaths.

[fdwr]

Thank you @egalli , I will re-build transformers.js dist once merging the microsoft/onnxruntime#22510 , also set transformers.env.backends.onnx.wasm.wasmPaths.

@ibelem - Tis merged.

[ibelem]

Hold the dists update since the microsoft/onnxruntime#22278 and microsoft/onnxruntime#22556 cause demos regression.

[fdwr]

Hold the dists update since the microsoft/onnxruntime#22278 and microsoft/onnxruntime#22556 cause demos regression.

Merged Revert "[WebNN] Fallback the node when its output doesn't have shape info" https://github.com/microsoft/onnxruntime/pull/22669.

[ibelem]

@fdwr

We need to wait for PRs below to be merged and integrated into the new development version of ORT distributions, otherwise more models may fail to run.

• [WebNN EP] Fix issues with MLTensor caching onnxruntime#22701 Fix issues with MLTensor caching

[fdwr]

@fdwr

We need to wait for PRs below to be merged and integrated into the new development version of ORT distributions, otherwise more models may fail to run.

• [WebNN EP] Fix issues with MLTensor caching onnxruntime#22701 Fix issues with MLTensor caching

@ibelem - I hadn't seen this one (not tagged). Thanks - looking at.

[ibelem]

microsoft/onnxruntime#22701 Fix issues with MLTensor caching was landed, waiting for next dev version of ONNX ORT dist later than 1.21.0-dev.20241106 @https://www.npmjs.com/package/onnxruntime-web?activeTab=versions .