feat: Vercel AI SDK v6 with tool-calling

[jakobhoeg]

This PR adds a Next.js example application of how to use Transformers.js models that support tool-calling with the Vercel AI SDK v6 (I also renamed the old next-vercel-ai-sdk to next-vercel-ai-sdk-v5).
It uses @browser-ai that acts as a Transformers.js model provider for the Vercel AI SDK, enabling seamless tool-call management.

The internals of the provider handles incremental streaming that detects which tool is being called before the model finishes generating arguments, which is like how native cloud-based API's handle tool-call streaming.
It also automatically handles different model tool-call outputs, such as XML tags (<tool_call>) and Python-style ([functionName(args)]) syntax, which enhances the Developer Experience dramatically. Easily switch between Qwen3, LFM2, Granite models without introducing a bunch of boilerplate code.

On the UI side, it also allows for HITL (human in the loop) tool-calls by just setting a simple needsApproval flag in the tool-call schema.

@xenova let me know if anything needs to be changed :)

Demo:

demooo.mp4

[xenova]

Wow that's really cool! I'm sure @nico-martin will be interested too.

[jakobhoeg]

Thank you, @xenova!

Hopefully I can get the internals updated to use Transformers.js v4 too when it's released :)

[jakobhoeg]

@nico-martin let me know if I need to change anything

[nico-martin]

Hi @jakobhoeg

That amazing!

I had one small issue on npm run dev:

./node_modules/@browser-ai/transformers-js/dist/index.js:2125:28
Module not found: Can't resolve '@huggingface/transformers'
  2123 |
  2124 | // src/transcription/transformers-js-transcription-model.ts
> 2125 | var import_transformers3 = require("@huggingface/transformers");
       |                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  2126 | function isBrowserEnvironment3() {
  2127 |   return typeof window !== "undefined";
  2128 | }

The problem is that @browser-ai/transformers-js/ has transformers.js as a peerDependency.
This means the package expects the next-vercel-ai-sdk-v5 needs transformers.js as a dependency. to provide @huggingface/transformers.

Edit: Sorry, wrong folder. But it seems like both, next-vercel-ai-sdk-v5 and next-vercel-ai-sdk-v6-tool-calling, have the same problem.

[nico-martin]

Pretty solid example 👍 !
Other than the Module not found: Can't resolve '@huggingface/transformers' I have nothing to add.
I might use some of the patterns for future projects. Its really good!

[jakobhoeg]

@nico-martin thank you for the feedback!
Strange! Are you using npm to install? And if so, what version? v7+ should handle this automatically and install peerDependencies for you 🤔 but for good practice, I'll just add it as a direct dependency, so others also won't run into the same issue.

[jakobhoeg]

@nico-martin should be fixed now

[nico-martin]

@nico-martin thank you for the feedback! Strange! Are you using npm to install? And if so, what version? v7+ should handle this automatically and install peerDependencies for you 🤔 but for good practice, I'll just add it as a direct dependency, so others also won't run into the same issue.

Thats strange. Yes, I use npm v 10.9.2 and did a normal npm install and npm run dev..

Anyways, thanks for the update!

[nico-martin]

lgtm 👍

[nico-martin]

I really like this pattern! Keeps the worker logic out of the app logic. I know WebLLM has a similar WorkerHandler and I think we should have one in Transformers.js too.