Support Flux multilingual model (flux-general-multi) and language_hint parameter in WorkersAIFluxSTT

[diroverflow]

Summary

WorkersAIFluxSTT in @cloudflare/voice currently only uses @cf/deepgram/flux which maps to Deepgram's English-only model (flux-general-en). There is no way to select the multilingual model (flux-general-multi) or pass language_hint parameters, making it impossible to build voice agents for non-English users.

Problem

Deepgram has released Flux Multilingual (flux-general-multi), a single model supporting 10 languages (English, Spanish, French, German, Hindi, Russian, Portuguese, Japanese, Italian, Dutch) with auto-detection and language_hint biasing. However:

1. Workers AI's @cf/deepgram/flux appears to only serve flux-general-en. There is no parameter to switch to flux-general-multi.
2. WorkersAIFluxSTT has no language or languageHint option in its constructor/config. Looking at the source, the WebSocket input only sends encoding, sample_rate, eot_threshold, etc. — no language-related fields.
3. The underlying ai.run("@cf/deepgram/flux", input, { websocket: true }) call provides no way to specify the Deepgram model variant or language_hint.

For context, WorkersAINova3STT already supports a language option — it would be great if WorkersAIFluxSTT followed a similar pattern.

Proposed Solution

1. Workers AI level: Add a model or language parameter to @cf/deepgram/flux that allows selecting flux-general-multi and passing language_hint values through to Deepgram.
2. @cloudflare/voice level: Add language / languageHints options to WorkersAIFluxSTT constructor, similar to how WorkersAINova3STT accepts language. When provided, pass these through to the ai.run() call.

Example API:

class MyAgent extends VoiceAgent<Env> {
  // Option A: single language hint
  transcriber = new WorkersAIFluxSTT(this.env.AI, { languageHints: ["zh"] });
  
  // Option B: multiple language hints for bilingual users
  transcriber = new WorkersAIFluxSTT(this.env.AI, { languageHints: ["en", "zh"] });
  
  // Option C: no hints — auto-detect
  transcriber = new WorkersAIFluxSTT(this.env.AI, { languageHints: [] });
}

Use Case

We're building an AI companion platform where characters can speak multiple languages. Currently, our voice call feature only works for English-speaking users. Supporting multilingual STT would allow us to serve our global user base without switching to a different STT provider.

Additional Context

• Deepgram Flux Multilingual docs: https://developers.deepgram.com/docs/flux/language-prompting
• Cloudflare Workers AI Flux docs: https://developers.cloudflare.com/workers-ai/models/flux/
• Flux Multilingual was GA'd on April 29, 2026
• Currently supported languages in flux-general-multi: en, es, fr, de, hi, ru, pt, ja, it, nl
• Chinese (zh) is not yet supported by Flux — a follow-up request for broader language coverage would also be valuable

[whoiskatrin]

hey hey @diroverflow, give a go these package in PR? #1459

Let know how this works and feels to you, thanks!

[diroverflow]

Thanks @whoiskatrin for working on this! We've been testing PR #1459 and ran into an important blocker that I think is worth documenting here.

What we tested

We built and linked the PR #1459 branch locally, then deployed it to production on Cloudflare Workers. Our voice agent overrides createTranscriber() to pass model: "flux-general-multi" and languageHints: ["ja"] when the character's locale is Japanese:

override createTranscriber(connection: Connection): WorkersAIFluxSTT {
  const url = new URL(connection.uri || "");
  const locale = url.searchParams.get("characterLocale") || "en";
  const languageHints = this.buildLanguageHints(locale);
  return new WorkersAIFluxSTT(this.env.AI\!, {
    model: "flux-general-multi",
    languageHints,
  });
}

The SDK code correctly puts model and language_hint into the input object passed to ai.run("@cf/deepgram/flux", input, { websocket: true }).

The problem

Despite the SDK changes, the STT still outputs romaji (English transliteration) instead of Japanese characters. For example, speaking Japanese produces "Konnichiwa." instead of "こんにちは".

Root cause

The Workers AI binding for @cf/deepgram/flux appears to not pass through model and language_hint to Deepgram's API. The Cloudflare docs list only encoding, sample_rate, eot_threshold, eager_eot_threshold, eot_timeout_ms, keyterm, mip_opt_out, tag, and request_id as supported input parameters — model and language_hint are not among them.

This makes sense: @cf/deepgram/flux likely maps internally to Deepgram's flux-general-en (English-only), and the binding strips unknown input fields rather than forwarding them as WebSocket query parameters.

For reference, Deepgram's native API accepts these as WebSocket query parameters:

wss://api.deepgram.com/v2/listen?model=flux-general-multi&language_hint=ja&encoding=linear16&sample_rate=16000

What needs to happen

PR #1459 solves the SDK layer (@cloudflare/voice), but the Workers AI binding layer also needs to:

1. Accept model as an input parameter and forward it to Deepgram's WebSocket URL as the model query parameter (defaulting to flux-general-en for backward compatibility)
2. Accept language_hint as an input parameter and forward it to Deepgram's WebSocket URL as the language_hint query parameter

Without the binding layer passing these through, the SDK-level changes in PR #1459 alone won't enable multilingual STT.

[whoiskatrin]

This one is on hold, waiting on Workers AI