feat(widget): add self-hosted orchestrator support via orchestrator-url attribute

[a1anfan]

Stacked on #921. Adds self-hosted orchestrator support to the Convai widget: setting the new orchestrator-url attribute connects the widget to a self-hosted orchestrator instead of the ElevenLabs cloud, and the optional orchestrator-agent-config attribute carries the exported agent configuration JSON (both the agent_config_dict/tools_config_list and agent_config/tools_config key spellings are accepted, plus an optional top-level bedrock_inference_profile). The parsed config feeds the client SDK's orchestrator session config from #921; the connection is forced to websocket, since self-hosted orchestrators only expose the conversation WebSocket.

Because self-hosted deployments have no HTTP API to serve a widget appearance config, orchestrator sessions skip the config fetch entirely and use a built-in default appearance (transcript and text input enabled), which the existing attributes and override-config can still override. The file upload button stays hidden by default, so no request leaves the customer network. Both changes are inert unless orchestrator-url is set; cloud behavior is untouched.

Testing

• unit tests
• build, lint, check-types across widget-core and widget-embed
• browser e2e and manual testing against a deployed orchestrator

---

Note

Medium Risk
Changes core session and config wiring and how connections are established; mistakes could break cloud sessions or mis-route self-hosted traffic, though cloud paths stay gated behind orchestrator-url.

Overview
Adds experimental self-hosted orchestrator support to the Convai widget via new orchestrator-url and optional orchestrator-agent-config attributes (minor bump in changeset). When orchestrator-url is set, it takes precedence over agent-id / signed-url, skips the ElevenLabs widget config HTTP fetch, and builds a websocket-only client orchestrator session from parsed export JSON (parseOrchestratorConfig, with unit tests).

Orchestrator mode uses a built-in default appearance (still overridable via existing attributes / override-config), exposes languageOverride for session agent language when overrides apply, and turns off cloud-only UI (file upload and end-of-call feedback). OrchestratorProvider is wired into the main widget and dev playground trees.

^{Reviewed by Cursor Bugbot for commit afcc6d2. Bugbot is set up for automated code reviews on this repo. Configure here.}

[a1anfan]

cursor review

[Copilot]

Pull request overview

Adds on-prem deployment support for the ConvAI widget by introducing new attributes that route sessions to a self-hosted orchestrator WebSocket and by providing a built-in default widget appearance when no HTTP config endpoint exists.

Changes:

• Add on-prem-url and on-prem-agent-config attributes and wire them into widget/session config providers.
• Introduce parseOnPremConfig (with unit tests) to map exported agent JSON into the client SDK’s onPremConfig.
• Add a changeset to release updated widget packages with the new on-prem functionality.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
packages/convai-widget-core/src/utils/parseOnPremConfig.ts	New helper to parse on-prem agent JSON into OnPremConfig for the client SDK.
packages/convai-widget-core/src/utils/parseOnPremConfig.test.ts	Unit tests covering expected key mappings and invalid JSON handling.
packages/convai-widget-core/src/types/attributes.ts	Adds the new on-prem custom attributes to the allowed attribute list.
packages/convai-widget-core/src/contexts/widget-config.tsx	Skips HTTP widget config fetch in on-prem mode and uses a built-in default appearance config.
packages/convai-widget-core/src/contexts/session-config.tsx	Creates onPremConfig session configs and forces websocket connection type when on-prem-url is set.
.changeset/olive-poems-brake.md	Releases convai-widget-core and convai-widget-embed with on-prem support changes.

Suppressed comments (1)

packages/convai-widget-core/src/utils/parseOnPremConfig.ts:31

• on-prem-agent-config values are parsed from a string attribute and then forwarded into OnPremConfig fields (agentConfig, toolsConfigList, etc.). The current implementation forwards whatever types are present, which can produce invalid wire payloads (e.g., tools_config_list being an object instead of an array) and hard-to-debug orchestrator errors. Consider validating/coercing the expected shapes and dropping invalid fields instead of passing them through.

  try {
    const parsed = JSON.parse(agentConfigJSON);
    return {
      conversationUrl,
      agentConfig: parsed.agent_config_dict ?? parsed.agent_config ?? undefined,

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[a1anfan]

cursor review

[Copilot]

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated no new comments.

Suppressed comments (1)

packages/convai-widget-core/src/contexts/session-config.tsx:124

• languageAttribute.value is force-cast to the client SDK Language type. Since HTML attributes are free-form strings, this can pass invalid language codes through to the SDK/orchestrator and cause hard-to-diagnose session-start failures.

Prefer validating the attribute (e.g., using the existing isValidLanguage helper in src/types/languages) and only setting overrides.agent.language when it’s valid; otherwise omit the field (and optionally console.warn that the value was ignored).

          agent: {
            ...overrides.value?.agent,
            language: (languageAttribute.value as Language) || undefined,
          },

[a1anfan]

cursor review

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 58c167c. Configure here.}

[Copilot]

Pull request overview

Copilot reviewed 24 out of 24 changed files in this pull request and generated no new comments.

Suppressed comments (1)

packages/convai-widget-core/src/utils/parseOrchestratorConfig.ts:32

• parseOrchestratorConfig normalizes http(s) to ws(s) but does not validate that the resulting url is actually a valid ws:///wss:// URL (e.g. ftp://... or a malformed value will pass through and fail later during WebSocket connection). Consider validating with new URL(...) and returning null with a clear error when the protocol is not ws/wss or the URL cannot be parsed.

  const url = rawUrl
    .replace(/^https:\/\//, "wss://")
    .replace(/^http:\/\//, "ws://");

  if (!agentConfigJSON) {
    return { url };
  }

[a1anfan]

cursor review

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 2ad172b. Configure here.}

[a1anfan]

cursor review

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 9e21fd1. Configure here.}

[kalvin807]

logic make sense. not a fan of this structure (session-config is already too long and complex). can we isolate orchestrator logic into a context?

import { computed, ReadonlySignal, useSignalEffect } from "@preact/signals";
import { ComponentChildren } from "preact";
import { createContext, useMemo } from "preact/compat";
import type { OrchestratorConfig } from "@elevenlabs/client";

import { useAttribute } from "./attributes";
import { useContextSafely } from "../utils/useContextSafely";
import { parseOrchestratorConfig } from "../utils/parseOrchestratorConfig";

const OrchestratorContext =
  createContext<ReadonlySignal<OrchestratorConfig | null> | null>(null);

export function OrchestratorProvider({ children }: { children: ComponentChildren }) {
  const url = useAttribute("orchestrator-url");
  const agentConfig = useAttribute("orchestrator-agent-config");
  const agentId = useAttribute("agent-id");
  const signedUrl = useAttribute("signed-url");

  const value = useMemo(
    () =>
      computed(() =>
        url.value ? parseOrchestratorConfig(url.value, agentConfig.value) : null
      ),
    []
  );

  useSignalEffect(() => {
    if (url.value && (agentId.value || signedUrl.value)) {
      console.warn(
        "[ConversationalAI] orchestrator-url takes precedence; agent-id and signed-url are ignored"
      );
    }
  });

  return (
    <OrchestratorContext.Provider value={value}>{children}</OrchestratorContext.Provider>
  );
}

export function useOrchestrator() {
  return useContextSafely(OrchestratorContext);
}

export function useIsOrchestratorSession() {
  const orchestrator = useOrchestrator();
  return useComputed(() => orchestrator.value !== null);
}

[kalvin807]

also tagging @giannagerton for widget related changes

[a1anfan]

cursor review

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 266a924. Configure here.}

[Copilot]

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated no new comments.

Suppressed comments (2)

packages/convai-widget-core/src/utils/parseOrchestratorConfig.ts:55

• OrchestratorConfig.url is documented as a WebSocket URL, but parseOrchestratorConfig currently accepts any non-empty string (e.g. example.com, ftp://...) and will pass it through to the client. Validate that the normalized URL starts with ws:// or wss:// and fail early with a clear error.

  const url = rawUrl
    .replace(/^https:\/\//, "wss://")
    .replace(/^http:\/\//, "ws://");

packages/convai-widget-core/src/contexts/orchestrator-config.tsx:44

• orchestrator-url is treated as enabled for any truthy string, including whitespace. That can inadvertently switch the widget into orchestrator mode while passing an empty/invalid URL into parseOrchestratorConfig, resulting in a broken session config and skipped cloud config fetch. Trim the attribute and base enabled/config on the trimmed value.

  const value = useMemo(
    () => ({
      enabled: computed(() => !!url.value),
      config: computed(() =>
        url.value ? parseOrchestratorConfig(url.value, agentConfig.value) : null

[a1anfan]

both done, thanks for the push - this ended up much cleaner

orchestrator context (266a924): went with your shape - OrchestratorProvider / useOrchestrator in orchestrator-config.tsx, mounted above WidgetConfigProvider, and the widget-config upload/feedback gates consume it too. one deviation: the context exposes enabled (attribute set) separately from the parsed config, because deriving "orchestrator session" from config-non-null would let an orchestrator url with an invalid agent config fall through to a cloud session when agent-id is also set.

language override (732f553, e1345ec): your one-liner drops the picker and browser matching (language.value already folds in override-language → picker → browser), and cursor caught two divergence bugs in my first simplification, both from the same root: for orchestrator sessions the widget appearance default isn't the agent's real language. final state is simpler than both: language-config exposes one languageOverride signal - "send the resolved language iff override-language is set or a supported set is declared", i.e. whenever the picker is visible the conversation follows it. no languageChosen flag, survives reloads, and bare boilerplate language="en" still never overrides the agent config. session-config is down to language: languageOverride.value.

one behavior note: with a declared supported set the override is now also sent on first load (previously silent until something diverged). this is deliberate because the picker was already showing a language the agent wasn't speaking

[kalvin807]

lgtm

[cursor]

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

^{Reviewed by Cursor Bugbot for commit afcc6d2. Configure here.}