Skip to content

[Draft] inworld tts auto mode #4655

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
ianbbqzy wants to merge 2 commits into
base: main
Choose a base branch
from
Open

[Draft] inworld tts auto mode #4655

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
0053464
inworld tts auto mode
Jan 29, 2026
0595314
always use auto mode for now
Jan 29, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions livekit-plugins/livekit-plugins-inworld/livekit/plugins/inworld/tts.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -355,6 +355,8 @@ async def _send_loop(self) -> None:
pkt["create"]["timestampType"] = opts.timestamp_type
if is_given(opts.text_normalization):
pkt["create"]["applyTextNormalization"] = opts.text_normalization
# Always enable auto_mode since we always use SentenceTokenizer
pkt["create"]["autoMode"] = True
Comment on lines +358 to +359
Copy link
Contributor

@coderabbitai coderabbitai bot Jan 29, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🔴 Critical

🧩 Analysis chain

🏁 Script executed:

wc -l "livekit-plugins/livekit-plugins-inworld/livekit/plugins/inworld/tts.py"

Repository: livekit/agents

Length of output: 133

🏁 Script executed:

sed -n '350,370p' "livekit-plugins/livekit-plugins-inworld/livekit/plugins/inworld/tts.py"

Repository: livekit/agents

Length of output: 1118

🏁 Script executed:

sed -n '1070,1100p' "livekit-plugins/livekit-plugins-inworld/livekit/plugins/inworld/tts.py"

Repository: livekit/agents

Length of output: 1339

🏁 Script executed:

rg "autoMode|auto_mode" "livekit-plugins/livekit-plugins-inworld/livekit/plugins/inworld/tts.py" -B 3 -A 3

Repository: livekit/agents

Length of output: 864

🏁 Script executed:

rg "create\[" "livekit-plugins/livekit-plugins-inworld/livekit/plugins/inworld/tts.py" -B 2 -A 5

Repository: livekit/agents

Length of output: 40

🏁 Script executed:

rg "version|Version|release|Release" "livekit-plugins/livekit-plugins-inworld/livekit/plugins/inworld/tts.py" -i | head -20

Repository: livekit/agents

Length of output: 802

🏁 Script executed:

rg "try|except|error|Error" "livekit-plugins/livekit-plugins-inworld/livekit/plugins/inworld/tts.py" -B 2 -A 2 | head -40

Repository: livekit/agents

Length of output: 1393

🏁 Script executed:

find "livekit-plugins/livekit-plugins-inworld" -name "*.py" -type f

Repository: livekit/agents

Length of output: 350

🏁 Script executed:

cat "livekit-plugins/livekit-plugins-inworld/livekit/plugins/inworld/version.py"

Repository: livekit/agents

Length of output: 658

🏁 Script executed:

cat "livekit-plugins/livekit-plugins-inworld/livekit/plugins/inworld/__init__.py"

Repository: livekit/agents

Length of output: 1447

🌐 Web query:

Inworld TTS WebSocket "autoMode" field API documentation release notes

💡 Result:

I couldn’t find any official Inworld TTS WebSocket API documentation or release-notes entry that defines an autoMode field.

What is documented for “automatic” behavior in the WebSocket TTS API is buffering + auto-flush via fields like bufferCharThreshold (start synthesis when buffered text reaches a threshold) and maxBufferDelayMs (time-based buffering), plus explicit flush_context messages. This is described in the WebSocket API reference and Inworld’s WebSocket/TTS announcement blog post. [1], [2]

Inworld’s published TTS release notes (as of the latest entry on January 21, 2026) don’t mention adding/renaming any WebSocket request field called autoMode. [3]

Sources
[1] Inworld API Reference — “Synthesize speech (WebSocket)” (voice:streamBidirectional)
[2] Inworld blog (June 11, 2025) — “Introducing timestamp alignment, WebSockets and more for Inworld TTS”
[3] Inworld Docs — TTS Release Notes

🏁 Script executed:

rg "autoMode|auto_mode" "livekit-plugins/livekit-plugins-inworld/" -r --type py

Repository: livekit/agents

Length of output: 518

🏁 Script executed:

rg "autoMode" "livekit-plugins/livekit-plugins-inworld/livekit/plugins/inworld/tts.py"

Repository: livekit/agents

Length of output: 110

🏁 Script executed:

rg "auto_mode" "livekit-plugins/livekit-plugins-inworld/livekit/plugins/inworld/tts.py"

Repository: livekit/agents

Length of output: 200

🏁 Script executed:

sed -n '340,365p' "livekit-plugins/livekit-plugins-inworld/livekit/plugins/inworld/tts.py"

Repository: livekit/agents

Length of output: 1478

🏁 Script executed:

rg "Inworld|API|compatibility" "livekit-plugins/livekit-plugins-inworld/livekit/plugins/inworld/tts.py" -i -B 2 -A 2 | head -50

Repository: livekit/agents

Length of output: 1514

Remove or conditionally gate undocumented autoMode field to prevent API failures.

The autoMode field is not documented in the Inworld TTS WebSocket API and the code comment ("To be removed once auto_mode is released") indicates it is a future/unreleased feature. Sending this unrecognized field unconditionally could cause stream creation failures if Inworld does not silently ignore unknown fields. Either:

  • Wait for official autoMode release and remove the comment, or
  • Make it conditional (similar to timestampType and applyTextNormalization which use is_given() checks), or
  • Add error handling to gracefully degrade if the field is rejected.
🤖 Prompt for AI Agents 
In `@livekit-plugins/livekit-plugins-inworld/livekit/plugins/inworld/tts.py`
around lines 358 - 359, The unconditional setting of pkt["create"]["autoMode"] =
True (in the TTS packet creation where SentenceTokenizer is used) should be
removed or gated to avoid sending an undocumented field; change this to follow
the existing pattern used for timestampType/applyTextNormalization (only set
autoMode if the corresponding config value is provided/is_given()), or otherwise
remove the line and/or wrap the send in graceful error handling to ignore server
rejections related to unknown fields; update the code around the packet
construction in the TTS send/create logic (the block that references
SentenceTokenizer and pkt["create"]) to implement the conditional check or
removal.

Copy link
Contributor

@cshape cshape Feb 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Auto mode will be released later this week, but this PR is good to merge as auto mode will be safely/silently ignored on our side before the release.

await self._ws.send_str(json.dumps(pkt))

elif isinstance(msg, _SendTextMsg):
Expand Down Expand Up @@ -1080,6 +1082,7 @@ async def _send_task() -> None:
for i in range(0, len(text), 1000):
connection.send_text(context_id, text[i : i + 1000])
self._mark_started()
# To be removed once auto_mode is released
connection.flush_context(context_id)
connection.close_context(context_id)

Expand Down
Loading