Skip to content

fix: update inworld tts to current api contract - #2277

Open
BenWeekes wants to merge 3 commits into
mainfrom
fix/inworld-tts-api
Open

fix: update inworld tts to current api contract#2277
BenWeekes wants to merge 3 commits into
mainfrom
fix/inworld-tts-api

Conversation

@BenWeekes

Copy link
Copy Markdown
Contributor

What

The inworld_tts_python extension is currently silent for everyone: the Inworld TTS API has drifted since the extension landed (#2010) and every synthesis request now fails.

Changes to inworld_tts.py:

  • Request fields renamed: voiceId/modelId (the previous voice/model are rejected with 400: voice_id can not be empty), and output format moved to audio_config (audio_encoding, sample_rate_hertz, verified honored at 16k/24k).
  • Streamed audio is now nested under result.audioContent in each NDJSON line; the old top-level paths are kept as fallbacks.
  • Each streamed chunk is a self-contained WAV; the RIFF header is stripped per chunk so raw PCM reaches the pipeline (previously this would have produced audible clicks).
  • Bad credentials now surface as HTTP 403 ("Invalid authorization credentials"); 401/403 are both treated as fatal invalid-key errors so the guarder -1000 contract holds.

Tests

  • New standalone suite under tests/ (dump, flush, params passthrough, error contracts incl. invalid key, TTFB metrics, robustness, state machine): 9 passed.
  • New tests/configs for the TTS guarder; full guarder run against the live API: 15 passed, 2 skipped (subtitle alignment, disabled by default).

All verified end-to-end in a voice-assistant graph (deepgram asr → openai llm → inworld tts) with audible output.

Ubuntu added 2 commits August 4, 2026 09:55
the service now requires voiceId/modelId (voice/model return 400
"voice_id can not be empty"), nests streamed audio under
result.audioContent, returns each streamed chunk as a self-contained
wav, and reports bad credentials as 403. send audio_config, strip the
riff header per chunk, and treat 401/403 as fatal invalid-key errors.
standalone suite covering dump, flush, params passthrough, error
contracts, ttfb metrics, robustness, and state transitions, plus
tests/configs for the tts guarder. guarder run: 15 passed, 2 skipped
(subtitle alignment, disabled by default).
@github-actions

github-actions Bot commented Aug 5, 2026

Copy link
Copy Markdown

Review

Good, well-scoped fix. The four API changes are clearly explained in the description and each one maps to a specific diff hunk, and keeping the external param names (voice, model, sample_rate) while only renaming the wire fields is the right call — no user-facing config breaks. The version bump is applied consistently to both manifest.json and pyproject.toml.

Two things I'd want addressed before merge, then some smaller notes.

1. Wrong env var in the new test configs (blocking)

All three guarder configs use ${env:INWORLD_TTS_API_KEY}:

"api_key": "${env:INWORLD_TTS_API_KEY}",

But everywhere else in the repo the key is INWORLD_API_KEYproperty.json, README.md, and .env.example:191. INWORLD_TTS_API_KEY appears nowhere else. Anyone who follows the README and then runs task tts-guarder-test EXTENSION=inworld_tts_python gets an empty key and a confusing 403 rather than a clear "not configured". Either rename to INWORLD_API_KEY in the three configs, or add the new name to .env.example — the former is less churn.

2. The fix itself has no test coverage (blocking-ish)

Every one of the ~1500 new test lines patches inworld_tts_python.extension.InworldTTSClient wholesale:

@patch("inworld_tts_python.extension.InworldTTSClient")
def test_params_passthrough(MockInworldTTSClient):

So the suite exercises the base-class plumbing (dump, flush, TTFB, state machine, error codes) but never inworld_tts.py — the only file this PR actually changes. None of voiceId/modelId, audio_config, result.audioContent, the RIFF strip, or the 403 branch is covered. That's how #2010 drifted silently in the first place, and this PR leaves the same hole open.

Worth adding one focused test against InworldTTSClient.get() with a stubbed aiohttp response that asserts:

  • the posted JSON contains voiceId / modelId / audio_config.sample_rate_hertz
  • a {"result": {"audioContent": <b64 of a small WAV>}} NDJSON line yields the PCM payload with the 44-byte header gone
  • status 403 yields INVALID_KEY_ERROR

That's the part that would have caught this regression, and it needs no live credentials.

3. RIFF strip silently forwards the header on a miss

if chunk[:4] == b"RIFF":
    data_pos = chunk.find(b"data", 12)
    if data_pos != -1:
        chunk = chunk[data_pos + 8 :]

If data_pos == -1 the whole RIFF blob — header and all — falls through to the byte-alignment code and gets yielded as PCM, which is exactly the audible click the change is meant to remove, just in the failure case. Since you've already established the chunk claims to be a WAV, a miss is a contract violation worth a log_warn at minimum, and arguably worth skipping the chunk.

Also, find(b"data", 12) scans for the literal bytes anywhere past offset 12 rather than walking the RIFF chunk list, so a LIST/INFO chunk containing those bytes would produce a bad offset. Low probability with a fixed-format vendor stream, but stepping chunk-by-chunk (read 4-byte id + 4-byte size, skip) is only a few lines more and can't mis-fire.

4. Mixed camelCase / snake_case in the payload

"voiceId": ...,
"modelId": ...,
"audio_config": {"audio_encoding": ..., "sample_rate_hertz": ...},

You've verified this works, and proto-JSON gateways do accept both spellings — but relying on that leaves the request half in each style. If the endpoint accepts audioConfig/audioEncoding/sampleRateHertz, using those makes the payload self-consistent and less exposed if the gateway ever tightens its field matching. If snake_case is what the docs actually specify for this nested message, a one-line comment saying so would save the next reader the same question.

Smaller notes

  • data.get("audio", {}).get("content") still throws AttributeError if the key is present but null. You guarded exactly this for result with (data.get("result") or {}) — worth the same treatment for consistency (it's caught by the outer handler today, but surfaces as a generic ERROR rather than a parse skip).
  • tests/test_state_machine.py config carries params from another vendor: "speaker": "celeste", "modelId": "arcana", "lang": "eng", "samplingRate": 16000. Harmless with a mocked client, but modelId as a params key reads as if it were meaningful when the extension looks up model — misleading for whoever edits this next. Suggest trimming to the params Inworld actually takes.
  • Same file uses eval(payload) to parse a JSON string. It matches the sibling test_state_machine.py files so I understand where it came from, but it breaks on any true/false/null in the payload and json.loads is a drop-in. Fine to leave if you'd rather stay consistent with the siblings.
  • tests/test_basic.py puts the sys.path manipulation and imports above the license header, and imports pathlib.Path twice. Several imports look unused (filecmp, shutil, threading, TTSFlush). .pylintrc has ignore=...tests... so CI won't flag it, but black does check these paths — worth a task format && task check pass.
  • tests/bin/start is missing a trailing newline.

Confirmed good

  • 401 and 403 both routed to INVALID_KEY_ERROR, so the guarder -1000 contract holds, and folding the status into the message (Invalid API key (HTTP 403)) makes the logs diagnosable.
  • RIFF strip is correctly placed before the odd-byte alignment cache, so the carry logic still sees a continuous PCM stream.
  • No credential material added to any log line; config.to_str() encryption path is untouched.
  • Commit messages are conventional, hard-wrapped, and free of tool attribution.

I have not run the suite or hit the live API — the notes above are from reading the diff against main, config.py/extension.py, and the repo's lint/format config.

@BenWeekes

Copy link
Copy Markdown
Contributor Author

Guarder test result (run against the live Inworld API):

$ task tts-guarder-test EXTENSION=inworld_tts_python
================== 15 passed, 2 skipped in 345.05s (0:05:45) ===================

The 2 skips are the subtitle-alignment tests, disabled by default for all TTS extensions.

Standalone extension suite:

$ task test-extension EXTENSION=agents/ten_packages/extension/inworld_tts_python
============================== 9 passed in 1.28s ===============================

Version bumped to 0.1.2 in both manifest.json and pyproject.toml.

@BenWeekes

Copy link
Copy Markdown
Contributor Author

Guarder test result (verbose run against the live Inworld API):

tts guarder result

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant