Skip to content

fix(cli): read the WAV in JS so Built-in transcription runs inside Electron - #5129

Merged
miguel-heygen merged 2 commits into
mainfrom
fix/cli-sherpa-wav-in-js
Oct 6, 2026
Merged

miguel-heygen merged 2 commits into
mainfrom
fix/cli-sherpa-wav-in-js

Conversation

@miguel-heygen

@miguel-heygen miguel-heygen commented Oct 6, 2026 •

Copy link
Copy Markdown
Collaborator

What

Built-in transcription (Parakeet through sherpa-onnx) fails in the packaged Desktop app, seen on 0.8.138. The worker has called readWave since 0.8.82; Desktop only recently started running it. The decode worker read the audio with sherpa-onnx's readWave, whose sample array points at memory outside V8. Desktop runs the CLI as ELECTRON_RUN_AS_NODE, and Electron's V8 memory cage refuses such arrays: readWave throws External buffers are not allowed. Plain Node allows them, which is why it only broke in the app.

The worker now reads the WAV in JS. The CLI always hands it a prepared 16-bit mono PCM WAV, so it parses that into a Float32Array and never calls readWave. The recognizer, acceptWaveform and decoding already worked under Electron with samples from JS.

  • New whisper/wav.ts holds the RIFF chunk scan, moved from transcribe.ts, which keeps using it, and readWav. A file that is not a 16-bit mono PCM WAV fails with a message naming it. An empty WAV, which ffmpeg writes for silent or zero-length input, reads as no samples, as before.
  • readWave(path, false) would also copy into V8 memory. A JS reader also reads WAVs readWave cannot, such as piped WAVs (where readWave returns no samples), and uses less peak memory. The engine has its own private readWav for audio effects (audioFxRender.ts); it is not exported, so the CLI keeps its own.
  • The worker tests' stand-in sherpa-onnx now throws Electron's exact error from readWave. If the worker ever calls it again, the tests fail. They write real WAV files instead of faking the reader.

Checks

  • Packaged app, Linux, at d02fa46. The commit after it only fixes how an empty WAV is read. I took the Desktop release build that showed the failure (build aea76732e) and ran its own CLI exactly as Desktop does: the app binary with ELECTRON_RUN_AS_NODE=1 and the bundled ffmpeg, running transcribe jfk.wav --engine parakeet --json, with the real Parakeet model and a fresh home directory.
    • Before, the shipped 0.8.138 CLI: exit 1, Parakeet failed: Parakeet decoder exited with code 1: External buffers are not allowed.
    • After, the same app with this branch's CLI bundle swapped into an extracted copy of app.asar (the worker now has no readWave call): exit 0, ok: true, 22 words, "And so, my fellow Americans, ask not what your country can do for you. Ask what you can do for your country."
  • Tests: wav.test.ts, sherpaWorker.test.ts, sherpa.test.ts, both transcribe test files and the command test pass 151/151, 3 runs in a row.
  • Putting the readWave call back fails 7 tests with External buffers are not allowed.
  • Typecheck, lint, format, comment checks and fallow pass.

@miguel-heygen
miguel-heygen marked this pull request as ready for review October 6, 2026 21:36
@github-actions

github-actions Bot commented Oct 6, 2026

Copy link
Copy Markdown

Edit accuracy: accurate 2059 (base branch 2059), smooth 1561 of those

The gate passes.
Smoothness is reported in the artifact, not gated. A case fails only if it fails 2 of 3 runs.

Quarantined, measured but not gated (0)

@jrusso1020 jrusso1020 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving at 812e5b4.

  • The root cause matches the description. sherpaWorker.ts was the only readWave caller in the repo (code search), and it now reads the WAV in JS. The recognizer still gets the same {samples, sampleRate} shape.
  • readWav checks RIFF/WAVE, PCM or EXTENSIBLE, mono and 16-bit. findWavChunk clamps the data size to the buffer, so a piped WAV with a placeholder size still reads.
  • The < to <= change in the chunk scan fixes the empty-WAV case. detectSpeechOnset in transcribe.ts gets the same fix, since it now shares the scanner rather than keeping its own copy.
  • Reuse: the engine's readWav in audioFxRender.ts is private, so a CLI-local reader is reasonable. If a third caller appears, export one from a shared spot.
  • The stand-in sherpa now throws Electron's exact error from readWave, so a regression fails 7 tests. That's the right guard.

Non-blocking: the per-sample readInt16LE loop is fine for typical clips. An Int16Array view (when the data offset is even) would be faster on hour-long audio.

Review by Rames

@miguel-heygen
miguel-heygen added this pull request to the merge queue Oct 6, 2026
Merged via the queue into main with commit f7bba22 Oct 6, 2026
85 of 86 checks passed
@miguel-heygen
miguel-heygen deleted the fix/cli-sherpa-wav-in-js branch October 6, 2026 23:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants