Flux stopped detecting speech mid-call, twice, with a healthy connection both times

[michaelgrafwebdev]

We run Flux on inbound phone calls over Twilio Media Streams. Twice now, two weeks apart, Flux has stopped detecting speech partway through a call and never recovered.

The strange part is that nothing looks broken. The socket stays open, our audio keeps flowing at a steady 50 packets a second, and Flux keeps sending Updates the entire time. They just stop containing anything, and no StartOfTurn or EndOfTurn ever fires again. Both callers kept talking to a silent agent until they gave up and hung up. This is production traffic, so one of those was a real customer.

I've gone as far as I can from our side. I think the rest of the answer is in your request log.

What we see

Most recent occurrence, request_id 019f9641-64bb-74c2-add4-1cd51a625bdf:

• Flux connected and worked normally for the first ~26 seconds
• At +30.8s the caller speaks a short answer (a first name) in response to a question from our agent
• From +30.8s to +45.4s we receive 113 Flux messages, every one of them a TurnInfo / Update carrying an empty transcript, with no StartOfTurn, no EndOfTurn, and no text at any point
• Caller hangs up at +45.5s

Message sizes make the empty transcripts unambiguous: those 113 messages were 231-232 bytes each, against 412-963 bytes for real transcript-carrying Updates earlier on the same connection.

The earlier call showed the same pattern: normal for about four seconds, then nothing for seventeen seconds, then the caller gave up. Across our production calls, healthy ones run 0.91 to 4.00 Deepgram events per second. These two sit at 0.40 and 0.42.

What we ruled out on our side

The audio bridge was healthy. We forwarded 2,274 packets at a steady 50/sec for the full call, 160 bytes each (mulaw, 8kHz, 20ms frames). At the 30-second mark we had sent exactly 1,500 packets, which is 30.0s of audio against a 30.002s wall clock. No backlog, no drops.

The WebSocket never dropped. Our heartbeat logged alive at 30s, 35s, 40s and 45s, with the gap since the last received message between 92ms and 211ms throughout. Flux was talking to us the entire time.

There were no errors anywhere: no Flux Error message, no WebSocket error, no exception on our side. Neither call overlaps an incident on your status page.

One detail I can't explain

Between +30.8s and +34.6s the message cadence tripled, from ~240ms to ~75ms, which is faster than the documented "roughly every 0.25 seconds of transcribed audio". That looks like a backlog being drained, but our side shows no backlog at all: our forwarding stayed locked to the wall clock the whole time. So whatever queued up appears to have queued on your side.

Questions

1. What does an empty-transcript Update at a steady ~240ms cadence with no StartOfTurn actually indicate? Reading the docs, that looks like the Initial/Ready state emitting keepalive-style Updates, which would mean Flux never registered speech onset for 19 seconds despite receiving audio the whole time. Is that reading right?

2. Why didn't eot_timeout_ms fire? We have it at 5000ms, and over a 19-second dead zone I expected the silence-based backup to produce something. Is eot_timeout_ms only armed once a turn has already started, so it can't help when the failure is that speech onset was never detected in the first place?

3. Is there anything in the request log for 019f9641-64bb-74c2-add4-1cd51a625bdf that shows why speech onset wasn't detected? That's the piece I can't get to from our side.

4. We send 20ms audio chunks (one Twilio frame straight through, 160 bytes) rather than the recommended 80ms. Could that contribute to this? I'm willing to buffer to 80ms regardless, but I'd like to know whether it's actually implicated before I treat it as the fix.

5. Does keyterm volume or quality affect turn detection? We were sending 55 keyterms on this connection and some were poor quality, internal document titles rather than plausible speech. I can see that hurting recognition. I want to know whether it can affect speech-onset detection specifically.

Config

model:               flux-general-en
encoding:            mulaw
sample_rate:         8000
eot_threshold:       0.7
eager_eot_threshold: 0 (disabled, not sent)
eot_timeout_ms:      5000

One long-lived connection per call, opened at call start and kept open for the whole conversation. We never reconnect mid-call.

I can provide the Project ID and full message dumps, or run a controlled reproduction on staging with verbose logging.

[deepgram-community[bot]]

Thanks for asking your question. Please be sure to reply with as much detail as possible so the community can assist you efficiently.
_{Consider joining our Discord community for more opportunity to engage with your fellow Deepgram users. You can earn points which can be redeemed for cool stuff by being active in our communities!}

[deepgram-community[bot]]

Hey there! It looks like you haven't connected your GitHub account to your Deepgram account. You can do this at https://community.deepgram.com - being verified through this process will allow our team to help you in a much more streamlined fashion.

[deepgram-community[bot]]

It looks like we're missing some important information to help debug your issue. Would you mind providing us with the following details in a reply?

• The programming language you are working in (e.g. JavaScript, Python).

[michaelgrafwebdev]

TypeScript on Node.js v24, connecting with a raw ws WebSocket (8.18.3) directly to wss://api.deepgram.com/v2/listen rather than through the JS SDK. We do have @deepgram/sdk 4.9.1 in the project, but it only drives our Nova-3 path, so nothing in the Flux connection above goes through it.

Project ID: f6240c14-1d82-4605-bf77-9bed00c027a3

Worth noting our staging and production traffic share one API key, so requests around that request_id will be a mix of both environments.

[michaelgrafwebdev]

is that sufficient information to draw any conclusions? If you're listening bot, you can ask for complicated details and I will attempt to provide them

[deepgram-community[bot]]

1. What does an empty-transcript Update at a steady ~240ms cadence with no StartOfTurn actually indicate? Reading the docs, that looks like the Initial/Ready state emitting keepalive-style Updates, which would mean Flux never registered speech onset for 19 seconds despite receiving audio the whole time. Is that reading ...
   Update messages are sent for every ~240ms of processed audio, whether or not any words were transcribed, regardless of an ongoing turn or not.

do you have request IDs for streams which you saw this behavior of missed StartOfTurn events?

This message was sent by nick kaimakis from Deepgram, via our community automation.

[michaelgrafwebdev]

Thanks for coming back on this, and for the answer to question 1. That one actually settles something on our end, so it was worth the wait.

One thing worth saying up front: I use AI tooling to draft these write-ups. I mention it because it can read a little clinical, and I would rather you know why than wonder about it. The upside for you is that every figure below comes straight out of our production logs and our internal incident writeup rather than from memory, so the timestamps, packet counts and message counts are exact rather than approximate. I have checked them against those same logs. If something looks wrong, it is my mistake and I will correct it.

Same stream I opened with:

request_id: 019f9641-64bb-74c2-add4-1cd51a625bdf
project: f6240c14-1d82-4605-bf77-9bed00c027a3

If you can pull the request log for that stream, that is the one piece I have no way to see from my side. From +30.8s to +45.4s of the call we received 113 Updates, every one of them carrying an empty transcript. No StartOfTurn, no EndOfTurn, no text. The caller was talking through that entire window, and our bridge kept forwarding audio at a steady 50 packets per second.

The one I most need answered is question 2. We run eot_timeout_ms at 5000. Across a 19 second dead zone, with audio arriving the whole time, it never fired. Is eot_timeout_ms armed only after StartOfTurn?

If it is, then it cannot help when the failure is that speech onset was never detected in the first place, and no config change on our side will mitigate this. A straight yes or no there would save us a lot of tuning against something that cannot move, and would tell us to build detection and recovery instead.

I would also like to check that I read your answer to question 1 correctly. If Updates fire roughly every 240ms regardless of transcription or turn state, then an empty Update stream looks identical in these two cases:

1. Flux is healthy and waiting on a quiet caller.
2. Flux is receiving audio but not registering a caller who is speaking.

If that is right, we have no way to tell those apart from the client side, and the caller's own audio energy is the only signal left to us. That is what we have built our own detection on, so a confirmation either way would help.

Three smaller ones, whenever you get to them:

3. Does the request log for that stream show anything about why onset was not detected?

4. We send 20ms chunks, one 160 byte Twilio mulaw frame straight through, rather than the recommended 80ms. Could that be implicated? We will buffer to 80ms regardless, but knowing whether it is the actual fix or just good hygiene changes how much we build around it.

5. We had 55 keyterms on that connection, and some were poor quality, internal document titles rather than anything a caller would realistically say. I can see that hurting recognition. Can it affect speech onset detection specifically?

There is also a second occurrence, an inbound customer call fourteen days earlier that went deaf about four seconds in and stayed that way for seventeen. Its request ID is sitting in a rotated production log. Happy to pull it if a second stream would help you narrow this down.

Appreciate you taking a look at this.

[deepgram-community[bot]]

I'm unable to look at the audio on our end and try to reproduce. do you have the original audio file or another reproducer?

it is not expected that Flux would not transcribe audible speech, but we'll need the audio to verify and dig in.

20ms chunks are fine. you'll just see no latency benefit going lower than 80ms chunks.

your keyterms certainly have room for improvement, but that should not lead to missed speech.

again we'll need to get the original audio to dig in further.

This message was sent by nick kaimakis from Deepgram, via our community automation.

[michaelgrafwebdev]

Nick, you asked for the original audio or a reproducer. A new production occurrence last night gives me both.
dg1652-reproducer.zip

The audio is attached: deploy-gate-test.raw (mulaw, 8kHz mono, 16,000 bytes = 2.0s), plus the same samples wrapped as WAV. It says "This is a deploy validation test." Our synthetic health probe streams these exact bytes over a live Flux session once an hour, followed by about 3 seconds of mulaw silence frames, then no further audio while the socket stays open. We close at 45s if the pipeline hasn't completed. Same bytes on every run.

The stalled session. 2026-08-15 06:40:26 UTC, project f6240c14-1d82-4605-bf77-9bed00c027a3, request_id 01a00426-7fcd-77d3-a8d2-6a9ce451197e:

+0.204s  Connected (handshake 204ms)
+0.524s  TurnInfo messages begin
+1.023s  StartOfTurn (turn 0)
+1.769s  Update: "the deploy validation test."
+2.0s..+4.7s  12 more Updates, transcript unchanged
+4.721s  last message we ever received
+45s     we close the session; EndOfTurn never arrived

Connect params: model=flux-general-en&encoding=mulaw&sample_rate=8000&eot_threshold=0.7&eot_timeout_ms=5000, 12 keyterms.

The control. Identical bytes 4 minutes later, same project: request_id 01a00429-f48e-7481-8e49-299dcfba2d38. EndOfTurn arrived within seconds, confidence 0.71, transcript exactly right: "This is a deploy validation test." The hourly runs before and after passed too. Same input, intermittent stall.

Two details:

1. This narrows my Q2. The turn was active (StartOfTurn had fired), and eot_timeout_ms=5000 still never produced an end of turn across the remaining 40 seconds. Does that timer advance on wall clock or on processed audio? If it counts audio time, note we only stream ~3s of trailing silence, so a 5000ms timeout would be unreachable in this injection pattern. That alone can't explain the miss, though: the acoustic endpoint closes the turn within seconds on every healthy run with the same trailing silence.

2. Your note said it's not expected that Flux would fail to transcribe audible speech. This occurrence is sharper than that: it DID transcribe (the Updates carry the sentence), it just never closed the turn. One more oddity for whoever digs in: the stalled run heard the opening as "the" where every healthy run hears "This is a", from the same bytes.

Thanks for the 20ms chunk answer, that settles my Q4.

[deepgram-community[bot]]

in this request, it looks like you stopped sending audio after ~4s of audio, which is why you stopped receiving messages from Flux, and never received an EndOfTurn.

This message was sent by nick kaimakis from Deepgram, via our community automation.

[michaelgrafwebdev]

Thanks Nick, that explains it. The test harness does stop streaming about 4.7 seconds in (2s of speech plus 3s of silence), so with eot_timeout_ms=5000 the timeout could never fire on that request. That one's on our reproducer, not on Flux. It also clears up something I'd been trying to pin down for a while: end-of-turn timing runs on processed audio, not wall clock, so the client has to keep streaming silence for EndOfTurn to show up.

I've added instrumentation on my side, deployed to production today. I've been using Anthropic's Claude Fable 5 to debug this, and it built out the new forensic layer: the test harness now streams continuous silence until EndOfTurn arrives, and production captures outbound audio plus per-event timelines, so the next occurrence will come with the exact audio we sent and a full event log. The July incidents (request 019f9641-64bb-74c2-add4-1cd51a625bdf) still puzzle me since audio was flowing continuously there, but I'd rather come back with something you can actually dig into than keep going back and forth on partial data.

Thanks for sticking with this. Next time it happens you'll get the whole picture, and if I figure it out on my side first I'll post the answer here so the next person who hits this has a trail to follow.