Skip to content

TEL-562: improve correctness of media timeout - #683

Merged
hechen-eng merged 2 commits into
mainfrom
media-timeout
May 14, 2026
Merged

TEL-562: improve correctness of media timeout#683
hechen-eng merged 2 commits into
mainfrom
media-timeout

Conversation

@hechen-eng

@hechen-eng hechen-eng commented May 12, 2026

Copy link
Copy Markdown
Contributor

Three improvements:

  1. When lastPacketTime was zero (no packet ever received), time.Since(lastPacketTime) overflowed time.Duration and logged as ~9.22e9s. Now sinceLast is 0 when there has been no packet.
  2. sinceLast exceeding the general timeout when isInitial:false. When media flowed briefly during the initial window and then stopped, the loop slept until the full initial deadline (e.g. 30s) before re-evaluating, at which point lastPacketTime was already older than the general timeout (e.g. sinceLast=20.3s with timeout=15). Fixed by capping the timer arm at the general timeout. This is not a regression. The previous media timeout mechanism has the same problem, e.g. last packet arrives at 9.7s then stopped. The first ticker triggers at 15s and then again at 15s. The "real" timeout is 20.3s.
  3. isInitial flipping back to true mid-call after SetTimeout. isInitial := lastPacketTime.Before(startTime) was relative to the most recently re-armed startTime, so the no-ACK fallback (SetTimeout(min(3s, MediaTimeoutInitial), MediaTimeout)) caused calls that had already received RTP to be re-classified as initial and killed by the shortened 3s window, producing logs like isInitial:true, packets:38, sinceLast:5.16s. Changed to isInitial := lastPacketTime.IsZero(). isInitial now means "no media has ever been received on this port." Policy change: once media is flowing, the no-ACK fallback no longer triggers the 3s shortcut. The call is killed only after the general 15s window of silence from the last packet. Signaling faults no longer aggressively tear down calls whose media is healthy.

datadog log: https://app.datadoghq.com/logs?query=%22triggering%20media%20timeout%22%20region%3Aofrankfurt1b&agg_m=count&agg_m_source=base&agg_t=count&clustering_pattern_field_path=message&cols=host%2Cservice%2C%40sinceLast%2C%40isInitial%2Cimage_tag%2C%40sinceStart%2C%40timeout&messageDisplay=inline&refresh_mode=paused&storage=hot&stream_sort=desc&viz=stream&from_ts=1778596320000&to_ts=1778610600000&live=false

@hechen-eng
hechen-eng requested a review from a team as a code owner May 12, 2026 18:32
@hechen-eng
hechen-eng merged commit d3e37ef into main May 14, 2026
6 checks passed
@hechen-eng
hechen-eng deleted the media-timeout branch May 14, 2026 16:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants