Conversation
…nprivileged range Devin review feedback on PR livekit#811: 1. inbound drainOnHangup: a remote CANCEL before the call is answered hit the 'cancelled' branch and slept the drain window with no media bridged. Gate the drain on the media actually being bridged (c.started broken) and drop 'cancelled' from the local-hangup set. Remote CANCEL is not a local hangup. 2. newTestMediaPort bound a random even port in 1-65535, which can land on a privileged port (<1024) and fail in CI with 'bind: permission denied' (seen on TestSetOfferReportsUnknownProvider). Constrain the test range to 10000-20000.
|
Addressed the Devin review finding (cancelled inbound call delayed): |
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #811 +/- ##
==========================================
+ Coverage 65.25% 69.25% +3.99%
==========================================
Files 51 43 -8
Lines 6588 8736 +2148
==========================================
+ Hits 4299 6050 +1751
- Misses 1915 2163 +248
- Partials 374 523 +149 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
…nprivileged range Devin review feedback on PR livekit#811: 1. inbound drainOnHangup: a remote CANCEL before the call is answered hit the 'cancelled' branch and slept the drain window with no media bridged. Gate the drain on the media actually being bridged (c.started broken) and drop 'cancelled' from the local-hangup set. Remote CANCEL is not a local hangup. 2. newTestMediaPort bound a random even port in 1-65535, which can land on a privileged port (<1024) and fail in CI with 'bind: permission denied' (seen on TestSetOfferReportsUnknownProvider). Constrain the test range to 10000-20000.
a871b74 to
16668d2
Compare
Review feedback on livekit#811: - inbound/outbound drain now uses select{ctx.Done, time.After} instead of time.Sleep so a context cancellation cuts the drain window short (inbound close() runs under WithoutCancel, so this preserves the full drain for clean hangups while allowing cancellation to pre-empt it). - outboundCall.close() releases c.mu for the drain window: Close, CloseWith and CloseWithTimeout invoke close() while holding c.mu, and sleeping under the lock would block Participant() readers and a concurrent shutdown for the whole drain duration.
|
Pushed cab53c5 addressing both review comments:
Verified: |
|
Thanks @genseric-ghiro for the review. I merged the latest
The latest head ( |
…nprivileged range Devin review feedback on PR livekit#811: 1. inbound drainOnHangup: a remote CANCEL before the call is answered hit the 'cancelled' branch and slept the drain window with no media bridged. Gate the drain on the media actually being bridged (c.started broken) and drop 'cancelled' from the local-hangup set. Remote CANCEL is not a local hangup. 2. newTestMediaPort bound a random even port in 1-65535, which can land on a privileged port (<1024) and fail in CI with 'bind: permission denied' (seen on TestSetOfferReportsUnknownProvider). Constrain the test range to 10000-20000.
6bc104d to
bac9dc1
Compare
…nprivileged range Devin review feedback on PR livekit#811: 1. inbound drainOnHangup: a remote CANCEL before the call is answered hit the 'cancelled' branch and slept the drain window with no media bridged. Gate the drain on the media actually being bridged (c.started broken) and drop 'cancelled' from the local-hangup set. Remote CANCEL is not a local hangup. 2. newTestMediaPort bound a random even port in 1-65535, which can land on a privileged port (<1024) and fail in CI with 'bind: permission denied' (seen on TestSetOfferReportsUnknownProvider). Constrain the test range to 10000-20000.
bac9dc1 to
fa26463
Compare
…nprivileged range Devin review feedback on PR livekit#811: 1. inbound drainOnHangup: a remote CANCEL before the call is answered hit the 'cancelled' branch and slept the drain window with no media bridged. Gate the drain on the media actually being bridged (c.started broken) and drop 'cancelled' from the local-hangup set. Remote CANCEL is not a local hangup. 2. newTestMediaPort bound a random even port in 1-65535, which can land on a privileged port (<1024) and fail in CI with 'bind: permission denied' (seen on TestSetOfferReportsUnknownProvider). Constrain the test range to 10000-20000.
fa26463 to
e2d85c0
Compare
…nprivileged range Devin review feedback on PR livekit#811: 1. inbound drainOnHangup: a remote CANCEL before the call is answered hit the 'cancelled' branch and slept the drain window with no media bridged. Gate the drain on the media actually being bridged (c.started broken) and drop 'cancelled' from the local-hangup set. Remote CANCEL is not a local hangup. 2. newTestMediaPort bound a random even port in 1-65535, which can land on a privileged port (<1024) and fail in CI with 'bind: permission denied' (seen on TestSetOfferReportsUnknownProvider). Constrain the test range to 10000-20000.
e2d85c0 to
10384ca
Compare
…nprivileged range Devin review feedback on PR livekit#811: 1. inbound drainOnHangup: a remote CANCEL before the call is answered hit the 'cancelled' branch and slept the drain window with no media bridged. Gate the drain on the media actually being bridged (c.started broken) and drop 'cancelled' from the local-hangup set. Remote CANCEL is not a local hangup. 2. newTestMediaPort bound a random even port in 1-65535, which can land on a privileged port (<1024) and fail in CI with 'bind: permission denied' (seen on TestSetOfferReportsUnknownProvider). Constrain the test range to 10000-20000.
10384ca to
6acf3b5
Compare
…nprivileged range Devin review feedback on PR livekit#811: 1. inbound drainOnHangup: a remote CANCEL before the call is answered hit the 'cancelled' branch and slept the drain window with no media bridged. Gate the drain on the media actually being bridged (c.started broken) and drop 'cancelled' from the local-hangup set. Remote CANCEL is not a local hangup. 2. newTestMediaPort bound a random even port in 1-65535, which can land on a privileged port (<1024) and fail in CI with 'bind: permission denied' (seen on TestSetOfferReportsUnknownProvider). Constrain the test range to 10000-20000.
6acf3b5 to
a17e816
Compare
…ping
When an agent ends a call (EndCall RPC, end_call tool, or participant
removed from the room) right after wait_for_playout() returns, the last
word of the utterance is clipped on the callee's end. The audio is
already past the agent — it is buffered in the room mixer (~100ms input
buffer), the encoder, and in-flight RTP — but the SIP bridge sends BYE
and closes media immediately, so the tail never clears the wire (issue
Add a configurable hangup drain window (hangup_drain_time, default
500ms): on a locally-initiated clean hangup, keep feeding media for the
window before sending BYE. Remote BYE, errors, timeouts, and
pre-connect failures are not delayed.
Regression test: an EndCall RPC on an established outbound call must not
emit BYE before the drain window has elapsed. Fails on the old code
('BYE sent before the media drain window elapsed'), passes with the
fix.
…nprivileged range Devin review feedback on PR livekit#811: 1. inbound drainOnHangup: a remote CANCEL before the call is answered hit the 'cancelled' branch and slept the drain window with no media bridged. Gate the drain on the media actually being bridged (c.started broken) and drop 'cancelled' from the local-hangup set. Remote CANCEL is not a local hangup. 2. newTestMediaPort bound a random even port in 1-65535, which can land on a privileged port (<1024) and fail in CI with 'bind: permission denied' (seen on TestSetOfferReportsUnknownProvider). Constrain the test range to 10000-20000.
Devin review feedback on PR livekit#811: The outbound drainOnHangup returned the full drain window for 'hangup'/'rpc'/'removed' regardless of connection state, so a CANCEL arriving during ringing waited HangupDrainTime with no media bridged. Gate the drain on c.started (media bridged) — same gate the inbound path got in the sibling commit — and add a regression test asserting the gate flips on the started fuse, not the reason set.
a17e816 to
fd7db18
Compare
|
Hi @davidzhao @Cloudish9 — following up on this one. I've pushed the last open review finding: the outbound Quick context: I'm working from a fork ( |
There was a problem hiding this comment.
Devin Review found 1 new potential issue.
2 flags not posted on this PR by your GitHub settings — view them in Devin Review. (Configure)
| // wire before the BYE tears the call down. Without this, the last | ||
| // word of a voicemail is clipped on abrupt hangups (issue #4737). | ||
| c.log().Debugw("draining media before hangup", "drain", drain) | ||
| time.Sleep(drain) |
There was a problem hiding this comment.
🔴 Shutdown strands draining inbound calls
If shutdown starts during time.Sleep, Shutdown cannot terminate the inbound call because done is already set. Server.Stop then closes SIP transport without waiting for BYE, leaving the peer connected.
Learn more
An inbound call marks itself done before the new drain window. Shutdown calls Shutdown on every registered call, but that method re-enters close, which returns immediately once done is set. Server.Stop then closes SIP transport while the original close is asleep. The original close later attempts CloseWithStatus, too late to reliably send BYE on that transport.
Example: A local EndCall begins a 500 ms drain. At 100 ms, service shutdown finds the call, gets an immediate return from Shutdown, and closes the SIP listener. At 500 ms the call attempts BYE; the peer may remain connected until its own timeout.
Recommended fix: Coordinate the in-progress close with Shutdown and Server.Stop. Interrupt the drain on shutdown and wait for the original close to finish sending BYE before closing signaling transport.
Was this helpful? React with 👍 or 👎 to provide feedback.
Fixes livekit/livekit#4737
Problem
When an agent ends a call right after
wait_for_playout()returns — EndCall RPC,end_calltool, or removing the SIP participant — the last word of the utterance is clipped on the callee's end (voicemail scenario). The audio is already past the agent: it is buffered in the room mixer (5-frame ~100ms input buffer), the codec encoder, and in-flight RTP.wait_for_playout()only tracks the local queue, so it cannot see this tail.The SIP bridge then sends BYE and closes media immediately, so whatever is still buffered never clears the wire. A trailing pause masks the bug because it gives the tail time to flush; an abrupt end-of-speech hangup clips the last word. The reporter's workaround (2s pre-hangup sleep) confirms the mechanism.
Fix
Add a configurable drain window
hangup_drain_time(default500ms): on a locally-initiated clean hangup, keep feeding media to the peer for the window before sending BYE. The window is bounded (default 500ms) and only applies to:EndCallRPC (reasonrpc)hangup)removed)cancelled)Remote BYE (the peer hung up), errors, timeouts, media failures, and pre-connect failures are not delayed. Set
hangup_drain_time: -1to disable.Test
TestOutboundHangupDrainsMediaBeforeBYE: on an established outbound call, anEndCallRPC must not emit BYE before the drain window has elapsed. Fails on the old code (BYE sent before the media drain window elapsed), passes with the fix. All media-port and outbound tests pass; the pre-existing auth-test failures on clean main are unrelated (they fail identically without this change).