Latency doubles in telephony contexts

[Rickaym]

Hi,

First of all, Livekit is great. I have been using it for over a year now to create web-based voice AI agents. More recently, I started applying livekit agents to telephony, to answer an inbound call at a Twilio number, for instance. And this is where latency becomes quite unbearable. Part of my question is how I can find the bottleneck/what I can do to reduce it, and another is if this is livekit-specific. Below is a comparison I made between the Engress recording and the microphone recording of the call. I know this is extremely rough, but this was easiest to do, and the latency difference is quite apparent. The code I used here is a prompt modified version of the starter example.

Based on the image below, what's seems apparent to me is telephony latency, that is, after the STT->LLM->TTS pipeline has ended, the audio which gets back to the phone (and vice versa) is causing the latency issue. When I compared the latency graphs (web vs telephony) of something like vapi, it was absolutely on-point. This was confusing to me.

I appreciate any thoughts here.

(first is the engress recording, second is the microphone recording)

[tinalenguyen]

Hi, it's great to see that you are exploring LiveKit's capabilities!

Could you provide any SCL_ call IDs and what STT/LLM/TTS models you are using? Also, it would be helpful if you could make distinctions of whether the user or the agent is speaking in the screenshots

[akshat-315]

Hi, any luck regarding this?
Facing the same issue. Using twilio as the SIP Provider but somehow the latency almost doubles over telephone calls. The agent normally runs flawlessly though

[Rickaym]

Hi @tinalenguyen, the first part of the audio where both of the recordings are sync is the user audio, so that is when the user stops speaking in the mic recording and the engress recording, the gap you see in the later section is the agent's response audio as it arrives to my device. It much later than the recording shows, which makes me believe this is purely telephony latency

[Rickaym]

No luck yet @akshat-315

[tinalenguyen]

@Rickaym Thank you for clarifying!

I think you're right that this is purely telephony latency. LiveKit -> Twilio latency should be around 100-150 ms. After the STT->LLM->TTS pipeline has ended, the rest of the latency should be from Twilio -> End User (PSTN).

The latency could be from the cellular network routing to the PSTN, or it could be PSTN geographic traversal latency. The audio is sent to Twilio's US East server, so maybe that plays a part.

[Rickaym]

Interesting, I actually tested from Asia, I'll have a US friend try this to see any results.
In any case, how can I decrease this latency for APAC calls?
Is there a way to adjust the Twilio server or be on the edge somehow?

Thanks tina

[tinalenguyen]

@Rickaym For APAC calls, I think you should try configuring to the Australia (AU1) server. I imagine the latency would be better routing from there compared to US East. There is also an Ireland server, but I think in most cases Australia is better.

Twilio docs:

• https://www.twilio.com/docs/global-infrastructure/regional-product-and-feature-availability
• https://www.twilio.com/docs/global-infrastructure/understanding-twilio-regions#twilios-region-isolation-model

[nishadmusthafa]

Hi @Rickaym , we've made some changes this week that should reduce the audio latency in the telephony context. These went out 7 hours back. Could you retry your tests and let me know if you are still seeing this ?

[Pavan-dev2004]

Hello,
I’m currently experiencing a latency of around 2000–3000 ms during telephonic calls. Could you please confirm whether this is the expected behavior for this functionality, or if further latency improvements are planned?

This issue appears only in telephonic calls integrated with Twilio. Additionally, any guidance or suggestions on how to reduce the latency would be greatly appreciated.

Thank you.

[deanihansen]

@tinalenguyen what's the recommended way to trace and benchmark the telephone part?
We are seeing consistent turn taking times around 1 sec without telephone / SIP trunk usage.

If we call our agent via SIP trunk both from Telnyx and Twilio, we are adding between 1 and 2.5 second of additional latency.
This is with region locking turned to avoid geo-hopping too far.

[Pavan-dev2004]

Hey @deanihansen, are you experiencing a similar latency issue when integrating the LiveKit agent with Twilio?

[nishadmusthafa]

@Pavan-dev2004 we aren't seeing any global reports of this. There's a good possibility it's due to cross continental packet hops. Could you share your architecture. Are these inbound/outbound calls. Where is the carrier, which region are the livekit calls running in, etc etc.

[Pavan-dev2004]

@nishadmusthafa I am experiencing this issue during outbound calls. The LiveKit calls are routed through the Indian region, while Twilio is using the default US region. I also tried changing the Twilio configuration to the Australia server, but I did not observe any improvement in latency. Currently, I am using LiveKit Cloud.
I’ve added the Twilio call log below showing the carrier metrics.

[nishadmusthafa]

Could you try the following

• When creating the sip trunk , set destination_country to be AU.
• Check if you have ever requested the account to be pinned to India, if not, then this could be happening because your API requests are coming from India(which is probably something to improve on our side)
• Ideally, have the person testing in Australia and not elsewhere.
  Let me know if these help