Skip to content

Conversation

@jeradf
Copy link
Contributor

@jeradf jeradf commented Feb 8, 2025

this should fix the utterance delays at true turn endings that some folks have noticed with the new model.

with this updated threshold the new model has the same TPR (true positive rate) as the old model, but a much higher TNR. in other words, it's better at preventing interruptions from the agent, without sacrificing higher latency at true turn endings.

for reference, here's the metrics comparison using our latest evaluation set:

                         threshold    tpr    tnr
OLD model OLD threshold      0.150  0.988  0.796
NEW model OLD threshold      0.150  0.943  0.976
NEW model NEW threshold      0.008  0.988  0.905

@changeset-bot
Copy link

changeset-bot bot commented Feb 8, 2025

🦋 Changeset detected

Latest commit: ac2d075

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package
Name Type
livekit-plugins-turn-detector Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@jeradf jeradf merged commit 7682654 into main Feb 8, 2025
14 checks passed
@jeradf jeradf deleted the fix-prob-cutoff branch February 8, 2025 05:11
jayesh-mivi pushed a commit to mivi-dev-org/custom-livekit-agents that referenced this pull request Jun 4, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants