Skip to content

Commit

Permalink
Fix FPS and latency bugs with VP9 screenshare. (#1069)
Browse files Browse the repository at this point in the history
* Fix FPS and latency bugs with VP9 screenshare.

Also restored L3T3_KEY publishing for screen share. It was disabled previously.

Chrome does not allow more than 5 fps with L1T3, and it has encoding bugs with L3T3
It has a different path for screenshare handling and it seems to be untested/buggy
As a workaround, we are setting contentHint to force it to go through the same
path as regular camera video. While this is not optimal, it delivers the performance
that we need

* changeset

* added info log when contentHint is overridden
  • Loading branch information
davidzhao authored Mar 13, 2024
1 parent 1720999 commit 282eff7
Show file tree
Hide file tree
Showing 3 changed files with 20 additions and 2 deletions.
5 changes: 5 additions & 0 deletions .changeset/smooth-glasses-jam.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"livekit-client": patch
---

Fix FPS and latency issues with VP9 screenshare
15 changes: 14 additions & 1 deletion src/room/participant/LocalParticipant.ts
Original file line number Diff line number Diff line change
Expand Up @@ -734,7 +734,20 @@ export default class LocalParticipant extends Participant {
if (isSVCCodec(videoCodec)) {
// vp9 svc with screenshare has problem to encode, always use L1T3 here
if (track.source === Track.Source.ScreenShare && videoCodec === 'vp9') {
opts.scalabilityMode = 'L1T3';
// Chrome does not allow more than 5 fps with L1T3, and it has encoding bugs with L3T3
// It has a different path for screenshare handling and it seems to be untested/buggy
// As a workaround, we are setting contentHint to force it to go through the same
// path as regular camera video. While this is not optimal, it delivers the performance
// that we need
if ('contentHint' in track.mediaStreamTrack) {
track.mediaStreamTrack.contentHint = 'motion';
this.log.info('forcing contentHint to motion for screenshare with VP9', {
...this.logContext,
...getLogContextFromTrack(track),
});
} else {
opts.scalabilityMode = 'L1T3';
}
}
// set scalabilityMode to 'L3T3_KEY' by default
opts.scalabilityMode = opts.scalabilityMode ?? 'L3T3_KEY';
Expand Down
2 changes: 1 addition & 1 deletion src/room/track/RemoteAudioTrack.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
import { TrackEvent } from '../events';
import { computeBitrate } from '../stats';
import type { AudioReceiverStats } from '../stats';
import { computeBitrate } from '../stats';
import type { LoggerOptions } from '../types';
import { isReactNative, supportsSetSinkId } from '../utils';
import RemoteTrack from './RemoteTrack';
Expand Down

0 comments on commit 282eff7

Please sign in to comment.