-
Notifications
You must be signed in to change notification settings - Fork 21
Prototype of latency reducer, only for Kick #114
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
Kaedriz
wants to merge
55
commits into
enhancer-app:master
Choose a base branch
from
Kaedriz:latency-reducer
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
55 commits
Select commit
Hold shift + click to select a range
e651345
Prototype of latency reducer, only for Kick
Kaedriz a512a70
Prototype for Twitch
Kaedriz 0ba1875
chore: lint
jvxz a85f3fc
feat(twitch): add mockup latency tab for latency-related options
jvxz 1439ef5
refactor(twitch): move stream latency display toggle to dedicated tab
jvxz 6f3944d
feat(twitch): add kick-identical mockup latency reducer options to la…
jvxz b28a867
feat(twitch): add latency reducer constants
jvxz ec9ce05
feat(twitch): extend twitch media player instance types to include ac…
jvxz 0a1394a
feat(twitch): add stream latency reducer module, add core functionali…
jvxz 9b85b87
fix(twitch): fix weird commit error
jvxz d71fc81
refactor(twitch): scrap first prototype for stream latency reduction …
jvxz 8cc0a2b
refactor(twitch): opt for a single playback rate value instead of min…
jvxz dd75926
feat(twitch): extend latency component to include playback rate
jvxz 67cdf96
feat(twitch): add getMediaPlayerPlaybackRate twitch util
jvxz e6b3e77
refactor(twitch): refactor stream latency reducer module to use playb…
jvxz 934d1df
chore: cleanup
jvxz ffcddfb
feat: add playbackRate signal to stream latency module with video rat…
jvxz 660a16c
refactor(twitch): organize latency settings declarations
jvxz 9435081
feat(twitch): re-add min/max playback rate settings, add min/max thre…
jvxz 8310d6e
feat(twitch): add latency offset for rate updates to prevent rapid up…
jvxz 82d84de
fix(twitch): fix twitch default settings
jvxz ecc72c7
chore: remove accidentally-commited vscode settings.json
jvxz 1ede871
Fixes playback not limited to 2 digits
Kaedriz 58891d2
WIP fix for twitch latency reducer
Kaedriz d4112f3
feat(twitch): enhance fix for latency reducer
jvxz f54ab42
Some changes to make it working again
Kaedriz 4cc0f81
Formatting
Kaedriz eb7222a
Fix compatibility with FFZ
Kaedriz 12d2b34
Final touches to twitch version
Kaedriz 9173f95
style: change ordering to be more logical
Kaedriz c437b68
refactor: Separate Reducer logic to separate module
Kaedriz 2f34c2b
Revert "feat(twitch): enhance fix for latency reducer"
Kaedriz 8e9e260
Merge remote-tracking branch 'origin/latency-reducer-2' into latency-…
Kaedriz 30c5aba
Merge branch 'enhancer-app:master' into latency-reducer
Kaedriz aa0fac6
Apply suggestion from @gemini-code-assist[bot]
Kaedriz 326cba2
Apply suggestion from @gemini-code-assist[bot]
Kaedriz dfc0081
Apply suggestion from @Copilot
Kaedriz 623429e
Apply suggestion from @Copilot
Kaedriz 360b3fc
Apply suggestion from @Copilot
Kaedriz 3c3f62c
Apply suggestion from @Copilot
Kaedriz 8d68151
Apply suggestion from @Copilot
Kaedriz 7e4bbd2
style: Fix incomplete comment
Kaedriz 446d483
refactor: Apply suggestion from Gemini
Kaedriz 033faff
style: formatting fixes
Kaedriz 0fbbd8d
Apply Gemini suggestion
Kaedriz 4baf6c1
refactor: remove obsolete check
Kaedriz 583ebe2
refactor: optimize settings calls for reducer
Kaedriz 235fad8
fix: add error logging
Kaedriz b239fb4
refactor: remove redundant code
Kaedriz cc038f3
fix: Prevent possible bug
Kaedriz c5fc137
style: fix typo
Kaedriz e836df7
refactor: remove redundant code
Kaedriz a332b1a
style: formatting
Kaedriz 1661c5d
refactor: extract part of latency getter to utils
Kaedriz 56fc659
refactor: temporary move these new feature to experimental to test th…
Kaedriz File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
135 changes: 135 additions & 0 deletions
135
src/platforms/kick/modules/stream-latency-reducer/stream-latency-reducer.module.tsx
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,135 @@ | ||
| import KickModule from "$kick/kick.module.ts"; | ||
| import type { KickModuleConfig } from "$types/shared/module/module.types.ts"; | ||
| import { signal } from "@preact/signals"; | ||
|
|
||
| export default class StreamLatencyReducerModule extends KickModule { | ||
| private updateInterval: NodeJS.Timeout | undefined; | ||
| private latencyTimings = signal<number[]>([]); | ||
|
|
||
| readonly config: KickModuleConfig = { | ||
| name: "stream-latency-reducer", | ||
| appliers: [ | ||
| { | ||
| type: "selector", | ||
| key: "stream-latency-reducer", | ||
| selectors: ["#channel-chatroom"], | ||
| callback: this.run.bind(this), | ||
| validateUrl: (url) => { | ||
| return !url.includes("/videos/") && !url.includes("/clips/"); | ||
| }, | ||
| once: true, | ||
| }, | ||
| ], | ||
| isModuleEnabledCallback: async () => await this.settingsService().getSettingsKey("streamLatencyReducerEnabled"), | ||
| }; | ||
|
|
||
| private run(): void { | ||
| if (this.updateInterval) clearInterval(this.updateInterval); | ||
| this.updateInterval = setInterval(async () => { | ||
| const status = await this.getPlaybackRateStatus(); | ||
|
|
||
| if (status === "catchingUpMax") { | ||
| this.setPlaybackRateMode("catchUpMax"); | ||
| } else if (status === "catchingUpMin") { | ||
| this.setPlaybackRateMode("catchUpMin"); | ||
| } else { | ||
| this.setPlaybackRateMode("reset"); | ||
| } | ||
| }, 1000); | ||
| } | ||
|
|
||
| private changePlaybackSpeed(video: HTMLVideoElement, rate: number) { | ||
| video.playbackRate = rate; | ||
| } | ||
|
|
||
| private async setPlaybackRateMode(mode: "catchUpMin" | "catchUpMax" | "reset") { | ||
| const videoPlayer = this.getPlayer(); | ||
| if (!videoPlayer) return; | ||
|
|
||
| let targetRate = 1; | ||
| const latency = this.computeLatency(videoPlayer); | ||
|
|
||
| // Always reset playback speed in reset mode, regardless of latency. | ||
| if (mode === "reset") { | ||
| this.changePlaybackSpeed(videoPlayer, 1); | ||
| return; | ||
| } | ||
|
|
||
| // When latency is zero, negative, or otherwise invalid, ensure playback speed is reset. | ||
| if (!latency || latency <= 0) { | ||
| this.changePlaybackSpeed(videoPlayer, 1); | ||
| return; | ||
| } | ||
| const { | ||
| minRate, | ||
| maxRate, | ||
| minThreshold: minSpeedThreshold, | ||
| maxThreshold: maxSpeedThreshold, | ||
| } = await this.getSettings(); | ||
|
|
||
| if (mode === "catchUpMax") { | ||
| targetRate = maxRate; | ||
| } else if (mode === "catchUpMin") { | ||
| if (maxSpeedThreshold === minSpeedThreshold) { | ||
| // Avoid division by zero when thresholds are equal; fall back to a step between minRate and maxRate. | ||
| targetRate = latency >= maxSpeedThreshold ? maxRate : minRate; | ||
| } else { | ||
| targetRate = | ||
| minRate + ((maxRate - minRate) * (latency - minSpeedThreshold)) / (maxSpeedThreshold - minSpeedThreshold); | ||
| } | ||
| } | ||
|
|
||
| this.changePlaybackSpeed(videoPlayer, targetRate); | ||
| } | ||
|
|
||
| private async getPlaybackRateStatus() { | ||
| const videoPlayer = this.getPlayer(); | ||
| if (!videoPlayer) return; | ||
| const latency = this.computeLatency(videoPlayer); | ||
| if (!latency) return "invalid"; | ||
|
|
||
| const { maxThreshold, minThreshold } = await this.getSettings(); | ||
| if (latency >= maxThreshold) return "catchingUpMax"; | ||
| if (latency > minThreshold) return "catchingUpMin"; | ||
| if (latency <= minThreshold) return "caughtUp"; | ||
| return "invalid"; | ||
| } | ||
|
|
||
| private async getSettings() { | ||
| const minRate = await this.settingsService().getSettingsKey("streamLatencyReducerMinRate"); | ||
| const maxRate = await this.settingsService().getSettingsKey("streamLatencyReducerMaxRate"); | ||
| const minThreshold = await this.settingsService().getSettingsKey("streamLatencyReducerMinThreshold"); | ||
| const maxThreshold = await this.settingsService().getSettingsKey("streamLatencyReducerMaxThreshold"); | ||
| return { minRate, maxRate, minThreshold, maxThreshold }; | ||
Kaedriz marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| } | ||
|
|
||
| // Calculates latency based on average of past latency snapshots | ||
| private computeLatency(video: HTMLVideoElement): number { | ||
| const computedLatency = this.kickUtils().getLatency(video); | ||
| this.latencyTimings.value.push(computedLatency); | ||
|
|
||
| // Reset timings array if experiences sudden increase in latency | ||
| if ( | ||
| this.latencyTimings.value.length > 1 && | ||
| computedLatency - this.latencyTimings.value[this.latencyTimings.value.length - 2] > 2 | ||
| ) { | ||
| this.latencyTimings.value = [computedLatency]; | ||
| } | ||
|
|
||
| const numberOfSamples = 10; | ||
| if (this.latencyTimings.value.length > numberOfSamples) this.latencyTimings.value.shift(); | ||
|
|
||
| return ( | ||
| this.latencyTimings.value.reduce((accumulator, currentValue) => accumulator + currentValue, 0) / | ||
| this.latencyTimings.value.length | ||
| ); | ||
| } | ||
|
|
||
| private getPlayer() { | ||
| const video = document.querySelector("video"); | ||
| if (!video || !this.kickUtils().isLiveVideo(video)) { | ||
| return null; | ||
| } | ||
| return video; | ||
| } | ||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change | ||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
|
@@ -8,6 +8,8 @@ export default class StreamLatencyModule extends KickModule { | |||||||||||||||
| private latencyCounter = signal(-1); | ||||||||||||||||
| private isLiveState = signal(false); | ||||||||||||||||
| private updateInterval: NodeJS.Timeout | undefined; | ||||||||||||||||
| private playbackRate = signal(1); | ||||||||||||||||
| private latencyTimings = signal<number[]>([]); | ||||||||||||||||
|
|
||||||||||||||||
| readonly config: KickModuleConfig = { | ||||||||||||||||
| name: "stream-latency", | ||||||||||||||||
|
|
@@ -31,6 +33,8 @@ export default class StreamLatencyModule extends KickModule { | |||||||||||||||
| this.logger.debug("Found multiple elements of chat room"); | ||||||||||||||||
| } | ||||||||||||||||
|
|
||||||||||||||||
| this.watchPlaybackRate(); | ||||||||||||||||
|
|
||||||||||||||||
| if (this.updateInterval) clearInterval(this.updateInterval); | ||||||||||||||||
| this.updateInterval = setInterval(() => this.updateLatency(), 1000); | ||||||||||||||||
|
|
||||||||||||||||
|
|
@@ -43,6 +47,7 @@ export default class StreamLatencyModule extends KickModule { | |||||||||||||||
| <LatencyComponent | ||||||||||||||||
| isLive={this.isLiveState} | ||||||||||||||||
| latencyCounter={this.latencyCounter} | ||||||||||||||||
| playbackRate={this.playbackRate} | ||||||||||||||||
| click={this.resetPlayer.bind(this)} | ||||||||||||||||
| />, | ||||||||||||||||
| span, | ||||||||||||||||
|
|
@@ -62,10 +67,32 @@ export default class StreamLatencyModule extends KickModule { | |||||||||||||||
| } | ||||||||||||||||
|
|
||||||||||||||||
| private computeLatency(video: HTMLVideoElement): number { | ||||||||||||||||
| const { currentTime, buffered } = video; | ||||||||||||||||
| if (buffered.length === 0) return -1; | ||||||||||||||||
| const bufferEnd = buffered.end(buffered.length - 1); | ||||||||||||||||
| return bufferEnd - currentTime; | ||||||||||||||||
| const computedLatency = this.kickUtils().getLatency(video); | ||||||||||||||||
| this.latencyTimings.value.push(computedLatency); | ||||||||||||||||
|
|
||||||||||||||||
| // Reset timings array if experiences sudden increase in latency | ||||||||||||||||
| if ( | ||||||||||||||||
| this.latencyTimings.value.length > 1 && | ||||||||||||||||
| computedLatency - this.latencyTimings.value[this.latencyTimings.value.length - 2] > 2 | ||||||||||||||||
| ) { | ||||||||||||||||
| this.latencyTimings.value = [computedLatency]; | ||||||||||||||||
Kaedriz marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||||||||||||
| } | ||||||||||||||||
|
|
||||||||||||||||
| const numberOfSamples = 10; | ||||||||||||||||
| if (this.latencyTimings.value.length > numberOfSamples) this.latencyTimings.value.shift(); | ||||||||||||||||
|
|
||||||||||||||||
| return ( | ||||||||||||||||
| this.latencyTimings.value.reduce((accumulator, currentValue) => accumulator + currentValue, 0) / | ||||||||||||||||
| this.latencyTimings.value.length | ||||||||||||||||
|
Comment on lines
+84
to
+86
|
||||||||||||||||
| return ( | |
| this.latencyTimings.value.reduce((accumulator, currentValue) => accumulator + currentValue, 0) / | |
| this.latencyTimings.value.length | |
| const length = this.latencyTimings.value.length; | |
| if (length === 0) return -1; | |
| return ( | |
| this.latencyTimings.value.reduce((accumulator, currentValue) => accumulator + currentValue, 0) / length |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.