Speed gets stuck and chart is empty

[zznidar]

Hey,
in the last couple of months, I have been experiencing a weird issue with JumpCutter on YouTube (using Firefox on Windows).

While watching a video, it every now and then randomly occurs that a loud portion is skipped or that a silent portion is not skipped. Looking at the chart, it seems as though the audio samples were not at all analysed -- there's no blue waveform visible for a couple of seconds. After it gets unstuck, the waveform is retrospectively shown (but then it's already too late).

Fig 1: Although the video was loud, the chart appears empty and it was played with silent speed

Fig 2: A moment later, the chart gets updated -- you can see that the empty area was now replaced with actual graph. And as shown by the bar, it was louder than the set threshold, yet still it was played with high speed.

Fig 3: Same thing sometimes happens if it gets stuck during speech -- then the silence won't be skipped (here I only have one screenshot with missing samples, but they fell below the threshold somewhere during that time)

I use the ✋ On some websites I visit, playback rate is getting changed in an unexpected manner. Don't let them change playback rate set by this extension. setting.

JumpCutter 1.27.0
Windows 10
Firefox 127.0.1 (but it also happened with a few previous versions)

[WofWca]

Hey! Thanks for the report.
This is strange indeed.

the waveform is retrospectively shown

Are you sure this happens this way? Like, in real time? The chart is polling-based, i.e. it requests the current loudness every couple of milliseconds, so it kind of seems like it takes time for polls to get delivered? I don't think I've ever seen this.
For me it also sometimes happens that a loud portion is skipped because it can't handle the volume change in time, but then the chart wouldn't show the previous loudness like that, it would just skip.

How often do you encounter this? Would it be possible to record a video some time?

On my part I'll try to do some testing as well.

[WofWca]

Oh, and I forgot to mention that sometimes it's just the chart that is wrong. Sometimes it starts showing red despite the playback being at normal speed. This usually happens when the silence range is really small, and it usually gets "fixed" by a new silence period.

But what you're describing looks a bit different.

[WofWca]

Plus I just noticed that you have both margins set to 0. That might have something to do with what we're seeing.

[zznidar]

Hey, I'll try to do make a screen-recording next week.

About the waveform being retrospectively shown: first, there's no waveform (like the right half of the Fig 1). Then, at some point, the whole empty area is at once "updated" (waveform is drawn) -- Fig 2 was taken a short moment after Fig 1. The largest red chunk in Fig 2 is the empty red chunk in Fig 1 (for some reason, it also shrinked on the graph).

I also sometimes experience the visually wrong chart at short silences you mention, but this one is also functional (i. e. the actual playback speed is different than it should be), so I think it's a different issue.

Good catch about margins being set to 0! This is indeed a change I made in the last couple of months! I'll try to do some more testing next week and come back with any results.

[zznidar]

I have done some tests. I tried it in a new FF profile to eliminate any other add-ons.

It seems the problem appears when CPU usage is very high. I suppose this is the reason for delayed audio analysis – if smooth video playback is prioritised by the browser over extensions, then it makes sense that the analysator runs out of CPU/is delayed at certain occasions.

I've tried setting Margin after to an arbitrarily low value (0.001), but there was no difference. I tried setting it to a bit more than 1 frame (for a 60-fps video, I set it to 0.018), but it didn't help much. (Because the analyser got stuck for longer)

My setup for testing:

I tested using this video (1080p60) https://www.youtube.com/watch?v=G9E9icM26l4 with sounded speed 1.5 (to be more CPU-intense) and silent 8.0 (to really notice when it gets stuck), both margins 0, threshold to something reasonable.

Simultaneously, I ran two 4k videos in another browser to use up the majority of CPU.

[WofWca]

Hmm, this is weird because usually when the CPU usage gets high the extension can't do close to anything. But it appears that in your case it is able to get the volume info but it doesn't have time to process the fact that the volume exceeded the threshold.

Actually, there are two separate pieces of code that are responsible for each of the two things.
Maybe we could try to prioritize the latter somehow.

Or at least detect the fact that the CPU is overloaded so we can't handle volume changes. In which case it would be ideal to just reset the speed to normal speed and wait until we have CPU time to work normally.

But all that applies only if our idea about the underlying cause of the problem is true.

[WofWca]

As a workaround, I can suggest not opening the pop-up, because it increases CPU usage.

And I still would appreciate a video recording if possible. You can email it to me in private if you wish.

[WofWca]

Hey, just want to note that I slightly changed the telemetry message exchange behavior in 1.27.1 - now it won't send more than one response within a 10ms interval. Though it should not affect this case, if it works the way we think it works.

[zznidar]

Hello there, I've just wanted to make a screen-recording, however YouTube started rolling out some bad updates for me this week (endless buffering, dropping all frames etc. on the videos I used last time). Therefore, right now, I cannot replicate the issue. It could also be that your telemetry change fixed it – I'll keep paying attention during normal usage in the future, and if I spot the issue again, I'll try to replicate & record it right away.

[WofWca]

Okay, thanks!