Mitigating long buffering on initial video playback

[karyogamy]

What is it?

• Bugfix (user facing)
• Feature (user facing)
• Codebase improvement (dev facing)
• Meta improvement to the project (dev facing)

Description of the changes in your PR

Hi team, long time no see.
I've long since had issue with the buffering time when starting out a new video in the built-in player. Sometimes this gets so bad, there's a 10 seconds wait before any picture would even show up. Of course, your experience might differ depending on the phones, network conditions, or luck etc.

After digging a bit, I found a continueLoadingCheckIntervalBytes parameter on ExoPlayer progressive media source that controls the size of data loaded per call from the playback loader. By default, ExoPlayer will load 1MB per call. Reducing this number seems to have solved the initial buffering woes on my side, though I have no idea how it affects other users. So if you have a similar issue with buffering under fast network speed, please give this a try and test extensively to see if it works and if lowering the values has any side effects.

Screenshots

Fixes the following issue(s)

• Fixes Slow initial video buffer #7630, Video player slow loading on first tap #6927

APK testing

The APK can be found by going to the "Checks" tab below the title. On the left pane, click on "CI", scroll down to "artifacts" and click "app" to download the zip file which contains the debug APK of this PR.

Due diligence

• I read the contribution guidelines.

[litetex]

Preamble: I'm not sure if this fixes the above issues in the first place. Usually people have at least >1MB/s downlink - so transferring the required data should take 1-2s at worst.

Anyway a few more notes:

[TobiGr]

Welcome back!

[allgasno]

Hey @karyogamy whats your internet speed? I don't think theres much difference between 1mb and 256kb, but I'm willing to test could you please direct me to the test apk for myself and others, thank you. Have an internet speed around 250Mbps.

[karyogamy]

Preamble: I'm not sure if this fixes the above issues in the first place. Usually people have at least >1MB/s downlink - so transferring the required data should take 1-2s at worst.

Thanks for the review @litetex!
This fix is not really dealing with issues arising from network speed. On my side, many of the videos load and play quickly enough, but sometimes, a video would buffer for a long time (~10s) at the start. When this buffering happens, I'd fast forward once and the video would start playing right away instead of buffering for another few seconds. It also didn't make sense to me why this initial buffering is so pronounced here but not on other similar apps or platforms. Though like I mentioned, this might just be affecting a small number of users, but if possible, please try clicking on a variety of individual videos and see if this can be reprod on your devices too.

Hey @karyogamy whats your internet speed? I don't think theres much difference between 1mb and 256kb, but I'm willing to test could you please direct me to the test apk for myself and others, thank you. Have an internet speed around 250Mbps.

Thanks @allgasno, my internet is at 500Mbps so this is not really caused by speed, but perhaps something internal to the player like mentioned.
Please do test it, I will ask for help on the other issue threads as well. You can find the zipped app by going to https://github.com/TeamNewPipe/NewPipe/pull/7919/checks, click on CI on the left, and scroll down to Artifacts.
NOTE you need to enable the fix by going to Settings -> Video and audio -> Playback load interval size -> choose a small value (e.g. 16 KiB).

[ghost]

Fixes buffer issues for me. I found all intervals have no difference in the latency.

[opusforlife2]

(This is speculation; I haven't tested yet) If it turns out that the 1MB value is better for general cases (loading other than the start of a stream) is it possible to use the smaller value for just the first few seconds, and 1MB (or something higher?) for the rest of the stream?

[karyogamy]

@opusforlife2 That's exactly the issue: This change adds a parameter that is passed directly from ProgressiveMediaSource factory to the media source and can't be changed on-the-fly unless we duplicate thousands of lines of code.

Also, this issue doesn't only affect the first video played, but every video in a play queue that hasn't been pre-buffered for seamless playback (e.g. when you skip a few songs videos, background player doesn't have this issue).

Honestly, I'm not sure how many users are affected by the same buffering issue, though my guess is a good majority are. That is why I agree with @litetex to keep the exoplayer setting (1MB) default for now. So if buffering complaint keeps coming in and gets fixed with this settings change, we can then consider making a smaller value default instead.

[teaerror]

Does anyone know why 1MB doesn't work for NewPipe, when ExoPlayer uses it?
Skipping 10 seconds resolves the issue but setting the playback load to 256 as default would help several users with buffering issues.
Also, this may help with the issue
https://exoplayer.dev/troubleshooting.html#why-do-some-mp4fmp4-files-play-incorrectly

[opusforlife2]

In my testing, none of the values give a consistently faster playback start. I've seen both slow and fast starts on every value. I'm assuming my internet latency isn't stable enough to test this. 🤷

[karyogamy]

@teaerror Yup, that was the page that eventually led me to this solution.
The workaround flags didn't seem to make a difference though.
This comment might help with why continueLoadingCheckIntervalBytes worked: google/ExoPlayer#4014 (comment)

@opusforlife2 Interesting. Initial buffering should become more consistent with the change.
Would you mind sharing your internet speed?
Also, did you make sure the players were fully shutdown and restarted after changing the settings?

[opusforlife2]

It varies between 50-100 Mbps, but as I said, latency is quite an issue.

I was using the main player only, and I closed it before every time I opened settings.

[litetex]

LGTM

Test worked fine. I think the initial loading is a bit better now 😄

Applied minor changes:

• Updated the branch
• Moved the preference to a better matching section
• Shortened a string

(@karyogamy you might also cleanup your repo's branches)

[sonarcloud]

Kudos, SonarCloud Quality Gate passed!   

0 Bugs
0 Vulnerabilities
0 Security Hotspots
0 Code Smells

0.0% Coverage
0.0% Duplication

[Stypox]

Code looks good to me, too, though it doesn't seem to make any difference for me, like opus

[opusforlife2]

Oh. I thought this was meant to be a test PR. To find out what the best value is and make that the default. Are we going to advise users to adjust the value based on their own experiences?

[Stypox]

Oh, I didn't get that ;-)
Should the PR be reverted and should we choose a specific value for everyone? This makes sense so that we don't introduce a new setting without a clear meaning except for us.

[karyogamy]

@opusforlife2 @Stypox I think the plan is to have the setting available to all for the time being, since there isn't a clear answer as to which value is best for everyone, especially now we know changing the value doesn't seem to make a difference for some. The side effect of choosing a smaller value is still unclear too. My guess is there has to be some kind of a trade-off.

We can either tell the users about this setting proactively, or when there's more buffering issues come in. A new default should be selected in a separate PR when the team is comfortable with it.

On a side note, would it make sense to have a separate settings section for these kind of fine-tuning details? e.g. Player internal buffer/cache size, extractor cache size, etc. Maybe a public Debug section if settings might disappear later?

[opusforlife2]

Sure. We can work with it in the blog. A temporary setting to poll our users. They could respond on our chosen platform(s), like here, Reddit, blog comments, etc.

On a side note, would it make sense to have a separate settings section for these kind of fine-tuning details? e.g. Player inter

#3192 (comment) ;)

[ghost]

The recent v0.22.2 still buffers like crazy, even though this fix claims to be merged. Something's strange.

[AudricV]

Nothing strange, only the extractor version and the required app changes have been made (that's why 0.22.2 is a hotfix version and not a normal version). It will be included in version 0.23.0.

[Stypox]

This issue is present even on dev though. I added a TODO point to #8230. The cause might be completely different from the one this PR fixes, since switching resolution almost instantly loads the video, and this behaviour could be observed only after updating the NewPipeExtractor (since I could notice it in the player refactor PR before YouTube broke). #8224 is actually relevant (since I am experiencing this issue, too, on 0.23.0 builds). @karyogamy could you look into this?

[karyogamy]

@Stypox Behaviour still seems fine on my end, did you test by setting a lower interval size?
Also, is the problem reproducible on every video? IIRC, there are some exceptions where the videos would still take a while to load even with the settings changed. In those cases, I added a hack to skip the first frames, but left it out of this PR since I'd much prefer a proper solution like #8153.

But if you are interested, try adding these lines here and see if it does anything:

        if (newPosition.contentPositionMs == 0
                && simpleExoPlayer.getTotalBufferedDuration() < 500L) {
            Log.d(TAG, "Playback - skipping to initial keyframe.");
            simpleExoPlayer.setSeekParameters(SeekParameters.CLOSEST_SYNC);
            simpleExoPlayer.seekTo(1L);
            simpleExoPlayer.setSeekParameters(PlayerHelper.getSeekParameters(context));
        }

[litetex]

→ Might be related: #8238

[ColeWunderlich]

For what it's worth I am also having issues with very long buffer times on v0.22.0.

I am able to consistently fix this by:

1. Attempting to play the video
2. Immediately hitting the "background" button (ie audio only)
3. Immediately switching back to full video after the background playback has started. Then everything works as normal.

This trick consistently works for me. Without the trick initial buffer times are usually over 30 sec for a video, but with this trick the video plays instantly without fail (background audio never takes more than a sec to kick in).

Perhaps this might provide some insight as to what's going on? I find it interesting that this "fix" doesn't involve changing any settings, which tells me the old defaults might be fine and something else might be going on.

PS. I have gigabit internet, so there's no way internet speed is causing buffering issues