Stream quality regression VAAPI Radeon Linux

[neatnoise]

Is there an existing issue for this?

• I have searched the existing issues

Is your issue described in the documentation?

• I have read the documentation

Is your issue present in the latest beta/pre-release?

This issue is present in the latest pre-release

Describe the Bug

I regularly update master git version of sunshine. The latest updates (probably this 38c13c8 ) introduced stream quality regression for VAAPI encoding (RX 580 GPU). The visual quality is much worse. The changed pixels update to achive good visual quality like with 1 second delay. At 25 Mb/s bitrate 1080p 60fps it shows very noticeable compression effects, it stutters, it looks like 5 Mbit bitrate before updates.

Expected Behavior

Nice looking stream at 25 Mbit

Additional Context

No response

Host Operating System

Linux

Operating System Version

Arch linux latest

Architecture

64 bit

Sunshine commit or version

sunshine-git-2024.713.205505.r0.g18e7dfb-1-x86_64.pkg.tar.zst

Package

Linux - AUR (Third Party)

GPU Type

AMD

GPU Model

RX 580

GPU Driver/Mesa Version

latest mesa-git

Capture Method

KMX (Linux)

Config

wan_encryption_mode = 2
cert = x
pkey = x
output_name = 0
amd_quality = default
external_ip = x
nv_rc = auto
channels = 3
address_family = both
resolutions = [
    1280x720,
    1920x1080
]
upnp = enabled
nv_preset = default
fps = [60]
qp = -1
encoder = vaapi
lan_encryption_mode = 2
amd_rc = auto
file_state = x

Apps

All, desktop

Relevant log output

lip 15 14:00:52 tm-stc sunshine[1294]: [2024:07:15:14:00:52]: Info: Trying encoder [vaapi]
lip 15 14:00:52 tm-stc sunshine[1294]: [2024:07:15:14:00:52]: Info: Screencasting with KMS
lip 15 14:00:52 tm-stc sunshine[1294]: [2024:07:15:14:00:52]: Info: /dev/dri/card1 -> amdgpu
lip 15 14:00:52 tm-stc sunshine[1294]: [2024:07:15:14:00:52]: Info: Found monitor for DRM screencasting
lip 15 14:00:52 tm-stc sunshine[1294]: [2024:07:15:14:00:52]: Info: Found connector ID [94]
lip 15 14:00:52 tm-stc sunshine[1294]: [2024:07:15:14:00:52]: Info: Found cursor plane [60]
lip 15 14:00:52 tm-stc sunshine[1294]: [2024:07:15:14:00:52]: Info: SDR color coding [Rec. 601]
lip 15 14:00:52 tm-stc sunshine[1294]: [2024:07:15:14:00:52]: Info: Color depth: 8-bit
lip 15 14:00:52 tm-stc sunshine[1294]: [2024:07:15:14:00:52]: Info: Color range: [JPEG]
lip 15 14:00:52 tm-stc sunshine[1294]: [2024:07:15:14:00:52]: Info: vaapi vendor: Mesa Gallium driver 24.2.0-devel for AMD Radeon RX 580 Series (radeonsi, po
laris10, LLVM 19.0.0, DRM 3.57, 6.9.9-arch1-1)
lip 15 14:00:52 tm-stc sunshine[1294]: [2024:07:15:14:00:52]: Error: [h264_vaapi @ 0x7240ec18a840] No usable encoding entrypoint found for profile VAProfileH
264High (7).
lip 15 14:00:52 tm-stc sunshine[1294]: [2024:07:15:14:00:52]: Info: Retrying with fallback configuration options for [h264_vaapi] after error: Nie zaimplemen
towana funkcja
lip 15 14:00:52 tm-stc sunshine[1294]: [2024:07:15:14:00:52]: Info: vaapi vendor: Mesa Gallium driver 24.2.0-devel for AMD Radeon RX 580 Series (radeonsi, po
laris10, LLVM 19.0.0, DRM 3.57, 6.9.9-arch1-1)
lip 15 14:00:52 tm-stc sunshine[1294]: [2024:07:15:14:00:52]: Warning: [h264_vaapi @ 0x7240ec38bf40] Driver does not support some wanted packed headers (want
ed 0xd, found 0x1).
lip 15 14:00:52 tm-stc sunshine[1294]: [2024:07:15:14:00:52]: Error: Invalid value at rbsp_stop_one_bit: bitstream ended.
lip 15 14:00:52 tm-stc sunshine[1294]: [2024:07:15:14:00:52]: Error: Failed to read unit 3 (type 12).
lip 15 14:00:52 tm-stc sunshine[1294]: [2024:07:15:14:00:52]: Error: Couldn't read packet: Invalid data found when processing input
lip 15 14:00:52 tm-stc sunshine[1294]: [2024:07:15:14:00:52]: Info: SDR color coding [Rec. 601]
lip 15 14:00:52 tm-stc sunshine[1294]: [2024:07:15:14:00:52]: Info: Color depth: 8-bit
lip 15 14:00:52 tm-stc sunshine[1294]: [2024:07:15:14:00:52]: Info: Color range: [JPEG]
lip 15 14:00:52 tm-stc sunshine[1294]: [2024:07:15:14:00:52]: Info: vaapi vendor: Mesa Gallium driver 24.2.0-devel for AMD Radeon RX 580 Series (radeonsi, po
laris10, LLVM 19.0.0, DRM 3.57, 6.9.9-arch1-1)
lip 15 14:00:52 tm-stc sunshine[1294]: [2024:07:15:14:00:52]: Error: [hevc_vaapi @ 0x7240ec38bf40] No usable encoding entrypoint found for profile VAProfileH
EVCMain (17).
lip 15 14:00:52 tm-stc sunshine[1294]: [2024:07:15:14:00:52]: Info: Retrying with fallback configuration options for [hevc_vaapi] after error: Nie zaimplemen
towana funkcja
lip 15 14:00:52 tm-stc sunshine[1294]: [2024:07:15:14:00:52]: Info: vaapi vendor: Mesa Gallium driver 24.2.0-devel for AMD Radeon RX 580 Series (radeonsi, po
laris10, LLVM 19.0.0, DRM 3.57, 6.9.9-arch1-1)
lip 15 14:00:52 tm-stc sunshine[1294]: [2024:07:15:14:00:52]: Warning: [hevc_vaapi @ 0x7240ec18a840] Driver does not advertise encoder features, using guesse
d defaults.
lip 15 14:00:52 tm-stc sunshine[1294]: [2024:07:15:14:00:52]: Warning: [hevc_vaapi @ 0x7240ec18a840] Driver does not advertise encoder block size, using gues
sed defaults.
lip 15 14:00:52 tm-stc sunshine[1294]: [2024:07:15:14:00:52]: Warning: [hevc_vaapi @ 0x7240ec18a840] Driver does not support some wanted packed headers (want
ed 0xd, found 0x1).
lip 15 14:00:52 tm-stc sunshine[1294]: [2024:07:15:14:00:52]: Info: SDR color coding [Rec. 601]
lip 15 14:00:52 tm-stc sunshine[1294]: [2024:07:15:14:00:52]: Info: Color depth: 8-bit
lip 15 14:00:52 tm-stc sunshine[1294]: [2024:07:15:14:00:52]: Info: Color range: [JPEG]
lip 15 14:00:52 tm-stc sunshine[1294]: [2024:07:15:14:00:52]: Info: vaapi vendor: Mesa Gallium driver 24.2.0-devel for AMD Radeon RX 580 Series (radeonsi, po
laris10, LLVM 19.0.0, DRM 3.57, 6.9.9-arch1-1)
lip 15 14:00:52 tm-stc sunshine[1294]: [2024:07:15:14:00:52]: Error: [av1_vaapi @ 0x7240ec18a840] No usable encoding profile found.
lip 15 14:00:52 tm-stc sunshine[1294]: [2024:07:15:14:00:52]: Info: Retrying with fallback configuration options for [av1_vaapi] after error: Nie zaimplement
owana funkcja
lip 15 14:00:52 tm-stc sunshine[1294]: [2024:07:15:14:00:52]: Info: vaapi vendor: Mesa Gallium driver 24.2.0-devel for AMD Radeon RX 580 Series (radeonsi, po
laris10, LLVM 19.0.0, DRM 3.57, 6.9.9-arch1-1)
lip 15 14:00:52 tm-stc sunshine[1294]: [2024:07:15:14:00:52]: Error: [av1_vaapi @ 0x7240ec38bf40] No usable encoding profile found.
lip 15 14:00:52 tm-stc sunshine[1294]: [2024:07:15:14:00:52]: Error: Could not open codec [av1_vaapi]: Nie zaimplementowana funkcja
lip 15 14:00:52 tm-stc sunshine[1294]: [2024:07:15:14:00:52]: Info: SDR color coding [Rec. 601]
lip 15 14:00:52 tm-stc sunshine[1294]: [2024:07:15:14:00:52]: Info: Color depth: 8-bit
lip 15 14:00:52 tm-stc sunshine[1294]: [2024:07:15:14:00:52]: Info: Color range: [JPEG]
lip 15 14:00:52 tm-stc sunshine[1294]: [2024:07:15:14:00:52]: Info: vaapi vendor: Mesa Gallium driver 24.2.0-devel for AMD Radeon RX 580 Series (radeonsi, po
laris10, LLVM 19.0.0, DRM 3.57, 6.9.9-arch1-1)
lip 15 14:00:52 tm-stc sunshine[1294]: [2024:07:15:14:00:52]: Error: [av1_vaapi @ 0x7240ec0eec80] No usable encoding profile found.
lip 15 14:00:52 tm-stc sunshine[1294]: [2024:07:15:14:00:52]: Info: Retrying with fallback configuration options for [av1_vaapi] after error: Nie zaimplement
owana funkcja
lip 15 14:00:52 tm-stc sunshine[1294]: [2024:07:15:14:00:52]: Info: vaapi vendor: Mesa Gallium driver 24.2.0-devel for AMD Radeon RX 580 Series (radeonsi, po
laris10, LLVM 19.0.0, DRM 3.57, 6.9.9-arch1-1)
lip 15 14:00:52 tm-stc sunshine[1294]: [2024:07:15:14:00:52]: Error: [av1_vaapi @ 0x7240ec38bf40] No usable encoding profile found.
lip 15 14:00:52 tm-stc sunshine[1294]: [2024:07:15:14:00:52]: Error: Could not open codec [av1_vaapi]: Nie zaimplementowana funkcja
lip 15 14:00:52 tm-stc sunshine[1294]: [2024:07:15:14:00:52]: Info: Screencasting with KMS
lip 15 14:00:52 tm-stc sunshine[1294]: [2024:07:15:14:00:52]: Info: /dev/dri/card1 -> amdgpu
lip 15 14:00:52 tm-stc sunshine[1294]: [2024:07:15:14:00:52]: Info: Found monitor for DRM screencasting
lip 15 14:00:52 tm-stc sunshine[1294]: [2024:07:15:14:00:52]: Info: Found connector ID [94]
lip 15 14:00:52 tm-stc sunshine[1294]: [2024:07:15:14:00:52]: Info: Found cursor plane [60]
lip 15 14:00:52 tm-stc sunshine[1294]: [2024:07:15:14:00:52]: Info: SDR color coding [Rec. 709]
lip 15 14:00:52 tm-stc sunshine[1294]: [2024:07:15:14:00:52]: Info: Color depth: 10-bit
lip 15 14:00:52 tm-stc sunshine[1294]: [2024:07:15:14:00:52]: Info: Color range: [JPEG]
lip 15 14:00:52 tm-stc sunshine[1294]: [2024:07:15:14:00:52]: Info: vaapi vendor: Mesa Gallium driver 24.2.0-devel for AMD Radeon RX 580 Series (radeonsi, po
laris10, LLVM 19.0.0, DRM 3.57, 6.9.9-arch1-1)
lip 15 14:00:52 tm-stc sunshine[1294]: [2024:07:15:14:00:52]: Error: [hevc_vaapi @ 0x7240ec38bf40] No usable encoding entrypoint found for profile VAProfileH
EVCMain10 (18).
lip 15 14:00:52 tm-stc sunshine[1294]: [2024:07:15:14:00:52]: Info: Retrying with fallback configuration options for [hevc_vaapi] after error: Nie zaimplemen
towana funkcja
lip 15 14:00:52 tm-stc sunshine[1294]: [2024:07:15:14:00:52]: Info: vaapi vendor: Mesa Gallium driver 24.2.0-devel for AMD Radeon RX 580 Series (radeonsi, po
laris10, LLVM 19.0.0, DRM 3.57, 6.9.9-arch1-1)
lip 15 14:00:52 tm-stc sunshine[1294]: [2024:07:15:14:00:52]: Error: [hevc_vaapi @ 0x7240ec0eec80] No usable encoding entrypoint found for profile VAProfileH
EVCMain10 (18).
lip 15 14:00:52 tm-stc sunshine[1294]: [2024:07:15:14:00:52]: Error: Could not open codec [hevc_vaapi]: Nie zaimplementowana funkcja
lip 15 14:00:52 tm-stc sunshine[1294]: [2024:07:15:14:00:52]: Warning: vaapi: h264 missing sps->vui parameters
lip 15 14:00:52 tm-stc sunshine[1294]: [2024:07:15:14:00:52]: Info: // Testing for available encoders, this may generate errors. You can safely ignore those 
errors. //
lip 15 14:00:52 tm-stc sunshine[1294]: [2024:07:15:14:00:52]: Info:
lip 15 14:00:52 tm-stc sunshine[1294]: [2024:07:15:14:00:52]: Info: // Ignore any errors mentioned above, they are not relevant. //
lip 15 14:00:52 tm-stc sunshine[1294]: [2024:07:15:14:00:52]: Info:
lip 15 14:00:52 tm-stc sunshine[1294]: [2024:07:15:14:00:52]: Info: Found H.264 encoder: h264_vaapi [vaapi]
lip 15 14:00:52 tm-stc sunshine[1294]: [2024:07:15:14:00:52]: Info: Found HEVC encoder: hevc_vaapi [vaapi]
lip 15 14:00:52 tm-stc sunshine[1294]: [2024:07:15:14:00:52]: Info: Executing [Desktop]
lip 15 14:00:52 tm-stc sunshine[1294]: [2024:07:15:14:00:52]: Info: Video encryption enabled
lip 15 14:00:52 tm-stc sunshine[1294]: [2024:07:15:14:00:52]: Info: CLIENT CONNECTED
lip 15 14:00:52 tm-stc sunshine[1294]: [2024:07:15:14:00:52]: Info: /dev/dri/card1 -> amdgpu
lip 15 14:00:52 tm-stc sunshine[1294]: [2024:07:15:14:00:52]: Info: Found display [wayland-0]
lip 15 14:00:52 tm-stc sunshine[1294]: [2024:07:15:14:00:52]: Info: Found interface: zxdg_output_manager_v1(31) version 3
lip 15 14:00:52 tm-stc sunshine[1294]: [2024:07:15:14:00:52]: Info: Found interface: wl_output(63) version 4
lip 15 14:00:52 tm-stc sunshine[1294]: [2024:07:15:14:00:52]: Info: Resolution: 1920x1080
lip 15 14:00:52 tm-stc sunshine[1294]: [2024:07:15:14:00:52]: Info: Offset: 0x0
lip 15 14:00:52 tm-stc sunshine[1294]: [2024:07:15:14:00:52]: Info: Logical size: 1280x720
lip 15 14:00:52 tm-stc sunshine[1294]: [2024:07:15:14:00:52]: Info: Name: HDMI-A-2
lip 15 14:00:52 tm-stc sunshine[1294]: [2024:07:15:14:00:52]: Info: Found monitor: Philips Consumer Electronics Company Philips FTV/0x01010101
lip 15 14:00:52 tm-stc sunshine[1294]: [2024:07:15:14:00:52]: Info: -------- Start of KMS monitor list --------
lip 15 14:00:52 tm-stc sunshine[1294]: [2024:07:15:14:00:52]: Info: Monitor 0 is HDMI-A-2: Philips Consumer Electronics Company Philips FTV/0x01010101
lip 15 14:00:52 tm-stc sunshine[1294]: [2024:07:15:14:00:52]: Info: --------- End of KMS monitor list ---------
lip 15 14:00:52 tm-stc sunshine[1294]: [2024:07:15:14:00:52]: Info: Screencasting with KMS
lip 15 14:00:52 tm-stc sunshine[1294]: [2024:07:15:14:00:52]: Info: /dev/dri/card1 -> amdgpu
lip 15 14:00:52 tm-stc sunshine[1294]: [2024:07:15:14:00:52]: Info: Found monitor for DRM screencasting
lip 15 14:00:52 tm-stc sunshine[1294]: [2024:07:15:14:00:52]: Info: Found connector ID [94]
lip 15 14:00:52 tm-stc sunshine[1294]: [2024:07:15:14:00:52]: Info: Found cursor plane [60]
lip 15 14:00:52 tm-stc sunshine[1294]: [2024:07:15:14:00:52]: Info: SDR color coding [Rec. 601]
lip 15 14:00:52 tm-stc sunshine[1294]: [2024:07:15:14:00:52]: Info: Color depth: 8-bit
lip 15 14:00:52 tm-stc sunshine[1294]: [2024:07:15:14:00:52]: Info: Color range: [MPEG]
lip 15 14:00:52 tm-stc sunshine[1294]: [2024:07:15:14:00:52]: Info: vaapi vendor: Mesa Gallium driver 24.2.0-devel for AMD Radeon RX 580 Series (radeonsi, po
laris10, LLVM 19.0.0, DRM 3.57, 6.9.9-arch1-1)
lip 15 14:00:52 tm-stc sunshine[1294]: [2024:07:15:14:00:52]: Error: [h264_vaapi @ 0x7240dc2f6380] No usable encoding entrypoint found for profile VAProfileH
264High (7).
lip 15 14:00:52 tm-stc sunshine[1294]: [2024:07:15:14:00:52]: Info: Retrying with fallback configuration options for [h264_vaapi] after error: Nie zaimplemen
towana funkcja
lip 15 14:00:52 tm-stc sunshine[1294]: [2024:07:15:14:00:52]: Info: vaapi vendor: Mesa Gallium driver 24.2.0-devel for AMD Radeon RX 580 Series (radeonsi, po
laris10, LLVM 19.0.0, DRM 3.57, 6.9.9-arch1-1)
lip 15 14:00:52 tm-stc sunshine[1294]: [2024:07:15:14:00:52]: Warning: [h264_vaapi @ 0x7240dc34cb80] Driver does not support some wanted packed headers (want
ed 0xd, found 0x1).
lip 15 14:00:52 tm-stc sunshine[1294]: [2024:07:15:14:00:52]: Error: Invalid value at rbsp_stop_one_bit: bitstream ended.
lip 15 14:00:52 tm-stc sunshine[1294]: [2024:07:15:14:00:52]: Error: Failed to read unit 3 (type 12).
lip 15 14:00:52 tm-stc sunshine[1294]: [2024:07:15:14:00:52]: Error: Couldn't read packet: Invalid data found when processing input
lip 15 14:00:53 tm-stc sunshine[1294]: connect: Przekroczony czas oczekiwania na połączenie
lip 15 14:00:53 tm-stc sunshine[1294]: [2024:07:15:14:00:53]: Error: A UPnP device has been found,  but it wasn't recognized as an IGD
lip 15 14:00:53 tm-stc sunshine[1294]: [2024:07:15:14:00:53]: Info: Found default monitor by name: alsa_output.pci-0000_00_14.2.iec958-stereo.monitor
lip 15 14:00:53 tm-stc sunshine[1294]: [2024:07:15:14:00:53]: Info: Opus initialized: 48 kHz, 2 channels, 512 kbps (total), LOWDELAY
lip 15 14:03:12 tm-stc sunshine[1294]: connect: Przekroczony czas oczekiwania na połączenie
lip 15 14:03:12 tm-stc sunshine[1294]: [2024:07:15:14:03:12]: Error: A UPnP device has been found,  but it wasn't recognized as an IGD
lip 15 14:05:22 tm-stc sunshine[1294]: connect: Przekroczony czas oczekiwania na połączenie
lip 15 14:05:22 tm-stc sunshine[1294]: [2024:07:15:14:05:22]: Error: A UPnP device has been found,  but it wasn't recognized as an IGD
lip 15 14:07:30 tm-stc sunshine[1294]: connect: Przekroczony czas oczekiwania na połączenie
lip 15 14:07:30 tm-stc sunshine[1294]: [2024:07:15:14:07:30]: Error: A UPnP device has been found,  but it wasn't recognized as an IGD
lip 15 14:09:40 tm-stc sunshine[1294]: connect: Przekroczony czas oczekiwania na połączenie
lip 15 14:09:40 tm-stc sunshine[1294]: [2024:07:15:14:09:40]: Error: A UPnP device has been found,  but it wasn't recognized as an IGD
lip 15 14:11:50 tm-stc sunshine[1294]: connect: Przekroczony czas oczekiwania na połączenie
lip 15 14:11:50 tm-stc sunshine[1294]: [2024:07:15:14:11:50]: Error: A UPnP device has been found,  but it wasn't recognized as an IGD
lip 15 14:13:58 tm-stc sunshine[1294]: connect: Przekroczony czas oczekiwania na połączenie
lip 15 14:13:58 tm-stc sunshine[1294]: [2024:07:15:14:13:58]: Error: A UPnP device has been found,  but it wasn't recognized as an IGD

[ns6089]

Working as intended now I'm afraid. The whole point of #2821 was putting bitrate back under control since it was overshooting through the roof. I guess someone can look into extending the NVENC option below into other encoders, but I have absolutely no idea how well it will work for AMD cards, especially through VAAPI.

[neatnoise]

@ns6089 so this option is not supported by vaapi, is it?

[ns6089]

VAAPI can support VBV/HRD size with max_frame_size acting as extra constraint (NVENC can't do this, at least without the manual buffer fullness manipulation). Depending on particular vendor implementation and hardware support.
FFmpeg supports both.
Sunshine exposes neither.

[gschintgen]

I can confirm the strong quality regression. For the last two days I was puzzled why I didn't notice earlier the very mushy blurry forests I was speeding through in Forza Horizon 4. At some point it was really bad and I couldn't clearly make out where I was supposed to head to. At first I thought it was related to the graphical characteristics of the landscape I was driving through or that I wasn't yet accustomed to the stream quality in a game with more realistic graphics (compared to the more artsy styles of the previous games I played.)

But I've now compiled sunshine from the very last commit before the AMD bitrate fixes (#2821) were merged and test-drove that build for some 75 minutes. The graphics were gorgeous as ever; like sitting in front of the host PC.

I do acknowledge that some people had actual issues with the occasional overly large frames (even going as far as having the stream break down) and I'm very grateful that this issue has been looked into! And fixed. But now the streaming is very degraded for other users as it seems.

That's obviously an unfortunate situation.

I also admit that my latest "gorgeous" streaming session resulted in the following log excerpt:

[2024:07:15:23:07:39]: Info: CLIENT CONNECTED
(...)
[2024:07:15:23:07:41]: Info: Adjust this behavior in the Applications tab or apps.json if this is not what you want.
[2024:07:15:23:37:44]: Warning: Number of fragments for reed solomon exceeds DATA_SHARDS_MAX
502 > 255, skipping error correction
[2024:07:15:23:37:44]: Warning: Number of fragments for reed solomon exceeds DATA_SHARDS_MAX
502 > 255, skipping error correction
[2024:07:15:23:37:44]: Warning: Number of fragments for reed solomon exceeds DATA_SHARDS_MAX
501 > 255, skipping error correction
[2024:07:16:00:23:33]: Info: CLIENT DISCONNECTED

My point is: I didn't even notice anything special (maybe a few small stutters in an hour long session). So at least for my use case the cure is worse than the disease.

This is all with Moonlight configured for 1080p with 60Mbps. The host has an AMD RX6650XT. (Everything wired)

Would it be possible to make the buffer limiting optional via the Web UI? (For the time being, or before the next stable release if nothing else changes until then.)

I'll gladly help play-testing any potential proof-of-concept patch or PR etc.

[ReenigneArcher]

Is it possible that changes to ffmpeg are causing this? This was merged 2 days ago: #2843

That update included a vaapi patch to ffmpeg, via LizardByte/build-deps#335

[cgutman]

I can try max_frame_size in place of rc_buffer_size to see if that can limit frame size without causing large quality degradations.

[ns6089]

Is it possible that changes to ffmpeg are causing this?

Very unlikely

I can try max_frame_size in place of rc_buffer_size

Looking at ffmpeg code, the initialization process fails when there is no support on the backend.
So we will need another encoder retry pass just for this option.
Also I think it was intended as a stricter constraint on top of VBV/HRD and won't improve quality if we keep it as a single frame at average bitrate. The only real way is to relax this single-frame constraint in some controllable manner.

[garza-utsa]

Also had to revert my sunshine too, RX550 here, and the regression indeed makes sunshine unusable, very blocky and unreadable.

Thanks for the commit reference by @neatnoise. For those wanting to revert, the magic commit to checkout was at 5cea1e1 for me. Was a little tricky as I'm on arch (btw), had to grab an older boost-libs, luckily had a copy in my cache at v1.83.0-9.1 which allowed me to compile, that was the only dep change I had to make.

To get a successful build, I did disable the docs and tests targets as it still wasn't the right boost-libs to get past all the tests and the docs had references to assets that no longer exist on lizardbyte's servers (logo svgs).

Don't forget to reinit your submodules after checking out.

[neatnoise]

@garza-utsa I have to agree with you unfortunately. In my opinion network little spikes are better option instead of large quality regression. I don't have to downgrade boost libs in my up to date arch system to compile it. I apply this patch in every dev build:

diff --git a/src/platform/linux/vaapi.cpp b/src/platform/linux/vaapi.cpp
index ada63700..cd1d81f6 100644
--- a/src/platform/linux/vaapi.cpp
+++ b/src/platform/linux/vaapi.cpp
@@ -129,16 +129,6 @@ namespace va {
       return 0;
     }

-    void
-    init_codec_options(AVCodecContext *ctx, AVDictionary *options) override {
-      // Don't set the RC buffer size when using H.264 on Intel GPUs. It causes
-      // major encoding quality degradation.
-      auto vendor = vaQueryVendorString(va_display);
-      if (ctx->codec_id != AV_CODEC_ID_H264 || (vendor && !strstr(vendor, "Intel"))) {
-        ctx->rc_buffer_size = ctx->bit_rate * ctx->framerate.den / ctx->framerate.num;
-      }
-    }
-
     int
     set_frame(AVFrame *frame, AVBufferRef *hw_frames_ctx_buf) override {
       this->hwframe.reset(frame);
diff --git a/src/video.cpp b/src/video.cpp
index 7758d68a..10c91d1e 100644
--- a/src/video.cpp
+++ b/src/video.cpp
@@ -859,7 +859,6 @@ namespace video {
       std::make_optional<encoder_t::option_t>("qp"s, &config::video.qp),
       "h264_vaapi"s,
     },
-    // RC buffer size will be set in platform code if supported
     LIMITED_GOP_SIZE | PARALLEL_ENCODING | SINGLE_SLICE_ONLY | NO_RC_BUF_LIMIT
   };
 #endif
@@ -1611,9 +1610,6 @@ namespace video {
         return nullptr;
       }

-      // Allow the encoding device a final opportunity to set/unset or override any options
-      encode_device->init_codec_options(ctx.get(), options);
-
       if (auto status = avcodec_open2(ctx.get(), codec, &options)) {
         char err_str[AV_ERROR_MAX_STRING_SIZE] { 0 };

[skunk-funk]

It appears I've opened the same issue here - #3181

Not sure how to consolidate these.

I will say, for some reason on intel iGPU this isn't an issue - so if you have an intel processor, use it to get around this!

The situation is really making this thing unusable.