Artifacts decoding H.264 1440p+ encoded w/ very slow preset #873
Description
It seems I have found VA-API decoding artifacting issue spanning multiple platforms, I've been able to reproduce the issue on Windows 10, Fedora Linux and Linux Mint using VLC and MPV. YouTube's video processing back-end is suffering from this issue as well.
First thought the issue lied in Kdenlive as I was exporting video using that, then tried Handbrake with a small slice of the original hours long video and had the same issue. Moved even more bare metal by trying using ffmpeg and took a 10 second slice of the cut down video were artifacting was obvious.
Long story short: Eventually found out that the underlying issue is hardware accelerated video DECODING and not the CPU encoding that I was doing using ffmpeg libraries.
I've been able to reproduce the decoding issue on 3 devices: An AMD Ryzen laptop with integrated graphics, a PC with a radeon RX 6700 XT and an older Intel laptop wih integrated graphics, the devices run Linux Mint (Ubuntu base), Fedora Linux and one with a Kubuntu and Windows 10 dual boot. Additionally, YouTube exhibits the same issue once a video like attached is processed indicating that they too likely are using VA-API.
During my testing, I thought that maybe the original file being processed in UT video codec might have been the problem. So I converted the files to huffyuv and prores raw. Then I encoded the files to h264 using ffmpeg, but the source codec made no difference since artifacting still appeared.
I've only been able to reproduce the issue with a 1440p or higher resolution video encoded to H.264 with the very slow and placebo preset, you can see the ffmpeg commands that I used for this below. Encoding a video with a different encoding preset like fast results in no visible artifacting when using VA-API hardware decoding
The issue reproduced on Windows using mpv
How to reproduce
Download a video from the Converted folder from my Nextcloud here: https://cloud.digiadvies.com/s/japLZNSZCPBpmN6
Or download this zip file, it contains a .mp4 file which exhibits the issue.
Test using mpv with hardware decoding enabled:
mpv --hwdec=auto ~/Downloads/CodecDebug/Converted/utvideo_converted.mp4
Notice artifacts popping in and out in all of the videos except for resize1280x720.mp4 and resize1920x1080.mp4.
OR
Play the videos in your browser with hardware acceleration enabled.
vainfo output on my Fedora system
Trying display: wayland
libva info: VA-API version 1.22.0
libva info: Trying to open /usr/lib64/dri-nonfree/radeonsi_drv_video.so
libva info: Trying to open /usr/lib64/dri-freeworld/radeonsi_drv_video.so
libva info: Found init function __vaDriverInit_1_22
libva info: va_openDriver() returns 0
vainfo: VA-API version: 1.22 (libva 2.22.0)
vainfo: Driver version: Mesa Gallium driver 25.3.4 for AMD Radeon RX 6700 XT (radeonsi, navi22, LLVM 21.1.8, DRM 3.64, 6.18.9-200.fc43.x86_64)
vainfo: Supported profile and entrypoints
VAProfileMPEG2Simple : VAEntrypointVLD
VAProfileMPEG2Main : VAEntrypointVLD
VAProfileVC1Simple : VAEntrypointVLD
VAProfileVC1Main : VAEntrypointVLD
VAProfileVC1Advanced : VAEntrypointVLD
VAProfileH264ConstrainedBaseline: VAEntrypointVLD
VAProfileH264ConstrainedBaseline: VAEntrypointEncSlice
VAProfileH264Main : VAEntrypointVLD
VAProfileH264Main : VAEntrypointEncSlice
VAProfileH264High : VAEntrypointVLD
VAProfileH264High : VAEntrypointEncSlice
VAProfileHEVCMain : VAEntrypointVLD
VAProfileHEVCMain : VAEntrypointEncSlice
VAProfileHEVCMain10 : VAEntrypointVLD
VAProfileHEVCMain10 : VAEntrypointEncSlice
VAProfileJPEGBaseline : VAEntrypointVLD
VAProfileVP9Profile0 : VAEntrypointVLD
VAProfileVP9Profile2 : VAEntrypointVLD
VAProfileAV1Profile0 : VAEntrypointVLD
VAProfileNone : VAEntrypointVideoProc
Commands used to convert original utvideo source to other codecs
ffmpeg -i /home/rick/Videos/Rat/CodecDebug/utvideo_source.mkv -t 10 -c:v prores_ks -profile:v 4 -an prores_raw.mov
ffmpeg -i /home/rick/Videos/Rat/CodecDebug/utvideo_source.mkv -t 10 -c:v huffyuv -an huffyuv.avi
Commands used to convert all 3 videos to h264
ffmpeg -i "/home/rick/Videos/Rat/CodecDebug/utvideo.avi" \
-map 0:v:0 \
-vf "format=yuv420p" \
-c:v libx264 \
-preset veryslow \
-crf 20 \
-an \
"/home/rick/Videos/Rat/CodecDebug/Converted/utvideo_converted.mp4"
ffmpeg -i "/home/rick/Videos/Rat/CodecDebug/huffyuv.avi" \
-map 0:v:0 \
-vf "format=yuv420p" \
-c:v libx264 \
-preset veryslow \
-crf 20 \
-an \
"/home/rick/Videos/Rat/CodecDebug/Converted/huffyuv_converted.mp4"
ffmpeg -i "/home/rick/Videos/Rat/CodecDebug/prores_raw.mov" \
-map 0:v:0 \
-vf "format=yuv420p" \
-c:v libx264 \
-preset veryslow \
-crf 20 \
-an \
"/home/rick/Videos/Rat/CodecDebug/Converted/prores_raw_converted.mp4"
Extra - Resize to 1280x720 (no artifacts appear in this one)
ffmpeg -i "/home/rick/Videos/Rat/CodecDebug/utvideo.avi" \
-map 0:v:0 \
-vf scale=1280:720 \
-pix_fmt yuv420p \
-c:v libx264 \
-preset veryslow \
-crf 20 \
-an \
"/home/rick/Videos/Rat/CodecDebug/Converted/resize1280x720.mp4"
Extra - Resize to 1920x1080 (no artifacts appear in this one)
ffmpeg -i "/home/rick/Videos/Rat/CodecDebug/utvideo.avi" \
-map 0:v:0 \
-vf scale=1920:1080 \
-pix_fmt yuv420p \
-c:v libx264 \
-preset veryslow \
-crf 20 \
-an \
"/home/rick/Videos/Rat/CodecDebug/Converted/resize1920x1080.mp4"
Extra - Resize to 3840x2160 artifacts do appear
ffmpeg -i "/home/rick/Videos/Rat/CodecDebug/utvideo.avi" \
-map 0:v:0 \
-vf scale=3840:2160 \
-pix_fmt yuv420p \
-c:v libx264 \
-preset veryslow \
-crf 20 \
-an \
"/home/rick/Videos/Rat/CodecDebug/Converted/resize3840x2160.mp4"