[Bug]: AMF encoding does not work on Linux past driver version 21.30

[hannesmann]

Describe the bug
Using any version of Radeon Software for Linux newer than 21.30 causes AMF to fail when attempting to encode video on an RX 6700 XT. This issue also occurred on an RX 590.

I usually use the unofficial AUR package for AMF to stream with OBS, but I can also reproduce the issue on Ubuntu 20.04 LTS which is a supported distro for AMF.

To Reproduce
Steps to reproduce the behavior:

1. Install any version of Radeon Software past 21.30.
2. Use the h264_amf encoder in FFmpeg or run the SimpleEncoder sample included in this repository.

Setup:

• OS: Arch Linux (also tested on Ubuntu 20.04 LTS)
• Driver Version: >= 21.40
• GPU: RX 6700 XT
• Which component has the issue: Encoder

Tests were done with AMD's proprietary Vulkan driver and not RADV. Using RADV causes a segfault on startup.

Debug Log:
When running ffmpeg -loglevel verbose -i input.mp4 -c:v h264_amf -log_to_dbg true output.mp4

[h264_amf @ 0x562bcbe259c0] AMF initialisation succeeded via Vulkan.
0122-06-06 23:09:34 BE10E040 [AMFEncoderCoreH264]   Error: ../../../../../runtime/src/components/EncoderCore/EncoderCoreH264Impl.cpp(187):Assertion failed:Failed to create service!
0122-06-06 23:09:34 BE10E040 [AMFEncoderCoreH264]   Error: ../../../../../runtime/src/components/EncoderCore/EncoderCoreH264Impl.cpp(319):AMF_ERROR 1 : AMF_FAIL: Failed to create encode service!
0122-06-06 23:09:34 BE10E040 [AMFEncoderCoreH264] Warning: CreateServices failed.
[h264_amf @ 0x562bcbd6b740] CreateComponent(AMFVideoEncoderVCE_AVC) failed with error 1
Error initializing output stream 0:0 -- Error while opening encoder for output stream #0:0 - maybe incorrect parameters such as bit_rate, rate, width or height

When running SimpleEncoder input.mp4

0122-06-06 23:02:48 E743F740 [AMFEncoderCoreH264]   Error: ../../../../../runtime/src/components/EncoderCore/EncoderCoreH264Impl.cpp(187):Assertion failed:Failed to create service!
0122-06-06 23:02:48 E743F740 [AMFEncoderCoreH264]   Error: ../../../../../runtime/src/components/EncoderCore/EncoderCoreH264Impl.cpp(187):Assertion failed:Failed to create service!
0122-06-06 23:02:48 E743F740 [AMFEncoderCoreH264]   Error: ../../../../../runtime/src/components/EncoderCore/EncoderCoreH264Impl.cpp(319):AMF_ERROR 1 : AMF_FAIL: Failed to create encode service!
0122-06-06 23:02:48 E743F740 [AMFEncoderCoreH264]   Error: ../../../../../runtime/src/components/EncoderCore/EncoderCoreH264Impl.cpp(319):AMF_ERROR 1 : AMF_FAIL: Failed to create encode service!
0122-06-06 23:02:48 E743F740 [AMFEncoderCoreH264] Warning: CreateServices failed.
0122-06-06 23:02:48 E743F740 [AMFEncoderCoreH264] Warning: CreateServices failed.
0122-06-06 23:02:48 E743F740 [SimpleEncoder]   Error: ../../../../public/samples/CPPSamples/SimpleEncoder/SimpleEncoder.cpp(177):AMF_ERROR 1 : AMF_FAIL: CreateComponent(AMFVideoEncoderVCE_AVC) failed
0122-06-06 23:02:48 E743F740 [SimpleEncoder]   Error: ../../../../public/samples/CPPSamples/SimpleEncoder/SimpleEncoder.cpp(177):AMF_ERROR 1 : AMF_FAIL: CreateComponent(AMFVideoEncoderVCE_AVC) failed
SimpleEncoder: ../../../../public/samples/CPPSamples/SimpleEncoder/SimpleEncoder.cpp:177: int main(int, char**): Assertion `0' failed.

Expected behavior
AMF should be able to encode the video successfully.

[MikhailAMD]

It looks like you have an installation issue, some mismatch between AMF/EncodeCore and VCN FW. What is your driver install command line?

[hannesmann]

some mismatch between AMF/EncodeCore and VCN FW

Output from dmesg:

[    5.285797] [drm] VCN(0) decode is enabled in VM mode
[    5.285798] [drm] VCN(0) encode is enabled in VM mode
[    7.368579] [drm] Found VCN firmware Version ENC: 1.20 DEC: 2 VEP: 0 Revision: 5
[    7.368582] amdgpu 0000:06:00.0: amdgpu: Will use PSP to load VCN firmware
[    8.063896] [drm] VCN decode and encode initialized successfully(under DPG Mode).

What is your driver install command line

I used a fresh install when testing on Ubuntu. All I did was download the 22.10.2 deb package from amd.com and run:

sudo amdgpu-install --usecase=workstation --vulkan=pro
sudo apt install amf-amdgpu-pro

[GloriousEggroll]

just dropping a note here to verify the same thing -- tested on both an rx 6600, 6700 XT, and 6900 XT, 21.40.2 with amf version 1.4.23 is the last version that works with these cards. I -did- also test 22.10.2 with an RX 5500 and an RX 580 -- both worked but none of the 6000 series worked.

I will test the 22.10.3 and 22.20 drivers on the RX 5500 and RX 580 and report back -- these also did -not- work for the 6000 series.

Another note/different bug -- the 6600 came up with an unidentified name in the 22.10.2 vulkan pro drivers. (the 6700 XT and 6900 XT were fine)

[GloriousEggroll]

more progress!

if i keep the vulkan-pro files from 21.40.2:
https://repo.radeon.com/amdgpu/21.40.2/ubuntu/pool/proprietary/v/vulkan-amdgpu-pro/vulkan-amdgpu-pro_21.40.2-1350683_amd64.deb
https://repo.radeon.com/amdgpu/21.40.2/ubuntu/pool/proprietary/v/vulkan-amdgpu-pro/vulkan-amdgpu-pro_21.40.2-1350683_i386.deb

and use the amf and libamdenc from 22.20:
https://repo.radeon.com/amdgpu/22.20/ubuntu/pool/proprietary/a/amf-amdgpu-pro/amf-amdgpu-pro_1.4.26-1438747~22.04_amd64.deb
https://repo.radeon.com/amdgpu/22.20/ubuntu/pool/proprietary/liba/libamdenc-amdgpu-pro/libamdenc-amdgpu-pro_1.0-1438747~22.04_amd64.deb

-- encoding works on my 6000 series cards!
-edit-
can confirm this method works on polaris and 5000 series cards also -- so the problem is whatever changed between amf and the vulkan-pro drivers after 21.40.2

[MikhailAMD]

In general, mixing components from different drivers is dangerous. One thing I noticed: the correct AMF install command; would be:
sudo amdgpu-install --usecase=workstation,amf --vulkan=pro
I am not sure what happens when you install amf explicitly via apt.

[Trillon008]

I believe I've been able to make both h264 & hevc work with latest 22.20.1

ffmpeg -i re3_speedrun_orig.webm -vf format=nv12 -c:v hevc_amf -c:a copy re3_amfhevc.mp4

Stream mapping:
  Stream #0:0 -> #0:0 (vp9 (native) -> hevc (hevc_amf))
  Stream #0:1 -> #0:1 (copy)

encoder         : Lavf59.16.100
  Stream #0:0(eng): Video: hevc (hev1 / 0x31766568), nv12(tv, bt709/unknown/unknown, progressive), 1280x720 [SAR 1:1 DAR 16:9], q=2-31, 2000 kb/s, 30 fps, 15360 tbn (default)
    Metadata:
      DURATION        : 02:53:07.965000000
      encoder         : Lavc59.18.100 hevc_amf
  Stream #0:1(eng): Audio: opus (Opus / 0x7375704F), 48000 Hz, stereo, fltp (default)
    Metadata:
      DURATION        : 02:53:08.041000000
frame= 4747 fps=658 q=-0.0 Lsize=   40912kB time=00:02:38.62 bitrate=2112.9kbits/s speed=  22x    
video:38099kB audio:2688kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.306913%

ffmpeg -i re3_speedrun_orig.webm -c:v h264_amf -c:a copy re3_amf.mp4

Stream mapping:
  Stream #0:0 -> #0:0 (vp9 (native) -> h264 (h264_amf))
  Stream #0:1 -> #0:1 (copy)

    encoder         : Lavf59.16.100
  Stream #0:0(eng): Video: h264 (avc1 / 0x31637661), yuv420p(tv, bt709/unknown/unknown, progressive), 1280x720 [SAR 1:1 DAR 16:9], q=2-31, 2000 kb/s, 30 fps, 15360 tbn (default)
    Metadata:
      DURATION        : 02:53:07.965000000
      encoder         : Lavc59.18.100 h264_amf
  Stream #0:1(eng): Audio: opus (Opus / 0x7375704F), 48000 Hz, stereo, fltp (default)
    Metadata:
      DURATION        : 02:53:08.041000000
frame= 3131 fps=714 q=-0.0 Lsize=   27281kB time=00:01:44.72 bitrate=2134.1kbits/s speed=23.9x    
video:25431kB audio:1767kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.305157%

Maybe one of you can retry to confirm. I've used glorious's method for installing AMF

[GloriousEggroll]

I believe I've been able to make both h264 & hevc work with latest 22.20.1

ffmpeg -i re3_speedrun_orig.webm -vf format=nv12 -c:v hevc_amf -c:a copy re3_amfhevc.mp4

Stream mapping:
  Stream #0:0 -> #0:0 (vp9 (native) -> hevc (hevc_amf))
  Stream #0:1 -> #0:1 (copy)

encoder         : Lavf59.16.100
  Stream #0:0(eng): Video: hevc (hev1 / 0x31766568), nv12(tv, bt709/unknown/unknown, progressive), 1280x720 [SAR 1:1 DAR 16:9], q=2-31, 2000 kb/s, 30 fps, 15360 tbn (default)
    Metadata:
      DURATION        : 02:53:07.965000000
      encoder         : Lavc59.18.100 hevc_amf
  Stream #0:1(eng): Audio: opus (Opus / 0x7375704F), 48000 Hz, stereo, fltp (default)
    Metadata:
      DURATION        : 02:53:08.041000000
frame= 4747 fps=658 q=-0.0 Lsize=   40912kB time=00:02:38.62 bitrate=2112.9kbits/s speed=  22x    
video:38099kB audio:2688kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.306913%

ffmpeg -i re3_speedrun_orig.webm -c:v h264_amf -c:a copy re3_amf.mp4

Stream mapping:
  Stream #0:0 -> #0:0 (vp9 (native) -> h264 (h264_amf))
  Stream #0:1 -> #0:1 (copy)

    encoder         : Lavf59.16.100
  Stream #0:0(eng): Video: h264 (avc1 / 0x31637661), yuv420p(tv, bt709/unknown/unknown, progressive), 1280x720 [SAR 1:1 DAR 16:9], q=2-31, 2000 kb/s, 30 fps, 15360 tbn (default)
    Metadata:
      DURATION        : 02:53:07.965000000
      encoder         : Lavc59.18.100 h264_amf
  Stream #0:1(eng): Audio: opus (Opus / 0x7375704F), 48000 Hz, stereo, fltp (default)
    Metadata:
      DURATION        : 02:53:08.041000000
frame= 3131 fps=714 q=-0.0 Lsize=   27281kB time=00:01:44.72 bitrate=2134.1kbits/s speed=23.9x    
video:25431kB audio:1767kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.305157%

Maybe one of you can retry to confirm. I've used glorious's method for installing AMF

22.20.1 amdgpu-pro-vulkan is still not compatible with 6000 series when installing just the amdgpu-pro and amf packages. Only 21.40.2's amdgpu-pro-vulkan works. I have -not- tried this method yet:

sudo amdgpu-install --usecase=workstation,amf --vulkan=pro

[DiarrheaMcgee]

i have the same [h264_amf @ 0x562bcbd6b740] CreateComponent(AMFVideoEncoderVCE_AVC) failed with error 1 problem

[GloriousEggroll]

From AUR, install vulkan-amdgpu-pro-fix amf-amdgpu-pro-fix and AMF works on RX 6000 series graphics cards!

Look at the PKGBUILD. They are doing the same thing I recommended -- using 21.40.2 vulkan:

https://aur.archlinux.org/cgit/aur.git/tree/PKGBUILD?h=amdgpu-pro-installer-fix#n14
https://aur.archlinux.org/cgit/aur.git/tree/PKGBUILD?h=amdgpu-pro-installer-fix#n54
https://aur.archlinux.org/cgit/aur.git/tree/PKGBUILD?h=amdgpu-pro-installer-fix#n55

As mentioned -- this is not a proper fix, it's a hacky workaround

[DiarrheaMcgee]

that works perfectly

[jarrard]

I still get freeze or seg fault with these amf 21-40-2 drivers using pkgbuild provided. Not sure whats wrong.

VK_ICD_FILESNAMES=/usr/share/vulkan/icd.d/amd_pro_icd64.json:/usr/share/vulkan/icd.d/amd_pro_icd32.json OBS_USE_EGL=1 obs

[liamtay07]

@jarrard
I recommend you to simplify things and use Vulkan Prefixes.
Install from AUR amd-vulkan-prefixes
Launch OBS with:
OBS_USE_EGL=1 vk_pro obs

And your games with vk_radv
https://wiki.archlinux.org/title/Vulkan#Selecting_via_AMD_Vulkan_Prefixes

Does this change anything? Does it work for you now?

[jarrard]

Cheers it does appear to work now. I just test it by recording the desktop.

I've been using obs_vkcapture for games which I think still works but i'll toss in vk_radv also to ensure radv gets used.

Seems OBS_USE_EGL=1 vk_pro obs works in terminal but not in desktop exec=OBS_USE_EGL=1 vk_pro obs
env/export makes no difference. Seems desktop shortcuts can't pass multiple commands.

Yeah OBS_USE_EGL=1 vk_pro obs works in terminal but won't work in .desktop or sh/bash script... puzzling!

[wallcarpet40]

@jarrard If you are using KDE, try adding env to your .desktop file like this:

env VK_ICD_FILENAMES=/opt/amdgpu-pro/vulkan/icd/amd_pro_icd64.json obs

I have my json files moved out of /usr/share/vulkan/icd.d/ folder, so that no other program will try to start with the pro drivers.