[jellyfin/ffmpeg] OpenCL missing from FFmpeg disables hardware acceleration on Intel

[WesSouza]

Is this a new Bug?

• I checked that the bug hasn't been reported before

Package Name

jellyfin

Package Version

10.8.13-13

Device Model

DS718+

Device Architecture

x86_64

Firmware Version

DSM 7.2.1-69057 Update 4

What happened?

Playing a video on Jellyfin with hardware acceleration enabled produces a very slow outcome compared to the same setup on a Docker container on the same machine.

It's also slower than software.

Reproduction steps

1. Configure transcoding hardware acceleration
2. Play a video that should be able to be remuxed
3. Transcoding happens at 8fps (150fps expected)

Install Log

2024/02/20 15:29:01	install jellyfin 10.8.13-13 Begin preinst
2024/02/20 15:29:01	Begin initialize_variables
2024/02/20 15:29:01	End initialize_variables
2024/02/20 15:29:01	===> Step preinst. STATUS=INSTALL USER=jellyfin GROUP=sc-media SHARE_PATH=
2024/02/20 15:29:01	install jellyfin 10.8.13-13 End preinst ret=[0]
2024/02/20 15:29:01	install jellyfin 10.8.13-13 Begin /bin/mkdir -p /volume1/@appstore/jellyfin -m 755
2024/02/20 15:29:01	install jellyfin 10.8.13-13 End /bin/mkdir -p /volume1/@appstore/jellyfin -m 755 ret=[0]
2024/02/20 15:29:01	install jellyfin 10.8.13-13 Begin /bin/rm -rf /volume1/@appstore/jellyfin
2024/02/20 15:29:01	install jellyfin 10.8.13-13 End /bin/rm -rf /volume1/@appstore/jellyfin ret=[0]
2024/02/20 15:29:01	install jellyfin 10.8.13-13 Begin /bin/mv -f /volume1/@tmp/synopkg/install.QbEjjq/package /volume1/@appstore/jellyfin
2024/02/20 15:29:01	install jellyfin 10.8.13-13 End /bin/mv -f /volume1/@tmp/synopkg/install.QbEjjq/package /volume1/@appstore/jellyfin ret=[0]
2024/02/20 15:29:01	install jellyfin 10.8.13-13 Begin /bin/rm -rf /var/packages/jellyfin
2024/02/20 15:29:01	install jellyfin 10.8.13-13 End /bin/rm -rf /var/packages/jellyfin ret=[0]
2024/02/20 15:29:01	install jellyfin 10.8.13-13 Begin /bin/mkdir -p /var/packages/jellyfin -m 755
2024/02/20 15:29:01	install jellyfin 10.8.13-13 End /bin/mkdir -p /var/packages/jellyfin -m 755 ret=[0]
2024/02/20 15:29:01	install jellyfin 10.8.13-13 Begin /bin/touch /var/packages/jellyfin/installing
2024/02/20 15:29:01	install jellyfin 10.8.13-13 End /bin/touch /var/packages/jellyfin/installing ret=[0]
2024/02/20 15:29:01	install jellyfin 10.8.13-13 Begin /bin/mv -f /volume1/@tmp/synopkg/install.QbEjjq/INFO /var/packages/jellyfin/INFO
2024/02/20 15:29:01	install jellyfin 10.8.13-13 End /bin/mv -f /volume1/@tmp/synopkg/install.QbEjjq/INFO /var/packages/jellyfin/INFO ret=[0]
2024/02/20 15:29:01	install jellyfin 10.8.13-13 Begin /bin/rm -rf /var/packages/jellyfin/scripts
2024/02/20 15:29:01	install jellyfin 10.8.13-13 End /bin/rm -rf /var/packages/jellyfin/scripts ret=[0]
2024/02/20 15:29:01	install jellyfin 10.8.13-13 Begin /bin/mv -f /volume1/@tmp/synopkg/install.QbEjjq/scripts /var/packages/jellyfin/scripts
2024/02/20 15:29:01	install jellyfin 10.8.13-13 End /bin/mv -f /volume1/@tmp/synopkg/install.QbEjjq/scripts /var/packages/jellyfin/scripts ret=[0]
2024/02/20 15:29:01	install jellyfin 10.8.13-13 Begin /bin/rm -rf /var/packages/jellyfin/WIZARD_UIFILES
2024/02/20 15:29:01	install jellyfin 10.8.13-13 End /bin/rm -rf /var/packages/jellyfin/WIZARD_UIFILES ret=[0]
2024/02/20 15:29:01	install jellyfin 10.8.13-13 Begin /bin/mv -f /volume1/@tmp/synopkg/install.QbEjjq/WIZARD_UIFILES /var/packages/jellyfin/WIZARD_UIFILES
2024/02/20 15:29:01	install jellyfin 10.8.13-13 End /bin/mv -f /volume1/@tmp/synopkg/install.QbEjjq/WIZARD_UIFILES /var/packages/jellyfin/WIZARD_UIFILES ret=[0]
2024/02/20 15:29:01	install jellyfin 10.8.13-13 Begin /bin/rm -rf /var/packages/jellyfin/conf
2024/02/20 15:29:01	install jellyfin 10.8.13-13 End /bin/rm -rf /var/packages/jellyfin/conf ret=[0]
2024/02/20 15:29:01	install jellyfin 10.8.13-13 Begin /bin/mv -f /volume1/@tmp/synopkg/install.QbEjjq/conf /var/packages/jellyfin/conf
2024/02/20 15:29:01	install jellyfin 10.8.13-13 End /bin/mv -f /volume1/@tmp/synopkg/install.QbEjjq/conf /var/packages/jellyfin/conf ret=[0]
2024/02/20 15:29:08	install jellyfin 10.8.13-13 Begin postinst
2024/02/20 15:29:08	Begin initialize_variables
2024/02/20 15:29:08	End initialize_variables
2024/02/20 15:29:08	===> Step postinst. STATUS=INSTALL USER=jellyfin GROUP=sc-media SHARE_PATH=
2024/02/20 15:29:08	Begin save_wizard_variables
2024/02/20 15:29:08	End save_wizard_variables
2024/02/20 15:29:08	Begin syno_sync_var_folder
2024/02/20 15:29:08	End syno_sync_var_folder
2024/02/20 15:29:08	Begin service_postinst
2024/02/20 15:29:08	End service_postinst
2024/02/20 15:29:08	install jellyfin 10.8.13-13 End postinst ret=[0]
2024/02/20 15:29:09	install jellyfin 10.8.13-13 Begin start-stop-status start
2024/02/20 15:29:09	install jellyfin 10.8.13-13 End start-stop-status start ret=[0]
2024/02/20 15:36:28	stop jellyfin 10.8.13-13 Begin start-stop-status stop
2024/02/20 15:36:29	stop jellyfin 10.8.13-13 End start-stop-status stop ret=[0]
2024/02/20 15:36:39	start jellyfin 10.8.13-13 Begin start-stop-status start
2024/02/20 15:36:39	start jellyfin 10.8.13-13 End start-stop-status start ret=[0]
2024/02/20 16:52:25	start jellyfin 10.8.13-13 Begin start-stop-status stop
2024/02/20 16:52:25	start jellyfin 10.8.13-13 End start-stop-status stop ret=[0]
2024/02/20 16:52:29	start jellyfin 10.8.13-13 Begin start-stop-status start
2024/02/20 16:52:29	start jellyfin 10.8.13-13 End start-stop-status start ret=[0]

Other Logs

Here's the ffmpeg logs from both installations, indicating a big speed difference:

• ffmpeg-docker.log
• ffmpeg-syno.log

Additional Information

I also tried to install ffmpeg5 and ffmpeg6 update the --ffmpeg arg on /var/packages/jellyfin/scripts/service-setup but the results were almost identical.

I followed this guide to setup the Docker container.

[mreid-tt]

@WesSouza, out of curiosity, on the same hardware what was the conversion rate in Docker?

[WesSouza]

I just reenabled the Docker project and realized the videos are being remuxed. I believe there's something wrong somewhere that prevents the SynoCommunity version of Jellyfin not to remux.

The conversion seems equally slow unfortunately.

Edit: I tested different videos. Transcoding is much slower on the Syno package than when using Docker for the same video.

[WesSouza]

@mreid-tt Added two ffmpeg logs to the description (one from this package, one from the docker instance), showing the differences.

So far:

• ffmpeg version is different (I tried with 5.1.3-2 from the SynoCommunity and got the same results)
• The JSON on docker sets "IsAVC": true and "NalLengthSize": "4" versus false and "0"

I also reenabled the same hardware acceleration settings on both instances and the ffmpeg command has two differences:

# Present on Docker, missing from Syno
-hwaccel_output_format vaapi

# Docker
-vf "setparams=color_primaries=bt709:color_trc=bt709:colorspace=bt709,deinterlace_vaapi=rate=frame,scale_vaapi=format=nv12:extra_hw_frames=24,hwmap=derive_device=qsv,format=qsv"

# Syno
-vf "setparams=color_primaries=bt709:color_trc=bt709:colorspace=bt709,yadif=0:-1:0,scale=trunc(min(max(iw\,ih*a)\,min(1920\,1080*a))/2)*2:trunc(min(max(iw/a\,ih)\,min(1920/a\,1080))/2)*2,format=nv12"

Both playbacks were attempted on the same video on the same browser (Safari on macOS), and Jellyfin's display version is the same on both instances.

[WesSouza]

The lack of -hwaccel_output_format vaapi seems to be caused by Syno's ffmpeg not providing the opencl hardware acceleration and filters.

From Jellyfin's code:

1. The code that adds -hwaccel_output_format vaapi checks hwSurface (here)
2. hwSurface requires isIntelVaapiOclSupported, which requires IsOpenclFullSupported()
3. isIntelVaapiOclSupported requires both opencl hardware acceleration, scale_opencl filter and specific filters tonemap_opencl and overlay_opencl (here)

All of these are available on the Docker compose instance, but absent from Syno's ffmpeg.

[mreid-tt]

@WesSouza, thank you for providing additional detail.

@th0ma7, I remember you working with the ffmpeg package before. Could you share any insights that might help resolve this issue?

[th0ma7]

@WesSouza there are a few caveats when using ffmpeg on you nas, your user must be part of the videodriver group.

vainfo will confirm what access you do have from your user account in use when accessing ffmpeg. See: https://github.com/SynoCommunity/spksrc/wiki/FAQ-FFmpeg

[WesSouza]

I have no idea how to add myself to the videodriver group on Synology, but I can run those as root.

root@WexDrive:~# vainfo 
Trying display: drm
libva info: VA-API version 1.18.0
libva info: Trying to open /var/packages/ffmpeg/target/lib/iHD_drv_video.so
libva info: Found init function __vaDriverInit_1_18
libva info: va_openDriver() returns 0
vainfo: VA-API version: 1.18 (libva 2.18.1)
vainfo: Driver version: Intel iHD driver for Intel(R) Gen Graphics - 23.1.6 (3359d2df)
vainfo: Supported profile and entrypoints
      VAProfileNone                   :	VAEntrypointVideoProc
      VAProfileNone                   :	VAEntrypointStats
      VAProfileMPEG2Simple            :	VAEntrypointVLD
      VAProfileMPEG2Main              :	VAEntrypointVLD
      VAProfileH264Main               :	VAEntrypointVLD
      VAProfileH264Main               :	VAEntrypointEncSlice
      VAProfileH264Main               :	VAEntrypointFEI
      VAProfileH264Main               :	VAEntrypointEncSliceLP
      VAProfileH264High               :	VAEntrypointVLD
      VAProfileH264High               :	VAEntrypointEncSlice
      VAProfileH264High               :	VAEntrypointFEI
      VAProfileH264High               :	VAEntrypointEncSliceLP
      VAProfileVC1Simple              :	VAEntrypointVLD
      VAProfileVC1Main                :	VAEntrypointVLD
      VAProfileVC1Advanced            :	VAEntrypointVLD
      VAProfileJPEGBaseline           :	VAEntrypointVLD
      VAProfileJPEGBaseline           :	VAEntrypointEncPicture
      VAProfileH264ConstrainedBaseline:	VAEntrypointVLD
      VAProfileH264ConstrainedBaseline:	VAEntrypointEncSlice
      VAProfileH264ConstrainedBaseline:	VAEntrypointFEI
      VAProfileH264ConstrainedBaseline:	VAEntrypointEncSliceLP
      VAProfileVP8Version0_3          :	VAEntrypointVLD
      VAProfileVP8Version0_3          :	VAEntrypointEncSlice
      VAProfileHEVCMain               :	VAEntrypointVLD
      VAProfileHEVCMain               :	VAEntrypointEncSlice
      VAProfileHEVCMain               :	VAEntrypointFEI
      VAProfileHEVCMain10             :	VAEntrypointVLD
      VAProfileVP9Profile0            :	VAEntrypointVLD

root@WexDrive:~# /var/packages/ffmpeg/target/bin/ffmpeg -hide_banner -hwaccels
Hardware acceleration methods:
vaapi
qsv
drm


root@WexDrive:~# /var/packages/ffmpeg/target/bin/ffmpeg -hide_banner -filters | grep opencl
root@WexDrive:~#

Now if I run these commands on the Docker container:

root@WexDrive:~# docker exec -i -t jellyfin bash
root@50ca4db67f1c:/# /usr/lib/jellyfin-ffmpeg/vainfo
Trying display: drm
libva info: VA-API version 1.20.0
libva info: Trying to open /usr/lib/jellyfin-ffmpeg/lib/dri/iHD_drv_video.so
libva info: Found init function __vaDriverInit_1_20
libva info: va_openDriver() returns 0
vainfo: VA-API version: 1.20 (libva 2.20.0)
vainfo: Driver version: Intel iHD driver for Intel(R) Gen Graphics - 24.1.1 (f5f09c4)
vainfo: Supported profile and entrypoints
      VAProfileNone                   :	VAEntrypointVideoProc
      VAProfileNone                   :	VAEntrypointStats
      VAProfileMPEG2Simple            :	VAEntrypointVLD
      VAProfileMPEG2Main              :	VAEntrypointVLD
      VAProfileH264Main               :	VAEntrypointVLD
      VAProfileH264Main               :	VAEntrypointEncSlice
      VAProfileH264Main               :	VAEntrypointFEI
      VAProfileH264Main               :	VAEntrypointEncSliceLP
      VAProfileH264High               :	VAEntrypointVLD
      VAProfileH264High               :	VAEntrypointEncSlice
      VAProfileH264High               :	VAEntrypointFEI
      VAProfileH264High               :	VAEntrypointEncSliceLP
      VAProfileVC1Simple              :	VAEntrypointVLD
      VAProfileVC1Main                :	VAEntrypointVLD
      VAProfileVC1Advanced            :	VAEntrypointVLD
      VAProfileJPEGBaseline           :	VAEntrypointVLD
      VAProfileJPEGBaseline           :	VAEntrypointEncPicture
      VAProfileH264ConstrainedBaseline:	VAEntrypointVLD
      VAProfileH264ConstrainedBaseline:	VAEntrypointEncSlice
      VAProfileH264ConstrainedBaseline:	VAEntrypointFEI
      VAProfileH264ConstrainedBaseline:	VAEntrypointEncSliceLP
      VAProfileVP8Version0_3          :	VAEntrypointVLD
      VAProfileVP8Version0_3          :	VAEntrypointEncSlice
      VAProfileHEVCMain               :	VAEntrypointVLD
      VAProfileHEVCMain               :	VAEntrypointEncSlice
      VAProfileHEVCMain               :	VAEntrypointFEI
      VAProfileHEVCMain10             :	VAEntrypointVLD
      VAProfileVP9Profile0            :	VAEntrypointVLD

root@50ca4db67f1c:/# /usr/lib/jellyfin-ffmpeg/ffmpeg -hide_banner -hwaccels
Hardware acceleration methods:
cuda
vaapi
qsv
drm
opencl
vulkan

root@50ca4db67f1c:/# /usr/lib/jellyfin-ffmpeg/ffmpeg -hide_banner -filters | grep opencl
 ... avgblur_opencl    V->V       Apply average blur filter
 ... boxblur_opencl    V->V       Apply boxblur filter to input video
 ... colorkey_opencl   V->V       Turns a certain color into transparency. Operates on RGB colors.
 ... convolution_opencl V->V       Apply convolution mask to input video
 ... deshake_opencl    V->V       Feature-point based video stabilization filter
 ... dilation_opencl   V->V       Apply dilation effect
 ... erosion_opencl    V->V       Apply erosion effect
 ... nlmeans_opencl    V->V       Non-local means denoiser through OpenCL
 ... overlay_opencl    VV->V      Overlay one video on top of another
 ... pad_opencl        V->V       Pad the input video.
 ... prewitt_opencl    V->V       Apply prewitt operator
 ... program_opencl    N->V       Filter video using an OpenCL program
 ... remap_opencl      VVV->V     Remap pixels using OpenCL.
 ... roberts_opencl    V->V       Apply roberts operator
 ... scale_opencl      V->V       Scale the input video size through OpenCL.
 ... sobel_opencl      V->V       Apply sobel operator
 ... tonemap_opencl    V->V       Perform HDR to SDR conversion with tonemapping.
 ... transpose_opencl  V->V       Transpose input video
 ... unsharp_opencl    V->V       Apply unsharp mask to input video
 ... xfade_opencl      VV->V      Cross fade one video with another video.
 ... openclsrc         |->V       Generate video using an OpenCL program

What am I missing here? All signs point to ffmpeg missing opencl.

[th0ma7]

@WesSouza at first read it wasn't that obvious that you where referring to opencl missing requirements. Indeed, opencl is not integrated currently in SynoCommunity's ffmpeg. It's on my todo list but didn't got to it yet.

[WesSouza]

@th0ma7 is there an open issue for it I should be watching? Or something I could do to help?

[th0ma7]

Just rename this ticket to be more relevant to opencl and I'll look into that during my next upgrade cycle.