Skip to content

libmpv: caller-owned Vulkan render API for gpu-next #18343

Description

@grepole121

Expected behavior of the wanted feature

I would like libmpv to expose a supported Vulkan render backend that renders
gpu-next output into a caller-owned VkImage, on a caller-owned Vulkan
device and queue, with explicit acquire/completion synchronization.

The use case is an all-Vulkan Wayland application that owns one raw toplevel
surface and its color-management protocol state. Qt Quick renders into one
FP16 image, mpv renders video/subtitles into another, and the application
composes both into its swapchain. The OpenGL render API cannot share this
Vulkan ownership graph, while letting mpv create a VO surface would introduce
a second surface/presentation/color owner and prevent application composition.

I have a separable three-commit prototype based on mpv v0.41.0 commit
41f6a645068483470267271e1d09966ca3b9f413. It adds:

  • MPV_RENDER_API_TYPE_VULKAN and a versioned render_vk.h ABI;
  • caller-supplied instance/device/graphics queue plus mandatory queue-lock
    callbacks;
  • caller-supplied image/view/format/usage/layout and target-generation data;
  • binary or timeline acquire and render-complete semaphores;
  • explicit surface-replaced, targets-retired and device-lost controls;
  • source/target color metadata and per-frame zero-copy/hardware-decode facts;
  • a gpu-next external-target backend that imports the device/image through
    libplacebo without presenting or using vkDeviceWaitIdle in the frame loop;
  • ABI/negative/synchronization tests and a buildable standalone Vulkan host.

The prototype passes all 35 configured mpv tests in Debug and fail-fast
ASan/UBSan builds. Five SDR/subtitle/HDR10/HLG cases per profile also run on
lavapipe. On AMD/RADV it renders VA-API/DRM PRIME video through the imported
device, reports the hardware mapping, and passes Vulkan validation. A native
Qt Quick host has also exercised repeated target replacement and device-loss
recovery.

Current mpv master at 1d15686142fd5d53c954aab7526cedab05ef9dc3
still exposes render.h and render_gl.h but no Vulkan public header or
MPV_RENDER_API_TYPE_VULKAN symbol. I could not find an existing feature
request for a caller-owned Vulkan libmpv target.

Before turning the prototype into a pull request, I would appreciate maintainer
direction on the intended abstraction and patch split, particularly:

  1. whether a public Vulkan-specific header is acceptable or the generic render
    API should expose a different backend-neutral external-image contract;
  2. whether libmpv should import a caller-created libplacebo/Vulkan device, or
    accept a smaller set of Vulkan handles/features and create its own wrapper;
  3. the preferred synchronization/target-retirement contract; and
  4. whether the gpu-next external-target work should land independently from
    the public ABI and example.

I can provide the complete patch series and rebase it onto current master once
the direction is agreed.

Alternative behavior of the wanted feature

  • Keep using the OpenGL render API. This adds an API/device boundary and does
    not satisfy an all-Vulkan host that must compose into its own image.
  • Give mpv a window/Wayland surface. This prevents same-pass application UI
    composition and gives mpv or its WSI path a second presentation and color
    owner.
  • Depend on mpv/libplacebo private internals. This has no stable ABI and is not
    suitable for a distributable client.
  • Maintain the prototype as a downstream fork. That works, but carries an
    avoidable security/rebase/API divergence cost.

Log File

This is a structural API request rather than a playback bug, so there is no
failure log. The prototype has fail-fast validation and synchronization tests;
I can attach their output or a focused Vulkan validation log if useful.

Sample Files

No media-specific sample is required. The standalone prototype host uses
generated SDR, HDR10, HLG and subtitle fixtures for validation.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions