Proposal: list of DRM modifiers as VASurfaceAttrib

[emersion]

Use-case: with AMD hardware, we want to decode NV12 buffers via VA-API, then directly display them via KMS. This works fine if the KMS planes aren't rotated, but fails when we try to rotate them.

The reason is that VA-API gives us some LINEAR buffers and AMD display hardware can't rotate LINEAR buffers. We need tiled buffers.

To fix this, it would be nice to be able to provide a list of DRM modifiers when creating a VASurface. The driver would only be allowed to pick a modifier from this list.

This is how other APIs handle this: both GBM and Vulkan let users pass a list of modifiers when allocating memory.

We could add VASurfaceAttribDRMFormatModifiers, with a pointer to a list of modifiers.

[XinfengZhang]

so , the surface attribute is used to report surface mortifiers types supported by dedicate config (h264 enc, VPP etc)

[emersion]

The goal here is to have the client pass a list of possible modifiers to the VA-API implementation. KMS doesn't accept all modifiers, if you want to have a direct VA-API → KMS path in a client you need to convince VA-API to use a modifier which is supported by KMS.

The goal is not to check the set of modifiers supported by a given VA-API configuration.

See for instance the GBM API: https://gitlab.freedesktop.org/mesa/mesa/-/blob/55e23f32c321cd51d1c23036edcf24d451ac63ad/src/gbm/main/gbm.h#L276. I'd like to have something like this for creating VA-API surfaces.

[emersion]

Ah, maybe I misunderstood your comment. Are you suggesting to make vaQuerySurfaceAttributes with VASurfaceAttribDRMFormatModifiers return the set of supported modifiers for a given VAConfigID? Yes, that sounds like this would be nice to have, although this is not 100% necessary for my use-case.

What the use-case described in the first comment would need is to accept VASurfaceAttribDRMFormatModifiers as a vaCreateSurfaces attrib.

I'd be willing to send a pull request to implement these two things, would you be happy with this design @XinfengZhang?

[emersion]

I've started an implementation in #505.

I've had trouble with the vaQuerySurfaceAttributes part:

• The list of DRM format modifiers needs to be per-format.
• The VA-API user has no way to provide an array of modifiers that the driver could fill, because the attributes are always overwritten.

So I've left it out for now.