Support creating GPU textures shared with other APIs

[slouken]

I still have to implement Vulkan texture import, but D3D12 and Metal work great!

Fixes #14077

[thatcosmonaut]

Don't have time to get into too many details here, but I wonder - is it unreasonable to try using the native video API extensions in Vulkan/D3D/Metal instead of doing all this mess with passing handles between an ffmpeg application? I'd have to imagine that's just what ffmpeg is doing anyway.

[slouken]

Don't have time to get into too many details here, but I wonder - is it unreasonable to try using the native video API extensions in Vulkan/D3D/Metal instead of doing all this mess with passing handles between an ffmpeg application? I'd have to imagine that's just what ffmpeg is doing anyway.

Yeah, there’s a lot of work going on in ffmpeg to support different codecs and in some cases do bitstream parsing that it doesn’t make sense to shove into SDL.

In addition, this is a generally useful way to do interop with other projects without having to expose internals.

[slouken]

(this also supports cross-process interop which is interesting for a number of applications)

[thatcosmonaut]

I'm not sure about the shape of the other APIs but Vulkan video can handle bitstream parsing and decoding, as seen in this post:
https://www.khronos.org/blog/an-introduction-to-vulkan-video

I think it would be reasonable to limit video support to formats that native hardware decode can handle.

I'm mostly just very trepidatious about allowing external contexts. I've done a lot of work to put guard rails on the GPU API but as soon as you expose contexts that's all gone. Modern graphics APIs are very sensitive to synchronization and layout issues and we cannot control that with external contexts at all. When we added external texture support to FNA3D it was a technical mess that I regretted immediately.

I would like to exhaust all possible designs and options before resorting to this.

[slouken]

I’m fine with that. My immediate motivation is to have the GPU renderer replace the metal, D3D12 and Vulkan renderers, but the metal renderer is used on iOS and macOS for playback of video. We support h264 and HEVC, and on macOS videotoolbox does require some basic parsing of the bitstream to set up the decode context.

If video decode were built into the GPU API, that would certainly reduce the number of external interop use cases.

In the meantime I’ll check in the first commit which is just about interop between the GPU API and the GPU renderer and we can come back to the rest of this later.

[slouken]

@thatcosmonaut, another use case was mentioned in #12617, where they're trying to integrate CEF for web based rendering. This is actually a pretty common use case for games that have React based UI and web authoring flows.

[lmurray]

I currently have several issues with the proposed API:

1. On Metal, CVPixelBuffer is being used which is a CoreVideo-specific API that, according to its documentation, explicitly has its pixel data stored in CPU memory. The underlying shared texture resource on Apple platforms is the IOSurface which may have its pixel data in either CPU or GPU memory. As SDL only needs the IOSurface and plane index I think the API should be adjusted to use this information instead of requiring a CVPixelBuffer which may require a wasteful GPU readback and reupload for GPU-backed IOSurfaces.

2. On D3D12, a D3D11 API is being used instead of the appropriate D3D12 API. I haven't used shared textures with D3D12 before but supposedly the D3D12_HEAP_FLAG_SHARED heap flag is used in some way.

3. On D3D12, SDL is making strong assumptions about how the keyed mutex should be acquired and released which may not be true for all external textures. SDL will need to expose appropriate properties to allow the app to inform SDL on how the keyed mutex should be used. Note that the mutex key may change for every access which may be extremely complicated to expose properly.

I have not used shared textures with Vulkan so I unfortunately cannot provide any guidance for backend.