Vulkan backend support.

[cheako]

I just learned about rust and now am lamenting my decision to write vkwayland in CPP. This project looks to have most everything required to support a Vulkan Backend and I'd be interested in contributing.

Might I suggest to make use of vulkan-malloc and by extension dacite. I looked at vulkano and fear that it goes against the spirit of Vulakn by implementing runtime checks when a big selling point of Vulkan is that it mostly eliminates them.

[elinorbgr]

This is at least partially related to #129

[elinorbgr]

To expand a little more, vulkan support is definitely somewhere on our roadmap, though where exactly is pretty unclear. If you're interested in helping us get there, well I can only thank you for your interest, this is very welcome!

I myself am not that familiar with the whole graphics stack though, this is more @Drakulix's domain. Some discussion has already been started in #129 about support for gfx-hal, which I believe is related to Vulkan support.

The exact specifics on how we'll support Vulkan are still pretty blurry though, and your expertise on that matter will likely help. :)

[cheako]

I took a look at gfx-rs/hal and it's a Vulkan oriented abstraction across a range of graphical APIs. Of note there is no support for Direct to Display drawing, a must for any serious Compositor.

It's not clear how legacy GL and Vulkan are made to be compatible. My assumption is that it's a case of gfx-hal being a Jack of All Trades. I don't think it's a suitable API for high performance(I.E. low latency) applications.

[cheako]

I started a branch to develop a Vulkan compositor, I figured that code changes would happen in parallel. Thus far I've only made changes to cargo, though I don't know what I'm doing.

cheako@7805efb#diff-80398c5faae3c069e4e6aa2ed11b28c0
cheako@7805efb#diff-7688ece93ac60e73de73f8ceaaf70408

[elinorbgr]

I don't think adding vulkan support would require creating a new crate. Rather, you might want to look into the backend structure of smithay:

• src/backend/egl contains most of the code for initializing and manipulating an EGL context, it uses the traits defined in egl/native.rs to abstract away the concept of "native display" and "native surface" so that the same code can handle EGL contetextes from x11/wayland environments (via winit) or DRM/GBM directly
• src/backend/drm contains the code for handling DRM and GBM, and provides type implement the required trait for being used with the EGL machinnery
• src/backend/winit.rs somilarly provides types built on top of winit implementing the same set of trait

My guess would be that the road forward for Vulkan support would be to introduce a new module on the same structure as the EGL one for handling Vulkan, similarly based on traits to abstract away the notion of "native stuff" (I don't know if Vulkan has a similar notion of display / surface ?), traits that would be implemented by the relevant types in the Drm and Winit modules. And maybe these traits could even be unified with the EGL ones?

[cheako]

I don't know much about EGL or how it works, so I'm not able to understand the existing code.

I've looked at src/backend/drm/mod.rs and I don't see how that would map to the Vulkan API. In particular you don't/can't pass a DRM device to Vulkan.

For the CPP vkwayland there are two impl objects that initialize Vulkan.

Firstly we have the usual GLFW path:

1. We create a *somewhat short-lived VkInstance. GLFW chooses the Instance Extensions it needs. Specifying the version of Vulkan your code is written against, it also gives Vulkan a chance to filter by Engine and Application names and versions. Note that GLFW does not support run-time detection of Wayland/X11, I'm interested in discovering how winit works.
   https://bitbucket.org/cheako/vkwayland/src/ab2ef75c73b142c716c529030836ca9188701ac3/src/glfw/vulkanio.cpp#lines-93

• It's used sporadically for generating function pointers for the DMABUF extension.

2. Create the Window and Surface using GLFW to abstract the X11/Wayland displays.
   https://bitbucket.org/cheako/vkwayland/src/ab2ef75c73b142c716c529030836ca9188701ac3/src/glfw/vulkanio.cpp#lines-111
3. Assume the first Physical Device, but probes the command queue to use. Normally this filtering would be a lot more advanced, but without a ton of systems to test against it's largely a guessing game.
   https://bitbucket.org/cheako/vkwayland/src/ab2ef75c73b142c716c529030836ca9188701ac3/src/glfw/vulkanio.cpp#lines-134
4. Create the Logical Device, this might be much like an OpenGL Context. Notice the Device Extensions.
   https://bitbucket.org/cheako/vkwayland/src/ab2ef75c73b142c716c529030836ca9188701ac3/src/glfw/vulkanio.cpp#lines-192

Then there is the fbcon(Direct to Display) path:

1. VkInstance, Notice line 256(VK_KHR_DISPLAY_EXTENSION_NAME).
   https://bitbucket.org/cheako/vkwayland/src/ab2ef75c73b142c716c529030836ca9188701ac3/src/fbcon/vulkanio.cpp#lines-315
2. There simply is no analog to creating Windows and Surfaces when using Direct to Display.
3. This time we use every Physical Device that has a graphics command queue. I might have a bug here, but it's not important... Like I said this part of Vulkan needs testing across many devices/systems.
   https://bitbucket.org/cheako/vkwayland/src/ab2ef75c73b142c716c529030836ca9188701ac3/src/fbcon/vulkanio.cpp#lines-342
4. Logical Device, wow look at all the Device Extensions.
   https://bitbucket.org/cheako/vkwayland/src/ab2ef75c73b142c716c529030836ca9188701ac3/src/fbcon/vulkanio.cpp#lines-412
5. There is some code run on each frame looking at the Displays and comparing that to the configured Displays. This is used to setup each Monitor for each Physical Device.
   https://bitbucket.org/cheako/vkwayland/src/ab2ef75c73b142c716c529030836ca9188701ac3/src/fbcon/vulkanio.cpp#lines-201

After the Logical Device is created the two code paths merge. That API is defined here:
https://bitbucket.org/cheako/vkwayland/src/ab2ef75c73b142c716c529030836ca9188701ac3/include/render.hpp#lines-89

Lines 102 to 104 are in reference to Devices and Displays and the following "Surface" are Wayland surfaces. At the time I was writing the Monitor hotplug I didn't know Wayland used Surfaces also, but CPP doesn't care they are named the same.

All these functions iterate over m_devices(line 153) and call similar functions where the functions are run against each "OutSurface" where I got a tad better at naming things:
https://bitbucket.org/cheako/vkwayland/src/ab2ef75c73b142c716c529030836ca9188701ac3/src/render/device.hpp#lines-27

[elinorbgr]

I'm sorry, this whole thing is getting way out of my expertise, I'll let @Drakulix take things from now... 😅

[elinorbgr]

But that won't stop me to try and understand anyway.

So first, to give you some more context about winit: as opposed to glfw, winit only handles the windowing itself, and doesn't know anything about opengl or vulkan. So it'll just provide pointers to the x11/wayland display and the x11/wayland surface, and we'd be left to setup the Vulkan WSI using them. The retreival of these pointers from a winit window is done via the WindowExt extension trait from winit: https://docs.rs/winit/0.19.0/winit/os/unix/trait.WindowExt.html

Now, regarding vulkan on tty, currently the GBM/DRM code of smithay allows to control how rendering devices are connected to monitors. With vulkan, all of this is handled directly via the vulkan API, rather than relying on libdrm and libgbm? And does Vulkan have some native mechanisms for importing dmabufs from clients?

[Drakulix]

Alright, I will try to clear up some confusion here.

I looked at vulkano and fear that it goes against the spirit of Vulkan by implementing runtime checks when a big selling point of Vulkan is that it mostly eliminates them.

Vulkano is still miles ahead in terms of performance compared to OpenGL and some user of smithay might prefer a safer way to use vulkan. No matter what solution we settle upon, I want to support vulkano as well. But as long as we do not close off vulkano support, I am open to support additional crates.

I myself am not that familiar with the whole graphics stack though, this is more @Drakulix's domain. Some discussion has already been started in #129 about support for gfx-hal, which I believe is related to Vulkan support.

gfx-hal is actually more a user-side issue, then a backend side. What gfx-hal does is mapping a vulkan-like API to different backends (vulkan, opengl, dx12, metal, etc).

So gfx-hal will be a useful abstraction for users to unify access to all the different backends, but we still need to implement a vulkan-backend.

I took a look at gfx-rs/hal and it's a Vulkan oriented abstraction across a range of graphical APIs. Of note there is no support for Direct to Display drawing, a must for any serious Compositor.

gfx-hal in general is missing support for direct display rendering in any backend. #129 tries to change gfx-hal's OpenGL backend to work on our provided implementation.

I am sure similar adjustments can be made once we have a working vulkan backend to gfx-hal to support that as well.

But yes, in its current state it is not usable for smithay.

It's not clear how legacy GL and Vulkan are made to be compatible. My assumption is that it's a case of gfx-hal being a Jack of All Trades. I don't think it's a suitable API for high performance(I.E. low latency) applications.

If the underlying backend is vulkan the overhead is almost zero, since calls are just mapped. The overhead is also quite small if dx12 or metal is used and absolutely usable as gfx-portability shows.

The GL layer requires at least OpenGL 3.2, I think, which I would not really describe as legacy. (There are still a bunch of OpenGL 2.1 cards on the market for embedded/mobile/low-cost systems.)
There is definitely some overhead involved in mapping those api's, but OpenGL is not the primary target of gfx-hal anyway.

I hope this helps to understand, how gfx-hal will work with smithay, and that it is irrelevant for this issue, so back to implementing a vulkan backend.

I've looked at src/backend/drm/mod.rs and I don't see how that would map to the Vulkan API. In particular you don't/can't pass a DRM device to Vulkan.

There are two ways right now to implement a compositor with Vulkan support.

One is the one you are talking about, namely the VK_KHR_display extension.

With vulkan, all of this is handled directly via the vulkan API, rather than relying on libdrm and libgbm?

And that solution does in fact not rely libdrm or libgbm. An implementation would require a completely new backend in smithay as an alternative to the existing drm-gbm-egl-stack.

And I am not opposed to add and support that as well, but I rather prefer another approach, that is rendering via gbm.

Rendering via gbm is not very straightforward, there is no vulkan extension that allows to create a VkSurface from a gbm-Surface. However it is desirable, a discussion similar to this one was held in the wlroots project: swaywm/wlroots#642.

To cite them:

Why would VK_KHR_display not work for this?

We're definitely are keeping GLES2 support, because it has much more hardware support than Vulkan, so the GBM/EGL stuff is sticking around. Trying to graft a VKDisplayKHR on top of all of DRM would be a complete mess; it would only be viable as a completely new backend, which I wouldn't actually be opposed to someone doing.

However, it just doesn't have all of the features we need. We can't do things like DPMS, gamma control, hotplugging(?), or many of the other nuanced things that DRM allows us to do. That's why I said I think it's for VR headsets; those kinds of features aren't important for those, but are expected from a proper display server.

Or:

On the server side, KhronosGroup/Vulkan-Docs#294 has a reasonably complete sketch of what Vulkan compositor support looks like. You can use VK_KHR_display to drive KMS for you (also handling buffer allocation through swapchains); it is easier to implement, but you lose some control over display timing, and it also means you need to composite all clients, and can't directly display clients on planes.

So basically we cannot do a lot of desirable features, if we choose to rely VK_KHR_display alone, especially not having a good way to do hot-plugging, other then frequently enumerating via vulkan is an absolute deal-breaker. Not mentioning planes, DPMS and gamma control.

That said, if you wanted to implement a backend based on VK_KHR_display, I have no objections, we can have multiple vulkan backends.

The approach I am favoring is also described in the linked wlroots issue:

The other approach is to use GBM as a buffer allocator, wrap those buffers in VkImages, render to them, extract the dma-fence FD from the rendering's signal VkSemaphore, and pass that into KMS with IN_FENCE_FD, whilst continuing to drive KMS like you do now. IN_FENCE_FD does require atomic; not sure if you have that.

So an implementation using this approach would still rely on our DrmDevice/Surface and GbmDevice/Surface code, but would wrap the GbmSurface into a VulkanSurface - I suppose - that would expose a VkImage to render into, which is mapped to the device directly using gbm, which in turn means this is also an approach using direct display rendering, although it does not appear to be at first glance.

However this does require atomic-modesetting support for the timing stuff via VkSemaphore, a feature we are currently lacking and is blocked on the drm-rs rewrite: Smithay/drm-rs#25.

Good news is, that I am working on that issue as well, because it also blocks plane-support and therefor nvidia-eglstream-support.

And does Vulkan have some native mechanisms for importing dmabufs from clients?

Yeah vulkan can import dmabufs.

I have not much experience with vulkan itself, this is mostly information I gathered by reading about it a lot, but I hope this is still helpful.

[cheako]

cheako@798ae3f

I've got to the point with winit.rs where the code will be shared with ?udev.rs? Where should I put the ten or twenty pages of common code? I already have some of it create_swapchain().

From https://gitlab.com/dennis-hamester/dacite/blob/master/examples/triangle/src/main.rs#L207 to 582.

[cheako]

I created a file called vulkan_init.rs.

I can't find where GliumDrawer??init() is called from?

[skyne98]

@cheako, nice to hear that someone is interested in writing the Vulkan support right now! Nice! Also, I wanted to mention, that I have opened an issue about GFX-hal because supporting it, even in the most inefficient way, in the beginning, will allow us to start using GFX-hal libraries (such as "rendy") to build up the compositors. Then, by the time a Vulkan backend for GFX-hal will be supported by Smithay, we will have code that can make full use of it.