Vulkan backend in renderdoc with wayland

[nical]

Description

The vulkan backend fails to initialize in renderdoc on wayland.

I am not sure where the responsibility lie between wgpu, winit and renderdoc. I'll document what I know here for future reference because I periodically forget and re-investigate the issue:

• The error happens in create_surface_from_wayland, VK_KHR_wayland_surface is missing.
• I see the following extensions present:
  • "VK_KHR_surface"
  • "VK_KHR_xlib_surface"
  • "VK_KHR_xcb_surface"
  • "VK_EXT_debug_utils"
  • "VK_EXT_swapchain_colorspace"
  • "VK_KHR_get_physical_device_properties2"
• There is a closed issue on renderdoc's repsoitory: vkCreateInstance fails with VK_ERROR_EXTENSION_NOT_PRESENT due to lack of VK_KHR_wayland_surface extension support on Ubuntu 22.04 baldurk/renderdoc#2711
  • The resolution is that wayland is currently not supported in renderdoc and it is up to the application to check the existence of the extensions.

I wonder whether when we get to create_surface_from_wayland it is still possible initialize the surface via another method or whether the presence of the extension should be checked earlier. Ideally wgpu applications would work in renderdoc with wayland without having to know about forcing the X11 backend.

Workaround

As far as local testing goes, using winit's x11 backend seems to be a sufficient workaround (set the WINIT_UNIX_BACKEND environment variable to "x11").

Repro steps

Run any of this repository's examples in renderdoc with the WGPU_BACKEND environment variable set to "vulkan" under wayland.

Platform

Linux with wayland

[cwfitzgerald]

I'm not really sure what a good option we have. By the time we have the wayland pointer, the window has already been made by winit.

We might be able to expose "Linux window system support" on an adapter, but winit has no way to decide programmatically but via an env var. This also feels very specific to this exact situation.

Maybe the best option is an error message that suggests the workaround (if renderdoc is found injected).

[nical]

Maybe the best option is an error message that suggests the workaround (if renderdoc is found injected).

That sounds very reasonable. I don´t know if we can easily know that renderdoc was injected but we can at least see that VK_KHR_xlib_surface is present and suggest starting the app in "x11 mode" and link back to a wiki page or this issue with specific instructions about winit (which is presubmably what most people use to create the window).

[SarthakSingh31]

For anyone running into this issue, the workaround is to force winit to run in x11 mode by setting the environment variable WAYLAND_DISPLAY=""