Steam and games crashing Xwayland

[Virkkunen]

Your system information

• Steam client version (build number or date): 1719618357
• Distribution (e.g. Ubuntu): endeavourOS
• Opted into Steam client beta?: Yes
• Have you checked for system updates?: Yes
• Steam Logs: steam-logs.tar.gz
• GPU: Nvidia
• Driver version: nvidia-dkms 555.58-2

Please describe your issue in as much detail as possible:

Describe what you expected should happen and what did happen. Please link any large code pastes as a Github Gist

After a while of playing any game, Steam and the game will crash Xwayland, either by immediately closing both (and Kwin sends me a notification that Xwayland has crashed) or by crashing my entire desktop, requiring a reboot. A few times Steam has crashed while closed to the tray. If it's the former type of crash, some related processes might still be running and if I try to drag a window, my whole desktop will freeze and there will be high CPU and GPU usage on kwin and Xwayland. I can still connect to my PC with ssh, but I have to reboot to be able to use my desktop again.

I've tried:

• both runtime and native Steam versions
• running with gamescope, mangohud, gamemode and a combination of each
• adaptive sync on automatic and off on Plasma's display settings
• removing a faulty plasmoid that could be causing issues
• Using nvidia-open-dkms driver
• setting nvidia.NVreg_EnableGpuFirmware=0 on my kernel for the proprietary drivers

Some useful logs:

• dmesg after the hard crash dragging a window: dmesg.txt
• journalctl --user, each ending on a dump after a crash: logs.txt logs_02.txt
• coredumps after each crash: coredumps.tar.gz

Steps for reproducing this issue:

1. Open steam and a game
2. Wait

[Virkkunen]

PROTON_LOG=1 on a game that has crashed: steam-1422450.log

[kisak-valve]

Hello @Virkkunen, in a preliminary skim of your logs, this looks more like an issue between QT6 and the NVIDIA driver.

From dmesg, I take it these are the relevant lines:

[ 2134.872424] [drm:nv_drm_gem_alloc_nvkms_memory_ioctl [nvidia_drm]] *ERROR* [nvidia-drm] [GPU ID 0x00000800] Failed to allocate NVKMS memory for GEM object
[ 2134.872474] [drm:nv_drm_gem_alloc_nvkms_memory_ioctl [nvidia_drm]] *ERROR* [nvidia-drm] [GPU ID 0x00000800] Failed to allocate NVKMS memory for GEM object
[ 2167.616215] gamescope-wl[14658]: segfault at 0 ip 0000641bc44bf2c7 sp 00007ffcfedc5b40 error 4 in gamescope[641bc4466000+16b000] likely on CPU 20 (core 10, socket 0)

I read this as the NVIDIA driver having an allocation issue, then a bit later gamescope falls over.

I suspect this is related to Added support for the linux-drm-syncobj-v1 protocol for Wayland explicit sync in EGL. in the NVIDIA 555.58 driver release. Can you easily test an older version of the NVIDIA driver?

Regardless, this issue should be reported to your desktop environment and/or the NVIDIA dev(s).

[Virkkunen]

I'll try running with driver 550, which was the previous driver I had. Because of the flickering issues with the lack of explicit sync, I never had Steam open for long enough to notice these type of crashes.

Also, shouldn't part of this issue be related to #8853?

[Virkkunen]

I've also just tried setting SDL_VIDEODRIVER=x11 as outlined by this issue, but I've had two crashes so far, with the same errors on the logs.

[Virkkunen]

I've reverted nvidia-dkms, nvidia-utils, opencl-nvidia, lib32-nvidia-utils and lib32-opencl-nvidia to version 550.90 and so far the issues aren't happening anymore. The crashes aren't happening, so they surely are due to the explicit sync protocol, and the window dragging freezes seemed to be an issue with plasma's cache, so both issues will be reported on their respective trackers.

[Virkkunen]

From my part, I think this issue can be closed.

For clarification, I've been doing some extensive testing and I can confirm that the issue is related to VRAM. Drivers 555 fill VRAM ridiculously quick and will crash Xwayland as soon as it hits 9.9Gib/10Gib on my GPU. Drivers 550 will take longer to fill, and it won't crash even if it hits 9.996Gib/10Gib, although it'll have a performance hit on my games.

This issue has been reported on NVIDIA's forums:
https://forums.developer.nvidia.com/t/555-release-feedback-discussion/293652/201
https://forums.developer.nvidia.com/t/555-crashes-xwayland-when-playing-games/298306/2