Skip to content

wlserver: check wlr_xwayland_server_create for failure - #2337

Open
adamsjo95 wants to merge 1 commit into
ValveSoftware:masterfrom
adamsjo95:wlserver-check-xwayland-server-create
Open

wlserver: check wlr_xwayland_server_create for failure#2337
adamsjo95 wants to merge 1 commit into
ValveSoftware:masterfrom
adamsjo95:wlserver-check-xwayland-server-create

Conversation

@adamsjo95

@adamsjo95 adamsjo95 commented Aug 20, 2026

Copy link
Copy Markdown

wlr_xwayland_server_create() returns NULL on failure, but
gamescope_xwayland_server_t's constructor uses the return value
unchecked, so the next line dereferences NULL and gamescope segfaults
with no indication of the cause.

wlroots' own caller checks it (xwayland/xwayland.c), and the function
has four return NULL paths — missing Xwayland binary, allocation
failure, and two error labels.

Reproduction

Any condition that makes Xwayland server creation fail will trigger it.
I hit it with a /tmp/.X11-unix that wlroots rejects on ownership
grounds, which exhausts all 33 display slots:

[Error] wlserver: [xwayland/sockets.c:83] /tmp/.X11-unix not owned by root or us
... (x33)
[Error] wlserver: [xwayland/sockets.c:200] No display available in the first 33
Segmentation fault (core dumped)

Backtrace before the patch:

#0  wl_signal_add (signal=0x90, listener=0x...) at wayland-server-core.h:490
#1  gamescope_xwayland_server_t::gamescope_xwayland_server_t (...) at src/wlserver.cpp:1816
#2  std::make_unique<gamescope_xwayland_server_t, wl_display*&, int&> ()
#3  wlserver_init () at src/wlserver.cpp:2204
#4  main (...) at src/main.cpp:1062

signal=0x90 is the offset of events.ready on a NULL base.

After

[Error] wlserver: [xwayland/sockets.c:200] No display available in the first 33
[Error] wlserver: Failed to create Xwayland server
abort (core dumped)

Exit status goes from 139 (SIGSEGV) to 134 (SIGABRT), with the cause
named.

Notes

I used abort() to match the wlr_output_commit_state() failure
handling twelve lines below in the same constructor. Happy to change it
to a cleaner exit if preferred — this failure is often a user-fixable
environment problem rather than an internal invariant violation, so a
non-abort path may read better.

Tested on gamescope fcc1341, wlroots 0.20.2. Builds clean with
--werror. meson test --suite gamescope passes 2/2 before and after.

wlr_xwayland_server_create() returns NULL on failure, but its return
value was used unchecked. When Xwayland server creation fails, the
following wl_signal_add() dereferences the NULL pointer and gamescope
segfaults with no indication of what went wrong.

wlroots' own caller in xwayland/xwayland.c checks the return value.

Log the failure and abort instead, matching the handling of the
wlr_output_commit_state() failure below.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant