Neptune D3D11 works on a plain Linux KVM host (experience report + spice-display.c GBM-path fix)

[neuromaniacMD]

Summary

I got the Neptune DirectX 11 stack running on a plain Linux KVM host (no UTM app involved) — Windows 11 guest, neptune_umd.dll forwarding D3D11 to DXVK-native on RADV. As far as I can tell this is the first Linux-host deployment, so I'm reporting what worked, a small spice-display.c fix needed on the Linux/GBM path (diff below), and one gap (RDP sessions).

Filing here since the pieces span utmapp/qemu, utmapp/virglrenderer and the guest tools — happy to move/split this wherever you prefer.

Environment

• Host: Fedora 44, kernel 7.1.5, AMD Strix Halo (Radeon 8060S, RADV/mesa 26.1.5)
• QEMU: utmapp/qemu branch utm-edition (10.0.12 base), built with --enable-spice --enable-virglrenderer --enable-opengl, libvirt 12
• virglrenderer: utmapp/virglrenderer branch neptune, -Dneptune=true
• D3D11 host backend: osy/dxvk master built native (-Dnative_headless=true), selected via NPT_D3D11_LIBRARY_PATH/NPT_DXGI_LIBRARY_PATH + DXVK_WSI_DRIVER=Headless
• Guest: Windows 11 24H2, driver from utm-guest-tools-3d-latest.iso (viogpu3d 100.6.101.58000)
• Device: virtio-vga-gl,blob=true,neptune=true,hostmem=4G, SPICE gl=on local, memfd shared memory backing

What works

• Desktop/DWM composition on the Neptune UMD is stable (login, WinUI apps, Device Manager, window management).
• dxdiag: Direct3D DDI 11.2, feature levels 11_1→9_1, WDDM 1.3, "Direct3D Acceleration: Enabled", "No problems found".
• Host GPU shows guest-driven load; the render server logs Found device: AMD Radeon 8060S Graphics (RADV STRIX_HALO).

Fix needed on Linux: spice-display.c GBM path (three small issues, one diff)

The utm-edition tree currently doesn't compile/run the SPICE-GL GBM path (macOS builds never exercise it):

1. qemu_spice_gl_update: width/height used but no longer declared in the CONFIG_GBM block (build error).
2. Same function: the egl_texture_blend call passes a stray 9th argument (false) that doesn't match this tree's 8-arg prototype (build error).
3. qemu_spice_display_early_init: the CONFIG_GBM branch calls egl_init() but never sets spice_opengl/spice_gl_ctx, so GL ops are never registered — the device fails with "The console requires a GL context", and if forced further, egl_get_fd_for_texture runs with no context current and aborts in epoxy (No provider of eglCreateImageKHR).

diff --git a/ui/spice-display.c b/ui/spice-display.c
--- a/ui/spice-display.c
+++ b/ui/spice-display.c
@@ -1548,6 +1548,7 @@ static void qemu_spice_gl_update(DisplayChangeListener *dcl,
     int fd;
     bool render_cursor = false;
     uint32_t texture;
+    uint32_t width, height;
 #endif
     bool y_0_top = false; /* FIXME */
     uint64_t cookie;
@@ -1620,7 +1621,7 @@ static void qemu_spice_gl_update(DisplayChangeListener *dcl,
         egl_texture_blit(ssd->gls, &ssd->blit_fb, &ssd->guest_fb,
                          !y_0_top);
         egl_texture_blend(ssd->gls, &ssd->blit_fb, &ssd->cursor_fb,
-                          !y_0_top, false, ptr_x, ptr_y, 1.0, 1.0);
+                          !y_0_top, ptr_x, ptr_y, 1.0, 1.0);
         glFlush();
     }
 #elif defined(CONFIG_IOSURFACE)
@@ -1763,6 +1764,8 @@ void qemu_spice_display_early_init(void)
         } else {
 #if defined(CONFIG_GBM)
             egl_init(qemu_opt_get(opts, "rendernode"), DISPLAY_GL_MODE_ON, &error_fatal);
+            spice_gl_ctx = qemu_egl_rn_ctx;
+            spice_opengl = DISPLAY_GL_MODE_ON;
 #elif defined(CONFIG_EGL)
             if (qemu_egl_init_dpy_cocoa(DISPLAY_GL_MODE_ES)) {
                 error_report("SPICE GL failed to initialize ANGLE display");

With this, the whole stack runs under libvirt. (One deployment note for others: libvirt's default seccomp sandbox kills the render server — first spawn=deny blocks the fork, and even with spawn allowed the child dies with SIGSYS in setpgid() from proxy_server_create. seccomp_sandbox = 0 in qemu.conf is required. A render-server-mode=thread-style option for the neptune proxy would remove that requirement.)

Also worth documenting for Linux users: the fork's dxvk_shared_resource.h extension means the dxvk libs must be the osy/dxvk build — if distro dxvk-native libs are also installed, libdxvk_d3d11.so's libdxvk_dxgi.so.0 dependency can silently resolve to the distro copy (no Headless WSI → D3D11CreateDevice: Failed to create a DXGI factory in the render server, and the guest DWM loops with 0x889800b0 while every RESOURCE_CREATE_BLOB fails ERR_UNSPEC). An rpath on the dxvk libs fixes it.

Remaining gap: RDP sessions

Console/SPICE is stable, but RDP into the guest fails: the RDP session's per-session dwm crash-cycles on the Neptune adapter and the Microsoft Remote Display Adapter (IddCx UMDF driver) crashes (Event 10111: offline due to a user-mode driver crash). Client sees a white screen (gfx pipeline) or black-with-cursor (bitmap path — screen capture produces no frames). Presumably the indirect-display/cross-adapter shared-surface path isn't implemented yet. Not a complaint — just flagging it as the one thing between this and headless-server use on Linux. Happy to test builds or provide more traces.