Implement VirtIO GPU and input devices

[shengwen-tw]

Summary

This pull request introduces initial support for virtio-gpu and virtio-input devices in semu, conforming to the VirtIO specification 1.3.

VirtIO-GPU

• Based on VirtIO MMIO transport layer (i.e., Linux platform device)
• Implements basic 2D rendering support.
• Provides two rendering backends:
  • Software-based backend (supports 2D rendering only, ready to be integrated into rv32emu).
  • virglrenderer backend (to enable hardware-accelerated 3D rendering in a future patch).
• Compatible with:
  • DirectFB2
  • X11 Window System
• Successfully tested with:
  • kmscube
  • glmark2
  • glxgears

VirtIO-Input

• Implements basic keyboard emulation.
• Implements basic mouse emulation.

Prerequisites

Note: this list is not yet exhaustive

$ sudo apt install libsdl2-dev libsdl2-2.0-0 libsdl2-image-dev libsdl2-image-2.0-0

Build kernel image and test rootfs image

$ make rootfs.cpio # Download prebuilt minimal rootfs for initrd
$ ./scripts/build-image.sh --all --x11 --external-root --extra-packages

Use --x11 for full setup; omit it for a minimal setup.

Run X11

1. Launch semu:

$ make check

2. Run startx

$ startx

3. After X11 shows up, launch the benchmark programs:

$ glmark2
$ glxgears

Run DirectFB2

1. Launch semu:

$ make check

2. Kill X11 (Skip if --x11 is not used)

$ ps | grep 'Xorg'
   <PID> root     /usr/bin/Xorg :0.0 vt01 -s 0 -noreset -allowMouseOpenFail
$ kill <PID>

3. Set up environment variables via run.sh:

$ source ./run.sh

4. Run DirectFB2 test programs:

df_andi        df_fire df_neo     df_window
df_texture     df_video           df_knuckles
df_input       df_neo             df_window
df_matrix      df_dok             df_fire
df_palette     df_cpuload         df_spacedream
df_layers      df_pss             df_input

Run kmscube

1. Launch semu:

$ make check

2. Kill X11 (Skip if --x11 is not used)

$ ps | grep 'Xorg'
   <PID> root     /usr/bin/Xorg :0.0 vt01 -s 0 -noreset -allowMouseOpenFail
$ kill <PID>

3. Run kmscube

$ kmscube

Run modetest

The simplest method to test VirtIO-GPU.

1. Enable BR2_PACKAGE_LIBDRM_INSTALL_TESTS in the Buildroot then rebuild the image.

2. Launch semu:

$ make check

3. Run modetest program

$ modetest -M virtio_gpu -s 34:1024x768

A test image should then show up in the display window.

[jserv]

How about using virglrenderer?
See https://mail.gnu.org/archive/html/qemu-devel/2021-02/msg04235.html

[shengwen-tw]

Yes, I do plan to support VirGL for acceleration.

As far as I know, the offloading of rendering operations to the host GPU is called 3D mode by VirtIO's spec, whereas this pull request only implemented 2D mode due to the overall complexity.

Before implementing 3D rendering mode with VirGL, I think we need to configure the Buildroot to have a minimal desktop environment or set the Mesa benchmark tools without a desktop manager (I'm not sure if this is feasible).

In conclusion, I'll recommend first merging this pull request as a checkpoint for virtio-gpu and then adding support for VirGL in a later pull request.

[shengwen-tw]

Reference: How to run headless unit tests for GUIs on GitHub actions

[jserv]

Rather than relying on Xvfb as a heavy-duty solution for validating Linux DRM/framebuffer, it might be more efficient to use lighter tools like drm-framebuffer. This tool, designed for testing drm devices, takes data from stdin and displays it, offering a simpler and more straightforward approach.

[shengwen-tw]

Rather than relying on Xvfb as a heavy-duty solution for validating Linux DRM/framebuffer, it might be more efficient to use lighter tools like drm-framebuffer. This tool, designed for testing drm devices, takes data from stdin and displays it, offering a simpler and more straightforward approach.

Actually, the intention of using Xvfb here is to make sure the SDL window can be launched in the CI environment by GitHub Action.

[jserv]

Merely executing Xvfb :1 -screen 0 1600x1200x24 is inadequate because, although it confirms the initialization of SDL with libX11, it does not guarantee the accuracy of the window content rendered in Xvfb.

[jserv]

Rebase the latest master branch.

[shengwen-tw]

@jserv :
I propose we begin the code review to merge the current progress in this pull request.
This will also make it easier to offload some tasks to additional contributors, if needed.
I will open a separate pull request to add support for the 3D rendering mode.

[jserv]

It is confusing to have VirGL here. Add comments.

[jserv]

Don't build X11 specific packages.

[shengwen-tw]

Introduced --x11 flag in scripts/build-image.sh for optional building.