Skip to content

vhost-device-console: add Unix domain socket backend support #821

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 6 commits into
base: main
Choose a base branch
from
Open

vhost-device-console: add Unix domain socket backend support #821

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
388d168
console: Improve error handling with descriptive messages
Mar 6, 2025
5a71380
console: Refactor network components with consistent name
Mar 6, 2025
085dd79
console: Refactor backend handling with pattern matching
Mar 6, 2025
45a778a
console: add Unix domain socket backend support
Mar 6, 2025
477f091
test(console): add UDS configuration validation tests
Mar 6, 2025
7f81834
console: check parameter mutual exclusivity
Mar 24, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions vhost-device-console/CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@
### Added

- [#811](https://github.com/rust-vmm/vhost-device/pull/811) Being able to specify max queue size
- [#821](https://github.com/rust-vmm/vhost-device/pull/821) Add Unix domain socket backend support

### Changed

Expand Down
54 changes: 44 additions & 10 deletions vhost-device-console/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,14 +6,17 @@ This program is a vhost-user backend that emulates a VirtIO Console device.
The device's binary takes as parameters a socket path, a socket number which
is the number of connections, commonly used across all vhost-devices to
communicate with the vhost-user frontend devices, and the backend type
"nested" or "network".
"nested" or "network" or "uds".

The "nested" backend allows input/output to the guest console through the
current terminal.

The "network" backend creates a local TCP port (specified on vhost-device-console
arguments) and allows input/output to the guest console via that socket.

The "uds" backend creates a unix domain socket (specified on vhost-device-console
arguments) and allows input/output to the guest console via that socket.

This program is tested with QEMU's `vhost-user-device-pci` device.
Examples' section below.

Expand All @@ -39,14 +42,23 @@ vhost-device-console --socket-path=<SOCKET_PATH>

The localhost's port to be used for each guest, this part will be increased with
0,1,2..socket_count-1.
This option is only valid when backend type is "network".

-- option:: -b, --backend=nested|network
-- option:: -b, --backend=nested|network|uds

The backend type vhost-device-console to be used. The current implementation
supports two types of backends: "nested", "network" (described above).
supports 3 types of backends: "nested", "network", "uds".
Note: The nested backend is selected by default and can be used only when
socket_count equals 1.

.. option:: --uds-path=uds-file-path

The unix domain socket to be used for each guest, this path will be suffixed with
0,1,2..socket_count-1. e.g.: `--uds-path=/tmp/vm.sock --socket-count=2`
leads to two connectable vhost-user sockets:
/tmp/vm.sock0, /tmp/vm.sock1
This option is only valid when backend type is "uds".

.. option:: -q, --max-queue-size=SIZE

The maximum size of virtqueues. It is optional, and the default value is
Expand All @@ -62,10 +74,10 @@ VIRTIO_CONSOLE_F_SIZE features.
## Features

The current device gives access to multiple QEMU guest by providing a login prompt
either by connecting to a localhost server port (network backend) or by creating an
nested command prompt in the current terminal (nested backend). This prompt appears
as soon as the guest is fully booted and gives the ability to user run command as a
in regular terminal.
either by connecting to a localhost server port (network backend) or a unix socket
file (uds backend) or by creating an nested command prompt in the current terminal
(nested backend). This prompt appears as soon as the guest is fully booted and
gives the ability to user run command as a in regular terminal.

## Examples

Expand All @@ -83,10 +95,15 @@ For testing the device the required dependencies are:
The daemon should be started first:
```shell
host# vhost-device-console --socket-path=/tmp/console.sock --socket-count=1 \
--tcp-port=12345 --backend=network
--tcp-port=12345 --backend=network # for network backend
```
or
```shell
host# vhost-device-console --socket-path=/tmp/console.sock --socket-count=1 \
--uds-path=/tmp/vm.sock --backend=uds # for uds backend
```
>Note: In case the backend is "nested" there is no need to provide
"--socket-count" and "--tcp-port" parameters.
"--socket-count", "--tcp-port" and "--uds-path" parameters.

The QEMU invocation needs to create a chardev socket the device can
use to communicate as well as share the guests memory over a memfd.
Expand Down Expand Up @@ -119,9 +136,26 @@ host# qemu-system
...
```

#### Test the device with UML
Start the daemon as above section, then run the following cmdline for
Kernel Mode Linux [`virtio console`](https://github.com/torvalds/linux/blob/848e076317446f9c663771ddec142d7c2eb4cb43/include/uapi/linux/virtio_ids.h#L34):
```text
host# linux root=/dev/ubda1 rw ubd0=$YOUR-PATH/kata-ubuntu-latest.image \
<normal UML options> \
virtio_uml.device=/tmp/console.sock0:3 console=tty0 console=hvc0 \
init=/bin/systemd \
systemd.unit=kata-containers.target agent.debug_console
Copy link
Contributor

@justxuewei justxuewei Apr 1, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think testing with UML does not need to depend on Kata Containers:

  • Remove staff regarding kata-containers: systemd.unit, debug_console.
  • Use /bin/bash instead.
$ sudo linux ubd0=kata-ubuntu-latest.image \
    root=/dev/ubda1 \
    init=/bin/bash \
    console=hvc0 \
    virtio_uml.device=/tmp/console.sock0:3

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BTW, Kata's rootfs image can still be used here.

```
Test with [kata-ubuntu-latest.image](https://github.com/kata-containers/kata-containers/releases/),
you can also use systemd to setup pty manually without kata-agent help.

Eventually, the user can connect to the console by running:
```test
host# stty -icanon -echo && nc localhost 12345 && stty echo
host# stty -icanon -echo && nc localhost 12345 && stty echo # for network backend
```
or
```test
host# stty -icanon -echo && nc -U /tmp/vm.sock0 && stty echo # for uds backend
```

>Note: `stty -icanon -echo` is used to force the tty layer to disable buffering and send / receive each character individually. After closing the connection please run `stty echo` so character are printed back on the local terminal console.
Expand Down
138 changes: 111 additions & 27 deletions vhost-device-console/src/backend.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,11 +42,16 @@ pub enum Error {
ThreadPanic(String, Box<dyn Any + Send>),
#[error("Error using multiple sockets with Nested backend")]
WrongBackendSocket,
#[error("Invalid cmdline option")]
InvalidCmdlineOption,
#[error("Invalid uds file")]
InvalidUdsFile,
}

#[derive(PartialEq, Eq, Debug)]
pub struct VuConsoleConfig {
pub socket_path: PathBuf,
pub uds_path: PathBuf,
pub backend: BackendType,
pub tcp_port: String,
pub socket_count: u32,
Expand All @@ -73,23 +78,48 @@ impl VuConsoleConfig {
(0..self.socket_count).map(make_socket_path).collect()
}

pub fn generate_tcp_addrs(&self) -> Vec<String> {
let tcp_port_base = self.tcp_port.clone();

let make_tcp_port = |i: u32| -> String {
let port_num: u32 = tcp_port_base.clone().parse().unwrap();
"127.0.0.1:".to_owned() + &(port_num + i).to_string()
};

(0..self.socket_count).map(make_tcp_port).collect()
pub fn generate_vm_socks(&self) -> Vec<String> {
match self.backend {
// if type is Nested, result will be dropped.
BackendType::Nested => {
vec![String::new()]
}

BackendType::Network => {
let port_base: u32 = self.tcp_port.parse().unwrap();
let make_tcp_port =
|i: u32| -> String { "127.0.0.1:".to_owned() + &(port_base + i).to_string() };
(0..self.socket_count).map(make_tcp_port).collect()
}

BackendType::Uds => {
let uds_filename = self.uds_path.file_name().expect("uds has no filename.");
let uds_parent = self
.uds_path
.parent()
.expect("uds has no parent directory.");
Copy link
Contributor

@justxuewei justxuewei Apr 1, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I prefer to create all directories, instead of doing expect() here.


let make_uds_path = |i: u32| -> String {
let mut filename = uds_filename.to_os_string();
filename.push(std::ffi::OsStr::new(&i.to_string()));
uds_parent
.join(&filename)
.to_str()
.expect("Path contains invalid UTF-8 characters")
.to_string()
};

(0..self.socket_count).map(make_uds_path).collect()
}
}
}
}

/// This is the public API through which an external program starts the
/// vhost-device-console backend server.
pub fn start_backend_server(
socket: PathBuf,
tcp_addr: String,
vm_sock: String,
backend: BackendType,
max_queue_size: usize,
) -> Result<()> {
Expand All @@ -104,7 +134,7 @@ pub fn start_backend_server(
vu_console_backend
.write()
.unwrap()
.assign_input_method(tcp_addr.clone())
.assign_input_method(vm_sock.clone())
.map_err(Error::CouldNotInitBackend)?;

let mut daemon = VhostUserDaemon::new(
Expand Down Expand Up @@ -132,26 +162,26 @@ pub fn start_backend_server(
pub fn start_backend(config: VuConsoleConfig) -> Result<()> {
let mut handles = HashMap::new();
let (senders, receiver) = std::sync::mpsc::channel();
let tcp_addrs = config.generate_tcp_addrs();
let vm_socks = config.generate_vm_socks();
let backend = config.backend;
let max_queue_size = config.max_queue_size;

for (thread_id, (socket, tcp_addr)) in config
for (thread_id, (socket, vm_sock)) in config
.generate_socket_paths()
.into_iter()
.zip(tcp_addrs.iter())
.zip(vm_socks.iter())
.enumerate()
{
let tcp_addr = tcp_addr.clone();
let vm_sock = vm_sock.clone();
info!("thread_id: {}, socket: {:?}", thread_id, socket);

let name = format!("vhu-console-{}", tcp_addr);
let name = format!("vhu-console-{}", vm_sock);
let sender = senders.clone();
let handle = Builder::new()
.name(name.clone())
.spawn(move || {
let result = std::panic::catch_unwind(move || {
start_backend_server(socket, tcp_addr.to_string(), backend, max_queue_size)
start_backend_server(socket, vm_sock.to_string(), backend, max_queue_size)
});

// Notify the main thread that we are done.
Expand Down Expand Up @@ -187,8 +217,9 @@ mod tests {
fn test_console_valid_configuration_nested() {
let args = ConsoleArgs {
socket_path: String::from("/tmp/vhost.sock").into(),
uds_path: None,
backend: BackendType::Nested,
tcp_port: String::from("12345"),
tcp_port: None,
socket_count: 1,
max_queue_size: DEFAULT_QUEUE_SIZE,
};
Expand All @@ -200,8 +231,9 @@ mod tests {
fn test_console_invalid_configuration_nested_1() {
let args = ConsoleArgs {
socket_path: String::from("/tmp/vhost.sock").into(),
uds_path: None,
backend: BackendType::Nested,
tcp_port: String::from("12345"),
tcp_port: None,
socket_count: 0,
max_queue_size: DEFAULT_QUEUE_SIZE,
};
Expand All @@ -216,8 +248,9 @@ mod tests {
fn test_console_invalid_configuration_nested_2() {
let args = ConsoleArgs {
socket_path: String::from("/tmp/vhost.sock").into(),
uds_path: None,
backend: BackendType::Nested,
tcp_port: String::from("12345"),
tcp_port: None,
socket_count: 2,
max_queue_size: DEFAULT_QUEUE_SIZE,
};
Expand All @@ -232,8 +265,9 @@ mod tests {
fn test_console_valid_configuration_network_1() {
let args = ConsoleArgs {
socket_path: String::from("/tmp/vhost.sock").into(),
uds_path: None,
backend: BackendType::Network,
tcp_port: String::from("12345"),
tcp_port: Some(String::from("12345")),
socket_count: 1,
max_queue_size: DEFAULT_QUEUE_SIZE,
};
Expand All @@ -245,8 +279,9 @@ mod tests {
fn test_console_valid_configuration_network_2() {
let args = ConsoleArgs {
socket_path: String::from("/tmp/vhost.sock").into(),
uds_path: None,
backend: BackendType::Network,
tcp_port: String::from("12345"),
tcp_port: Some(String::from("12345")),
socket_count: 2,
max_queue_size: DEFAULT_QUEUE_SIZE,
};
Expand All @@ -257,16 +292,16 @@ mod tests {
fn test_backend_start_and_stop(args: ConsoleArgs) -> Result<()> {
let config = VuConsoleConfig::try_from(args).expect("Wrong config");

let tcp_addrs = config.generate_tcp_addrs();
let vm_socks = config.generate_vm_socks();
let backend = config.backend;
let max_queue_size = config.max_queue_size;

for (socket, tcp_addr) in config
for (socket, vm_sock) in config
.generate_socket_paths()
.into_iter()
.zip(tcp_addrs.iter())
.zip(vm_socks.iter())
{
start_backend_server(socket, tcp_addr.to_string(), backend, max_queue_size)?;
start_backend_server(socket, vm_sock.to_string(), backend, max_queue_size)?;
}
Ok(())
}
Expand All @@ -275,8 +310,9 @@ mod tests {
fn test_start_backend_server_success() {
let args = ConsoleArgs {
socket_path: String::from("/not_a_dir/vhost.sock").into(),
uds_path: None,
backend: BackendType::Network,
tcp_port: String::from("12345"),
tcp_port: Some(String::from("12345")),
socket_count: 1,
max_queue_size: DEFAULT_QUEUE_SIZE,
};
Expand All @@ -288,6 +324,7 @@ mod tests {
fn test_start_backend_success() {
let config = VuConsoleConfig {
socket_path: String::from("/not_a_dir/vhost.sock").into(),
uds_path: PathBuf::new(),
backend: BackendType::Network,
tcp_port: String::from("12346"),
socket_count: 1,
Expand All @@ -296,4 +333,51 @@ mod tests {

assert!(start_backend(config).is_err());
}

#[test]
fn test_console_invalid_uds_path() {
let args = ConsoleArgs {
socket_path: PathBuf::from("/tmp/vhost.sock"),
uds_path: Some("/non_existing_dir/test.sock".to_string().into()),
backend: BackendType::Uds,
tcp_port: Some(String::new()),
socket_count: 1,
max_queue_size: 128,
};

assert_matches!(VuConsoleConfig::try_from(args), Err(Error::InvalidUdsFile));
}

#[test]
fn test_generate_vm_sock_addrs_uds() {
let config = VuConsoleConfig {
socket_path: PathBuf::new(),
uds_path: "/tmp/vm.sock".to_string().into(),
backend: BackendType::Uds,
tcp_port: String::new(),
socket_count: 3,
max_queue_size: 128,
};

let addrs = config.generate_vm_socks();
assert_eq!(
addrs,
vec!["/tmp/vm.sock0", "/tmp/vm.sock1", "/tmp/vm.sock2"]
);
}

#[test]
fn test_start_uds_backend_with_invalid_path() {
let config = VuConsoleConfig {
socket_path: PathBuf::from("/tmp/vhost.sock"),
uds_path: "/invalid/path/uds.sock".to_string().into(),
backend: BackendType::Uds,
tcp_port: String::new(),
socket_count: 1,
max_queue_size: 128,
};

let result = start_backend(config);
assert!(result.is_err());
}
}
1 change: 1 addition & 0 deletions vhost-device-console/src/console.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ pub enum BackendType {
#[default]
Nested,
Network,
Uds,
}

#[derive(Debug)]
Expand Down
Loading