Skip to content

feat: add embedded SSH server - #1042

Merged
DorianZheng merged 5 commits into
mainfrom
agent/embed-ssh-server
Jul 28, 2026
Merged

DorianZheng merged 5 commits into
mainfrom
agent/embed-ssh-server

Conversation

@DorianZheng

@DorianZheng DorianZheng commented Jul 26, 2026 •

Copy link
Copy Markdown
Member

Summary

  • embed a russh SSH endpoint in boxlite-guest with CA-signed certificate authentication
  • add live configure/status/disable control outside Guest.Init, plus host APIs and a stdio ProxyCommand transport
  • support shell/exec, PTY and signals, SFTP, TCP and Unix-socket forwarding, and bounded lifecycle cleanup
  • document setup, credential rotation, security boundaries, compatibility, and troubleshooting

Why

SSH access was coupled to gateway and image assumptions and could not be safely reconfigured at runtime. The guest now owns SSH protocol termination while the host owns mutable policy and credential reconciliation.

User impact

Standard OpenSSH clients can connect without an sshd package in the image. CA credentials can rotate without restarting the guest: existing authenticated sessions drain normally, while terminal shutdown closes all sessions. Password authentication, raw public-key authentication, agent forwarding, and unsafe forwarding modes remain disabled.

Validation

  • make test:unit:guest — 210 unit tests plus 7 SSH integration tests passed on Linux
  • BOXLITE_DEPS_STUB=1 make test:unit:rust FILTER=ssh — 6 tests passed
  • BOXLITE_DEPS_STUB=1 make test:integration:cli FILTER=network_tunnel_accepts_stdio_proxy_mode SETUP_DONE=1
  • make clippy
  • make guest
  • make fmt:check:rust
  • git diff --check

A full make fmt:check was also attempted, but the apps dependency bootstrap failed while building native sqlite3 before formatting ran.

Summary by CodeRabbit

  • New Features
    • Added an embedded certificate-authenticated SSH service with interactive shell/exec, SFTP, and TCP/streamlocal forwarding (including reverse streamlocal).
    • Extended TTY handling with RFC 4254 terminal modes and process-group-aware termination.
    • Added guest-side rootfs sealing (read-only) and root login profile resolution.
  • Bug Fixes
    • Hardened security by enforcing guest root filesystem sealing while preserving expected writable paths.
    • Restricted guest RPC processing to host-origin vsock peers.
  • Tests
    • Added/expanded integration coverage for TTY resize, sealed-root security enforcement, and execution/session behavior.
  • Chores
    • Improved lint workflow for macOS Rust cross-linting to Linux musl.

@coderabbitai

coderabbitai Bot commented Jul 26, 2026 •

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: d3602e2e-4827-41cb-b6f7-618d18f4d949

📥 Commits

Reviewing files that changed from the base of the PR and between a5c79de and 1d0059b.

📒 Files selected for processing (3)
  • src/guest/src/service/guest.rs
  • src/guest/src/service/ssh/mod.rs
  • src/guest/src/service/ssh/reverse_streamlocal.rs
🚧 Files skipped from review as they are similar to previous changes (2)
  • src/guest/src/service/ssh/reverse_streamlocal.rs
  • src/guest/src/service/ssh/mod.rs

📝 Walkthrough

Walkthrough

Changes

The PR adds a Linux guest SSH service with certificate authorization, session and forwarding support, SFTP handling, typed workload execution, PTY mode propagation, process-group signaling, execution cleanup, guest-root sealing, host-key provisioning, and related integration tests.

Embedded SSH execution

Layer / File(s) Summary
Protocol and PTY execution contracts
src/shared/proto/..., src/guest/src/service/exec/*, src/boxlite/src/portal/interfaces/exec.rs
Adds SSH RPC messages, PTY terminal modes, process-group kill requests, typed execution errors, and termios application.
Execution lifecycle and container wiring
src/guest/src/service/exec/*, src/guest/src/container/zygote.rs
Centralizes execution operations, propagates typed SSH workloads, classifies exits, tracks forwarders, and releases ephemeral executions.
SSH listener, authentication, and server control
src/guest/src/service/ssh/{mod,auth,control,server,host_key,limits}.rs, src/guest/src/service/server.rs
Adds listener lifecycle management, host-key persistence, certificate authorization, SSH channel handling, and host-vsock filtering.
Typed SSH workloads and login profiles
src/guest/src/service/ssh/{workload,session,bridge}.rs, src/guest/src/container/{command,lifecycle,user_profile}.rs
Routes trusted SSH shell, exec, SFTP, and streamlocal workloads through container execution with validated root profiles and controlled environments.
TCP, streamlocal, and SFTP forwarding
src/guest/src/service/ssh/{forward,streamlocal,reverse_streamlocal,sftp}.rs
Implements bounded direct/reverse forwarding, tokenized streamlocal relays, and SFTP v3 operations and extensions.
Guest startup hardening and validation
src/guest/src/{main,mounts,reaper}.rs, src/boxlite/tests/*, make/test.mk, .github/workflows/lint.yml
Provisions SSH keys, seals the guest root read-only, adds PTY and filesystem enforcement tests, expands the guest nextest target, and configures macOS musl cross-compilation for linting.

Estimated code review effort: 5 (Critical) | ~120 minutes

Possibly related PRs

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 73.55% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title concisely and accurately captures the main change: adding an embedded SSH server.
Description check ✅ Passed It covers summary, why, user impact, and validation; adding an explicit Changes section would better match the template.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch agent/embed-ssh-server

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@cla-assistant

cla-assistant Bot commented Jul 26, 2026

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.


tester seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
You have signed the CLA already but the status is still pending? Let us recheck it.

boxlite-guest terminates SSH itself with russh, so a standard OpenSSH client
reaches a box without an sshd package in the image. Scope is the guest only:
the host APIs, gateway, CLI surface and docs are left for follow-ups.

Authentication accepts one credential — a CA-signed OpenSSH user certificate
naming this box. `none`, passwords and raw public keys are rejected outright,
and the listener advertises publickey alone.

Sessions run inside the container, not beside it. A reserved argv selects a
typed workload that the zygote's executor validates and runs after
libcontainer has applied the tenant's namespaces and credentials, so shell,
exec, SFTP and socket forwarding all get the container's rootfs, uid and
capability set with no helper binary in the image. Shell sessions exec the
account's login shell with OpenSSH's argv[0] convention; PTY requests carry
RFC 4254 terminal modes through to the pty.

Those direct workloads enter the container by fork without execve, so their
/proc/self/exe still names this binary. The guest root is remounted read-only
once its tmpfs mounts are up, which closes that reopen without the memfd copy
and re-exec a sealed self-image would cost.

The server drives executions through the guest's own execution core in native
types rather than a loopback gRPC client, so stdin and output are not
protobuf-encoded to reach a struct in the same process.

Ssh.Configure/Status/Disable let the host enable, rotate and disable the
listener at runtime; rotation rejects certificates authenticated against a
stale policy while established sessions drain.

Two wire-contract additions come with it: KillRequest.process_group, so a
session can tear down a shell's whole process group, and TtyConfig.modes for
the terminal modes an SSH client sends. Existing hosts default both.
@DorianZheng
DorianZheng force-pushed the agent/embed-ssh-server branch from c77528c to d006431 Compare July 28, 2026 07:14
@DorianZheng
DorianZheng marked this pull request as ready for review July 28, 2026 08:09
@boxlite-agent

boxlite-agent Bot commented Jul 28, 2026 •

Copy link
Copy Markdown

📦 BoxLite review — looks good · 9b9cc63

Review evidence

  • ✅ git diff --numstat origin/main...HEAD && git diff origin/main...HEAD — ~9.6k lines added, embedded SSH server feature + 4 fixups
  • ✅ git show 9b9cc63 (user_profile.rs, bridge.rs, session.rs, workload.rs) — passwd-in-container refactor consistent, tests updated to match
  • ✅ git show 1d0059b + grep run_internal call site — umask race checked: bind runs in dedicated single-purpose process, safe
  • ✅ git show 13699c9 (guest.rs, ssh/mod.rs) — teardown now best-effort past quiesce failure, error deferred, test added
  • ⚪ cargo test -p boxlite-guest — no rust toolchain in sandbox (cargo/rustc absent)

Risk notes

  • passwd resolution via getpwuid_r — root_session_profile() now reads via nix::unistd::User inside container mount ns after libcontainer applies namespaces; verified session.rs ordering (namespace entry before lookup) and HOME/SHELL handling in enter_root_home; looks correct
  • reverse-forward socket permission race — bind_owner_only mutates process-wide umask; confirmed via run_internal that this executes in a short-lived current_thread runtime inside a dedicated forked helper process with a single bind call, so no cross-task umask race in practice
  • shutdown teardown ordering — quiesce error deferred until after containers stopped/exit records written/fsync; new test shutdown_writes_exit_records_when_ssh_fails_to_quiesce exercises the degraded path
  • coverage gaps — auth.rs, host_key.rs, server.rs, sftp.rs, forward.rs, streamlocal.rs, control.rs, backoff.rs, limits.rs, exec/state.rs, exec/tty.rs, exec/registry.rs, security_enforcement.rs, resize_tty.rs and the proto changes (~7k of the ~9.6k added lines) were only skimmed via file listing/stat, not read in depth, due to the 8-call budget; these are the largest unreviewed risk surface (auth logic, SFTP protocol handling, TTY/exec state machine)
src/guest/src/container/user_profile.rs
  root_session_profile  +98/… (net simplification within 9b9cc63)  switched to getpwuid_r lookup
src/guest/src/service/ssh/session.rs
  run_internal/enter_root_home  +11/-6 (9b9cc63)  HOME/SHELL set post-namespace-entry
src/guest/src/service/ssh/bridge.rs
  session_exec_env/execution_launch  in 9b9cc63  drops client HOME/SHELL instead of overwrite
src/guest/src/service/ssh/reverse_streamlocal.rs
  bind_owner_only  +31/-1 (1d0059b)  owner-only bind via umask
src/guest/src/service/guest.rs
  shutdown  +54/-6ish (13699c9)  defer quiesce error past teardown
src/guest/src/service/ssh/mod.rs
  SshManager::shutdown  +7/-0  test-only budget close helper
src/guest/src/service/ssh/*.rs (auth, sftp, forward, server, workload, bridge, streamlocal, control, host_key, session)
  ~7000 new lines total  largely unreviewed, sampled only

reviewed 9b9cc63 in a BoxLite microVM · @boxlite-agent review to re-run · powered by BoxLite

@DorianZheng DorianZheng added the e2e-local Triggers the local (in-process) E2E suite on the self-hosted runner label Jul 28, 2026

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 4

🧹 Nitpick comments (14)
src/boxlite/tests/resize_tty.rs (1)

55-58: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

The final stdout drain is unbounded, unlike the readiness loop.

If stty never runs or stdout never closes, this loop blocks forever and the harness kills the test without ever printing the diagnostic on Line 62 — precisely the regression this test exists to catch. The wait() below is bounded, but it is unreachable until the drain finishes.

💚 Suggested change
-    while let Some(chunk) = stream.next().await {
-        stdout.push_str(&chunk);
-    }
+    while let Ok(Some(chunk)) = tokio::time::timeout(Duration::from_secs(30), stream.next()).await {
+        stdout.push_str(&chunk);
+    }
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/boxlite/tests/resize_tty.rs` around lines 55 - 58, Bound the final
stdout-drain loop in the resize TTY test so it cannot wait indefinitely when
stty does not run or stdout remains open. Apply a timeout around the stream
consumption after the readiness loop, while preserving the existing stdout
collection and allowing execution.wait() to run afterward so the diagnostic
remains reachable.
src/guest/src/service/ssh/reverse_streamlocal.rs (2)

442-458: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Simplify the ingress address extraction.

local_addr().ok().unwrap_or_else(|| ...UNSPECIFIED...) fabricates a sentinel purely so the following two checks reject it. A direct Ok(SocketAddr::V4(addr)) match expresses the same rejection without the placeholder.

♻️ Suggested change
-        let SocketAddr::V4(ingress_address) = ingress
-            .local_addr()
-            .ok()
-            .unwrap_or_else(|| SocketAddr::V4(SocketAddrV4::new(Ipv4Addr::UNSPECIFIED, 0)))
-        else {
+        let Ok(SocketAddr::V4(ingress_address)) = ingress.local_addr() else {
             return false;
         };
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/guest/src/service/ssh/reverse_streamlocal.rs` around lines 442 - 458,
Update the ingress address extraction after TcpListener::local_addr in the
reverse streamlocal setup to directly accept only Ok(SocketAddr::V4(addr)) and
return false for errors or non-IPv4 addresses. Remove the fabricated UNSPECIFIED
sentinel and retain the existing localhost and nonzero-port validation.

847-857: 🔒 Security & Privacy | 🔵 Trivial | ⚡ Quick win

Use a constant-time comparison for the ingress bearer token.

received != expected_token is a short-circuiting byte compare on a shared secret that an unprivileged loopback peer can probe repeatedly (no rate limit or attempt cap on the ingress accept path). A constant-time equality keeps this from being a byte-at-a-time oracle.

♻️ Suggested change
-    if received != expected_token {
+    let matched = received
+        .iter()
+        .zip(expected_token)
+        .fold(0_u8, |acc, (a, b)| acc | (a ^ b))
+        == 0;
+    if !matched {
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/guest/src/service/ssh/reverse_streamlocal.rs` around lines 847 - 857,
Replace the short-circuiting comparison in the ingress token validation flow
with the project’s available constant-time byte-equality helper, comparing
received against expected_token while preserving the existing PermissionDenied
error and successful Ok(stream) behavior.
src/guest/src/service/ssh/sftp.rs (1)

915-922: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Capture the OS error before building the status reply.

cvt passes last_os_error() into io_status and reads it again for .with_message, so if the status packet travels between the two calls the code and message can describe different errors. Use a single std::io::Error value and format it once.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/guest/src/service/ssh/sftp.rs` around lines 915 - 922, Update cvt so it
captures std::io::Error::last_os_error() once in the nonzero-result branch, then
reuse that same error value for both io_status and the formatted message,
ensuring the status code and message describe one OS error.
src/guest/src/service/exec/state.rs (1)

448-545: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Test relies on raw fd numbers, which the harness can recycle.

fd_is_open(task_raw_fd) / fd_is_open(tracked_fds) check fd numbers; another test thread in the same binary can allocate the same number between release and the assertion, turning a correct release into a spurious failure. The zygote test in src/guest/src/container/zygote.rs avoids this by probing pipe identity (POLLERR / EOF on the retained peer) instead. Not blocking, just a flake source.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/guest/src/service/exec/state.rs` around lines 448 - 545, Update the
release tests around state_with_tracked_handle and
release_closes_handle_fds_and_aborts_forwarders_idempotently to verify
descriptor closure through retained pipe-peer behavior or pipe identity, rather
than raw fd-number availability via fd_is_open. Preserve the existing assertions
for idempotent release, task cancellation, and retained state cleanup, and apply
the same race-safe approach to tracked and task descriptors.
src/guest/src/service/exec/timeout.rs (1)

37-37: 🩺 Stability & Availability | 🔵 Trivial | ⚡ Quick win

Timeout escalation never targets the process group, so background descendants can survive.

Both the SIGTERM and SIGKILL stages pass process_group: false. The PR adds process-group signaling specifically to reach descendants that leader-only signals miss (see the process_group_kill_reaches_a_background_descendant test), but the timeout watcher — the guest's own backstop for runaway executions — never uses it. A shell/exec workload that backgrounds a child (or traps SIGTERM) can leave that descendant running indefinitely even after the exec "times out" and is SIGKILLed.

♻️ Suggested fix: escalate to the process group on timeout
-        if !exec_state.kill(Signal::SIGTERM, false).await {
+        if !exec_state.kill(Signal::SIGTERM, true).await {
             // Process already exited on its own; nothing more to do.
             return;
         }
@@
-        if exec_state.kill(Signal::SIGKILL, false).await {
+        if exec_state.kill(Signal::SIGKILL, true).await {

Please confirm whether container-scoped executions get equivalent cleanup via container teardown independent of this path — if not, this gap applies there too.

Also applies to: 54-54

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/guest/src/service/exec/timeout.rs` at line 37, Update the timeout
escalation in the watcher around exec_state.kill so both the SIGTERM and SIGKILL
calls target the process group by passing process_group: true. Preserve the
existing escalation order and behavior while ensuring timed-out executions also
terminate background descendants.
src/guest/src/service/ssh/server.rs (1)

328-335: 🎯 Functional Correctness | 🔵 Trivial | 💤 Low value

env_request rejects updates to already-set variables once at the cap.

At MAX_ENV_VARS, re-sending an existing name fails even though the map size would not grow. Consider state.env.len() >= MAX_ENV_VARS && !state.env.contains_key(variable_name), matching the guard already used in apply_pty_request.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/guest/src/service/ssh/server.rs` around lines 328 - 335, Update the
environment-cap condition in env_request so it rejects only new variable names
when state.env has reached MAX_ENV_VARS; allow updates to names already present
in state.env while preserving the existing name, value-size, and NUL validation
checks.
src/guest/src/service/ssh/host_key.rs (1)

70-73: 🔒 Security & Privacy | 🔵 Trivial | 💤 Low value

Create the host-key directory with restrictive mode.

create_dir_all uses the process umask (typically 0o755). The key file itself is 0600, so this isn't exploitable today, but tightening the directory to 0o700 matches the sshd convention and the 0o077 check enforced on the key.

🛡️ Proposed hardening
-    std::fs::create_dir_all(parent).map_err(HostKeyError::Io)?;
+    std::fs::create_dir_all(parent).map_err(HostKeyError::Io)?;
+    std::fs::set_permissions(parent, std::fs::Permissions::from_mode(0o700))
+        .map_err(HostKeyError::Io)?;

(use std::os::unix::fs::PermissionsExt; is already needed in scope.)

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/guest/src/service/ssh/host_key.rs` around lines 70 - 73, Update the
directory creation in the host-key path setup around path.parent() and
create_dir_all to enforce restrictive 0o700 permissions, using PermissionsExt as
indicated. Preserve the existing parent validation and HostKeyError::Io mapping
while ensuring the resulting host-key directory is not group- or
world-accessible.
src/guest/src/service/ssh/mod.rs (1)

595-613: 🩺 Stability & Availability | 🔵 Trivial | 💤 Low value

Pre-authentication budget is effectively 2× AUTHENTICATION_TIMEOUT.

run_stream is wrapped in a 30s timeout, and monitor_session then starts a fresh 30s auth timer, so an idle peer can hold a connection permit for ~60s. Consider passing a deadline (Instant) through instead of two independent timers.

Also applies to: 701-707

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/guest/src/service/ssh/mod.rs` around lines 595 - 613, Use one shared
authentication deadline across the pre-authentication flow instead of starting
independent timers around run_stream and monitor_session. Update the relevant
handshake and monitor_session paths to derive remaining time from a single
Instant deadline and pass that deadline through, ensuring an idle peer cannot
exceed AUTHENTICATION_TIMEOUT overall while preserving shutdown and
connection-policy cancellation behavior.
src/guest/src/service/server.rs (1)

134-153: 🩺 Stability & Availability | 🔵 Trivial | 💤 Low value

Rejected peers are logged at warn per connection attempt.

A tenant process can loop loopback vsock connects and flood guest logs. Consider rate-limiting or demoting to debug after the first rejection.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/guest/src/service/server.rs` around lines 134 - 153, Update the
rejected-peer logging in the listener’s incoming connection filter to avoid
emitting a warn record for every rejected connection attempt. Demote the
per-peer rejection messages in the non-host and missing-address branches to
debug-level logging, or reuse an existing rate-limited warning mechanism if
available, while preserving rejection before tonic receives the stream.
src/guest/src/service/ssh/bridge.rs (3)

741-746: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

tokio::spawn(async {}) as a placeholder output task reads as a workaround.

Making spawn_execution_cleanup take Option<JoinHandle<()>> would express "no output task" directly.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/guest/src/service/ssh/bridge.rs` around lines 741 - 746, Update
spawn_execution_cleanup and its callers to accept an Option<JoinHandle<()>>,
allowing cleanup_failed_execution_start to pass None when no output task exists.
Remove the placeholder tokio::spawn(async {}) handle while preserving the
existing cleanup behavior for callers that provide an output task.

329-341: 🚀 Performance & Scalability | 🔵 Trivial | ⚡ Quick win

Blocking filesystem I/O runs on the runtime thread while the container mutex is held.

root_session_profile does canonicalize + read of the container's /etc/passwd (and symlink_metadata per path component) synchronously. It happens on every channel open, under the per-container mutex that also serializes exec builds. Consider resolving it once per container and caching it, or moving the call into spawn_blocking.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/guest/src/service/ssh/bridge.rs` around lines 329 - 341, Update
resolve_single_container_profile so the synchronous root_session_profile
filesystem work does not run on the async runtime thread while holding the
container mutex. Prefer resolving and caching the profile once per container;
otherwise clone the container handle, release the mutex before calling
root_session_profile, and perform that call via spawn_blocking while preserving
the existing return and error behavior.

295-315: 🚀 Performance & Scalability | 🔵 Trivial | 💤 Low value

Teardown always burns the full grace window, even when the process is already gone.

sleep_until(deadline) is unconditional, so a channel whose process exited on SIGHUP still holds the teardown task for the whole PROCESS_TERMINATION_GRACE. Racing the sleep against the execution's exit signal would release resources sooner.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/guest/src/service/ssh/bridge.rs` around lines 295 - 315, Update
terminate_process_group to stop waiting once the execution’s exit signal is
received, while retaining PROCESS_TERMINATION_GRACE as the maximum deadline
before sending FORCED_TERMINATION_SIGNAL. Replace the unconditional
sleep_until(deadline) with a race between the deadline and the execution exit
notification, then only send the forced signal if the process has not already
exited.
src/guest/src/container/command.rs (1)

382-391: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Guard duplicates BoxliteWorkloadExecutor::validate_spec, but the early check is worth keeping.

Fails before the zygote round-trip; just keep the two error strings in sync (src/guest/src/service/ssh/workload.rs Line 164-168 uses the same wording today).

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/guest/src/container/command.rs` around lines 382 - 391, Keep the early
PTY compatibility guard in the command path, but synchronize its
BoxliteError::Config message with the validation message in
BoxliteWorkloadExecutor::validate_spec. Ensure both checks use the exact same
wording while preserving the existing pre-zygote return behavior.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/guest/src/service/guest.rs`:
- Around line 100-108: Change the SSH quiesce handling around
self.ssh_manager.shutdown() so a failure is recorded and logged but does not
return early; allow the remaining teardown steps, including filesystem sync, to
run. Preserve the degraded quiesce error for reporting after Step 3 if the
function’s existing return flow supports it, while keeping successful shutdown
behavior unchanged.

In `@src/guest/src/service/ssh/bridge.rs`:
- Around line 317-327: Update ChannelBridge::drop and the terminate_running path
so cleanup does not panic when the Tokio runtime is shutting down: guard the
spawned teardown task and skip spawning if the runtime no longer accepts tasks.
Preserve normal channel-close behavior and ensure terminal shutdown still relies
on closing all sessions.

In `@src/guest/src/service/ssh/reverse_streamlocal.rs`:
- Around line 106-122: Update the reverse streamlocal bind flow around
UnixListener::bind to temporarily set the process umask to 0o177 before creating
the socket, then restore the previous umask immediately after bind, including
when bind fails. Keep the existing set_permissions normalization and
BoundSocket::capture behavior unchanged.

In `@src/guest/src/service/ssh/workload.rs`:
- Around line 663-689: Update the subprocess test around SUBPROCESS_CASE "exec"
to derive the expected directory from the process’s actual current working
directory after enter_root_home, rather than from inherited_cwd or the stale PWD
environment variable. Keep the shell command and output assertion unchanged; do
not modify enter_root_home unless explicitly choosing the separate
session-environment fix.

---

Nitpick comments:
In `@src/boxlite/tests/resize_tty.rs`:
- Around line 55-58: Bound the final stdout-drain loop in the resize TTY test so
it cannot wait indefinitely when stty does not run or stdout remains open. Apply
a timeout around the stream consumption after the readiness loop, while
preserving the existing stdout collection and allowing execution.wait() to run
afterward so the diagnostic remains reachable.

In `@src/guest/src/container/command.rs`:
- Around line 382-391: Keep the early PTY compatibility guard in the command
path, but synchronize its BoxliteError::Config message with the validation
message in BoxliteWorkloadExecutor::validate_spec. Ensure both checks use the
exact same wording while preserving the existing pre-zygote return behavior.

In `@src/guest/src/service/exec/state.rs`:
- Around line 448-545: Update the release tests around state_with_tracked_handle
and release_closes_handle_fds_and_aborts_forwarders_idempotently to verify
descriptor closure through retained pipe-peer behavior or pipe identity, rather
than raw fd-number availability via fd_is_open. Preserve the existing assertions
for idempotent release, task cancellation, and retained state cleanup, and apply
the same race-safe approach to tracked and task descriptors.

In `@src/guest/src/service/exec/timeout.rs`:
- Line 37: Update the timeout escalation in the watcher around exec_state.kill
so both the SIGTERM and SIGKILL calls target the process group by passing
process_group: true. Preserve the existing escalation order and behavior while
ensuring timed-out executions also terminate background descendants.

In `@src/guest/src/service/server.rs`:
- Around line 134-153: Update the rejected-peer logging in the listener’s
incoming connection filter to avoid emitting a warn record for every rejected
connection attempt. Demote the per-peer rejection messages in the non-host and
missing-address branches to debug-level logging, or reuse an existing
rate-limited warning mechanism if available, while preserving rejection before
tonic receives the stream.

In `@src/guest/src/service/ssh/bridge.rs`:
- Around line 741-746: Update spawn_execution_cleanup and its callers to accept
an Option<JoinHandle<()>>, allowing cleanup_failed_execution_start to pass None
when no output task exists. Remove the placeholder tokio::spawn(async {}) handle
while preserving the existing cleanup behavior for callers that provide an
output task.
- Around line 329-341: Update resolve_single_container_profile so the
synchronous root_session_profile filesystem work does not run on the async
runtime thread while holding the container mutex. Prefer resolving and caching
the profile once per container; otherwise clone the container handle, release
the mutex before calling root_session_profile, and perform that call via
spawn_blocking while preserving the existing return and error behavior.
- Around line 295-315: Update terminate_process_group to stop waiting once the
execution’s exit signal is received, while retaining PROCESS_TERMINATION_GRACE
as the maximum deadline before sending FORCED_TERMINATION_SIGNAL. Replace the
unconditional sleep_until(deadline) with a race between the deadline and the
execution exit notification, then only send the forced signal if the process has
not already exited.

In `@src/guest/src/service/ssh/host_key.rs`:
- Around line 70-73: Update the directory creation in the host-key path setup
around path.parent() and create_dir_all to enforce restrictive 0o700
permissions, using PermissionsExt as indicated. Preserve the existing parent
validation and HostKeyError::Io mapping while ensuring the resulting host-key
directory is not group- or world-accessible.

In `@src/guest/src/service/ssh/mod.rs`:
- Around line 595-613: Use one shared authentication deadline across the
pre-authentication flow instead of starting independent timers around run_stream
and monitor_session. Update the relevant handshake and monitor_session paths to
derive remaining time from a single Instant deadline and pass that deadline
through, ensuring an idle peer cannot exceed AUTHENTICATION_TIMEOUT overall
while preserving shutdown and connection-policy cancellation behavior.

In `@src/guest/src/service/ssh/reverse_streamlocal.rs`:
- Around line 442-458: Update the ingress address extraction after
TcpListener::local_addr in the reverse streamlocal setup to directly accept only
Ok(SocketAddr::V4(addr)) and return false for errors or non-IPv4 addresses.
Remove the fabricated UNSPECIFIED sentinel and retain the existing localhost and
nonzero-port validation.
- Around line 847-857: Replace the short-circuiting comparison in the ingress
token validation flow with the project’s available constant-time byte-equality
helper, comparing received against expected_token while preserving the existing
PermissionDenied error and successful Ok(stream) behavior.

In `@src/guest/src/service/ssh/server.rs`:
- Around line 328-335: Update the environment-cap condition in env_request so it
rejects only new variable names when state.env has reached MAX_ENV_VARS; allow
updates to names already present in state.env while preserving the existing
name, value-size, and NUL validation checks.

In `@src/guest/src/service/ssh/sftp.rs`:
- Around line 915-922: Update cvt so it captures std::io::Error::last_os_error()
once in the nonzero-result branch, then reuse that same error value for both
io_status and the formatted message, ensuring the status code and message
describe one OS error.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 69805489-9590-4fa3-aa46-d8af62848243

📥 Commits

Reviewing files that changed from the base of the PR and between e3563b3 and d006431.

⛔ Files ignored due to path filters (1)
  • Cargo.lock is excluded by !**/*.lock
📒 Files selected for processing (40)
  • make/test.mk
  • src/boxlite/src/portal/interfaces/exec.rs
  • src/boxlite/tests/resize_tty.rs
  • src/boxlite/tests/security_enforcement.rs
  • src/guest/Cargo.toml
  • src/guest/src/container/command.rs
  • src/guest/src/container/lifecycle.rs
  • src/guest/src/container/mod.rs
  • src/guest/src/container/user_profile.rs
  • src/guest/src/container/zygote.rs
  • src/guest/src/main.rs
  • src/guest/src/mounts.rs
  • src/guest/src/reaper.rs
  • src/guest/src/service/exec/error.rs
  • src/guest/src/service/exec/exec_handle.rs
  • src/guest/src/service/exec/executor.rs
  • src/guest/src/service/exec/mod.rs
  • src/guest/src/service/exec/registry.rs
  • src/guest/src/service/exec/state.rs
  • src/guest/src/service/exec/timeout.rs
  • src/guest/src/service/exec/tty.rs
  • src/guest/src/service/guest.rs
  • src/guest/src/service/mod.rs
  • src/guest/src/service/server.rs
  • src/guest/src/service/ssh/auth.rs
  • src/guest/src/service/ssh/backoff.rs
  • src/guest/src/service/ssh/bridge.rs
  • src/guest/src/service/ssh/control.rs
  • src/guest/src/service/ssh/forward.rs
  • src/guest/src/service/ssh/host_key.rs
  • src/guest/src/service/ssh/limits.rs
  • src/guest/src/service/ssh/mod.rs
  • src/guest/src/service/ssh/reverse_streamlocal.rs
  • src/guest/src/service/ssh/server.rs
  • src/guest/src/service/ssh/session.rs
  • src/guest/src/service/ssh/sftp.rs
  • src/guest/src/service/ssh/streamlocal.rs
  • src/guest/src/service/ssh/workload.rs
  • src/shared/proto/boxlite/v1/service.proto
  • src/shared/src/lib.rs

Comment thread src/guest/src/service/guest.rs Outdated
Comment on lines +317 to +327
impl Drop for ChannelBridge {
fn drop(&mut self) {
// Transport loss may drop the russh handler without delivering a
// channel-close callback. Keep SSH-owned processes tied to the channel
// in that path as well; terminate_running takes the cancel sender so a
// normal channel-close does not start teardown twice.
if self.output_cancel.is_some() {
self.terminate_running();
}
}
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🩺 Stability & Availability | 🟡 Minor | ⚡ Quick win

tokio::spawn in Drop panics if the bridge is dropped while the runtime is shutting down.

Transport loss during guest shutdown drops handlers from the runtime shutdown path, where spawn panics with "A Tokio 1.x context was found, but it is being shut down". Since terminal shutdown closes all sessions anyway, a guarded spawn is enough.

🛡️ Proposed guard
     fn drop(&mut self) {
-        if self.output_cancel.is_some() {
+        if self.output_cancel.is_some() && tokio::runtime::Handle::try_current().is_ok() {
             self.terminate_running();
+        } else if let Some(cancel) = self.output_cancel.take() {
+            let _ = cancel.send(());
         }
     }
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
impl Drop for ChannelBridge {
fn drop(&mut self) {
// Transport loss may drop the russh handler without delivering a
// channel-close callback. Keep SSH-owned processes tied to the channel
// in that path as well; terminate_running takes the cancel sender so a
// normal channel-close does not start teardown twice.
if self.output_cancel.is_some() {
self.terminate_running();
}
}
}
impl Drop for ChannelBridge {
fn drop(&mut self) {
// Transport loss may drop the russh handler without delivering a
// channel-close callback. Keep SSH-owned processes tied to the channel
// in that path as well; terminate_running takes the cancel sender so a
// normal channel-close does not start teardown twice.
if self.output_cancel.is_some() && tokio::runtime::Handle::try_current().is_ok() {
self.terminate_running();
} else if let Some(cancel) = self.output_cancel.take() {
let _ = cancel.send(());
}
}
}
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/guest/src/service/ssh/bridge.rs` around lines 317 - 327, Update
ChannelBridge::drop and the terminate_running path so cleanup does not panic
when the Tokio runtime is shutting down: guard the spawned teardown task and
skip spawning if the runtime no longer accepts tasks. Preserve normal
channel-close behavior and ensure terminal shutdown still relies on closing all
sessions.

Comment thread src/guest/src/service/ssh/reverse_streamlocal.rs
Comment thread src/guest/src/service/ssh/workload.rs Outdated
Comment on lines +663 to +689
let command = r#"printf '\036%s\0%s\037' "$PWD" 'literal spaces ; $(not reparsed)'"#;
let output = Command::new(std::env::current_exe().unwrap())
.args([
"--exact",
SUBPROCESS_TEST,
"--nocapture",
"--test-threads=1",
])
.env(SUBPROCESS_CASE, "exec")
.env(SUBPROCESS_COMMAND, command)
.current_dir(inherited_cwd.path())
.output()
.unwrap();
assert!(
output.status.success(),
"{}",
String::from_utf8_lossy(&output.stderr)
);
let expected = format!(
"\u{1e}{}\0literal spaces ; $(not reparsed)\u{1f}",
expected_home.display()
);
assert!(output
.stdout
.windows(expected.len())
.any(|window| window == expected.as_bytes()));
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🔴 Critical | ⚡ Quick win

Failing test: $PWD is not updated by set_current_dir, so the assertion compares against the inherited tempdir.

enter_root_home uses std::env::set_current_dir and exports only HOME; the PWD variable the subprocess inherits still points at inherited_cwd (set via .current_dir(...)). A non-interactive sh -c (dash and friends) does not recompute PWD, so the printed value is the tempdir, not the resolved root home — which matches the CI failure on both arches. Query the real cwd instead.

🐛 Proposed fix
-        let command = r#"printf '\036%s\0%s\037' "$PWD" 'literal spaces ; $(not reparsed)'"#;
+        let command = r#"printf '\036%s\0%s\037' "$(pwd)" 'literal spaces ; $(not reparsed)'"#;

If the intent is that SSH sessions also see a correct PWD, the durable fix is to export it alongside HOME in enter_root_home (src/guest/src/service/ssh/session.rs Line 53-57) and keep the assertion as-is.

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
let command = r#"printf '\036%s\0%s\037' "$PWD" 'literal spaces ; $(not reparsed)'"#;
let output = Command::new(std::env::current_exe().unwrap())
.args([
"--exact",
SUBPROCESS_TEST,
"--nocapture",
"--test-threads=1",
])
.env(SUBPROCESS_CASE, "exec")
.env(SUBPROCESS_COMMAND, command)
.current_dir(inherited_cwd.path())
.output()
.unwrap();
assert!(
output.status.success(),
"{}",
String::from_utf8_lossy(&output.stderr)
);
let expected = format!(
"\u{1e}{}\0literal spaces ; $(not reparsed)\u{1f}",
expected_home.display()
);
assert!(output
.stdout
.windows(expected.len())
.any(|window| window == expected.as_bytes()));
}
let command = r#"printf '\036%s\0%s\037' "$(pwd)" 'literal spaces ; $(not reparsed)'"#;
let output = Command::new(std::env::current_exe().unwrap())
.args([
"--exact",
SUBPROCESS_TEST,
"--nocapture",
"--test-threads=1",
])
.env(SUBPROCESS_CASE, "exec")
.env(SUBPROCESS_COMMAND, command)
.current_dir(inherited_cwd.path())
.output()
.unwrap();
assert!(
output.status.success(),
"{}",
String::from_utf8_lossy(&output.stderr)
);
let expected = format!(
"\u{1e}{}\0literal spaces ; $(not reparsed)\u{1f}",
expected_home.display()
);
assert!(output
.stdout
.windows(expected.len())
.any(|window| window == expected.as_bytes()));
}
🧰 Tools
🪛 GitHub Actions: Test / Rust Tests (linux-arm64-gnu)

[error] 685-685: Test failed in boxlite_guest::service::ssh::workload::tests::exec_workload_preserves_one_command_argument_and_root_home. Panic: assertion failed: output.stdout.windows(expected.len()).any(|window| window == expected.as_bytes()).

🪛 GitHub Actions: Test / Rust Tests (linux-x64-gnu)

[error] 685-685: Test failed: assertion failed in exec_workload_preserves_one_command_argument_and_root_home. Assertion: output.stdout.windows(expected.len()).any(|window| window == expected.as_bytes()).

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/guest/src/service/ssh/workload.rs` around lines 663 - 689, Update the
subprocess test around SUBPROCESS_CASE "exec" to derive the expected directory
from the process’s actual current working directory after enter_root_home,
rather than from inherited_cwd or the stale PWD environment variable. Keep the
shell command and output assertion unchanged; do not modify enter_root_home
unless explicitly choosing the separate session-environment fix.

Source: Pipeline failures

tester added 4 commits July 28, 2026 16:39
Two workload subprocess tests encoded assumptions about the runner, and the
guest's new C dependency needs a cross toolchain the macOS lint job lacks.

selected_home mirrored session::enter_root_home's fallback with is_dir(),
but root's home is a directory for everyone and traversable only by root —
an unprivileged run legitimately lands on / while the test still expected
/root. It now asks for search permission, which is what set_current_dir
needs. This is the failure CI reported.

The descriptor-boundary test dup2'd onto a fixed fd 4096. dup2 rejects a
target at or above RLIMIT_NOFILE with EBADF, so the call fails before
close_inherited_fds runs wherever the soft limit is lower; it now picks the
highest descriptor the process may actually hold. CI cancelled after the
first failure, so whether it reached this test is unknown — the fix is for
robustness, not a diagnosed CI failure.

Only the macOS branch of `make clippy` cross-compiles boxlite-guest to a
linux-musl target, and the guest links C since russh brought in ring. That
job now installs the same bottled cross toolchain and linker config
scripts/setup/setup-macos.sh gives developers. Linux clippy builds the guest
for its host target and needs nothing new.
The Shutdown RPC returned as soon as the SSH quiesce timed out — before
containers were stopped, init exit records were written, or filesystems
were synced. That left COW disks inconsistent on the next start and the
box's exit status unrecorded, and nothing else picks the teardown back
up: `shutting_down` has already told the reaper to stand down.

The quiesce only buys ordering against session handlers registering new
executions, so a stuck session now logs and the teardown continues. The
failure is still reported, after the remaining steps have run.
`bind` creates a pathname socket with `0777 & ~umask`, and `connect(2)`
on one needs only write permission, so the chmod that followed left a
window in which any container process could reach the listener. The
socket may sit anywhere the target validator allows, including a
world-writable directory, so the umask has to be narrow across the bind.
The root account's home and shell were resolved from the guest, against the
container's mounted rootfs. Reaching into another mount namespace by path
meant hand-rolling a chroot-like symlink walker, and its result was then
discarded: every session workload re-resolves the profile inside the
container and chdirs there itself, and the login shell never launched
anything — the program comes from the workload placeholder.

Resolution now happens once, where passwd is actually readable, through the
same `getpwuid_r` libcontainer uses to seed HOME. The shell fallback list
goes with it: OpenSSH substitutes /bin/sh only for an empty passwd field and
otherwise execs what passwd names, so a shell an image does not ship now
fails visibly instead of silently becoming a different shell.

HOME and SHELL are dropped from the exec request rather than overwritten,
since neither has a correct value outside the container.
@DorianZheng
DorianZheng requested a review from a team as a code owner July 28, 2026 12:18
@DorianZheng
DorianZheng merged commit 17f0865 into main Jul 28, 2026
39 of 41 checks passed
@DorianZheng
DorianZheng deleted the agent/embed-ssh-server branch July 28, 2026 12:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

e2e-local Triggers the local (in-process) E2E suite on the self-hosted runner

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant