Skip to content

[codex] Sync embedded computer-use-linux v0.2.5 - #419

Merged
ilysenko merged 2 commits into
ilysenko:mainfrom
avifenesh:codex/sync-computer-use-v0.2.5
Jun 6, 2026
Merged

[codex] Sync embedded computer-use-linux v0.2.5#419
ilysenko merged 2 commits into
ilysenko:mainfrom
avifenesh:codex/sync-computer-use-v0.2.5

Conversation

@avifenesh

Copy link
Copy Markdown
Collaborator

Summary

  • sync embedded codex-computer-use-linux to the standalone computer-use-linux v0.2.5 screenshot payload behavior
  • add bounded PNG/JPEG screenshot payload controls and coordinate metadata to get_app_state / screenshot
  • opt JavaScript GitHub Actions into Node 24 runtime while keeping existing job Node 24 setup

Tracks agent-sh/computer-use-linux#19.

Validation

  • cargo fmt --check -p codex-computer-use-linux
  • cargo check -p codex-computer-use-linux --all-targets
  • cargo test -p codex-computer-use-linux
  • cargo clippy -p codex-computer-use-linux --all-targets -- -D warnings
  • node --test linux-features/agent-workspace/test.js
  • cargo build -p codex-computer-use-linux
  • node /home/avifenesh/projects/computer-use-linux/scripts/zod-check/check.mjs --command target/debug/codex-computer-use-linux
  • cargo check --workspace --all-targets
  • cargo test --workspace --all-targets
  • git diff --check

@avifenesh

Copy link
Copy Markdown
Collaborator Author

@ilysenko @Leay15 I'm considering to extract comp use out and adding it directly to the mcp, instead of correlating between the two on any change. Thoughts?

@ilysenko

ilysenko commented Jun 5, 2026

Copy link
Copy Markdown
Owner

One suggestion before merging: can we make the new window-targeted click path fail closed where we already can verify the target?

type_text / press_key already use focus_target_for_input(...) and avoid sending input when focus verification fails. For click, the new target focus result is ignored, and with relative=true list/resolve/bounds failures can leave the original x/y to be sent as absolute desktop coordinates.

I think click should reuse the same target verification path when a window target is provided, and relative=true should require resolved window bounds before sending input.

The screenshot payload changes look fine to me; this is mostly about input safety.

@ilysenko

ilysenko commented Jun 5, 2026

Copy link
Copy Markdown
Owner

@avifenesh
Decision is yours, of course. The way I see it: extracting it is possible, but it may create more maintenance than the current embedded sync.

With the current layout, the backend is in the workspace, CI builds it, Nix sees it, native packages include it, and updater rebuilds use the same source.

If it moves out, we need pinned versions/checksums, reproducible Nix builds, CI coverage for the exact shipped backend, native package integration, updater rebuild consistency, and no accidental dependency on a moving external release.

So the main requirement would be: keep it fully pinned and reproducible. Otherwise the embedded workspace copy is fine as-is.

@avifenesh avifenesh left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is an auto review done by revuto.


I found one input-safety issue in the new targeted click path.

Comment thread computer-use-linux/src/server.rs Outdated
// Raise the target window first (if specified) so the click lands on the
// intended app rather than whatever is stacked on top at that pixel.
if let Some(target) = params.window_target() {
let _ = focus_window_target(&target).await;

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is an auto review done by revuto.


Targeted clicks should fail closed here. focus_window_target can return an error or a focus result that does not satisfy the requested target (the keyboard paths handle this via focus_target_for_input/focus_satisfies_target), but this call discards that result and still proceeds to send the click. In the same block, relative=true bounds resolution is best-effort, so if listing/resolution/bounds are unavailable the original x/y are sent as absolute desktop coordinates. For this destructive tool that can click the currently focused or occluding window instead of the requested target; please require verified focus, and require resolved bounds before translating relative clicks.

@ilysenko ilysenko left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed current head 8676f8d; targeted click now fails closed and checks pass.

@ilysenko
ilysenko merged commit f44140f into ilysenko:main Jun 6, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants