Skip to content

Evolve experimental PiP into a multi-target Agent View #3365

Description

@f-trycua

Problem

The experimental macOS PiP surface is a process-global, single-frame preview. It captures only the latest post-action native screenshot, so concurrent sessions overwrite one another and exact Chrome tabs are not represented independently.

For the shared-computer model, the visual surface should group exact targets by the existing lifecycle session without adding claim/release ownership APIs.

Scope

Evolve experimental PiP into a multi-target Agent View:

  • preserve existing --experimental-pip and config compatibility;
  • group cards by the existing session/runtime identity;
  • automatically identify native targets by pid + window_id;
  • identify browser targets by target_id + tab_id;
  • capture Chrome tabs through the existing exact CDP binding, without activating the tab or foregrounding its window;
  • keep multiple target cards visible instead of replacing one global frame;
  • remove session cards when that session ends;
  • keep target assignment automatic and internal: no claim/release MCP tools;
  • retain Windows/Linux stubs, with native backends remaining tracked by PiP preview (experimental): native Windows + Linux backends #1729.

Non-goals

  • virtual machines or macOS display creation;
  • security ownership/lease enforcement between agents;
  • continuous video streaming;
  • human input forwarding from Agent View into a target;
  • Windows/Linux native PiP implementation.

Acceptance evidence

  • focused shared-model and routing tests;
  • macOS build/compile verification;
  • live macOS smoke proving two native windows and two Chrome tabs remain distinct in Agent View;
  • existing CLI/config compatibility remains intact.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions