Skip to content

Control overlay repaints the viewport every frame, and observe leaves the Accessibility domain enabled (GPU/CPU during sessions) #391

Open

Description

@code-yeongyu

Summary

While an agent session controls a page, two things keep the browser busy even when the agent is idle between commands:

  1. ControlOverlay.tsx renders a full-viewport div with animation: bsk-breathe 3s ease-in-out infinite, and bsk-breathe animates an inset box-shadow. box-shadow is not compositor-animatable, so the whole viewport is repainted every frame for the entire session. It also ignores prefers-reduced-motion.
  2. Every observe/snapshot sends Accessibility.enable (tools/vom/capture-coordinator.ts) and never sends Accessibility.disable. Per the CDP docs this "turns on accessibility for the page, which can impact performance until accessibility is disabled", so the renderer keeps the AX tree live for the rest of the session.

Measured (extension 0.3.2, bsk 0.3.2, Chromium 153-based browser, macOS, Apple Silicon; CPU of every browser process sampled from ps once a second, grouped by process role; one session: navigate, observe x3, snapshot x3, click, screenshot x2, evaluate x3, 5 s apart)

Controlled A/B: a fresh headless Chrome for Testing 154 profile per run with only the extension under test loaded unpacked, connected to the local daemon; builds alternated so machine load hits both equally. Percent of one core, mean over the window:

Run Build Idle baseline: all browser processes / GPU process During session: all / GPU 0-30 s after session stop: all
1 0.3.2 3.2 / 1.7 37.2 / 21.5 1.2
2 patched 3.1 / 1.1 14.9 / 6.7 0.9
3 0.3.2 2.0 / 0.8 33.5 / 19.5 1.3
4 patched 2.1 / 0.9 10.1 / 4.4 0.7

Mean: GPU process 20.5% -> 5.6% (-73%), whole browser 35.4% -> 12.5% (-65%) during a session. In a real, in-use desktop browser the same session on 0.3.2 took the GPU process from roughly 30% to 107-133% of a core.

Expected

The "agent is in control" cue costs near-zero GPU while the agent is idle, and observation leaves no ongoing accessibility work behind.

Fix we are running (small, happy to send it as a PR)

  • Paint the glow once as a static inset shadow; for the onset cue, pulse the opacity of a pre-painted shadow layer (compositor-only) for two cycles, then rest; no pulse under prefers-reduced-motion: reduce.
  • After the capture has read the AX tree, send Accessibility.disable to each target whose AX reads finished (skipping targets with a timed-out read so nothing queues behind it), through the capture's existing worker pool. Refs are keyed by backend DOM node id, so nothing depends on AX ids across captures.

Diff and tests: code-yeongyu#1 (MIT, credits upstream). The two new tests fail on main (3f10983) and pass with the change; the extension suite, tsc --noEmit and biome pass.

Workaround

Turn off "Control hints" in the extension popup (covers the overlay only).

Activity

  1. NianJiuZst commented on Oct 3, 2026

    @NianJiuZst
    Contributor

    👍 I support this direction.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions