Skip to content

fix(extension): reduce idle overlay and Accessibility overhead - #408

Open
NianJiuZst wants to merge 5 commits into
Tencent:mainfrom
NianJiuZst:fix/391-idle-overlay-ax-cost
Open

NianJiuZst wants to merge 5 commits into
Tencent:mainfrom
NianJiuZst:fix/391-idle-overlay-ax-cost

Conversation

@NianJiuZst

@NianJiuZst NianJiuZst commented Oct 8, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Fixes #391.

An agent session can keep doing browser work between commands: the control overlay continuously animates a viewport-sized inset shadow, and observation enables Chrome's Accessibility domain without disabling it afterward. This PR makes the control glow settle after two opacity pulses and releases Accessibility on the targets whose capture reads have finished.

Problem and resulting behavior

Area Before After
Control glow An infinite box-shadow animation runs for the entire time the overlay is visible. Two static shadow layers are painted; only the brighter layer's opacity animates, for two 3-second cycles, before the resting glow remains.
Reduced motion The breathing animation ignores the user's motion preference. prefers-reduced-motion: reduce disables the pulse and leaves the static control glow.
Observation Each target receives Accessibility.enable, but completed captures leave the domain enabled. Successfully enabled targets receive Accessibility.disable after the AX read phase, before document-identity and geometry work.
Capture interruption There is no capture-scoped Accessibility release. Failure or cancellation triggers release for the targets already enabled, while attachment and pending-read checks protect cleanup dispatch.

Implementation

Control overlay

ControlOverlay.tsx keeps the resting inset shadow on the existing overlay and adds a brighter child layer. The bsk-breathe keyframes now contain only opacity changes, with a finite iteration count of two. The child has a resting opacity of zero, and a reduced-motion media query disables its animation. The control indicator, interruption button, and input-blocking behavior remain available.

Accessibility lifetime

capture-coordinator.ts records each successfully enabled target together with its debugger attachment ID. Release uses the capture's existing worker helper, with at most four cleanup requests in flight, and is awaited before the capture completes.

  • Release runs when target collection fails or is cancelled, and in finally around the AX read phase.
  • Cleanup uses sendGuarded with the original attachment ID in production, so an old capture cannot reconnect or send cleanup to a replacement attachment.
  • A target whose AX read times out is removed from the release set because the underlying Chrome command can still be running.
  • command-deadline.ts guards Accessibility.disable against earlier pending reads and gives cleanup itself a 1-second deadline. A timed-out disable retains its target gate until Chrome replies or the debugger detaches; late replies cannot clear a replacement attachment.
  • Rejected cleanup requests are treated as best-effort cleanup and do not replace the capture result or its original error.

Observation refs retain their backend DOM node IDs and document/attachment identity; the implementation does not depend on Accessibility node IDs surviving between captures. The changelog records both user-visible changes.

Regression coverage

Eight new cases extend three existing test files:

  • ControlOverlay.test.tsx (2): static shadows, opacity-only keyframes, exactly two pulses, no infinite animation, and the reduced-motion override.
  • capture-coordinator.test.ts (5): release for root and child targets after their AX reads; release before identity/geometry work; dispatch on the original attachment; no cleanup on a replacement attachment; skipping a timed-out child while releasing the root; and cleanup during cancellation. Some cases assert multiple related behaviors.
  • command-deadline.test.ts (1): a pending child-session AX read blocks its own cleanup without blocking root-target cleanup.

Validation — 2026-10-20

Merged upstream main e8777ab71bb851196895515b53eea2481462dd57; current revision d305f8de8867bc42081d23c765502ab2291ce44b.

  • Extension suite: 2,683 passed, 244 skipped. The three directly affected test files pass 82 tests.
  • Extension TypeScript, changed-file Biome, production build and diff checks passed.
  • Two additional out-of-tree capture regressions cover completion and cancellation with a stuck child cleanup. Both settle within the cleanup deadline; the root remains usable, the affected child stays gated, and its late response restores access.
  • Real Chrome 153: after AX reads, an intentionally blocked renderer kept Accessibility.disable pending. The production read gate returned its timeout after 1,002 ms, rejected another read while pending, and recovered after the original command settled following execution termination.
  • Independent full-PR P0–P2 review passed. New proof scripts/logs remain outside the repository.

Limits and performance evidence

  • Capture regressions use simulated CDP; the blocked-renderer deadline was also checked against real Chrome. CPU/GPU profiling was not rerun; skipped tests are not counted as passes.
  • The measurements in issue #391 belong to the reporter's combined overlay/Accessibility experiment. They motivate the change but are not independently reproduced results for this branch or separate measurements of each optimization.
  • Cleanup is best effort. A timed-out target can retain Accessibility mode until a later successful capture or debugger detachment. The pending-read guard prevents queuing behind a known stuck read, and the new deadline bounds a disable request that itself stops responding. A deadline does not cancel Chrome's underlying command.

Thanks to @code-yeongyu for the report, measurements, and original prototype.

NianJiuZst and others added 5 commits October 8, 2026 12:19
The control overlay animated an inset box-shadow on a full-viewport element
forever. box-shadow is not compositor-animatable, so the whole viewport was
repainted on every frame for as long as the agent held control, even while it
was idle. The animation also ignored prefers-reduced-motion.

Paint the resting glow and the brighter glow as two static shadows and fade
only the brighter layer's opacity, for two pulses. Under reduced motion only
the resting glow is shown.

Refs Tencent#391.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LZRCYsFEKFsAZe6i8WAc9o
Every capture sent Accessibility.enable to each target but never disabled it,
so the renderer kept its accessibility tree live and reported changes for the
rest of the debugger attachment.

Once a target's AX trees are read, or the capture fails or is cancelled first,
disable the domain again on each target that enabled it. The cleanup shares
the capture's worker bound, settles before the capture does, only uses the
attachment that enabled AX, and is never queued behind a read still pending
past its deadline: such a target is skipped, and the read gate now refuses
Accessibility.disable behind a pending read.

Refs Tencent#391.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LZRCYsFEKFsAZe6i8WAc9o

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Control overlay repaints the viewport every frame, and observe leaves the Accessibility domain enabled (GPU/CPU during sessions)

1 participant