feat(dashboard): live annotation toggle in viewport toolbar#1374
Open
techcomthanh wants to merge 2 commits into
Open
feat(dashboard): live annotation toggle in viewport toolbar#1374techcomthanh wants to merge 2 commits into
techcomthanh wants to merge 2 commits into
Conversation
Adds a `#` toggle button to the viewport toolbar (left of the snapshot icon) that injects the same numbered red boxes used by `screenshot --annotate` onto the live page, so element refs are visible in the streaming viewport without taking a screenshot. CLI equivalent: agent-browser --session <name> annotate-overlay on agent-browser --session <name> annotate-overlay off Numbers shown on the live overlay match `@eN` from `snapshot -i`. Rust side: - screenshot.rs: extract a public `set_live_annotation` that reuses the existing `inject_annotation_overlay` / `remove_annotation_overlay` helpers. - actions.rs: new `annotate_overlay` action handler. On enable it refreshes the ref map via a fresh snapshot so overlay numbers match the current DOM. - commands.rs: new `annotate-overlay on|off` CLI verb. Dashboard side: - viewport.tsx: new toolbar toggle button (lucide `Hash`) with a local `annotateOn` state; auto re-applies the overlay 300ms after URL changes so labels track navigation.
Contributor
|
@techcomthanh is attempting to deploy a commit to the Vercel Labs Team on Vercel. A member of the Team first needs to authorize it. |
![vercel[bot]](https://avatars.githubusercontent.com/in/8329?s=60&v=4){kind=small}
The local `annotateOn` state was scoped to the Viewport component, not to the session. Switching sessions kept the toggle's previous state but the new session has no overlay injected (and vice versa), so the UI was out of sync with reality. Reset to `false` whenever `sessionName` changes — the user re-enables explicitly per session, which matches the actual overlay state on the daemon side. Addresses PR vercel-labs#1374 review feedback.
Author
|
Addressed in 259430c: reset |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a
#toggle in the dashboard viewport toolbar (left of the snapshot icon) that injects the same numbered red boxes used byscreenshot --annotateonto the live page, so element refs are visible in the streaming viewport without taking a screenshot.Motivation: when picking elements interactively from the dashboard,
--annotateonly labels stills. The labels disappear the moment streaming continues, which is awkward for human-in-the-loop selection. This toggle reuses the existing overlay injector so labels stay on screen while the user identifies the target, then can be removed in one click.CLI equivalent (also new):
The numbers shown on the live overlay match
@eNfromsnapshot -i— the action takes a fresh snapshot on enable so the ref map is current.Changes
Rust (
cli/)native/screenshot.rs: extract a publicset_live_annotation(client, session_id, ref_map, on)that reuses the existinginject_annotation_overlay/remove_annotation_overlayhelpers (no new injection code).native/actions.rs: newannotate_overlayaction. On enable, clears + refills the ref map viasnapshot::take_snapshotso overlay numbers match the current DOM.commands.rs: newannotate-overlay on|offCLI verb.Dashboard (
packages/dashboard/)components/viewport.tsx: newHashicon toggle button placed left of the snapshot button; localannotateOnstate; callsrunCmd(\"annotate-overlay\", on?\"on\":\"off\"). A 300ms-debounceduseEffectonurlre-applies the overlay after navigation so labels track the new DOM.Test plan
cargo build --release --manifest-path cli/Cargo.toml— clean build.pnpm --filter dashboard build— clean build.agent-browser open https://example.com→annotate-overlay on→ DOM contains#__agent_browser_annotations__overlay; viewer shows numbered boxes.offremoves them. Verified viaagent-browser evalpost-toggle.screenshot --annotate).Notes / Open questions
setTimeoutonurlchange). SPA route changes that don't updateactiveUrlAtomwon't trigger it; users re-click the toggle. A CDPPage.frameNavigatedhook on the server side would be more robust — happy to follow up if preferred.Hashfromlucide-react) is a guess; open to alternatives.