Skip to content

fix(screenshot): draw box overlays on the image, and let the frame scale - #25

Merged
abrichr merged 1 commit into
mainfrom
fix/inspector-embed
Oct 14, 2026
Merged

abrichr merged 1 commit into
mainfrom
fix/inspector-embed

Conversation

@abrichr

@abrichr abrichr commented Oct 14, 2026

Copy link
Copy Markdown
Member

screenshot_display drew box overlays in the wrong place and without a border, so a box was invisible and, once you could see it, off target. This PR draws boxes on the image itself and lets a screenshot scale down to a phone-width page.

The bug

Overlays were positioned against the container, not the image. The default container is 800x450, so a 1280x800 screenshot is pillarboxed to 720x450 inside it, and every overlay fraction was a fraction of the wrong box. Box overlays also inherited the click marker's translate(-50%, -50%), and core.css had no .oa-overlay-box rule at all.

Measured in Chromium on origin/main, a box at x=0.25, y=0.5, width=0.25 on a 1280x800 frame:

x y width border
Asked for 0.25 0.5 0.25
origin/main 0.083 0.4375 0.278 0px none
This branch 0.25 0.5 0.25 2px solid

The fix

  • The image and its overlays now share one positioned frame that shrink-wraps the rendered image, so overlay fractions are fractions of the picture whether it's letterboxed or not. Click markers benefit too.
  • .oa-overlay-box gets a border, no transform, and a label that sits above the box.
  • width=None fills the parent and height=None lets the height follow the image. With both set to None, a screenshot scales down on a 390 px page with no sideways scroll.
  • Caption, label, alt text, placeholder and path are HTML-escaped. There's a new alt argument, and the documented color overlay key is now applied.
  • docs/COMPONENTS.md shows the new signature.

The defaults (width=800, height=450) still produce a fixed-size container, so existing callers keep their layout.

Why now

I used PageBuilder(include_alpine=False) and screenshot_display to build a self-contained, step-by-step run inspector for the website: the engine's own screenshot for each step, with a box on the target the engine resolved. Every box landed off target, and at 390 px the fixed-size container overflowed the page. With this branch, the boxes land within 0.4% of the engine's region at 390 px and 1280 px. The inspector itself lives outside this repository.

Tests

  • tests/test_components/test_screenshot.py: 9 new string checks for the markup (overlays inside the image frame, fluid mode, the pixel cap on a fixed container, escaping, color) and for the stylesheet rules.
  • tests/test_screenshot_overlay_browser.py: renders a page and measures the box and click marker against the rendered image in Chromium, letterboxed at 800x450 and fluid at 390 px. It skips when Chromium isn't installed, as on CI today. Against origin/main it fails with x=0.083.

How I checked it:

  • uv run ruff check . passes.
  • uv run pytest tests/: 170 passed, 79 skipped (159 passed before; the 11 new tests pass, including both browser tests with Chromium installed locally).

This doesn't touch the files in #24 or #23, so it merges in either order.

🤖 Generated with Claude Code

screenshot_display placed overlays against its container, not the image.
With a 1280x800 frame in the default 800x450 container the image is
pillarboxed to 720x450, so a box at x=0.25 landed at x=0.083 of the
picture. Box overlays also inherited the click marker's centering
transform, and core.css had no .oa-overlay-box rule, so a box rendered
with no border at all: invisible, and in the wrong place.

- The image and its overlays now share a positioned frame that
  shrink-wraps the rendered image, so overlay fractions are fractions of
  the picture in fixed and fluid layouts.
- .oa-overlay-box has a border, no transform, and a label above it.
- width=None fills the parent and height=None lets the height follow the
  image, so a screenshot scales down on a 390 px page without a sideways
  scroll.
- Caption, label, alt, placeholder and path are HTML-escaped. New alt
  argument. The documented overlay color key is applied.

Tests: string checks for the markup and stylesheet, and a Chromium test
that measures the box and click marker against the rendered image
(letterboxed at 800x450 and fluid at 390 px). It skips when Chromium is
not installed. On origin/main the browser test measures x=0.083.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown
Contributor

📸 Generated Screenshots Preview

Screenshots have been generated. Download the artifacts to preview them.

Generated Files:

  • demo_new_controls.png (0.10 MB)
  • demo_new_events.png (0.09 MB)
  • demo_new_full.png (0.12 MB)
  • demo_viewer.png (0.29 MB)
  • turn-off-nightshift_controls.png (0.10 MB)
  • turn-off-nightshift_events.png (0.10 MB)
  • turn-off-nightshift_full.png (0.12 MB)

🔗 Download screenshots artifact

@abrichr
abrichr merged commit 2759149 into main Oct 14, 2026
5 checks passed
@abrichr
abrichr deleted the fix/inspector-embed branch October 14, 2026 08:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant