Repository navigation
[feat]: add position to locator.click() - #3112
Open
antonvishal wants to merge 5 commits into
Open
antonvishal wants to merge 5 commits into
antonvishal wants to merge 5 commits into
Conversation
Click a point relative to the element's padding box, matching Playwright and MouseEvent.offsetX/Y, so canvas and map UIs can be targeted through a locator. Exposed in the protocol, TypeScript, Python, Go, and the Playwright facade. Co-authored-by: Cursor <cursoragent@cursor.com>
🦋 Changeset detectedLatest commit: 253a3cb The changes in this PR will be included in the next version bump. This PR includes changesets to release 25 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
Contributor
|
This PR is from an external contributor and must be approved by a stagehand team member with write access before CI can run. |
antonvishal
marked this pull request as ready for review
October 5, 2026 19:02
# Conflicts: # packages/sdk-go/internal/extensionassets/stagehand-extension.zip
# Conflicts: # packages/sdk-go/internal/extensionassets/stagehand-extension.zip
# Conflicts: # packages/sdk-go/internal/extensionassets/stagehand-extension.zip
# Conflicts: # packages/sdk-go/internal/extensionassets/stagehand-extension.zip
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
why
I was navigating across https://teleoperator.mindblown.ai/?mb=game and needed to click specific points on a canvas map. Stagehand's
locator.click()could only click an element's center, and passing Playwright'spositionoption was rejected by the protocol schema, so agents written against the Playwright facade failed there.what changed
locator.click({ position: { x, y } })clicks at a point measured from the top-left of the element's padding box. That matches Playwright andMouseEvent.offsetX/Y, so{x: 0, y: 0}lands exactly on the padding edge even when the element has padding and a border.LocatorClickPosition), the extension locator and frame/deep locator delegates, and the Playwright facade (locator.clickand frame locators, with input validation). The generated Go and Python models are regenerated, Python'sLocator.clickgainsposition=, and the docs are updated.force: truecombined withpositiondispatches a real click at the point rather than the DOM-click shortcut.test plan
clickCount.test.tsintegration test: page-reported offsets equal the requested position for a padded and bordered element ({0,0}and{37,19}) and for a rotated element. The old protocol schema rejects the option.facade-frames.test.ts: positioned clicks land exactly on canvases in the main frame and in same-origin and out-of-process iframes.click(position=...), and GoLocatorClickOptions.Position.Summary by cubic
Adds a
positionoption tolocator.click()so a click can target a point relative to the element's padding box instead of only its center, matching Playwright so Playwright-facade agents can interact with canvas and map UIs.MouseEvent.offsetX/Y;{0,0}lands on the padding edge even with padding and borders, and the point follows rotated or transformed elements.force: truecombined withpositiondispatches a real click at the point.positionto the Python and Go SDKs, regenerates generated models, and updates the docs.Written for commit 253a3cb. Summary will update on new commits.