Skip to content

fix(extension): count XPath positions among the siblings of each parent - #3096

Open
mikhail-koviazin wants to merge 2 commits into
browserbase:mainfrom
mikhail-koviazin:fix/xpath-positional-predicates
Open

mikhail-koviazin wants to merge 2 commits into
browserbase:mainfrom
mikhail-koviazin:fix/xpath-positional-predicates

Conversation

@mikhail-koviazin

@mikhail-koviazin mikhail-koviazin commented Oct 3, 2026 •

Copy link
Copy Markdown
Contributor

why

Part of #2693, the two positional items from its "Where it comes from" list.

When the document contains a shadow root, XPath locators go through the composed-tree parser instead of document.evaluate(). For a // step that parser collects every matching descendant into one list and then applies [n] to the list, so //div[2] is evaluated as (//div)[2]. In XPath the position counts along the child axis: //div[2] is every div that is the second div child of its parent.

Separately, [0] is clamped to 1 (Math.max(1, Number(input))), so a predicate that selects nothing in XPath returns the first match.

Measured on a build of main (50f3ab81), fixture served over http. Without the shadow root every row agrees with native. With one unrelated attachShadow() call on the page:

XPath document.evaluate() Stagehand
//div[2] 3 1
//div[1] 3 1
//section//div[2] 3 2
//div[@class='x'][2] 1, B2 1, the nested wrapper div
//div[0] 0 1, A1
//section[@id='s1']/div[2] 1 1 (control: child steps are already correct)

Fixture:

<section id="s1"><div>A1</div><div>A2</div><div class="x">A3</div></section>
<section id="s2"><div class="x"><div>C1</div><div>C2</div></div><div class="x">B2</div></section>

The [@class='x'][2] row is the one I'd worry about. The count is 1 on both sides, so nothing looks off, but it is a different element, and a click lands on it without an error.

what changed

  • applyPredicatesPerParent in xpathParser.ts groups the candidates of a step by parentNode, applies the predicates inside each group and returns the survivors in the order they came in. Steps without a positional predicate skip the grouping.
  • xpathResolver.ts uses it for // steps in both resolveXPathComposedMatches and resolveStagehandShadowHopMatches. Child steps keep calling applyPredicates: their pool is already the children of one node.
  • [0] is parsed as written and matches nothing.

Not in this PR: position(), last() and (//div)[n] from the same issue are still unsupported.

test plan

packages/extension/tests/xpath-positional-predicates.test.ts (new, unit): per-parent counting, a position after another predicate and before it, [0], and that order is kept when candidates from different parents are interleaved.

packages/sdk-ts/tests/integration/locatorXPathPositionalPredicates.test.ts (new, integration): the table above against real Chrome, plus //em[2] inside the shadow root to check that piercing still works. Served over http, because data: URLs stay on the native engine and never reach this parser. Registered in scripts/test-integration.ts.

On main the integration test fails with expected 1 to be 3 and with the wrapper div's HTML where B2 is expected. With the change it passes, and so do locatorXPathTextPredicates, locatorCount, locatorNth and waitForSelector, each run as its own file.


Summary by cubic

Fixes XPath positional predicates in the composed-tree parser so //div[2] counts among the siblings of each parent, matching document.evaluate(), and [0] now selects nothing instead of clamping to the first match.

  • Applies positional predicates per parent only for descendant steps; child steps keep existing behavior.
  • Adds unit and integration coverage, including a shadow-root case to verify piercing still works.

Written for commit e5510e5. Summary will update on new commits.

Review in cubic

The composed-tree parser runs whenever the document contains a shadow root, and for a
descendant step it applied a positional predicate to the flat list of every match, so
`//div[2]` behaved like `(//div)[2]`. It returned one element where document.evaluate()
returns one per parent, and with a preceding predicate it returned the wrong element.

- a descendant step now filters candidates within their own sibling group and keeps
  document order; child steps already pooled the children of a single parent
- the same grouping applies to the first step of the shadow-hop resolver
- regression coverage runs over http, since data: URLs stay on the native engine
XPath positions start at 1, so `//div[0]` selects nothing. The composed-tree parser
clamped the index to 1 and returned the first match instead.

- `[0]` is now parsed as written and yields an empty set, as document.evaluate() does
@changeset-bot

changeset-bot Bot commented Oct 3, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: e5510e5

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 21 packages
Name Type
@browserbasehq/stagehand-extension Patch
@browserbasehq/stagehand-go Patch
@browserbasehq/stagehand Patch
@browserbasehq/stagehand-python Patch
browse Patch
@browserbasehq/stagehand-examples Patch
@browserbasehq/stagehand-integrations Patch
@browserbasehq/eve Patch
@browserbasehq/stagehand-integrations-example-pi-facade Patch
@browserbasehq/stagehand-integrations-claude-agent-sdk Patch
@browserbasehq/stagehand-integrations-example-claude-code-facade Patch
@browserbasehq/stagehand-integrations-codex-sdk Patch
@browserbasehq/stagehand-integrations-example-codex-facade Patch
@browserbasehq/stagehand-integrations-cursor-sdk Patch
@browserbasehq/stagehand-integrations-deepagents-sdk Patch
@browserbasehq/stagehand-integrations-eve-sdk Patch
@browserbasehq/stagehand-integrations-fx-sdk Patch
@browserbasehq/stagehand-integrations-mastra-sdk Patch
@browserbasehq/stagehand-integrations-example-mastra-facade Patch
@browserbasehq/stagehand-integrations-pi-sdk Patch
@browserbasehq/stagehand-integrations-example-vercel-ai-facade Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@github-actions

github-actions Bot commented Oct 3, 2026

Copy link
Copy Markdown
Contributor

This PR is from an external contributor and must be approved by a stagehand team member with write access before CI can run.
Approving the latest commit mirrors it into an internal PR owned by the approver.
If new commits are pushed later, the internal PR stays open but is marked stale until someone approves the latest external commit and refreshes it.

@github-actions github-actions Bot added external-contributor Tracks PRs mirrored from external contributor forks. external-contributor:awaiting-approval Waiting for a stagehand team member to approve the latest external commit. labels Oct 3, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

external-contributor:awaiting-approval Waiting for a stagehand team member to approve the latest external commit. external-contributor Tracks PRs mirrored from external contributor forks.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant