Skip to content

Conversation

@seanmcguire12
Copy link
Member

@seanmcguire12 seanmcguire12 commented Jul 17, 2025

why

  • incorrect/broken xpaths for nested iframes with siblings meant act often failed on websites where the HTML follows this pattern
  • getAccessibilityTreeWithFrames() populates an internal map: seg: Map<Frame | null, string>
  • for every iframe it visits, the map stores the frameXpath the XPath segment (/html/body/iframe[2], …) pointing from that iframe’s parent document to the iframe element itself
  • previously, the map key was the parent frame: seg.set(snapshot.parentFrame, snapshot.frameXpath);
  • if a website contains 2+ sibling iframes, the first siblings frameXpath would get overwritten/lost

what changed

  • keyed the seg map by the frame itself so that siblings are no longer overwritten

test plan

  • added an eval that has nested iframes with siblings

@changeset-bot
Copy link

changeset-bot bot commented Jul 17, 2025

🦋 Changeset detected

Latest commit: 7b5d562

The changes in this PR will be included in the next version bump.

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@seanmcguire12 seanmcguire12 marked this pull request as ready for review July 17, 2025 21:44
@seanmcguire12 seanmcguire12 added act These changes pertain to the act function extract These changes pertain to the extract function labels Jul 17, 2025
Copy link
Contributor

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Greptile Summary

This PR fixes a critical bug in the XPath generation system for nested iframes. The core issue was that when multiple sibling iframes existed on a page, the XPath mapping would incorrectly overwrite previous siblings' paths, causing act() operations to fail. This was due to using the parent frame as the key in the internal seg map.

The fix changes the seg map to be keyed by the frame itself rather than its parent frame, ensuring each iframe's XPath information is preserved independently. This is a robust solution that maintains the integrity of the XPath hierarchy while fixing the sibling iframe issue.

The changes include:

  • Modified lib/a11y/utils.ts to use frame instead of parentFrame as map key
  • Added a new required frame field to FrameSnapshot interface
  • Added a new eval test case 'nested_iframes_2' to validate the fix

Confidence score: 5/5

  1. This PR is very safe to merge as it fixes a clear bug without introducing new complexity
  2. The score is 5 because the change is well-isolated, includes tests, and fixes a clear logical error in the original implementation
  3. While all changes look good, pay special attention to lib/a11y/utils.ts as it contains the core fix

4 files reviewed, no comments
Edit PR Review Bot Settings | Greptile

@MrHaikuSwan
Copy link

@seanmcguire12 = 🐐

seanmcguire12 pushed a commit that referenced this pull request Jul 31, 2025
This PR was opened by the [Changesets
release](https://github.com/changesets/action) GitHub action. When
you're ready to do a release, you can merge this and the packages will
be published to npm automatically. If you're not ready to do a release
yet, that's fine, whenever you add more changesets to main, this PR will
be updated.


# Releases
## @browserbasehq/stagehand@2.4.2

### Patch Changes

- [#865](#865)
[`6b4e6e3`](6b4e6e3)
Thanks [@seanmcguire12](https://github.com/seanmcguire12)! - improve
type safety for trimTrailingTextNode

- [#897](#897)
[`e77d018`](e77d018)
Thanks [@miguelg719](https://github.com/miguelg719)! - Fix selfHeal to
remember intially received arguments

- [#920](#920)
[`c20adb9`](c20adb9)
Thanks [@seanmcguire12](https://github.com/seanmcguire12)! - fix: tab
handling on API

- [#882](#882)
[`b86df93`](b86df93)
Thanks [@seanmcguire12](https://github.com/seanmcguire12)! - remove
elements that don't have xpaths from observe response

- [#905](#905)
[`023c2c2`](023c2c2)
Thanks [@tkattkat](https://github.com/tkattkat)! - Delete old images
from anthropic cua client

- [#925](#925)
[`8c28647`](8c28647)
Thanks [@miguelg719](https://github.com/miguelg719)! - Remove
\_refreshPageFromApi()

- [#887](#887)
[`87e09c6`](87e09c6)
Thanks [@seanmcguire12](https://github.com/seanmcguire12)! - fix: allow
xpaths with prepended 'xpath=' for targeted extract

- [#864](#864)
[`a611115`](a611115)
Thanks [@miguelg719](https://github.com/miguelg719)! - Temporarily patch
custom clients serialization error on api

- [#881](#881)
[`69913fe`](69913fe)
Thanks [@miguelg719](https://github.com/miguelg719)! - Pass sdk version
number to API for debugging

- [#913](#913)
[`b1b83a1`](b1b83a1)
Thanks [@seanmcguire12](https://github.com/seanmcguire12)! - move iframe
out of 'experimental'

- [#891](#891)
[`be8497c`](be8497c)
Thanks [@seanmcguire12](https://github.com/seanmcguire12)! - fix: nested
iframe xpath bug

- [#883](#883)
[`98704c9`](98704c9)
Thanks [@seanmcguire12](https://github.com/seanmcguire12)! - add timeout
for JS click

- [#907](#907)
[`04978bd`](04978bd)
Thanks [@seanmcguire12](https://github.com/seanmcguire12)! - store
mapping of CDP frame ID -> page

## @browserbasehq/stagehand-evals@1.0.6

### Patch Changes

- Updated dependencies
\[[`6b4e6e3`](6b4e6e3),
[`e77d018`](e77d018),
[`c20adb9`](c20adb9),
[`b86df93`](b86df93),
[`023c2c2`](023c2c2),
[`8c28647`](8c28647),
[`87e09c6`](87e09c6),
[`a611115`](a611115),
[`69913fe`](69913fe),
[`b1b83a1`](b1b83a1),
[`be8497c`](be8497c),
[`98704c9`](98704c9),
[`04978bd`](04978bd)]:
    -   @browserbasehq/stagehand@2.4.2

## @browserbasehq/stagehand-examples@1.0.6

### Patch Changes

- Updated dependencies
\[[`6b4e6e3`](6b4e6e3),
[`e77d018`](e77d018),
[`c20adb9`](c20adb9),
[`b86df93`](b86df93),
[`023c2c2`](023c2c2),
[`8c28647`](8c28647),
[`87e09c6`](87e09c6),
[`a611115`](a611115),
[`69913fe`](69913fe),
[`b1b83a1`](b1b83a1),
[`be8497c`](be8497c),
[`98704c9`](98704c9),
[`04978bd`](04978bd)]:
    -   @browserbasehq/stagehand@2.4.2

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

act These changes pertain to the act function extract These changes pertain to the extract function

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants