Snapshot is unusable with Power Apps/Dynamics 365

[rajyraman]

When I take a Snapshot using the tools only IFrames are captured.

Page

Snapshot

Even if I force it to take a snapshot of a specific element, I don't see any change.

[pavelfeldman]

I'm seeing iframe contents just fine. Note that zero-size iframes are not populated by design.

[rajyraman]

@pavelfeldman - When I try to get the selectors from VSCode manually Playwright can see all the elements, but inside MCP it just sees the IFrames.

Left hand navigation

Main grid

I tried with google.com to see it it behaves like this.

It does not. So, I not sure what is special about Dynamics 365/Power Apps that Playwright fails to discover all the elements.

[gbanne12]

I'm seeing iframe contents just fine. Note that zero-size iframes are not populated by design.

Just to add to the original description, the issue is not that one or more expected iframes are not present in the generated snapshot but rather that they seem to be the only thing the model can see from the given output.

Example output when running the page snapshot via the MCP is below:

• Ran Playwright code:

// <internal code to capture accessibility snapshot>

• Page URL: https://org9e533c5d.crm4.dynamics.com/main.aspx?appid=[private-id]&pagetype=entitylist&etn=account&viewid=00000000-0000-0000-00aa-000010001002&viewType=1039
• Page Title: Accounts Active Accounts -
• Page Snapshot

- generic [ref=e1]:
  - iframe

In contrast, if I run the playwright codegen and select to 'Assert snapshot' on a specific element in the tree on the same page as above the generated snapshot has all the expected page content. The iframe is omitted as indicated above that it should be if it has zero size and the rest of the page content is generated.

await expect(page.locator('#dataSetRoot_0')).toMatchAriaSnapshot(`
- button "Press Enter to go back." [disabled]
- separator
- menubar "Account Commands":
- menuitem
- menuitem "New"
- menuitem "Delete Delete More Commands. Delete":
- button "Delete"
- separator
- button "Delete More Commands. Delete"
- menuitem "Refresh"
- menuitem "Visualize this view"
- menuitem "Email a Link Email a Link More Commands. Email a Link":
- button "Email a Link"
- separator
- button "Email a Link More Commands. Email a Link"
- menuitem "Flow. Run Flow"
- menuitem "More commands for Account"
- button "Share"
- heading "Active Accounts" [level=1]:
- button "Active Accounts": Active Accounts Open popup to change view.
- button "Edit columns": Edit columns Edit the view by adding, removing or re-ordering columns.
- button "Edit filters": Edit filters Edit the view with advanced filter queries.
- searchbox "Account Filter by keyword"
- treegrid "Active Accounts":
- rowgroup:
- 'row "Toggle selection of all rows Account Name Main Phone Address 1: City Primary Contact Email (Primary Contact)"':
- columnheader "Toggle selection of all rows":
- checkbox "Toggle selection of all rows"
- columnheader "Account Name":
- button "Account Name"
- columnheader "Main Phone":
- button "Main Phone"
- 'columnheader "Address 1: City"':
- 'button "Address 1: City"'
- columnheader "Primary Contact":
- button "Primary Contact"
- columnheader "Email (Primary Contact)":
- button "Email (Primary Contact)"
- rowgroup:
- row "Press SPACE to select this row.":
- gridcell "select or deselect the row":
- checkbox "select or deselect the row"
- gridcell
- gridcell "Empty phone number":
- link "Empty phone number":
- /url: "tel:"
- gridcell
- gridcell
- gridcell
- row "Press SPACE to select this row.":
- gridcell "select or deselect the row":
- checkbox "select or deselect the row"
- gridcell
- gridcell "Empty phone number":
- link "Empty phone number":
- /url: "tel:"
- gridcell
- gridcell
- gridcell
- row "Press SPACE to select this row.":
- gridcell "select or deselect the row":
- checkbox "select or deselect the row"
- gridcell
- gridcell "Empty phone number":
- link "Empty phone number":
- /url: "tel:"

.......

Apologies if I have missed it but I don't see any way to generate a snapshot without specifying some form of locator. This appears to be slightly different from it happening at a whole page level via the MCP.

[Skn0tt]

In my testing, it can see the iframe contents just fine on the PowerApps Preview screen. @rajyraman @gbanne12 if it's public, could you share a link to the page where you're experiencing this? Or if it's not, maybe there's a different page on the interwebs that shows the same behaviour, so we can use that for debugging?

[gbanne12]

Are you able to access the trial environments?
https://www.microsoft.com/en-gb/dynamics-365/free-trial

I tried with the Dynamics 365 Sales Demo site via the above link and it shows the behaviour.

If not, I'll need to go and have a think about how we could share it with you. Not currently aware of any that are publicly available.

[Skn0tt]

That worked, thanks! I can repro and will take a look what's going on.

[Skn0tt]

It looks like this line is returning null when we try to resolve the iframe element:

https://github.com/microsoft/playwright/blob/d1eb9589f88fa17fe42531c082242918eea0e270/packages/playwright-core/src/server/frameSelectors.ts#L164

Not sure what that means though. I'll try reproducing it outside of PowerApps.

[Skn0tt]

@gbanne12 @rajyraman i'm able to reproduce it outside of Playwright MCP. It only happens when I connect via CDP to a browser that already has the PowerApps page open, it doesn't happen when I make Playwright load the PowerApps page. Does that mirror what you're doing?

So is your MCP server connecting via CDP, and then do you instruct your agent to take a snapshot of the already opened page? Or are you asking it to browser_navigate there in before?

[Skn0tt]

Here's the closest I got to a repro:

it('repro', async ({ playwright }) => {
  const browser = await playwright.chromium.connectOverCDP('http://localhost:5123'); // CDP connection to my browser where i'm logged in and have a PowerApps app open
  const page = browser.contexts()[0].pages()[0];
  expect(page.url()).toContain('https://apps.powerapps.com/play/');
  await page.reload(); // test fails without this line
  await expect(page.frameLocator('#fullscreen-app-host').locator('#publishedCanvas')).toBeVisible();
});

Still trying to repro it outside of PowerApps.

[gbanne12]

@gbanne12 @rajyraman i'm able to reproduce it outside of Playwright MCP. It only happens when I connect via CDP to a browser that already has the PowerApps page open, it doesn't happen when I make Playwright load the PowerApps page. Does that mirror what you're doing?

So is your MCP server connecting via CDP, and then do you instruct your agent to take a snapshot of the already opened page? Or are you asking it to browser_navigate there in before?

@Skn0tt Hey, I have been asking the agent to navigate to the page itself and then perform the snapshot and haven't had any joy from that method.

[Skn0tt]

Just to clarify: You're using --cdp-endpoint, and you originally didn't navigate to the page itself, and now you're trying that and it shows the exact same behaviour? Or is it broken in a different way now?

[rajyraman]

@Skn0tt - I just loaded the context with the saved state (mainly for auth) and navigated to the home page of a model driven app. When I checked the tool output, I could not see any elements apart from empty iframes. So, I didn't proceed any further.

I used both --isolated true and --storage-state for the session state.

[gbanne12]

Sorry, for the confusion. I am not specifying the --cdp-endpoint flag and never have been. If this is what you meant above I misunderstood that.

My general approach has been to ask the agent to perform all the actions. i.e. open browser, navigate to the app, generate snapshot. However, I have also tried to handling navigating to the page myself before asking for the snapshot to be generate and I see the same result.

[Skn0tt]

Thanks for the clarification. If neither of you are using --cdp-endpoint, them i'm looking at the bug from the wrong angle.