Add E2E tests setup for Auto Sizes

[mukeshpanchal27]

Summary

Part of #1511

Relevant technical choices

This PR adds the basic infrastructure for E2E testing for the Auto Sizes plugin.

More E2E test coverage will be added over time once this is merged into the feature branch.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 72.48%. Comparing base (2f01c04) to head (775ce25).
Report is 301 commits behind head on feature/1511-incorporate-layout-constraints-from-ancestors.

Additional details and impacted files

@@                                      Coverage Diff                                       @@
##           feature/1511-incorporate-layout-constraints-from-ancestors    #1988      +/-   ##
==============================================================================================
+ Coverage                                                       69.60%   72.48%   +2.87%     
==============================================================================================
  Files                                                              86       85       -1     
  Lines                                                            7018     7051      +33     
==============================================================================================
+ Hits                                                             4885     5111     +226     
+ Misses                                                           2133     1940     -193     

Flag	Coverage Δ
multisite	72.48% <ø> (+2.87%)	⬆️
single	40.43% <ø> (+0.75%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[mukeshpanchal27]

In b4758e6, I tried adding E2E tests to verify the image src before and after the plugin deactivation/activation.

When I run npm run test-e2e, it shows the same image URL (no change), but when I run npm run test-e2e:debug, it shows the expected smaller version image URL in debug mode.

Does anyone know why this might be happening or what could be causing the difference in behavior between normal and debug modes? cc. @joemcgill @felixarntz

[joemcgill]

@mukeshpanchal27 this is a tricky thing to try to test in a Playwright test. The src attribute of the image will not change when the plugin is activated. Instead, the sizes attribute is updated, which is what the browser uses to choose an image file from the srcset list to render on the image, which will be set as that image's currentSrc property.

I'm not entirely sure if the headless browser Playwright uses will process srcset and sizes the same way a real browser will, so it may be best to just test that the sizes attribute has been updated correctly in the test.

[github-actions]

The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs. You can also manually ask me to refresh this list by adding the props-bot label.

If you're merging code through a pull request on GitHub, copy and paste the following into the bottom of the merge commit message.

Co-authored-by: mukeshpanchal27 <mukesh27@git.wordpress.org>
Co-authored-by: swissspidy <swissspidy@git.wordpress.org>
Co-authored-by: joemcgill <joemcgill@git.wordpress.org>

To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook.

[swissspidy]

wp-scripts already has a good global-setup config. Why do we need a custom one here?

[swissspidy]

Why do we need a custom config over what's provided by wp-scripts? IMO the default one should suffice for our needs.

[swissspidy]

What if we ever want to have e2e tests in other plugins too? It would be weird to copy these config files everywhere, so I think it would be better to move them to tools/ or so. But then again see my other comments about their purpose.

[swissspidy]

It's typically better to use locators, for example getByAltText, or alternatively CSS locators. But not waitFor*.

[swissspidy]

This doesn't do anything, this isn't an assertion that fails the test.

Ideally we use something like await expect(locator).toHaveAttribute('src', /leaves-768x512\.jpeg$/ );