Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Fixed flaky test for custom screen size #27067

Merged
merged 2 commits into from
Sep 17, 2024

Conversation

hjetpoluru
Copy link
Contributor

@hjetpoluru hjetpoluru commented Sep 11, 2024

Description

The fix for the flaky test is related to the custom screen size of the window during execution.

The flaky test was identified in multiple test scenarios with a similar pattern of false negatives in the automation tests. There were browser errors that appeared after navigating and opening the devTools, with the following error mentioned:

2024-08-22T20:14:57.507Z [driver] Called 'navigate' with arguments []

-----Received an error from Chrome-----
Request Storage.getStorageKeyForFrame failed. {"code":-32602,"message":"Frame tree node for given frame not found"}
Request Storage.getStorageKeyForFrame failed. {"code":-32602,"message":"Frame tree node for given frame not found"}
----------End of Chrome error----------

2024-08-22T20:15:04.239Z [driver] Called 'fill' with arguments ["#password","correct horse battery staple"]

And due to this error, the code mistakenly flags the test as failed and attempts to capture a screenshot, leading to Due to this error, the code mistakenly flags the test as failed and attempts to capture a screenshot, leading to another false error: NoSuchWindowError: no such window: target window already closed.

  • The test specific to responive UI requires a specific window size inorder to perform correctly and with help of @danjm suggestion I have include the constrainWindowSize: true which meets the test criteria.
  • Also I have noticed that the windows size is not required for the swap-send-test-utils.ts and hence removed it.

Open in GitHub Codespaces

Related issues

Fixes:
#26898
#26900
#26637
#26112
#24624

Manual testing steps

Run the below commands locally or in codespaces
Webpack build
yarn
yarn build:test:webpack
yarn build:test:webpack
ENABLE_MV3=false yarn test:e2e:single test/e2e/tests/swap-send/swap-send-erc20.spec.ts --browser=chrome
ENABLE_MV3=false yarn test:e2e:single test/e2e/tests/swap-send/swap-send-eth.spec.ts --browser=chrome
ENABLE_MV3=false yarn test:e2e:single test/e2e/tests/account/metamask-responsive-ui.spec.js --browser=chrome

Pre-merge author checklist

Pre-merge reviewer checklist

  • I've manually tested the PR (e.g. pull and build branch, run the app, test code being changed).
  • I confirm that this PR addresses all acceptance criteria described in the ticket it closes and includes the necessary testing evidence such as recordings and or screenshots.

@hjetpoluru hjetpoluru added team-extension-platform area-qa Relating to QA work (Quality Assurance) labels Sep 11, 2024
@hjetpoluru hjetpoluru self-assigned this Sep 11, 2024
Copy link
Contributor

CLA Signature Action: All authors have signed the CLA. You may need to manually re-run the blocking PR check if it doesn't pass in a few minutes.

Copy link

codecov bot commented Sep 11, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 70.11%. Comparing base (a61b938) to head (e4fbc69).

Additional details and impacted files
@@           Coverage Diff            @@
##           develop   #27067   +/-   ##
========================================
  Coverage    70.11%   70.11%           
========================================
  Files         1426     1426           
  Lines        49689    49689           
  Branches     13902    13902           
========================================
  Hits         34835    34835           
  Misses       14854    14854           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@hjetpoluru hjetpoluru marked this pull request as ready for review September 11, 2024 16:53
@hjetpoluru hjetpoluru requested a review from a team as a code owner September 11, 2024 16:53
@metamaskbot metamaskbot added the INVALID-PR-TEMPLATE PR's body doesn't match template label Sep 11, 2024
@@ -266,7 +266,6 @@ export const getSwapSendFixtures = (
) => {
const ETH_CONVERSION_RATE_USD = 3010;
return {
driverOptions: { responsive: true },
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The window size is not mandatory for the test, so I have removed it.

@@ -266,7 +266,6 @@ export const getSwapSendFixtures = (
) => {
const ETH_CONVERSION_RATE_USD = 3010;
return {
driverOptions: { responsive: true },
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would like to confirm with the original author before we delete this. @BZahory, was there a reason that this was originally written so that the dev tools would be open during the test run (here is where your PR first added this: d0a0037#diff-f2647a24cd0065e584933d0fdc32081d6f28f75b7d1123533b91e02109d885fcR244_

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@danjm Sure, I agree with you we should confirm. Just to add BZahory may not reply as he is no longer working with the Extension. I am tagging the team @MetaMask/metamask-assets here for faster confirmation and this is my observation according to the PR in the screenshot and the test scenario where the extension is just clicked and hence I am thinking the size of the window is not mandatory.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can't currently see how the responsive screen size was needed, so I think it is fine to remove for now.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks Dan.

Copy link

sonarcloud bot commented Sep 11, 2024

@metamaskbot
Copy link
Collaborator

Builds ready [e4fbc69]
Page Load Metrics (1786 ± 174 ms)
PlatformPageMetricMin (ms)Max (ms)Average (ms)StandardDeviation (ms)MarginOfError (ms)
ChromeHomefirstPaint149228261788362174
domContentLoaded144427771760365175
load148128281786363174
domInteractive2087392110
Bundle size diffs
  • background: 0 Bytes (0.00%)
  • ui: 0 Bytes (0.00%)
  • common: 0 Bytes (0.00%)

@hjetpoluru hjetpoluru merged commit c1a4386 into develop Sep 17, 2024
84 checks passed
@hjetpoluru hjetpoluru deleted the fix-flaky-test-custom-screen branch September 17, 2024 15:01
@github-actions github-actions bot locked and limited conversation to collaborators Sep 17, 2024
@metamaskbot metamaskbot added the release-12.6.0 Issue or pull request that will be included in release 12.6.0 label Sep 17, 2024
@metamaskbot metamaskbot added release-12.5.0 Issue or pull request that will be included in release 12.5.0 and removed release-12.6.0 Issue or pull request that will be included in release 12.6.0 labels Sep 29, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
area-qa Relating to QA work (Quality Assurance) INVALID-PR-TEMPLATE PR's body doesn't match template release-12.5.0 Issue or pull request that will be included in release 12.5.0 team-extension-platform
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

5 participants