-
Notifications
You must be signed in to change notification settings - Fork 4.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix: Fixed flaky test for custom screen size #27067
Conversation
CLA Signature Action: All authors have signed the CLA. You may need to manually re-run the blocking PR check if it doesn't pass in a few minutes. |
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## develop #27067 +/- ##
========================================
Coverage 70.11% 70.11%
========================================
Files 1426 1426
Lines 49689 49689
Branches 13902 13902
========================================
Hits 34835 34835
Misses 14854 14854 ☔ View full report in Codecov by Sentry. |
@@ -266,7 +266,6 @@ export const getSwapSendFixtures = ( | |||
) => { | |||
const ETH_CONVERSION_RATE_USD = 3010; | |||
return { | |||
driverOptions: { responsive: true }, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The window size is not mandatory for the test, so I have removed it.
@@ -266,7 +266,6 @@ export const getSwapSendFixtures = ( | |||
) => { | |||
const ETH_CONVERSION_RATE_USD = 3010; | |||
return { | |||
driverOptions: { responsive: true }, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would like to confirm with the original author before we delete this. @BZahory, was there a reason that this was originally written so that the dev tools would be open during the test run (here is where your PR first added this: d0a0037#diff-f2647a24cd0065e584933d0fdc32081d6f28f75b7d1123533b91e02109d885fcR244_
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@danjm Sure, I agree with you we should confirm. Just to add BZahory may not reply as he is no longer working with the Extension. I am tagging the team @MetaMask/metamask-assets here for faster confirmation and this is my observation according to the PR in the screenshot and the test scenario where the extension is just clicked and hence I am thinking the size of the window is not mandatory.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I can't currently see how the responsive screen size was needed, so I think it is fine to remove for now.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks Dan.
Quality Gate passedIssues Measures |
Builds ready [e4fbc69]
Page Load Metrics (1786 ± 174 ms)
Bundle size diffs
|
Description
The fix for the flaky test is related to the custom screen size of the window during execution.
The flaky test was identified in multiple test scenarios with a similar pattern of false negatives in the automation tests. There were browser errors that appeared after navigating and opening the devTools, with the following error mentioned:
And due to this error, the code mistakenly flags the test as failed and attempts to capture a screenshot, leading to Due to this error, the code mistakenly flags the test as failed and attempts to capture a screenshot, leading to another false
error: NoSuchWindowError: no such window: target window already closed.
constrainWindowSize: true
which meets the test criteria.swap-send-test-utils.ts
and hence removed it.Related issues
Fixes:
#26898
#26900
#26637
#26112
#24624
Manual testing steps
Run the below commands locally or in codespaces
Webpack build
yarn
yarn build:test:webpack
yarn build:test:webpack
ENABLE_MV3=false yarn test:e2e:single test/e2e/tests/swap-send/swap-send-erc20.spec.ts --browser=chrome
ENABLE_MV3=false yarn test:e2e:single test/e2e/tests/swap-send/swap-send-eth.spec.ts --browser=chrome
ENABLE_MV3=false yarn test:e2e:single test/e2e/tests/account/metamask-responsive-ui.spec.js --browser=chrome
Pre-merge author checklist
Pre-merge reviewer checklist