Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Firefox e2e test with Cypress are failing #194

Closed
karolina-siemieniuk-morawska opened this issue Nov 17, 2022 · 1 comment
Closed

Firefox e2e test with Cypress are failing #194

karolina-siemieniuk-morawska opened this issue Nov 17, 2022 · 1 comment

Comments

@karolina-siemieniuk-morawska
Copy link
Collaborator

karolina-siemieniuk-morawska commented Nov 17, 2022

Continuation of: #85

Since the release of Firefox version 102 Cypress has trouble connecting to this browser during test. The issue appears to be common, as it was reported many times to Cypress:

cypress-io/cypress#22086
cypress-io/cypress#23215

Solution for this proposed by Cypress was to upgrade to latest version. This indeed resolved the problem with establishing connection, unfortunately created new problems.

  1. Snapshot tests are now failing on Github (they work fine locally) because of a differences between locally generated snapshots and those in CI. These are almost invisible, <1-2% differences in how fonts are rendered, but our failure threshold is 0.005.
    image
    The reason for this is most probably caused by the fact that every system renders fonts slightly differently, and locally snapshots were generated using MacOS, on github with Ubuntu 20.4.

jaredpalmer/cypress-image-snapshot#219 (comment)

  1. To recreate the same exact environment for tests locally that we have on CI, it's necessary to run Cypress tests on Docker. Unfortunately, docker image for cypress 11 does not have chrome and firefox browsers installed for arm64 architecture, which results with following error when trying to run tests:
Can't run because you've entered an invalid browser name.
Browser: 'firefox' was not found on your system or is not supported by Cypress.

https://github.com/cypress-io/cypress-docker-images/tree/master/included/11.0.1#:~:text=Note%3A%20Currently%2C%20the%20linux/arm64%20build%20of%20this%20image%20does%20not%20contain%20any%20browsers%20except%20Electron.%20See%20%23695%20for%20more%20information.
cypress-io/cypress-docker-images#695
cypress-io/cypress#15258

Additionally, passing the path to browser installed locally also results with following error:

We could not identify a known browser at the path you provided

cypress-io/cypress#4679

This makes it impossible to locally create snapshots that will match the ones on CI.

We need to find a solution for using Firefox in e2e tests.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant