Removes Percy based visual regression tests #139252
Conversation
tylersmalley
force-pushed
the
rm-percy
branch
2 times, most recently
from
6d09cf0 to
c46799f
Compare
tylersmalley
changed the title
|
Pinging @elastic/kibana-operations (Team:Operations) |
tylersmalley
added
the
release_note:skip
|
Less code is the best code 😉 |
|
Holding off on this until a meeting we're having on Monday, August 29th. |
There was a problem hiding this comment.
I'm fine with removing this now based on the small number of tests and responses from the teams that own the features of those tests. I believe the risk of missing a bug that no other automated tests or humans would find without these tests to be pretty low.
We can still review a future PR that uses the latest Percy API if someone wants to create that.
There was a problem hiding this comment.
From our meeting last week, we agreed to remove the deprecated Percy package and existing tests from mainline. A new evaluation of Percy testing will be done per comment: #138629 (comment). If the evaluation goes well, I will put up another PR to add back a new set of visual regression tests.
liza-mae
added
v8.5.0
v8.4.0
v7.17.7
v8.4.2
auto-backport
When Percy was chosen, we targeted a specific workflow that allowed these tests to run in the context of a PR and for developers to see and address failures much like any other test. That workflow ended up not being possible therefore, the Percy project was abandoned. QA has maintained running the handful (14) of existing tests during releases. While that has been a somewhat frictionless process, the Percy dependency is deprecated requiring many needless dependencies to be downloaded and installed for development environments. All teams have [confirmed](elastic#138629 (comment)) that they do not rely on nor need these tests, so I feel it's best to remove this code and focus our efforts elsewhere. Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
tylersmalley
force-pushed
the
rm-percy
branch
from
c46799f to
b9fe9ff
Compare
💚 Build Succeeded
Metrics [docs]
History
To update your PR or re-run it, just comment with: |
💔 All backports failed
Manual backportTo create the backport manually run: Questions ?Please refer to the Backport tool documentation and see the Github Action logs for details |
This PR is a recreation of #139252 applied to 7.17 Just confirmed with @dmlemeshko that we don't need to run percy anymore also on 7.17. \cc @legrego --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Reapplies #136359 reverted in #136437
When Percy was chosen for visual regression testing, we targeted a specific workflow that allowed these tests to run in the context of a PR and for developers to see and address failures much like any other test. That workflow was not possible; therefore, the Percy project was largely abandoned.
For three years now, these tests have not been run as part of the Kibana CI job. QA has maintained running the handful (14) of existing tests during releases. While that has been a somewhat frictionless process, the Percy dependency is deprecated, requiring many needless dependencies to be downloaded and installed for development environments.
All teams have confirmed that they do not rely on nor need these tests. With all that said, we should remove this code. A separate discussion can be started if visual testing is necessary and a priority.
closes #138629
closes #43127
closes #54149