Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ftr] add first-class support for playwrite journeys #140680

Merged
merged 22 commits into from
Sep 22, 2022

Conversation

spalger
Copy link
Contributor

@spalger spalger commented Sep 14, 2022

Fixes #140676

This PR enables the FTR to load a custom "journey" format config file (example). This config file format is a big iteration on the FTR config file format, but using the @kbn/journeys package can be converted into the components necessary for journeys to behave like standard FTR configs, meaning that journeys can be defined cleanly and in a focused way without making major changes to the FTR or creating a new test runner.

For a comprehensive example of how journeys look in this new format, checkout x-pack/performance/journeys/web_logs_dashboard.ts.

In addition to improving the ergonomics of journeys this PR ensures that failed steps trigger failures, something that wasn't happening before. This requires that journeys are very stable, so the stability of existing journeys was improved and debugging was improved in this PR by adding screenshots taken at the end of each step to failure reports and ensuring that these reports are uploaded in hourly SUPT jobs (example)

Verified changes to performance pipeline in https://buildkite.com/elastic/kibana-pull-request/builds/72898

@spalger spalger force-pushed the implement/ftr-journey-loading branch 25 times, most recently from 0ba6e73 to 1161887 Compare September 16, 2022 06:26
@spalger spalger added Team:Operations Team label for Operations Team release_note:skip Skip the PR/issue when compiling release notes labels Sep 16, 2022
@spalger spalger marked this pull request as ready for review September 16, 2022 17:37
@spalger spalger requested review from a team as code owners September 16, 2022 17:37
@spalger
Copy link
Contributor Author

spalger commented Sep 21, 2022

Sorry about that @dmlemeshko, my migration of the config file to not need the removed base.config.js file wasn't good, looks like I've got it working now though.

@spalger spalger requested a review from dmlemeshko September 21, 2022 21:45
@dmlemeshko
Copy link
Member

Thank you @spalger, I retested and scalability run works perfectly now

@spalger spalger enabled auto-merge (squash) September 22, 2022 07:05
@kibana-ci
Copy link
Collaborator

💚 Build Succeeded

Metrics [docs]

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id before after diff
@kbn/dev-cli-runner 64 85 +21
@kbn/ftr-common-functional-services - 28 +28
@kbn/journeys - 57 +57
@kbn/test 213 217 +4
total +110

Any counts in public APIs

Total count of every any typed public API. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats any for more detailed information.

id before after diff
@kbn/test 5 4 -1

Public APIs missing exports

Total count of every type that is part of your API that should be exported but is not. This will cause broken links in the API documentation system. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats exports for more detailed information.

id before after diff
@kbn/ftr-common-functional-services - 2 +2
@kbn/journeys - 5 +5
total +7
Unknown metric groups

API count

id before after diff
@kbn/dev-cli-runner 65 101 +36
@kbn/ftr-common-functional-services - 28 +28
@kbn/journeys - 62 +62
@kbn/test 254 261 +7
total +133

ESLint disabled in files

id before after diff
@kbn/failed-test-reporter-cli - 1 +1
@kbn/test 3 1 -2
total -1

ESLint disabled line counts

id before after diff
@kbn/journeys - 1 +1

Total ESLint disabled count

id before after diff
@kbn/failed-test-reporter-cli - 1 +1
@kbn/journeys - 1 +1
@kbn/test 10 8 -2
total -0

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport:skip This commit does not require backporting release_note:skip Skip the PR/issue when compiling release notes Team:Operations Team label for Operations Team v8.6.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Single User Performance Journey Ergonomics
8 participants