[test-optimization] [SDTEST-2024] Fix Playwright exit code status when it fails before running the test #5733
Conversation
Overall package sizeSelf size: 9.37 MB Dependency sizes| name | version | self size | total size | |------|---------|-----------|------------| | @datadog/libdatadog | 0.5.1 | 29.73 MB | 29.73 MB | | @datadog/native-appsec | 8.5.2 | 19.33 MB | 19.34 MB | | @datadog/pprof | 5.8.0 | 12.55 MB | 12.92 MB | | @datadog/native-iast-taint-tracking | 4.0.0 | 11.72 MB | 11.73 MB | | @opentelemetry/core | 1.30.1 | 908.66 kB | 7.16 MB | | protobufjs | 7.4.0 | 2.77 MB | 5.42 MB | | @datadog/wasm-js-rewriter | 4.0.1 | 2.85 MB | 3.58 MB | | @datadog/native-metrics | 3.1.1 | 1.02 MB | 1.43 MB | | @opentelemetry/api | 1.8.0 | 1.21 MB | 1.21 MB | | import-in-the-middle | 1.13.1 | 117.64 kB | 839.26 kB | | source-map | 0.7.4 | 226 kB | 226 kB | | opentracing | 0.14.7 | 194.81 kB | 194.81 kB | | lru-cache | 7.18.3 | 133.92 kB | 133.92 kB | | pprof-format | 2.1.0 | 111.69 kB | 111.69 kB | | @datadog/sketches-js | 2.1.1 | 109.9 kB | 109.9 kB | | lodash.sortby | 4.7.0 | 75.76 kB | 75.76 kB | | ignore | 5.3.2 | 53.63 kB | 53.63 kB | | istanbul-lib-coverage | 3.2.0 | 29.34 kB | 29.34 kB | | rfdc | 1.4.1 | 27.15 kB | 27.15 kB | | @isaacs/ttlcache | 1.4.1 | 25.2 kB | 25.2 kB | | dc-polyfill | 0.1.8 | 25.08 kB | 25.08 kB | | tlhunter-sorted-set | 0.1.0 | 24.94 kB | 24.94 kB | | shell-quote | 1.8.2 | 23.54 kB | 23.54 kB | | limiter | 1.1.5 | 23.17 kB | 23.17 kB | | retry | 0.13.1 | 18.85 kB | 18.85 kB | | semifies | 1.0.0 | 15.84 kB | 15.84 kB | | jest-docblock | 29.7.0 | 8.99 kB | 12.76 kB | | crypto-randomuuid | 1.0.0 | 11.18 kB | 11.18 kB | | ttl-set | 1.0.0 | 4.61 kB | 9.69 kB | | mutexify | 1.4.0 | 5.71 kB | 8.74 kB | | path-to-regexp | 0.1.12 | 6.6 kB | 6.6 kB | | koalas | 1.0.2 | 6.47 kB | 6.47 kB | | module-details-from-path | 1.0.3 | 4.47 kB | 4.47 kB |🤖 This report was automatically generated by heaviest-objects-in-the-universe |
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## master #5733 +/- ##
=======================================
Coverage 79.08% 79.08%
=======================================
Files 515 515
Lines 23578 23578
=======================================
Hits 18646 18646
Misses 4932 4932 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
Datadog ReportBranch report: ✅ 0 Failed, 510 Passed, 0 Skipped, 16m 8.34s Total Time |
BenchmarksBenchmark execution time: 2025-05-14 15:14:05 Comparing candidate commit f891b7a in PR branch Found 0 performance improvements and 0 performance regressions! Performance is the same for 1272 metrics, 51 unstable metrics. |
|
|
||
| if (totalFailedTestCount === totalAttemptToFixFailedTestCount) { | ||
| if (totalFailedTestCount > 0 && | ||
| totalAttemptToFixFailedTestCount > 0 && |
There was a problem hiding this comment.
Nit: this condition is not needed. It would be caught with the next condition comparing the number of failures.
There was a problem hiding this comment.
Yeah it makes sense. Next time that I touch this file I'll change this :)
…n it fails before running the test (#5733)
ghost
mentioned this pull request
|
@Mariovido I just looked at our release proposal and saw that this is marked as minor. Being a fix, isn't it actually a patch? I just updated it, but please change it back, in case you think it should be a minor instead. |
|
@BridgeAR You're right it should be a patch. Thank you for noticing and changing it. :) |
…n it fails before running the test (#5733)
…n it fails before running the test (#5733)
What does this PR do?
With this change we make sure that we don't change the status of the exit code of the session if it fails before running any test.
Motivation
We want to solve an issue raised by a user
Fixes #5702
Plugin Checklist
Additional Notes