Fix 50+ year duration in flaky-monitor report#1391
Merged
KCarretto merged 1 commit intoflaky-test-monitor-3414294539595760241from Dec 24, 2025
Merged
Conversation
The `go-ctrf-json-reporter` produces absolute Unix timestamps for start/stop times. When these reports are merged with other reports (e.g., from `junit-to-ctrf`) that use relative timestamps starting at 0, the duration calculation (stop - start) results in the current Unix epoch duration (~56 years). This commit updates the `Tests` workflow to normalize the Tavern CTRF report timestamps by shifting the start time to 0 while preserving the duration, ensuring correct reporting in the Flaky Monitor.
Contributor
Author
|
👋 Jules, reporting for duty! I'm here to lend a hand with this pull request. When you start a review, I'll add a 👀 emoji to each comment to let you know I've read it. I'll focus on feedback directed at me and will do my best to stay out of conversations between you and other bots or reviewers to keep the noise down. I'll push a commit with your requested changes shortly after. Please note there might be a delay between these steps, but rest assured I'm on the job! For more direct control, you can switch me to Reactive Mode. When this mode is on, I will only act on comments where you specifically mention me with For security, I will only act on instructions from the user who triggered this task. New to Jules? Learn more at jules.google/docs. |
KCarretto
approved these changes
Dec 24, 2025
KCarretto
merged commit 7b7bbc3
into
flaky-test-monitor-3414294539595760241
1 check passed
KCarretto
added a commit
that referenced
this pull request
Dec 24, 2025
* Implement flaky test monitoring system - Update `tests.yml` to generate JUnit XML reports for Go, Rust, and Node.js tests. - Add `flaky-monitor` job to aggregate and analyze historical test results. - Create `scripts/analyze_flaky_tests.py` to calculate test flakiness grouped by OS. - Configure automatic reporting to GitHub Step Summary and PR comments. - Use `test-results` branch for persistent history storage. * Implement flaky test monitoring using CTRF - Update `tests.yml` to generate CTRF reports for Go, Rust, and Node.js tests. - Use `go-ctrf-json-reporter` for Go tests. - Use `junit-to-ctrf` for Rust and Node.js tests (converting JUnit XML). - Add `flaky-monitor` job using `ctrf-io/github-test-reporter` to track and report flaky tests. - Configure historical data tracking via artifact retention. * Implement CTRF-based flaky test monitoring - Update `tests.yml` to generate CTRF reports for Go, Rust, and Node.js tests. - Use `go-ctrf-json-reporter` for Go tests. - Use `junit-to-ctrf` for Rust and Node.js tests (converting JUnit XML). - Add `flaky-monitor` job using `ctrf-io/github-test-reporter` to track and report flaky tests. - Configure historical data tracking via artifact retention. - Fix argument passing for `cargo llvm-cov` by using `--` separator. * Implement CTRF-based flaky test monitoring - Update `tests.yml` to generate CTRF reports for Go, Rust, and Node.js tests. - Use `go-ctrf-json-reporter` for Go tests. - Use `junit-to-ctrf` for Rust and Node.js tests (converting JUnit XML). - Add `flaky-monitor` job using `ctrf-io/github-test-reporter` to track and report flaky tests. - Configure historical data tracking via artifact retention. - Fix argument passing for `cargo llvm-cov` by configuring `nextest.toml` for JUnit output. * Implement CTRF-based flaky test monitoring - Update `tests.yml` to generate CTRF reports for Go, Rust, and Node.js tests. - Use `go-ctrf-json-reporter` for Go tests. - Use `junit-to-ctrf` for Rust and Node.js tests (converting JUnit XML). - Add `flaky-monitor` job using `ctrf-io/github-test-reporter` to track and report flaky tests. - Configure historical data tracking via artifact retention. - Fix argument passing for `cargo llvm-cov` by configuring `nextest.toml` for JUnit output. - Fix PATH issues for `go-ctrf-json-reporter` by explicitly adding GOPATH/bin to GITHUB_PATH. - Ensure `npx` is available in Rust job by adding `setup-node` step. * Implement CTRF-based flaky test monitoring - Update `tests.yml` to generate CTRF reports for Go, Rust, and Node.js tests. - Use `go-ctrf-json-reporter` for Go tests. - Use `junit-to-ctrf` for Rust and Node.js tests (converting JUnit XML). - Add `flaky-monitor` job using `ctrf-io/github-test-reporter` to track and report flaky tests. - Configure historical data tracking via artifact retention. - Fix argument passing for `cargo llvm-cov` by configuring `nextest.toml` for JUnit output. - Fix PATH issues for `go-ctrf-json-reporter` by calling binary with full path. - Ensure `npx` is available in Rust job by adding `setup-node` step. * Implement CTRF-based flaky test monitoring - Update `tests.yml` to generate CTRF reports for Go, Rust, and Node.js tests. - Use `go-ctrf-json-reporter` for Go tests. - Use `junit-to-ctrf` for Rust and Node.js tests (converting JUnit XML). - Add `flaky-monitor` job using `ctrf-io/github-test-reporter` to track and report flaky tests. - Configure historical data tracking via artifact retention. - Fix argument passing for `cargo llvm-cov` by configuring `nextest.toml` for JUnit output. - Fix PATH issues for `go-ctrf-json-reporter` by calling binary with full path. - Ensure `npx` is available in Rust job by adding `setup-node` step. - Pin `gotestsum` to v1.12.0 to support Go 1.23.4. * Implement CTRF-based flaky test monitoring - Update `tests.yml` to generate CTRF reports for Go, Rust, and Node.js tests. - Use `go-ctrf-json-reporter` for Go tests. - Use `junit-to-ctrf` for Rust and Node.js tests (converting JUnit XML). - Add `flaky-monitor` job using `ctrf-io/github-test-reporter` to track and report flaky tests. - Configure historical data tracking via artifact retention. - Fix argument passing for `cargo llvm-cov` by configuring `nextest.toml` for JUnit output. - Fix PATH issues for `go-ctrf-json-reporter` by calling binary with full path. - Ensure `npx` is available in Rust job by adding `setup-node` step. - Pin `gotestsum` to v1.12.0 to support Go 1.23.4. - Use `shell: bash` explicitly for steps using bash syntax to prevent PowerShell errors on Windows runners. * Implement CTRF-based flaky test monitoring - Update `tests.yml` to generate CTRF reports for Go, Rust, and Node.js tests. - Use `go-ctrf-json-reporter` for Go tests. - Use `junit-to-ctrf` for Rust and Node.js tests (converting JUnit XML). - Add `flaky-monitor` job using `ctrf-io/github-test-reporter` to track and report flaky tests. - Configure historical data tracking via artifact retention. - Fix argument passing for `cargo llvm-cov` by configuring `nextest.toml` for JUnit output. - Fix PATH issues for `go-ctrf-json-reporter` by calling binary with full path. - Ensure `npx` is available in Rust job by adding `setup-node` step. - Pin `gotestsum` to v1.12.0 to support Go 1.23.4. - Use `shell: bash` explicitly for steps using bash syntax to prevent PowerShell errors on Windows runners. - Revert unintended version downgrade in `.devcontainer/devcontainer.json`. * Implement CTRF-based flaky test monitoring - Update `tests.yml` to generate CTRF reports for Go, Rust, and Node.js tests. - Use `go-ctrf-json-reporter` for Go tests. - Use `junit-to-ctrf` for Rust and Node.js tests (converting JUnit XML). - Add `flaky-monitor` job using `ctrf-io/github-test-reporter` to track and report flaky tests. - Configure historical data tracking via artifact retention. - Fix argument passing for `cargo llvm-cov` by configuring `nextest.toml` for JUnit output. - Fix PATH issues for `go-ctrf-json-reporter` by calling binary with full path. - Ensure `npx` is available in Rust job by adding `setup-node` step. - Pin `gotestsum` to v1.12.0 to support Go 1.23.4. - Use `shell: bash` explicitly for steps using bash syntax to prevent PowerShell errors on Windows runners. - Revert unintended version downgrade in `.devcontainer/devcontainer.json`. - Restore missing documentation in `docs/_docs/user-guide/imix.md`. * Implement CTRF-based flaky test monitoring - Update `tests.yml` to generate CTRF reports for Go, Rust, and Node.js tests. - Use `go-ctrf-json-reporter` for Go tests. - Use `junit-to-ctrf` for Rust and Node.js tests (converting JUnit XML). - Add `flaky-monitor` job using `ctrf-io/github-test-reporter` to track and report flaky tests with comprehensive reporting options enabled. - Configure historical data tracking via artifact retention. - Fix argument passing for `cargo llvm-cov` by configuring `nextest.toml` for JUnit output. - Fix PATH issues for `go-ctrf-json-reporter` by calling binary with full path. - Ensure `npx` is available in Rust job by adding `setup-node` step. - Pin `gotestsum` to v1.12.0 to support Go 1.23.4. - Use `shell: bash` explicitly for steps using bash syntax to prevent PowerShell errors on Windows runners. - Revert unintended version downgrade in `.devcontainer/devcontainer.json`. - Restore missing documentation in `docs/_docs/user-guide/imix.md`. - Correctly locate `nextest.toml` in `.config/` directory. * switching to manual * attempt #2 * overwrite-comment * Fix CI workflow to correctly locate nested JUnit reports (#1388) The `implants` test job was failing to generate CTRF reports because the `junit.xml` output from `cargo nextest` was located in a subdirectory (e.g., `target/nextest/default/junit.xml`) rather than the workspace root. This caused the `mv` command to fail, skipping the conversion step and ultimately causing the artifact upload to fail with "No files were found". This change updates the workflow to dynamically search for `junit.xml` using `find` (piped to `head -n 1` for macOS compatibility), ensuring the report is correctly located and processed regardless of the output directory structure. This also restores the `implants/.config/nextest.toml` configuration to ensure JUnit generation is enabled. Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> * Fix: Normalize timestamps in Tavern CTRF reports (#1391) * Update flaky-monitor workflow to show fail rate and history (#1395) --------- Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> Co-authored-by: KCarretto <Kcarretto@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Normalized timestamps in Tavern CTRF report generation to prevent incorrect duration calculations in aggregated reports.
PR created automatically by Jules for task 10428402154428350700 started by @KCarretto