Skip to content

ci: diagnose flaky JaCoCo coverage - #9

Closed
warku123 wants to merge 1 commit into
developfrom
test/flaky-jacoco-coverage
Closed

warku123 wants to merge 1 commit into
developfrom
test/flaky-jacoco-coverage

Conversation

@warku123

@warku123 warku123 commented Jul 16, 2026 •

Copy link
Copy Markdown
Owner

Purpose

Temporary P0 instrumentation for repeated JaCoCo coverage runs on one fixed commit.

Scope

  • adds isolated JaCoCo baseline workflow
  • captures exec/XML manifests, semantic coverage counters, JUnit results, Gradle logs, and crash signals
  • does not change test parallelism, retries, Gradle configuration, or production/test behavior

Experiment

Re-run the same workflow run 10 times. Compare semantic JaCoCo counters and class-level line coverage; raw exec/XML hashes are evidence only.

Lifecycle

Draft experiment only. Do not merge into develop. Remove diagnostics after evidence collection.


Summary by cubic

Adds a branch-scoped CI workflow and a Python tool to collect stable JaCoCo coverage evidence and logs so we can compare repeated runs and diagnose flaky coverage. Runs only on the test/flaky-jacoco-coverage branch or via manual dispatch and uploads a manifest and summary with class-level counters and test failures.

  • New Features
    • New GitHub Actions workflow “JaCoCo Flake Baseline”: runs clean build, :framework:testWithRocksDb, and jacocoTestReport on JDK 8; gated to the experimental branch.
    • Adds .github/scripts/jacoco_baseline_summary.py to write evidence/summary.json and evidence/manifest.json with report-level counters, class-level line coverage, JUnit totals/failed tests, and Gradle log OOM/crash/retry signals; tolerant of malformed input.
    • Uploads evidence even when Gradle steps fail (then fails the job afterward): JaCoCo .exec and XML, JUnit results, test reports, Gradle logs, JVM crash artifacts, and evidence/metadata.json.
    • No changes to product code, test configs, parallelism, or Gradle settings.

Written for commit 33f647e. Summary will update on new commits.

Review in cubic

@coderabbitai

coderabbitai Bot commented Jul 16, 2026

Copy link
Copy Markdown

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 1f1d22c4-a69f-496f-a9ef-d6925eca550e

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch test/flaky-jacoco-coverage

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@warku123 warku123 closed this Aug 3, 2026
@warku123
warku123 deleted the test/flaky-jacoco-coverage branch August 7, 2026 09:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant