-
-
Notifications
You must be signed in to change notification settings - Fork 366
Add parser coverage enforcement at 84% threshold #8858
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Changes SummaryThis PR adds code coverage enforcement to the parser build system with an 84% minimum threshold, prevents coverage regression, and adds snapshot tests for operators (pizza Type: mixed Components Affected: Build system (build.zig), Parser source documentation (NodeStore.zig, tokenize.zig), Test suite (snapshot tests) Files Changed
Architecture Impact
Risk Areas: Coverage threshold is hardcoded - increasing it requires code changes, Excludes debug statements which may hide untested error paths in production debugging code, HTML.zig exclusion assumes it's truly a visualization-only utility with no critical logic Suggestions
Full review in progress... | Powered by diffray |
kcov's ptrace-based approach doesn't work reliably with Zig-generated binaries on Linux (reports 0 lines). On macOS, kcov uses LLDB which works. Developers should run `zig build coverage` locally before merging. The enforcement at 84% minimum threshold still works locally. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This reverts commit d9dd394.
kcov cannot instrument musl binaries (it uses ptrace/DWARF which expects native glibc). The default target on Linux is musl for static linking, so coverage tests were building musl binaries that kcov couldn't instrument (reported 0% coverage). This fix explicitly uses a native glibc target for the coverage test binaries, while leaving the main roc binary as musl for static linking. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
|
Superseded by #8878 which includes additional fixes for coverage enforcement |
The previous fix used an empty target query which may still default to musl in some contexts. This explicitly specifies .abi = .gnu to ensure glibc is used for coverage binaries. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Use ./src/parse prefix like TigerBeetle uses --include-path=./src. Remove extra exclude flags that may be causing issues. Temporarily lower threshold to 50% to verify kcov works.
Instead of using setExecCmd which wraps the test, build the test binary first, then run kcov as a separate system command. This is how TigerBeetle runs kcov: kcov --include-path=./src output_dir binary_path
When running kcov on a single binary (not merging), it creates a subdirectory named after the binary: parse_unit_coverage/coverage.json
Also set threshold to 0 so we can see the output without failing.
kcov has compatibility issues with Zig binaries on Linux, but works reliably on macOS. Run coverage on the macos-15 (ARM64) runner only. - Remove Linux glibc workaround (no longer needed) - Restore 84% coverage threshold - Only enable coverage step for macOS in build.zig
The nix flake only includes kcov on Linux, so install it via brew before running coverage on macOS ARM.
Coverage runs on macOS ARM where kcov works correctly (via LLDB). kcov doesn't work on Linux with Zig binaries (ptrace issue). Coverage workflow: - Runs in parallel with main CI - Does not block CI completion (takes 5+ hours) - Uploads coverage report as artifact - Has 6-hour timeout 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
|
Closing in favor of #8864 |
Summary
Technical Details
Test plan
zig build coveragepasses locally with 84.69% coverage🤖 Generated with Claude Code