Repository navigation
ci: scope the C++ coverage report and add per-directory rows - #29
Merged
Merged
Conversation
Build with clang coverage instrumentation through CFLAGS and CXXFLAGS. Both builds in build-macos run cmake from scratch and read them. The test step writes one profile for each process and instrumented binary. After the tests, merge the profiles and report line coverage for the C++ test binary and the installed Python extension and libraries. Write the totals to the job summary and upload the full report as an artifact. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The first report counted the generated sources in the two build folders and the headers of the uv Python. Ignore every path under a build folder and the uv Python folder, so that the totals count only the mlx sources. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The job summary now shows one table with Lines, Branches and Functions for all host C++ under mlx/ and for backend/metal, backend/cpu, backend/common, distributed and io, each against the 80% target. The target is reported only. The report now also leaves out the Metal shader source in mlx/backend/metal/kernels/, the CUDA backend and the vendored pocketfft header. The step reads each value from the llvm-cov TOTAL row by its column name and fails when a value is missing. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The kernels exclude dropped steel/conv/params.h, a header that host C++ includes and that has coverage records. The shader source has no records, so the exclude is removed. The text says "link" instead of "run", and a comment notes that value_of rejects the "-" of a column without counters. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This was referenced Sep 30, 2026
# Conflicts: # .github/workflows/fork-tests.yml
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Proposed changes
The C++ coverage step of
Fork Testsnow says what its number measures. The ruling (David Tai, 2026-09-27 and 2026-09-29): the target is 80% line coverage, reported and never enforced, measured on host code only.mlx/. One row each coversmlx/backend/metal,mlx/backend/cpu,mlx/backend/common,mlx/distributedandmlx/io. The step runsllvm-cov reportonce for each row, with the directory as the source path.python/, build folders, virtual environment and Python headers). It addsmlx/backend/cuda/(the CUDA backend, which macOS builds only as a stub) andmlx/3rdparty/(the vendored pocketfft header). The Metal shader source needs no pattern: the host compiler never builds it, so it has no coverage records. The few headers inmlx/backend/metal/kernels/that host C++ includes, such assteel/conv/params.h, stay counted.llvm-cov exportcall that fed it are gone.Nothing else in the job changes.
fork.yamllists.github/workflows/fork-tests.ymlin the "Fork tooling" section, so nofork.yamlchange is needed.Test plan
actionlint .github/workflows/fork-tests.yml: no findings./usr/bin/python3 scripts/check_forkdiff.py --upstream-ref refs/remotes/upstream/main: passes.Job summary of that run, as the step wrote it (values checked against the TOTAL rows in the step log):
Before this PR (#28 head, run 36505380323) the same job counted 53027 lines at 68.26%. The difference is the CUDA stub (9 lines) and the pocketfft header (2281 lines).
steel/conv/params.h(17 lines) stays counted.CONTRIBUTING checklist
actionlintpasses on it.Dependencies
#28 is not merged yet. This branch is based on
ci/coverage-on-main(#28's head, ec82b43), so the diff againstmainshows #28's two commits too. Merge #28 first; this PR then adds one commit. If #28 is squash-merged, rebase this branch withgit rebase --onto main ec82b436.Authorship
Written by Claude (Opus 5.5) under David Tai's direction.
🤖 Generated with Claude Code