Skip to content

feat: introduce code coverage and mutation testing infrastructure#343

Draft
Copilot wants to merge 2 commits into
mainfrom
copilot/introduce-code-coverage-mutation-testing
Draft

feat: introduce code coverage and mutation testing infrastructure#343
Copilot wants to merge 2 commits into
mainfrom
copilot/introduce-code-coverage-mutation-testing

Conversation

Copilot AI commented Apr 18, 2026

Copy link
Copy Markdown
Contributor

📦 What

Adds pytest-cov (7.1.0) for code coverage and mutmut (3.5.0) for mutation testing, with CI workflows, wrapper scripts, and a ratcheting threshold mechanism.

  • requirements.txt — pinned pytest-cov==7.1.0, mutmut==3.5.0
  • cmd/scadm/pyproject.toml[tool.coverage.*] (branch coverage, exclusions) and [tool.mutmut] config
  • cmd/scadm/.coverage-threshold — persisted threshold (starts at 80%), ratchets upward on success
  • cmd/test/test-coverage.sh — runs coverage, enforces threshold, updates ratchet file
  • cmd/test/test-mutmut.sh — runs mutation testing locally
  • .github/workflows/coverage.yml — PRs + main pushes (scoped to cmd/scadm/**)
  • .github/workflows/mutation-tests.yml — weekly cron (Monday 03:00 UTC) + workflow_dispatch
  • docs/decisions/ADR-002-code-coverage-mutation-testing.md — tool selection rationale
  • Updated TESTING.md, .github/instructions/python.instructions.md, .gitignore

💡 Why

Coverage (quantity) + mutation testing (quality) as deterministic guardrails for AI-assisted development. Tool choices documented in ADR-002:

Tool Over Reason
pytest-cov Codecov/Coveralls SaaS Local-first, no external dependency
mutmut cosmic-ray, mutpy Active maintenance, incremental runs, simpler config

Mutation tests run weekly (not per-PR) because they're CPU-intensive. The ratchet mechanism ensures coverage monotonically increases — the threshold file is updated to the actual value on each passing run.

⚠️ Current unit test coverage is 50%. The coverage workflow will fail until test backfill lands (depends on #326).

🔧 How

# Coverage: enforce threshold (fails below stored value)
./cmd/test/test-coverage.sh

# Coverage: HTML report only (no threshold check)
./cmd/test/test-coverage.sh --report

# Mutation testing: full run
./cmd/test/test-mutmut.sh

# Mutation testing: show last results
./cmd/test/test-mutmut.sh --results

Config lives in cmd/scadm/pyproject.toml under [tool.coverage.*] and [tool.mutmut]. The ratchet threshold is in cmd/scadm/.coverage-threshold — do not lower it manually.

Add pytest-cov (7.1.0) and mutmut (3.5.0) with:
- Coverage config in pyproject.toml (branch coverage, 80% threshold)
- Ratcheting threshold file (.coverage-threshold)
- Wrapper scripts: test-coverage.sh, test-mutmut.sh
- CI workflows: coverage.yml (PRs + main), mutation-tests.yml (weekly)
- ADR-002 documenting tool selection rationale
- Updated TESTING.md and python.instructions.md

Agent-Logs-Url: https://github.com/kellerlabs/homeracker/sessions/404faffd-118c-417f-a6f3-e468a836906f

Co-authored-by: kellervater <33324812+kellervater@users.noreply.github.com>
Copilot AI changed the title [WIP] Add code coverage and mutation testing integration feat: introduce code coverage and mutation testing infrastructure Apr 18, 2026
Copilot AI requested a review from kellervater April 18, 2026 13:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Introduce Code Coverage and Mutation Testing

2 participants