Skip to content

chore: stryker applies and verifies all mutants — config, report-only CI, docs - #103

Merged
Bonobo791 merged 8 commits into
mainfrom
chore-stryker-runs-the-mutants
Aug 5, 2026
Merged

chore: stryker applies and verifies all mutants — config, report-only CI, docs#103
Bonobo791 merged 8 commits into
mainfrom
chore-stryker-runs-the-mutants

Conversation

@Bonobo791

@Bonobo791 Bonobo791 commented Aug 5, 2026

Copy link
Copy Markdown
Owner

User description

What

Makes Stryker the only way mutants are applied and verified in this repo, per the approved plan. Hand-editing source to simulate a mutation stops being a workflow and becomes a documented, PR-justified exception.

1. stryker.config.json productionized (plus reports/ added to .gitignore — it was unignored while holding the baseline report and the incremental cache):

  • coverageAnalysis: "perTest" — biggest speedup; only tests covering each mutant run.
  • incremental: true — cache at reports/stryker-incremental.json.
  • json reporter added — reports/mutation/mutation.json is the machine-readable survivor list for the agent loop.
  • timeoutMS 10000, timeoutFactor 1.5, concurrency 4. No thresholds (ratchet stays a maintainer-approved step).

Verified: scoped run on src/lib/relative-time.ts clean (51/53 killed, 96.23%), incremental rerun reuses the cache (4s → 2s), reports/mutation/mutation.html + mutation.json land where the CI artifact upload expects them.

2. Report-only CI — .github/workflows/mutation.yml: on every PR to main, Stryker runs --since main --incremental (full-history checkout, svelte-kit sync first, incremental cache via actions/cache) and uploads reports/mutation/ as an artifact. It never fails on survivors — there is no thresholds.break; the check only goes red if Stryker itself errors. Note: the workflow's first real run happens on this very PR.

3. Docs eliminate the manual path:

  • AGENTS.md role section: Stryker applies and verifies ALL mutants; a kill is proven by survived→killed on a scoped re-run (that re-run IS the both-directions confirmation); the CI report run is documented.
  • Skill SKILL.md ("In the Moderaty repo" section rewritten, also fixing the now-stale "bare config, no mutate globs" description): never hand-edit source to simulate a mutation; mental review escalates to a scoped npx stryker run --mutate, not a physical mutation.
  • references/agent-mutation-loop.md: "Manual adversarial mode" → "Hand-applied mutants: exception only" (commit first, confirm red, revert immediately, justify in the PR); loop step 6 names the Stryker re-run as the validation mechanism.

Installed skill re-synced to ~/.agents/skills/mutation-testing (diff-verified).

Verification

  • npm run test — 519 passed (58 files); npm run check — 0/0.
  • codacy-analysis analyze on all changed files — 0 issues.
  • actionlint unavailable locally; workflow YAML manually reviewed, and reporter output paths verified by an actual run (see above).

Out of scope (per plan)

thresholds.break gating, killing the baseline's survivors, scheduled full sweeps.


CodeAnt-AI Description

Standardize mutation testing for pull requests with scoped, report-only checks

What Changed

  • Pull requests now run mutation testing only against changed production TypeScript files, while excluding tests, development helpers, and static legal pages.
  • Mutation reports are uploaded as HTML and JSON artifacts, and surviving mutants do not fail the pull request.
  • Incremental results and per-test coverage reduce repeated mutation work.
  • A shared scope command keeps local and CI mutation runs consistent and skips checks when no mutable source files changed.
  • Documentation now requires Stryker to apply and verify mutants, with manual mutations limited to rare unsupported cases.
  • Added tests cover file filtering, mixed changes, and empty mutation scopes.

Impact

✅ Scoped mutation feedback on pull requests
✅ Faster repeat mutation runs
✅ Consistent local and CI mutation results

💡 Usage Guide

Checking Your Pull Request

Every time you make a pull request, our system automatically looks through it. We check for security issues, mistakes in how you're setting up your infrastructure, and common code problems. We do this to make sure your changes are solid and won't cause any trouble later.

Talking to CodeAnt AI

Got a question or need a hand with something in your pull request? You can easily get in touch with CodeAnt AI right here. Just type the following in a comment on your pull request, and replace "Your question here" with whatever you want to ask:

@codeant-ai ask: Your question here

This lets you have a chat with CodeAnt AI about your pull request, making it easier to understand and improve your code.

Example

@codeant-ai ask: Can you suggest a safer alternative to storing this secret?

Preserve Org Learnings with CodeAnt

You can record team preferences so CodeAnt AI applies them in future reviews. Reply directly to the specific CodeAnt AI suggestion (in the same thread) and replace "Your feedback here" with your input:

@codeant-ai: Your feedback here

This helps CodeAnt AI learn and adapt to your team's coding style and standards.

Example

@codeant-ai: Do not flag unused imports.

Retrigger review

Ask CodeAnt AI to review the PR again, by typing:

@codeant-ai: review

Check Your Repository Health

To analyze the health of your code repository, visit our dashboard at https://app.codeant.ai. This tool helps you identify potential issues and areas for improvement in your codebase, ensuring your repository maintains high standards of code health.

@cla-bot cla-bot Bot added the cla-signed label Aug 5, 2026
@codeant-ai

codeant-ai Bot commented Aug 5, 2026

Copy link
Copy Markdown

🤖 CodeAnt AI — Review Status

Status Commit Started (UTC) Finished (UTC)
✅ Incremental review completed 95ccd59 Aug 05, 2026 · 16:35 16:38
✅ Incremental review completed 9f10007 Aug 05, 2026 · 16:23 16:26
✅ Incremental review completed cbf7fe7 Aug 05, 2026 · 16:11 16:13
✅ Reviewed your PR 0e795bb Aug 05, 2026 · 15:23 15:24

@codeant-ai

codeant-ai Bot commented Aug 5, 2026

Copy link
Copy Markdown

Thanks for using CodeAnt! 🎉

We're free for open-source projects. if you're enjoying it, help us grow by sharing.

Share on X ·
Reddit ·
LinkedIn

@netlify

netlify Bot commented Aug 5, 2026

Copy link
Copy Markdown

Deploy Preview for moderaty ready!

Name Link
🔨 Latest commit 095a548
🔍 Latest deploy log https://app.netlify.com/projects/moderaty/deploys/6a7367770fbc900008ba7f70
😎 Deploy Preview https://deploy-preview-103--moderaty.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.
Lighthouse
Lighthouse
1 paths audited
Performance: 90
Accessibility: 97
Best Practices: 100
SEO: 100
PWA: -
View the detailed breakdown and full score reports
🤖 Make changes Run an agent on this branch

To edit notification comments on pull requests, go to your Netlify project configuration.

@codeant-ai codeant-ai Bot added the size:L This PR changes 100-499 lines, ignoring generated files label Aug 5, 2026
@codeant-ai

codeant-ai Bot commented Aug 5, 2026

Copy link
Copy Markdown

Sequence Diagram

This PR makes Stryker the required path for applying and verifying mutants, while adding a report-only mutation check for every pull request. Survivors are reviewed from the generated report and verified as killed through a scoped Stryker rerun.

sequenceDiagram
    participant Developer
    participant GitHub Actions
    participant Stryker
    participant Test Suite
    participant Artifact

    Developer->>GitHub Actions: Open or update pull request
    GitHub Actions->>Stryker: Run mutations since main
    Stryker->>Test Suite: Verify each generated mutant
    Test Suite-->>Stryker: Mutation results
    Stryker-->>GitHub Actions: HTML and JSON reports
    GitHub Actions->>Artifact: Upload mutation reports
    Developer->>Stryker: Rerun scoped survivors after adding tests
    Stryker->>Test Suite: Confirm survivors become killed
Loading

Generated by CodeAnt AI

@codacy-production

Copy link
Copy Markdown

Up to standards ✅

🟢 Issues 0 issues

Results:
0 new issues

View in Codacy

NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.

@amazon-q-developer amazon-q-developer Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review Complete

This PR successfully productionizes Stryker as the authoritative mutation testing workflow. The changes are well-structured and documented:

Configuration & Tooling:

  • stryker.config.json properly configured with performance optimizations (coverageAnalysis: perTest, incremental: true) and machine-readable output (json reporter)
  • CI workflow correctly implements report-only mode with proper caching strategy and artifact uploads
  • .gitignore appropriately excludes generated reports

Documentation:

  • AGENTS.md, SKILL.md, and agent-mutation-loop.md consistently establish that Stryker applies and verifies ALL mutants
  • Clear workflow guidance eliminates manual mutation application as a standard practice
  • Exception handling for rare cases where Stryker's operators are insufficient is properly documented

The PR description verification claims are substantiated by the code changes. No blocking defects identified.


You can now have the agent implement changes and create commits directly on your pull request's source branch. Simply comment with /q followed by your request in natural language to ask the agent to make changes.

@codeant-ai

codeant-ai Bot commented Aug 5, 2026

Copy link
Copy Markdown

🏁 CodeAnt Quality Gate Results

Commit: 095a5483
Scan Time: 2026-08-05 16:41:51 UTC

✅ Overall Status: PASSED

Quality Gate Details

Quality Gate Status Details
Secrets ✅ PASSED 0 secrets found
Duplicate Code ✅ PASSED 0.0% duplicated
SAST ✅ PASSED No security issues
Bugs ✅ PASSED Rating S: No bugs
IAC ✅ PASSED Rating S: No issues

View Full Results

@coderabbitai

coderabbitai Bot commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Warning

Review limit reached

@Bonobo791, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 43 minutes

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 2ccc0d57-9b3e-4172-965c-62dbe2223fe7

📥 Commits

Reviewing files that changed from the base of the PR and between 9f10007 and 095a548.

📒 Files selected for processing (5)
  • .agents/skills-src/mutation-testing/SKILL.md
  • .agents/skills-src/mutation-testing/references/tools-by-language.md
  • .github/workflows/mutation.yml
  • scripts/stryker-pr-scope.mjs
  • scripts/stryker-pr-scope.test.mjs
📝 Walkthrough

Summary by CodeRabbit

  • New Features

    • Added automated mutation testing for pull requests, focused on changed production files.
    • Added mutation reports in JSON, HTML, clear-text, and progress formats.
    • Added incremental test execution to reduce repeat analysis time.
  • Bug Fixes

    • Corrected pull-request mutation scoping to avoid unsupported options and exclude tests and non-production files.
  • Documentation

    • Updated mutation-testing guidance, validation workflows, and exceptional manual-mutation procedures.

Walkthrough

The PR adds repository-specific Stryker configuration, a Git-diff scope utility, pull-request mutation CI, report handling, tests, and updated mutation-testing guidance.

Changes

Mutation testing workflow

Layer / File(s) Summary
Stryker configuration and reporting
stryker.config.json, .gitignore
Stryker now uses incremental execution, per-test coverage, JSON reports, timeout settings, and concurrency of 4. Mutation reports are ignored.
Changed-file scope calculation
scripts/stryker-pr-scope.mjs, scripts/stryker-pr-scope.test.mjs
The scope utility selects changed production TypeScript files under src, excludes configured paths, and handles empty scopes. Vitest tests cover these filters.
Pull-request mutation workflow
.github/workflows/mutation.yml
The workflow prepares the repository, computes the changed-file scope, runs report-only Stryker checks when needed, and uploads reports.
Mutation-testing execution guidance
AGENTS.md, .agents/skills-src/mutation-testing/*
Guidance replaces Stryker.NET-style --since usage with diff-based --mutate scoping. It requires survivor re-runs and limits hand-applied mutants to unsupported operators.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Sequence Diagram(s)

sequenceDiagram
  participant PullRequest
  participant GitHubActions
  participant ScopeScript
  participant Stryker
  participant Reports
  PullRequest->>GitHubActions: Trigger mutation workflow
  GitHubActions->>ScopeScript: Compute changed mutable TypeScript files
  ScopeScript-->>GitHubActions: Return mutation scope
  GitHubActions->>Stryker: Run scoped report-only mutation test
  Stryker->>Reports: Generate mutation reports
  GitHubActions->>Reports: Upload reports
Loading

Possibly related PRs

  • Bonobo791/Moderaty#55: Introduced the mutation-testing skill and agent feedback-loop guidance extended by this PR.
🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title clearly summarizes the main changes to Stryker configuration, report-only CI, and documentation.
Description check ✅ Passed The description directly explains the Stryker workflow, CI changes, documentation updates, and verification results.
✨ Finishing Touches 💡 2
📝 Generate docstrings 💡
  • Create stacked PR
  • Commit on current branch
🛠️ Fix failing CI checks 💡
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch chore-stryker-runs-the-mutants

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@qodo-code-review

Copy link
Copy Markdown

PR Summary by Qodo

Productionize Stryker mutation workflow (config + report-only CI + docs)

✨ Enhancement ⚙️ Configuration changes 📝 Documentation 🕐 20-40 Minutes

Grey Divider

AI Description

• Productionize stryker.config.json for faster, cached mutation runs and machine-readable output.
• Add PR-time, report-only Stryker GitHub Action that uploads HTML/JSON reports as artifacts.
• Document Stryker as the sole mutation mechanism; hand-applied mutations are exception-only.
Diagram

graph TD
  PR["Pull request"] --> GA["GitHub Actions"] --> Cache[("Incremental cache")] --> Stryker["Stryker run"] --> Vitest["Vitest runner"] --> Reports[/"Mutation reports"/] --> Artifact["Uploaded artifact"]
  Config["Stryker config"] --> Stryker
  Docs["Mutation docs"] -.-> Stryker

  subgraph Legend
    direction LR
    _proc["Process/Step"] ~~~ _db[("Cache/State")] ~~~ _art[/"Artifact/Output"/]
  end
Loading
High-Level Assessment

The following are alternative approaches to this PR:

1. Add a `thresholds.break` gate now
  • ➕ Turns mutation score into an enforced quality bar immediately
  • ➕ Prevents new survivors from landing unnoticed
  • ➖ High risk of noisy failures early (equivalents/no-coverage) and developer friction
  • ➖ Requires an agreed ratchet policy and triage workflow to avoid disabling the gate
2. Post results as a PR check summary/comment (in addition to artifact)
  • ➕ Makes survivors visible without downloading artifacts
  • ➕ Can highlight top survivors and link directly to mutation.html
  • ➖ More workflow complexity (parsing JSON + formatting)
  • ➖ Needs guardrails to avoid spam on frequent pushes
3. Nightly/weekly full-sweep workflow on main
  • ➕ Finds survivors outside PR diffs and tracks global health over time
  • ➕ Avoids PR latency while still improving coverage quality
  • ➖ Does not give immediate feedback to PR authors
  • ➖ Requires scheduling + retention decisions for reports/caches

Recommendation: Current report-only PR workflow is the right first step: it standardizes Stryker as the single source of truth, produces machine-readable survivors, and avoids premature gating while the project establishes a ratchet/triage process. Consider adding a lightweight PR summary (from mutation.json) next, then introduce thresholds.break once maintainers agree on the ratchet policy.

Files changed (6) +120 / -27

Documentation (3) +40 / -24
SKILL.mdMake Stryker the required mutation workflow (hand edits exception-only) +8/-5

Make Stryker the required mutation workflow (hand edits exception-only)

• Rewrites the repo-specific section to state Stryker applies/verifies all mutants, adds guidance on proving kills via survived→killed scoped re-runs, and documents the new CI report workflow. Clarifies incremental behavior and reframes mental mutation testing as review-time reasoning backed by scoped Stryker runs.

.agents/skills-src/mutation-testing/SKILL.md

agent-mutation-loop.mdAlign agent mutation loop validation with Stryker re-runs +16/-9

Align agent mutation loop validation with Stryker re-runs

• Updates the loop to validate tests by flipping a target mutant survived→killed via scoped Stryker re-runs. Replaces “manual adversarial mode” with an explicit exception-only process for hand-applied mutants when Stryker cannot express a mutation.

.agents/skills-src/mutation-testing/references/agent-mutation-loop.md

AGENTS.mdUpdate mutation-testing role to require Stryker verification +16/-10

Update mutation-testing role to require Stryker verification

• Clarifies that Stryker is mandatory for applying/verifying mutants, defines kill proof as survived→killed on scoped re-run, and documents the PR-time report-only CI workflow. Keeps thresholds gating explicitly out-of-scope pending maintainer approval.

AGENTS.md

Other (3) +80 / -3
mutation.ymlAdd report-only Stryker GitHub Action for PRs to main +70/-0

Add report-only Stryker GitHub Action for PRs to main

• Introduces a PR-triggered workflow that checks out full history, runs 'svelte-kit sync', restores/saves Stryker incremental cache, runs 'npx stryker run --since main', and uploads 'reports/mutation/' as an artifact. Workflow is explicitly non-gating (fails only on tool error, not survivors).

.github/workflows/mutation.yml

.gitignoreIgnore Stryker reports directory +2/-1

Ignore Stryker reports directory

• Adds 'reports/' to '.gitignore' so incremental cache and mutation reports are not committed. Keeps 'stryker.log' tracked as ignored with a newline fix.

.gitignore

stryker.config.jsonProductionize Stryker config (perTest, incremental, json reporter, tuning) +8/-2

Productionize Stryker config (perTest, incremental, json reporter, tuning)

• Enables 'coverageAnalysis: perTest' and 'incremental: true' for faster scoped mutation testing. Adds JSON reporting alongside existing reporters and sets timeouts/concurrency defaults to stabilize CI/local runs.

stryker.config.json

@qodo-code-review

qodo-code-review Bot commented Aug 5, 2026

Copy link
Copy Markdown

Code Review by Qodo

🐞 Bugs (0) 📘 Rule violations (0) 📜 Skill insights (0)

Grey Divider


Action required

1. Missing base ref for --since ✓ Resolved 🐞 Bug ☼ Reliability
Description
.github/workflows/mutation.yml runs npx stryker run --since main, but the job never ensures a
local main ref exists after checkout; depending on how the PR ref is checked out (often detached
HEAD with only origin/main), Stryker’s git diff step can error and the workflow will fail before
producing artifacts.
Code

.github/workflows/mutation.yml[63]

+      - run: npx stryker run --since main --incremental --reporters html,json,clear-text
Relevance

●●● Strong

CI reliability fix to prevent --since diff failures; workflow robustness changes are typically
accepted.

PR-#83

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
The workflow checks out the repo and then invokes Stryker diff-scoped to main, but there is no
step that fetches/creates a local main ref; this makes the --since main dependency brittle in
CI.

.github/workflows/mutation.yml[37-63]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The mutation-report workflow calls Stryker with `--since main`, but the workflow does not guarantee that a local git ref named `main` exists in the runner’s checkout. On PR jobs, this can cause `--since main` resolution/diffing to fail, turning the whole report job red.

## Issue Context
The workflow uses `actions/checkout` with `fetch-depth: 0`, which fetches history but does not explicitly create a local branch named `main`. The Stryker invocation hardcodes `--since main`.

## Fix Focus Areas
- .github/workflows/mutation.yml[37-63]

## Suggested fix
Either:
1) Ensure `main` exists locally before running Stryker (e.g., add a step after checkout: `git fetch origin main:main`), then keep `--since main`, or
2) Change the Stryker invocation to diff against a ref guaranteed to exist in CI (commonly `origin/main`), if Stryker supports it (`--since origin/main`).

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools



Remediation recommended

2. Stryker run missing --ignoreStatic ✓ Resolved 📘 Rule violation ▣ Testability
Description
The new CI workflow runs npx stryker run without the required --ignoreStatic flag. This can
cause static mutants to be included, violating the mandated Stryker invocation policy.
Code

.github/workflows/mutation.yml[63]

+      - run: npx stryker run --since main --incremental --reporters html,json,clear-text
Relevance

●●● Strong

Compliance-mandated CLI policy; team has accepted workflow hardening changes before.

PR-#83

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
PR Compliance ID 2545812 requires every CLI-based Stryker command to include --ignoreStatic. The
workflow step at .github/workflows/mutation.yml:63 runs `npx stryker run --since main
--incremental ... without --ignoreStatic`.

Rule 2545812: Enforce --ignoreStatic flag in Stryker mutation test runs
.github/workflows/mutation.yml[63-63]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The GitHub Actions workflow runs Stryker without the required `--ignoreStatic` flag.

## Issue Context
Compliance rule PR Compliance ID 2545812 requires that every CLI-based Stryker invocation includes `--ignoreStatic` (or an equivalent config setting).

## Fix Focus Areas
- .github/workflows/mutation.yml[63-63]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


Grey Divider

Context used
✅ Compliance rules (platform): 82 rules

To customize comments, go to the Qodo configuration screen, or learn more in the docs.

Qodo Logo

Comment thread .github/workflows/mutation.yml Outdated
Comment thread .github/workflows/mutation.yml Outdated
@codeant-ai

codeant-ai Bot commented Aug 5, 2026

Copy link
Copy Markdown

Thanks for using CodeAnt! 🎉

We're free for open-source projects. if you're enjoying it, help us grow by sharing.

Share on X ·
Reddit ·
LinkedIn

@codeant-ai codeant-ai Bot added size:L This PR changes 100-499 lines, ignoring generated files and removed size:L This PR changes 100-499 lines, ignoring generated files labels Aug 5, 2026
@Bonobo791

Copy link
Copy Markdown
Owner Author

Review triage — all bot comments addressed

@qodo-code-review

  • Stryker run missing --ignoreStatic (rule violation) — valid, fixed in cbf7fe7 as the config-equivalent the compliance rule allows: "ignoreStatic": true in stryker.config.json, so it covers every CLI invocation, local and CI, not just the workflow.
  • Missing base ref for --since (bug) — valid, and the diagnosis underneath it was worse: verifying the fix surfaced that StrykerJS has no --since flag at all (that is Stryker.NET) — the invocation died with unknown option '--since' (reproduced locally), and the same broken command had shipped in the docs: stryker is installed — update mutation-testing role and approved deps #94/docs: skill documents Stryker as the repo's mutation runner #97 docs. Fixed in cbf7fe7: the workflow now computes a git-diff-driven --mutate scope (git diff --name-only main...HEAD, filtered to the config's mutate set) with a loud skip notice for docs-only PRs, keeps the explicit git fetch origin main:main for the detached-HEAD case, and every --since reference in AGENTS.md, SKILL.md, and references/tools-by-language.md (selection table, lever list, CI YAML example) now uses the diff-driven pattern. The PR body is updated to match.

@sonarqubecloud — Quality Gate failed: C Security Rating on New Code. Declined. The hotspots are the @v4 version tags on GitHub Actions; the repo's existing workflow (db-backup.yml, on main) uses the identical unpinned-tag convention, so pinning only this file to SHAs would be incoherent. A repo-wide SHA-pinning policy is a separate maintainer decision; the hotspots can be reviewed/accepted in SonarCloud.

@codacy-production — up to standards, 0 issues. Nothing to fix.
@CodeAnt-AI — quality gate PASSED; sequence diagram and review-status comments noted, informational.
@coderabbitai — rate-limited this run; no comments to triage. Noted.
@netlify — deploy preview ready; informational.

Gates on cbf7fe7: 519 tests green, svelte-check 0/0, local codacy-analysis 0 issues, workflow YAML parses, CI scope pipeline verified locally (empty-scope skip and non-empty run). Holding for human review.

@codeant-ai

codeant-ai Bot commented Aug 5, 2026

Copy link
Copy Markdown

Question: "ignorestatic": trueinstryker.config.json`, so it covers every cli invocation, local and ci, not just the workflow.

  • missing base ref for --since (bug) — valid, and the diagnosis underneath it was worse: verifying the fix surfaced that strykerjs has no --since flag at all (that is stryker.net) — the invocation died with unknown option '--since' (reproduced locally), and the same broken command had shipped in the docs: stryker is installed — update mutation-testing role and approved deps #94/docs: skill documents Stryker as the repo's mutation runner #97 docs. fixed in cbf7fe7: the workflow now computes a git-diff-driven --mutate scope (git diff --name-only main...head, filtered to the config's mutate set) with a loud skip notice for docs-only prs, keeps the explicit git fetch origin main:main for the detached-head case, and every --since reference in agents.md, skill.md, and references/tools-by-language.md (selection table, lever list, ci yaml example) now uses the diff-driven pattern. the pr body is updated to match.

@sonarqubecloud — quality gate failed: c security rating on new code. declined. the hotspots are the @v4 version tags on github actions; the repo's existing workflow (db-backup.yml, on main) uses the identical unpinned-tag convention, so pinning only this file to shas would be incoherent. a repo-wide sha-pinning policy is a separate maintainer decision; the hotspots can be reviewed/accepted in sonarcloud.

@codacy-production — up to standards, 0 issues. nothing to fix.
@CodeAnt-AI — quality gate passed; sequence diagram and review-status comments noted, informational.
@coderabbitai — rate-limited this run; no comments to triage. noted.
@netlify — deploy preview ready; informational.

gates on cbf7fe7: 519 tests green, svelte-check 0/0, local codacy-analysis 0 issues, workflow yaml parses, ci scope pipeline verified locally (empty-scope skip and non-empty run). holding for human review.

Answer:
The --since diagnosis and workflow fix look correct: StrykerJS does not support --since, and creating a local main ref after a full-history checkout is appropriate for the detached PR checkout.

One remaining inconsistency should be fixed before merging:

  • The documented diff-driven commands in SKILL.md and references/tools-by-language.md exclude only *.test.ts:
    git diff --name-only main...HEAD |
      grep -E '^src/.+\.ts$' |
      grep -v '\.test\.ts$'
  • Because passing --mutate overrides the config’s mutate list, those commands can still mutate:
    • src/lib/server/testuser.ts
    • src/routes/terms/+page.ts
    • src/routes/privacy/+page.ts
    • src/routes/dpa/+page.ts

The workflow correctly filters those files, but the documented local/CI examples do not, despite the docs claiming they use the same scope as the config. The examples should use the same exclusion expression as the workflow, or preferably call a shared script so the scope cannot drift. They should also handle an empty result explicitly; otherwise a docs-only invocation may execute --mutate "" rather than produce the workflow’s clear skip notice.

The report-only behavior, global ignoreStatic: true, cache setup, and removal of the invalid --since references otherwise align with the stated plan.

@coderabbitai

coderabbitai Bot commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

Tip

For best results, initiate chat on the files or code changes.

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

printf '%s\n' '--- changed files near cbf7fe7 ---'
git show --name-status --format='%H%n%s' cbf7fe7 -- 2>/dev/null || true

printf '%s\n' '--- candidate files ---'
fd -a -t f '^(stryker\.config\.json|mutation\.yml|AGENTS\.md|SKILL\.md|tools-by-language\.md)$' . 2>/dev/null || true

printf '%s\n' '--- configuration and workflow excerpts ---'
for f in stryker.config.json .github/workflows/mutation.yml AGENTS.md; do
  if [ -f "$f" ]; then
    echo "### $f"
    rg -n -C 3 '"ignoreStatic"|coverageAnalysis|incremental|reporters|--mutate|git diff|--since|git fetch|mutation/' "$f" || true
  fi
done

printf '%s\n' '--- SKILL.md and tools reference excerpts ---'
while IFS= read -r f; do
  echo "### $f"
  rg -n -C 3 -- '--since|--mutate|git diff|Stryker|ignoreStatic' "$f" || true
done < <(fd -a -t f '^(SKILL\.md|tools-by-language\.md)$' . 2>/dev/null)

printf '%s\n' '--- remaining StrykerJS --since references in tracked text ---'
rg -n -i -C 2 'stryker.{0,120}--since|--since.{0,120}stryker' \
  -g '*.md' -g '*.yml' -g '*.yaml' -g '*.json' . || true

Length of output: 5176


🏁 Script executed:

#!/bin/bash
set -euo pipefail

printf '%s\n' '--- stryker.config.json ---'
cat -n stryker.config.json

printf '%s\n' '--- mutation workflow ---'
cat -n .github/workflows/mutation.yml

printf '%s\n' '--- changed skill documentation: relevant excerpts ---'
for f in \
  .agents/skills-src/mutation-testing/SKILL.md \
  .agents/skills-src/mutation-testing/references/tools-by-language.md
do
  echo "### $f"
  rg -n -i -C 4 -- 'Stryker|--since|--mutate|git diff|ignoreStatic' "$f" || true
done

printf '%s\n' '--- remaining --since references in repository documentation/configuration ---'
rg -n -i -C 2 -- '--since' \
  -g '*.md' -g '*.yml' -g '*.yaml' -g '*.json' . || true

Length of output: 19804


@Bonobo791 Verified the reported fixes in cbf7fe7.

  • stryker.config.json sets "ignoreStatic": true. This applies to local and CI Stryker runs.
  • The workflow does not use StrykerJS --since.
  • The workflow creates a filtered git diff --name-only main...HEAD scope and passes it with --mutate.
  • The workflow skips an empty scope with a visible notice.
  • The workflow uploads reports/mutation/.
  • AGENTS.md, SKILL.md, and references/tools-by-language.md document that StrykerJS has no --since option and use git-diff-driven --mutate scoping.

I found no blocking issue in the inspected changes.


✏️ Learnings added
Learnt from: Bonobo791
URL: https://github.com/Bonobo791/Moderaty/pull/103

Timestamp: 2026-08-05T16:13:27.760Z
Learning: In the Moderaty repository, StrykerJS mutation runs must use `stryker.config.json`, where `ignoreStatic: true` applies to all runs. For PR-scale StrykerJS mutation testing, scope `--mutate` from `git diff --name-only main...HEAD`; StrykerJS does not support the Stryker.NET `--since` option.

Note: Learnings are effective only in the context of similar code segments. To apply general coding guidelines, please configure review instructions. You can manage existing learnings in the app.

You are interacting with an AI system.

@qodo-code-review

Copy link
Copy Markdown

Review triage — all bot comments addressed

@qodo-code-review

  • Stryker run missing --ignoreStatic (rule violation) — valid, fixed in cbf7fe7 as the config-equivalent the compliance rule allows: "ignoreStatic": true in stryker.config.json, so it covers every CLI invocation, local and CI, not just the workflow.
  • Missing base ref for --since (bug) — valid, and the diagnosis underneath it was worse: verifying the fix surfaced that StrykerJS has no --since flag at all (that is Stryker.NET) — the invocation died with unknown option '--since' (reproduced locally), and the same broken command had shipped in the docs: stryker is installed — update mutation-testing role and approved deps #94/docs: skill documents Stryker as the repo's mutation runner #97 docs. Fixed in cbf7fe7: the workflow now computes a git-diff-driven --mutate scope (git diff --name-only main...HEAD, filtered to the config's mutate set) with a loud skip notice for docs-only PRs, keeps the explicit git fetch origin main:main for the detached-HEAD case, and every --since reference in AGENTS.md, SKILL.md, and references/tools-by-language.md (selection table, lever list, CI YAML example) now uses the diff-driven pattern. The PR body is updated to match.

@sonarqubecloud — Quality Gate failed: C Security Rating on New Code. Declined. The hotspots are the @v4 version tags on GitHub Actions; the repo's existing workflow (db-backup.yml, on main) uses the identical unpinned-tag convention, so pinning only this file to SHAs would be incoherent. A repo-wide SHA-pinning policy is a separate maintainer decision; the hotspots can be reviewed/accepted in SonarCloud.

@codacy-production — up to standards, 0 issues. Nothing to fix.
@CodeAnt-AI — quality gate PASSED; sequence diagram and review-status comments noted, informational.
@coderabbitai — rate-limited this run; no comments to triage. Noted.
@netlify — deploy preview ready; informational.

Gates on cbf7fe7: 519 tests green, svelte-check 0/0, local codacy-analysis 0 issues, workflow YAML parses, CI scope pipeline verified locally (empty-scope skip and non-empty run). Holding for human review.

Thanks for the detailed triage. The fixes address both Qodo findings:

  • finding 1 — the invalid base-ref/--since approach was replaced with a local main ref plus a git-diff-driven --mutate scope.
  • finding 2"ignoreStatic": true in stryker.config.json applies to the workflow and local Stryker invocations, satisfying the requirement without duplicating the CLI flag.

The documented commands, workflow behavior, and validation results are consistent with the updated implementation. No further Qodo action is needed from this review.

@codeant-ai codeant-ai Bot added size:L This PR changes 100-499 lines, ignoring generated files and removed size:L This PR changes 100-499 lines, ignoring generated files labels Aug 5, 2026
@codacy-production

Copy link
Copy Markdown

Up to standards ✅

🟢 Issues 0 issues

Results:
0 new issues

View in Codacy

🟢 Metrics 13 complexity · 0 duplication

Metric Results
Complexity 13 (≤ 100 complexity)
Duplication 0 (≤ 1 duplication)

View in Codacy

NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.

@Bonobo791

Copy link
Copy Markdown
Owner Author

Review triage follow-up

@CodeAnt-AI — your consistency catch was right on both counts, fixed in 9f10007:

  • The documented commands filtered only *.test.ts, so a CLI --mutate (which overrides the config globs) could still mutate testuser.ts and the legal-page loaders — and the docs claimed config parity they didn't have.
  • Docs-only diffs would have produced --mutate "" instead of a clean skip.

Adopted your preferred fix: a shared script, scripts/stryker-pr-scope.mjs — prints the changed-vs-main src scope filtered to the config's mutate set (the exclusions mirror stryker.config.json, noted in the file header), empty output means skip. The workflow and the documented local commands (SKILL.md, AGENTS.md) now both call it, so they cannot drift; tools-by-language.md's generic example explicitly warns about the override and the empty-scope case. Ships with behavior tests (scripts/stryker-pr-scope.test.mjs, 6 tests covering keeps/drops/join/empty), verified CLI-side both ways: empty line on a docs-only diff, filtered scope on a src-heavy diff. Gates: 525 tests green, svelte-check 0/0, codacy-analysis 0 issues on all changed files.

One process note, self-inflicted and fixed: while verifying I accidentally popped a stale stash from another branch (feat-reversible-actions), which conflicted ROADMAP.md; I restored the file to HEAD immediately and confirmed it byte-identical (git diff HEAD --quiet) and the stash entry intact. No content was lost or altered.

@coderabbitai — thanks for the independent verification run confirming the cbf7fe7 fixes; no blocking issues stands. @qodo-code-review — awaiting your re-review, nothing further from your side so far. @sonarqubecloud — the re-posted gate failure is the same unpinned-tag hotspot set already declined (matches db-backup.yml convention; repo-wide SHA pinning is a separate maintainer decision).

Holding for human review.

@codeant-ai

codeant-ai Bot commented Aug 5, 2026

Copy link
Copy Markdown

Question: - the documented commands filtered only *.test.ts, so a cli --mutate (which overrides the config globs) could still mutate testuser.ts and the legal-page loaders — and the docs claimed config parity they didn't have.

  • docs-only diffs would have produced --mutate "" instead of a clean skip.

adopted your preferred fix: a shared script, scripts/stryker-pr-scope.mjs — prints the changed-vs-main src scope filtered to the config's mutate set (the exclusions mirror stryker.config.json, noted in the file header), empty output means skip. the workflow and the documented local commands (skill.md, agents.md) now both call it, so they cannot drift; tools-by-language.md's generic example explicitly warns about the override and the empty-scope case. ships with behavior tests (scripts/stryker-pr-scope.test.mjs, 6 tests covering keeps/drops/join/empty), verified cli-side both ways: empty line on a docs-only diff, filtered scope on a src-heavy diff. gates: 525 tests green, svelte-check 0/0, codacy-analysis 0 issues on all changed files.

one process note, self-inflicted and fixed: while verifying i accidentally popped a stale stash from another branch (feat-reversible-actions), which conflicted roadmap.md; i restored the file to head immediately and confirmed it byte-identical (git diff head --quiet) and the stash entry intact. no content was lost or altered.

@coderabbitai — thanks for the independent verification run confirming the cbf7fe7 fixes; no blocking issues stands. @qodo-code-review — awaiting your re-review, nothing further from your side so far. @sonarqubecloud — the re-posted gate failure is the same unpinned-tag hotspot set already declined (matches db-backup.yml convention; repo-wide sha pinning is a separate maintainer decision).

holding for human review.

Answer:
Blocking issue: .agents/skills-src/mutation-testing/references/tools-by-language.md still documents a raw git diff | grep | paste command:

npx stryker run --mutate "$(git diff --name-only main...HEAD | grep ...)"

That command does not use scripts/stryker-pr-scope.mjs and only excludes *.test.ts. Because CLI --mutate overrides the config, it can still mutate the explicitly excluded files:

  • src/lib/server/testuser.ts
  • src/routes/terms/+page.ts
  • src/routes/privacy/+page.ts
  • src/routes/dpa/+page.ts

It also passes an empty string to --mutate for docs-only diffs, despite the new documented requirement to skip the run when the scope is empty.

Please update this generic reference example to call the shared scope script and guard the invocation, for example:

SCOPE=$(node scripts/stryker-pr-scope.mjs) &&
  if [ -n "$SCOPE" ]; then
    npx stryker run --mutate "$SCOPE"
  else
    echo "No mutable source files changed; skipping Stryker."
  fi

Alternatively, keep the example as pseudocode but explicitly state that it is illustrative only and must not be used for this repository. As written, the docs still provide the exact unsafe workflow that this PR claims to eliminate.

@coderabbitai

coderabbitai Bot commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

Tip

For best results, initiate chat on the files or code changes.

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

printf '%s\n' '--- follow-up commit summary ---'
git show --stat --oneline --decorate --no-renames 9f10007

printf '%s\n' '--- follow-up commit diff ---'
git show --format=fuller --find-renames --find-copies --no-ext-diff 9f10007 -- \
  scripts/stryker-pr-scope.mjs \
  scripts/stryker-pr-scope.test.mjs \
  stryker.config.json \
  .github/workflows/mutation.yml \
  AGENTS.md \
  .agents/skills-src/mutation-testing/SKILL.md \
  .agents/skills-src/mutation-testing/references/tools-by-language.md

printf '%s\n' '--- source and test structure ---'
ast-grep outline scripts/stryker-pr-scope.mjs --items all 2>/dev/null || true
ast-grep outline scripts/stryker-pr-scope.test.mjs --items all 2>/dev/null || true

printf '%s\n' '--- remaining mutation scope invocations and exclusions ---'
rg -n -C 3 --glob '!*node_modules*' \
  'stryker-pr-scope|--mutate|mutate["'\'']?\s*:|ignoreStatic|git diff --name-only' \
  stryker.config.json .github/workflows/mutation.yml AGENTS.md .agents scripts 2>/dev/null || true

Length of output: 34022


@Bonobo791 Verified 9f10007.

scripts/stryker-pr-scope.mjs filters the PR scope to the same effective set as stryker.config.json. It excludes test files, src/lib/server/testuser.ts, and the legal-page loaders.

The workflow and repository-specific PR instructions now use the shared script. The workflow skips Stryker when the computed scope is empty.

scripts/stryker-pr-scope.test.mjs covers included files, exclusions, mixed diffs, and empty diffs.

I found no blocking issue in the inspected follow-up changes. 🐇


✏️ Learnings added
Learnt from: Bonobo791
URL: https://github.com/Bonobo791/Moderaty/pull/103

Timestamp: 2026-08-05T16:24:59.602Z
Learning: In the Moderaty repository, use `node scripts/stryker-pr-scope.mjs` for PR-scale StrykerJS mutation scopes. The script computes the changed-versus-main TypeScript files under `src/`, mirrors the `stryker.config.json` exclusions, prints a comma-separated `--mutate` scope, and produces empty output when the Stryker run must be skipped.

Note: Learnings are effective only in the context of similar code segments. To apply general coding guidelines, please configure review instructions. You can manage existing learnings in the app.

You are interacting with an AI system.

@qodo-code-review

Copy link
Copy Markdown

Review triage follow-up

@CodeAnt-AI — your consistency catch was right on both counts, fixed in 9f10007:

  • The documented commands filtered only *.test.ts, so a CLI --mutate (which overrides the config globs) could still mutate testuser.ts and the legal-page loaders — and the docs claimed config parity they didn't have.
  • Docs-only diffs would have produced --mutate "" instead of a clean skip.

Adopted your preferred fix: a shared script, scripts/stryker-pr-scope.mjs — prints the changed-vs-main src scope filtered to the config's mutate set (the exclusions mirror stryker.config.json, noted in the file header), empty output means skip. The workflow and the documented local commands (SKILL.md, AGENTS.md) now both call it, so they cannot drift; tools-by-language.md's generic example explicitly warns about the override and the empty-scope case. Ships with behavior tests (scripts/stryker-pr-scope.test.mjs, 6 tests covering keeps/drops/join/empty), verified CLI-side both ways: empty line on a docs-only diff, filtered scope on a src-heavy diff. Gates: 525 tests green, svelte-check 0/0, codacy-analysis 0 issues on all changed files.

One process note, self-inflicted and fixed: while verifying I accidentally popped a stale stash from another branch (feat-reversible-actions), which conflicted ROADMAP.md; I restored the file to HEAD immediately and confirmed it byte-identical (git diff HEAD --quiet) and the stash entry intact. No content was lost or altered.

@coderabbitai — thanks for the independent verification run confirming the cbf7fe7 fixes; no blocking issues stands. @qodo-code-review — awaiting your re-review, nothing further from your side so far. @sonarqubecloud — the re-posted gate failure is the same unpinned-tag hotspot set already declined (matches db-backup.yml convention; repo-wide SHA pinning is a separate maintainer decision).

Holding for human review.

The scope-consistency issue is fixed: the shared script correctly filters excluded files and skips empty scopes, and finding 1 is implemented.

One item remains: finding 2 is still open. The workflow invokes npx stryker run ... --incremental without the required explicit --ignoreStatic flag. Although ignoreStatic is set in stryker.config.json, this finding requires the CLI invocation itself to include it. Add --ignoreStatic to .github/workflows/mutation.yml, then this re-review should be clear.

@codeant-ai

codeant-ai Bot commented Aug 5, 2026

Copy link
Copy Markdown

PR Code Suggestions ✨

Previous suggestions up to commit 9f10007
CategorySuggestion                                                                                                                                    SeverityGenerated at (UTC)
Api mismatch
Deleted or renamed source paths can be passed to Stryker as nonexistent mutation targets

The git diff --name-only result includes deleted paths and the old side of renames.
Those paths pass the filter and are forwarded to Stryker as mutation targets even
though they no longer exist in the checkout, causing the mutation step to fail for
PRs that delete or rename a mutable TypeScript file. Exclude deleted paths with an
appropriate diff filter or verify that each path still exists before adding it to
the scope.

scripts/stryker-pr-scope.mjs [59]

Why it matters? 🤔
  • ❌ Mutation CI can fail on source-file deletion PRs.
  • ⚠️ Changed-file mutation reports become unavailable.
  • ⚠️ Pull-request checks report tooling errors instead of results.

Fix in Cursor Fix in VSCode Claude

(Use Cmd/Ctrl + Click for best experience)

Prompt for AI Agent 🤖
This is a comment left during a code review.

**Path:** scripts/stryker-pr-scope.mjs
**Line:** 59:59
**Comment:**
	*Api Mismatch: The `git diff --name-only` result includes deleted paths and the old side of renames. Those paths pass the filter and are forwarded to Stryker as mutation targets even though they no longer exist in the checkout, causing the mutation step to fail for PRs that delete or rename a mutable TypeScript file. Exclude deleted paths with an appropriate diff filter or verify that each path still exists before adding it to the scope.

Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise.
Once fix is implemented, also check other comments on the same PR, and ask user if the user wants to fix the rest of the comments as well. if said yes, then fetch all the comments validate the correctness and implement a minimal fix
Major2026-08-05 16:25

Latest suggestions up to commit 95ccd59
CategorySuggestion                                                                                                                                    SeverityGenerated at (UTC)
Logic error
Local usage invokes Stryker with an empty mutation scope

The documented command invokes Stryker whenever scope computation succeeds,
including when the computed scope is empty. The surrounding contract explicitly
requires callers to skip Stryker for an empty scope, but --mutate "" can result in a
failed or unintended mutation invocation for docs-only or fully filtered changes.
Add a non-empty guard to the example.

scripts/stryker-pr-scope.mjs [34-35]

Why it matters? 🤔
  • ⚠️ Docs-only local runs invoke Stryker unnecessarily.
  • ⚠️ Empty scopes may produce failed or unintended mutation runs.
  • ⚠️ Local and CI mutation behavior diverge.

Fix in Cursor Fix in VSCode Claude

(Use Cmd/Ctrl + Click for best experience)

Prompt for AI Agent 🤖
This is a comment left during a code review.

**Path:** scripts/stryker-pr-scope.mjs
**Line:** 34:35
**Comment:**
	*Logic Error: The documented command invokes Stryker whenever scope computation succeeds, including when the computed scope is empty. The surrounding contract explicitly requires callers to skip Stryker for an empty scope, but `--mutate ""` can result in a failed or unintended mutation invocation for docs-only or fully filtered changes. Add a non-empty guard to the example.

Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise.
Once fix is implemented, also check other comments on the same PR, and ask user if the user wants to fix the rest of the comments as well. if said yes, then fetch all the comments validate the correctness and implement a minimal fix
Major2026-08-05 16:37
Api mismatch
Relative script invocation prevents the CLI from computing and printing the mutation scope

The executable check compares the absolute import.meta.url with a URL built directly
from the usually relative process.argv[1]. When invoked as documented with node
scripts/stryker-pr-scope.mjs, the URLs do not match, so the CLI body never runs and
produces no scope, causing local and workflow mutation runs to be skipped or receive
an empty scope. Resolve process.argv[1] to an absolute file URL before comparing.

scripts/stryker-pr-scope.mjs [63]

Why it matters? 🤔
  • ❌ CI skips mutation reports for every PR.
  • ❌ Local PR-scale mutation commands receive no scope.
  • ⚠️ Survivor reports are absent despite successful workflow completion.

Fix in Cursor Fix in VSCode Claude

(Use Cmd/Ctrl + Click for best experience)

Prompt for AI Agent 🤖
This is a comment left during a code review.

**Path:** scripts/stryker-pr-scope.mjs
**Line:** 63:63
**Comment:**
	*Api Mismatch: The executable check compares the absolute `import.meta.url` with a URL built directly from the usually relative `process.argv[1]`. When invoked as documented with `node scripts/stryker-pr-scope.mjs`, the URLs do not match, so the CLI body never runs and produces no scope, causing local and workflow mutation runs to be skipped or receive an empty scope. Resolve `process.argv[1]` to an absolute file URL before comparing.

Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise.
Once fix is implemented, also check other comments on the same PR, and ask user if the user wants to fix the rest of the comments as well. if said yes, then fetch all the comments validate the correctness and implement a minimal fix
Major2026-08-05 16:37

Repository owner deleted a comment from codeant-ai Bot Aug 5, 2026
…gnore-scripts + local bins in CI (codeant, sonarqube)

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Warning

CodeRabbit couldn't request changes on this pull request because it doesn't have sufficient GitHub permissions.

Please grant CodeRabbit Pull requests: Read and write permission and re-run the review.

👉 Steps to fix this

Actionable comments posted: 4

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.agents/skills-src/mutation-testing/references/agent-mutation-loop.md:
- Line 33: Update the validation instruction in the mutation-testing loop to
specify that Stryker’s --mutate option must target the production file
containing the target mutant, not the generated test file. Preserve the existing
scoped re-run and survived-to-killed validation requirements.

In @.agents/skills-src/mutation-testing/references/tools-by-language.md:
- Line 71: Replace both PR-mode Stryker examples with the shared scope helper
command node scripts/stryker-pr-scope.mjs, ensuring the generated mutation scope
excludes test files, testuser.ts, and excluded legal page loaders. Add a guard
so Stryker is not run when the helper returns an empty scope, avoiding an empty
--mutate value.

In @.agents/skills-src/mutation-testing/SKILL.md:
- Around line 74-76: Update the mutation-testing instruction to explicitly pass
the production file containing the suspected mutant to Stryker’s --mutate
option, rather than referring ambiguously to “that file” or the generated test
file. Preserve the existing guidance to inspect survivors and add or rerun kill
tests.

In `@AGENTS.md`:
- Around line 63-65: Update the Stryker CI description in AGENTS.md to state
that scope is computed for every pull request, while Stryker runs only when
mutable source files changed; mention that docs-only or fully filtered pull
requests receive a skip notice.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 7fdc1650-c711-43f9-9382-ea15907d62cb

📥 Commits

Reviewing files that changed from the base of the PR and between 5cbf018 and 9f10007.

📒 Files selected for processing (9)
  • .agents/skills-src/mutation-testing/SKILL.md
  • .agents/skills-src/mutation-testing/references/agent-mutation-loop.md
  • .agents/skills-src/mutation-testing/references/tools-by-language.md
  • .github/workflows/mutation.yml
  • .gitignore
  • AGENTS.md
  • scripts/stryker-pr-scope.mjs
  • scripts/stryker-pr-scope.test.mjs
  • stryker.config.json

4. **Classify** every survivor: genuine gap vs equivalent vs no-coverage (see surviving-mutant-triage.md). Discard equivalents before prompting — they poison the loop.
5. **Prompt with the survivors**: for each genuine survivor, describe the mutant concretely and require a test that fails under it and passes on the original.
6. **Validate generated tests** before adding them: syntax/compile clean, passes on the original code, and (sampled) fails under its target mutant. Discard tests that only pass in both worlds — they assert nothing about the mutation.
6. **Validate generated tests** before adding them: syntax/compile clean, passes on the original code, and flips its target mutant survived→killed on a scoped Stryker re-run (`--mutate` the touched file; sample across rounds rather than re-running per test). Discard tests that only pass in both worlds — they assert nothing about the mutation.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

Name the production mutation target explicitly.

Line 33 says to --mutate the touched file after generating a test. That wording can select the generated test file, while Stryker must mutate the production file containing the target mutant. Change the instruction to --mutate the production file containing the target mutant.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.agents/skills-src/mutation-testing/references/agent-mutation-loop.md at
line 33, Update the validation instruction in the mutation-testing loop to
specify that Stryker’s --mutate option must target the production file
containing the target mutant, not the generated test file. Preserve the existing
scoped re-run and survived-to-killed validation requirements.

Comment thread .agents/skills-src/mutation-testing/references/tools-by-language.md Outdated
Comment thread .agents/skills-src/mutation-testing/SKILL.md
Comment thread AGENTS.md
Comment on lines +63 to +65
verdict. CI runs a report-only Stryker pass over the PR's changed source
files on every PR
(`.github/workflows/mutation.yml`, report uploaded as an artifact); wiring a

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Describe the empty-scope behavior accurately.

The workflow starts on every pull request, but the Stryker step runs only when steps.scope.outputs.mutate != ''. Docs-only and fully filtered pull requests receive a skip notice. Change the text to state that CI computes the scope on every pull request and runs Stryker only when mutable source files changed.

🧰 Tools
🪛 LanguageTool

[uncategorized] ~65-~65: The official name of this software platform is spelled with a capital “H”.
Context: ... PR's changed source files on every PR (.github/workflows/mutation.yml, report uploade...

(GITHUB)

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@AGENTS.md` around lines 63 - 65, Update the Stryker CI description in
AGENTS.md to state that scope is computed for every pull request, while Stryker
runs only when mutable source files changed; mention that docs-only or fully
filtered pull requests receive a skip notice.

@Bonobo791

Copy link
Copy Markdown
Owner Author

Review triage follow-up

@CodeAnt-AI — correct again: the generic reference still showed the raw git diff | grep | paste command as its primary example, undercutting the shared-script guarantee. Fixed in 9b05190: tools-by-language.md's StrykerJS PR-mode lever now leads with the guarded shared-script form (SCOPE=$(node scripts/stryker-pr-scope.mjs) + explicit skip on empty), demotes the raw pattern to "illustrative only, must NOT be used verbatim in Moderaty", and the CI YAML example carries the same pointer to .github/workflows/mutation.yml.

@sonarqubecloud — I owe you a correction: I declined the C security rating as an unpinned-action-tags issue, but the actual findings were different and valid. Fixed in 9b05190:

  • npm ci without --ignore-scripts → now npm ci --ignore-scripts. Verified no dependency needs its lifecycle script: the only hasInstallScript entries in the lockfile are esbuild (binaries arrive via optionalDependencies, the postinstall is validation-only) and fsevents (macOS-only; CI is ubuntu). If that reasoning is wrong, the workflow on this very PR fails loudly and we'll see it.
  • npx on-demand install / unverified releases → both invocations replaced with the exact local binaries (./node_modules/.bin/svelte-kit sync, ./node_modules/.bin/stryker run), verified present and working locally. npx no longer appears in the workflow.

Note for the maintainer: db-backup.yml has the same npm ci/npx-free pattern already except npm ci without --ignore-scripts — worth a follow-up decision repo-wide.

@qodo-code-review, @codacy-production, @netlify — nothing new from you this round; prior verdicts stand.

Gates on 9b05190: 525 tests green, svelte-check 0/0, codacy-analysis 0 issues, skill re-synced. Holding for human review.

@codeant-ai

codeant-ai Bot commented Aug 5, 2026

Copy link
Copy Markdown

Question: the generic reference still showed the raw git diff | grep | paste command as its primary example, undercutting the shared-script guarantee. fixed in 9b05190: tools-by-language.md's strykerjs pr-mode lever now leads with the guarded shared-script form (scope=$(node scripts/stryker-pr-scope.mjs) + explicit skip on empty), demotes the raw pattern to "illustrative only, must not be used verbatim in moderaty", and the ci yaml example carries the same pointer to .github/workflows/mutation.yml.

@sonarqubecloud — i owe you a correction: i declined the c security rating as an unpinned-action-tags issue, but the actual findings were different and valid. fixed in 9b05190:

  • npm ci without --ignore-scripts → now npm ci --ignore-scripts. verified no dependency needs its lifecycle script: the only hasinstallscript entries in the lockfile are esbuild (binaries arrive via optionaldependencies, the postinstall is validation-only) and fsevents (macos-only; ci is ubuntu). if that reasoning is wrong, the workflow on this very pr fails loudly and we'll see it.
  • npx on-demand install / unverified releases → both invocations replaced with the exact local binaries (./node_modules/.bin/svelte-kit sync, ./node_modules/.bin/stryker run), verified present and working locally. npx no longer appears in the workflow.

note for the maintainer: db-backup.yml has the same npm ci/npx-free pattern already except npm ci without --ignore-scripts — worth a follow-up decision repo-wide.

@qodo-code-review, @codacy-production, @netlify — nothing new from you this round; prior verdicts stand.

gates on 9b05190: 525 tests green, svelte-check 0/0, codacy-analysis 0 issues, skill re-synced. holding for human review.

Answer:

Blocking: stryker-pr-scope.mjs does not execute when invoked from the documented command

The entry-point check is incorrect for the documented invocation:

node scripts/stryker-pr-scope.mjs

When Node is invoked with that relative path, process.argv[1] is typically:

scripts/stryker-pr-scope.mjs

but import.meta.url is an absolute file:///.../scripts/stryker-pr-scope.mjs URL. Constructing:

new URL(`file://${process.argv[1]}`)

does not reliably produce the same absolute file URL (for example, file://scripts/stryker-pr-scope.mjs is parsed with scripts as the host). Consequently, isMain can be false, the changedFiles() call never runs, and the script emits no scope.

That would cause the PR workflow to treat every PR as having no mutable files and skip Stryker, while the documented local command also silently does nothing.

Use a filesystem-path comparison instead, for example:

import { fileURLToPath } from 'node:url';
import { resolve } from 'node:path';

const isMain =
  process.argv[1] &&
  fileURLToPath(import.meta.url) === resolve(process.argv[1]);

Alternatively, use a standard import.meta.url/process.argv[1] comparison after normalizing both paths. A regression test should invoke the script as a subprocess with the relative path and verify that it prints the expected scope.

@qodo-code-review

Copy link
Copy Markdown

Review triage follow-up

@CodeAnt-AI — correct again: the generic reference still showed the raw git diff | grep | paste command as its primary example, undercutting the shared-script guarantee. Fixed in 9b05190: tools-by-language.md's StrykerJS PR-mode lever now leads with the guarded shared-script form (SCOPE=$(node scripts/stryker-pr-scope.mjs) + explicit skip on empty), demotes the raw pattern to "illustrative only, must NOT be used verbatim in Moderaty", and the CI YAML example carries the same pointer to .github/workflows/mutation.yml.

@sonarqubecloud — I owe you a correction: I declined the C security rating as an unpinned-action-tags issue, but the actual findings were different and valid. Fixed in 9b05190:

  • npm ci without --ignore-scripts → now npm ci --ignore-scripts. Verified no dependency needs its lifecycle script: the only hasInstallScript entries in the lockfile are esbuild (binaries arrive via optionalDependencies, the postinstall is validation-only) and fsevents (macOS-only; CI is ubuntu). If that reasoning is wrong, the workflow on this very PR fails loudly and we'll see it.
  • npx on-demand install / unverified releases → both invocations replaced with the exact local binaries (./node_modules/.bin/svelte-kit sync, ./node_modules/.bin/stryker run), verified present and working locally. npx no longer appears in the workflow.

Note for the maintainer: db-backup.yml has the same npm ci/npx-free pattern already except npm ci without --ignore-scripts — worth a follow-up decision repo-wide.

@qodo-code-review, @codacy-production, @netlify — nothing new from you this round; prior verdicts stand.

Gates on 9b05190: 525 tests green, svelte-check 0/0, codacy-analysis 0 issues, skill re-synced. Holding for human review.

Thanks for the follow-up. The reported 9b05190 changes address the shared-scope and workflow hardening points described. One Qodo item remains open: finding 2 (.github/workflows/mutation.yml) still flags the Stryker invocation because it does not explicitly pass --ignoreStatic; setting it in stryker.config.json may not satisfy the repository’s CLI-invocation policy. The prior finding 1 is implemented.

@Bonobo791

Copy link
Copy Markdown
Owner Author

Review triage follow-up

@qodo-code-review — understood: the invocation policy keys on the CLI invocation itself, and the config setting alone doesn't satisfy it. Fixed in the latest push: the workflow's Stryker step now passes --ignoreStatic explicitly, and so do every documented CLI invocation — the scoped-audit and PR-scale commands in SKILL.md and the guarded shared-script example in tools-by-language.md (which now also states the policy: every CLI Stryker invocation carries the flag). The config keeps "ignoreStatic": true as the behavior default; the flag makes each invocation self-evidently compliant. Verified the flag on a real run (96.23%, 51/53 on the smoke module). Gates: 525 tests green, codacy-analysis 0 issues, skill re-synced.

Holding for human review.

@codeant-ai

codeant-ai Bot commented Aug 5, 2026

Copy link
Copy Markdown

Thanks for using CodeAnt! 🎉

We're free for open-source projects. if you're enjoying it, help us grow by sharing.

Share on X ·
Reddit ·
LinkedIn

@qodo-code-review

Copy link
Copy Markdown

Review triage follow-up

@qodo-code-review — understood: the invocation policy keys on the CLI invocation itself, and the config setting alone doesn't satisfy it. Fixed in the latest push: the workflow's Stryker step now passes --ignoreStatic explicitly, and so do every documented CLI invocation — the scoped-audit and PR-scale commands in SKILL.md and the guarded shared-script example in tools-by-language.md (which now also states the policy: every CLI Stryker invocation carries the flag). The config keeps "ignoreStatic": true as the behavior default; the flag makes each invocation self-evidently compliant. Verified the flag on a real run (96.23%, 51/53 on the smoke module). Gates: 525 tests green, codacy-analysis 0 issues, skill re-synced.

Holding for human review.

Thanks for the update. The reported changes address finding 2: the workflow and documented CLI invocations now pass --ignoreStatic explicitly, while retaining the config default. The smoke-run result and validation signals are consistent with the fix. Human review can proceed; no further action is needed from me on this finding.

@codacy-production

codacy-production Bot commented Aug 5, 2026

Copy link
Copy Markdown

Up to standards ✅

🟢 Issues 0 issues

Results:
0 new issues

View in Codacy

🟢 Metrics 18 complexity · 0 duplication

Metric Results
Complexity 18 (≤ 100 complexity)
Duplication 0 (≤ 1 duplication)

View in Codacy

NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.

@codeant-ai codeant-ai Bot added size:L This PR changes 100-499 lines, ignoring generated files and removed size:L This PR changes 100-499 lines, ignoring generated files labels Aug 5, 2026
@Bonobo791

Copy link
Copy Markdown
Owner Author

Review triage follow-up

@CodeAnt-AI (entry-point check finding) — valid in principle, fixed. On Node 24 process.argv[1] is resolved absolute, so the documented relative invocation happened to work (which is why my CLI spot-checks passed); but the new URL(\file://${argv[1]}`)construction genuinely breaks on paths needing encoding — reproduced with a#in the script's directory, whereisMainevaluated false and the script printed nothing at all. The check is nowfileURLToPath(import.meta.url) === resolve(process.argv[1])`, exactly your suggested shape.

Test-first per the repo rule: two subprocess regression tests in scripts/stryker-pr-scope.test.mjs — the documented relative invocation, and the #-in-path case that goes red on the old comparison and green on the new one (confirmed both directions). Gates: 527 tests green, svelte-check 0/0, codacy-analysis 0 issues on both files.

Holding for human review.

@codeant-ai

codeant-ai Bot commented Aug 5, 2026

Copy link
Copy Markdown

Question: //${argv[1]}`)construction genuinely breaks on paths needing encoding — reproduced with a#in the script's directory, whereismainevaluated false and the script printed nothing at all. the check is nowfileurltopath(import.meta.url) === resolve(process.argv[1])`, exactly your suggested shape.

test-first per the repo rule: two subprocess regression tests in scripts/stryker-pr-scope.test.mjs — the documented relative invocation, and the #-in-path case that goes red on the old comparison and green on the new one (confirmed both directions). gates: 527 tests green, svelte-check 0/0, codacy-analysis 0 issues on both files.

holding for human review.

Answer:
No blocking issues found.

The isMain fix is correct: comparing fileURLToPath(import.meta.url) with resolve(process.argv[1]) avoids URL parsing problems for paths containing #, spaces, or other characters requiring encoding. The two subprocess tests cover both the documented relative invocation and the regression case, and the scope filtering mirrors the configured Stryker exclusions.

The CI workflow also correctly:

  • creates a local main ref before computing the diff;
  • skips Stryker for docs-only or otherwise non-mutable changes;
  • avoids passing an empty --mutate value;
  • runs Stryker in report-only mode without a survivor threshold;
  • caches the incremental report and uploads HTML/JSON artifacts.

One minor edge case remains: joining filenames with commas assumes changed paths cannot contain commas, since Stryker parses the --mutate value as a comma-separated list. That is not normally relevant for this repository, but could be avoided in the future by rejecting or separately handling such paths. Otherwise, the implementation and regression coverage look sound.

@sonarqubecloud

sonarqubecloud Bot commented Aug 5, 2026

Copy link
Copy Markdown

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cla-signed size:L This PR changes 100-499 lines, ignoring generated files

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant