feat(telemetry): track which rules users disable and suppress

[rayhanadev]

Why

Rule rejection — turning a rule off in config or silencing a finding with an inline comment — is the strongest false-positive signal we have, and none of it was measurable: no rule identity ever rode telemetry for silenced rules.

Before:

Wide event: scan.rulesDisabled = 3        # a count — which 3? unknown
Pipeline:   isRuleIgnored(...) -> return null   # silent drop, no tally
Metrics:    rule.fired only                # survivors only; suppressed findings invisible

After:

rule.disabled    counter — once per scan per config off-switch, keyed rule + source ("rules" | "ignore")
rule.suppressed  counter — findings the pipeline dropped per user intent, keyed rule + source ("config" | "override" | "inline")
Wide event:      diag.suppressed / diag.suppressedConfig / diag.suppressedOverride / diag.suppressedInline

Group either counter by rule in Sentry for the rule-rejection leaderboard; the diag.suppressed* dims make per-scan suppression rates queryable in Trace Explorer.

What changed

• Two counters are needed because the mechanisms never meet: rules: "off" entries are stripped from the generated oxlint config upstream (runners/oxlint/config.ts) — those rules never run, so only the scan-level rule.disabled counter can see them. ignore.rules, per-path ignore.overrides, and inline disables are enforced post-lint in the pipeline, where rule.suppressed tallies them per finding.
• buildDiagnosticPipeline now counts drops at exactly the four user-intent sites via a suppress(diagnostic, source) helper and exposes summarizeSuppressions(). Engine-owned drops (test-file auto-suppression, the library gate, ignore.files patterns, the warnings hide, textComponents/runtimeGlobals knobs) are deliberately not counted — they say nothing about the user rejecting a specific rule.
• Tallies thread InspectOutput → CachedScanPayload (SCAN_RESULT_CACHE_SCHEMA_VERSION 2 → 3) so cache hits replay the same telemetry. Public InspectResult and the JSON report are untouched.
• New core export canonicalizeUserRuleKey (in rule-key-aliases.ts, wrapping the existing canonicalizer) so react/jsx-key, bare jsx-key, and react-doctor/jsx-key group under one rule attribute.
• Attributes carry only rule keys and fixed source tokens — the same exposure class as the existing rule.fired — and pass through beforeSendMetric scrubbing like every other metric.

Test plan

• pnpm typecheck — 14/14 tasks green.
• New tests: pipeline source attribution for all four drop sites plus the not-counted ignore.files boundary (packages/core/tests/merge-and-filter-diagnostics.test.ts), summarizeDisabledRules canonicalization/dedup (record-scan-metrics.test.ts), wide-event rollup incl. absent-vs-zero semantics (build-run-event.test.ts).
• Full suites: react-doctor 2039 passed; core green except the two known env-file tests that fail under a global .env* gitignore (pre-existing environment trap, unrelated).
• pnpm lint / pnpm format:check clean.

🤖 Generated with Claude Code

---

Note

Low Risk
Low risk: telemetry-only paths and cache schema bump; diagnostic filtering behavior is unchanged aside from counting at existing drop sites.

Overview
Adds anonymized telemetry so rule rejection (config off-switches, ignores, overrides, inline disables) is measurable per canonical rule key.

The diagnostic pipeline now tallies user-intent drops via suppress() and summarizeSuppressions() (config / override / inline), while engine-owned filters stay untracked. rule.suppressed counters and diag.suppressed* wide-event rollups carry those tallies; rule.disabled records once per scan for rules: "off" and ignore.rules (including rules stripped from oxlint before they fire). canonicalizeUserRuleKey groups legacy and short rule spellings in metrics.

Tallies flow through InspectOutput, CachedScanPayload (cache schema v3), scan metrics, and run events. Public InspectResult / JSON reports are unchanged.

^{Reviewed by Cursor Bugbot for commit eb41a4d. Bugbot is set up for automated code reviews on this repo. Configure here.}

[pkg-pr-new]

Open in StackBlitz

npm i https://pkg.pr.new/eslint-plugin-react-doctor@1016

npm i https://pkg.pr.new/oxlint-plugin-react-doctor@1016

npm i https://pkg.pr.new/react-doctor@1016

commit: eb41a4d

[Copilot]

Pull request overview

This PR adds telemetry to measure rule rejection by tracking (1) which rules users disable via config and (2) which findings get suppressed by user intent during the diagnostic filtering pipeline, while keeping the public InspectResult and JSON report unchanged.

Changes:

• Introduces rule.disabled and rule.suppressed counters (keyed by canonical rule + source) and threads suppression tallies through the scan payload/cache for replay on cache hits.
• Extends the core diagnostic pipeline to tally user-intent suppression sites and exposes summarizeSuppressions() via InspectOutput.
• Adds canonicalization for user-provided rule keys (canonicalizeUserRuleKey) plus tests for canonicalization, suppression attribution, and wide-event rollups.

Reviewed changes

Copilot reviewed 15 out of 15 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
packages/react-doctor/tests/scan-result-cache.test.ts	Updates cache test expectations for new cached payload fields.
packages/react-doctor/tests/record-scan-metrics.test.ts	Adds tests for summarizeDisabledRules canonicalization/dedup behavior.
packages/react-doctor/tests/build-run-event.test.ts	Adds tests for diag.suppressed* rollups and absent-vs-zero semantics.
packages/react-doctor/src/inspect.ts	Threads suppressedRuleCounts into cached payload, scan metrics, and run events.
packages/react-doctor/src/cli/utils/scan-result-cache.ts	Persists suppression tallies in cached scan payload (schema bump support).
packages/react-doctor/src/cli/utils/record-scan-metrics.ts	Emits rule.disabled and rule.suppressed metrics; adds disabled-rule summarizer.
packages/react-doctor/src/cli/utils/constants.ts	Bumps scan cache schema version; adds new metric names.
packages/react-doctor/src/cli/utils/build-run-event.ts	Rolls suppression tallies into wide-event diag.suppressed* attributes.
packages/core/tests/merge-and-filter-diagnostics.test.ts	Adds tests validating suppression-source attribution and non-counted boundaries.
packages/core/src/types/index.ts	Re-exports SuppressedRuleCount type.
packages/core/src/types/diagnostic.ts	Introduces SuppressedRuleCount telemetry-only type.
packages/core/src/run-inspect.ts	Adds suppressedRuleCounts to InspectOutput and populates it from the pipeline.
packages/core/src/rule-key-aliases.ts	Adds canonicalizeUserRuleKey for config-key canonicalization.
packages/core/src/build-diagnostic-pipeline.ts	Adds suppression tallying + summarizeSuppressions() implementation.
.changeset/rule-ignore-telemetry.md	Publishes the telemetry feature as a react-doctor patch changeset.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[devin-ai-integration]

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no bugs or issues to report.