Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/example-report.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# DebtLens Report

Scanned **3** files with **36** rules in **162ms**.
Scanned **3** files with **35** rules in **162ms**.

## Summary

Expand Down
56 changes: 56 additions & 0 deletions docs/feature-flags-rfc.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
# Feature-flag debt RFC

Status: shipped with the opt-in `feature-flags` pack.

## Contract

The `stale-feature-flag` rule recognizes two conservative sources of flag identity:

1. top-level boolean constants whose names match `featureFlags.constantNamePatterns`; and
2. boolean properties or top-level boolean constants in files matched by
`featureFlags.registryGlobs` (paths are relative to the scan target).

Configured access patterns identify registry keys read through a call. `callee` is the
exact source-level callee text and `keyArgument` is a zero-based argument index. Only
string literals and no-substitution template literals are treated as keys.

```json
{
"pack": "feature-flags",
"featureFlags": {
"accessPatterns": [
{ "callee": "isEnabled", "keyArgument": 0 },
{ "callee": "featureClient.enabled", "keyArgument": 1 }
],
"registryGlobs": ["src/flags.ts", "packages/*/src/flags/**"],
"constantNamePatterns": ["^(?:enable|disable)[A-Z]"]
}
}
```

`accessPatterns` and `constantNamePatterns` replace their defaults when configured;
`registryGlobs` extend across root and package configuration. Supported glob operators
are `*`, `**`, and `?`. Defaults recognize `isEnabled(key)`, `useFlag(key)`, and
`flags(key)`, plus common flag-like top-level constant names. The rule remains opt-in.

## Findings

- A literal boolean definition is always-on/off only when its identifier, property, or
configured literal-key access participates in conditional control flow.
- A configured registry entry is unreferenced only after all scanned files are
aggregated. Cross-file constant references therefore do not become false unused
findings.
- If a configured access call uses a dynamic key, unreferenced-registry findings are
suppressed for that scan because the detector cannot prove which entry it reads.

## Non-goals

- No flag-provider SDK is inferred without configuration.
- No dynamic key, computed registry property, remote rollout state, flag age, or rollout
percentage is resolved.
- No dead branch is rewritten automatically.
- Registry formats other than TS/JS boolean constants and object properties are not
parsed in this version.

These limits favor missed findings over noisy cleanup advice. Add project-specific
patterns instead of broadening names globally.
47 changes: 47 additions & 0 deletions docs/parallel-scans.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
# Parallel scans and shared cache

DebtLens stays serial by default. For a large, CPU-heavy repository scan, enable
the bounded worker pool with either the automatic setting or an explicit size:

```sh
debtlens scan . --parallel
debtlens scan . --concurrency 4
```

Use `--concurrency 1` when comparing behavior, profiling startup, or running in a
single-CPU container. Parallel and serial scans have the same findings and stable
ordering. Cross-file rules still see the whole repository; they are not evaluated
independently on incomplete shards.

Workers help when detector work is large enough to repay startup and source
transfer. Small repositories, narrow `--changed` scans, and source-tree execution
through `tsx` may be faster with `--concurrency 1`. The published built CLI avoids
the per-worker TypeScript runtime startup cost.

## Restore the cache in CI

`--cache-dir` enables the scan cache and writes `cache.json` below the supplied
directory:

```sh
debtlens scan . --parallel --cache-dir .cache/debtlens
```

Save and restore `.cache/debtlens` with the CI provider's normal cache or artifact
mechanism. Cache entries use checkout-relative file identities and content hashes,
so a cache created at one runner's checkout path can hit after restoration at a
different path. The key also includes DebtLens version, selected rules, and all
finding-affecting rule configuration.

Do not share a writable cache directory between simultaneous scans. Each cache
file is atomically replaced, but the store is a last-writer-wins local artifact,
not a network coordination service. Give parallel jobs separate writable paths,
then let the CI cache service publish one completed artifact.

`--cache [path]` remains available for a specific cache file. Plugin-enabled scans
do not cache results because DebtLens cannot hash arbitrary plugin implementation
code. Plugin detectors also run in-process when worker concurrency is selected;
built-in file-local rules still use workers.

For the design contract and comparative benchmark, see
[`performance-rfc.md`](./performance-rfc.md).
91 changes: 91 additions & 0 deletions docs/performance-rfc.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
# Parallel scan and portable cache RFC

Status: implemented in the scanner core.

## Goals and invariants

Large scans may use Node worker threads, but concurrency must never change the
finding contract. For the same files, DebtLens version, selected rules, and rule
configuration, serial and parallel scans emit byte-identical `issues` arrays.
Worker scheduling is not observable in finding or warning order.

`--concurrency 1` is the reference serial implementation. `--parallel` selects a
CPU-based default capped at four workers; `--concurrency <n>` selects an explicit
pool size. Small scans can be slower in parallel because worker startup is real
work, so serial remains the default unless parallelism is requested.

## Sharding and aggregation model

Discovery and the canonical full source model are built by the coordinator.
File-local built-in rules are then given deterministic round-robin file shards.
Each worker parses its shard once and runs all selected file-local rules. The
coordinator concatenates shard findings in shard order and then restores detector
registry order before the existing normalization and stable issue sort.

Cross-file rules are an explicit aggregation phase. Rules whose result depends on
repository-wide duplicates, graphs, imports, or paired instruction files run once
on the coordinator with the complete file set. This includes `duplicate-logic`
(and language variants), `duplicated-literal`, `import-cycle`,
`test-duplication`, `story-only-component`, `config-drift`, and the AI instruction
duplication/contradiction rules, plus `stale-feature-flag`, whose registry
definitions and uses may live in different files. They are never run independently
on file shards.

Third-party plugin detectors are JavaScript functions and cannot be safely sent
through the structured-clone boundary. When worker concurrency is enabled,
built-in file-local rules use workers and plugin detectors retain the compatible
in-process path. Plugin findings are still merged in selected-rule order. Scan
caching remains disabled when plugins are loaded because their implementations
cannot be content-hash invalidated.

## Worker protocol and failure behavior

Workers receive source snapshots, clone-safe scan options, and built-in detector
IDs. They import the built-in registry themselves; detector functions are never
serialized. A response contains the detector ID, issues, warnings, and optional
profile timing. A worker error fails the scan instead of silently retrying with a
different correctness model.

The source tree uses the TypeScript worker entry under `tsx`; built packages use
the compiled JavaScript entry. Benchmarks use built JavaScript because loading a
TypeScript runtime in every development worker adds startup cost that consumers of
the published CLI do not pay.

## Portable cache contract

Cache format version 3 is intentionally incompatible with earlier absolute-path
entries. Its scan key is SHA-256 over:

- cache format version and DebtLens package version;
- checkout-root-relative scan target and changed-file identities;
- selected detector IDs and all finding-affecting scan/rule configuration.

The entry also stores a sorted scan manifest as target-relative file identity plus
SHA-256 content hash. Cache hits therefore require the same logical paths and
contents, while the checkout may be restored under a different absolute root.
Absolute target and cache paths are not persisted in the cached result; they are
rehydrated for the current invocation. Writes continue to use a temporary file
followed by an atomic rename.

Changing a file, rule configuration, selected rules, cache format, or DebtLens
version produces a miss. Concurrency is deliberately absent from the key because
it cannot affect findings.

## Verification and performance gate

Core tests compare serialized serial and parallel findings, exercise cross-file
rules, verify `--concurrency 1`, preserve deterministic warnings, and restore one
cache artifact into a different checkout root.

After `npm run build`, this command generates a 240-file CPU-oriented fixture,
warms both modes, alternates execution order, compares median timings, and rejects
any byte-level finding difference:

```sh
node scripts/benchmark.mjs --small-only --compare-parallel
```

The dedicated comparison requires at least a 1.05x median speedup by default.
`--runs` and `--min-speedup` make the sample count and machine-specific gate
explicit. The ordinary one-line benchmark fixtures retain their absolute runtime
budgets; they are intentionally not presented as evidence of parallel speedup.
27 changes: 26 additions & 1 deletion docs/rules.md
Original file line number Diff line number Diff line change
Expand Up @@ -1043,7 +1043,7 @@ When this is a false positive:
- deliberate fire-and-forget marked with `void`
- callbacks passed directly to APIs that manage promise lifecycle

Confidence: **0.65–0.82**. Higher when the callee is clearly async or returns `Promise<...>`.
Confidence: **0.68–0.88**. Higher when the callee is clearly async or returns `Promise<...>`.

## `commented-out-code`

Expand Down Expand Up @@ -1211,6 +1211,31 @@ When this is a false positive:

Confidence: **0.62**. Co-occurring domain synonyms are often legitimate vocabulary, so this rule stays advisory.

## `stale-feature-flag`

Flags feature flags that are hardcoded on/off in conditional control flow and configured
registry entries that are not referenced anywhere in the scan. This rule runs only in
the opt-in `feature-flags` pack (or when selected explicitly).

Configuration:

- `featureFlags.accessPatterns`: exact callee and zero-based literal-key argument shapes
- `featureFlags.registryGlobs`: registry paths/globs relative to the scan target
- `featureFlags.constantNamePatterns`: regexes for top-level boolean flag constants

Why it matters: completed rollouts leave unreachable branches and unused registry entries
that continue to tax testing and maintenance.

When this is a false positive:

- the literal is a deliberate build-time switch rather than a rollout flag
- a registry is consumed through an unconfigured provider or non-TypeScript manifest
- a computed/dynamic access cannot be attributed to one literal key

Dynamic configured accesses suppress unused-registry claims, and non-registry constants
must control a branch before they are reported. Confidence: **0.82–0.90**. See the
[feature-flag RFC](./feature-flags-rfc.md) for the exact contract and non-goals.

## `ai-instruction-duplication`

Flags the same normalized instruction block repeated across assistant instruction files such as `AGENTS.md`, `CLAUDE.md`, `.cursor/rules/**`, and `.github/copilot-instructions.md`.
Expand Down
45 changes: 45 additions & 0 deletions schema/debtlens.config.schema.json
Original file line number Diff line number Diff line change
Expand Up @@ -1396,6 +1396,51 @@
},
"additionalProperties": false
},
"featureFlags": {
"type": "object",
"description": "Configuration for the opt-in stale feature-flag detector.",
"properties": {
"accessPatterns": {
"type": "array",
"description": "Call shapes that read a literal feature-flag key.",
"items": {
"type": "object",
"additionalProperties": false,
"properties": {
"callee": {
"type": "string",
"minLength": 1
},
"keyArgument": {
"type": "integer",
"minimum": 0,
"default": 0
}
},
"required": [
"callee"
]
}
},
"registryGlobs": {
"type": "array",
"items": {
"type": "string",
"minLength": 1
},
"description": "Registry file globs relative to the scan target."
},
"constantNamePatterns": {
"type": "array",
"items": {
"type": "string",
"minLength": 1
},
"description": "Regexes identifying top-level boolean flag constants outside registries."
}
},
"additionalProperties": false
},
"failOn": {
"enum": [
"info",
Expand Down
Loading