Tags: profullstack/threatcrush
Tags
feat(scan): LDAP/XPath/NoSQL/host-header rules; fix two that never fi… …red (#104) Detection coverage 79.8% to 83.0% (103 to 107 of 129), FPR still 0%, and zero new findings on capacitor (10), the self-scan (67), ShortsStudio (0) or koajs/router (0). Two of the rules shipped in 0.10.0 passed their unit tests but did nothing on the real corpus — the tests used simplified fixtures. Both fixed, and the tests now reproduce the real conditions: - py-ldap-injection: the whole fixture is under `if False:` with a module-level `from ldap.filter import escape_filter_chars`, and the `/escap/i` guard matched that import in every window — exonerating the unescaped filter too. The guard is now an escaper *call* (`escape…(`), which the import is not. - py-xpath-injection: the `"`-delimited f-string contains a `'` (`text()='{x}'`), and a class excluding both quotes stopped before the interpolation. Now one variant per delimiter quote. Two new, precise rules: - js-nosql-injection: a request object passed straight to `find`/`update` (CWE-943) — a Mongo operator-injection / auth bypass. - js-host-header-trust: a URL built from `req.headers.host` (CWE-346) — the password-reset-poisoning shape. Also built and dropped: a recursive-merge prototype-pollution rule. The bare `target[key] = source[key]` copy is the safe allow-listed shape (`updates[field] = body[field]` over an `allowedFields` list) as often as the vulnerable one — whether it is a sink depends on the key's origin and which of endless guard idioms filters it, which line matching cannot decide. It flagged legitimate merges in Capacitor and in this repo's own web app, so it is left to KNOWN_GAPS; `js-prototype-pollution` still catches the explicit `__proto__` literal. Gate floor moves 76 to 80. 157 tests, up from 154. Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
feat(scan): 13 misconfiguration, weak-crypto and injection rules (#103) Raises detection coverage against the testbed from 71.3% to 79.8% (92 to 103 of 129), false-positive rate still 0%, and — the number that matters for a tool people choose to install — zero new findings on capacitor (10), the self-scan (67), ShortsStudio (0) or koajs/router (0). Each rule targets a defect the line itself shows, so the corrected shape in the corpus differs on something visible here rather than three functions away: py-framework-debug-enabled debug mode left on (CWE-489) js-cors-wildcard-credentials `*` origin with credentials (CWE-942) js-cookie-insecure-flag cookie with secure:false (CWE-614) js-hardcoded-crypto-key key literal in crypto code (CWE-321) py-hardcoded-secret-key signing secret literal (CWE-798) py-ldap-injection unescaped LDAP filter (CWE-90) py-xpath-injection interpolated XPath (CWE-643) py-fast-password-hash SHA-2 on a password, no KDF (CWE-759) py-plaintext-password-retained raw password stored (CWE-256) js-timing-unsafe-mac-compare === on a signature (CWE-208) js-predictable-cipher-iv static IV (CWE-329) js-mass-assignment Object.assign(x, req.body) (CWE-915) js-header-injection request input into a response header (CWE-113) Two decisions in service of "safe to install": - The timing-unsafe compare first flagged the corpus's own safe line — a `.length ===` length check whose `timingSafeEqual` sits forward of the backward guard window. A length comparison is not the value compare, so the rule now excludes `.length`. - A `Math.random().toString(36)` token rule was written and then dropped: the exact shape generates security tokens AND benign callback ids (Capacitor's native bridge does the latter), with no line-visible signal between them. Three false positives on a real repo is precisely what gets a scanner uninstalled, so the bare shape is left to the credential-scoped rule that already covers the `token = …Math.random…` case. Deliberately still absent: CSRF, IDOR, TOCTOU, missing authorization, session fixation, and the sanitize-upstream classes (CSV/log injection) whose vulnerable and safe lines are identical — they need whole-function reasoning this engine does not fake. See KNOWN_GAPS. Gate floor moves 68 to 76. 154 tests, up from 141. Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
feat(scan): weak-crypto rules for Python (ciphers, hashes, predictabl… …e seed) (#102) Raises detection coverage against the testbed from 65.9% to 71.3% (85 to 92 of 129), false-positive rate still 0%. The CWE-327/338 cluster was the largest recoverable gap: primitives threatcrush had generic rules for, but scoped to a credential on the matched line, so Python's integrity- and confidentiality-use cases slipped through. Three Python rules, each keyed on a signal that survives without trusting a name: - py-broken-cipher: DES/RC2/RC4/Blowfish construction, and AES in ECB mode. No safe use, so inherent — flagged wherever it appears. AES matches only on ECB, leaving GCM/CTR/CBC alone. - py-weak-hash: hashlib.md5/sha1, unless the line carries Python's own `usedforsecurity=False` opt-out for a non-security digest. - py-predictable-random-seed: random.seed() from the clock or pid, which makes the whole sequence reproducible. A fixed integer seed (reproducible tests) is left alone. What is deliberately NOT added: a rule for a `random`-drawn token whose only signal is the enclosing function name (`generate_session_id`). Guard windows exclude definition lines on purpose — a name is not evidence, the same reason a `def sanitize_…` does not count as sanitisation — so there is no line-level signal to key on. The generic `insecure-randomness-for-secret` still catches the common `token = …random…` shape. Those name-only cases are the documented tail, pinned by a test that asserts they stay silent. The coverage gate floor moves 60 to 68 to lock the gain in; a regression below the new baseline now fails the job. No new findings on capacitor (10, unchanged) or the self-scan (67, unchanged) — the rules are Python-only and precise. 141 tests, up from 134. Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
feat(scan): path exclusion via --exclude and .threatcrushignore (v0.8… ….0) (#101) * feat(scan): path exclusion via --exclude and .threatcrushignore Release 0.8.0. Scanning the ThreatCrush repo with ThreatCrush reported 141 findings, and 57 of them were the scanner detecting its own reflection: the rule definitions (regexes and example strings that match the very patterns they describe) and the test fixtures (deliberately-vulnerable sample code). Real findings in the product were buried under them. Adds a general exclusion mechanism — useful to any consumer for generated output or vendored trees, not just the self-scan: - `--exclude <glob>` on `threatcrush scan`, repeatable - a `.threatcrushignore` file at the scan root, read automatically and merged with `--exclude` Globs are gitignore-flavoured: a bare name matches at any depth, a pattern with a slash is anchored to the root, `*` stays within a path segment and `**` crosses them. A directory match prunes the whole subtree. Excluding is not the same as finding nothing. `ScanReport.excluded` counts the skipped paths — a pruned directory once, not per file — and the CLI prints it, so a scan quieted by a broad glob cannot be mistaken for a clean one, the same guarantee `suppressed` already carries. A committed `.threatcrushignore` excludes this repo's rule sources and fixtures. The self-scan drops 141 to 67, and the residual is product code plus two example-config secrets — real targets, not reflections. The daemon and any scan honour the file too, since it is read in `scanPath`. Coverage gate unchanged (the testbed has no ignore file): TPR 65.9% / FPR 0%. 132 package tests, up from 125; the glob matcher is pinned in both directions (over- and under-matching) by its own tests. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(scan): match exclusion globs segment by segment (no ReDoS surface) CodeQL flagged js/polynomial-redos in the glob compiler added for --exclude, and it was right twice over: - whatever cross-segment regex the globs compiled to backtracks on a path full of slashes, the exact shape this scanner's own redos-nested-quantifier rule exists to catch; - the trailing-slash trim `pattern.replace(/\/+$/, '')` is unanchored, so `replace` retries at every start position and is quadratic on a pattern value that is all slashes. So the path is no longer run through one generated regex, and the trims no longer use a backtracking one. `compileExcludes` splits both the pattern and the path on `/` and aligns them with a two-pointer — the classic `**` match — where `**` consumes zero or more segments and every other part matches exactly one. Each per-segment matcher is a trivial `^…$` with single quantifiers (consecutive `*` are collapsed so no `[^/]*[^/]*` adjacency survives), tested against one bounded segment, never the whole path. Trailing slashes are trimmed with a loop rather than an unanchored regex. Semantics unchanged — every existing exclusion test still passes — plus cases pinning a trailing `**` (matches files beneath, not the directory itself) and that a 50,000-slash path and a 50,000-slash / 5,000-star pattern all resolve in under a millisecond. walk.ts carries no findings and needs no suppression; the ReDoS surface is gone, not silenced. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
fix: repair npm install (workspace: protocol) and the SQL verb false … …positive (v0.7.2) (#99) * fix(scan): require SQL structure so a verb-shaped word is not read as injection Release 0.7.2. The SQL keyword list matched bare verbs — INSERT, UPDATE, DELETE, DROP, a lone SELECT — each with a trailing word boundary. A word boundary sits at the hyphen in a React key `insert-${i}`, and after `Update` in a log line `Update finished in ${ms}ms`, so both read as SQL injection at critical severity. On ralyodio/ShortsStudio that was the only finding; on ionic-team/capacitor it was three of thirteen. Real SQL pairs the verb with the clause that makes it a statement: SELECT … FROM, INSERT INTO, UPDATE … SET, DELETE FROM, DROP TABLE. Requiring that structure keeps every injection shape the corpus and the unit tests exercise — all of which are SELECT … FROM or DELETE FROM — while a verb standing alone as prose no longer qualifies. The SELECT and UPDATE look-aheads are bounded to one string literal so the clause must be in the same statement. ShortsStudio: 1 finding to 0 (clean). capacitor: 13 to 10, removing all three sql-template-interpolation false positives and nothing else. Testbed coverage unchanged at TPR 65.9% / FPR 0% — no true positive lost. 125 tests, up from 123; the new false-positive test was confirmed to fail against the old pattern. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(cli): make @threatcrush/scan a devDependency so npm can install the CLI Extracting the scan engine (#94) added `@threatcrush/scan: workspace:*` to the CLI's dependencies. pnpm understands `workspace:*`; the release runs `npm publish`, which does not rewrite it — so the published package.json ships `workspace:*` verbatim, and every `npm install -g @profullstack/threatcrush` fails with EUNSUPPORTEDPROTOCOL. This broke all installs of 0.7.0 and 0.7.1, including the malware-test-prs scan workflow that installs @latest. The package is bundled into the CLI by tsup (`noExternal`), so it is a build-time dependency, not a runtime one — nothing requires it from the published artifact. Moving it to devDependencies is the correct classification and the fix: npm does not install a package's devDependencies, so the `workspace:*` spec is never resolved by consumers, while pnpm still links it for the build. Verified end to end: packed the 0.7.2 tarball and confirmed its published `dependencies` carries no `workspace:` spec; `npm install` of that tarball into a clean project succeeds (209 packages, no protocol error); @threatcrush/scan is not installed separately; and the installed CLI scans and detects correctly from the bundle alone. `pnpm install --frozen-lockfile` passes with the updated lockfile. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
ci: gate on detection coverage against the testbed corpus (#98) Adds a CI job that scores the CLI against profullstack/malware-test-prs — the public catalog of vulnerable/safe line pairs the rule set has been tuned against all along — and fails if the true-positive rate falls or the false-positive rate rises. Unit tests prove a rule fires on one hand-written line. This proves the whole set still catches what it caught across 67 real cases, and still stays silent on the 78 corrected implementations sitting beside them. A change that trades one false positive for three misses passes every unit test and fails here. At the pinned corpus commit the CLI scores TPR 65.9% (85/129) and FPR 0% (0/78) — up from the 15.6% baseline recorded in code-rules.ts when the code rules were only secret detection, with the false-positive rate held at zero throughout. The remaining misses are dominated by classes KNOWN_GAPS deliberately declines to approximate with line-oriented matching — CSRF, IDOR, TOCTOU, NoSQL injection, session fixation — so the floor is set at TPR 60 / FPR 2, under the current result with room for noise but not for a real regression. The corpus is pinned to a commit, not tracked to its default branch: a new upstream test case would otherwise lower the rate and fail this repository's gate on someone else's change. Bump the pin in a PR whose diff is the rule that covers the new case. Verified locally against the pinned commit: gate passes at 60/2, fails at 90/5, and scores identically from a clean master build. Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
feat(scan): command injection, SSRF and traversal for Java; traversal… … and template escaping for Go (v0.7.0) (#91) * feat(scan): command injection, SSRF and traversal for Java; traversal and template escaping for Go Release 0.7.0. Command injection, SSRF and path traversal were implemented for JavaScript and, in part, for Go, and never for Java. The same defect in the same codebase was reported or not depending on which file it lived in — Java had five rules to JavaScript's thirteen, and none of them covered a class this common. Java: Runtime.exec built by concatenation (CWE-78), outbound request to a computed URL (CWE-918), file path built from request data (CWE-22), broken cipher or ECB mode (CWE-327). Go: file path built from request data (CWE-22), value marked as pre-escaped HTML (CWE-79). Deliberately not added: TLS verification, weak hashing and insecure randomness. `tls-verification-disabled`, `weak-hash-on-credential` and `insecure-randomness-for-secret` are language-agnostic and already cover both languages, including Go's InsecureSkipVerify and Java's MessageDigest. Duplicating them per-language would double-report. Each rule is built against the corrected shape: the argv form of exec is not matched, a constant URL is not matched, a canonical-path containment check exonerates the traversal rules, AES/GCM is not matched, and template.HTML on a literal is not matched. Bare Cipher.getInstance("AES") is matched, because the JCE resolves it to AES/ECB/PKCS5Padding. Verified against ionic-team/capacitor: 146 Java files, findings unchanged at 13 — no false positives. The two concatenated `new File` sites there build a config path from a constant and are correctly left alone by needsContext. debtap unchanged at 8. 111 tests, up from 105. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * test: suppress the scanner's own findings on the new PHP and Java fixtures The self-scan flagged three lines of this file. All three are string literals holding sample code for another language — a PHP `eval` and a Java `Runtime.exec` — which the JavaScript rules match because the file they sit in is TypeScript. Same treatment as the existing fixtures: an inline disable naming the rule and why. The Java one is lifted into a variable first, so the suppression comment attaches to the line the finding is actually on rather than to the middle of a wrapped expression. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
PreviousNext