fix(core): add args regex capture group for 13 languages (#1209) - #1212
Merged
Conversation
Ports #1199's fix (PR #1207, Python-only) to 13 more languages with the same zero-capture-group `args` regex precondition: c, cpp, go, kotlin, swift, php, perl, lua, apex, rust, solidity, scala, dart. Without a capture group, detector.py's _calculate_block_metrics falls back to args_match.group(0) -- the WHOLE match, including the keyword/name prefix -- and a zero/one-arg signature with no comma hits a whitespace- split fallback meant for genuinely space-separated languages, overcounting by +1 (e.g. `func main()` reported args=2 instead of 0). Each language got individual attention, not a blind port -- several needed more than "wrap the trailing parens": - c: also required recognizing `(void)` as 0 args, not 1 (the shared _count_top_level_args helper now filters a lone "void" segment). - cpp, kotlin, swift, rust, scala: have a second (or third) alternation branch for lambdas/closures/arrow functions with their own parameter span, each needed its own capture group to fix the same bug there too (e.g. cpp's `[]()`, scala's bare `x =>`). - Every language needed a name-capturing group added alongside the args group, purely so the existing per-language extraction tests (which assert the captured group contains the expected function name once any group exists) keep passing -- detector.py already resolves to the highest-numbered participating group via `lastindex`, so this doesn't affect counting. Also adds persisted regression coverage that didn't exist before (neither #1199 nor this fix had any until now, only ad hoc verification): a direct unit test for the shared `_count_top_level_args` helper (empty parens, trailing comma, bare `*`/`/`/`void` markers, nested brackets), and a per-language parametrized test running the real pipeline against a small fixed snippet per language, asserting exact `args` values. Remaining scope on #1209: csharp/java/javascript/typescript/groovy/ruby/ powershell (multi-branch regexes needing the same per-branch treatment, deferred as a distinct follow-up) and the 5-language Tier 2 (objective-c/ scheme/matlab/haskell/fortran, structurally unlike Python's model). Golden master fixtures regenerated via update_golden_master.py per the Differential Scan protocol -- the drift is the real, intentional args-count correction across the crucible corpus (e.g. Go's `handlePanic()` now correctly reports 0 params instead of 2), confirmed clean by crucible_check.py afterward (both full-precision and zero-dependency modes). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Contributor
5 tasks
squid-protocol
added a commit
that referenced
this pull request
Aug 10, 2026
…1216) Completes #1209's Tier 1 mechanical list: csharp, java, javascript, typescript, groovy, ruby, powershell (the remaining 7 of 20, following PR #1212's first 13). Same root cause as #1199/#1212: without a capture group isolating the parameter-list span, detector.py's _calculate_block_metrics falls back to the WHOLE regex match, and a zero/one-arg signature with no comma hits a whitespace-split fallback meant for space-separated languages, overcounting by +1. These 7 are the genuinely multi-branch regexes flagged as a distinct, costlier follow-up when PR #1212 stopped -- each has 3-4 real alternatives (standard method, constructor, lambda/arrow-function, class-method shorthand) that all needed their own capture group, not just one: - csharp/java: standard method + constructor + lambda/method-ref branches. - javascript/typescript: function decl + arrow function + class-method shorthand (typescript also has a bare-identifier arrow branch). - groovy: standard method + closure (`->`), where the closure branch's params are entirely optional -- needed an empty-string alternative added inside the group so a zero-param closure (`{ ->`) still has something to capture (the shared test harness requires a non-empty captured-groups list once any group exists in the pattern; same shape as perl's bare `shift` fix in PR #1212). - ruby: def-with-parens + `do |...|` + `{ |...| }` + stabby lambda `->(...)`, four independent parameter spans. - powershell: `param(...)` block + `function name(...)` + class-method constructor-shaped call. tests/extraction/languages/test_ruby.py's test_ruby_args needed updating: its "expected" values were the WHOLE matched payload (a different, older convention than every other language's name-based expected values), which no longer holds now that the match is split across capture groups. Switched the 21 cases to check the method name (or parameter span for blocks/lambdas) instead -- this doesn't lose coverage, since assert_valid_match still separately asserts the payload matches at all regardless of what "expected" is. Extends the ARGS_COUNT_FIXTURES regression suite added in PR #1212 with these 7 languages (7 new parametrized real-pipeline cases, 21 total now). Golden master fixtures regenerated via update_golden_master.py per the Differential Scan protocol -- confirmed clean by crucible_check.py afterward (both full-precision and zero-dependency modes). #1209's remaining scope: the 5-language Tier 2 (objective-c/scheme/ matlab/haskell/fortran), structurally unlike Python's single-paren- comma-list model and each needing bespoke handling. Co-authored-by: Joe Esquibel <squid-protocol@users.noreply.github.com> Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
5 tasks
squid-protocol
added a commit
that referenced
this pull request
Aug 10, 2026
… + Tier 2) (#1218) Fixes the 5 Tier 2 languages: objective-c, scheme, matlab, haskell, fortran. Unlike Tier 1's mechanical capture-group port, each of these has a signature shape that doesn't fit Python's single-paren-comma-list model at all, so each needed its own counting strategy, not just a regex tweak: - objective-c: keyword-message selectors (`doThing:(int)x withOther:(int)y`) scatter one parameter per repeated `label:(Type)name` segment across the signature -- there's no comma-separated list to count at all. The args regex's first branch now captures the WHOLE repeated selector span (was matching only the FIRST segment, meaning every multi-param method silently undercounted to 1 regardless of real arity, and could even false-match a `:(Type)` cast sitting inside an unrelated method's body). New `_count_colon_selector_segments` counts top-level `:(` occurrences instead of commas. Also fixed a real, pre-existing-shape ReDoS: adding an optional leading-label group to match 2nd+ segments introduced unbounded `\w*` backtracking at every string position (confirmed quadratic via scaling sweep before bounding it to `{0,80}`). - scheme: `(define (name a b) ...)` arguments are space-separated inside the SAME parens as the function name, not comma-separated. Split the name and parameter-list into separate capture groups so the existing whitespace-split fallback (already used for genuinely space-separated languages) sees just "a b", not "(define name a b)". Also fixed a real, pre-existing ReDoS in this exact regex found while verifying the fix: two adjacent unbounded quantifiers over near-identical character classes produced confirmed O(n^2) behavior (247x time for 16x input) on an unterminated `(define (xxxx...` with no closing paren -- bounding the name quantifier to `{1,100}` restores linear behavior. This ReDoS predates #1209 entirely; the capture-group refactor just meant actually re-deriving and scaling-testing this regex for the first time in a while. - matlab: `function [out1, out2] = name(in1, in2)` has an output-variable list before the `=` in addition to the real input-parameter parens. The new capture group targets ONLY the trailing input parens, so the output list is never miscounted as if it were part of the input signature (a distinct bug from the generic overcount, specific to MATLAB's syntax). - haskell: `name :: Int -> Int -> Int` signatures don't use parens or commas at all -- curried arity is the top-level arrow count. New `_count_haskell_type_arrows` counts top-level "->" after skipping any leading typeclass-constraint clause (`Show a => ...`) and ignoring an arrow nested inside a higher-order parameter's own type (`(Int -> Int) -> Int`). Gated on a new `_args_arrow_count_groups` flag in haskell's rules dict (an underscore-prefixed metadata key, matching the existing `_dependency_capture` convention) naming which SPECIFIC capture-group index is a type signature -- content-sniffing for "->" can't work here, since a real signature can have ZERO arrows (`noop :: IO ()`) and still need a correct zero count. Also hardened language_lens.py's two rule-iteration loops and test_language_standards_strict.py's global regex-integrity sweep to skip underscore-prefixed keys (matching detector.py's own `coding_analysis` convention already) -- they previously assumed every value in a language's `rules` dict was a compiled regex, which the new non-regex flag value would have broken (caught exceptions / the global integrity test, not silent wrongness, but worth aligning properly). - fortran: `SUBROUTINE Foo` (no parens at all) is valid zero-arg syntax alongside `SUBROUTINE Foo(a, b)`. The args group's capture now includes a trailing empty alternative so it participates even when the parens are genuinely absent -- without it, a bare zero-arg subroutine's only captured content would be the name itself, miscounted as 1 argument via the generic whitespace-split fallback. Two of these languages (scheme, matlab -- plus haskell/fortran for functions without an explicit signature or with expression-body syntax) aren't actually extracted by the real detector pipeline at all: they use paren-/keyword-delimited scoping, not braces, and the generic slicer dispatcher's Mode B brace-slicer never finds a body for them. This is a real, pre-existing recall gap, but a separate one from args-counting (out of scope here -- #1209 only fixes the count once a function IS found). New regression tests for these call `_calculate_block_metrics` directly with a hand-built block, bypassing the slicer, to isolate and pin the counting fix itself rather than being blocked by the unrelated gap. New tests: `objective-c` added to the existing ARGS_COUNT_FIXTURES real-pipeline suite; a new ARGS_COUNT_FIXTURES_DIRECT_BLOCK suite for scheme/matlab/haskell/fortran; direct unit coverage for both new counting helpers (`_count_colon_selector_segments`, `_count_haskell_type_arrows`). This closes #1209 -- all 25 languages in scope (20 Tier 1 + 5 Tier 2) are now fixed, across PRs #1207 (Python, the originating fix), #1212 (13 languages), #1216 (7 languages), and this one (5 languages). Golden master fixtures regenerated via update_golden_master.py per the Differential Scan protocol -- the diff is dominated by objective-c (the only Tier 2 language the real pipeline actually extracts at scale) plus a handful of real fortran corpus files, confirmed clean by crucible_check.py afterward (both full-precision and zero-dependency modes). Co-authored-by: Joe Esquibel <squid-protocol@users.noreply.github.com> Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Part of #1209 (13 of 25 languages — see issue for full tier breakdown). Ports #1199's fix (PR #1207, Python-only) to
c, cpp, go, kotlin, swift, php, perl, lua, apex, rust, solidity, scala, dart.Root cause (same as #1199): without a capture group isolating the parameter-list span,
detector.py's_calculate_block_metricsfalls back toargs_match.group(0)— the WHOLE match, including the keyword/name prefix. A zero/one-arg signature with no comma then hits a whitespace-split fallback meant for genuinely space-separated languages, overcounting by +1 (func main()→args=2instead of0).Not a blind port — each language got individual verification, and several needed more than "wrap the trailing parens":
c: also required recognizing(void)as 0 args, not 1 (the shared_count_top_level_argshelper now filters a lone"void"segment).cpp,kotlin,swift,rust,scala: have a second (or third) alternation branch for lambdas/closures/arrow functions with their own independent parameter span — each needed its own capture group to fix the same bug there too (e.g. cpp's[](), scala's barex =>).detector.pyalready resolves to the highest-numbered participating group vialastindex, so this doesn't affect counting logic.New persisted regression coverage (didn't exist before — neither #1199 nor #1207 left any, only ad hoc verification):
test_count_top_level_args_shared_helper— direct unit coverage of the shared counting helper (empty parens, trailing comma, bare*///voidmarkers, nested brackets).test_args_count_real_pipeline[lang]— parametrized across all 13 languages, running the realLANGUAGE_DEFINITIONSregex + real detector pipeline against a small fixed snippet, asserting exactargsvalues per function.Remaining scope on #1209 (not in this PR):
csharp/java/javascript/typescript/groovy/ruby/powershell(multi-branch regexes needing the same per-branch treatment as cpp/kotlin/swift here — deferred as a distinct follow-up since each is materially more expensive to verify), and the 5-language Tier 2 (objective-c/scheme/matlab/haskell/fortran— structurally unlike Python's single-paren-comma-list model, need bespoke handling).Type of change
gitgalaxy/core/detector.py,language_standards.py)CI checklist
python tests/tools/audit_check.pypassespython -m pytest tests/passes (6801 passed)python tests/tools/crucible_check.pypasses (both full-precision and zero-dependency modes)Differential Scan target
Corpus-wide fix across 13 languages, not a single-repo target.
golden_master_audit.json/golden_master_zero_dep_audit.jsonregenerated viaupdate_golden_master.pyagainst the standard ~80-repo language-crucible baseline — the diff is the real, intentional args-count correction (e.g. Go'shandlePanic()now correctly reports 0 params instead of 2; Go'sMain(a, b, c)now correctly reports 3 instead of an overcounted value), confirmed clean bycrucible_check.pyafterward.Verification
Co-Authored-By: Claude Sonnet 5 noreply@anthropic.com