fix(analytics): resolve registry-mounted package routers via their own APIRouter prefix (#12945)

[mrveiss]

Closes #12945.

Thinking Path

#12947 handled registry entries naming a module file and deliberately skipped those naming a package, because my first attempt at packages made things measurably worse: applying the registry prefix to each submodule produced 182 endpoints of which 4 were real, pushing scanned-but-not-real from 70 to 209.

Reading how LLC is actually mounted explains why, and what the correct rule is:

feature_routers.py:679   ("llc.api", "", ["llc"], "llc"),     <- empty prefix

llc/api/__init__.py:42   router = APIRouter(prefix="/llc", tags=["llc"])
                  :43+   router.include_router(activity_router)   # no extra prefix
llc/api/costs.py:46      router = APIRouter(prefix="/costs")

So a submodule serves /api + /llc (package router) + /costs (submodule router) + the route path. There is a third prefix level the module-file case does not have, and it lives in the package's own __init__.py — nowhere near the registry entry that names it. Confirmed the sub-routers are included with no additional prefix, so the package router is the only intermediate level.

_scan_file already applies a file's own APIRouter(prefix=…), so only the package-level part belongs in the prefix map.

Two guards against the failure mode that made the first attempt worse:

• the package must actually mount routers (include_router present in __init__.py), otherwise the entry is not a router package;
• a submodule is included only if it declares an APIRouter, so helper modules contribute nothing rather than endpoints at a guessed path.

What Changed

api/codebase_analytics/api_endpoint_scanner.py:

• _registry_router_files() now dispatches a package entry to _package_router_files().
• _package_router_files() reads the package's __init__.py, takes its router prefix, and maps each router-declaring submodule to registry_prefix + package_prefix.

api_endpoint_scanner_test.py: the placeholder "packages are skipped" test is replaced with four real ones — the prefix coming from the package's own router, the full served path end-to-end (/api/llc/costs/by-agent), helper modules without a router being skipped, and a package that mounts nothing being skipped.

Verification

$ python3 -m pytest api/codebase_analytics/api_endpoint_scanner_test.py -q
19 passed in 1.75s

$ python3 -m pytest api/codebase_analytics/ -q
233 passed, 11 skipped in 5.92s        (was 230 passed / 11 skipped)

$ python3 -m flake8 …/api_endpoint_scanner.py …/api_endpoint_scanner_test.py
(clean)

Measured on the live install's indexed source c5939a51-…, read-only, against app.openapi() (2159 unique normalized paths):

	#12947	this PR
router files scanned outside api/	6	28
endpoints found	2041	2160
real routes matched	1789 (82.9%)	1885 (87.3%)
real routes missed	370	274
scanned but not real	70	71

+96 real routes for +1 spurious. 119 LLC endpoints discovered, 96 of them confirmed present in app.openapi().

End-user metric across the whole thread, on the same source:

	before #12943	#12943	#12947	this PR
backend endpoints	0	1986	2041	2160
missing findings	~2400	488	435	330
false positives among them	~97%	95.4%	94.8%	93.1%
genuine findings	—	20	20	20

The genuine count holding at exactly 20 across all three fixes is the useful signal: the number of real drift findings is stable, and everything removed so far was scan gap rather than real drift.

What remains (not this issue)

274 real routes are still unmatched and 270 of the 290 unique missing findings remain false positives. That residue is no longer about registry-mounted modules — all 7 are now covered — so it belongs to a different gap (routes reached by neither the api/ walk nor a registry entry, e.g. nested include_router chains). Worth a fresh issue with these numbers as the baseline rather than widening this one.

Model Used

claude-opus-5

[github-actions]

✅ SSOT Configuration Compliance: Passing

🎉 No hardcoded values detected that have SSOT config equivalents!

[mrveiss]

Reviewed this diff while dogfooding the new secreview skill (#12951 / #12952).
Two confirmed findings, both in _package_router_files — and both are the same
defect class this PR is fixing: a submodule served under a prefix it does not
actually have.

1. rglob descends into nested subpackages, but only the top-level package prefix is applied

sorted(package.rglob("*.py")) walks the whole tree, while
not py_file.name.startswith("__") filters out the nested __init__.py that
carries that subpackage's own prefix. So a nested router package's submodules are
mapped to the parent package's served prefix:

rglob:                          ['__init__.py', 'a.py', 'sub/__init__.py', 'sub/b.py']
filtered (not startswith __):   ['a.py', 'sub/b.py']

sub/b.py is included with the parent's served_prefix; sub/__init__.py — the
only place sub's own APIRouter(prefix=...) could be read — is dropped. If any
registry-mounted package ever nests a router subpackage, this invents endpoints
under the wrong path, exactly the failure mode the PR docstring warns about.

Fix: either glob("*.py") for the flat case, or recurse through
_package_router_files per nested __init__.py so each level contributes its prefix.

2. Docstring overclaims the include_router check

Submodules are included only when the package actually mounts them via include_router

The code checks the package mounts anything:

if not _INCLUDE_ROUTER_RE.search(init_content):
    return {}

then includes every submodule with an APIRouter(prefix=...), mounted or not. A
module that declares a router the package never mounts still contributes routes —
the same phantom-endpoint problem, just narrower. Either match the mounted router
names, or soften the docstring to what the code guarantees.

Not a finding

The package path is built from the registry via joinpath(*module_path.split(".")),
so a .. segment would escape backend_dir — but the registry is code-controlled,
not user input. Noting it only so it is not re-raised later.

Both are pre-existing in this PR's new code rather than regressions of merged
behaviour, so happy for them to land here or as a follow-up — your call.