Skip to content

perf(analyzer): index qualified reference lookups - #769

Merged
duriantaco merged 2 commits into
duriantaco:mainfrom
mcdigman:codex/index-qualified-reference-lookups-upstream
Aug 25, 2026
Merged

perf(analyzer): index qualified reference lookups#769
duriantaco merged 2 commits into
duriantaco:mainfrom
mcdigman:codex/index-qualified-reference-lookups-upstream

Conversation

@mcdigman

@mcdigman mcdigman commented Aug 25, 2026

Copy link
Copy Markdown
Contributor

Closes #768

Summary

  • index non-import definitions by simple name and source file
  • resolve qualified references with logarithmic binary searches instead of repeated candidate scans

Knocks down the empirical O(n^2) scaling.

Measured Performance Improvement

  • homeassistant/core/homeassistant: _mark_refs cumulative time: 1213.7 s -> 84.5s
    • direct str.startswith calls from _mark_refs: 2.661 billion -> 1.965 million
  • scipy: 23.1s -> 14.3s in _mark_refs (scipy specifically is not as badly hit by this particular quadratic scaling relative to its size as a repo)
  • Not much change in timing (maybe ~5% improvement within ~inter-run noise) on skylos's current SHA pinned liveness_primer corpus (small to medium sized repositories not as affected by quadratic scaling)

Note: I doubt this completely maxes out possible perf wins in _mark_refs; I thought the maintainer might find a smaller footprint/relatively straightforward code change from me that specifically targets the O(n^2) scaling with indexing more palatable than a larger refactor, given how cognitively complex and important this function is.

Validation

  • Home Assistant findings were byte-for-byte identical across both 2,218-line outputs
  • 188 analyzer tests passed
  • 85 dead-code and framework-aware tests passed
  • targeted Ruff undefined-name, unbound-name, and closure-binding checks passed
  • liveness_primer shows 0 diffs on corpus

AI Use Disclosure
Profiled targeted and and potential O(n^2) identified by hand using cProfile. Codex implemented an indexing scheme and the sorting/bisection. Reviewed according to skylos repository review guidelines using GPT-5.6 Sol and Opus 5.0. Second commit was partly based on my own by-hand simplification findings.

@mcdigman
mcdigman requested a review from duriantaco as a code owner August 25, 2026 03:22

@duriantaco duriantaco left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks @mcdigman

@duriantaco
duriantaco merged commit b0fbfdb into duriantaco:main Aug 25, 2026
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Perf: O(n^2) scaling in _mark_refs

2 participants