Skip to content

perf(hooks): eliminate multi-second foreground stalls before the detached launch#1601

Closed
TNRealtorApp wants to merge 1 commit into
Graphify-Labs:v8from
TNRealtorApp:perf/hook-foreground-cost
Closed

perf(hooks): eliminate multi-second foreground stalls before the detached launch#1601
TNRealtorApp wants to merge 1 commit into
Graphify-Labs:v8from
TNRealtorApp:perf/hook-foreground-cost

Conversation

@TNRealtorApp

Copy link
Copy Markdown
Contributor

Problem

On Windows (and any machine with a cold import cache or AV-scanned site-packages), git commit stalls for 10-40+ seconds inside the graphify post-commit hook before the #1161 detached launcher even starts:

  1. Interpreter probes import the full package. Each probe runs the equivalent of python -c 'import graphify', which executes the whole package import — measured 13s per probe (cold) on a Windows 11 dev box — and up to four probes can run synchronously per commit.
  2. The shebang probe reads a binary. Git for Windows' command -v graphify can return the launcher path without its .exe suffix, so the *.exe) guard misses and head -1 reads a PE binary: the shell prints warning: command substitution: ignored null byte in input on every commit, and the garbage 'shebang' always fails the allowlist and falls through to the slowest fallbacks (including the Microsoft Store python3 alias).
  3. Redundant git rev-parse. git already exports GIT_DIR to hooks; the unconditional re-derive costs another ~1.3s on machines where every git exec is AV-scanned.

Fix

  • Probes use importlib.util.find_spec('graphify') — locates the package without executing it, so each probe costs interpreter startup only. A broken install under the selected interpreter still fails loudly in the rebuild log (same as today). Probe order and fallback semantics are unchanged.
  • The Windows pip layout is resolved directly: Scripts/graphify(.exe) → sibling ..\python.exe (or .\python.exe for venv Scripts dirs) — no binary parsing, works with or without the .exe suffix.
  • The remaining POSIX shebang read strips NUL bytes first (head -c 256 | tr -d '\000' | head -n 1).
  • GIT_DIR=${GIT_DIR:-$(git rev-parse --git-dir 2>/dev/null)} in both hooks (rev-parse remains the manual-invocation fallback).

Measured (Windows 11, Python 3.13, real repo)

before after
hook foreground (cold) 26s+ (2 x 13s import probes) ~1.4s (one find_spec probe)
null-byte warnings every commit none

Tests

tests/test_hooks.py: 47 passed (4 new — find_spec probes, NUL-safe shebang read, sibling python.exe resolution, GIT_DIR reuse; plus the _launcher_payload docstring updated to match).

🤖 Generated with Claude Code

…ched launch

Three foreground costs ran on every commit before the detached rebuild
launcher even started:

1. Interpreter probes imported the full package. Each probe executed
   'import graphify' wholesale — measured 13s per probe cold on a Windows 11
   dev box with AV-scanned site-packages — and up to four probes could run
   synchronously. Probes now use importlib.util.find_spec, which locates the
   package without executing it (interpreter startup cost only). A broken
   install under the selected interpreter still fails loudly in the rebuild
   log, as before.

2. The shebang probe read a binary. Git for Windows' command -v can return
   the launcher path WITHOUT its .exe suffix, so the '*.exe)' guard missed
   and head -1 read a PE binary: the shell warned 'ignored null byte in
   input' on every commit and the garbage always fell through to the slow
   python3/python fallbacks. The Windows pip layout is now resolved directly
   (Scripts/graphify -> sibling ../python.exe, or ./python.exe for venvs),
   and the remaining POSIX shebang read strips NULs first.

3. GIT_DIR was re-derived. git exports GIT_DIR to hooks; the unconditional
   rev-parse added ~1.3s more on machines where every git exec is scanned.
   Now reused from the environment with rev-parse as the manual-run fallback.

Measured on the affected machine: hook foreground drops from 26s+ (cold) to
~1.4s, warnings gone. Behavior is unchanged on healthy POSIX setups — probe
order and fallback semantics are preserved.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@safishamsi

Copy link
Copy Markdown
Collaborator

Merged into v8 as 1256d65 (your authorship). Verified: probes now use find_spec (no full package import), NUL-safe shebang read, Windows sibling-python resolution, and GIT_DIR reuse — 4 new tests + full suite 2800 green. The find_spec/detached tradeoff is sound (foreground picks the interpreter; the detached rebuild still logs a broken install). Genuinely nice perf work for Windows/cold-cache hooks. Ships next release.

@safishamsi safishamsi closed this Jul 2, 2026
safishamsi added a commit that referenced this pull request Jul 2, 2026
Two 0.9.4 regressions (CLI cross-file indirect_call, stale community labels on
re-cluster), the case-folding god-node fix (#1581), ~15 language extractor
fixes (Ruby/Groovy/Elixir/Fortran/Rust/Julia/SystemVerilog/Scala/PowerShell/
ObjC/PHP/C#/C++/Swift), merge-graphs mixed-type handling (#1606), Swift
singleton-into-local resolution (#1604), Homebrew python@ shebang (#1586),
hooks foreground-stall perf (#1601), serve query stopwords (#1597) and
multi-project MCP serving (#1594), plus JSON-loading hardening, dedup collision
warning, and Windows hook worker limit.

Built wheel validated in a clean venv: CLI reports 0.9.5, import resolves to the
installed package, the case-folding and Swift-singleton fixes verified, and a
real `graphify extract` produces indirect_call + inherits edges end-to-end.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants