Skip to content

fix(server): keep Git inspection off the event loop - #2103

Merged
RaghavChamadiya merged 1 commit into
repowise-dev:mainfrom
justonemorenight:codex/fix-async-server-git
Sep 7, 2026
Merged

RaghavChamadiya merged 1 commit into
repowise-dev:mainfrom
justonemorenight:codex/fix-async-server-git

Conversation

@justonemorenight

Copy link
Copy Markdown
Contributor

Summary

  • offload scheduler repository state + Git HEAD inspection as one blocking unit
  • offload incremental GitPython change planning, including graph cascade selection
  • offload synchronous state stamping for sync and scoped generation jobs
  • add regression coverage for the async delegation boundaries

Root cause

APScheduler coroutine jobs and server job coroutines were invoking filesystem reads, git rev-parse, and ChangeDetector synchronously on the asyncio event loop. Moving only rev-parse would leave GitPython change detection blocking, so the complete inspection/planning units now run through the existing asyncio.to_thread pattern.

Validation

  • uv run ruff check ...
  • uv run pytest tests/unit/server/test_scheduler.py tests/unit/server/test_job_executor.py -q (26 passed)

Fixes #832

@justonemorenight

Copy link
Copy Markdown
Contributor Author

Root cause addressed as a complete blocking boundary: state-file reads, git rev-parse, GitPython ChangeDetector, and graph cascade planning are now offloaded together via the existing asyncio.to_thread pattern. Added tests that lock the scheduler and incremental-planning delegation seams. Focused server tests: 26 passed; Ruff and diff checks pass.

@justonemorenight

Copy link
Copy Markdown
Contributor Author

Rebased onto current main and preserved the newer stale-page prioritization path: its DB lookup remains async, while the resulting stale-page ages are passed into the offloaded Git/ChangeDetector/graph planning unit. Added the regression assertion for that boundary. Validation: 53 focused server/persistence/ChangeDetector tests passed; Ruff and git diff --check pass.

@justonemorenight
justonemorenight force-pushed the codex/fix-async-server-git branch from b847f07 to b322344 Compare September 6, 2026 04:11

@RaghavChamadiya RaghavChamadiya left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @justonemorenight. The thing that makes this the right version of the change is that you moved a whole unit rather than the one obvious call: _plan_incremental_page_regen keeps the git rev-parse, the GitPython-backed ChangeDetector, the pagerank ranking and the parsed-file filtering together in the worker thread, and leaves the DB stale-page lookup on the loop where it belongs. Offloading only the subprocess would have left get_affected_pages blocking, which is the expensive half on a large graph.

Reusing _read_head_sha and _load_state instead of re-inlining subprocess.run and json.loads is worth noting too. It picks up the exception handling those already have.

Both tests assert on the to_thread call itself, so they will fail if someone later inlines it back. Green on the full matrix, merging.

@RaghavChamadiya
RaghavChamadiya merged commit cf15f12 into repowise-dev:main Sep 7, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Avoid blocking Git subprocesses in async server jobs

2 participants