feat(v0.6.1): heartbeat-stream /query/ + /upload/ (survive mobile/Tailscale tunnel drops)#1073
Merged
Hashevolution merged 1 commit intoJun 26, 2026
Conversation
β¦ailscale tunnel drops) Operator (phone over Tailscale): a long query returned "μλ² μ°κ²° μ€ν¨: Failed to fetch" before the answer, and the composer upload chip stayed stuck even though the server finished (entities formed). Both share one root cause: a request that holds an IDLE connection for 30β90s (full RAG query / image ingest with vision OCR) is evicted by the mobile network or a Tailscale re-handshake. The blocking call inside the async handler also pinned the event loop, so /trace/poll couldn't stream reasoning stages live. Fix: `core/http_heartbeat.stream_json_with_heartbeat` runs the blocking work in a worker thread (frees the loop β /trace/poll streams stages live) and emits JSON-insignificant whitespace every ~12s while it runs, so bytes keep flowing and the tunnel stays up. The heartbeat is leading whitespace, which r.json()/JSON.parse ignore β NO client change. current_trace_id (ContextVar) propagates into the thread via anyio.to_thread, preserving trace correlation (unit-tested). - core/http_heartbeat.py: the helper (3.3KB) - routes/query.py: wrap the rag_engine.query pipeline in _work + stream - routes/auth.py: wrap the /upload/ ingest slow-block in _work + stream (validation/size/auth errors still raise proper status before streaming; a rare slow-path error now surfaces in the 200 body, file marked failed) - tests/test_http_heartbeat.py: 4 tests (stream+parse / error / fast / contextvar propagation) core/retrieval + core/graph traversal + core/reasoning 0 lines (new util is core/http_heartbeat). Live end-to-end verify pending on the operator's server (helper + propagation unit-tested; routes import-OK + structurally wrapped). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01B7gaaBkpM3xYTzxhYrGHnR
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Operator (phone over Tailscale): a long query returned "Failed to fetch" before the answer, and the composer upload chip stayed stuck even though the server finished (entities formed). Shared root cause: a request that holds an idle connection for 30β90s (full RAG query / image ingest with vision OCR) is evicted by the mobile network or a Tailscale re-handshake. The blocking call inside the
asynchandler also pinned the event loop, so/trace/pollcouldn't stream reasoning stages live.Fix:
core/http_heartbeat.stream_json_with_heartbeatruns the blocking work in a worker thread (frees the loop β/trace/pollstreams stages live) and emits JSON-insignificant whitespace every ~12s while it runs, so bytes keep flowing and the tunnel stays up. The heartbeat is leading whitespace, whichr.json()/JSON.parseignore β no client change.current_trace_id(ContextVar) propagates into the thread viaanyio.to_thread, preserving trace correlation.Bonus
Threading the blocking pipeline also fixes the "reasoning not reflecting live" symptom β the event loop is now free to serve
/trace/pollduring the request.Verification
tests/test_http_heartbeat.py4/4: stream+parse, error-in-body, fast-path, contextvar propagation.routes.query/routes.authimport OK; slow blocks wrapped, validation (400/413/403) still raises proper status before streaming.test_measurement_critical_surfaces33/33.Trade-off
A rare slow-path error (after streaming starts, status already 200) now surfaces in the body (
{"answer":"","error":...}) instead of a 5xx; the file is still saved + markedfailedserver-side. Validation errors are unaffected (raised before streaming).Quality delta
Quality delta: exempt (label: feat) β transport-robustness wrapper on two routes + a new
core/http_heartbeatutil;core/retrieval+core/graphtraversal +core/reasoning0 lines.π€ Generated with Claude Code