fix(gateway): live-tail malformed redis reconnect ids

[Huixin615]

Fixes #4011

Why

When the Redis stream bridge is enabled, a reconnecting LangGraph SDK client can send Last-Event-ID: -1 after a page refresh. The bridge previously passed that value directly to Redis XREAD; Redis rejected it, and the error fallback reset the stream id to 0-0.

That fallback replayed the retained stream buffer from the beginning, so long active runs could re-stream the whole retained response on every reconnect and hang the chat UI.

What changed

Malformed Redis stream reconnect ids now live-tail new events instead of replaying retained history.

• Valid Redis stream ids (<ms>-<seq>) still resume after the provided Last-Event-ID.
• Missing Last-Event-ID still starts from 0-0, preserving initial subscribe behavior.
• Malformed values such as -1 now start from Redis $, so active reconnects wait for new events only.
• Redis ResponseError no longer falls back to 0-0, avoiding accidental full-buffer replay.
• Redis stream bridge docs now mention the malformed reconnect-id behavior.

Surface area

• Frontend UI — page / component / setting / interaction under frontend/
• Backend API — SSE reconnect behavior through the Redis stream bridge
• Agents / LangGraph — agent node, graph wiring, langgraph.json, or prompt change
• Sandbox — docker/ or sandboxed execution
• Skills — change under skills/
• Dependencies — new/upgraded entry in backend/pyproject.toml or frontend/package.json
• Default behavior change — malformed Redis reconnect ids now live-tail instead of replaying retained history
• Docs / tests / CI only — no runtime behavior change

Screenshots / Recording

N/A. Backend Redis stream reconnect behavior only.

Bug fix verification

Test path that reproduces the bug:

• backend/tests/test_stream_bridge.py::test_redis_invalid_last_event_id_tails_live_events
• backend/tests/test_stream_bridge.py::test_redis_integration_invalid_last_event_id_tails_live_events

Did it go red on main and green on this branch?

• Not rerun on main in this checkout. The previous implementation passed malformed ids to Redis, handled the resulting ResponseError, and reset to 0-0; the updated tests assert that malformed ids skip retained metadata and receive only newly published events.

If a red test wasn't cheap to write, explain why and what you did instead.

• A focused fake Redis regression covers the SDK-style -1 reconnect path without requiring Redis. The real Redis integration test is also updated for environments with DEER_FLOW_TEST_REDIS_URL.

Validation

cd backend && UV_CACHE_DIR=/tmp/uv-cache uv run pytest tests/test_stream_bridge.py -k redis
# 14 passed, 5 skipped, 20 deselected

cd backend && UV_CACHE_DIR=/tmp/uv-cache uv run ruff check packages/harness/deerflow/runtime/stream_bridge tests/test_stream_bridge.py
# All checks passed

cd backend && UV_CACHE_DIR=/tmp/uv-cache uv run ruff format --check packages/harness/deerflow/runtime/stream_bridge tests/test_stream_bridge.py
# 6 files already formatted

git diff --check
# passed

[zhfeng]

Thanks for this - clean fix, and the xrevrange-to-concrete-id approach is nicer than a naive $ (uniform XREAD loop, clean empty-stream -> 0-0 fallback, smaller miss window). CI is green and the regression tests (fake + real-redis) look valid. Approving with two notes below.

Non-blocking: malformed Last-Event-ID on an already-terminal run now hangs until TTL.

The replaced test (test_redis_integration_invalid_last_event_id_falls_back) subscribed to a completed run with a malformed id and relied on the 0-0 fallback to replay the retained events + END. With the new resolver, a malformed id on a terminal run whose stream is still buffered (within stream_ttl_seconds, default 86400s) resolves to the END marker's id -> XREAD after it returns nothing -> the subscriber heartbeats until the TTL expires, because _terminal_record_stream_missing stays False while the stream still exists.

This isn't reachable from the frontend (shouldSkipReconnect skips terminal runs before joinStream, and the SDK only sends -1 on reconnect), so it only affects a direct API client sending a bogus Last-Event-ID to a finished run. If that's acceptable, a one-line note in the resolver docstring would help; if not, a small guard works: in _resolve_start_stream_id, if the xrevrange last entry's kind == _KIND_END, return "0-0" (bounded replay delivers END and returns - no hang, since no new events pile on a terminal run; the hang only bites active runs).

Trivial: dead $ branch in _FakeRedis.xread.

The if last_id == "$": … branch isn't reachable, since the resolver produces a concrete id via xrevrange and never "$". Harmless, but can be dropped for clarity.

[willem-bd]

Reviewed the malformed reconnect-id fix. The core change is sound: pre-validating the id before XREAD, live-tailing on a malformed token, and propagating ResponseError (now safe because validation happens up front) all check out, and the two regression tests are meaningful. None of the items below are blocking — they're accuracy / coverage / cleanup nits. Details inline.

[willem-bd]

The PR body says malformed ids "start from Redis $", but this resolves to the last retained entry's id via XREVRANGE instead. They're equivalent for live-tailing a non-empty stream, and the XREVRANGE approach is arguably the better choice here — it also covers the empty-stream case by returning 0-0, whereas $ on a non-existent stream has murkier BLOCK semantics.

One loose end: _FakeRedis.xread gained a last_id == "$" branch (test_stream_bridge.py:49) that is now dead — nothing in production or tests ever passes $ (confirmed by grep). It reads like an incomplete refactor. Suggest aligning the PR-body wording to "live-tail from the last retained id" and either dropping the fake's $ branch or leaving a comment that it's intentional future-proofing.

[willem-bd]

Minor: this is stricter than Redis's own stream-id grammar. Redis accepts bare-ms partial ids like 1234 (== 1234-0), but this regex rejects them, so such an id now falls into the malformed path and live-tails instead of resuming — silently dropping any retained events with id >= that value. No known client emits partial ids (the SDK echoes the server's full <ms>-<seq>), so the risk is theoretical, but ^\d+(-\d+)?$ would preserve resume semantics if you want belt-and-suspenders.

Nit: with fullmatch, the ^…$ anchors are redundant.

[willem-bd]

Coverage gap: this empty-stream fallback (XREVRANGE -> [] -> "0-0") isn't exercised — both new tests pre-populate the stream, so the malformed path always takes the XREVRANGE branch. A small test subscribing with a malformed id to an empty/expired stream would cover this branch and confirm it live-tails future events.