fix: stream SSE responses end-to-end through Next.js proxy

[hyeonho-park]

Problem

The /api/search/ask and /api/sources/{source_id}/chat/sessions/{session_id}/messages endpoints emit Server-Sent Events (data: ...\n\n) but declare media_type="text/plain". Combined with Next.js 16 rewrites() proxy buffering non-SSE content types, the frontend's ReadableStream never receives chunks progressively. The UI sits at Processing... indefinitely even though the backend logs 200 OK right away.

Root cause

Two layers need to change for streaming to actually reach the browser:

1. Backend advertises the wrong Content-Type. StreamingResponse(..., media_type="text/plain") makes intermediaries assume the body is plain text and buffer it.
2. Next.js 16 rewrites() proxy buffers text/event-stream too. Even after fixing (1), Next.js' built-in rewrites handler in standalone mode reads the upstream body to completion before forwarding. There is no stable config flag in Next 15/16 to disable this for SSE.

Reproducible with curl -N from the host:

• curl -N :5055/api/search/ask streams chunks in real time after fix 1.
• curl -N :8502/api/search/ask still buffers after fix 1; needs fix 2.

Fix

Backend (api/routers/search.py, api/routers/source_chat.py): switch to media_type="text/event-stream" and add Cache-Control: no-cache, Connection: keep-alive, X-Accel-Buffering: no.

Frontend: add src/app/api/_sse-proxy.ts helper and two App Router route handlers (/api/search/ask, /api/sources/[sourceId]/chat/sessions/[sessionId]/messages). They return upstream.body as a ReadableStream, which Next.js streams without buffering. Reuses the existing INTERNAL_API_URL env var so all deployment topologies keep working.

Verification

• curl -Ni -X POST http://localhost:8502/api/search/ask ... now returns Content-Type: text/event-stream with chunks at distinct timestamps.
• Playwright: browser fetch('/api/search/ask') receives strategy / answer / final_answer / complete events progressively; the Ask UI renders each stage as it arrives.
• GET /api/search/ask returns 405 Method Not Allowed (confirms the new route handler is routed, not the rewrite).
• Non-streaming API paths continue through next.config.ts rewrites unchanged.

Scope

• No public API changes.
• No frontend hook changes (same relative URLs).
• No new dependencies.
• Route handlers set runtime = 'nodejs' + dynamic = 'force-dynamic' for output: 'standalone' compatibility.

[cubic-dev-ai]

2 issues found across 5 files

Confidence score: 3/5

• There is some concrete regression risk: frontend/src/app/api/_sse-proxy.ts does not handle network-level fetch failures, which can surface upstream outages as unhandled 500s instead of controlled proxy errors.
• api/routers/search.py sets Connection: keep-alive on SSE responses, which is invalid as a hop-by-hop header for HTTP/2+ and can cause interoperability issues through proxies/clients.
• Given both issues are medium severity (6/10) with high confidence (9/10), this looks mergeable with caution rather than low-risk.
• Pay close attention to frontend/src/app/api/_sse-proxy.ts and api/routers/search.py - error handling for upstream failures and HTTP-compliant SSE headers need tightening.

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="api/routers/search.py">

<violation number="1" location="api/routers/search.py:153">
P2: `Connection: keep-alive` is a hop-by-hop header and is invalid for HTTP/2+ responses; remove it from SSE response headers.</violation>
</file>

<file name="frontend/src/app/api/_sse-proxy.ts">

<violation number="1" location="frontend/src/app/api/_sse-proxy.ts:9">
P2: `sseProxy` does not catch network-level `fetch` failures, so upstream outages can bubble as unhandled 500s instead of controlled proxy errors.</violation>
</file>

_{Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.}

[cubic-dev-ai]

P2: Connection: keep-alive is a hop-by-hop header and is invalid for HTTP/2+ responses; remove it from SSE response headers.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At api/routers/search.py, line 153:

<comment>`Connection: keep-alive` is a hop-by-hop header and is invalid for HTTP/2+ responses; remove it from SSE response headers.</comment>

<file context>
@@ -147,7 +147,12 @@ async def ask_knowledge_base(ask_request: AskRequest):
+            media_type="text/event-stream",
+            headers={
+                "Cache-Control": "no-cache",
+                "Connection": "keep-alive",
+                "X-Accel-Buffering": "no",
+            },
</file context>

[cubic-dev-ai]

P2: sseProxy does not catch network-level fetch failures, so upstream outages can bubble as unhandled 500s instead of controlled proxy errors.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At frontend/src/app/api/_sse-proxy.ts, line 9:

<comment>`sseProxy` does not catch network-level `fetch` failures, so upstream outages can bubble as unhandled 500s instead of controlled proxy errors.</comment>

<file context>
@@ -0,0 +1,36 @@
+  const body = await req.text()
+  const auth = req.headers.get('authorization')
+
+  const upstream = await fetch(`${INTERNAL_API_URL}${upstreamPath}`, {
+    method: 'POST',
+    headers: {
</file context>