You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Hand-written streaming parser retains the whole response in the request pool #27
In streaming mode the hand-written parser backend (NGX_HTTP_FFI_CLIENT_USE_LLHTTP=0) still accumulates the entire response, which defeats the point of incremental reads.
Detail
ngx_http_ffi_client_response_append() (src/ngx_http_ffi_client_response.c:198-230) appends every received byte into parser->raw, growing it by doubling into the request pool — and an nginx pool never frees the old buffers, so a grown raw costs the sum of all its generations. Consumed bytes are never reclaimed: ngx_http_ffi_client_response_update_body() (:480-510) only advances parser->taken as a cursor into the still-complete buffer.
Result: streaming a 1 GB body or a long-lived SSE stream on the fallback backend holds the whole thing in the request pool until the request ends, even though the caller copied and dropped each chunk. The llhttp backend hands spans out of the read buffer instead (src/ngx_http_ffi_client_response_llhttp.c:384-405), so this is fallback-specific.
Proposal
Either:
make the fallback streaming path memory-bounded — compact/reset raw once a span has been handed out and taken catches up, so peak usage is bounded by the read buffer plus one pending chunk; or
accept that the hand-written parser is a migration-only fallback (README "Project status" §2) and refuse stream mode on it with a clear error rather than silently buffering everything.
Whichever is chosen should be stated in the README so the two backends' streaming guarantees are not silently different.
Acceptance
A streaming test on the fallback backend over a body far larger than the read buffer shows bounded memory (or a clean, documented refusal).
Both backends' streaming semantics are documented.
Related: README "Project status" §2, #12. Part of #18.
api7/rfcs#115 - RFC: bring ngx_http_ffi_client to parity with lua-resty-http [merged] api7/aisix#794 - feat(usage): estimate tokens locally when the upstream reports no usage [merged] api7/aisix#809 - fix(azure): apply stream_timeout per chunk, not as a whole-response ceiling [merged]
api7/api7-ee-3-gateway#2022 - feat(ai-aws-content-moderation): moderate LLM responses, including streams [open] #22 - feat: stateful client object with buffered and streaming reads [open]
📝 Issue Planner
Check the box below or use the @coderabbitai plan command to generate an implementation plan and prompts that you can use with your favorite coding assistant.
Create Plan
🧪 Issue enrichment is currently in open beta.
You can configure auto-planning by selecting labels in the issue_enrichment configuration.
To disable automatic issue enrichment, add the following to your .coderabbit.yaml:
issue_enrichment:
auto_enrich:
enabled: false
💬 Have feedback or questions? Drop into our discord!
Activity
coderabbitai commentedon Jul 27, 2026
🔗 Related PRs
api7/rfcs#115 - RFC: bring ngx_http_ffi_client to parity with lua-resty-http [merged]
api7/aisix#794 - feat(usage): estimate tokens locally when the upstream reports no usage [merged]
api7/aisix#809 - fix(azure): apply stream_timeout per chunk, not as a whole-response ceiling [merged]
api7/api7-ee-3-gateway#2022 - feat(ai-aws-content-moderation): moderate LLM responses, including streams [open]
#22 - feat: stateful client object with buffered and streaming reads [open]
📝 Issue Planner
Check the box below or use the
@coderabbitai plancommand to generate an implementation plan and prompts that you can use with your favorite coding assistant.🧪 Issue enrichment is currently in open beta.
You can configure auto-planning by selecting labels in the issue_enrichment configuration.
To disable automatic issue enrichment, add the following to your
.coderabbit.yaml:💬 Have feedback or questions? Drop into our discord!