feat(api): SSE-proxy /v1/responses with mock function_call events by cursor[bot] · Pull Request #628 · ContextualWisdomLab/contextual-orchestrator

cursor · 2026-08-16T16:09:45Z

Summary

Successor to #617 at a1823cc (mock chat delta.tool_calls + streamed route top_p/penalties). Unique commits close the remaining buyer-visible Responses gap on that tip:

/v1/responses stream=true SSE-proxies a single pool agent as named response.* events instead of 400 invalid_stream.
Offline mock:// function tools emit output[].type=function_call on the JSON path and response.function_call_arguments.delta on the stream path. Invoice lookup_balance binds INV- from input (default INV-9). Chat-style tools[].function.name and Responses-native tools[].name / tool_choice.name both bind.
tool_choice=none and content-only streams stay output_text / stop.
stream_options.include_usage=true / include_obfuscation=true and non-boolean stream still fail closed.
Direct python3 tests/test_responses_tools_shape_http_honesty.py now calls the renamed empty-tools no-op case.

Buyer next action

Send stream=true on /v1/responses. Include the invoice id in input. Omit stream_options.include_usage. Content-only Responses streams still work without tools.

Prefer this head over #617 for the Responses stream slice. Do not merge #606, #615, #613, #601, or #617 in parallel with this head. #619 remains the KV host-allowlist landing on the #617 tip — do not fold it here and do not open another allowlist PR. Independent non-author APPROVE + Full unit/Semgrep are still required. This automation will not APPROVE or merge.

Test plan

python3 tests/test_passthrough_sse_responses_http_honesty.py
python3 tests/test_responses_stream_reject_http_honesty.py
python3 tests/test_stream_null_noop_http_honesty.py
python3 tests/test_responses_stream_options_http_honesty.py
python3 tests/test_passthrough_sse_tools_http_honesty.py
python3 tests/test_openai_passthrough.py test_responses_tools_shape_http_honesty.py
python3 tests/test_paper_contracts.py test_self_check.py test_api_contract.py test_conventions.py
CI Full unit + Semgrep

Docs

docs/rest_api_design.md, docs/architecture.md, docs/library_research.md, docs/papers/README.md, CHANGELOG.md, README
APA 7th: OpenAI. (2024). Create a model response. https://platform.openai.com/docs/api-reference/responses/create
OpenAI. (2024). Streaming API responses. https://platform.openai.com/docs/guides/streaming-responses
WHATWG. (n.d.). Server-sent events. https://html.spec.whatwg.org/multipage/server-sent-events.html

Next loop

After #619 lands the KV host allowlist on this honesty tip: remaining __main__.py bootstrap tokens (env only as KV seed). Do not open another Responses-stream or SSE-tools PR; extend this head.

…closed otherwise Chat history: message-level audio and legacy function_call are null/empty omit no-ops; non-empty fail closed with named errors (including tools passthrough). Tip substrate from #577 assistant refusal/annotations honesty. Local full unit: 940 passed.

…ed otherwise OpenAI fine-tune style message weight is not applied on this gateway. Accept null/0/1 as honest no-ops; reject other types and values with invalid_message_weight. Tip substrate from #578. Local full unit: 943 passed.

…ion role Reject unsupported message keys with named unknown_message_fields (not silent strip or tools-passthrough smuggle). Reject legacy function role with invalid_message_role migration to tool. Tip substrate from #579. Local full unit: 947 passed.

OpenAI partial-assistant prefix flag is not applied on this gateway. null/false are honest no-ops; true and non-booleans fail closed with invalid_message_prefix. Tip substrate from #580. Local full unit: 950 passed.

…therwise Named invalid_max_tool_calls on /v1/chat/completions instead of opaque unknown_fields. Aligns with Responses max_tool_calls honesty; gateway has no multi-step tool loop.

…losed otherwise Legacy /v1/completions treated max_tool_calls as unknown_fields. Accept the key for named invalid_max_tool_calls (null/empty/whitespace omit-equivalent), matching chat/Responses honesty so SDKs get a clear migration path.

SDK clients often send include_usage/include_obfuscation as JSON null. Drop null flag values before validation so null (and null+false mixes) match omit / all-false no-ops on chat, Completions, and Responses. True flags remain fail-closed with invalid_stream_options.

…age honesty Null include_usage/include_obfuscation stay omit-equivalent, but unknown stream_options keys no longer become no-ops just because their value is null. Weight, prefix, refusal, annotations, developer role, empty user/system content, and participant name now use the same named errors on the tools passthrough path as on orchestration. Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>

Hoist stream, required model, stream_options, and temperature/top_p range checks before proxy_completion so a tools or response_format body cannot return a billed JSON completion when the SDK asked for SSE, or silently pick a pool model when model is omitted. Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>

Hoist attribution and routing validation before proxy_completion so a tools or response_format body cannot bill a sync completion with an unknown spend dimension or a batch/latency_tolerant hint. Tools passthrough has no batch job plane. Buyer next action: send known sync attribution; omit routing.channel=batch and latency_tolerant=true on tool-calling requests. Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>

Call the orchestration message, max_tokens, attribution, and routing validators before proxy_completion so a tools or response_format body cannot bill a completion with no prompt, crash on a non-object entry, or silently drop unknown spend/routing keys. Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>

Match _validate_messages: tools + user content 123 must 400 invalid_message instead of billing a sync completion. Assistant/tool JSON null stays omit-equivalent. Buyer next action: send user/system content as a non-empty string or a content-parts array. Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>

Match the tools-path honesty contract to the invalid_message behavior so SDK clients send a string or content-parts array. Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>

Passthrough has no batch job plane. Reject routing.channel=batch and latency_tolerant=true before proxy_completion so a tool-calling body cannot bill a silent sync completion. Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>

Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>

Hoist the remaining chat request knobs before proxy_completion so an OpenAI SDK tool-calling body cannot bill a sync completion for seed, stop, n>1, logprobs, logit_bias, out-of-range token/penalty values, unsupported reasoning_effort, or a non-default service_tier. Buyer next action: omit those fields on tool-calling requests; they are not applied on this gateway. Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>

Call _validate_messages before proxy_completion so tools + [] / omitted / null / non-list / non-object messages raise invalid_message instead of billing a completion with no prompt. Buyer next action: always send a non-empty messages array of objects on tool-calling requests. Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>

Hoist the orchestration request-knob validators before proxy_completion so a tools or response_format body cannot bill a completion that silently drops seed, stop, n>1, logprobs, logit_bias, or out-of-range penalties. Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>

stream_chat defaulted to 0.2 even after the HTTP path wrote default_temperature from the request. A streamed invoice summary at temperature=0.8 no longer silently falls back to 0.2. Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>

… proxy SDK optional defaults serialize omitted fields as JSON null. Accepting those keys without popping them is not omit-equivalent: proxy_completion forwards the body and several providers reject a null JSON Schema object. Pop the keys in place, keep non-null wrong types on invalid_tools, and assert omit-real via mock echo on chat and Responses. Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>

Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>

OpenAI SDK tool callers send stream=true. Returning 400 was honest but left every streaming tool client failing. Pipe a single pool agent as chat.completion.chunk frames so concatenated mock content matches the non-stream JSON body, and pipe live provider text so tool_calls deltas survive. Keep include_usage, empty messages, and unsupported knobs fail-closed. Buyer next action: send stream=true on tool-calling requests; omit stream_options.include_usage. Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>

Keep tools/response_format SSE proxy from this branch and take #601's null tool.function omit-pop, request-knob HTTP cases, and streamed route temperature. Conflict resolution keeps stream=true as SSE, not 400. CHANGELOG records the combined unreleased slice. Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>

Unknown assistant tool_calls entry/function fields, non-boolean include_orchestration_trace, and unknown mode used to bill a JSON or SSE completion on the tools proxy. They now return the same named 400s as the orchestration path before the first byte. Buyer next action: send only id/type/function/optional index on assistant tool_calls; send a boolean or omit include_orchestration_trace; send auto/route/conduct or omit mode. Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>

mock:// lookup_balance now returns assistant tool_calls and streams delta.tool_calls with finish_reason=tool_calls so offline SDK clients match the live _stream_raw path. Invoice identifiers bind from the user text (default INV-9). Buyer next action: send stream=true on tool-calling requests and include the invoice id in the user message. Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>

stream_chat recorded temperature but omitted the request-scoped top_p, presence_penalty, and frequency_penalty that chat() already forwards. A streamed invoice summary at top_p=0.1 no longer drops those knobs. Buyer next action: send the nucleus and penalty values you want on stream=true route requests; streaming matches the JSON path. Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>

OpenAI Responses SDKs send stream=true. Returning 400 left every streaming Responses client failing. Pipe a single pool agent as named response.* events so a streamed invoice lookup_balance reconstructs to the same function_call as the JSON body (INV-9 from the prompt). Content-only streams still match output_text. Live providers stay verbatim. include_usage=true and non-boolean stream still fail closed. Buyer next action: send stream=true on /v1/responses; include the invoice id in input; omit stream_options.include_usage. Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>

The script entrypoint still named the old rejects_empty_tools_array helper after the case became an omit-equivalent no-op. Direct python3 tests/test_responses_tools_shape_http_honesty.py now runs. Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>

cursor · 2026-08-16T16:11:49Z

Bugbot is not enabled for your account, so this pull request was not reviewed.

Enable Bugbot in the Cursor dashboard to get automatic reviews on future PRs.

Official openai-python reconstructs argument chunks with item_id from output_item.added, and .done also carries name. The mock stream now emits those fields. HTTP tests POST the JSON twin, require two-plus argument deltas (INV-20260816009), and keep tool_choice=none on output_text. Buyer next action: send stream=true on /v1/responses; correlate function_call_arguments.delta with item_id; omit include_usage. Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>

cursor

Stale comment

Review (unique slice + 37d5cc3)

CodeRabbit CLI 0.7.3 is installed here; coderabbit auth login --agent timed out (Automatic login timed out). This assessment is the unique #617→#628 slice (a1823cc..37d5cc3), not the 155-file honesty stack vs main.

What the buyer can do now

Send stream=true on /v1/responses. Include the invoice id in input (default bind is still INV-9). Correlate response.function_call_arguments.delta with item_id from response.output_item.added. Omit stream_options.include_usage. Content-only and tool_choice=none streams stay output_text.

Unique slice

Transport: stream=true is text/event-stream named response.* events, not 400 invalid_stream.

Mock JSON output[].type=function_call and stream function_call_arguments.delta reconstruct to the same id / call_id / name / arguments as the HTTP JSON twin.

37d5cc3 adds the official envelope keys the first tip omitted: item_id on argument (and output_text) deltas; .done also carries name. HTTP tests now POST the JSON twin, require two-plus argument chunks (INV-20260816009), and keep tool_choice=none on output_text.

Chat-style tools[].function.name and Responses-native tools[].name / tool_choice.name both bind (in-process).

include_usage=true and non-boolean stream still fail closed.

Direct python3 tests/test_responses_tools_shape_http_honesty.py calls the renamed empty-tools no-op.

Local: test_passthrough_sse_responses_http_honesty.py, test_responses_stream_reject_http_honesty.py, test_stream_null_noop_http_honesty.py, test_responses_tools_shape_http_honesty.py, paper/self/api/conventions printed ok.

Residual — do not fold into this head

KV host allowlist remains #619 / landing #621. Do not open another allowlist PR.

Remaining __main__.py bootstrap tokens (env only as KV seed) wait until that allowlist lands.

Official sequence_number, response.in_progress, and dropping the Chat Completions data: [DONE] trailer after response.completed.

Failed streams still record analytics status_code: 200 because the writer swallows the exception after headers.

Prefer this head over #617 and #627 for the Responses-stream slice. Do not merge #606, #615, #613, #601, or #617 in parallel. Independent non-author APPROVE + Full unit/Semgrep are still required. This automation will not APPROVE or merge.

_{Sent by Cursor Automation: Fix Issues}

cursor

Review (unique slice + `37d5cc3`)

CodeRabbit CLI is not authenticated in this environment (coderabbit binary missing; prior run's auth login --agent timed out). This assessment is the unique #617→#628 slice (a1823cc..37d5cc3), not the honesty stack vs main.

What the buyer can do on this head

Send stream=true on /v1/responses. Include the invoice id in input (default bind is still INV-9). Correlate response.function_call_arguments.delta with item_id from response.output_item.added. Omit stream_options.include_usage. Content-only and tool_choice=none streams stay output_text.

Unique slice (still SOUND)

Transport: stream=true is text/event-stream named response.* events, not 400 invalid_stream.
Mock JSON output[].type=function_call and stream function_call_arguments.delta reconstruct to the same id / call_id / name / arguments as the HTTP JSON twin.
item_id on argument (and output_text) deltas; .done also carries name. HTTP tests POST the JSON twin and require two-plus argument chunks (INV-20260816009).
Chat-style tools[].function.name and Responses-native tools[].name / tool_choice.name both bind.
include_usage=true and non-boolean stream still fail closed.

Envelope residual — do not fold into this head

Official sequence_number, response.in_progress, and dropping the Chat Completions data: [DONE] trailer after response.completed landed on successor #648 (6de6eb0 on this tip). Prefer #648 over this head for the official envelope slice. Do not merge this PR in parallel with #648.

KV host allowlist remains #619 / landing #621. Do not open another allowlist PR. Remaining __main__.py bootstrap tokens wait until that allowlist lands. Failed streams still record analytics status_code: 200 because SSE headers are already sent.

Do not merge #606, #615, #613, #601, or #617 in parallel. Independent non-author APPROVE + Full unit/Semgrep are still required. This automation will not APPROVE or merge.

View PR

_{Sent by Cursor Automation: Fix Issues}

seonghobae and others added 29 commits August 16, 2026 21:18

fix(api): treat message prefix null/false as omit; fail-closed on true

82038d3

OpenAI partial-assistant prefix flag is not applied on this gateway. null/false are honest no-ops; true and non-booleans fail closed with invalid_message_prefix. Tip substrate from #580. Local full unit: 950 passed.

fix(api): treat chat max_tool_calls null/empty as omit; fail-closed o…

1a196b0

…therwise Named invalid_max_tool_calls on /v1/chat/completions instead of opaque unknown_fields. Aligns with Responses max_tool_calls honesty; gateway has no multi-step tool loop.

docs(api): tell buyers non-string user content fails closed on tools

f6dfae5

Match the tools-path honesty contract to the invalid_message behavior so SDK clients send a string or content-parts array. Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>

docs: tell buyers to omit batch routing on tool-calling requests

dbfa0e5

Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>

docs: record passthrough knob and stream temperature fixes in CHANGELOG

7e9a339

Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>

cursor Bot requested a review from seonghobae August 16, 2026 16:10

cursor Bot mentioned this pull request Aug 16, 2026

feat(api): frame /v1/responses stream=true as official SSE events #627

Open

seonghobae marked this pull request as ready for review August 16, 2026 16:11

seonghobae enabled auto-merge (squash) August 16, 2026 16:11

cursor Bot mentioned this pull request Aug 16, 2026

fix(api): treat padded tool_choice none as omit on mock tools #639

Open

cursor Bot commented Aug 16, 2026

View reviewed changes

Comment thread contextual_orchestrator/orchestrator.py

Comment thread tests/test_passthrough_sse_responses_http_honesty.py

This was referenced Aug 16, 2026

feat(embeddings): split batch inputs on meaning units #643

Open

feat(api): stamp Responses SSE sequence_number and drop Chat [DONE] #648

Closed

cursor Bot commented Aug 16, 2026

View reviewed changes

Navigation Menu

Uh oh!

feat(api): SSE-proxy /v1/responses with mock function_call events - #628

feat(api): SSE-proxy /v1/responses with mock function_call events#628
cursor[bot] wants to merge 30 commits into
mainfrom
cursor/bc-81a85314-9dcf-4bbe-8c80-d0393c59da91-9cfb

cursor Bot commented Aug 16, 2026

Uh oh!

cursor Bot commented Aug 16, 2026

Uh oh!

cursor Bot left a comment •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

cursor Bot left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

cursor Bot commented Aug 16, 2026

Summary

Buyer next action

Test plan

Docs

Next loop

Uh oh!

cursor Bot commented Aug 16, 2026

Uh oh!

cursor Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Review (unique slice + 37d5cc3)

What the buyer can do now

Unique slice

Residual — do not fold into this head

Uh oh!

Uh oh!

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Review (unique slice + 37d5cc3)

What the buyer can do on this head

Unique slice (still SOUND)

Envelope residual — do not fold into this head

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

cursor Bot left a comment •

edited

Loading

Review (unique slice + `37d5cc3`)

Review (unique slice + `37d5cc3`)