fix(api): synthesize mock content after bound tool results - #660
fix(api): synthesize mock content after bound tool results#660cursor[bot] wants to merge 33 commits into
Conversation
…closed otherwise Chat history: message-level audio and legacy function_call are null/empty omit no-ops; non-empty fail closed with named errors (including tools passthrough). Tip substrate from #577 assistant refusal/annotations honesty. Local full unit: 940 passed.
…ed otherwise OpenAI fine-tune style message weight is not applied on this gateway. Accept null/0/1 as honest no-ops; reject other types and values with invalid_message_weight. Tip substrate from #578. Local full unit: 943 passed.
…ion role Reject unsupported message keys with named unknown_message_fields (not silent strip or tools-passthrough smuggle). Reject legacy function role with invalid_message_role migration to tool. Tip substrate from #579. Local full unit: 947 passed.
OpenAI partial-assistant prefix flag is not applied on this gateway. null/false are honest no-ops; true and non-booleans fail closed with invalid_message_prefix. Tip substrate from #580. Local full unit: 950 passed.
…therwise Named invalid_max_tool_calls on /v1/chat/completions instead of opaque unknown_fields. Aligns with Responses max_tool_calls honesty; gateway has no multi-step tool loop.
…losed otherwise Legacy /v1/completions treated max_tool_calls as unknown_fields. Accept the key for named invalid_max_tool_calls (null/empty/whitespace omit-equivalent), matching chat/Responses honesty so SDKs get a clear migration path.
SDK clients often send include_usage/include_obfuscation as JSON null. Drop null flag values before validation so null (and null+false mixes) match omit / all-false no-ops on chat, Completions, and Responses. True flags remain fail-closed with invalid_stream_options.
…age honesty Null include_usage/include_obfuscation stay omit-equivalent, but unknown stream_options keys no longer become no-ops just because their value is null. Weight, prefix, refusal, annotations, developer role, empty user/system content, and participant name now use the same named errors on the tools passthrough path as on orchestration. Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>
Hoist stream, required model, stream_options, and temperature/top_p range checks before proxy_completion so a tools or response_format body cannot return a billed JSON completion when the SDK asked for SSE, or silently pick a pool model when model is omitted. Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>
Hoist stream, required model, stream_options, and temperature/top_p range checks before proxy_completion so a tools or response_format body cannot return a billed JSON completion when the SDK asked for SSE, or silently pick a pool model when model is omitted. Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>
Hoist attribution and routing validation before proxy_completion so a tools or response_format body cannot bill a sync completion with an unknown spend dimension or a batch/latency_tolerant hint. Tools passthrough has no batch job plane. Buyer next action: send known sync attribution; omit routing.channel=batch and latency_tolerant=true on tool-calling requests. Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>
Call the orchestration message, max_tokens, attribution, and routing validators before proxy_completion so a tools or response_format body cannot bill a completion with no prompt, crash on a non-object entry, or silently drop unknown spend/routing keys. Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>
Match _validate_messages: tools + user content 123 must 400 invalid_message instead of billing a sync completion. Assistant/tool JSON null stays omit-equivalent. Buyer next action: send user/system content as a non-empty string or a content-parts array. Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>
Match the tools-path honesty contract to the invalid_message behavior so SDK clients send a string or content-parts array. Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>
Passthrough has no batch job plane. Reject routing.channel=batch and latency_tolerant=true before proxy_completion so a tool-calling body cannot bill a silent sync completion. Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>
Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>
Hoist the remaining chat request knobs before proxy_completion so an OpenAI SDK tool-calling body cannot bill a sync completion for seed, stop, n>1, logprobs, logit_bias, out-of-range token/penalty values, unsupported reasoning_effort, or a non-default service_tier. Buyer next action: omit those fields on tool-calling requests; they are not applied on this gateway. Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>
Call _validate_messages before proxy_completion so tools + [] / omitted / null / non-list / non-object messages raise invalid_message instead of billing a completion with no prompt. Buyer next action: always send a non-empty messages array of objects on tool-calling requests. Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>
Hoist the orchestration request-knob validators before proxy_completion so a tools or response_format body cannot bill a completion that silently drops seed, stop, n>1, logprobs, logit_bias, or out-of-range penalties. Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>
stream_chat defaulted to 0.2 even after the HTTP path wrote default_temperature from the request. A streamed invoice summary at temperature=0.8 no longer silently falls back to 0.2. Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>
… proxy SDK optional defaults serialize omitted fields as JSON null. Accepting those keys without popping them is not omit-equivalent: proxy_completion forwards the body and several providers reject a null JSON Schema object. Pop the keys in place, keep non-null wrong types on invalid_tools, and assert omit-real via mock echo on chat and Responses. Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>
Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>
OpenAI SDK tool callers send stream=true. Returning 400 was honest but left every streaming tool client failing. Pipe a single pool agent as chat.completion.chunk frames so concatenated mock content matches the non-stream JSON body, and pipe live provider text so tool_calls deltas survive. Keep include_usage, empty messages, and unsupported knobs fail-closed. Buyer next action: send stream=true on tool-calling requests; omit stream_options.include_usage. Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>
Keep tools/response_format SSE proxy from this branch and take #601's null tool.function omit-pop, request-knob HTTP cases, and streamed route temperature. Conflict resolution keeps stream=true as SSE, not 400. CHANGELOG records the combined unreleased slice. Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>
Unknown assistant tool_calls entry/function fields, non-boolean include_orchestration_trace, and unknown mode used to bill a JSON or SSE completion on the tools proxy. They now return the same named 400s as the orchestration path before the first byte. Buyer next action: send only id/type/function/optional index on assistant tool_calls; send a boolean or omit include_orchestration_trace; send auto/route/conduct or omit mode. Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>
mock:// lookup_balance now returns assistant tool_calls and streams delta.tool_calls with finish_reason=tool_calls so offline SDK clients match the live _stream_raw path. Invoice identifiers bind from the user text (default INV-9). Cite Toolformer, ReAct, and OpenAI function calling (APA 7th) in docs/papers and the honesty test. Buyer next action: send stream=true on tool-calling requests and include the invoice id in the user message. Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>
stream_chat recorded temperature but omitted the request-scoped top_p, presence_penalty, and frequency_penalty that chat() already forwards. A streamed invoice summary at top_p=0.1 no longer drops those knobs. Buyer next action: send the nucleus and penalty values you want on stream=true route requests; streaming matches the JSON path. Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>
Streamed top_p / penalties now match chat(). Record the ICLR 2020 nucleus-sampling paper (APA 7th) so the buyer can see why those knobs must not silently drop on stream=true. Buyer next action: send the nucleus and penalty values you want on streamed route requests. Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>
The HTTP validator already accepted " none " / tab-newline none as none but discarded the stripped token. Mock selection required an exact "none", so a padded none still emitted tool_calls after a 200. Strip in the selector and write the token back so live providers see none. JSON and SSE honesty tests cover exact and padded none. Buyer next action: send tool_choice=none to skip tools; incidental padding stays omit-equivalent. Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>
Buyers type "invoice 4419" more often than "INV-4419". The mock binder only matched an INV- prefix, so a realistic prompt still defaulted to INV-9. Accept invoice / invoice id / invoice # plus digits. Buyer next action: put the invoice number in the user text; prefixed and bare forms both bind. Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>
After the first tool_calls hop, POST role=tool with the matching tool_call_id. Mock JSON and SSE now return content/stop that reproduces the observed invoice id, balance, and status instead of emitting another lookup_balance. Empty or whitespace tool_choice with tools is written back as none. Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>
Cover invoice 4419 → INV-4419 through _post_raw on both the JSON body and reconstructed delta.tool_calls, matching the padded-none HTTP honesty already in this file. Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>
There was a problem hiding this comment.
Unique slice vs #639 (2356202…d074598) is SOUND. Local tests/test_tool_result_continuation_http_honesty.py and tests/test_passthrough_sse_tools_http_honesty.py printed ok.
Buyer next action: after the first tool_calls hop, POST the tool output as role=tool with the same tool_call_id and read content / stop (or streamed content deltas). Send none when you want no first-hop tool call. Do not omit tool_choice on the first hop if you want to skip tools — omitted tool_choice stays auto and still emits lookup_balance.
What this tip uniquely locks
- Bound
role=tool+ matchingtool_call_idsynthesizescontent/stopon mock JSON and SSE from the observed values. Invoice 4419 observationbalance_usd=128.50/status=openis reproduced. - Unmatched
tool_call_idis ignored and first-hoplookup_balancestill fires. - Empty/whitespace
tool_choicewithtoolsis written back asnone(not popped). Writeback is correct: pop would make""indistinguishable from a missing key, and omittedtool_choicemust stay auto. - Bound observations ending the loop before
tool_choice=requiredis the ReAct second hop this slice adds, not a defect.
Do not merge in parallel. Prefer this over #639 / #620 / #606 / #617 for the stream-path tip. Keep KV/token #621, Responses stream #628/#648, verify #634, omit-budget #641, meaning-unit embeddings #643/#652, Responses text.format #657, and process-bootstrap #645 on their own tips.
Residual (not merge-blocking)
- CHANGELOG /
docs/rest_api_design.mdstill say “sendnoneor omittool_choicewhen you want no tool call.” First-hop omit is auto. - Unmatched-id test has no assistant
tool_callshistory. A forgedcall_fakebeside a real assistant id should still emitlookup_balanceand must not contain128.50(JSON and SSE). invoice no/invoice nr/inv#4419still default toINV-9. That is the next bind gap, not a hole in this unique synthesizer.
Independent non-author APPROVE + Full unit / Semgrep required. Do not self-approve. Reviewer seonghobae is already requested.
Sent by Cursor Automation: Fix Issues
There was a problem hiding this comment.
Unique slice vs #639 remains SOUND (bound tool_call_id synthesizer + empty tool_choice writeback as none). Local honesty tests printed ok.
Prefer #682 #682 as the stream-path landing vehicle. That tip contains this synthesizer plus clerk-alias bind (invoice no. / nr / inv#4419 → INV-4419) and the forged-tool_call_id leak lock.
Buyer next action on this tip: after the first tool_calls hop, POST the tool output as role=tool with the same tool_call_id and read content / stop. Send none when you want no first-hop tool call. Do not merge this in parallel with #682 / #674 / #639.
Independent non-author APPROVE still required if this tip is taken instead. Do not self-approve. Reviewer seonghobae is already requested.
Sent by Cursor Automation: Fix Issues
|
Bugbot is not enabled for your account, so this pull request was not reviewed. Enable Bugbot in the Cursor dashboard to get automatic reviews on future PRs. |


Successor to #639. Same padded-
none+ bare-invoice stream tip, plus the second-hop tool-result hole LangChain/OpenAI SDK clients hit immediately after the firsttool_callsresponse.Unique vs #639 (
2356202)role=toolobservations (tool_call_idmatches a prior assistanttool_callsid) synthesizecontent/stopon mock JSON and SSE instead of emitting anotherlookup_balance.balance_usd=128.50/status=openis reproduced in the answer (tests/test_tool_result_continuation_http_honesty.py).tool_call_idis not treated as a result — first-hoplookup_balancestill fires.tool_choicewithtoolsis written back asnoneso HTTP omit and mock selection stay content/stop.Buyer next action
After the first
tool_callshop, POST the tool output asrole=toolwith the sametool_call_idand readcontent/stop(or streamed content deltas). Sendnoneor omittool_choicewhen you want no tool call. Do not merge #639 in parallel — this tip already contains that stack plus the second-hop fix.Do not merge in parallel
Prefer this over #639 / #620 / #606 / #617 for the stream-path tip. Keep KV/token #621, Responses stream #628, verify #634, omit-budget #641, and meaning-unit embeddings #643 on their own tips. Independent non-author APPROVE + Full unit/Semgrep required.
Local checks printed
ok: tool-result continuation HTTP JSON+SSE, SSE tools honesty, empty-string tool_choice, chat tool_choice functions, tool_choice strip, paper/API/self-check/conventions, and related tool-message honesty files.References
Yao, S., Zhao, J., Yu, D., Du, N., Shafran, I., Narasimhan, K., & Cao, Y. (2023). ReAct: Synergizing reasoning and acting in language models. International Conference on Learning Representations. https://arxiv.org/abs/2210.03629
Schick, T., Dwivedi-Yu, J., Dessì, R., Raileanu, R., Lomeli, M., Hambro, E., Zettlemoyer, L., Cancedda, N., & Scialom, T. (2023). Toolformer: Language models can teach themselves to use tools. Advances in Neural Information Processing Systems, 36. https://arxiv.org/abs/2302.04761
OpenAI. (2024). Function calling. OpenAI API documentation. https://platform.openai.com/docs/guides/function-calling