[SEV-2 / BUG] BAML rejects valid Fireworks Chat Completions responses containing both completion_tokens_details and output_tokens_details

[zeke-john]

Product

BAML

Describe the bug

For transparency I used Fable 5 to help write & verify the info provided

We had a pretty severe production incident yesterday using BAML 0.212.0 with Fireworks' OpenAI-compatible Chat Completions endpoint.

The model returned a valid completion, but the response's usage object contained both:

• completion_tokens_details
• output_tokens_details

BAML rejected the entire response with:

duplicate field `output_tokens_details`

it seems like the cause is in engine/baml-runtime/src/internal/llm_client/primitive/openai/types.rs (still present on canary):

#[serde(alias = "completion_tokens_details")]
pub output_tokens_details: Option<serde_json::Value>,

Because of the alias, serde treats the two names as the same field. When a response has both, it errors as a duplicate and the whole response gets thrown away, even though the actual completion in choices[0].message.content is fine.

The same thing happens with input_tokens_details and prompt_tokens_details one line up. We tested a response with both of those and it fails with duplicate field `input_tokens_details` .

This caused successful model generations to be discarded and made the affected production workflow fail. Retrying the same request did not resolve the problem because subsequent responses contained the same fields. We mitigated the incident by routing affected calls away from the Fireworks client.

Here is the sanitized response shape:

{
  "id": "chatcmpl-example",
  "object": "chat.completion",
  "model": "accounts/fireworks/models/deepseek-v4-flash-0731",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "synthetic summary text"
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 100,
    "completion_tokens": 20,
    "total_tokens": 120,
    "completion_tokens_details": {
      "reasoning_tokens": 0
    },
    "output_tokens_details": {
      "reasoning_tokens": 0
    }
  }
}

We first hit the problem on BAML 0.212.0 and confirmed via the same local reproduction that 0.225.0 fails the same way:

• openai-generic provider: fails
• dedicated openai provider: fails (same deserializer)
• streaming (b.stream, usage delivered in the final SSE chunk): fails

So there is currently no BAML-side configuration that tolerates this response shape.

Reproduction Steps

Here is a loom of me repro'ing this for extra context ->

https://www.loom.com/share/e5f0bb5047a7482ebb79107f03ed0991

Save the response above as response.json, so like this:

{
  "id": "chatcmpl-example",
  "object": "chat.completion",
  "created": 1787007845,
  "model": "accounts/fireworks/models/deepseek-v4-flash-0731",
  "choices": [
    {
      "index": 0,
      "message": { "role": "assistant", "content": "synthetic summary text", "tools": null },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 100,
    "completion_tokens": 20,
    "total_tokens": 120,
    "prompt_tokens_details": { "cached_tokens": 99 },
    "completion_tokens_details": { "reasoning_tokens": 0 },
    "output_tokens_details": { "reasoning_tokens": 0 }
  }
}

baml_src/main.baml:

generator ts {
    output_type "typescript"
    output_dir "../"
    version "0.225.0"
    default_client_mode async
}

client<llm> Mock {
    provider "openai-generic"
    options {
        base_url "http://127.0.0.1:8787/v1"
        api_key "x"
        model "x"
    }
}

function Extract(text: string) -> string {
    client Mock
    prompt #"{{ text }}"#
}

server.js:

const fs = require('fs')
require('http').createServer((req, res) => res.end(fs.readFileSync('response.json'))).listen(8787)

test.ts:

import { b } from './baml_client'
b.Extract('hi').then(console.log, (e) => console.error(String(e)))

then you can run ->

npm i @boundaryml/baml@0.225.0 tsx
npx baml-cli generate --from baml_src
node server.js & npx tsx test.ts

you get the error ->

Failed to parse into a response accepted by baml_runtime::internal::llm_client::primitive::openai::types::ChatCompletionGeneric<baml_runtime::internal::llm_client::primitive::openai::types::ChatCompletionChoice>: {"id":"chatcmpl-example","object":"chat.completion","created":1787007845,"model":"accounts/fireworks/models/deepseek-v4-flash-0731","choices":[{"index":0,"message":{"role":"assistant","content":"synthetic summary text","tools":null},"finish_reason":"stop"}],"usage":{"prompt_tokens":100,"completion_tokens":20,"total_tokens":120,"prompt_tokens_details":{"cached_tokens":99},"completion_tokens_details":{"reasoning_tokens":0},"output_tokens_details":{"reasoning_tokens":0}}}

Delete the output_tokens_details entry from response.json and rerun: the call works (Same on 0.212.0)

BAML Version

0.212.0 (also repros on 0.225.0)

Language/Framework

Node.js

LLM Provider

Fireworks

LLM Model

deepseek-v4-flash-0731

Operating System

Linux

Browser

n/a

Code Editor

n/a

[aaronvg]

taking a look, thanks for the report

[zeke-john]

Thank you @aaronvg, please let us know if you would like any clarification or more info on this issue 🙂