ChatOpenAI with structured output, outputs always all field of my pydantic object.

[MagMueller]

Checked other resources

• I added a very descriptive title to this issue.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.
• I am sure that this is a bug in LangChain rather than my code.
• The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package).

Example Code

input_messages = [
SystemMessage(content='You are a helpful assistant please output valid json.'),
HumanMessage(content='What is the weather in Tokyo?'),
]

structured_llm = llm.with_structured_output(
output_class, include_raw=True
)
response: dict[str, Any] = await structured_llm.ainvoke(input_messages)  # type: ignore

Our output_class pydantic object decoded with

output_class.model_json_schema()

Looks like

"$defs": {"ActionModel": {"properties": {"search_google": {"anyOf": [
                    {"$ref": "#/$defs/SearchGoogleAction"
                    },
                    {"type": "null"
                    }
                ], "default": None
            }, "go_to_url": {"anyOf": [
                    {"$ref": "#/$defs/GoToUrlAction"
                    },
                    {"type": "null"
                    }
                ], "default": None
            }, "go_back": {"anyOf": [
                    {"$ref": "#/$defs/go_backParams"
                    },
                    {"type": "null"
                    }
                ], "default": None
            }, "click_element": {"anyOf": [
                    {"$ref": "#/$defs/ClickElementAction"
                    },
                    {"type": "null"
                    }
                ], "default": None
            }, "input_text": {"anyOf": [
                    {"$ref": "#/$defs/InputTextAction"
                    },
                    {"type": "null"
                    }
                ], "default": None
            }, "switch_tab": {"anyOf": [
                    {"$ref": "#/$defs/SwitchTabAction"
                    },
                    {"type": "null"
                    }
                ], "default": None
            }, "open_tab": {"anyOf": [
                    {"$ref": "#/$defs/OpenTabAction"
                    },
                    {"type": "null"
                    }
                ], "default": None
            }, "extract_content": {"anyOf": [
                    {"$ref": "#/$defs/ExtractPageContentAction"
                    },
                    {"type": "null"
                    }
                ], "default": None
            }, "done": {"anyOf": [
                    {"$ref": "#/$defs/DoneAction"
                    },
                    {"type": "null"
                    }
                ], "default": None
            }, "scroll_down": {"anyOf": [
                    {"$ref": "#/$defs/ScrollAction"
                    },
                    {"type": "null"
                    }
                ], "default": None
            }, "scroll_up": {"anyOf": [
                    {"$ref": "#/$defs/ScrollAction"
                    },
                    {"type": "null"
                    }
                ], "default": None
            }, "send_keys": {"anyOf": [
                    {"$ref": "#/$defs/SendKeysAction"
                    },
                    {"type": "null"
                    }
                ], "default": None
            }, "scroll_to_text": {"anyOf": [
                    {"$ref": "#/$defs/scroll_to_textParams"
                    },
                    {"type": "null"
                    }
                ], "default": None
            }, "get_dropdown_options": {"anyOf": [
                    {"$ref": "#/$defs/get_dropdown_optionsParams"
                    },
                    {"type": "null"
                    }
                ], "default": None
            }, "select_dropdown_option": {"anyOf": [
                    {"$ref": "#/$defs/select_dropdown_optionParams"
                    },
                    {"type": "null"
                    }
                ], "default": None
            }
        }, "title": "ActionModel", "type": "object"
    }, "AgentBrain": {"description": "Current state of the agent", "properties": {"evaluation_previous_goal": {"title": "Evaluation Previous Goal", "type": "string"
            }, "memory": {"title": "Memory", "type": "string"
            }, "next_goal": {"title": "Next Goal", "type": "string"
            }
        }, "required": ["evaluation_previous_goal", "memory", "next_goal"
        ], "title": "AgentBrain", "type": "object"
    }, "ClickElementAction": {"properties": {"index": {"title": "Index", "type": "integer"
            }, "xpath": {"anyOf": [
                    {"type": "string"
                    },
                    {"type": "null"
                    }
                ], "default": None, "title": "Xpath"
            }
        }, "required": ["index"
        ], "title": "ClickElementAction", "type": "object"
    }, "DoneAction": {"properties": {"text": {"title": "Text", "type": "string"
            }
        }, "required": ["text"
        ], "title": "DoneAction", "type": "object"
    }, "ExtractPageContentAction": {"properties": {"include_links": {"title": "Include Links", "type": "boolean"
            }
        }, "required": ["include_links"
        ], "title": "ExtractPageContentAction", "type": "object"
    }, "GoToUrlAction": {"properties": {"url": {"title": "Url", "type": "string"
            }
        }, "required": ["url"
        ], "title": "GoToUrlAction", "type": "object"
    }, "InputTextAction": {"properties": {"index": {"title": "Index", "type": "integer"
            }, "text": {"title": "Text", "type": "string"
            }, "xpath": {"anyOf": [
                    {"type": "string"
                    },
                    {"type": "null"
                    }
                ], "default": None, "title": "Xpath"
            }
        }, "required": ["index", "text"
        ], "title": "InputTextAction", "type": "object"
    }, "OpenTabAction": {"properties": {"url": {"title": "Url", "type": "string"
            }
        }, "required": ["url"
        ], "title": "OpenTabAction", "type": "object"
    }, "ScrollAction": {"properties": {"amount": {"anyOf": [
                    {"type": "integer"
                    },
                    {"type": "null"
                    }
                ], "default": None, "title": "Amount"
            }
        }, "title": "ScrollAction", "type": "object"
    }, "SearchGoogleAction": {"properties": {"query": {"title": "Query", "type": "string"
            }
        }, "required": ["query"
        ], "title": "SearchGoogleAction", "type": "object"
    }, "SendKeysAction": {"properties": {"keys": {"title": "Keys", "type": "string"
            }
        }, "required": ["keys"
        ], "title": "SendKeysAction", "type": "object"
    }, "SwitchTabAction": {"properties": {"page_id": {"title": "Page Id", "type": "integer"
            }
        }, "required": ["page_id"
        ], "title": "SwitchTabAction", "type": "object"
    }, "get_dropdown_optionsParams": {"properties": {"index": {"title": "Index", "type": "integer"
            }
        }, "required": ["index"
        ], "title": "get_dropdown_optionsParams", "type": "object"
    }, "go_backParams": {"properties": {}, "title": "go_backParams", "type": "object"
    }, "scroll_to_textParams": {"properties": {"text": {"title": "Text", "type": "string"
            }
        }, "required": ["text"
        ], "title": "scroll_to_textParams", "type": "object"
    }, "select_dropdown_optionParams": {"properties": {"index": {"title": "Index", "type": "integer"
            }, "text": {"title": "Text", "type": "string"
            }
        }, "required": ["index", "text"
        ], "title": "select_dropdown_optionParams", "type": "object"
    }
}, "properties": {"current_state": {"$ref": "#/$defs/AgentBrain"
    }, "action": {"items": {"$ref": "#/$defs/ActionModel"
        }, "title": "Action", "type": "array"
    }
}, "required": ["current_state", "action"
], "title": "AgentOutput", "type": "object"
}

Error Message and Stack Trace (if applicable)

Output:

{
    "current_state": {
        "evaluation_previous_goal": "Determine the current weather in Tokyo.",
        "memory": "The user is interested in the current weather conditions in Tokyo.",
        "next_goal": "Search for the current weather in Tokyo using an online weather service or search engine."
    },
    "action": [
        {
            "search_google": {
                "query": "current weather in Tokyo"
            },
            "go_to_url": null,
            "go_back": null,
            "click_element": null,
            "input_text": null,
            "switch_tab": null,
            "open_tab": null,
            "extract_content": null,
            "done": null,
            "scroll_down": null,
            "scroll_up": null,
            "send_keys": null,
            "scroll_to_text": null,
            "get_dropdown_options": null,
            "select_dropdown_option": null
        }
    ]
}

Expected output:

{
    "current_state": {
        "evaluation_previous_goal": "Determine the current weather in Tokyo.",
        "memory": "The user is interested in the current weather conditions in Tokyo.",
        "next_goal": "Search for the current weather in Tokyo using an online weather service or search engine."
    },
    "action": [
        {
            "search_google": {
                "query": "current weather in Tokyo"
            }
        }
    ]
}

Description

Hey, I want to upgrade our library browser-use to your new 3.0.0 version.

But with ChatOpenAI gpt-4 outputs all fields for the action key. But we only want to have one action of the keys, e.g. only "search_google".
This save many tokens and also prevents the model from outputting multiple actions in the same element.
In version 2.24 it worked and gave us our desired output.

System Info

System Information

OS: Darwin
OS Version: Darwin Kernel Version 23.1.0: Mon Oct 9 21:33:00 PDT 2023; root:xnu-10002.41.9~7/RELEASE_ARM64_T6031
Python Version: 3.11.10 (main, Oct 16 2024, 08:56:36) [Clang 18.1.8 ]

Package Information

langchain_core: 0.3.29
langchain: 0.3.14
langchain_community: 0.3.14
langsmith: 0.2.10
langchain_anthropic: 0.3.1
langchain_aws: 0.2.10
langchain_fireworks: 0.2.5
langchain_google: 0.1.1
langchain_google_genai: 2.0.8
langchain_mistralai: 0.2.4
langchain_ollama: 0.2.2
langchain_openai: 0.3.0
langchain_text_splitters: 0.3.4

Optional packages not installed

langserve

Other Dependencies

aiohttp: 3.11.11
anthropic: 0.42.0
async-timeout: 5.0.1
boto3: 1.35.97
dataclasses-json: 0.6.7
defusedxml: 0.7.1
filetype: 1.2.0
fireworks-ai: 0.15.8
google-generativeai: 0.8.3
httpx: 0.28.1
httpx-sse: 0.4.0
jsonpatch: 1.33
langsmith-pyo3: Installed. No version info available.
numpy: 1.26.4
ollama: 0.4.5
openai: 1.59.7
orjson: 3.10.14
packaging: 24.2
pydantic: 2.10.5
pydantic-settings: 2.7.1
PyYAML: 6.0.2
requests: 2.32.3
requests-toolbelt: 1.0.0
SQLAlchemy: 2.0.36
tenacity: 9.0.0
tiktoken: 0.8.0
tokenizers: 0.21.0
typing-extensions: 4.12.2
zstandard: 0.23.0
(.venv) magnus@full-speed agentweb %

[MagMueller]

So in a nutshell I think the problem is that Langchain or openAI forces every key to be set, even the key is optionally. So maybe you fill it up somewhere with None

[ccurme]

Hi @MagMueller, thanks for submitting this.

This is likely stemming from an update in langchain-openai==0.3, which changed the default method for with_structured_output. You should be able to restore langchain-openai==0.2.x behavior by explicitly passing in method="function_calling" into with_structured_output.

Version 0.3 passes Pydantic models straight to OpenAI's structured output feature, which currently emulates optional parameters via union types with null.

Release notes for 0.3 are here, please shout if this does not solve your issue.