standard-tests: refactoring and fixes #31703

mdrxy · 2025-06-23T14:54:11Z

libs/core/langchain_core/messages/base.py: add model name to examples per docs ("0.3.17: Additionally check for the presence of model_name in the response metadata, which is needed for usage tracking in callback handlers")
libs/core/langchain_core/utils/function_calling.py: correct typo
libs/standard-tests/langchain_tests/integration_tests/chat_models.py:
- magic_function(input) -> magic_function(_input) to prevent warning about redefining built in input
- relocate a few tests for better grouping and narrative flow
- suppress some type hint warnings following suit from similar tests
- fix a few more typos
- validate not only that model_name is defined, but that it is not empty (test_usage_metadata)

…ad of tokens

version 0.3.17 checks for the presence of `model_name` in `response_metadata`, which is needed for usage tracking in callback handlers (source: https://python.langchain.com/api_reference/standard_tests/integration_tests/langchain_tests.integration_tests.chat_models.ChatModelIntegrationTests.html#langchain_tests.integration_tests.chat_models.ChatModelIntegrationTests.test_usage_metadata)

…the structure of the rest of the file)

…ition of built-in function

…ace issue

…llow similar positioning as js

vercel · 2025-06-23T14:54:17Z

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
langchain	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Jun 23, 2025 11:22pm

codspeed-hq · 2025-06-23T14:55:42Z

CodSpeed WallTime Performance Report

Merging #31703 will not alter performance

_{Comparing mdrxy/chat_models-updates (d0ac6b8) with master (b6f74bf)}

⚠️

Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

Summary

✅ 13 untouched benchmarks

codspeed-hq · 2025-06-23T15:02:18Z

CodSpeed Instrumentation Performance Report

Merging #31703 will not alter performance

_{Comparing mdrxy/chat_models-updates (d0ac6b8) with master (b6f74bf)}

Summary

✅ 13 untouched benchmarks

sydney-runkle · 2025-06-23T16:16:39Z

magic_function(input) -> magic_function(_input) to prevent warning about redefining built in input

Eventually we should enforce more linting / type checking rules, good to fix these now though!

libs/standard-tests/langchain_tests/integration_tests/chat_models.py

sydney-runkle

Seems like a reasonable cleanup, let's confirm with @ccurme who owns integration test stuff :)

libs/standard-tests/langchain_tests/integration_tests/chat_models.py

mdrxy · 2025-06-23T20:52:08Z

closing some ties here, @ccurme will you merge after a final review?

sydney-runkle

Changes looking good, but a few nit picks to make this a bit more pythonic / less redundant!

libs/standard-tests/langchain_tests/integration_tests/chat_models.py

…ol call validations plus additional clean ups

sydney-runkle

Great!

mdrxy added 17 commits June 18, 2025 15:51

core[refactor]: update streaming test semantics to count chunks inste…

64fb442

…ad of tokens

docs: add some inline documentation

fc35023

test: add type ignores to match _streaming tests

a06cf33

test: enhance assertion for model_name to check for emptiness

c332fe1

test: put test_tool_calling_async next to test_tool_calling (per …

9a361f2

…the structure of the rest of the file)

test: move tool call tests to be with the rest of the group

b86cd78

fix: rename parameter in magic_function to prevent issue with redefin…

27c06aa

…ition of built-in function

test: add type ignores for TypedDict access in tool call validation

407a937

test: move error status tool test to be with rest of tool test group

afb1930

test: add straggler type ignore

fffd827

test: further type ignores and some pylint appeasement

e51f393

fix: correct typo in test description for HumanMessage

f9d084e

fix: "ToolMessage: contains..." was incorrectly bolded due to whitesp…

1a05580

…ace issue

fix: correct typo in height description from "METERs" to "METERS"

8aaad35

fix: move test for structured few-shot examples with tool calls to fo…

2ca77b6

…llow similar positioning as js

test: move test for binding runnables to more logical place

839bb12

mdrxy requested a review from ccurme June 23, 2025 14:54

dosubot bot added size:XXL This PR changes 1000+ lines, ignoring generated files. 🤖:nit Small modifications/deletions, fixes, deps or improvements to existing code or docs labels Jun 23, 2025

sydney-runkle reviewed Jun 23, 2025

View reviewed changes

libs/standard-tests/langchain_tests/integration_tests/chat_models.py Outdated Show resolved Hide resolved

sydney-runkle approved these changes Jun 23, 2025

View reviewed changes

dosubot bot added the lgtm PR looks good. Use to confirm that a PR is ready for merging. label Jun 23, 2025

ccurme approved these changes Jun 23, 2025

View reviewed changes

libs/standard-tests/langchain_tests/integration_tests/chat_models.py Outdated Show resolved Hide resolved

libs/standard-tests/langchain_tests/integration_tests/chat_models.py Show resolved Hide resolved

mdrxy added 2 commits June 23, 2025 16:45

refactor: rewrite assertions to prevent needing type ignores

cb29c79

refactor: simplify assertions for test_stream

b91ff98

vercel bot deployed to Preview June 23, 2025 21:04 View deployment

sydney-runkle reviewed Jun 23, 2025

View reviewed changes

refactor: replace direct dictionary access with safe get method in to…

d0ac6b8

…ol call validations plus additional clean ups

sydney-runkle approved these changes Jun 23, 2025

View reviewed changes

sydney-runkle enabled auto-merge (squash) June 23, 2025 23:18

vercel bot deployed to Preview June 23, 2025 23:22 View deployment

sydney-runkle merged commit 6d71b6b into master Jun 23, 2025
155 checks passed

sydney-runkle deleted the mdrxy/chat_models-updates branch June 23, 2025 23:22

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

standard-tests: refactoring and fixes #31703

standard-tests: refactoring and fixes #31703

mdrxy commented Jun 23, 2025

Uh oh!

vercel bot commented Jun 23, 2025 •

edited

Loading

Uh oh!

codspeed-hq bot commented Jun 23, 2025 •

edited

Loading

Uh oh!

codspeed-hq bot commented Jun 23, 2025 •

edited

Loading

Uh oh!

sydney-runkle commented Jun 23, 2025

Uh oh!

Uh oh!

sydney-runkle left a comment

Uh oh!

Uh oh!

Uh oh!

mdrxy commented Jun 23, 2025

Uh oh!

sydney-runkle left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

sydney-runkle left a comment

Uh oh!

Uh oh!

Uh oh!

standard-tests: refactoring and fixes #31703

standard-tests: refactoring and fixes #31703

Conversation

mdrxy commented Jun 23, 2025

Uh oh!

vercel bot commented Jun 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codspeed-hq bot commented Jun 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

CodSpeed WallTime Performance Report

Merging #31703 will not alter performance

Summary

Uh oh!

codspeed-hq bot commented Jun 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

CodSpeed Instrumentation Performance Report

Merging #31703 will not alter performance

Summary

Uh oh!

sydney-runkle commented Jun 23, 2025

Uh oh!

Uh oh!

sydney-runkle left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

mdrxy commented Jun 23, 2025

Uh oh!

sydney-runkle left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

sydney-runkle left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

vercel bot commented Jun 23, 2025 •

edited

Loading

codspeed-hq bot commented Jun 23, 2025 •

edited

Loading

codspeed-hq bot commented Jun 23, 2025 •

edited

Loading