Skip to content

Conversation

@aymeric-roucher
Copy link
Contributor

@aymeric-roucher aymeric-roucher commented Jan 19, 2025

What does this PR do?

Adds more supported types in chat templates.
Why not use an empty type for "any"?
=> Sometimes users want to specify a type hint like this:

def get_weather(location: Any) -> None:
    return "Nice weather today."

=> In that case, I think it makes it more explicit to have the type hint being precised in the json schema as "any" rather than left empty as is currently the case.

I don't know if these changes are compatible with the current usage of chat template in transformers (in particular, maybe some models are trained with the convention to let the type for 'Any' blank), but they allow finer definition of inputs which is useful for using transformers chat templates in smolagents. Let me know if this is an appropriate change @Rocketknight1!

@CISC
Copy link
Contributor

CISC commented Jan 20, 2025

any is not part of the JSON Schema specification, so I think this should be left unset.

However setting NoneType to null is the right thing to do and seems to have been an omission.

@Rocketknight1 I noticed some of your chat templates are missing the handling of null as well, and as I've reused your json_to_python_type macro I added this in f.ex. MiniCPM-4B and Dria-Agent models' templates.

@Rocketknight1
Copy link
Member

Seconding @CISC's comment, I don't think we can use {"type": "any"}, but the type for None is definitely appreciated!

@aymeric-roucher
Copy link
Contributor Author

That's updated @Rocketknight1 ! Thanks to you and @CISC for your feedback! 🤗

Copy link
Member

@Rocketknight1 Rocketknight1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@aymeric-roucher aymeric-roucher merged commit 44393df into main Jan 20, 2025
26 checks passed
@aymeric-roucher aymeric-roucher deleted the tool-calling-add-json-schema-types branch January 20, 2025 18:15
bursteratom pushed a commit to bursteratom/transformers that referenced this pull request Jan 31, 2025
* Tool calling: support NoneType for function return type
elvircrn pushed a commit to elvircrn/transformers that referenced this pull request Feb 13, 2025
* Tool calling: support NoneType for function return type
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants