feat: Use structured outputs for more control over response #5195
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Looks good! I will try to find some time this weekend to toy around with it and make sure it works as expected, but based on my understanding of how structured outputs works this should work great.
This is correct, we mock the call to OpenAI (since otherwise, as you said, you'd need to provide a key). I'll test manually with my key when I get the time (and welcome others to do the same!). |
Where was the invalid invite? We should get that updated. Here's a new one: https://discord.gg/qA9zCWB5ay |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What this PR does / why we need it:
Hi,
This pull request leverages structured outputs to provide more granular control + guarantees over the responses from LLM providers. In a few of the usages in this project we expect a specific json object back. While mealie is already providing the expected json schema in the prompt this PR goes one step further and enforces that this schema is (allegedly) guaranteed to be respected in the response.
In theory this makes LLM calling more reliable and possibly makes prompting a bit more ergonomic in the future as well as pleading with LLMs to
please for the love of all that is holy return json in the desired format or else my whole family will dieshould be less necessary.Which issue(s) this PR fixes:
No specific related issue that I saw.
Special notes for your reviewer:
I tried to join the discord server but the invite was invalid. Happy to hear if I should have communicated prior to submitting this PR somewhere.
Testing
I ran unit test using
task py:checkin the devcontainer and gotI don't believe the tests actually executed any calls to OpenAI, especially since I didn't provide my API key, so I'm happy to take feedback or suggestions on additional testing I can do to validate this PR. I'm just getting started with mealie and don't have an extensive database of recipes nor URLs, etc, to test it on so would appreciate any suggestions how to go about that.