feat: Import + Translate recipe images with OpenAI

[michael-genson]

What type of PR is this?

(REQUIRED)

• feature

What this PR does / why we need it:

(REQUIRED)

This adds a new recipe creation method to create recipes from images using AI. Optionally, users may opt to translate the recipe into their own language as well.

Here's the new interface:

Copying the work from recipe timeline events, you can rotate/crop the image after uploading it:

It can take a while, so we have a loading text/animation:

And voilà, a handwritten recipe translated from German to English:

I don't speak German, so I hope it's accurate 😅 But it seems to check out (ingredient/step counts, similar words, etc.).

Which issue(s) this PR fixes:

(REQUIRED)

Closes #660
Closes #1588
Closes #3899
Kind of addresses #3011 (see special notes below)

Special notes for your reviewer:

(fill-in or delete this section)

The translate feature is super helpful even without the image processing. At some point I plan to have a more general "Import with AI" that can accept URLs, text, images, etc.

Testing

(fill-in or delete this section)

Tested with a ton of images of recipes from Google, as well as some cookbooks I have at home. Works great!

[lindell]

This is great! I've tested this on a few Swedish recipes and it generally works as expected.

I will add some examples of where I find things going wrong. Most might be what gpt4-o returned. But it might also be something we might improve upon with better prompting. Nothing critical, but some might be fixable.

Example one:

1. I got a servings/total_yield of 4 port, ca 20 min. For some reason, it added the total time to it. Prompting looks reasonable, I think this is mostly an error from the model.
2. 1. / 2. etc was included as a part of the recipe instructions. I see this is in the json schema. So I'm not sure there is anything to do about it?

The recipe:

[freakshock88]

This is a great idea indeed!
Just another idea.. could this not be easily adapted to parse unsupported (unscrapable) recipe websites?
A workaround is of course to make a screenshot.. but a url would be more user friendly.

[lindell]

This is a great idea indeed! Just another idea.. could this not be easily adapted to parse unsupported (unscrapable) recipe websites? A workaround is of course to make a screenshot.. but a url would be more user friendly.

@michael-genson has already added this! 😄
#3690

[freakshock88]

This is a great idea indeed! Just another idea.. could this not be easily adapted to parse unsupported (unscrapable) recipe websites? A workaround is of course to make a screenshot.. but a url would be more user friendly.

@michael-genson has already added this! 😄 #3690

Oh sorry about that, I tried it before I fixed my OpenAI setup I see now ;) It works indeed, awesome! Thanks for your response!

[lindell]

A note on the translation.

I use Mealie (and all other software) in English. But I and those I share the instance with have all my recipes in my first language (Swedish). To translate a recipe, I have to change the Mealie language to Swedish, to the importing, and then change the language back 😛

I will almost never use the translate feature, so it does not matter for me. But thought it was worth noting.

[michael-genson]

1. / 2. etc was included as a part of the recipe instructions

Yeah I tried building that into the prompt:

Sometimes, but not always, recipes will include their number in front of the text, such as ... In the case where they are directly numbered ... you should not include the number in the text.

It just seems to ignore it sometimes ¯\_(ツ)_/¯

To translate a recipe, I have to change the Mealie language to Swedish, to the importing, and then change the language back

I figure this is an edge-case, but maybe we can build it into the UI somehow. Maybe in the next iteration

[williamkray]

wow this is rad, i have a ton of recipe cards we've saved from our old meal-box plans back in the day (blue apron, martha & marley spoon, etc) and would love to be able to import them just by running them through the scanner!

[boc-the-git]

🚀

[felixschndr]

This is really cool! Showed the feature to my girlfriend, has huge wife-approvalfactor!

[boc-the-git]

This is really cool! Showed the feature to my girlfriend, has huge wife-approvalfactor!

This feature got you married @felixschndr ?
@michael-genson works wonders!

[felixschndr]

This feature got you married @felixschndr ?

(Sadly) not :D I didn't know wheter there was a word for girlfriend approval factor so I set wife in italic to mean girlfriend ^^