Batch API Support for Open AI

[Mahgozar]

Batch API offers significantly reduced costs specially for embedding models that you need a lot of content to be indexed

[logancyang]

OpenAI Batch API is not realtime and it takes up to 24hr to get the result. https://platform.openai.com/docs/guides/batch

[Mahgozar]

exactly because its not real time it can be useful for the first time you want to index a large vault, it's gonna take a long time anyway, might as well make it cost a lot less

[logancyang]

@Mahgozar are you ok with having your index ready in 24hr? Most people are not. Might as well just use a local model for free and it's much faster as well.