support base64 for embeddings

[stillmatic]

Is your feature request related to a problem? Please describe.
The OpenAI embeddings API has an option to return the results as base64 encoded float32's. The Python library uses it to retrieve the results: https://github.com/openai/openai-python/blob/main/openai/api_resources/embedding.py#L25

Doing so would result in a small performance improvement - the results can be marshaled directly to float32 instead of being sent over the wire as JSON float (untyped) and converting back to float32.

Describe the solution you'd like

The embeddings method should include encoding_format=base64 in its call to OpenAI and decode the results.

Describe alternatives you've considered

The python library uses this unless the encoding format is set to anything. I think that's reasonable, and we should match the Python behavior.

Additional context
https://news.ycombinator.com/item?id=37135823

[vvatanabe]

@stillmatic That sounds very convenient. However, the encoding_format is not officially documented. It may be removed or changed in the future.

[stillmatic]

It's possible. However, the official Python package uses that encoding format, so I think the risk of OpenAI changing it is quite low - they need to support users of the official packages, even if outdated. Backwards compatibility 😉

[henomis]

Here a pull request that implements the base64 support: #485

[stillmatic]

closed in #485 , thanks @henomis !!!