Skip to content

fix: pass encoding_format="float" in LiteLLM SDK embedding calls#434

Merged
nicoloboschi merged 1 commit intovectorize-io:mainfrom
franchb:fix/litellm-encoding-format
Feb 25, 2026
Merged

fix: pass encoding_format="float" in LiteLLM SDK embedding calls#434
nicoloboschi merged 1 commit intovectorize-io:mainfrom
franchb:fix/litellm-encoding-format

Conversation

@franchb
Copy link
Contributor

@franchb franchb commented Feb 24, 2026

Summary

  • Explicitly pass encoding_format="float" in all litellm.embedding() / litellm.aembedding() calls in LiteLLMSDKEmbeddings
  • DeepInfra (and other OpenAI-compatible providers) reject requests when encoding_format is null/None
  • LiteLLM defaults encoding_format to None to prevent the OpenAI SDK from using "base64", but this None value causes 400 errors on providers that validate the field

Problem

When using DeepInfra embedding models (e.g., Qwen/Qwen3-Embedding-8B, BAAI/bge-small-en-v1.5) via the litellm-sdk provider, requests fail with:

BadRequestError: Error code: 400 - 'encoding_format' only support with [float, base64]

This happens because:

  1. LiteLLM explicitly sets encoding_format=None when not specified (source) to override OpenAI SDK's default of "base64"
  2. DeepInfra's API validates the field and rejects null — it only accepts "float" or "base64"

This is a known issue across the ecosystem: microsoft/graphrag#2194, BerriAI/litellm#12110, RooCodeInc/Roo-Code#7199.

Fix

Pass encoding_format="float" explicitly. This is always safe because LiteLLMSDKEmbeddings.encode() returns list[list[float]] — float is the only compatible format.

Test plan

  • Unit tests updated to verify encoding_format="float" is passed
  • Run cd hindsight-api && uv run pytest tests/test_litellm_sdk_embeddings.py -v

🤖 Generated with Claude Code

DeepInfra rejects requests when encoding_format is null. LiteLLM sets
it to None by default, so we explicitly pass "float" — the only format
compatible with our list[list[float]] return type.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Copy link
Collaborator

@nicoloboschi nicoloboschi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@nicoloboschi nicoloboschi merged commit 58f2de7 into vectorize-io:main Feb 25, 2026
17 of 31 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants