Skip to content

[Feature]: Set BaseURL for Embedding, reranker and LLM models #340

@qdrddr

Description

@qdrddr

Use Case

I'd like to be able to easily switch between the providers and models and specify a different BaseURL for Embedding, Reranker, and LLM models.

Problem Statement

I could not specify a proxy to cache my model requests. Such as LiteLLM Proxy or CloudFlare AI Gateway proxy.

How This Feature Would Help

Saves money. Improves speed for cached requests.

Proposed Solution

Implement LiteLLM Client SDK so I could set BASE_URL and switch easily between models and providers.

Alternatives Considered

No response

Priority

Important - affects my workflow

Additional Context

No response

Checklist

  • I would be willing to contribute this feature

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions