Skip to content

refactor(ask-a-sailor): swap OpenAI for any-llm provider abstraction#25

Draft
Copilot wants to merge 3 commits into
mainfrom
copilot/add-llm-provider-abstraction
Draft

refactor(ask-a-sailor): swap OpenAI for any-llm provider abstraction#25
Copilot wants to merge 3 commits into
mainfrom
copilot/add-llm-provider-abstraction

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Mar 11, 2026

AskASailorAgent hard-codes OpenAI(), locking the agent to a single vendor. This swaps in mozilla-ai/any-llm so clubs can switch LLM providers via env var.

Agent refactor

  • Replace from openai import OpenAIfrom any_llm import completion, embedding
  • Remove self.client = OpenAI(); call module-level completion() / embedding() directly
  • Model resolved from LLM_MODEL env var, falling back to gpt-4o-mini
# Switch providers without code changes
LLM_MODEL=gpt-4o-mini              # OpenAI (default)
LLM_MODEL=mistral-small-latest     # Mistral
LLM_MODEL=anthropic:claude-3-haiku # Anthropic
LLM_MODEL=ollama:llama3            # Local/offline

Dependencies

  • openai>=1.20.0any-llm-sdk[openai]>=1.10.0 in both root and package requirements (openai remains as transitive dep)

Config & CI

  • New .env.example documenting LLM_MODEL and provider API keys
  • LLM_MODEL added to CI test env

Tests

  • test_agent_answer_with_mocked_any_llm — end-to-end with mocked completion/embedding
  • test_agent_respects_llm_model_env — verifies env var propagates through to the completion call

All 30 tests pass (28 original + 2 new), 6 integration tests skipped as before.

Original prompt

This section details on the original issue you should resolve

<issue_title>[ask-a-sailor] mozilla-ai/any-llm provider abstraction: swap LLMs without code changes</issue_title>
<issue_description>Currently AskASailorAgent hard-codes OpenAI. mozilla-ai/any-llm (github.com/mozilla-ai/any-llm) provides a single interface across OpenAI, Anthropic Claude, Mistral, and local models (Ollama) with one import swap. This is critical for: (1) cost flexibility -- switch to Mistral 7B for lower-cost club deployments, (2) offline/local mode -- llamafile for clubs with no cloud budget or data privacy requirements, (3) funder credibility -- not locked to any single vendor. Work: (1) pip install any-llm, refactor AskASailorAgent to use any_llm.get_llm(model=os.environ['LLM_MODEL']) instead of OpenAI() directly, (2) add LLM_MODEL env var to .env.example and CI, (3) verify tests still pass with mocked any-llm client, (4) document the swap in README.</issue_description>

Comments on the Issue (you are @copilot in this section)


✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

Copilot AI and others added 2 commits March 11, 2026 07:53
…gent

- Replace direct OpenAI calls with any_llm.completion() and any_llm.embedding()
- Add LLM_MODEL env var support (default: gpt-4o-mini)
- Update requirements.txt (root + ask-a-sailor) to use any-llm-sdk[openai]
- Create .env.example with LLM_MODEL and provider key docs
- Add LLM_MODEL to CI env vars
- Add mocked any-llm tests (2 new tests)
- Document provider swap in README

Co-authored-by: fullharbor <237832340+fullharbor@users.noreply.github.com>
Co-authored-by: fullharbor <237832340+fullharbor@users.noreply.github.com>
Copilot AI changed the title [WIP] Add LLM provider abstraction for AskASailorAgent refactor(ask-a-sailor): swap OpenAI for any-llm provider abstraction Mar 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[ask-a-sailor] mozilla-ai/any-llm provider abstraction: swap LLMs without code changes

2 participants