Skip to content

feat: propagate api_base property for local LLM support#239

Open
ahmednabiled wants to merge 3 commits intomesa:mainfrom
ahmednabiled:feature/propagate-api-base
Open

feat: propagate api_base property for local LLM support#239
ahmednabiled wants to merge 3 commits intomesa:mainfrom
ahmednabiled:feature/propagate-api-base

Conversation

@ahmednabiled
Copy link

Description

This PR propagates api_base across the Mesa-LLM stack so users can configure local or self-hosted LLM endpoints (e.g., remote Ollama, vLLM, LM Studio) from application-level APIs.

Previously, api_base was supported in ModuleLLM but not consistently exposed through higher-level abstractions such as LLMAgent and memory modules, which made custom endpoint configuration incomplete in end-to-end usage.

This PR also removes hardcoded model usage in example memory initialization paths so summarization uses the configured llm_model.

Changes

  • Propagate api_base through LLMAgent, Memory, STLTMemory, EpisodicMemory, and LongTermMemory.
  • Forward api_base from top-level abstractions into ModuleLLM.
  • Update examples to pass api_base from app.py → model.py → agents.py.
  • Update docs for ModuleLLM, LLMAgent, and Memory with api_base usage examples.
  • Remove deprecated api_key references from documentation.

Checklist

  • I have read the CONTRIBUTING document.
  • My code follows project style guidelines.
  • I added/updated tests for new behavior.
  • I ran existing tests and all passed.
  • I updated documentation.

- Add api_base parameter through LLMAgent, Memory, and ModuleLLM layers
- Support custom API endpoints for self-hosted LLMs (e.g. remote Ollama)
- Fix hardcoded openai/gpt-4o-mini in example agent memory modules
- Update documentation in module_llm.md, llm_agent.md, and memory.md
- Remove deprecated api_key references from documentation
@coderabbitai
Copy link
Contributor

coderabbitai bot commented Mar 18, 2026

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 27b04dc8-2210-47df-86bf-2eafecd81204

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link

@IlamaranMagesh IlamaranMagesh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems related to the earlier discussion in #172, sharing it here for context.

@ahmednabiled
Copy link
Author

@jackiekazil @colinfrisch @IlamaranMagesh I'd appreciate any feedback on these changes. Are they relevant, good, or bad? I did this because when I first ran the examples, it was a bit confusing trying to run them using local models. This was my starting point.

@wang-boyu wang-boyu added the enhancement Release notes label label Mar 23, 2026
@codecov
Copy link

codecov bot commented Mar 23, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 90.78%. Comparing base (8fe8992) to head (f94b6f2).
⚠️ Report is 8 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #239      +/-   ##
==========================================
+ Coverage   90.67%   90.78%   +0.10%     
==========================================
  Files          19       19              
  Lines        1555     1573      +18     
==========================================
+ Hits         1410     1428      +18     
  Misses        145      145              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@wang-boyu
Copy link
Member

Thanks for the improvement. I've moved the new api_base parameter after all previous parameters to make the API backward compatible. Could you pull my changes and verify that it's still working for you?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement Release notes label

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants