Development Roadmap (2025 S1)

[huangyz0918]

For anyone who wants to contribute (add features, report bugs, or just simply discuss and learn), join our Discord 👋
Or you can just comment here for open discussions! 👨‍💻

RAG Module for Code Indexing

• Complete a PoC with LanceDB by implementing a memory.py file. [MRG] add lancedb as memory #262 @leeeizhang
• Build a pipeline to generate embeddings for local code RAG. Enhance Local Code Generation with RAG Module #259 [MRG] Code RAG for Chatbot #265 @huangyz0918
• Enable storing the "successfully" project (generated code, plan, suggestions, and settings), for RAG and enhancement.
• Add a CLI entry for managing memory (list embedded files, allow CRUD). @leeeizhang Add manage API for the memory #266 Add a set of new CLI commands, called mle memory which provides memory CRUD #270

Research Topic

FYI @HuaizhengZhang

• PoC: How to sync up the knowledge updating (e.g., the code will update frequently)
• PoC: How to efficiently scan the file as memory? (the embedding costs time, for a large amount of codebase/files)
• PoC: How to do the chunking for code different types of textual (image/audio, other modality) data
• PoC: Using graphRAG for overall (code) information summarization

Enhance mle chat

• Improved the mle chat's agent calling, allow calling agents (now we can call functions) Enable unordered agent interactions in mle chat #260 @YuanmingLeee

Prompting

• Integrated the reference code search to Advisor/Debugger/Coder
• Add tracking tool, with integration to Langfuse or other tools. Set the default tracking to False, or letting users know while first run the mle in the terminal about usage tracking. [Prompt Ops] It seems that we are continuously improving our prompts. now its time to use some prompt tracking tools to help us do some A/B testings #255

Other LLM Framework Support

• Integrate with the Ollama backend. (Ollama is added, test and make it better) -- local
• Integrate with vLLM backend -> vLLM cloud backend -- on-prem

Function Calls

• Add local logging of tool/function calling.
• Added functions to preview multi-modality data, start with the image data.
• Function store: [Function Store] shall we build a function store to avoid change function code everytime? #268
• Added support for local LLMs (tested the APIs with vLLM), disable the function calling without errors if the LLM doesn't support. [llama3.2] test new llama3.2 1b & 3b models with MLE-agent #227

Documentation

• Updated the doc site with new release/features @huangyz0918 @leeeizhang
• New demo video on in the README.md @HuaizhengZhang

[TimeLordRaps]

What are your thoughts on the library nano-graphrag?

[huangyz0918]

@TimeLordRaps Do you mean this https://github.com/gusye1234/nano-graphrag?

I think it is an elegant, small, and clean implementation of GraphRAG. However, to implement GraphRAG, a graph-based data store and (usually) a KV store must be introduced, which brings problems in 1) extract storage/dependency and 2) compatibility with other stores. Moreover, I am not sure how such graph-based indexing performs on code generation tasks.

But I think it is worth trying in the chat mode for our project -- since the user may ask very high-level, or summarized questions based on the large code base. For the advisor it can also help, but the very first problem is how we handle the function call with the graph search (maybe use graph query for project summarization as the pre-retrieval before calling function like web search).

[TimeLordRaps]

https://github.com/CEDARScript/cedarscript-grammar This should help. A friend of mines project I'm helping work forward into an eventual shared resource of personified coding graphs. Ie think aiders --edit-format, which is where its currently being applied, that said, we were just discussing branching out to other coding frameworks. Combining cedarscript with a graph database has power that hasn't been tested yet. That said cedarscript improved gemini-1.5-flash refactored benchmark with these highlights:

48% of tests (43 total) showed improvements
103% increase in Pass 1 success rate (75 tests)
Test duration reduced by 93% (from 5:17:26 to 0:25:17)
Token efficiency greatly improved:
Sent tokens: -37% (7.59M)
Received tokens: -96% (180K)
Error reduction:
Error outputs: -94% (35 total)
Malformed outputs: -94% (6 cases)
Syntax errors: -85% (3 cases)
Indent errors eliminated (100% reduction)