🧠 AI Agent Server (RAG + Memory + Plugins)

This project implements a pluggable AI Agent backend built with TypeScript (Node.js) that supports:

• Conversation memory per session
• RAG (Retrieval-Augmented Generation) from local markdown docs
• Plugin system (Weather + Math evaluation)
• LLM response generation using Google Gemini

---

✅ Features

• Endpoint: POST /agent/message
  Handles user queries with:
  • Session-based memory
  • Context retrieved from documents
  • Optional plugin outputs (weather, math)
• Vector Search for RAG: Uses cosine similarity with Gemini embeddings.
• Plugins:
  • Weather (OpenWeather API or mock fallback)
  • Math expression evaluation
• Prompt Engineering: System prompt includes:
  • Agent instructions
  • Last 2 messages as memory
  • Top 3 relevant context chunks
  • Plugin outputs

---

⚙️ Tech Stack

• Language: TypeScript
• Framework: Express
• LLM: Google Gemini (via @google/generative-ai)
• Embeddings: Gemini embedding-001
• Vector Similarity: compute-cosine-similarity
• Plugins: Axios for weather API
• Deployment: On Render

---

📂 Project Structure

ai-agent-server/
│
├── src/
│   ├── index.ts          # Server entry point
│   ├── routes/agent.ts   # Agent endpoint
│   ├── services/
│   │   ├── llm.ts        # Gemini LLM integration
│   │   ├── rag.ts        # RAG system with embeddings
│   │   └── plugins.ts    # Plugin execution (weather, math)
│   ├── memory.ts         # Session-based memory
│
├── documents/            # Markdown docs for RAG
├── .env                  # Environment variables
├── package.json
├── tsconfig.json
└── README.md

---

🔑 Environment Variables

Create a .env file in the project root:

PORT=8080
GEMINI_API_KEY=your_gemini_api_key
OPENWEATHER_API_KEY=your_weather_api_key  # optional

---

▶️ Setup and Run

1. Install dependencies:

npm install

2. Run in development:

npm run dev

3. Build and run:

npm run build
npm start

---

✅ API Usage

POST /agent/message

Request Body:

{
  "message": "search weather in Bangalore",
  "session_id": "123"
}

Response:

{
  "response": "The current weather in Bangalore is 28°C and sunny."
}

Sample Curl:

curl -X POST http://https://ai-agent-server-y85l.onrender.com/agent/message \
-H "Content-Type: application/json" \
-d '{"message":"What is markdown?","session_id":"123"}'

---

🧠 Agent Architecture & Flow

1. Receive user input → /agent/message
2. Store message in session memory
3. Retrieve last 2 messages for context
4. Perform RAG:
   • Embed query → Compute similarity with document chunks → Get top 3
5. Detect and execute plugins
6. Build prompt:
   • Instructions + Memory + RAG + Plugins
7. Call Gemini LLM → Generate response
8. Return response & update memory

---

🔌 Plugins

• Weather:
  • Uses OpenWeather API if key exists
  • Fallback to mock data
• Math Evaluator:
  • Evaluates arithmetic safely with Function()

---

✅ Deployment

• Use Render or Railway:
  • Set NODE_ENV=production
  • Add GEMINI_API_KEY in environment variables
• Start command:

npm run build && npm start

---

📚 Documents for RAG

• daext-blogging-with-markdown-complete-guide.md
• john-apostol-custom-markdown-blog.md
• just-files-nextjs-blog-with-react-markdown.md
• webex-boosting-ai-performance-llm-friendly-markdown.md
• wikipedia-lightweight-markup-language.md

---

✅ Author

Vividh Kanaujia