Agents/monitor

[pankhuri-mt]

PR Review: Agent Module & Monitor Agent

Context & Motivation

This PR introduces a new Agent Module for Mindtrace addressing following (A sub sub issue under Github Issue #310):

1. Discoverability: Previously, agents and their capabilities were not easily discoverable. Users had to know exactly which agent to use and how to configure it.
2. Configurability: Agent configuration was scattered and difficult to manage. There was no unified way to configure LLM providers, models, and agent-specific settings.
3. Extensibility: Adding new agents required significant boilerplate and lacked a standardized pattern.

This PR solves these issues by introducing:

• A registry-based agent discovery system (AgentRegistry)
• A unified configuration framework (BaseAgentWorkflowConfig) that supports multiple LLM providers and agent models
• A generic CLI interface that automatically discovers and exposes registered agents
• A monitor agent use case demonstrating natural language to LogQL query generation for Loki

Key Features

1. Discoverability & Registry System

The AgentRegistry provides a centralized way to discover and register agents:

# Agents can be registered with metadata
AgentRegistry.register(
    name="monitor",
    description="AI-powered log monitoring and analysis agent",
    cli_module="mindtrace.agents.monitor.cli",
    cli_class="MonitorAgentCLI",
)

# Users can list all available agents
mindtrace agents --list

2. Configurability & BaseAgentWorkflowConfig

The BaseAgentWorkflowConfig extends CoreConfig to provide a unified configuration system:

Key Components:

• LLM Providers: Support for multiple providers (OpenAI, Ollama) with flexible configuration
• Agent Models: One agent feature can have multiple models with different purposes
• Environment Variable Overrides: Configuration can be overridden via environment variables using __ delimiter

3. Monitor Agent Use Case

The Monitor Agent demonstrates the framework's capabilities with a real-world use case:

Natural Language → LogQL Query Generation

The monitor agent converts natural language queries into LogQL queries for Loki:

User: "Show me all error logs for EchoService"
→ LogQL: {service_name="EchoService"} |= "error"

How It Works:

1. Query Generator Agent: Takes natural language input and generates LogQL queries

   • Uses extract_logs tool to sample logs if needed
   • Validates queries by checking if logs exist
   • Retries with error feedback if query fails (up to 3 retries)

2. Log Analyzer Agent: Analyzes extracted logs and provides insights

   • Takes user query and log data
   • Returns structured JSON response with insights

3. Loki Integration: Direct integration with Loki API

   • Queries Loki using LogQL
   • Handles time ranges (default: last 7 days)
   • Returns structured log data with timestamps and labels

Example Flow:

# CLI usage
mindtrace agents monitor query -s MockPLCService -q "check if any errors exist" 

# Programmatic usage
agent = MonitorAgent()
result = await agent.query("show errors", "MockPLCService")
# Returns: {"logql": "{service_name=\"MockPLCService\"} |= \"error\"", 
#           "AI_message": "Found 5 error logs...", 
#           "sample_logs": [...]}

Commands

Agent Discovery

# List all available agents
mindtrace agents --list

# Get help for agents
mindtrace agents --help

Monitor Agent Commands

# Configure monitor agent
mindtrace agents monitor configure --config '{"LOKI_URL": "http://localhost:3100"}'

# Query logs using natural language
mindtrace agents monitor query \
  --query "show all errors for the last hour" \
  --service EchoService

Architecture: Agent & Workflow Pattern

Core Concept

Agent = Class, Methods = Workflows

An agent is a class that inherits from BaseAgent. Each method in the agent class represents a workflow - a specific command or feature the agent provides. A BaseAgent acts as the boundary for an agent domain. It can orchestrate multiple models and providers, with domain-specific agent commands exposed as methods on the class.

Example:

class MonitorAgent(BaseAgent):
    agent_name = "monitor"
    
    # Workflow 1: Query logs
    async def query(self, query: str, service: str) -> dict:
        # Natural language → LogQL → Loki → Analysis
        ...
    
    # Workflow 2: Set alert (future)
    async def set_alert(self, condition: str, service: str) -> dict:
        # Define alert conditions
        ...
    

Multiple Commands per Agent

Agents can support multiple workflows, each serving a different purpose. For instance a Monitoring Agent can have following features:

1. Chat Workflow: Interactive conversation with the agent
2. Query Logs Workflow: Natural language log querying (implemented in monitor agent)
3. Set Alert Workflow: Define and manage alert conditions
4. Log Analyzer: RCA analysis

Each workflow is a method that can be:

• Exposed via CLI commands
• Served as HTTP endpoints (via Service integration)
• Triggered by hooks from frontends or other agents

Chat Interface & Service Integration

Current State:

• CLI provides command-based interface (query, configure)
• Each workflow is exposed as a CLI command

Future Vision:

• Generic Chat Interface: A unified chat CLI that works with any agent

  mindtrace agents chat monitor
  # Interactive chat session with monitor agent

• Service Integration: Agents can be served as FastAPI services

  # Agent methods become HTTP endpoints
  class MonitorAgentService(Service):
      def __init__(self):
          self.agent = MonitorAgent()
          self.add_endpoint("/query", self.agent.query, schema=QuerySchema)
          self.add_endpoint("/chat", self.agent.chat, schema=ChatSchema)

• Workflow Hooks: Workflows can be triggered by:

  • Frontend applications (via HTTP/MCP)
  • Other agents (programmatic calls)
  • Scheduled tasks
  • Event-driven triggers

Testing & Examples

See samples/agents/monitor_setup/plc_service/ for:

• Complete setup with Loki, Grafana, Promtail
• Mock PLC service generating logs