MCP Server Latency Test

[MervinPraison]

Discussed in #725

^{Originally posted by victoriaaajiang July 4, 2025}
Hello!
I'm building an MCP server using a local Ollama LLM. I want to measure latency breakdown for:

• Planning process
• Tool usage
• LLM answer generation

Which functions mark the start and end of each phase in the code?

I'm looking among agent.py, llm.py, task.py, mcp.py and main.py. But I find it quite hard to keep track of the interaction between functions.

Thanks!

[MervinPraison]

@claude is there any way to implement this , but with minimal code change

[claude]

Claude encountered an error —— View job

---

I'll analyze this and get back to you.

[MervinPraison]

@claude is there any way to implement this , but with minimal code change

[claude]

Claude encountered an error —— View job

---

I'll analyze this and get back to you.

[MervinPraison]

@claude is there any way to implement this , but with minimal code change

[claude]

Claude encountered an error —— View job

---

I'll analyze this and get back to you.

[MervinPraison]

@claude is there any way to implement this , but with minimal code change

[claude]

Claude encountered an error —— View job

---

I'll analyze this and get back to you.

[claude]

Claude finished @MervinPraison's task —— View job • claude/issue-733-20250705_064329 • Create PR ➔

---

✅ Latency Tracking Implementation for MCP Server

I've successfully implemented a minimal latency tracking solution for PraisonAI that measures performance across three key phases with minimal code changes.

Implementation Summary

1. Created Latency Tracking Module (src/praisonai-agents/praisonaiagents/monitoring/)

• latency_tracker.py - Core tracking implementation with thread-safe metrics collection
• __init__.py - Module exports for easy importing
• README_LATENCY.md - Comprehensive documentation

2. Minimal Code Changes

Added tracking to just 3 key locations:

• Planning Phase: Wrapped agent.chat() method with track_phase("planning")
• Tool Usage Phase: Wrapped agent.execute_tool() method with track_phase("tool_usage")
• LLM Generation Phase: Wrapped llm.get_response() method with track_phase("llm_generation")

3. Key Features

• Zero overhead when disabled - Uses environment variable PRAISON_LATENCY_TRACKING
• Thread-safe - Supports concurrent MCP requests
• Minimal intrusion - Only 3 lines added per tracked method
• Rich metrics - Tracks total time, count, average, min/max per phase

4. Usage Examples

Created two example files:

• examples/python/api/agent-latency-tracking.py - Basic agent usage
• examples/python/api/mcp-server-with-latency.py - MCP server integration

How to Use

1. Enable tracking:

   export PRAISON_LATENCY_TRACKING=true

   Or programmatically:

   from praisonaiagents.monitoring import enable_tracking
   enable_tracking()

2. Track requests:

   from praisonaiagents.monitoring import start_request, end_request
   
   start_request("my_request")
   response = agent.chat("What time is it?")
   metrics = end_request()
   
   print(f"Total: {metrics['total_time']:.3f}s")
   print(f"Planning: {metrics['phases']['planning']['total']:.3f}s")
   print(f"Tools: {metrics['phases'].get('tool_usage', {}).get('total', 0):.3f}s")
   print(f"LLM: {metrics['phases']['llm_generation']['total']:.3f}s")

3. Get summary:

   from praisonaiagents.monitoring import get_latency_summary
   summary = get_latency_summary()

Functions that Mark Phase Boundaries

As requested, here are the key functions that mark the start and end of each phase:

1. Planning Process:

   • Start: Agent.chat() method entry (line 1213 in agent.py)
   • End: When chat() returns or calls LLM

2. Tool Usage:

   • Start: Agent.execute_tool() method entry (line 897 in agent.py)
   • End: When tool execution completes

3. LLM Answer Generation:

   • Start: LLM.get_response() method entry (line 290 in llm.py)
   • End: When LLM API call completes

Changes Made

• ✅ Created monitoring module with latency tracking
• ✅ Added imports and context managers to 3 key methods
• ✅ Created examples and documentation
• ✅ Committed all changes to branch

Branch: claude/issue-733-20250705_064329

[MervinPraison]

@claude I can see that you are making numerous changes in the core files. Make it minimal.

Try to use existing. Mostly try to use the tracking as an external tool or as a custom tool if possible, rather than integrating it with the core

[claude]

Claude finished @MervinPraison's task —— View job • claude/issue-733-20250705_072344 • Create PR ➔

---

✅ Minimal Latency Tracking Solution Created

I've successfully created a minimal latency tracking solution as a custom tool without modifying any PraisonAI core files.

Todo List:

• Read existing codebase to understand custom tool implementation
• Create a custom latency tracking tool
• Create wrapper functions for tracking without core modifications
• Add examples showing how to use the tool
• Test and document the solution

What I Created:

1. latency_tracker_tool.py - Core tracking implementation with multiple integration methods:

   • Custom tool function for agents
   • Decorators for automatic tracking
   • Context managers for fine-grained control
   • Wrapper classes for transparent tracking

2. Multiple Example Files:

   • minimal_latency_example.py - Simplest usage example
   • example_latency_tracking.py - Comprehensive examples
   • mcp_server_latency_example.py - MCP server specific integration
   • tools_with_latency.py - Example tools.py with built-in tracking

3. README_LATENCY_TRACKING.md - Complete documentation

Key Features:

✨ Zero Core Modifications - All tracking is external to PraisonAI core files
🔧 Multiple Integration Options:

• Use as a custom tool that agents can call
• Use decorators on your functions
• Use context managers for manual tracking
• Use wrapper classes for automatic tracking

📊 Tracks All Three Phases:

• Planning process
• Tool usage
• LLM answer generation

🔒 Thread-Safe - Supports concurrent MCP requests

Quick Usage Example:

from latency_tracker_tool import tracker

# Track MCP request phases
with tracker.track("planning", "request_1"):
    plan = agent.chat("Plan the task")

with tracker.track("tool_usage", "request_1"):
    result = execute_tools()

with tracker.track("llm_generation", "request_1"):
    response = agent.chat("Generate response")

# Get metrics
metrics = tracker.get_metrics("request_1")

Branch: claude/issue-733-20250705_072344