docs: Add comprehensive MCP agent integration guide and examples #1509

datradito · 2025-12-29T23:50:45Z

Overview

This PR adds comprehensive documentation, examples, and tests for the MCP server introduced in PR #1502. It provides developers with complete guidance on integrating AI agents with Echidna's fuzzing campaigns through the Model Context Protocol interface.

What's Added

📚 Documentation

AGENT_TESTING_GUIDE.md (430+ lines) - Complete testing and integration guide covering:
- Quick start setup instructions
- Manual testing with curl
- Python integration snippets
- Troubleshooting common issues
- Custom agent development patterns

🤖 Agent Examples

examples/simple_agent.py - Autonomous monitoring agent
- Real-time fuzzing progress tracking
- Coverage stagnation detection
- Automated transaction injection
- Quick test mode for validation
examples/langgraph_agent.py - LLM-powered agent
- AI-driven coverage analysis with Claude
- Intelligent transaction sequence generation
- Autonomous fuzzing strategy decisions
VS Code + GitHub Copilot Integration
- .vscode/mcp.json configuration guide
- Conversational AI-driven fuzzing workflow
- No-code agent integration

🧪 Testing Suite

test-mcp-client.py - Validates all 7 MCP tools:
- status - Campaign status and metrics
- show_coverage - Coverage reports
- target - Contract ABI information
- inject_fuzz_transactions - Priority injection
- clear_fuzz_priorities - Reset priorities
- reload_corpus - Corpus reloading
- dump_lcov - LCOV coverage export
tests/mcp/ - Comprehensive test infrastructure:
- pytest integration tests
- JSON schema validators
- Test contracts (EchidnaMCPTest.sol, SimpleToken.sol)
- Helper scripts for validation

🔧 Python Dependencies

Added .gitignore patterns for Python artifacts
requirements.txt for test dependencies

Key Features

✅ Critical Fix: Interactive UI Blocking

Documented the required --format text flag to disable Echidna's interactive TUI, which blocks the MCP server thread. This is essential for proper MCP operation.

echidna contract.sol --server 8080 --format text

- Add test-mcp-client.py for tool validation - Add examples/simple_agent.py (autonomous monitoring) - Add examples/langgraph_agent.py (LLM-powered agent) - Add AGENT_TESTING_GUIDE.md (comprehensive testing guide) - Add tests/mcp/ directory with test suite and contracts - Update .gitignore for Python/MCP artifacts

The clean PR branch is based on upstream/dev-agents (PR crytic#1502) which uses the '--server' flag name. Documentation was written for the development branch which used '--mcp-port', causing inconsistency. Changes: - AGENT_TESTING_GUIDE.md: Update all command examples - test-mcp-client.py: Fix error message - examples/README.md: Update all 3 command examples - examples/simple_agent.py: Fix error message - examples/langgraph_agent.py: Fix error message - tests/mcp/conftest.py: Fix pytest fixture command - .gitignore: Add Python cache patterns This ensures documentation matches the actual upstream implementation.

The interactive TUI blocks the MCP server thread. Users must run echidna with '--format text' to disable the TUI when using MCP. Also added --contract flag to examples for better clarity. Tested: All 5 MCP tools working correctly with these changes.

- Add tests for reload_corpus and dump_lcov tools (read_logs is commented out in upstream) - Add VS Code + GitHub Copilot integration section to AGENT_TESTING_GUIDE.md - Document .vscode/mcp.json configuration for Copilot users - Handle reload_corpus errors gracefully (can fail if corpus is locked) - Update expected output to show 7/7 tools passing - Include usage example for conversational AI-driven fuzzing with Copilot

CLAassistant · 2025-12-29T23:50:55Z

All committers have signed the CLA.

Copilot

Pull request overview

This PR adds comprehensive documentation, examples, and testing infrastructure for the MCP (Model Context Protocol) server integration with Echidna. It enables AI agents to interact with Echidna's fuzzing campaigns through a standardized protocol, providing both simple and advanced (LLM-powered) agent examples for monitoring and guiding fuzzing operations.

Key Changes:

Complete test suite with pytest fixtures, schema validation, and integration tests covering all MCP tools
Two agent examples: a simple autonomous monitoring agent and an advanced LangGraph-based AI agent using Claude
Comprehensive documentation including setup guides, troubleshooting, and VS Code integration instructions

Reviewed changes

Copilot reviewed 28 out of 29 changed files in this pull request and generated 32 comments.

Show a summary per file

File	Description
tests/mcp/test_*.py	Comprehensive pytest test suite covering all 9 MCP tools with schema validation, performance tests, and integration workflows
tests/mcp/conftest.py	pytest fixtures providing MCP client and Echidna campaign management for testing
tests/mcp/scripts/*.py	Helper utilities for MCP client wrapper and JSON schema validation
tests/mcp/contracts/*.sol	Test Solidity contracts (SimpleToken, EchidnaMCPTest) for validating MCP functionality
tests/mcp/contracts/*.json	JSON schemas defining expected response structures for each MCP tool
tests/mcp/requirements.txt	Python dependencies for testing framework and LangChain integration
examples/simple_agent.py	Autonomous monitoring agent with coverage stagnation detection and transaction injection
examples/langgraph_agent.py	Advanced LLM-powered agent using LangGraph for intelligent fuzzing strategy
examples/README.md	Documentation for agent examples with usage instructions and strategy ideas
AGENT_TESTING_GUIDE.md	Comprehensive guide covering server setup, manual testing, agent integration, and troubleshooting
test-mcp-client.py	Command-line test script validating all 7 core MCP tools
.gitignore	Added patterns for Python cache files and temporary test artifacts

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2025-12-29T23:56:03Z

tests/mcp/scripts/mcp_client_wrapper.py

+        self.base_url = base_url
+        self.client = httpx.Client(timeout=30.0)
+
+    def _call_tool(self, tool_name: str, parameters: Dict[str, Any]) -> Dict[str, Any]:


The MCP client wrapper has inconsistency between the method name and the tool it calls. The method is named call_tool but internally uses the generic _call_tool helper, which is correct. However, there's a public method call_tool that should be named something more specific like call_mcp_tool to match the internal implementation pattern, or the internal helper should be named differently to avoid confusion with the public API.

Copilot · 2025-12-29T23:56:04Z

tests/mcp/conftest.py

+
+    # Build command
+    cmd = [
+        'echidna-test',


The conftest fixture spawns an Echidna subprocess but uses 'echidna-test' as the command. This should likely be just 'echidna' to match the actual binary name used elsewhere in the documentation (e.g., in AGENT_TESTING_GUIDE.md where it uses '~/.local/bin/echidna'). The inconsistency may cause the test fixture to fail.

Suggested change

'echidna-test',

'echidna',

Copilot · 2025-12-29T23:56:04Z

tests/mcp/conftest.py

+        contract_path,
+        '--server', str(port),
+        '--test-mode', 'assertion',
+        '--test-limit', '1000'


The test fixture does not include the '--format text' flag when starting Echidna, which according to the PR description and AGENT_TESTING_GUIDE.md is a critical requirement to disable the interactive TUI that blocks the MCP server thread. Without this flag, the MCP server may not function correctly.

Suggested change

'--test-limit', '1000'

'--test-limit', '1000',

'--format', 'text',

Copilot · 2025-12-29T23:56:04Z

tests/mcp/scripts/mcp_client_wrapper.py

+    """
+    Enhanced MCP client with convenience methods for each tool.
+
+    Wraps httpx and provides typed methods for all 9 MCP tools.


The docstring comment claims this provides "typed methods for all 9 MCP tools" but the wrapper only implements methods for 8 tools (read_logs, show_coverage, dump_lcov, get_corpus_size, inspect_corpus, find_transaction, inject_transaction, prioritize_function, clear_priorities). The count should be updated to match the actual number of tools implemented.

Suggested change

Wraps httpx and provides typed methods for all 9 MCP tools.

Wraps httpx and provides typed methods for all 8 MCP tools.

AGENT_TESTING_GUIDE.md

Copilot · 2025-12-29T23:56:11Z

tests/mcp/test_corpus.py

+Tests for corpus inspection MCP tools.
+"""
+
+import pytest


Import of 'pytest' is not used.

Suggested change

import pytest

Copilot · 2025-12-29T23:56:12Z

tests/mcp/test_coverage.py

+Tests for show_coverage and dump_lcov MCP tools.
+"""
+
+import pytest


Import of 'pytest' is not used.

Suggested change

import pytest

Copilot · 2025-12-29T23:56:12Z

tests/mcp/test_coverage.py

+
+import pytest
+import time
+import re


Import of 're' is not used.

Suggested change

import re

Copilot · 2025-12-29T23:56:12Z

tests/mcp/test_read_logs.py

+
+import pytest
+import time
+from tests.mcp.scripts.mcp_client_wrapper import MCPClientV2


Import of 'MCPClientV2' is not used.

Suggested change

from tests.mcp.scripts.mcp_client_wrapper import MCPClientV2

Copilot · 2025-12-29T23:56:12Z

tests/mcp/test_schemas.py

+Tests validating MCP tool responses against JSON schemas.
+"""
+
+import pytest


Import of 'pytest' is not used.

Suggested change

import pytest

…outdated links

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

- Update examples/README.md to list correct 7 tools - Fix mcp_client_wrapper.py to use upstream tool names: * inject_fuzz_transactions (not inject_transaction) * clear_fuzz_priorities (not clear_priorities) * status, target, show_coverage, reload_corpus, dump_lcov - Mark old test files as skipped (use old tool names): * test_corpus.py - get_corpus_size, inspect_corpus, find_transaction * test_injection.py - inject_transaction (old signature) * test_prioritization.py - prioritize_function * test_read_logs.py - read_logs (commented out in upstream) - Update docstring: 7 active tools (not 9) These tests are preserved for reference but skipped until updated to match upstream API from PR crytic#1502.

- Rewrite test_command_logging.py: Fix call() → call_tool(), update tool names, fix response parsing - Rewrite test_schemas.py: Test 7 upstream tools instead of 9 old tools (all 7 passing) - Rewrite test_integration_workflows.py: Use upstream tools in workflows (status, inject_fuzz_transactions, clear_fuzz_priorities, show_coverage, dump_lcov, target) - Rewrite test_coverage.py: Fix tool references and response parsing for MCP format - Fix test_read_logs.py: Skip test_read_logs_empty_params (tool commented out) - Fix conftest.py: Use 'echidna' instead of 'echidna-test', add --format text flag All tests now use upstream's 7 active tools: - status: Campaign status and metrics - target: Contract name and ABI - show_coverage: Coverage report with annotated source - dump_lcov: LCOV format coverage export - reload_corpus: Reload corpus from disk - inject_fuzz_transactions: Inject transaction sequence (string parameter) - clear_fuzz_priorities: Clear all priorities Test results: - test_schemas.py: 7/7 PASSING ✅ - Old tool tests: 24/24 SKIPPED (correctly marked) - Command logging/workflows: Need running echidna campaign - Coverage tests: Need running echidna campaign Removed references to old/non-existent tools: - read_logs (commented out in upstream) - get_corpus_size (doesn't exist) - inspect_corpus_transactions (doesn't exist) - find_transaction_in_corpus (doesn't exist) - inject_transaction (old name, wrong signature) - prioritize_function (doesn't exist) - clear_priorities (old name)

- Added logging for control commands in MCP server. - Updated pytest fixture to use temporary directories for campaigns. - Modified test cases to utilize new fixture and ensure proper logging. - Increased timeout for read_logs tests to prevent false failures.

gustavo-grieco

I like the direction of this PR, here is a couple of comments:

Why exactly we need to log all the commands to a file in the MCP?
Start addressing all the copilot comments, discarding with a comment the ones that makes no sense.
You added some basic testing code for the MCP using Python, that's a great, but we need to make sure it is used by our CI. Include a new CI test for that (which should run in parallel). If you have questions about that, @elopez is our Github Action expert.
The added documentation is very useful. We usually have the documentation for tools like echidna in building-secure-contracts, but we usually update it when it is close to release. However, don't remove the documentation, we can keep it until we are more confident on how to use this agentic capabilities, and move them into building secure contracts when we are close to release.

datradito added 4 commits December 29, 2025 17:32

Copilot AI review requested due to automatic review settings December 29, 2025 23:50

datradito requested review from arcz, elopez and gustavo-grieco as code owners December 29, 2025 23:50

Copilot started reviewing on behalf of datradito December 29, 2025 23:51 View session

Copilot AI reviewed Dec 29, 2025

View reviewed changes

datradito and others added 5 commits December 29, 2025 21:02

docs: Update README with placeholder for Echidna MCP path and remove …

a35e254

…outdated links

Update AGENT_TESTING_GUIDE.md

e2b3262

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

gustavo-grieco requested changes Jan 2, 2026

View reviewed changes

	'--test-limit', '1000'
	'--test-limit', '1000',
	'--format', 'text',

	Wraps httpx and provides typed methods for all 9 MCP tools.
	Wraps httpx and provides typed methods for all 8 MCP tools.

docs: Add comprehensive MCP agent integration guide and examples #1509

Are you sure you want to change the base?

docs: Add comprehensive MCP agent integration guide and examples #1509

Uh oh!

Conversation

datradito commented Dec 29, 2025

Overview

What's Added

📚 Documentation

🤖 Agent Examples

🧪 Testing Suite

🔧 Python Dependencies

Key Features

✅ Critical Fix: Interactive UI Blocking

Uh oh!

CLAassistant commented Dec 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Dec 29, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 29, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 29, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 29, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Copilot AI Dec 29, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 29, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 29, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 29, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 29, 2025

Choose a reason for hiding this comment

Uh oh!

gustavo-grieco left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

CLAassistant commented Dec 29, 2025 •

edited

Loading