Skip to content

Conversation

Copy link
Contributor

Copilot AI commented Jan 19, 2026

Created test suite to validate Serena MCP server behavior through the MCP gateway, revealing fundamental protocol differences between stdio and HTTP transports.

Changes

  • New test script test/serena-mcp-tests/test_serena_via_gateway.sh

    • Launches gateway container with Serena backend
    • Implements proper MCP handshake (initialize + notifications/initialized)
    • Parses SSE responses from gateway
    • 23 test cases covering initialization, tools listing, and tool execution
  • Makefile targets

    • make test-serena-gateway - Run gateway tests
    • Updated make test-serena description to clarify direct connection
  • Documentation

    • GATEWAY_TEST_FINDINGS.md - Behavioral analysis
    • IMPLEMENTATION_SUMMARY.md - Implementation details
    • Updated README.md with usage instructions

Key Finding

Stdio-based MCP servers like Serena require persistent connection state:

# Direct stdio: Single persistent stream (68/68 tests pass)
echo '{"method":"initialize"...}
{"method":"notifications/initialized"}
{"method":"tools/call"...}' | docker run -i serena

# HTTP gateway: Independent stateless requests (7/23 tests pass)
curl -X POST /mcp/serena -d '{"method":"initialize"...}'        # ✓ Works
curl -X POST /mcp/serena -d '{"method":"notifications/..."}'    # ✓ Works  
curl -X POST /mcp/serena -d '{"method":"tools/call"...}'        # ✗ "invalid during session initialization"

Each HTTP request creates a new session. Serena rejects tool calls outside the initialization stream. This is expected given current gateway architecture - tests document the limitation and provide regression baseline for future session persistence features.

Test Results

Working: Container startup, initialization handshake, error handling
Limited: Tool execution requires connection-level session state not maintained across HTTP requests

Both test suites can run independently for comparison.


✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

Copilot AI and others added 6 commits January 19, 2026 19:04
Co-authored-by: lpcox <15877973+lpcox@users.noreply.github.com>
Co-authored-by: lpcox <15877973+lpcox@users.noreply.github.com>
Co-authored-by: lpcox <15877973+lpcox@users.noreply.github.com>
Co-authored-by: lpcox <15877973+lpcox@users.noreply.github.com>
Co-authored-by: lpcox <15877973+lpcox@users.noreply.github.com>
Co-authored-by: lpcox <15877973+lpcox@users.noreply.github.com>
Copilot AI changed the title [WIP] Add tests for Serena through MCP gateway Add Serena gateway tests and document stdio/HTTP behavioral differences Jan 19, 2026
Copilot AI requested a review from lpcox January 19, 2026 19:17
@lpcox lpcox marked this pull request as ready for review January 19, 2026 19:28
@lpcox lpcox merged commit bd43e98 into main Jan 19, 2026
3 checks passed
@lpcox lpcox deleted the copilot/add-serena-tests-with-gateway branch January 19, 2026 19:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants