vMCP: Implement backend session persistence with client pooling

[jhrozek]

Problem

When a client makes multiple tool calls through vMCP within the same MCP session, each call to a backend MCP server creates a new backend client and new backend session. This breaks stateful backends that rely on session persistence (e.g., Playwright browser contexts, database transactions, conversation state).

Current Behavior (Bug)

Client Session (single)
    │
    ├─► Tool Call 1 ──► vMCP ──► NEW Backend Client ──► Backend (session A)
    │                              └─► Initialize + CallTool + Close
    │
    ├─► Tool Call 2 ──► vMCP ──► NEW Backend Client ──► Backend (session B)
    │                              └─► Initialize + CallTool + Close
    │
    └─► Tool Call 3 ──► vMCP ──► NEW Backend Client ──► Backend (session C)
                                   └─► Initialize + CallTool + Close

The backend sees 3 separate sessions (A, B, C) instead of one persistent session.

Expected Behavior

Client Session (single)
    │
    ├─► Tool Call 1 ──► vMCP ──► Cached Backend Client ──► Backend (session X)
    ├─► Tool Call 2 ──► vMCP ──► Cached Backend Client ──► Backend (session X)
    └─► Tool Call 3 ──► vMCP ──► Cached Backend Client ──► Backend (session X)

Solution

Implement a Session-Scoped Client Pool that caches initialized MCP clients per (vmcpSessionID, backendID) tuple.

Architecture

Each VMCPSession owns its own pool. Users never share connections:

VMCPSession "user-A"                VMCPSession "user-B"
    └─► pool["backend1"] = clientA      └─► pool["backend1"] = clientB  (DIFFERENT)

Key Interface

type BackendClientPool interface {
    // GetOrCreate returns cached client for backendID, or creates new one.
    GetOrCreate(ctx context.Context, target *vmcp.BackendTarget) (*client.Client, error)

    // MarkUnhealthy marks a client for replacement on next GetOrCreate.
    // Called when connection errors occur (reset, EOF, timeout).
    MarkUnhealthy(backendID string)

    // Close shuts down all pooled clients. Called on session termination.
    Close() error
}

Implementation Steps

1. Create BackendClientPool interface and implementation - pkg/vmcp/client/pool.go
2. Add session ID context propagation - pkg/vmcp/discovery/middleware.go
3. Extend VMCPSession with pool - pkg/vmcp/session/vmcp_session.go
4. Initialize pool on session registration - pkg/vmcp/server/server.go
5. Cleanup pool on session termination - pkg/vmcp/server/session_adapter.go
6. Modify httpBackendClient to use pool - pkg/vmcp/client/client.go
7. Add connection error detection - for MarkUnhealthy functionality

Files to Modify/Create

File	Action
pkg/vmcp/client/pool.go	CREATE - Pool interface + implementation
pkg/vmcp/client/pool_test.go	CREATE - Pool unit tests
pkg/vmcp/session/vmcp_session.go	MODIFY - Add pool field, Close()
pkg/vmcp/client/client.go	MODIFY - Use pool, remove defer Close()
pkg/vmcp/discovery/middleware.go	MODIFY - Propagate session ID
pkg/vmcp/server/server.go	MODIFY - Initialize pool in hook
pkg/vmcp/server/session_adapter.go	MODIFY - Cleanup pool in Terminate()

Error Handling

Error Type	Action
Connection refused/reset	MarkUnhealthy, return error
Timeout	Return error (keep client)
MCP tool error	Return error (keep client)
Pool closed	Return error

Testing

Unit tests:

• GetOrCreate creates client on first access
• GetOrCreate returns same client for same backend
• Concurrent access creates only one client (double-checked locking)
• MarkUnhealthy causes recreation
• Close() closes all clients

Integration tests:

• Multiple tool calls use same backend session
• Session termination cleans up pool
• Error recovery works

[jerm-dro]

@yrobla and I identified some technical debt around session management in vMCP that makes this fix complex. We're working on an RFC (stacklok/toolhive-rfcs#38) to pay down this debt. In the meantime, we'll not be working on this issue. If someone is blocked, we can prioritize a quick fix which adds to the debt.