Add Virtual MCP server support

examples/mcp-servers/README.md

@@ -161,6 +161,91 @@ You should see a successful initialization response from MCP Optimizer.
 
 For client configuration (Cursor, VSCode, Claude Desktop), see [Connecting Clients](../../docs/kubernetes-integration.md#connecting-clients).
 
+## Virtual MCP (vMCP) Setup
+
+Virtual MCP allows you to aggregate multiple MCP servers behind a single endpoint. This is useful when you want to group related servers together and expose them as a unified service.
+
+### Using the vMCP Configuration
+
+The `vmcp-github-fetch.yaml` file provides a complete vMCP setup that aggregates GitHub and Fetch servers:
+
+```bash
+# First, ensure the shared service account exists
+kubectl apply -f examples/mcp-servers/shared-serviceaccount.yaml
+
+# Create GitHub secrets (required for GitHub server)
+export GITHUB_TOKEN=your_token_here
+./examples/mcp-servers/create-github-secrets.sh
+
+# Apply the complete vMCP configuration
+kubectl apply -f examples/mcp-servers/vmcp-github-fetch.yaml
+
+# Verify the deployment
+kubectl get mcpgroup github-fetch-group -n toolhive-system
+kubectl get mcpserver -n toolhive-system
+kubectl get virtualmcpserver github-fetch-vmcp -n toolhive-system
+```
+
+The vMCP configuration includes:
+- **MCPGroup** (`github-fetch-group`): Logical container grouping the backend servers
+- **MCPServer** (GitHub and Fetch): Backend MCP servers
+- **VirtualMCPServer** (`github-fetch-vmcp`): Virtual server that aggregates the backends
+
+### Accessing the vMCP Service
+
+Once deployed, you can access the vMCP through its service:
+
+```bash
+# Port forward the vMCP service
+kubectl port-forward -n toolhive-system svc/github-fetch-vmcp 8080:8080
+
+# Test the connection
+curl -s http://localhost:8080/mcp \
+  -X POST \
+  -H "Content-Type: application/json" \
+  -d '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2024-11-05","capabilities":{},"clientInfo":{"name":"test","version":"1.0"}}}'
+```
+
+For production deployments, configure an Ingress or Gateway API for TLS termination and controlled access.
+
+### Using MCP Optimizer with vMCP
+
+To deploy MCP Optimizer that discovers and aggregates tools from the vMCP group:
+
+```bash
+# First, ensure the vMCP setup is deployed
+kubectl apply -f examples/mcp-servers/vmcp-github-fetch.yaml
+
+# Deploy MCP Optimizer configured for the github-fetch-group
+kubectl apply -f examples/mcp-servers/mcpserver_mcp-optimizer-vmcp.yaml
+
+# Verify deployment
+kubectl get mcpserver mcp-optimizer -n toolhive-system
+kubectl get pods -n toolhive-system | grep mcp-optimizer
+
+# Check logs to verify tool discovery from the group
+kubectl logs -n toolhive-system -l app.kubernetes.io/name=mcp-optimizer --tail=50
+```
+
+The `mcpserver_mcp-optimizer-vmcp.yaml` configuration includes:
+- **Enhanced RBAC**: Permissions to read `mcpservers`, `virtualmcpservers`, and `mcpgroups`
+- **Group Filtering**: `ALLOWED_GROUPS` set to `github-fetch-group` to discover only servers in that group
+- **ServiceAccount**: Dedicated service account with imagePullSecrets for ghcr.io
+
+Access MCP Optimizer:
+
+```bash
+# Port forward the MCP Optimizer service
+kubectl port-forward -n toolhive-system svc/mcp-optimizer-proxy 9900:9900
+
+# Test the connection
+curl -s http://localhost:9900/mcp \
+  -X POST \
+  -H "Content-Type: application/json" \
+  -H "Accept: application/json, text/event-stream" \
+  -d '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2024-11-05","capabilities":{},"clientInfo":{"name":"test","version":"1.0"}}}'
+```
+
 ## Files
 
 - **`create-github-secrets.sh`** - Convenience script to create both GitHub secrets from GITHUB_TOKEN environment variable
@@ -169,6 +254,8 @@ For client configuration (Cursor, VSCode, Claude Desktop), see [Connecting Clien
 - **`mcpserver_github.yaml`** - GitHub API integration server (uses shared ServiceAccount)
 - **`mcpserver_toolhive-doc-mcp.yaml`** - ToolHive documentation search and retrieval server (uses shared ServiceAccount, shares github-token secret)
 - **`mcpserver_mcp-optimizer.yaml`** - MCP Optimizer server that aggregates tools from all MCP servers (includes its own ServiceAccount with imagePullSecrets and RBAC)
+- **`mcpserver_mcp-optimizer-vmcp.yaml`** - MCP Optimizer configured for vMCP setup with enhanced RBAC to read virtualmcpservers and mcpgroups, filters by github-fetch-group
+- **`vmcp-github-fetch.yaml`** - Complete vMCP configuration with MCPGroup, GitHub server, Fetch server, and VirtualMCPServer that aggregates them
 
 ## Complete Documentation
 

examples/mcp-servers/mcpserver_mcp-optimizer-vmcp.yaml

@@ -0,0 +1,109 @@
+# MCP Optimizer configured to work with Virtual MCP Server setup
+# This configuration discovers and aggregates tools from servers in the github-fetch-group
+#
+# Prerequisites:
+# 1. Deploy the vMCP setup first: kubectl apply -f vmcp-github-fetch.yaml
+# 2. Ensure shared-serviceaccount.yaml is deployed
+# 3. GitHub secrets created (see create-github-secrets.sh)
+#
+# Usage:
+#   kubectl apply -f examples/mcp-servers/mcpserver_mcp-optimizer-vmcp.yaml
+#   kubectl port-forward -n toolhive-system svc/mcp-optimizer-proxy 9900:9900
+
+---
+apiVersion: v1
+kind: ServiceAccount
+metadata:
+  name: mcp-optimizer
+  namespace: toolhive-system
+imagePullSecrets:
+  - name: ghcr-pull-secret
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: ClusterRole
+metadata:
+  name: mcp-optimizer-reader
+rules:
+  # Allow reading ToolHive CRDs to discover MCP servers, groups, and virtual servers
+  - apiGroups: ["toolhive.stacklok.dev"]
+    resources: ["mcpservers", "virtualmcpservers", "mcpgroups"]
+    verbs: ["get", "list", "watch"]
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: ClusterRoleBinding
+metadata:
+  name: mcp-optimizer-reader
+roleRef:
+  apiGroup: rbac.authorization.k8s.io
+  kind: ClusterRole
+  name: mcp-optimizer-reader
+subjects:
+  - kind: ServiceAccount
+    name: mcp-optimizer
+    namespace: toolhive-system
+---
+apiVersion: toolhive.stacklok.dev/v1alpha1
+kind: MCPServer
+metadata:
+  name: mcp-optimizer
+  namespace: toolhive-system
+spec:
+  image: mcp-optimizer:latest
+  transport: streamable-http
+  proxyMode: streamable-http
+  port: 9900
+  targetPort: 9900
+  serviceAccount: mcp-optimizer
+  permissionProfile:
+    name: network
+    type: builtin
+  podTemplateSpec:
+    spec:
+      securityContext:
+        fsGroup: 1000
+      volumes:
+        - name: data
+          emptyDir: {}
+        - name: tmp
+          emptyDir: {}
+      containers:
+        - name: mcp
+          imagePullPolicy: IfNotPresent
+          volumeMounts:
+            - name: data
+              mountPath: /data
+            - name: tmp
+              mountPath: /tmp
+          env:
+            - name: SQLITE_TMPDIR
+              value: "/tmp"
+            - name: RUNTIME_MODE
+              value: "k8s"
+            - name: K8S_ALL_NAMESPACES
+              value: "true"
+            - name: LOG_LEVEL
+              value: "INFO"
+            - name: WORKLOAD_POLLING_INTERVAL
+              value: "60"
+            - name: REGISTRY_POLLING_INTERVAL
+              value: "300"
+            - name: MAX_TOOLS_TO_RETURN
+              value: "8"
+            - name: MAX_SERVERS_TO_RETURN
+              value: "5"
+            - name: HYBRID_SEARCH_SEMANTIC_RATIO
+              value: "0.5"
+            - name: ASYNC_DB_URL
+              value: "sqlite+aiosqlite:///data/mcp_optimizer.db"
+            - name: DB_URL
+              value: "sqlite:///data/mcp_optimizer.db"
+            # Configure to discover servers in the github-fetch-group
+            - name: ALLOWED_GROUPS
+              value: "github-fetch-group"
+  resources:
+    limits:
+      cpu: 1000m
+      memory: 2Gi
+    requests:
+      cpu: 250m
+      memory: 256Mi

examples/mcp-servers/vmcp-github-fetch.yaml

@@ -0,0 +1,79 @@
+# Complete vMCP setup with GitHub and Fetch MCP servers
+# This configuration creates a Virtual MCP Server that aggregates
+# GitHub and Fetch backend servers into a single endpoint
+
+---
+apiVersion: toolhive.stacklok.dev/v1alpha1
+kind: MCPGroup
+metadata:
+  name: github-fetch-group
+  namespace: toolhive-system
+spec:
+  description: "Virtual MCP group with GitHub and Fetch servers"
+
+---
+apiVersion: toolhive.stacklok.dev/v1alpha1
+kind: MCPServer
+metadata:
+  name: github
+  namespace: toolhive-system
+spec:
+  groupRef: github-fetch-group
+  image: ghcr.io/github/github-mcp-server:v0.18.0
+  transport: stdio
+  proxyMode: streamable-http
+  port: 8080
+  targetPort: 8080
+  serviceAccount: mcp-shared-sa
+  permissionProfile:
+    name: network
+    type: builtin
+  secrets:
+    - name: github-token
+      key: token
+      targetEnvName: GITHUB_PERSONAL_ACCESS_TOKEN
+  resources:
+    limits:
+      cpu: 200m
+      memory: 256Mi
+    requests:
+      cpu: 100m
+      memory: 128Mi
+
+---
+apiVersion: toolhive.stacklok.dev/v1alpha1
+kind: MCPServer
+metadata:
+  name: fetch
+  namespace: toolhive-system
+spec:
+  groupRef: github-fetch-group
+  image: ghcr.io/stackloklabs/gofetch/server
+  transport: streamable-http
+  proxyMode: streamable-http
+  port: 8080
+  targetPort: 8080
+  serviceAccount: mcp-shared-sa
+  permissionProfile:
+    name: network
+    type: builtin
+  resources:
+    limits:
+      cpu: 100m
+      memory: 128Mi
+    requests:
+      cpu: 50m
+      memory: 64Mi
+
+---
+apiVersion: toolhive.stacklok.dev/v1alpha1
+kind: VirtualMCPServer
+metadata:
+  name: github-fetch-vmcp
+  namespace: toolhive-system
+spec:
+  config:
+    groupRef: github-fetch-group
+  serviceType: ClusterIP
+  incomingAuth:
+    type: anonymous

migrations/versions/2025_08_18_0743-d2977d4c8c53_create_initial_tables.py

@@ -68,6 +68,7 @@ def upgrade() -> None:
             description TEXT,
             server_embedding BLOB,
             "group" TEXT NOT NULL DEFAULT 'default',
+            virtual_mcp INTEGER NOT NULL DEFAULT 0,
             last_updated TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
             created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
             FOREIGN KEY (registry_server_id)