Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
64 changes: 27 additions & 37 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,11 @@
# ChatBotKit Terraform Provider

[![Terraform Registry](https://img.shields.io/badge/Terraform-Registry-purple.svg)](https://registry.terraform.io/providers/chatbotkit/chatbotkit/latest)

This directory contains the ChatBotKit Terraform Provider, which allows you to manage ChatBotKit resources using Terraform.

**Terraform Registry:** https://registry.terraform.io/providers/chatbotkit/chatbotkit/latest

## Building

```bash
Expand Down Expand Up @@ -52,43 +56,29 @@ CHATBOTKIT_API_KEY=your-api-key go test -v ./internal/provider/ -run "^TestAcc"
## Directory Structure

```
sdks/terraform/
├── main.go # Provider entry point
├── go.mod # Go module definition
├── go.sum # Go dependencies
├── .goreleaser.yml # Release configuration
├── terraform-registry-manifest.json # Registry manifest
├── docs/ # Terraform Registry documentation
│ ├── index.md # Provider documentation
│ ├── resources/ # Resource documentation
│ └── data-sources/ # Data source documentation
terraform-provider-chatbotkit/
├── main.go # Provider entry point
├── go.mod # Go module definition
├── go.sum # Go dependencies
├── .goreleaser.yml # Release configuration
├── terraform-registry-manifest.json # Registry manifest
├── docs/ # Terraform Registry documentation
│ ├── index.md # Provider documentation
│ ├── resources/ # Resource documentation
│ └── data-sources/ # Data source documentation
├── types/
│ └── types.go # Generated Go types
│ └── types.go # Generated Go types
├── internal/
│ └── provider/
│ ├── client.go # GraphQL API client
│ ├── client_test.go # Client unit tests
│ ├── provider.go # Provider configuration
│ ├── provider_test.go # Provider tests
│ ├── resource_*.go # Resource implementations
│ └── resource_*_test.go # Resource tests
│ ├── client.go # GraphQL API client
│ ├── client_test.go # Client unit tests
│ ├── provider.go # Provider configuration
│ ├── provider_test.go # Provider tests
│ ├── resource_*.go # Resource implementations
│ └── resource_*_test.go # Resource tests
└── examples/
└── basic/
└── main.tf # Example Terraform configuration
```

## Regenerating Code

The provider code is generated from the GraphQL schema:

```bash
cd sites/main

# Generate Go types
pnpm script:sync-api-spec-to-terraform

# Generate provider resources and client
pnpm script:gen-terraform-stubs
└── main.tf # Example Terraform configuration
```

## Resources
Expand Down Expand Up @@ -122,12 +112,12 @@ The provider supports the following resources:

The provider supports the following data sources for reading existing resources:

| Data Source | Description |
| -------------------- | ----------------------------------- |
| `chatbotkit_bot` | Read information about an existing bot |
| `chatbotkit_dataset` | Read information about an existing dataset |
| Data Source | Description |
| ---------------------- | -------------------------------------------- |
| `chatbotkit_bot` | Read information about an existing bot |
| `chatbotkit_dataset` | Read information about an existing dataset |
| `chatbotkit_blueprint` | Read information about an existing blueprint |
| `chatbotkit_skillset` | Read information about an existing skillset |
| `chatbotkit_skillset` | Read information about an existing skillset |

## Example Usage

Expand Down
2 changes: 1 addition & 1 deletion VERSION
Original file line number Diff line number Diff line change
@@ -1 +1 @@
1.0.0
1.0.1
37 changes: 36 additions & 1 deletion examples/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,9 @@ These examples demonstrate production-ready architectures based on ChatBotKit bl
|---------|-------------|--------------|
| [skillset-based-dynamic-skill](./skillset-based-dynamic-skill/) | Agent that dynamically discovers and loads skills | Dynamic skill loading, Modular architecture, List & Install abilities |
| [ai-employee](./ai-employee/) | Comprehensive AI Employee for professional environments | Workspace/Space, Shell execution, Gmail integration, Notion integration, Dynamic skillsets |
| [matillion-operations](./matillion-operations/) | AI-powered operations center for Matillion Data Productivity Cloud | 4 specialized skillsets, Pipeline management, Schedule management, Monitoring, Scheduled health checks |
| [simple-self-improving-agent](./simple-self-improving-agent/) | Self-improving agent that learns from interactions | Backstory file management, Read/Write abilities, Continuous learning |
| [mcp-factory](./mcp-factory/) | Factory-style architecture with multiple MCP server integrations | 4 service skillsets, Modular MCP servers, Clear separation of concerns |
| [dynamic-mcp-search-and-install](./dynamic-mcp-search-and-install/) | Agent that dynamically discovers and installs MCP servers | MCP registry search, On-demand installation, Self-extending capabilities |

## Getting Started

Expand Down Expand Up @@ -112,6 +114,39 @@ Demonstrates a sophisticated AI employee with workspace and multiple integration

**Use when:** Building autonomous digital team members with access to tools and services.

### Simple Self-improving Agent Example
Demonstrates a self-improving AI agent that learns from interactions.

**What you'll learn:**
- File resource management for backstory storage
- Read and write abilities for file manipulation
- Self-improvement patterns
- Continuous learning architectures

**Use when:** Building agents that need to adapt and improve based on real-world usage.

### MCP Factory Example
Factory-style architecture with multiple independent MCP server integrations.

**What you'll learn:**
- Modular MCP server organization
- Multiple skillset patterns
- Service isolation and boundaries
- MCP server integration setup

**Use when:** Providing a suite of distinct AI functionalities to multiple clients or users.

### Dynamic MCP Search and Install Example
Agent that discovers and installs MCP servers on-demand from a registry.

**What you'll learn:**
- MCP registry search capabilities
- Dynamic MCP installation patterns
- Self-extending agent architectures
- On-demand capability loading

**Use when:** Building adaptable agents that discover and integrate tools as needed.

### Matillion Operations Example
AI-powered operations center for data pipeline management.

Expand Down
208 changes: 208 additions & 0 deletions examples/dynamic-mcp-search-and-install/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,208 @@
# Dynamic MCP Search and Install Architecture

This example demonstrates a reference architecture for an AI agent that can search for and install Model Context Protocol (MCP) servers dynamically.

## Overview

This blueprint demonstrates a dynamic architecture for an MCP server that allows the AI agent to search for and install Model Context Protocol (MCP) servers from a known registry. By combining search and installation capabilities, this blueprint enables agents to discover and integrate new MCP servers on-demand, creating a self-extending system that grows its functionality as needed.

## Architecture

```
┌──────────────────────────────────────────────────────────┐
│ Dynamic MCP Agent │
│ │
│ ┌─────────────────────────────────────────────────────┐ │
│ │ MCP Discovery Skillset │ │
│ │ ┌──────────────────┐ ┌──────────────────┐ │ │
│ │ │ Search MCP │ │ Install MCP │ │ │
│ │ │ Servers │ │ (by URL) │ │ │
│ │ └──────────────────┘ └──────────────────┘ │ │
│ └─────────────────────────────────────────────────────┘ │
│ │
│ ┌───────────────┐ │
│ │ MCP Registry │ │
│ └───────┬───────┘ │
│ ┌──────────────┼──────────────┬──────────────┐ │
│ │ │ │ │ │
│ ┌─────────┐ ┌─────────┐ ┌─────────┐ ┌─────────┐ │
│ │ File │ │ GitHub │ │ Database│ │ Slack │ │
│ │ System │ │ MCP │ │ MCP │ │ MCP │ │
│ │ MCP │ │ │ │ │ │ │ │
│ └─────────┘ └─────────┘ └─────────┘ └─────────┘ │
│ │
│ (Installed on-demand as needed) │
└──────────────────────────────────────────────────────────┘
```

## Key Features

1. **Dynamic Discovery**: Search for MCP servers by name or description in a registry
2. **On-Demand Installation**: Install MCP servers only when needed for specific tasks
3. **Self-Extending**: Agent gains new capabilities without manual reconfiguration
4. **Flexible Integration**: Connect to any MCP server available in the registry

## Core Abilities

### Search MCP Servers
Uses the `registry/server/search` template to find available MCP servers. The agent can:
- Search by server name
- Search by description or keywords
- Discover servers that provide needed functionality
- Review server metadata before installation

### Install MCP
Uses the `conversation/mcp/install[url]` template to activate an MCP server by its URL. Once installed:
- The server's tools become available to the agent
- Functions can be called directly in the conversation
- Capabilities persist for the duration of the conversation

## How It Works

### Discovery Flow

1. **User Request**: User asks for functionality the agent doesn't have
2. **Search**: Agent searches the MCP registry for relevant servers
3. **Selection**: Agent identifies the most appropriate MCP server
4. **Installation**: Agent installs the MCP server using its URL
5. **Utilization**: Agent uses the newly available tools to complete the task

### Example Interaction

**User**: "Can you help me analyze the files in my project directory?"

**Agent**:
1. Searches for MCP servers related to file system access
2. Finds a "filesystem" MCP server in the registry
3. Installs it using the server's URL
4. Uses the filesystem tools to list and analyze files
5. Provides the requested analysis to the user

## Usage

1. Set your ChatBotKit API key:
```bash
export CHATBOTKIT_API_KEY="your-api-key"
```

2. Initialize Terraform:
```bash
terraform init
```

3. Review the planned changes:
```bash
terraform plan
```

4. Apply the configuration:
```bash
terraform apply
```

5. Test the agent by:
- Asking it to search for specific MCP servers
- Requesting functionality that requires MCP installation
- Observing how it discovers and integrates new capabilities

## Testing the Agent

### Search for MCP Servers
```
User: "What MCP servers are available for working with GitHub?"
Agent: [Searches registry and lists GitHub-related MCP servers]
```

### Install and Use an MCP Server
```
User: "I need to check my GitHub repositories"
Agent: [Searches for GitHub MCP, installs it, then uses its tools to list repositories]
```

## MCP Registry

The agent searches a registry of available MCP servers. Common MCP servers include:

- **Filesystem**: Access and manipulate local files
- **GitHub**: Interact with GitHub repositories and issues
- **PostgreSQL**: Query and manage PostgreSQL databases
- **Slack**: Send messages and interact with Slack
- **Google Drive**: Access and manage Google Drive files
- **Memory**: Persistent key-value storage
- **Fetch**: Make HTTP requests to external APIs

## Customization

### Adding Search Filters

Enhance the search ability with filters:

```hcl
resource "chatbotkit_skillset_ability" "search_mcp_advanced" {
skillset_id = chatbotkit_skillset.mcp_discovery.id
name = "Advanced MCP Search"
description = "Search MCP servers with filters"
instruction = <<-EOT
template: registry/server/search
parameters:
query: $[query! ys|search terms]
category: $[category ys|server category (e.g., database, filesystem)]
EOT
}
```

### Installing Multiple MCP Servers

The agent can install multiple MCP servers in a single conversation:

```hcl
# The agent can call the install ability multiple times
# Each installation adds to the available tools
```

## When to Use This Pattern

This pattern is ideal when:
- Agent needs diverse, specialized capabilities that vary by use case
- You want to avoid pre-installing all possible tools
- Requirements evolve and new MCP servers become available
- Users need access to different sets of integrations
- You want a self-service approach to capability extension

## Benefits

1. **Reduced Initial Complexity**: Start simple, add capabilities as needed
2. **Efficient Resource Usage**: Only load tools when required
3. **Future-Proof**: Automatically gain access to new MCP servers as they're added to the registry
4. **User-Driven**: Agent adapts to specific user needs dynamically
5. **Maintainable**: No need to reconfigure for new integrations

## Important Considerations

1. **Registry Quality**: Ensure the MCP registry contains trustworthy servers
2. **Installation Limits**: Consider rate limits or caps on installations per conversation
3. **Security**: Validate MCP servers before installation to prevent malicious code
4. **Performance**: Installing multiple large MCP servers may impact response times

## Comparison with Static MCP Configuration

| Aspect | Dynamic (This Pattern) | Static Configuration |
|--------|----------------------|---------------------|
| Setup Complexity | Low (just search/install abilities) | High (configure each MCP server) |
| Runtime Flexibility | High (discover new servers) | Low (fixed set of servers) |
| Resource Efficiency | High (load on-demand) | Low (all loaded upfront) |
| User Experience | Adaptive (fits needs) | Consistent (same tools always) |

## Cleanup

To destroy all created resources:
```bash
terraform destroy
```

## Learn More

- [Model Context Protocol Specification](https://modelcontextprotocol.io)
- [ChatBotKit MCP Documentation](https://chatbotkit.com/docs/integrations/mcp)
- [ChatBotKit Registry Documentation](https://chatbotkit.com/docs/registry)
- [Blueprint Reference Architecture Examples](https://chatbotkit.com/blueprints)
Loading
Loading