KB Map Generator - doc-map-builder

A high-performance knowledge-base map generator CLI (rust language) tool that scans a structured directory of documents and compiles a comprehensive, sorted index map (map.json).

This tool is designed to generate machine-readable sitemaps for multi-agent systems, AI assistants (such as Windsurf, Cursor, or Claude Code), and static site generators. By maintaining a single map.json index, AI agents can immediately understand your entire knowledge base structure, file descriptions, and paths without wasting token context scanning the filesystem.

Features

• 🚀 High Performance: Written in Rust for blazing-fast recursive directory scanning, parsing, and JSON serialization.
• 📂 Flexible Schema: Supports nested directories, tracking files and categories recursively.
• 📝 Intelligent Metadata Extraction:
  • YAML Frontmatter: Parses Markdown files (.md) starting with --- for explicit name and description tags.
  • Smart Fallback for Markdown: If frontmatter is missing, it auto-generates a kebab-case name from the filename and extracts the first non-empty paragraph or heading as the description.
  • Programming Language Source Code Files Integration: Parses .zig, .ts, .py, .c, .cpp, .h, .hpp source files, automatically using the first line-level comment (//, /**, /*, ///, #!, //!) as the description.
• 🔀 Deterministic Sorting: Alphabetically sorts all paths and files recursively to ensure git diffs are clean and predictable.

System Requirements

• Rust / Cargo: Rust Toolchain (v1.75+ or newer recommended)

Getting Started

1. Structure Your Knowledge Base

The generator expects a directory named docs/ in the working directory where it is run:

my-knowledge-base/

• docs/
  • index.md # Markdown with/without frontmatter
  • getting-started.md
  • advanced-topics/
    • configuration.md
    • example.rust # Code samples with comments

2. Add Metadata to Your Files (Optional but Recommended)

Markdown (.md) Frontmatter

Add a YAML frontmatter block at the top of your markdown files:

---
name: getting-started
description: A beginner's guide to setting up and configuring the environment.
---

# Getting Started

This is the body of the markdown document...

If no frontmatter is found, the tool will kebab-case the filename (e.g. getting-started) and use the first paragraph/heading as the description.

Programming Source Code (like .zig, .ts, .py, .c, .cpp, .h, .hpp)

Add a documentation comment at the very top of your programming source code files:

//! An example demonstrating standard library HashMap usage.
const std = @import("std");
...

3. Build & Run the Generator

Clone this scripts directory or place it in your repository:

# Navigate to the scripts directory
cd scripts/

# Build the release binary
cargo build --release

To run the tool and generate the map, execute the compiled binary from your workspace root (where the docs/ folder is located):

# From workspace root
./scripts/target/release/md-map

This will read your ./docs/ folder and output a ./map.json file in the same workspace root directory:

Updated map.json with 3 top-level entries.

Output Schema (map.json)

The generated map.json is a deterministic tree representing your documentation structure.

Schema Specification

• Root Level: A sorted JSON array of entries.
• File Entry: Contains the file's metadata and relative path.

{
  "name": "getting-started",
  "description": "A beginner's guide to setting up and configuring the environment.",
  "path": "getting-started.md"
}

• Directory / Category Entry: Contains a title, a trailing slash relative path, and a children array.

{
  "description": "Advanced Topics",
  "path": "advanced-topics/",
  "children": [
    {
      "name": "configuration",
      "description": "Deep-dive into advanced configuration parameters.",
      "path": "advanced-topics/configuration.md"
    }
  ]
}

Generalizing for Any Knowledge Base

This tool is entirely context-agnostic. Any workspace containing structured files under a docs/ directory can immediately use it to index its contents.

Adding New File Parsers

To support other language specifications or custom metadata extraction (e.g. parsing Python docstrings, JSDoc comment blocks, or JSON config files), modify file_metadata in src/main.rs:

// In src/main.rs
fn file_metadata(full_path: &Path, basename: &str) -> (String, String) {
    ...
    // Add custom handlers here for other extensions like .py, .ts, etc.
}

Integrating with AI Agents

To optimize interactions with AI code assistants (like Windsurf, Cursor, or Claude Code), add the index map directly to your agent system instructions (.agents or workspace README.md):

### AI Prompting Guideline

To understand the structure of the documentation and knowledge base, read `./map.json` at the start of your workflow.

When creating or updating files, always run:
`./scripts/target/release/md-map` (or the equivalent map update command) to ensure map.json remains synchronized.