docs: complete the documentation revamp — consolidated tree, schemas + CI gates, node migration, new site

.claude/CLAUDE.md

@@ -1,81 +1 @@
-# RocketRide Server
-
-## Overview
-
-Monorepo for the RocketRide (Aparavi) data toolchain. Contains a C++ engine, Python SDK/tools, TypeScript VS Code extension, and supporting packages.
-
-## Stack
-
-- **Engine**: C++ core (`packages/server/`), built via `./builder`
-- **Python**: SDK (`packages/client-python/`), tools, tests. Python 3.10+, ruff for lint/format, pytest
-- **TypeScript**: VS Code extension (`apps/vscode/`), Node.js, tabs, single quotes, semicolons (see rules)
-- **Build system**: `./builder` CLI wraps platform-specific builds and test orchestration
-
-## Common Commands
-
-```bash
-# Build
-./builder build                  # Full build
-./builder build server           # Server/engine core only
-./builder build vscode           # VS Code extension only
-
-# Test
-./builder test                   # All tests
-python -m pytest <path>          # Python tests directly
-npx tsc --noEmit                 # TypeScript type-check
-
-# Lint & Format
-python -m ruff check <path>      # Python lint
-python -m ruff format <path>     # Python format
-npx prettier --check <path>      # TS/JS format check
-```
-
-## API Documentation
-
-The co-located per-package docs (assembled into the docs site by `builder docs:build`) are the **source of truth** (see the co-located documentation rule below). The agent integration docs at `docs/agents/` are the assistant-facing integration reference within it: **read them first before writing any RocketRide integration code.**
-
-1. `ROCKETRIDE_README.md`: the platform map, setup, task router — start here
-2. `ROCKETRIDE_CONCEPTS.md`: workspace, connection, deploy/publish lifecycle
-3. `ROCKETRIDE_PIPELINES.md`: `.pipe` format, lanes, patterns, pitfalls
-4. `ROCKETRIDE_COMPONENT_REFERENCE.md`: pipeline component catalog + schemas
-5. `ROCKETRIDE_APPS.md` + `ROCKETRIDE_UI_COMPONENTS.md`: app building + shell UI
-6. `ROCKETRIDE_python_API.md` or `ROCKETRIDE_typescript_API.md`: client methods
-7. `ROCKETRIDE_INTEGRATIONS.md`: MCP, n8n, webhooks, Telegram, CI
-8. `ROCKETRIDE_OBSERVABILITY.md`: runtime events, logs, monitoring
-
-## Conventions
-
-- **Commits**: conventional commits: `feat(scope):`, `fix(scope):`, `chore(scope):`, etc.
-- **Branches**: `feat/RR-<ticket>-short-description`, `fix/RR-<ticket>-short-description`
-- **PR base branch**: `develop`
-- **Error handling (VS Code extension)**: use `Callout.call()` wrapper, never raw try/catch
-- **Error types (VS Code extension)**: always throw `AppError`, never plain `Error`
-- **Logging (VS Code extension)**: use `logger.*`, never `console.log`
-- These three rules govern the VS Code extension only. The docs-site theme components (`packages/docs`) use plain React with raw try/catch by design; do not apply them there.
-- **Python quotes**: single quotes (enforced by ruff)
-- **TS quotes**: single quotes preferred
-
-## Co-located documentation rule
-
-Documentation is co-located with the code it documents and is gathered into the
-docs site by `builder docs:build`. When a change alters a public contract, update
-the corresponding co-located doc in the SAME change:
-
-- A node's inputs, outputs, or config schema -> prose in `nodes/src/nodes/<name>/doc.md`.
-  Never hand-edit content between `<!-- ROCKETRIDE:GENERATED:PARAMS START -->` and
-  `<!-- ROCKETRIDE:GENERATED:PARAMS END -->`; it is regenerated from `services*.json`
-  by `nodes:docs-generate`.
-- A public TypeScript SDK signature -> `packages/client-typescript/docs/`. Reference
-  under `docs/reference/` is generated by `client-typescript:docs-generate` (do not edit).
-- A public Python SDK signature -> `packages/client-python/docs/` (reference generated).
-- The MCP protocol surface -> `packages/client-mcp/docs/`.
-- The WebSocket (5565) / engine protocol surface -> `packages/server/docs/`.
-- The `.pipe` schema (`packages/client-typescript/src/client/types/pipeline.ts`) -> its
-  reference regenerates to `/pipeline-reference`.
-- The VS Code extension surface -> `apps/vscode/docs/`.
-- Spine/landing pages (Home, Quickstart, Concepts, Cloud, Troubleshooting, Glossary,
-  Cursor/Windsurf stubs) -> `packages/docs/content-static/`.
-
-Prose-only edits and internal refactors that do not change a public contract do not
-require doc updates. Treat the doc as part of the change, not a follow-up. Do not create
-a separate docs repo or a top-level docs site folder. Verify with `builder docs:build`.
+@../AGENTS.md

.coderabbit.yaml

@@ -35,6 +35,24 @@ reviews:
       instructions: |
         C++ code: C++17 standard. This is the core engine — review for
         memory safety, thread safety, and performance.
+    - path: 'nodes/src/nodes/**/README.md'
+      instructions: |
+        Node READMEs follow docs/development/nodes/readme-schema.md (structure is
+        machine-checked by scripts/validate-node-readme.py; focus review on
+        accuracy). Verify the prose matches the node's code and services*.json:
+        the "As a tool" table must list exactly the functions the node exposes
+        to agents, and lane and connection descriptions must match declared
+        behavior. Never accept hand edits inside the ROCKETRIDE:GENERATED:PARAMS
+        region. If a PR changes node behavior, config fields, or tools without
+        updating the README, flag it.
+    - path: 'docs/public/**/*.md'
+      instructions: |
+        Client docs follow docs/development/clients/readme-schema.md. The Python
+        and TypeScript readmes must stay in parity: an API method, type, or
+        exception documented in one must be documented in the other or be
+        explicitly marked language-specific. Package READMEs are generated
+        from these sources via `builder docs:export` — never review edits to
+        packages/client-*/README.md as hand-written files.
   tools:
     eslint:
       enabled: true

.cursorrules

@@ -1,66 +1,9 @@
-# Rocket Ride Data Toolchain - AI Assistant Instructions
+# RocketRide — AI assistant instructions
 
-## Read ALL Documentation First
+The canonical contributor guide is `AGENTS.md` at the repository root. Read it
+first; it routes every task (integration code, nodes, clients, MCP, engine) to
+the right documentation under `docs/`.
 
-**Before ANY Rocket Ride code, read ALL 6 files at `docs/agents/`:**
-
-1. **`ROCKETRIDE_QUICKSTART.md`** ← Start here, copy working examples
-2. `ROCKETRIDE_README.md`
-3. `ROCKETRIDE_python_API.md` or `ROCKETRIDE_typescript_API.md`
-4. `ROCKETRIDE_PIPELINE_RULES.md`
-5. `ROCKETRIDE_COMPONENT_REFERENCE.md`
-6. `ROCKETRIDE_COMMON_MISTAKES.md`
-
-**If user request is ambiguous: STOP and ask. Do NOT guess.**
-
----
-
-## ✓ Before Submitting Code
-
-- [ ] Read ROCKETRIDE_QUICKSTART.md and used as template
-- [ ] Read all 6 documentation files
-- [ ] Verified code follows documented patterns
-- [ ] Quoted relevant documentation in response
-
-**Cannot check all boxes? Don't submit code.**
-
----
-
-## Documentation Guide
-
-| Need | File |
-|------|------|
-| Working examples | ROCKETRIDE_QUICKSTART.md |
-| Setup checklist | ROCKETRIDE_README.md |
-| Client methods | ROCKETRIDE_python_API.md or ROCKETRIDE_typescript_API.md |
-| Pipeline rules | ROCKETRIDE_PIPELINE_RULES.md |
-| Components | ROCKETRIDE_COMPONENT_REFERENCE.md |
-| Troubleshooting | ROCKETRIDE_COMMON_MISTAKES.md |
-
-**The co-located per-package docs — assembled into the docs site by `builder docs:build` — are the source of truth. The agent integration docs at `docs/agents/` are the assistant-facing integration reference within it; read them first before writing any RocketRide integration code.**
-
-
-## Co-located documentation rule
-
-Documentation is co-located with the code it documents and is gathered into the
-docs site by `builder docs:build`. When a change alters a public contract, update
-the corresponding co-located doc in the SAME change:
-
-- A node's inputs, outputs, or config schema -> prose in `nodes/src/nodes/<name>/doc.md`.
-  Never hand-edit content between `<!-- ROCKETRIDE:GENERATED:PARAMS START -->` and
-  `<!-- ROCKETRIDE:GENERATED:PARAMS END -->`; it is regenerated from `services*.json`
-  by `nodes:docs-generate`.
-- A public TypeScript SDK signature -> `packages/client-typescript/docs/`. Reference
-  under `docs/reference/` is generated by `client-typescript:docs-generate` (do not edit).
-- A public Python SDK signature -> `packages/client-python/docs/` (reference generated).
-- The MCP protocol surface -> `packages/client-mcp/docs/`.
-- The WebSocket (5565) / engine protocol surface -> `packages/server/docs/`.
-- The `.pipe` schema (`packages/client-typescript/src/client/types/pipeline.ts`) -> its
-  reference regenerates to `/pipeline-reference`.
-- The VS Code extension surface -> `apps/vscode/docs/`.
-- Spine/landing pages (Home, Quickstart, Concepts, Cloud, Troubleshooting, Glossary,
-  Cursor/Windsurf stubs) -> `packages/docs/content-static/`.
-
-Prose-only edits and internal refactors that do not change a public contract do not
-require doc updates. Treat the doc as part of the change, not a follow-up. Do not create
-a separate docs repo or a top-level docs site folder. Verify with `builder docs:build`.
+Before writing any RocketRide integration code, read
+`docs/agents/context/ROCKETRIDE_README.md` and the other `ROCKETRIDE_*` files in
+`docs/agents/context/`. If a request is ambiguous, stop and ask — do not guess.

.github/SUPPORT.md

@@ -5,8 +5,8 @@ Looking for help with RocketRide? Here's where to go.
 ## Documentation
 
 - **[Documentation site](https://docs.rocketride.org)**: guides, tutorials, API reference
-- **[Build system docs](../docs/develop/README-builder.md)**: building from source
-- **[Client library docs](../docs/develop/README-clients.md)**: TypeScript, Python, MCP SDKs
+- **[Build system docs](../docs/development/builder/reference.md)**: building from source
+- **[Client library docs](https://docs.rocketride.org/develop/clients)**: TypeScript, Python, MCP SDKs
 
 ## Getting Help
 

.github/copilot-instructions.md

@@ -6,25 +6,28 @@ Use RocketRide when building AI pipelines, document processing, RAG systems, or
 
 ## Documentation
 
-Full docs: `.rocketride/docs/`
+Full docs: `docs/agents/context/`
 
 **Read the relevant doc(s) before generating any RocketRide code.**
 
 | File                              | Read when...                                                      |
 | --------------------------------- | ----------------------------------------------------------------- |
-| ROCKETRIDE_README.md              | Starting any RocketRide work: overview + mandatory setup steps   |
-| ROCKETRIDE_QUICKSTART.md          | Writing first pipeline: complete working examples (Python & TS)  |
-| ROCKETRIDE_PIPELINE_RULES.md      | Defining pipelines: structure, lane wiring, config rules         |
+| ROCKETRIDE_README.md              | Starting any RocketRide work: platform map, setup, task router     |
+| ROCKETRIDE_CLI.md                 | The `rocketride` command: validate, start/upload runs, store, deploy, app verbs; when to prefer it over SDK code |
+| ROCKETRIDE_CONCEPTS.md            | Workspace, connection, deploy/publish lifecycle                    |
+| ROCKETRIDE_PIPELINES.md           | Defining pipelines: `.pipe` format, lanes, patterns, pitfalls     |
 | ROCKETRIDE_COMPONENT_REFERENCE.md | Choosing/configuring components: all providers and config fields |
-| ROCKETRIDE_COMMON_MISTAKES.md     | Before finalizing: known pitfalls to avoid                       |
+| ROCKETRIDE_APPS.md                | Building RocketRide apps (App Builder, deploy, publish)           |
+| ROCKETRIDE_UI_COMPONENTS.md       | Shell UI components for apps                                       |
+| ROCKETRIDE_INTEGRATIONS.md        | MCP, n8n, webhooks, Telegram, CI                                   |
 | ROCKETRIDE_python_API.md          | Python SDK: client methods, types, patterns                      |
 | ROCKETRIDE_typescript_API.md      | TypeScript SDK: client methods, types, patterns                  |
 | ROCKETRIDE_OBSERVABILITY.md       | Consuming runtime logs, lifecycle events, and pipeline traces     |
 
 ## Before Writing ANY RocketRide Code
 
-1. Read `.rocketride/docs/ROCKETRIDE_README.md` for mandatory setup requirements
+1. Read `docs/agents/context/ROCKETRIDE_README.md` for mandatory setup requirements
 2. Read the relevant API doc (Python or TypeScript) for your language
-3. Read `.rocketride/docs/ROCKETRIDE_PIPELINE_RULES.md` + `.rocketride/docs/ROCKETRIDE_COMPONENT_REFERENCE.md`
-4. Read `.rocketride/docs/ROCKETRIDE_COMMON_MISTAKES.md` before finalizing
+3. Read `docs/agents/context/ROCKETRIDE_PIPELINES.md` + `docs/agents/context/ROCKETRIDE_COMPONENT_REFERENCE.md`
+4. For apps, read `docs/agents/context/ROCKETRIDE_APPS.md`
 <!-- ROCKETRIDE:END -->

.github/workflows/docs-schemas.yml

@@ -0,0 +1,76 @@
+# ---------------------------------------------------------------------------
+#  Documentation schema checks — the PR gate for doc contracts.
+#
+#  Runs the docs builder gates (docs:test, which carries the schema
+#  validators, plus docs:build and docs:check) on doc-path pull requests into
+#  develop (and direct pushes to it). ci.yml deliberately does not carry
+#  them — the paths filter below keeps doc-only machinery from spending the
+#  shared pipeline's minutes, and non-doc changes from spending these.
+# ---------------------------------------------------------------------------
+name: Doc schemas
+
+on:
+  pull_request:
+    branches: [develop]
+    # GitHub Actions YAML has no anchors — keep this list identical to push's.
+    paths:
+      - 'docs/**'
+      - 'nodes/src/nodes/**/README.md'
+      - 'nodes/src/nodes/**/services*.json'
+      - 'scripts/validate-node-readme.py'
+      - 'scripts/validate-client-docs.py'
+      - 'tests/test_validate_node_readme.py'
+      - 'pnpm-lock.yaml'
+      - '.github/workflows/docs-schemas.yml'
+  push:
+    branches: [develop]
+    paths:
+      - 'docs/**'
+      - 'nodes/src/nodes/**/README.md'
+      - 'nodes/src/nodes/**/services*.json'
+      - 'scripts/validate-node-readme.py'
+      - 'scripts/validate-client-docs.py'
+      - 'tests/test_validate_node_readme.py'
+      - 'pnpm-lock.yaml'
+      - '.github/workflows/docs-schemas.yml'
+
+concurrency:
+  group: docs-schemas-${{ github.head_ref || github.run_id }}
+  cancel-in-progress: true
+
+permissions:
+  contents: read
+
+jobs:
+  # -------------------------------------------------------------------------
+  #  Stage and compile the docs site. Catches what only a full build can:
+  #  broken internal links, files under docs/ not covered by a mount, and
+  #  spine entries with no backing page. docs:test guards the gather/export
+  #  logic itself and runs the node README and client-doc validators;
+  #  docs:check is the export drift gate. Blocking — every failure mode here
+  #  is deterministic. (docs.yml's deploy build re-runs the same commands on
+  #  develop with the full-history checkout its sitemap <lastmod> stamping
+  #  needs; a shallow clone is fine for a PR gate.)
+  # -------------------------------------------------------------------------
+  build:
+    name: Docs site build
+    runs-on: ubuntu-latest
+    timeout-minutes: 15
+    steps:
+      - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
+        with:
+          persist-credentials: false
+      - uses: pnpm/action-setup@fc06bc1257f339d1d5d8b3a19a8cae5388b55320 # v4
+      - uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4
+        with:
+          node-version: 20
+          cache: pnpm
+      - run: pnpm install --frozen-lockfile
+      - name: Install validator test dependencies
+        run: python3 -m pip install --disable-pip-version-check pytest==8.4.1
+      - name: Docs unit tests
+        run: node scripts/build.js docs:test
+      - name: Build docs site
+        run: node scripts/build.js docs:build
+      - name: Check docs export drift
+        run: node scripts/build.js docs:check

.github/workflows/docs.yml

@@ -8,10 +8,14 @@ on:
   push:
     branches: [develop]
     paths:
-      - 'packages/docs/**'
+      - 'docs/**'
       - 'nodes/src/nodes/**/README.md'
       - 'packages/*/docs/**'
       - 'apps/*/docs/**'
+      - 'packages/client-typescript/README.md'
+      - 'packages/client-python/README.md'
+      - 'packages/client-mcp/README.md'
+      - 'packages/n8n-nodes/README.md'
       - '.github/workflows/docs.yml'
   workflow_dispatch: {}
 
@@ -44,6 +48,8 @@ jobs:
       - run: pnpm install --frozen-lockfile
       - name: Build docs
         run: node scripts/build.js docs:build
+      - name: Check docs export drift
+        run: node scripts/build.js docs:check
       - uses: actions/upload-pages-artifact@56afc609e74202658d3ffba0e8f6dda462b719fa # v3.0.1
         with:
           path: dist/docs

.gitignore

@@ -24,6 +24,15 @@ out/
 .claude/*
 !.claude/CLAUDE.md
 .cursor/
+.windsurf/
+# Agent stubs planted by the RocketRide VS Code extension's agent installer
+# (apps/vscode/src/agents/). It gitignores its own .rocketride/ docs but not
+# these, so they land in commits. Anchored to the root so the deliberate
+# .claude/CLAUDE.md pointer above still applies. AGENTS.md cannot be listed
+# here — it is a real tracked file — so the installer is also switched off in
+# .vscode/settings.json, which is what stops it appending to AGENTS.md.
+/CLAUDE.md
+/.github/copilot-instructions.md
 
 # -----------------------------------------------------------------------------
 # Git hooks (local overrides)
@@ -210,13 +219,6 @@ user.json
 # Build version info (generated)
 version.json
 
-# Package READMEs (copied from docs/ by build; packaging requires them here)
-packages/chat-widget/README.md
-packages/client-typescript/README.md
-packages/client-python/README.md
-packages/client-mcp/README.md
-apps/vscode/README.md
-
 downloads/jdk-17.zip
 downloads/jre-17.zip
 downloads/jdk-17.tar.gz