Repository navigation
docs: complete the documentation revamp — consolidated tree, schemas + CI gates, node migration, new site - #2139
Merged
Merged
Conversation
…xport pipeline (#1856) * feat(docs): move spine pages to docs/product/, re-point gather Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(docs): mount client/mcp/engine docs from top-level docs/ tree Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(docs): move contributor docs to docs/internal/ Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(docs): add docs:export/docs:check; readme sources under docs/ Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(docs): docs:check skips gitignored export destinations Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(docs): commit exported package READMEs (un-gitignore) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(docs): remove docs/images and all references Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(docs): restore list spacing in vscode docs Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs: update doc-location rules, folder guide, CI drift gate Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * ci(docs): trigger docs workflow on top-level docs/ changes Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(clients): source package READMEs from committed docs:export copies Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(docs): close CI drift-gate gap, restore .pipe rule bullet Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs: drop reserved skills/ section from docs/ tree Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs: move top-level images/ into docs/images/ All doc-referenced images now live under docs/. Root README relative paths and main/develop-pinned raw.githubusercontent URLs rewritten; generated package READMEs refreshed via docs:export. Orphans (banner-root, screenshot-ide, screenshot-sdk) carried over unchanged. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs: fix stale README-* link labels and broken post-move links in internal docs CodeRabbit bucket-1 findings plus three broken links the sweep surfaced: README-vscode.md and README-*-client.md references now point at the new clients/ and mcp/ locations. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs: address CodeRabbit review findings across doc sources Verified each finding against current SDK/node code before fixing: - H1s, fence languages, MD031 blank lines, heading levels (lint batch) - Example correctness: import os, double-wrapped pipeline args (4 sites), undefined vars, Answer instance API documented from Question.ts, validate-gated use(), deploy rollback to recorded version, getTaskStatus/validate signatures from client.ts - Config correctness: postgres questions-lane wiring (both examples), qdrant serverName->host, neo4j password field, execution-model documents lane, node count 100+ - Security-hardening in crash-reporting.md examples (pinned checksum install, mktemp, env-var instead of interpolation), self-hosting default host now 127.0.0.1 - builder.md skip conditions include --force; LICENSE link depth (2 files) - Generated package READMEs refreshed via docs:export Deferred as code-behavior (not doc) issues: firecrawl unbounded 429 retry, vscode ROCKETRIDE_APIKEY persistence on disconnect. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
…e cleanup (#1912) * docs(agents): make AGENTS.md the single-source contributor router AGENTS.md becomes the one canonical guide: repo overview, task routing table into docs/, common commands (absorbed from .claude/CLAUDE.md), conventions, and the documentation rules. .claude/CLAUDE.md becomes an @AGENTS.md import and .cursorrules a pointer, eliminating the three divergent copies of the doc-location rules. A nested nodes/AGENTS.md gives agents working in the nodes tree the README schema and validator command with zero indirection. docs/README.md references the documentation contracts in docs/internal/. * docs(nodes): add node README schema and deterministic validator docs/internal/node-readme-schema.md defines the per-node README contract: a fixed section set where services*.json triggers decide which sections are required (invoke→Connections, lanes→Lanes, tool∈classType→As a tool, ≥2 profiles→Profiles, gpu→Requirements, restricted capabilities→ Limitations), core sections for every node (summary, What it does, Configuration usage guidance, Example pipelines), and optional sections with placement rules (About vendor, Authentication, Notes, Upstream docs). The generated ROCKETRIDE:GENERATED:PARAMS region owns the field reference; hand-written Configuration is usage guidance only. scripts/validate-node-readme.py enforces the structural rules per node: section presence/order, forbidden-when-untriggered, table parity against declared invoke keys/lanes/profiles, and warnings when a field with objective complexity signals lacks a detail subsection. Multi-service directories are merged; fragment-only files (no protocol) are skipped. * docs(nodes): migrate six sample node READMEs to the README schema llm_anthropic, db_postgres, tool_github, store_qdrant, ocr, and agent_deepagent restructured to the schema in docs/internal/node-readme-schema.md. Content-preserving: existing prose kept, sections reordered into the canonical layout, bespoke headings folded into their owning sections or Notes, and the generated ROCKETRIDE:GENERATED:PARAMS regions untouched byte-for-byte. Substantive fixes made along the way: - llm_anthropic: Profiles table now lists all 21 declared profiles (previously 7 — Opus 5, Fable 5, Sonnet 5, 4.8 variants, latest aliases and 3 Haiku were declared in services.json but undocumented) - tool_github: removed a duplicated all-tools table that shadowed the per-group tool tables; grouped tables are now the single source - All six gain Example pipelines; db_postgres gains the Limitations section its noremote capability requires; vendor nodes gain About sections Validator: Profiles parity accepts key-or-title in the first two columns; grouped detail subsections satisfy the complex-field check via token containment. Schema: preset services with identical wiring may be documented once (db_postgres/Supabase case). * ci(docs): advisory node README schema check + CodeRabbit doc rules New node-docs CI job runs scripts/validate-node-readme.py over the node tree on every PR. Advisory for now (continue-on-error, not in ci-ok): the corpus is mid-migration to the schema, so the job surfaces results without blocking merges; flipping it to a hard gate is a two-line change once migration completes. CodeRabbit path_instructions direct review attention where the deterministic validator cannot reach: prose accuracy of node READMEs against code/services*.json (structure is machine-checked), and Python/TypeScript client-doc parity. * docs(clients): client readme schema, parity checker, and parity fixes docs/internal/client-readme-schema.md codifies the shared section order the Python and TypeScript readmes already follow, fixes the insertion points for language-specific sections, and states the parity rule: any documented method/type/exception exists in both readmes (snake_case and camelCase compare equal) or carries a language-specific marker. scripts/validate-client-docs.py enforces it deterministically: section presence/order/ownership plus API symbol parity harvested from method tables across the six shared API sections. Parity gaps it found, fixed against client source: - python readme was missing the attach/detach/is_attached/login/logout/ is_authenticated/set_env connection lifecycle (implemented in mixins/connection.py, previously undocumented) and Answer.setAnswer - typescript readme was missing Question.addGoal (Question.ts:264) - dap_request (python convenience) and withConnection (typescript convenience) marked language-specific with cross-references Package READMEs regenerated via docs:export; docs:check green; docs:build page count unchanged (179 pages, 125 nodes, 6 mounts). * refactor(docs): split protocols out, rename internal to development The docs/ tree is organized by audience, but docs/internal/ was a mixed bag: it held the WebSocket protocol surface (published to the site as Protocols > WebSocket) alongside contributor-only material, and an integration guide that belonged with the other integrations. - New docs/protocols/ holds both members of the site's Protocols section: websocket/ (was docs/internal/engine-protocol/) and mcp/ (was docs/mcp/). The section's two halves previously lived in unrelated trees. - docs/internal/ -> docs/development/. Every file that remains is contributor documentation: setup, builder, engine internals, node authoring, pre-commit hooks. The old name also read as company-private in a public repository, which is the opposite of the signal contributor docs should send. - docs/internal/n8n.md -> docs/product/integrations/n8n.md and onto the spine, joining the six existing integration guides. Prepared for publication: frontmatter added, repo-relative links rewritten to site routes or blob/develop URLs, and a reference to a local-only test harness removed. Every folder under docs/ is now entirely published or entirely not. docs/README.md documents that invariant and groups the sections by it. Paths updated in gather.js (mounts + root glob), export.js (MCP readme source), spine.js (integrations), AGENTS.md, nodes/AGENTS.md, docs/README.md, .coderabbit.yaml, ci.yml, SUPPORT.md, examples/README.md, tool_n8n's README, both validators, and the regenerated MCP package README. Both validators also reformatted to the repo's ruff style. Verification: docs:test 42/42, docs:check in sync, docs:build green, docs:gather 180 pages (179 + the newly published n8n page) / 125 nodes / 6 mounts. Node and client validators unchanged in behavior. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * ci(docs): build the docs site on doc-touching PRs; run client parity checker Review of the doc-CI surface found two genuine gaps; everything else was judged adequate or deliberately rejected to avoid CI sprawl. Gap 1: the docs site was only ever built AFTER merge (docs.yml on push to develop), so deterministic breakage — broken internal links, docs/ files not covered by a mount, spine entries without a page — merged clean and failed on develop's deploy. New path-filtered docs-build job runs docs:test + docs:build on PRs touching doc sources, wired into ci-ok (skip-tolerant, same pattern as helm/shell-contract). Blocking is justified: every failure mode is deterministic, no network flakiness. Gap 2: scripts/validate-client-docs.py was committed but nothing executed it. Added to the advisory node-docs job (renamed 'Doc schemas (advisory)'). Rejected as sprawl: an external-link checker (network-flaky by construction), prose/spell linting (noise on a technical corpus), a separate node-site-coverage check and a docs-folder-invariant check (both subsumed — the first by the advisory schema validator, the second by docs:build's unmounted-file abort now running on PRs). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Krish Garg <krish.garg@rocketride.ai> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
…irement (#1960) * docs(agents): make AGENTS.md the single-source contributor router AGENTS.md becomes the one canonical guide: repo overview, task routing table into docs/, common commands (absorbed from .claude/CLAUDE.md), conventions, and the documentation rules. .claude/CLAUDE.md becomes an @AGENTS.md import and .cursorrules a pointer, eliminating the three divergent copies of the doc-location rules. A nested nodes/AGENTS.md gives agents working in the nodes tree the README schema and validator command with zero indirection. docs/README.md references the documentation contracts in docs/internal/. * docs(nodes): add node README schema and deterministic validator docs/internal/node-readme-schema.md defines the per-node README contract: a fixed section set where services*.json triggers decide which sections are required (invoke→Connections, lanes→Lanes, tool∈classType→As a tool, ≥2 profiles→Profiles, gpu→Requirements, restricted capabilities→ Limitations), core sections for every node (summary, What it does, Configuration usage guidance, Example pipelines), and optional sections with placement rules (About vendor, Authentication, Notes, Upstream docs). The generated ROCKETRIDE:GENERATED:PARAMS region owns the field reference; hand-written Configuration is usage guidance only. scripts/validate-node-readme.py enforces the structural rules per node: section presence/order, forbidden-when-untriggered, table parity against declared invoke keys/lanes/profiles, and warnings when a field with objective complexity signals lacks a detail subsection. Multi-service directories are merged; fragment-only files (no protocol) are skipped. * docs(nodes): migrate six sample node READMEs to the README schema llm_anthropic, db_postgres, tool_github, store_qdrant, ocr, and agent_deepagent restructured to the schema in docs/internal/node-readme-schema.md. Content-preserving: existing prose kept, sections reordered into the canonical layout, bespoke headings folded into their owning sections or Notes, and the generated ROCKETRIDE:GENERATED:PARAMS regions untouched byte-for-byte. Substantive fixes made along the way: - llm_anthropic: Profiles table now lists all 21 declared profiles (previously 7 — Opus 5, Fable 5, Sonnet 5, 4.8 variants, latest aliases and 3 Haiku were declared in services.json but undocumented) - tool_github: removed a duplicated all-tools table that shadowed the per-group tool tables; grouped tables are now the single source - All six gain Example pipelines; db_postgres gains the Limitations section its noremote capability requires; vendor nodes gain About sections Validator: Profiles parity accepts key-or-title in the first two columns; grouped detail subsections satisfy the complex-field check via token containment. Schema: preset services with identical wiring may be documented once (db_postgres/Supabase case). * ci(docs): advisory node README schema check + CodeRabbit doc rules New node-docs CI job runs scripts/validate-node-readme.py over the node tree on every PR. Advisory for now (continue-on-error, not in ci-ok): the corpus is mid-migration to the schema, so the job surfaces results without blocking merges; flipping it to a hard gate is a two-line change once migration completes. CodeRabbit path_instructions direct review attention where the deterministic validator cannot reach: prose accuracy of node READMEs against code/services*.json (structure is machine-checked), and Python/TypeScript client-doc parity. * docs(clients): client readme schema, parity checker, and parity fixes docs/internal/client-readme-schema.md codifies the shared section order the Python and TypeScript readmes already follow, fixes the insertion points for language-specific sections, and states the parity rule: any documented method/type/exception exists in both readmes (snake_case and camelCase compare equal) or carries a language-specific marker. scripts/validate-client-docs.py enforces it deterministically: section presence/order/ownership plus API symbol parity harvested from method tables across the six shared API sections. Parity gaps it found, fixed against client source: - python readme was missing the attach/detach/is_attached/login/logout/ is_authenticated/set_env connection lifecycle (implemented in mixins/connection.py, previously undocumented) and Answer.setAnswer - typescript readme was missing Question.addGoal (Question.ts:264) - dap_request (python convenience) and withConnection (typescript convenience) marked language-specific with cross-references Package READMEs regenerated via docs:export; docs:check green; docs:build page count unchanged (179 pages, 125 nodes, 6 mounts). * refactor(docs): split protocols out, rename internal to development The docs/ tree is organized by audience, but docs/internal/ was a mixed bag: it held the WebSocket protocol surface (published to the site as Protocols > WebSocket) alongside contributor-only material, and an integration guide that belonged with the other integrations. - New docs/protocols/ holds both members of the site's Protocols section: websocket/ (was docs/internal/engine-protocol/) and mcp/ (was docs/mcp/). The section's two halves previously lived in unrelated trees. - docs/internal/ -> docs/development/. Every file that remains is contributor documentation: setup, builder, engine internals, node authoring, pre-commit hooks. The old name also read as company-private in a public repository, which is the opposite of the signal contributor docs should send. - docs/internal/n8n.md -> docs/product/integrations/n8n.md and onto the spine, joining the six existing integration guides. Prepared for publication: frontmatter added, repo-relative links rewritten to site routes or blob/develop URLs, and a reference to a local-only test harness removed. Every folder under docs/ is now entirely published or entirely not. docs/README.md documents that invariant and groups the sections by it. Paths updated in gather.js (mounts + root glob), export.js (MCP readme source), spine.js (integrations), AGENTS.md, nodes/AGENTS.md, docs/README.md, .coderabbit.yaml, ci.yml, SUPPORT.md, examples/README.md, tool_n8n's README, both validators, and the regenerated MCP package README. Both validators also reformatted to the repo's ruff style. Verification: docs:test 42/42, docs:check in sync, docs:build green, docs:gather 180 pages (179 + the newly published n8n page) / 125 nodes / 6 mounts. Node and client validators unchanged in behavior. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * ci(docs): build the docs site on doc-touching PRs; run client parity checker Review of the doc-CI surface found two genuine gaps; everything else was judged adequate or deliberately rejected to avoid CI sprawl. Gap 1: the docs site was only ever built AFTER merge (docs.yml on push to develop), so deterministic breakage — broken internal links, docs/ files not covered by a mount, spine entries without a page — merged clean and failed on develop's deploy. New path-filtered docs-build job runs docs:test + docs:build on PRs touching doc sources, wired into ci-ok (skip-tolerant, same pattern as helm/shell-contract). Blocking is justified: every failure mode is deterministic, no network flakiness. Gap 2: scripts/validate-client-docs.py was committed but nothing executed it. Added to the advisory node-docs job (renamed 'Doc schemas (advisory)'). Rejected as sprawl: an external-link checker (network-flaky by construction), prose/spell linting (noise on a technical corpus), a separate node-site-coverage check and a docs-folder-invariant check (both subsumed — the first by the advisory schema validator, the second by docs:build's unmounted-file abort now running on PRs). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(docs): shipped example bundle — screenshot + downloadable .pipe per node Every node's Example pipelines section now opens with a shipped example: a minimal runnable pipeline committed as example.pipe next to the node, a canvas screenshot of that same pipeline (example.png), and a shields.io download badge. Readers browsing the docs see the node wired on the canvas and can download the .pipe straight into their own canvas. - Schema: the standalone optional screenshot (formerly under What it does) is folded into Example pipelines as part of the bundle. Authors reference example.png / example.pipe by bare relative name so node folders stay self-contained and GitHub renders them natively. - gather.js: stageNodeMarkdown rewrites those two refs to develop-pinned repository URLs when staging site pages (raw for the image, blob for the .pipe download) — same pattern as the generated Source link. Verified end-to-end on a staged page. - Validator: bundle is all-or-nothing (pipe file, screenshot file, embed, download link + alt text) — any part present makes every part required; nothing at all is a warning until the node's bundle lands. New nodes must ship it (CodeRabbit instruction added). docs:test 42/42, staging parity (180 pages / 125 nodes / 6 mounts), 6 samples still pass (now with the expected bundle warning), ruff clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(nodes): move Example pipelines above the reference sections; migrate agent_llamaindex Example pipelines now sits directly under What it does, so a reader sees what the node is, a canvas screenshot, and a downloadable .pipe before any reference tables. Schema sections renumbered; the six migrated samples reordered to match. agent_llamaindex migrated to the schema: About LlamaIndex, three example pipelines, Connections/Lanes/As a tool tables matching services.json, and Configuration reduced to usage guidance — the duplicated hand-written field table is gone (the generated Schema table owns it), with detail subsections for the three fields that need them (Agent description as the parent-agent selection signal, Instructions, Require tool call as the fabrication guard). Loop internals and SSE progress moved under Notes. 7/125 nodes pass. docs:test 42/42, staging parity, ruff clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(docs): trigger the example bundle on references, ship agent_llamaindex's .pipe The all-or-nothing rule counted a loose file as a bundle part, so committing example.pipe before its screenshot existed would fail the node — blocking the natural two-step path (author the pipeline, screenshot it later). The trigger is now a README *reference*: referencing either half obliges the other and every referenced file must exist (no broken links, never a half-rendered README), while an unreferenced file is a warning naming what is present and what is still needed. agent_llamaindex's example.pipe is the existing, working examples/agent-llamaindex.pipe — chat -> agent (llm_anthropic on the llm channel, tool_http_request on tool) -> response_answers — rather than an authored approximation. Its screenshot and the two references land when the pipeline is captured on the canvas. Verified all three paths: unreferenced file warns, a lone badge fails, a referenced-but-missing file fails. docs:test 42/42, staging parity, 7/125 nodes pass, ruff clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(nodes): ship example.pipe for all seven migrated nodes; fix a broken example Every migrated node now carries the .pipe half of its example bundle, so completing each one is capture-a-screenshot plus two pasted lines. docs/development/example-bundle-todo.md holds the per-node instructions, the exact markdown to paste (alt text written per node), and provenance. Three are existing working pipelines copied from examples/: agent_llamaindex <- agent-llamaindex.pipe, ocr <- document-processor.pipe, store_qdrant <- rag-pipeline.pipe. The other four (llm_anthropic, db_postgres, tool_github, agent_deepagent) were authored from those proven shapes and statically validated: every provider is a declared node protocol, every input lane is one the upstream node produces, every control channel is declared in the target's invoke. They are unrun — opening each on the canvas for its screenshot is the verification step. Also fixes examples/document-processor.pipe, which referenced provider 'anonymize' while the node declares 'anonymize_text' — a broken example that predates this branch. All 17 files in examples/ now reference only real providers. 7/125 nodes pass, each with the expected unreferenced-bundle warning. docs:test 42/42, staging parity. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(docs): centre the example screenshot and download button, matching the root README The markdown image form rendered left-aligned at natural size; the root README gets its look from an HTML block, so node examples now use the same house style: a centred div with a width=100% screenshot above the shields.io download button. Safe on the docs site because Docusaurus is configured markdown.format 'detect', so .md is parsed as CommonMark rather than MDX (tags are still self-closed for portability). - gather.js: the staging rewrite now matches HTML attribute forms (src=, href=) as well as markdown targets, so the centred block resolves to repository URLs on the site. The shields.io URL is deliberately untouched. - Validator accepts either form and reads alt text out of the img tag, so the alt-text requirement still holds. - Schema documents the HTML block as house style and records why the link stays relative: the VS Code extension registers a custom editor for *.pipe, so clicking the button in the IDE opens the pipeline directly on the canvas — an absolute URL would bounce the reader to a browser. - agent_llamaindex converted, its example.png committed, and its first example corrected to describe the pipeline actually shipped (chat + tool_http_request, not webhook + tool_tavily). First fully complete bundle in the repo. - Checklist snippets regenerated in the new form. docs:build green (HTML compiles on the site), docs:test 42/42, 7/125 nodes pass, ruff clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(docs): keep the example block renderable in the VS Code preview The raw-HTML form centred correctly on GitHub but rendered as empty space in the VS Code / Cursor markdown preview: that preview rewrites relative paths only for markdown-syntax images, so an <img src="example.png"> is left unresolved and blocked. The IDE is the surface where the download button matters most (clicking it opens the pipeline on the canvas), so it cannot be the one that breaks. Blank lines inside the wrapper fix it. A blank line ends the HTML block per CommonMark, so the image and badge parse as markdown — resolving in the preview — while the surrounding div still centres them, and GitHub scales the screenshot to the content width on its own. Renders in all three surfaces: IDE preview, GitHub, and the docs site. Schema records why the blank lines are load-bearing so nobody tidies them away. Checklist snippets regenerated. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(nodes): complete shipped example bundles * docs(nodes): park the Example pipelines section and its bundle checks The shipped-example bundle (canvas screenshot + downloadable .pipe) is complete for seven nodes but is a per-node manual step, and it blocks the migration of the remaining ~120 READMEs behind screenshot capture. Park the requirement so the migration can proceed on prose alone. `## Example pipelines` is commented out in the schema with its full text preserved in place, and the validator's bundle checks are commented out alongside it — un-parking is uncommenting both and re-numbering the schema's sections. The section stays in the validator's CANONICAL_ORDER, so it remains tolerated in its slot rather than flagged as an unknown heading: the seven migrated nodes keep their examples and keep passing. It is simply no longer required. gather.js keeps rewriting relative example refs for the staged site — inert with nothing to rewrite, needed when this returns. * ci: scope the doc schema checks to fix/docs, leave the shared pipeline alone The doc schema work added four things to config that every branch shares: a docs path filter and two new jobs in ci.yml, a pnpm install plus a docs:check step bolted onto the always-on ruff-check job, and two path instructions in .coderabbit.yaml. None of that belongs on develop while the schemas are still being rolled out — and the ruff-check change in particular would have added an install and a docs build to every pull request in the repository, not just documentation ones. ci.yml is restored byte-for-byte to its pre-branch state, so merging fix/docs into develop is now a no-op for the shared pipeline. The same checks now live in .github/workflows/docs-schemas.yml, which triggers only on pull requests and pushes targeting fix/docs. It stays inert on develop even after this branch merges. Validators are advisory (the node corpus is mid-migration and would paint every PR red); the site build and export drift gate are blocking, since both pass today. CodeRabbit has no per-branch config file, but it reads .coderabbit.yaml from a pull request's base branch — so adding fix/docs to base_branches here enables review for PRs into this branch while develop keeps using its own copy. That entry and the two doc path instructions are fenced with TEMPORARY markers to be removed in the fix/docs -> develop PR. * ci: drop the unusable workflow_dispatch trigger from the doc schema checks workflow_dispatch only fires when the workflow file lives on the repository's default branch. This file is deliberately scoped to fix/docs and is meant never to reach develop, so the trigger could never have worked — gh confirms it with 'workflow not found on the default branch'. Removed rather than left as dead configuration. * chore: drop agent stubs the VS Code extension planted in this repo The extension's agent installer (apps/vscode/src/agents/) writes AI assistant instruction files into whatever workspace is open, on startup, with rocketride.integrations.autoAgentIntegration defaulting to true. It adds its own .rocketride/ docs directory to .gitignore but not the stubs it plants, so three of them were swept into b7c9918, and its marker-based merge appended a thirtieth-line block to AGENTS.md because that file had no ROCKETRIDE markers to replace. That block is the reason CodeRabbit flagged AGENTS.md as self-contradicting on #1960: line 21 sends contributors to docs/agents/ROCKETRIDE_QUICKSTART.md while the appended block made .rocketride/docs/ROCKETRIDE_README.md step 1. AGENTS.md is the contributor router, and .rocketride/docs/ is customer-facing SDK documentation — the wrong guidance for someone working on the engine. Removes the three files and the appended block, and gitignores the stub targets not already covered (.cursor/ and .claude/* already were, which is the only reason the Cursor stub never landed). AGENTS.md cannot be gitignored, so the installer is also disabled in .vscode/settings.json, which is tracked and therefore fixes this for everyone who opens the repo rather than one machine. The underlying extension bug is not fixed here. --------- Co-authored-by: Krish Garg <krish.garg@rocketride.ai> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
…locking (#1971) * ci(docs): gate the node schema on the PR's own diff, and make it blocking The corpus-wide run was advisory because 109 of 124 nodes fail the schema today, so a hard gate would have been red on every PR from the day it was added. But a check that has always failed is a check nobody reads, and the day it fails for a new reason looks identical to the two hundred days before it. The interesting failures were going to be buried under structural noise like "unknown heading '## Modes'". Scoping it to the nodes the PR actually touches inverts that: green by default, so red means this PR broke something. It also lets the check be a hard gate immediately rather than waiting on a 109-node cleanup to finish before it protects anything, and the backlog drains on its own — touching a node obliges you to bring its README along — while never being able to grow. services*.json counts as a trigger alongside README.md, since the metadata is what decides which sections are required or forbidden; a metadata-only change can invalidate a README nobody edited. That also closes the gap where the docs path filter watched only README.md. --all keeps a home in a new informational job that writes pass/fail counts to the run summary, where a large number is a trend line rather than a merge blocker. Retire that job when the count reaches zero. Checked against the existing docs:check gate to be sure two checks cannot disagree about what a good README is: docs:check verifies that exported client package READMEs match their docs/ sources, which is export drift, not document structure. No overlap. Verified under bash (CI's shell, not the zsh this was written in, which does not word-split unquoted variables): 8 nodes detected from the diff, all pass, exit 0; including an unmigrated node exits 1. * ci(docs): stop the scoped schema gate failing on a diff with no node files Two bugs CodeRabbit caught in the previous commit, both reproduced before fixing. Under `set -euo pipefail`, grep exits 1 when nothing matches, so a PR that touched no node README or services*.json died on the command substitution before reaching the empty check that was meant to report a clean skip. Since that job is now blocking, an unrelated PR into fix/docs would have been blocked outright. The `-d` test in the trailing loop had the same shape when the last candidate was a deleted node. `grep -c` already prints 0 when it matches nothing and exits 1 as well, so `|| echo 0` emitted a second zero and the run summary rendered "0 0". Verified: empty diff reaches the skip path and exits 0; the real diff still resolves 8 nodes; counts render as a single 0. --------- Co-authored-by: Krish Garg <krish.garg@rocketride.ai>
…1973) `table_col` stripped `[*_`]` from every table cell to remove markdown decoration before comparing names against services*.json. Underscores are markdown emphasis, but they are also legal in the identifiers this function exists to compare, so the strip silently corrupted them: `_source` became `source` and no longer matched the lane that declared it. Caught on tool_pipe, whose README was correct and failed anyway. Thirteen nodes carry an underscore in a lane, connection, or profile name, and the LLM family is the worst affected — `gemini-2_5-flash`, `llama3_2-vision-11b`, `ai21_jamba-1_5-large` — so this would have produced a wave of false failures on the largest batch still to come. Profiles masked the bug because their comparison falls back to fuzzy containment across two columns; lanes and connections compare strictly, so they failed outright. Emphasis is now only removed when a pair of underscores wraps the whole cell, which is the only case where an underscore is decoration rather than part of a name. Verified against `_source`, `z_score`, `gemini-2_5-flash`, `streamable_http`, `_emphasised_`, and both the `*(default)*` and `_(default)_` markers already in use. No change to the corpus pass count. Co-authored-by: Krish Garg <krish.garg@rocketride.ai>
…ges (#1969) * refactor(docs): reorganize docs/ by audience and retire duplicated pages Collapses docs/ to three top-level folders keyed by who reads them, brings all five clients under one roof, and removes the hand-written pages that duplicated generated node documentation. docs/ public/ humans outside the repo: typescript, python, vscode, mcp, n8n (each with a distribution README.md and assets/), product/ (the site spine), assets/ development/ contributors; never published, no exceptions agents/ AI assistants: ROCKETRIDE_*.md, stubs/, skills/ clients/, protocols/, stubs/ and images/ are dissolved. Site mount destinations are unchanged, so no published URL moves except where noted below. Five clients, one place. mcp graduates from protocols/ to a client folder (its README already was the published rocketride-mcp README). n8n joins as the fifth: packages/n8n-nodes/README.md is now generated by docs:export like the other three, so it can no longer drift. Integration pages retired. Six of seven pages under product/integrations were node-README duplicates at 73-90% overlap, and postgres.md had already drifted into publishing Supabase instructions that the db_postgres README says will fail to resolve. All six are deleted and redirected to their node pages; n8n survives because it documents webhook ingress, credential-vault and human-in-the-loop flows that no single node README can own. development/ is now genuinely internal. Its websocket docs split by audience: the wire-protocol reference (external integrators) moved to the site spine and keeps its URLs, while crash-reporting.md (lldb, breakpad) is unpublished. apps.md split likewise, its published-SDK half becoming develop/apps with shell-ui imports normalized to rocketride/app-sdk. gather no longer sweeps development/ at all, so it cannot publish by accident. The remainder is grouped by subsystem (builder/, engine/, nodes/, clients/, apps/). product/ regrouped. The real quickstart was filed under Evaluate while /quickstart was an empty card grid pointing at it; it is now a landing page with the tutorial split into an IDE walkthrough and an SDK guide. Loose root files became operate/ (cloud, self-hosting, troubleshooting) and reference/ (cli, glossary, pipeline-reference). Broken distribution assets fixed. Every banner in the published READMEs pointed at a main-pinned raw URL for a path that had never existed there, so all of them 404 on npm, PyPI and the Marketplace. They now target the per-client assets/ paths; the root README uses relative paths that resolve on any branch. New guardrail: a doc id is a public URL here, and a spine id with no backing file used to publish a silent "coming soon" page. assertNoUnexpectedPlaceholders now fails the build and names the offending id, with an allowlist for intentional stubs. It caught four moved URLs hardcoded in the site footer during this work. Also adds development/docs-pipeline.md and development/ci-gates.md, deletes a service catalog that had drifted to 118/148 against a real 143/163, and fixes setup-guide and root-file link depths. Verified: docs:build 176 pages / 125 nodes / 5 mounts, docs:check in sync, docs:test 48/48, validate-client-docs.py passing, docs:export idempotent. 14 retired routes all have emitted redirects. * docs: address CodeRabbit review on #1969 Twelve of thirteen findings accepted; several were symptoms of wider problems and were fixed at the root. Wrong technical claims, verified against the source and rewritten: - Data lanes and agent tools were described as mutually exclusive. Nine of 27 `tool` services declare lanes, and validate-node-readme.py gates the two sections independently. Replaced with a 2x2 matrix. - The node loader contract was pseudocode for a `MyNode`/`process()` API that does not exist. python-instance.cpp requires `IInstance`, python-global.cpp treats `IGlobal` as optional, and bindMethod binds callbacks by name from a 24-entry list, silently no-oping on a typo. Replaced with a working three-file template. - `./builder test server` is not an alias for the server tests: bare args fan out to every module. Corrected here and two more sites in index.md. Host and path corrections: - The SDK `uri` examples used cloud.rocketride.ai, which is the web console. The API host is api.rocketride.ai (CONST_DEFAULT_WEB_CLOUD in both SDKs since #1078). Only SDK/URI contexts were changed; ~20 sites across the Python, TypeScript and VS Code pages, which had drifted away from the package READMEs that were already correct. - The exported agent doc linked into docs/development/, which docs:export never copies to .rocketride/docs/. Repointed at a sibling exported doc and published URLs. - docs.rocketride.ai -> .org in apps/index.md, matching docusaurus.config.ts. Also: documented the four app-SDK methods missing from the API list, corrected the n8n trigger's authentication description (it declares no credentials, only a `secret` parameter checked against the Authorization header), fixed rsbuild dependency drift in the app template, normalized bare `builder` to `./builder` in 14 places, tagged three untagged code fences, and ran prettier over redirects.ts (all 702 entry lines predated this PR; array data verified identical before and after). Rejected: the suggestion to unify updateEditorViewState and updateEditorViewport. They are distinct methods with different signatures in both Documents.tsx and shell-api/v0.d.ts; unifying them would introduce a bug. Also rejected `npx` -> `pnpm exec`, which the apps' own tasks.js contradict. Two source-level issues found and left for separate PRs: app-sdk/index.ts is missing four methods the runtime exposes, and tool_firecrawl/README.md was already schema-nonconformant.
* docs(nodes): migrate tool_apify, tool_bland_ai, tool_chartjs, tool_cognee, tool_daytona, tool_deepl to the README schema * docs(nodes): migrate tool_exa_search, tool_filesystem, tool_firecrawl, tool_git, tool_guild, tool_http_request to the README schema * docs(nodes): migrate tool_laserdata_memory, tool_mcp_client, tool_mem0, tool_n8n, tool_oura, tool_pipe to the README schema * docs(nodes): migrate tool_google_workspace, tool_pipedrive, tool_python, tool_slack, tool_tavily, tool_v0, tool_xtrace_memory to the README schema * docs(tool_pipedrive): clarify bearer token format * docs(nodes): use American spelling in tool_n8n --------- Co-authored-by: Krish Garg <krish.garg@rocketride.ai>
) * docs(nodes): migrate LLM family with progressive model tables * docs(nodes): settle the Profiles contract before the other families rebase This PR's schema and validator changes become the rules the five in-flight node-README PRs are held to, so four gaps get closed here rather than after 74 more nodes have adopted them. Each protocol-bearing service keeps its own default. A second registration is a separate node to the engine — a branded preset, a second provider, a second backend — so its preconfig.default is a fact about that service, not a competing claim about the primary one. The old single-default rule false-failed cloud_tts (#1975) and store_elasticsearch (#1970), which are correct as written, and it also hid a real defect here: llm_openai_api's Nebius default llama-3-3-70b rendered unmarked, and the intro bolded a title the metadata does not declare. Both fixed. Four tests cover the multi-service case; there were none. The visible-table rule said "until the table reaches six rows" while the validator enforced "at most six", so ten of twelve large-layout nodes under-fill against the written text. Six is now stated as a ceiling, not a quota, with an explicit clause for the nodes that front several vendors' catalogues — llm_bedrock was collapsing Llama 4 Scout and Maverick while showing Llama 3.3 above the fold, which that clause now forbids and this change corrects. How a row names its profile is now stated: the declared key as code, or the declared title reproduced exactly, one form per node. That is what the validator already enforced through resolve_profile_row, but leaving it unwritten is why guardrails (#1978) fails on re-cased display names. The default marker is **(default)**. The validator stays decoration-agnostic so pages using *(default)* keep passing; the schema now says which to write. No regression across the 124-node corpus: same failure set before and after. 21 validator tests pass. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: Dylan Savage <dylanmsavage@hotmail.com> Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
* docs(nodes): migrate store family to README schema * docs(nodes): name each store profile default in its Profiles intro #2022 landed the tightened Profiles contract: every protocol-bearing service's declared default has to appear before the first table, with its key and its title in bold, and be marked on its row. Six of the store nodes marked the row but never named the default in prose, so they fail the new validator. store_elasticsearch registers two backends with two separate defaults — Elasticsearch defaults to self-managed, OpenSearch to local. Both rows were already marked, which read as a contradiction with nothing explaining it; the intro now says which default belongs to which service. That shape is what the new multi-service clause in the schema describes. store_qdrant is not part of this migration — it was already on fix/docs and already passing the old validator — but it is the same one-line fix in the same family and it fails the new rules, so it is folded in here rather than left as an orphan nothing re-checks. Default markers switched from *(default)* to **(default)**, which is the form the schema now specifies. The validator accepts either. All nine store nodes pass; 21 validator tests pass. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: Dylan Savage <dylanmsavage@hotmail.com> Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
) * docs(cloud_tts): seed the generated params region so the node reaches the docs site gather.js stages a node only when its README carries the ROCKETRIDE:GENERATED:PARAMS marker, and nodes:docs-generate refuses to create one (resolveDocPath returns null without it), so a node that never had the marker can never acquire it on its own. cloud_tts was in that state and has been invisible on the site. Seeded an empty marker pair and ran the generator to fill it, which produced both service tables (ElevenLabs and OpenAI). Content is entirely machine-generated; nothing here is hand-written. * docs(nodes): migrate agent_crewai, agent_langchain, agent_rocketride, cloud_tts, db_clickhouse, db_mysql to the README schema * docs(nodes): migrate embedding_image, embedding_openai, embedding_transformer, embedding_video, graph_arango, graph_falkordb to the README schema * docs(nodes): migrate graph_hydradb, graph_neo4j, memory_internal, memory_persistent, preprocessor_code, preprocessor_langchain, preprocessor_llm to the README schema * fix(docs): clarify supported OpenAI embedding profiles * fix(docs): document embedding API key resolution * fix(docs): list custom node profiles * docs(nodes): bring the small families up to the new Profiles contract Nine nodes fail the contract #2022 landed. Most need only the intro sentence naming the declared default, but three carried real gaps: embedding_transformer and memory_persistent never listed their `custom` profile, so a profile the configuration panel offers was undocumented. memory_persistent went further and explained the omission in prose; the row is now present and that sentence says what `custom` is for instead. embedding_openai rendered its declared context window as "8191 tokens." in a semantic column, which the validator compares against the metadata value. The column is now `Context tokens` carrying the number alone. cloud_tts registers OpenAI and ElevenLabs with a default each. Both rows were marked with nothing explaining why there were two; the intro now names each default and the service it belongs to. Its Profile column also mixed bare keys for one provider with key-plus-title for the other — every row now uses the declared title, one convention per node as the schema asks. Profile titles are padded to align in the configuration panel's dropdown ('Text Small - ...', 'Text Large - ...'). That alignment is deliberate and belongs in services.json, but requiring a README to reproduce it verbatim in bold prose is not reasonable, so title matching now collapses whitespace runs on both sides. Two tests cover it, including that a genuine wording mismatch still fails. All 19 nodes in this PR pass; 23 validator tests pass. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(preprocessor_code): correct the parser cache's scope The README said the parser cache is retained for the lifetime of the global node instance. languageParsers is declared as a class attribute on PreProcessor (code.py:166) and the write at code.py:235 is an item assignment, so it mutates the class-level dict rather than rebinding a per-instance one. A grammar loaded once is therefore reused by every instance in the process and outlives any single global node instance. Raised by CodeRabbit on #1975. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: Krish Garg <krish.garg@rocketride.ai> Co-authored-by: Dylan Savage <dylanmsavage@hotmail.com> Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
…ad a README (#1976) * docs(nodes): migrate detect_segment, face_detection, pose_estimation, video_composer to the README schema * docs(nodes): migrate background_removal, caption, depth_estimate, detect to the README schema * docs(nodes): migrate accessibility_describe, frame_grabber, image_cleanup, thumbnail to the README schema * docs(nodes): migrate audio_player, audio_transcribe, audio_tts, twelvelabs to the README schema * docs(nodes): seed generated params regions for the eight newly documented nodes These nodes had no README at all, so they had no marker region either, and nodes:docs-generate cannot create one (resolveDocPath returns null when the marker is absent). Seeded an empty pair per node and ran the generator to fill it. Content is entirely machine-generated. * fix(docs): clarify audio player video input * fix(docs): clarify background removal output size * fix(docs): label RT-DETR fallback link * docs(nodes): bring the media nodes up to the new Profiles contract Eight nodes needed the intro sentence naming the declared default, which is all the new contract asks of this family — no missing rows, no semantic-column mismatches, no multi-service defaults. Markers switched to **(default)**. The review also turned up a boilerplate problem worth fixing while the family is open. Four ## Requirements sections opened with "This node is marked as GPU-capable. Its metadata declares local CPU, Apple Silicon (MPS), and CUDA operation" and closed with a variant of "The source does not specify VRAM requirements or relative CPU performance." Both halves describe the metadata and the writing process rather than the node: a reader who opens ## Requirements wants to know what hardware to provision, and being told what the source failed to mention answers nothing. Those sections now say where inference runs and what the device lock does, and stop. depth_estimate keeps the one fact the closer was carrying — its output is relative depth, not calibrated distance. Same fix in two other places: caption pointed at sibling nodes via "the node metadata directs object-detection use cases to Object Detection", and audio_player attributed its own truncation note to "The source notes that...". All 16 nodes pass. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: Krish Garg <krish.garg@rocketride.ai> Co-authored-by: Dylan Savage <dylanmsavage@hotmail.com>
* docs(nodes): migrate dictionary, extract_data, extract_facts, ner to the README schema * docs(nodes): migrate anomaly_detector, anonymize, answer_documents, currency_convert_explicit to the README schema * docs(nodes): migrate prompt, question, schema_validate, summarization to the README schema * docs(nodes): migrate landing_ai, llamaparse, reducto, vectorizer to the README schema * docs(landing_ai): seed the generated params region so the node reaches the docs site Last of the thirteen nodes gather.js was silently skipping. Its two variant sub-pages (extract, parse) were hidden with it, since variants are only staged once their parent is. * fix(docs): clarify anonymizer classifier fallback * fix(docs): use canonical LlamaParse links * fix(docs): document JSON scalar lane metadata * fix(docs): use American spelling consistently * fix(docs): clarify severity fallback warnings * docs(nodes): bring the text family up to the new Profiles contract Three nodes fail the contract #2022 landed. anomaly_detector needs only the intro sentence naming the declared default; anonymize and ner also never listed their `custom` profile, so a model choice the configuration panel offers was undocumented in both. The review turned up a second, larger thing. Seven nodes carried an `## About RocketRide` section. That slot is for third-party services a node wraps, and RocketRide is not third-party: on a page inside RocketRide's own documentation, "RocketRide connects data-processing components into pipelines" tells a reader nothing, and the sentence after it describes what the node does, which is what `## What it does` is for and where each of these already says it. Each one also dragged along an `## Upstream docs` link back to docs.rocketride.org — circular, and present only because `## About` requires that section. Both are removed from all seven. question's `## Requirements` said the node "is marked GPU-capable in its metadata, but its implementation only strips text". A reader opening that section wants to know what to provision, not what the metadata claims, so it now says the node needs no hardware of its own and that any GPU requirement belongs downstream. All 16 nodes pass. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: Krish Garg <krish.garg@rocketride.ai> Co-authored-by: Dylan Savage <dylanmsavage@hotmail.com> Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
* docs(nodes): migrate local_text_output, telegram, text_output, webhook to the README schema * docs(nodes): migrate autopipe, core, remote, response to the README schema * docs(nodes): migrate aparavi_aql, rocketride_graph, rocketride_sql, rocketride_vector to the README schema * docs(nodes): migrate guardrails, rerank_cohere, search_exa to the README schema * fix(docs): explain local-only node execution * docs(guardrails): document custom profile * docs(search-exa): warn against production mock bypass * docs(webhook): clarify endpoint security constraints * docs(nodes): bring the infrastructure nodes up to the new Profiles contract guardrails named its profiles "Basic", "Strict" and "Custom" — display names the metadata never declares, so no row resolved to a profile and the declared default was not marked on any of them. Rows now use the declared keys, and the prose that referred to the display names follows. remote and rerank_cohere needed the intro sentence; rerank_cohere also described its `custom` profile in a trailing sentence rather than listing it, so a profile the configuration panel offers had no row. Five nodes carried an `## About` section that was not about a third-party service. webhook and local_text_output were filler — "RocketRide is the pipeline system that hosts this node", "Python provides the filesystem functions used by this node" — and each dragged along an `## Upstream docs` link back to docs.rocketride.org that existed only because `## About` requires that section. The three managed-database nodes were different: their About carried a real fact, that RocketRide provisions a per-tenant database and the node resolves it from the signed-in identity rather than from connection fields. That is configuration guidance, and each `## Configuration` already gestured at it, so it moves there in full rather than being dropped. search_exa said the same thing twice: the third paragraph of `## What it does` restated `## Notes` → URL safety almost sentence for sentence, and took the section to nine sentences. The summary now states the guarantee and points at Notes, which keeps the detail and regains the address classes the summary had been carrying. All 15 nodes pass. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: Krish Garg <krish.garg@rocketride.ai> Co-authored-by: Dylan Savage <dylanmsavage@hotmail.com> Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
…er (#2124) These three were already on fix/docs before the six family migrations, so no PR owned them and nothing re-validated them once #2022 tightened the contract. ocr was not a documentation problem. Its table is the best-formed profile table in the corpus — twelve profiles with the key and the title in separate columns — but the first column is headed "Profile key", which PROFILE_HEADERS did not alias, so every row resolved to an empty cell and the validator reported all twelve profiles missing and twelve unknown rows. The alias is added, with a test, because separating key from title is a shape worth supporting rather than a mistake to correct. ocr additionally needed its default marked and its title in the intro. tool_mcp_client registers MCP Client and the Butterbase preset, each with its own default, and only the RocketRide one was marked — so the Butterbase preset's default was undocumented. Both are now marked and named. tool_laserdata_memory needed the intro sentence. All 124 nodes pass; 24 validator tests pass. Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
# Conflicts: # .coderabbit.yaml # .github/SUPPORT.md # docs/README-nodes.md # docs/development/builder/authoring.md # docs/public/product/develop/apps.md # docs/public/vscode/cloud-sign-in.md # docs/public/vscode/microsoft-oauth.md # nodes/src/nodes/audio_player/README.md # nodes/src/nodes/audio_transcribe/README.md # nodes/src/nodes/dictionary/README.md # nodes/src/nodes/embedding_video/README.md # nodes/src/nodes/frame_grabber/README.md # nodes/src/nodes/llm_qwen/README.md # nodes/src/nodes/llm_vision_mistral/README.md # nodes/src/nodes/thumbnail/README.md # nodes/src/nodes/tool_filesystem/README.md # nodes/src/nodes/tool_git/README.md # nodes/src/nodes/tool_mcp_client/README.md
The MCP docs described only the stdio PyPI package, while the HTTP MCP server merged from develop (PR #1880) carried its docs as a 613-line co-located doc.md — the wrong depth for a public surface, in a location the docs tree does not allow. - docs/public/mcp/ now holds two folders: stdio/ and http/, each with a README.md, rendered in the sidebar as an MCP category with two pages. The two near-duplicate stdio docs (README.md export source + index.md site page) are merged into stdio/README.md, which serves as both the /protocols/mcp/stdio page and the packages/client-mcp README source (regenerated copy included). - docs/public/mcp/http/README.md is a new public doc sized to how GitHub, Sentry, and Linear document their hosted MCP servers: endpoint, auth in two sentences, per-client setup, the 27 tools by capability group, example prompts, resources, widgets, self-hosting env vars, security, FAQ. - packages/ai/src/ai/modules/mcp/doc.md is deleted; the few contributor facts that live nowhere else (engine-client seam, credentials-catalog builder gates, widget workspace) move to docs/development/engine/mcp-module.md. Everything else stays in git history. - Pipeline: DOCS_ROOT_MOUNTS split per folder; a README.md at a mount root with no index sibling is now gathered as that mount's page (the blanket export-source skip is otherwise unchanged); export mapping, spine, redirects, footer link, and spine tests updated; stale /protocols/mcp links retargeted to /protocols/mcp/stdio. Verified: docs:build (broken-link gate), docs:check, spine tests, validate-client-docs.py all green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ed audit Every fix traces to a finding confirmed against source by an independent adversarial verification pass (10 audit lanes, 10 verifiers, 10 fix executors; ~252 confirmed fixes applied, 12 refuted findings dropped). Highlights: - SDK docs (TS/Python): remove fictional APIs and behavior (.env auto-parsing, exponential backoff, Answer.parseJson, dapRequest, MIME table), correct signatures/defaults, add missing parameter rows - Examples: pipeline samples now validate (components key, real providers/fields, embedded question lanes for qdrant) - WebSocket protocol pages: correct wire commands, token semantics, response shapes, event names; document the run-log continuum - VS Code docs: migrate to the grouped settings schema, real command surface and workflows - Development docs: builder module table from live --list output, ci-gates matches actual CI, runnable command fixes - Operate/reference: ROCKETRIDE_APIKEY vs ROCKETRIDE_AUTH scoping, CLI table corrections, working tar command - Regenerated client-mcp/client-python/client-typescript READMEs via docs:export Gates: docs:build (broken-link check), docs:check, validate-client-docs. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ls/ (#2125) * docs(agents): land the pipeline-builder skill set in docs/agents/skills/ First occupants of the reserved skills directory: the five hand-written pipeline-builder skills (orchestrator + design/configure/run/debug) plus MCP_TOOL_CONTRACT.md, the frozen tool-name/result-shape contract against the 27-tool HTTP MCP server surface. The skills are weak-model-first: verification by tools (validate_pipeline is the compiler, the run result is the proof) and hard gates (Waiting = STOP, cost approval, 17 forcing functions in GATE_PROTOCOL.md) rather than model intelligence. Bundled L1 node index regenerated 2026-08-25 from a live v3.3.1 engine (124 nodes, matches this branch's README corpus). Deliberately not exported by docs:export (subdirectories are excluded); README documents the layout and the future export hook. Bundled shims conformed to repo ruff style; anti-example keys shortened to obvious placeholders that clear the gitleaks rules. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(agents): apply review fixes to the pipeline-builder skill set Code-verified review fixes on top of the initial drop: - MCP_TOOL_CONTRACT: error envelope is {error_type, message, hint}; add pipelineTraceLevel to run_pipeline/run_dropper_pipe inputs (server default summary); list_integrations detail key is 'setup' - L1 node index reconciled against the branch's node corpus: 167 providers (renamed graph nodes, store tool flags, missing lanes, 45 absent providers added); meta provenance + counts synced - Error strings now match what the code emits verbatim (or are worded as symptom patterns): 'The service <provider> was not found', 'references unknown component id', 'input has unknown lane', 'Connection closed unexpectedly', 'Pipeline is already running.' - ${ROCKETRIDE_*} prefix requirement restored in PIPELINE_ANTIPATTERNS and the rules summary (non-prefixed refs come back <REDACTED>) - Shims: MIT headers, corrupt-cache reads follow the ERROR_JSON contract, fetch-doc docstring drift fixed, cross-skill hint path - New tools/generate-index.py — the index regenerator three docs reference (SDK get_services(), atomic writes, --dry-run) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Krish Garg <krish.garg@rocketride.ai> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: Dylan Savage <dylanmsavage@hotmail.com>
…vamp (#2138) * feat(docs): restyle the docs site to the RocketRide marketing design language Port the marketing/cloud landing look onto the Docusaurus theme: the red-orange star (#F93822) as sole accent over a warm paper field with a dot grid (light) or the Abyss Deep near-black (dark); Ubuntu Mono uppercase kickers; squared surfaces with hairline borders; contrast- inversion hovers on cards and sidebar rows; a hard-orange-offset CTA block; a floating form-fitting dark sidebar panel; and a rocket-mark "you are here" indicator on the page TOC. Rebuild the homepage as a Dify-style hero + numbered card sections using new global Card/CardGrid MDX components. Cascade notes: the build injects Infima/theme rules at (2,x,y) hacked specificity and wraps CSS in cascade layers, so overrides variously match the :not(#\#) hack, exceed it, or pin with !important — each case is documented inline in custom.css. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(docs): make sidebar group kickers readable in the mobile drawer The sidebar menu markup is reused by the mobile navbar drawer, whose surface is light — the white kicker text scoped to the menu vanished there. Kickers now default to the star orange (readable on the drawer in both themes); only the dark desktop panel re-whitens them. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(site): restructure the IA into journey order (phases 1-3) Rebuild the docs spine around the reader journey — Get Started, Concepts, Clients, Guides, Examples, Nodes, Connect, Deploy & Operate, Reference, Support & Community — and make the docs/public/product/ tree, doc ids, and URLs one namespace again. - Move 12 pages, split 4 by audience (performance, error-handling, apps, cli), retire 2 into their canonical homes; dissolve evaluate/, develop/, protocols/, integrations/, ide-extensions/ - Promote the cloud-vs-self-host table to the /operate landing; dedupe the decision prose, lane table, and memory tables to single canonical copies - Re-slot 4 mounts (clients/typescript, clients/python, clients/vscode, connect/mcp); 15 exact + 4 wildcard redirects, legacy targets re-aimed - Add content-pass placeholder slots (guarded by EXPECTED_PLACEHOLDERS): observability, docker, kubernetes, get-help, contributing, release-notes - Fold why-rocketride into the homepage with audited catalog figures (140+ nodes, 15+ LLM providers, 10+ vector stores; census in claude/tasks/docs-ia-restructure) - Update footer, spine/gather/llms tests (48/48), contributor pipeline doc, agents observability mirror pointer, and the pipeline-reference generator Full build passes with zero broken links; docs:check in sync. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(docs): generate /support/release-notes from GitHub releases New docs:release-notes build step between gather and index: fetches the repo's releases from the public GitHub API (token honoured when set), keeps stable releases, and renders a reverse-chronological page with component/version/date bylines. format: md front matter keeps GitHub- flavored release bodies out of the MDX compiler. On fetch failure the gather placeholder ships, so offline builds never break. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(clients): restructure the client docs into code-verified page sets Replace the long, drifted README+index pairs with an industry-standard split: slim package READMEs (~95 parallel lines — install, quickstart, early deferral to the docs site) and 13 journey-ordered site pages per client (configuration, connection, pipelines, deploy, data, storage, chat, logs, errors, reference, examples, analytics + card-grid landing), plus TS-only database-sequelize. Method tables live only in reference.md; typescript/methods/ and the index/log pages are absorbed, with exact redirects for every retired URL. Two audit passes (6 agents before, 10 fresh agents after) verified every claim against the SDK sources: fabricated APIs removed, set_events documented as deprecated in favor of add_monitor/removeMonitor, real return shapes and reconnect/backoff behavior, the inert `public` flag, and ~60 further accuracy fixes. Pipeline: client mounts render as nested sidebar categories (spine mount+nest); validate-client-docs.py rewritten to check README sections, site page-set parity, env-var parity, and reference.md symbol parity; readme-schema.md rewritten as the two-layer contract. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs: port the dd908a4 audit fixes into their restructured successor pages The accuracy audit landed against pages this branch had moved, absorbed, or rewritten; a code-checked sweep ported the fixes that had no home: the app-sdk export surface on the apps reference, lane-table scopes on execution-model, agent_rocketride max_waves, docFilter limit and memory_persistent max_history, the CLI local-engine wording, vscode agent-rule detection, and the TS reference/configuration return shapes. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(mcp): give the HTTP MCP server the client-docs treatment Replace docs/public/mcp/http/README.md with five code-verified pages — overview (card landing), connect (per-client setup, the OAuth bootstrap chain, transport details), tools (all 27 with parameters, return shapes, and the wire contract), resources-and-widgets, and self-hosting — nested in the sidebar as Connect > MCP > HTTP Server. The stdio docs are unchanged; the deleted README had no package to export to (the server is built into the engine). Two extraction audits plus two fresh adversarial verification passes checked every claim against packages/ai/src/ai/modules/mcp/. Notable: the docs now state send_files' actual behavior — paths resolve on the engine host's disk, not the account file store, contradicting the tool's own schema string (code fix tracked separately). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(support): write the FAQ / Get Help page Support channels (Discussions, Stack Overflow, Discord, issue templates, the private security path), a before-you-ask checklist, and fourteen FAQ entries harvested from the docs' own recurring gotchas — each a short answer deep-linking to the page that owns the detail. Code-verified, including the pk_ webhook-Bearer nuance. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(support): write the Contributing router page Industry-standard router (9 of 10 surveyed docs sites carry one): a ways-to-contribute menu where each path gets a paragraph plus a link to its canonical home — issue templates, docs edits, the build-a-node ladder, and code contributions via CONTRIBUTING.md/AGENTS.md on GitHub. States the scope policy: docs and examples PR directly; new nodes, engine core, and protocol changes open an issue first. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(guides): write the Observability guide The unifying page no single surface owned: the DVR mental model (continuum, chapters, beginSeq), live vs recorded planes, the two keying regimes (task token vs projectId+source+teamId), one authoritative trace-level table reconciling the three per-surface defaults (wire none, MCP summary, VS Code full), retention, a which-surface-when map, two worked workflows, and the on_trace naming trap. Execution Model now links to it. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(operate): write the Docker self-hosting page The published ghcr.io/rocketride-org/rocketride-engine image: quickstart docker run (loopback-bound by default), tag policy, amd64-only, keyless cosign verification, configuration via environment, /opt/data persistence, health and upgrades, and a minimal production compose example — with the repo compose file labeled as the dev stack it is. Health checks use /version everywhere (including the self-hosting overview, which previously said /ping): the engine never registers /ping, so it 401s even with no auth configured. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(operate): write the Kubernetes self-hosting page; retire the stub allowlist The deploy/helm/rocketride chart: install from a source checkout (no published registry), the values that matter, the four examples (KEDA one flagged as a kubectl-applied ScaledObject, not a values file), the no-bundled-databases stance, and a probes section that redirects all three probes to /version — the chart's /ping defaults 401 against the shipped engine and recycle healthy pods (chart fix tracked separately). With every stub filled, EXPECTED_PLACEHOLDERS shrinks to the build-time release-notes entry, and the gather test pins the new list. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(support): align the contributing page with the every-PR-links-an-issue policy Root CONTRIBUTING.md requires every PR to link an issue (Fixes #123, CI-enforced), so the page no longer offers an issue-less path for docs and examples — the distinction is now whether to wait for discussion before building, which nodes, engine core, and protocol changes do. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(vscode): document the App Builder The feature existed only as two settings-table rows. New page covers the code-verified working core: the New App wizard (Blank/Dashboard templates, frame options, <publisher>.<name> ids), the Develop view — live preview against the engine shell with the per-user ephemeral dev overlay, the 47-entry component gallery, the merged Console feed, the watch loop and its two settings — and the publish-then-pin deploy model (@user/@team/@org rungs, repoint-never-rebuild). States today's limits plainly: single-file bundles, prompt-based deploy target, Store tab visible but unwired. Cross-linked both ways with the Shell Apps guide. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(site): cut the home hero to a definitional intro The docs home opened with marketing copy: a slogan headline and a feature-checklist paragraph of hand-maintained counts. A docs visitor needs orientation, not persuasion — the hero is now the site name plus the one sentence that carries the model (portable JSON, one engine, runs the same everywhere), straight into the CTAs. Nothing on the page hardcodes catalog counts anymore. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(site): polish card icons, hovered-code contrast, and sidebar labels Three review nits from Dylan's pass: card icon glyphs grow from 19px to ~29px so they hold their own against the bold card titles; inline code in a hovered card body gets a field matched to the inverted card (the rule carries Infima's :not(#\#) ID-level specificity hack — without it the resting light background always wins and the pill goes illegible in light mode); and the top-level sidebar labels drop the uppercase-mono kicker treatment for the same face and size as every other menu row. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Docusaurus hardcodes webpack resolve.symlinks: true, so symlink-staged MDX compiles under its real source path outside the docs plugin's content dir. The plugin's mdx-loader rule then never attaches the doc metadata export and every page crashes at runtime in DocItem with "Cannot read properties of undefined (reading 'id')". Copy mode matches what docs:gather already does for prod builds; source edits now need a docs:dev re-run to show up. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
dylan-savage
requested review from
Rod-Christensen,
jmaionchi,
kwit75 and
stepmikhaylov
as code owners
August 27, 2026 21:57
Contributor
|
Important Review skippedToo many files! This PR contains 591 files, which is 291 over the limit of 300. To get a review, reduce the PR to 300 files or fewer by splitting it into smaller PRs or changing its base branch. Usage-priced reviews support at most 300 files. ⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: ASSERTIVE Plan: Advanced Run ID: ⛔ Files ignored due to path filters (49)
📒 Files selected for processing (591)
You can disable this status message by setting the Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
github-actions
Bot
added
docs
Contributor
🤖 Internal: Discord sync markerAuto-managed by the Discord notification workflow. Stores the linked Discord message ID and forum thread ID. Do not edit or delete. |
nihalnihalani
added a commit
to nihalnihalani/rocketride-server
that referenced
this pull request
Sep 14, 2026
…DME schema The documentation revamp (rocketride-org#2139) deleted docs/develop/README-nodes.md and with it the hand-written node catalog table this PR appended a row to. The catalog is now generated at /nodes from each node's own README.md, and docs/development/nodes/readme-schema.md fixes that README's shape: scripts/validate-node-readme.py --all gates the whole corpus in docs:validate, so a node README that predates the schema fails the build. Restructure this node's README to the schema rather than resurrecting the deleted path: - drop the Docusaurus frontmatter and <head> block (the docs build now owns the page's title and position), and lead with an H1 matching the directory name plus a summary paragraph; - lift the lane table out of "What it does" into its own ## Lanes section; - add the required ## Profiles section — auto, manual and the test-only ci_tiny profile, with the declared default named in the introduction; - replace the hand-written field table under ## Configuration, which duplicated the generated ## Schema table, with per-field usage guidance, and move the algorithm detail ("what is budgeted", window resolution, offline tiktoken) under ## Notes where the schema puts it. The budget-percentage guidance is corrected while it moves: the optimizer scales a set summing over 100 back to 100 and only warns below 90, so the earlier "keep the total at or below 100" advice described a constraint the code does not impose. No behaviour change; documentation only. The generated PARAMS region is carried over untouched. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
nihalnihalani
added a commit
to nihalnihalani/rocketride-server
that referenced
this pull request
Sep 14, 2026
Adds examples/rag-pipeline.eval.json (three cases over the RAG example: deterministic assertions plus one llm_judge case) and lists it in the examples README. The prose half of the original commit — the eval spec reference, the assertion table, the judge notes and the CI recipe — targeted docs/README-python-client.md and packages/docs/content-static/cli.mdx, both of which the documentation revamp (rocketride-org#2139) removed. That content is re-homed into docs/public/product/connect/cli.mdx in a later commit on this branch rather than resurrecting the deleted paths here. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
nihalnihalani
added a commit
to nihalnihalani/rocketride-server
that referenced
this pull request
Sep 14, 2026
Two reporters.py docstrings described the human eval report as mirroring ``rocketride validate`` output, but at the time that subcommand was only proposed in the still-open rocketride-org#1573, so the comparison pointed at a command that did not exist. Rewords both to describe the format directly. The prose half of the original commit edited docs/README-python-client.md and packages/docs/content-static/cli.mdx, which the documentation revamp (rocketride-org#2139) removed; the eval prose is re-homed into docs/public/product/connect/cli.mdx later on this branch instead. rocketride-org#1573 has since merged, so a follow-up commit restores the pairing where it is now accurate. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
nihalnihalani
added a commit
to nihalnihalani/rocketride-server
that referenced
this pull request
Sep 14, 2026
A spec whose run raised (e.g. its pipeline failed to start) was printed to
stderr and then dropped: it produced no EvalReport, so render_json and
render_junit described only the specs that did run. With one spec broken and
another passing, the run exited 1 while its machine-readable reports claimed
zero failures — a CI job could publish a green artifact for a failed run.
Collect those failures as SpecError records (spec path, message, exception
type) alongside the reports and thread them through all three renderers:
- render_json gains a top-level `spec_errors` list of {spec, error} and a
`summary.spec_errors` count. Both are always present, so a consumer can
read them without probing for the key first.
- render_junit renders each one as a one-test <testsuite> holding a single
<testcase> with an <error>, and counts it in the <testsuites> totals.
- render_human lists them and appends ", E spec error(s)" to its summary
line. They still go to stderr as before.
The exit-code contract is unchanged: 0 all passed, 1 any case failed or any
spec could not run, 2 usage/spec/connection error or no case produced a
result. Reports written on the all-specs-errored path (exit 2) now carry the
errors too, and --json still owns stdout.
Also documents that contract accurately: the eval parser description and the
docs said 1 meant only "at least one case failed", and the eval docs listed
--token / ROCKETRIDE_TOKEN, which eval does not register — it takes only
--uri and --apikey.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The doc edits of the original commit landed in docs/README-python-client.md,
packages/client-python/docs/index.md and packages/docs/content-static/cli.mdx,
all three removed by the documentation revamp (rocketride-org#2139); the eval prose is
re-homed into docs/public/product/connect/cli.mdx later on this branch, in
its final form, instead of resurrecting the deleted paths.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
nihalnihalani
added a commit
to nihalnihalani/rocketride-server
that referenced
this pull request
Sep 14, 2026
…e expander `eval` registered `--json` with `action='store_true'`, making it the one subcommand whose `--json` refused a destination file. Every other verb takes the shared `--json [FILE]` (bare = stdout, `--json FILE` = write there), so `rocketride eval evals/*.eval.json --json report.json` was a usage error: argparse rejected the leftover path with "unrecognized arguments" and exit 2, and written flag-first (`--json report.json evals/*.eval.json`) the report path was swallowed by the `files` positional and reported as a missing spec. Either way the run exited 2 with nothing evaluated. Register the shared option on `eval` and route its report through the shared `Output` channel: bare `--json` still owns stdout (human report suppressed, stdout carries exactly one JSON document), and `--json FILE` writes that document to the file while the human report keeps flowing on stdout — the same three-way behavior `validate --json` already has, including creating a missing parent directory. `--json FILE` and `--junit <path>` now compose, so one run can publish both CI artifacts. The exit-code contract is untouched: the code still follows the results (0/1) and never the old usage-error 2. `eval` still does not route through `run_cli_command` — that runner maps any raised error to exit 1, while this command reserves 1 for "a case failed" — so it flushes the Output itself alongside its client teardown. With no caller left, the `json_arg` keyword added to `_add_connection_args()` for this command goes away and the helper is byte-identical to its former shape again. Also lifts `_expand_files()` — duplicated verbatim in `eval.py` and `validate.py` — into a single `expand_file_patterns()` next to `find_files()` in `cli/utils/file_utils.py`, with behavior unchanged. It stays separate from `find_files()` on purpose: it returns paths exactly as typed and keeps an unmatched pattern, both of which a per-file reporting command needs to name a missing file the way the user asked for it. Tests: the parser test that pinned the boolean flag now pins the shared option and the two argument orders that used to break; new cases cover `--json FILE` writing a parseable document, the 0/1 exit codes, spec errors landing in the file, and `--json FILE` with `--junit`; a new unit test covers the shared expander directly. Docs updated in all three hand-maintained CLI pages plus the `--help` text and the module docstring. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> The three hand-maintained CLI pages the original commit updated — docs/README-python-client.md, packages/client-python/docs/index.md and packages/docs/content-static/cli.mdx — were all removed by the documentation revamp (rocketride-org#2139); the eval prose is re-homed into docs/public/product/connect/cli.mdx later on this branch instead. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
nihalnihalani
added a commit
to nihalnihalani/rocketride-server
that referenced
this pull request
Sep 14, 2026
`eval` routes its report through the shared `Output`, but neither path that
exits 2 recorded a payload: a spec that fails to parse returned before the
`Output` existed, and a connection failure returned with it created but
nothing recorded. `finish()` is a no-op when no payload was recorded, so
`--json report.json` wrote no file at all on either path — while every other
subcommand, going through `run_cli_command`, always leaves an
`{"error": {...}}` envelope behind.
The failure that costs someone time is the stale one: run 1 passes and writes
`report.json`, run 2 in the same workspace has the server down and exits 2,
`report.json` is never touched, and the reporting step parses run 1's green
summary as run 2's result.
Open the `Output` before the spec-loading loop and route both paths through
`out.fail()`, which prints the one `Error: <message> — <hint>` sentence on
stderr (replacing the manual prints, so it still appears exactly once) and
records the envelope for `finish()` to flush; the `finally` block already ran
`disconnect_all()` then `finish()`, the order `run_cli_command` uses, and now
covers every return. The exit codes are untouched — both paths still return an
explicit 2, since this command reserves 1 for "a case failed" — as are
`--junit`, stdout purity under bare `--json` (the envelope is the one document
there), and human mode (stderr only, stdout empty).
The connection message keeps its `: <err>` cause rather than being reduced to
a bare "Unable to connect to server": `hint_for()` only matches messages that
literally say "refused" / "getaddrinfo" / "timed out", and the errors this
path actually raises do not — a local refusal arrives as "[Errno 61] Connect
call failed" and an unknown host as "[Errno 8] nodename nor servname
provided", both of which yield an empty hint. Dropping the cause would leave
the user with a sentence naming no reason at all.
Tests: five cases pin the contract — the envelope in `--json FILE` for a spec
parse error and for a connection failure (with its hint), a pre-existing green
`report.json` overwritten by it, bare `--json` carrying exactly one JSON
document on stdout, and human mode leaving stdout empty with a single `Error:`
sentence on stderr. Docs updated in all three hand-maintained CLI pages plus
the `--help` text and the module docstring.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The CLI pages the original commit updated were removed by the documentation
revamp (rocketride-org#2139); the eval prose is re-homed into
docs/public/product/connect/cli.mdx later on this branch instead.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
nihalnihalani
added a commit
to nihalnihalani/rocketride-server
that referenced
this pull request
Sep 14, 2026
…cs tree This branch documented `eval` in three hand-maintained pages that the documentation revamp (rocketride-org#2139) has since removed: docs/README-python-client.md, packages/client-python/docs/index.md, and packages/docs/content-static/cli.mdx (renamed to docs/public/product/connect/cli.mdx). Replaying those hunks would have resurrected deleted paths, so the prose lands once, in the new home. Adds `### eval` to docs/public/product/connect/cli.mdx between `### validate` and `### list` — the two CI-gate commands now sit together — in that page's voice and shape: intro paragraph, one commented ```bash block, a "Key flags:" table using its `<files...>` positional spelling, an "Exit codes:" paragraph in its `0` — … ; `1` — … form, and a closing blockquote note. The spec-format, assertion-type, judge, CI and troubleshooting subsections carry over from the old page unchanged. `eval` is also listed in the Commands table. Two corrections the move made possible: - The parity note claimed both packages ship the same commands; `eval` is Python-only, so the note now says so. - The CI recipe and the gating paragraph pair `eval` with `validate` again. That pairing was removed in "docs(cli): do not reference the unmerged validate subcommand" because rocketride-org#1573 was still open; it merged on 2026-09-08 and `rocketride validate` is documented on this very page, so the structural + behavioral pair is accurate once more and the anchor resolves. No eval section is added under docs/public/python/. That tree deliberately delegates the whole CLI surface to /connect/cli, and scripts/validate-client-docs.py enforces page and symbol parity between the Python and TypeScript folders — a Python-only CLI page there would break it. The old co-located mirror in packages/client-python/docs/index.md is therefore dropped rather than moved. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
nihalnihalani
added a commit
to nihalnihalani/rocketride-server
that referenced
this pull request
Sep 14, 2026
The rag-pipeline.eval.json entry linked to docs/README-python-client.md#rocketride-eval for the spec format and assertion reference. The documentation revamp (rocketride-org#2139) deleted that file, so the link was dead the moment this branch rebased onto develop. Retargets it at docs/public/product/connect/cli.mdx#eval, where the eval reference now lives — the same relative-path style the neighbouring n8n entry uses for its guide link. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
nihalnihalani
added a commit
to nihalnihalani/rocketride-server
that referenced
this pull request
Sep 14, 2026
Syncs feat/RR-1609-otel-bridge onto upstream/develop (94b8d50). The docs revamp (rocketride-org#2139) restructured the documentation tree while this branch was open, so all five conflicts are modify/delete or rename-follow conflicts against paths that no longer exist. Each is resolved by moving the content into its new home, never by resurrecting the old file: - docs/README-python-client.md and packages/client-python/docs/index.md are deleted on develop (the Python client docs now live under docs/public/python/). Their +18-line "OpenTelemetry export" section is dropped from both mirrors; the same material now lives in the CLI reference and in the bridge guide. - docs/develop/README-clients.md -> docs/public/product/clients/index.md. Kept develop's "both SDK packages install the CLI" sentence and appended the OTel bridge note, rewritten as a site route (/clients/python/otel-bridge) instead of a relative repo path. - packages/docs/content-static/cli.mdx -> docs/public/product/connect/cli.mdx. The `### otel` section is re-homed between `### list` and `### store`, matching the shape of the surrounding `### validate` / `### list` sections, and the `otel` row is added to the command table. develop's rewritten "Python vs TypeScript CLI" callout now claims both packages ship the same commands and that continuous monitoring is deliberately out of scope, so the callout is amended rather than reverted: `otel` is named as the one Python-only command, and as an exporter to your own backend rather than an in-terminal monitor. Stale /protocols/websocket/observability links are rewritten to /connect/websocket/observability. - packages/client-python/docs/otel-bridge.md -> docs/public/python/otel-bridge.md (git's rename detection placed it; develop carries no competing version, so there is nothing to reconcile). Frontmatter gains sidebar_position: 13 so the page sorts after Analytics in the nested "Python SDK" sidebar, and the "See also" links are rewritten from the old /develop/python and GitHub blob URLs to /connect/cli, /clients/python and /clients. Merged rather than rebased: the branch already carries a merge commit (1d16eed) that ported the whole otel CLI command from the class-based Command architecture to develop's run_* dispatch. A rebase would replay the pre-port commits and force that port to be redone against every one of them, with no way to verify each intermediate resolution. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
nihalnihalani
added a commit
to nihalnihalani/rocketride-server
that referenced
this pull request
Sep 14, 2026
Every other page under docs/public/python/ has a card in the SDK overview's "Explore the SDK" grid; the newly re-homed otel-bridge page did not, so it was reachable only from the sidebar and the CLI reference. Adds the card last in the grid, matching sidebar_position: 13. Uses LuRadar, which is not already taken by another card in this file and is present in the Lucide set react-icons v5 ships. This replaces the "OpenTelemetry export (`rocketride otel`)" section that used to live in docs/README-python-client.md and packages/client-python/docs/index.md — both deleted by the docs revamp (rocketride-org#2139). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
krishhgg
added a commit
that referenced
this pull request
Sep 14, 2026
Merging develop brought the documentation revamp (#2139) and its node README validator, which rejects any non-canonical `##` heading and any hand-written field table. Re-home the Chroma top_k guidance to fit: - store_chroma: drop the Fields table and the `## Retrieval tuning` section; add a `### Top K` subsection under `## Configuration` in develop's layout. - rerank_cohere: move the `## Wiring` block under `## Notes`. Its example called `top_k: 20` a "higher" pool, but the data-lane default is 25, so it narrowed retrieval. Recommend 50 and say plainly that only Chroma exposes Top K. - core: add `vector.top_k` to the hand-written common-vector table and regenerate the generated block via nodes:docs-generate. The regeneration also emits the gcp and microsoft tables, which were added to the catalogue after the block was last generated. Verified: validate-node-readme.py --all passes; docs:test, docs:build and docs:check pass; store_chroma 114 passed; all store_* suites 188 passed, 1 pre-existing skip. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
5 of 7 tasks
madhumitha-chandrasekaran-1
pushed a commit
to madhumitha-chandrasekaran-1/rocketride-server
that referenced
this pull request
Sep 14, 2026
… paths Rebased onto develop (was 38 commits behind, and the doc restructuring in rocketride-org#2139 relocated/removed the paths this PR previously edited). Reapplied as a fresh commit rather than a replay, per kgarg2468's review on PR rocketride-org#1866: - docs/agents/ROCKETRIDE_{python,typescript}_API.md (root) no longer exist on develop -- rocketride-org#2139 moved the shipped copies to docs/agents/context/, which is what client-docs:agent packs into docs.zip (see docs/agents/README.md). The previous commit's edits landed on the now-deleted root paths, leaving the actually-served context/ copies with the original wrong '_trace' claim. Fixed at the context/ paths instead; the root paths are gone, nothing to reconcile them to. - packages/client-typescript/docs/guide/methods/use.md was deleted wholesale by rocketride-org#2139 (the whole docs/guide/ tree, superseded by docs/public/ and the generated docs/reference/); dropped that edit, there is no live equivalent carrying the same wrong claim. - Corrected the substance, not just relocated the old text: task_engine.py (develop, post-#2xxx) now guards derived apaevt_flow / analytics rollup with `self._pipelineTraceLevel != 'none'` (previously truthy-only), so explicit 'none' and omission are equivalent -- neither produces flow events or analytics. The old wording ("'none' only suppresses lane-write/invoke traces; tool-call traces are still captured") is now false and was corrected accordingly wherever this doc block appears: docs/agents/context/ROCKETRIDE_python_API.md, docs/agents/context/ROCKETRIDE_typescript_API.md, and both pipelineTraceLevel doc comments in client.ts (the @PARAM block and the inline options-type comment). - Left packages/client-typescript/contract/versions/*.d.ts and packages/shell/contract/versions/*.d.ts untouched -- generated frozen SDK compatibility snapshots (`./builder client-typescript:freeze`), not live documentation. Verified every referenced symbol still resolves on current develop (TASK_STATUS, getTaskStatus/get_task_status, LogEventStream, LogApi, open_event_stream/openEventStream, get_traces/getTraces n<=50 via _MAX_TRACES). tsc --noEmit clean other than the pre-existing, unrelated 'ignore' module errors in app-pack/index.ts. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
madhumitha-chandrasekaran-1
pushed a commit
to madhumitha-chandrasekaran-1/rocketride-server
that referenced
this pull request
Sep 14, 2026
… paths Rebased onto develop (was 38 commits behind, and the doc restructuring in rocketride-org#2139 relocated/removed the paths this PR previously edited). Reapplied as a fresh commit rather than a replay, per kgarg2468's review on PR rocketride-org#1866: - docs/agents/ROCKETRIDE_{python,typescript}_API.md (root) no longer exist on develop -- rocketride-org#2139 moved the shipped copies to docs/agents/context/, which is what client-docs:agent packs into docs.zip (see docs/agents/README.md). The previous commit's edits landed on the now-deleted root paths, leaving the actually-served context/ copies with the original wrong '_trace' claim. Fixed at the context/ paths instead; the root paths are gone, nothing to reconcile them to. - packages/client-typescript/docs/guide/methods/use.md was deleted wholesale by rocketride-org#2139 (the whole docs/guide/ tree, superseded by docs/public/ and the generated docs/reference/); dropped that edit, there is no live equivalent carrying the same wrong claim. - Corrected the substance, not just relocated the old text: task_engine.py (develop, post-#2xxx) now guards derived apaevt_flow / analytics rollup with `self._pipelineTraceLevel != 'none'` (previously truthy-only), so explicit 'none' and omission are equivalent -- neither produces flow events or analytics. The old wording ("'none' only suppresses lane-write/invoke traces; tool-call traces are still captured") is now false and was corrected accordingly wherever this doc block appears: docs/agents/context/ROCKETRIDE_python_API.md, docs/agents/context/ROCKETRIDE_typescript_API.md, and both pipelineTraceLevel doc comments in client.ts (the @PARAM block and the inline options-type comment). - Left packages/client-typescript/contract/versions/*.d.ts and packages/shell/contract/versions/*.d.ts untouched -- generated frozen SDK compatibility snapshots (`./builder client-typescript:freeze`), not live documentation. Verified every referenced symbol still resolves on current develop (TASK_STATUS, getTaskStatus/get_task_status, LogEventStream, LogApi, open_event_stream/openEventStream, get_traces/getTraces n<=50 via _MAX_TRACES). tsc --noEmit clean other than the pre-existing, unrelated 'ignore' module errors in app-pack/index.ts.
madhumitha-chandrasekaran-1
pushed a commit
to madhumitha-chandrasekaran-1/rocketride-server
that referenced
this pull request
Sep 14, 2026
The "Docs site build" CI check (node scripts/build.js docs:test) started failing after merging develop: the docs revamp (rocketride-org#2139) added docs/development/nodes/readme-schema.md and its validator (scripts/validate-node-readme.py) as a blocking CI gate, and this README predates that schema. python3 scripts/validate-node-readme.py nodes/src/nodes/cloud_stt reported: x cloud_stt: FAIL (7 failed, 0 warnings, 6 passed) FAIL: H1 equals directory name -- 'Cloud Speech To Text (`cloud_stt`)' != 'cloud_stt' FAIL: unknown heading '## Limits' -- move under ## Notes FAIL: unknown heading '## Code layout' -- move under ## Notes FAIL: unknown heading '## Adding a vendor' -- move under ## Notes FAIL: unknown heading '## Related nodes' -- move under ## Notes FAIL: section order -- got: What it does, Configuration, Lanes, Limits, Code layout, Adding a vendor, Related nodes FAIL: Lanes rows match declared lanes -- missing lane-in: ['audio', 'video'] Fixes, each mapped straight to a FAIL line above: - H1 -> plain `# cloud_stt` (directory name, per section 0 of the schema). - `## Limits`, `## Code layout`, `## Adding a vendor`, `## Related nodes` are node-specific free-form content with no matching schema section -- moved under `## Notes` as `###` subsections, the schema's designated home for exactly this. - Reordered the remaining CORE/CONDITIONAL sections to the schema's fixed order: What it does (2) -> Lanes (4) -> Configuration (7) -> Notes (11). Lanes sorts before Configuration; it was previously reversed. - `## Lanes` was prose ("`audio`, `video` -> `text`.") instead of the required `| Lane in | Lane out | Description |` table, so the validator couldn't find matching rows for either declared lane (`lanes: {audio: [text], video: [text]}` in services.json) -- replaced with a proper table, one row per declared lane. Verified: `python3 scripts/validate-node-readme.py nodes/src/nodes/cloud_stt` -> `ok cloud_stt: PASS (0 failed, 0 warnings, 9 passed)`. Re-ran the validator against every other node README in the corpus (none regressed -- this was the only failure) and the full `node scripts/build.js docs:test` the CI job actually runs, which now completes clean end-to-end.
asclearuc
pushed a commit
that referenced
this pull request
Sep 15, 2026
…ement (continues #527) (#1489) * feat(nodes): add context_optimizer token-budget node RAG pipelines push a system prompt, the user question, retrieved documents and chat history into an LLM node with no accounting of the model's context window. When the total exceeds the limit the provider rejects the request or silently truncates, and no node in the catalog trims content by token budget before the LLM call. context_optimizer is a questions -> questions filter that counts tokens with tiktoken, allocates a per-component budget (system prompt / query / documents / history, configurable percentages), truncates on sentence boundaries, keeps the highest-scoring documents that fit, and compacts the middle of the conversation history. Requests that already fit pass through unchanged and no extra model call is made. Context-window sizes are resolved from the live model catalog first -- the modelTotalTokens values on the preconfig profiles of the sibling llm_* nodes, which tools/sync_models keeps in step with the providers -- so ids such as gpt-5.4, claude-sonnet-4-6 and models/gemini-3.1-pro-preview resolve without this node hard-coding them. MODEL_LIMITS remains as a fallback for deployments without the llm_* nodes and for the abbreviated family aliases no provider publishes; it was refreshed against that catalog. This answers the open MODEL_LIMITS note from #527 without reaching for IInvokeLLM.GetContextLength(), which is only available after beginFilter. classType is "text" so the docs sidebar files the node under Text rather than the unmapped "transform" fallback category (packages/docs/scripts/lib/ gather.js NODE_CATEGORIES). util-text.svg is co-located because the engine resolves icons relative to services.json since 488d076 and otherwise logs "Icon : **** MISSING" and renders the unknown glyph. The unit suite guards that fallback: one test asserts MODEL_LIMITS and the live catalog never state a different window for an id they both carry, so a stale table entry fails loudly instead of changing the budget depending on whether the llm_* nodes are deployed. The catalog discovery test asserts structure -- non-empty mapping, positive integer windows, no discovered provider silently dropped -- rather than naming model ids that tools/ sync_models is free to rename. The services.json test block carries a comment noting that tiktoken downloads its BPE vocabulary on first use and that TIKTOKEN_CACHE_DIR pre-seeds it. Continues #527. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs(nodes): list context_optimizer in the node catalog and changelog Every node PR merged since July adds a row to the docs/README-nodes.md catalog and a line under CHANGELOG.md [Unreleased] / Added; do the same for context_optimizer. The hand-maintained "121 node directories -> 151 services" sentence and the lane-ontology produced/consumed counts are left untouched: both are already stale on develop (137 directories / 183 services*.json today), so bumping them by one would propagate a wrong number and add conflict surface with Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(nodes): keep every question entry and hold every context_optimizer budget Addresses the six CodeRabbit threads on the rebased head of #1489. - IInstance.py:102 (Data Integrity, Major) -- FIXED. writeQuestions no longer collapses question.questions to its first entry. Every QuestionText is kept and rewritten in place; when the merged query is truncated the entries share the query allowance, each keeping a slice proportional to its own size via the new ContextOptimizer.truncate_each_to_budget. No other filter node on develop destroys the list (rerank_cohere/IInstance.py:55 and store_elasticsearch/IInstance.py:52 read questions[0] but forward it intact, guardrails/IInstance.py:58 joins every entry and forwards unchanged), while embedding_transformer/sentenceTransformer.py:135 writes an embedding per entry -- so the drop was losing real downstream data. - optimizer.py:334 (Functional Correctness, Minor) -- FIXED. resolve_model_limit builds one candidate list (full id, then the bare name for a scoped id) and uses it for both the catalog and MODEL_LIMITS. With an empty catalog, 'openai/gpt-5' resolved to DEFAULT_MODEL_LIMIT (128000) instead of 400000 and warned; it now resolves through the table, which is what the table is for. - optimizer.py:357 (Stability, Minor) -- FIXED. _parse_pct rejects NaN, which satisfied neither range check and then raised "ValueError: cannot convert float NaN to integer" inside allocate_budget, failing the whole optimize() call. inf/-inf keep clamping as before. - optimizer.py:541 (Functional Correctness, Minor) -- FIXED. truncate_to_budget re-measures the joined sentences before returning them. The per-sentence tally excludes the joining spaces, so the result could exceed max_tokens: measured, 'Alpha beta gamma. 12345 67890.' costs 11 cl100k tokens against a 10-token budget. - optimizer.py:588 (Functional Correctness, Minor) -- FIXED. summarize_history reserves the placeholder's widest wording (the omitted count is substituted only after the budget is fixed, and costs 5 more tokens than the bare marker), so the returned list stays inside max_tokens -- measured, 12 messages against a 25-token budget returned 29. The adjacent single-message path had the same defect (it ignored the role overhead the docstring promises to reserve) and is fixed the same way. - optimizer.py:762 (Refactor, Major) -- FIXED. optimize() derives the encoding from effective_model and threads it through every count, so a model override can no longer apply one model's limit with another model's tokenizer. The encoding is reported in metadata. 26 new tests; each fix was mutation-checked by reverting it and confirming the matching test fails. Suite: 124 passed under real tiktoken 0.14.0 and under the suite's whitespace stub. The three ci_tiny dynamic cases in services.json were replayed and produce identical results, so no expectation changed. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(nodes): return an empty history when the budget cannot cover role overhead `ContextOptimizer.summarize_history` promises never to exceed `max_tokens`, but two branches could still overshoot. Every returned message costs `count_tokens(role) + 4` before a single content token is spent, so when `max_tokens` is below that floor even an empty-content message breaks the contract: the `len(messages) == 1` branch and the `budget_for_recent <= 0` fallback both computed a negative `content_budget`, got `''` back from `truncate_to_budget`, and returned a ~5-token message anyway. Guard both branches with `if content_budget < 0: return []` and state the empty-list case in the docstring. A budget exactly equal to the overhead still returns one empty-content message, which costs exactly the budget. Adds `TestHistorySummarization::test_budget_below_role_overhead_returns_empty`, covering both branches across every budget below the floor plus the boundary at the floor itself. Mutation-checked by disabling the guard, which fails the new test. Reported by CodeRabbit as an outside-diff comment on 2026-07-08. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(nodes): tokenize provider-scoped model ids with the right encoding resolve_model_limit() already falls back from a provider-scoped id to its bare name, so 'openai/gpt-5' resolves the 400k gpt-5 context window. encoding_name_for_model() did not strip that prefix: the _MODEL_ENCODING_OVERRIDES lookup missed, tiktoken.encoding_for_model() raised KeyError for the scoped name and the count silently fell back to cl100k_base. The gpt-5 window was therefore measured with the gpt-4 tokenizer -- on Japanese text that overcounts by ~21% (510 vs 420 tokens for the same string), so the node truncates content it did not need to. encoding_name_for_model() now builds the same (full, bare) candidate tuple resolve_model_limit() uses and tries the override table and then tiktoken against each. Unscoped ids produce a one-element tuple and are byte-for-byte unaffected. Also validate model_name at the config boundary. Config.getNodeConfig() merges profile defaults into the raw pipeline config without type checking, so a hand-written "model_name": null reached rsplit() and raised AttributeError inside beginGlobal, taking the node down before it served a single record. Non-strings and blank strings now fall back to the documented DEFAULT_MODEL_NAME with a warning, matching how max_context_tokens and the budget percentages are already handled. Adds TestScopedIdEncoding (15 cases) and TestModelNameValidation (17 cases); the suite goes 125 -> 157. Both were mutation-checked by reverting each fix in isolation: 6 and 12 failures respectively. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(nodes): context_optimizer reads the model catalog through the engine registry and budgets unknown models conservatively The catalog was discovered by globbing llm_*/services*.json relative to __file__, which only works while the node sits inside the nodes tree: a node installed into a user's store or materialised into --node_path has no siblings, the catalog came back empty, and an unknown model fell to the optimistic 128 000-token default -- four times the real window of a 32k model, so the provider rejected the request. - Read the catalog through the engine's own registry: getServiceDefinitions() for the index, getServiceDefinition() for each raw definition, keeping every definition that publishes preconfig.profiles[*].modelTotalTokens (chat, vision, embedding and search nodes alike). No filesystem access, no local JSONC parser, layout-independent. - Drop the class-level cache: the registry is re-read at every pipeline start and each optimizer keeps its own snapshot, so a reloaded registry is never masked by a stale one. - Replace the 128 000 default with a conservative fallback: the smallest window either the catalog or MODEL_LIMITS publishes (CONSERVATIVE_MODEL_LIMIT_FLOOR only if both are empty). Erring low costs unused context; erring high costs the request. Tests drive the production registry path through substituted rocketlib functions: real source-tree definitions, synthetic registries, unusable indexes, per-type failures, refresh without restart, and the no-filesystem guarantee. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(nodes): count the un-budgeted prompt sections in context_optimizer `optimize()` totalled only the four budgeted components (system prompt, question, documents, history), but `Question.getPrompt()` also renders `instructions` (where the `prompt` node appends), `examples`, `context` (where retrieval lands), `goals`, and a skeleton of `###` headers, `Document N) Content:` prefixes and CRLF joins. A question whose four fields fit was therefore waved through pass 1 while the rendered prompt was already over the window -- the exact failure this node exists to prevent. `IInstance` now measures that part from the real rendering -- `getPrompt()` on a copy with the four budgeted fields emptied, counted with the node's own tokenizer -- and hands it to `optimize(overhead_tokens=...)`, which adds it to the pass-1 total and allocates the per-component budgets from `total_limit - overhead_tokens` instead of from the whole window. When the overhead alone fills the window every budget is 0, all four components are emptied and the node warns once, mirroring `summarize_history`'s refusal to overshoot a budget that cannot even cover a message's role overhead. README.md now states which fields are budgeted and which are only counted. Also: - Pin `tiktoken>=0.7.0,<1.0.0`, matching `chunker`. The node relies on `get_encoding()`, on `encoding_for_model()` raising `KeyError`, and on `Encoding.name`, none of which a 1.x major is bound by. - Read `max_context_tokens` with the shared `ai.common.utils.config_int` rather than a node-local try/except with identical semantics. The local parser's warning on a non-numeric value is dropped rather than pushed into `config_int`, which its other adopters rely on being silent. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(nodes): recalibrate the context_optimizer ci_tiny profile for the measured overhead The CI-only `ci_tiny` profile was tuned for the old accounting: an 80-token window that an 81-token question just failed to fit, and a history budget that just held the summary placeholder. With the section markup counted (27 tokens for the history case) the placeholder no longer fit, and `test_dynamic.py::TestDynamicNodes::test_node_cases[context_optimizer:services:test2:ci_tiny]` failed on `history[1]`. The profile is now 112 tokens and the over-budget question is 160 words. Every case still exercises its truncation path and passes with margin under real cl100k tokens: the question is truncated to 125 characters (limit 160), the history budget is 29 tokens and keeps the summary placeholder, and the oversized document is still excluded (the cutover is at a 140-token window). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(nodes): re-home the context_optimizer docs onto the new node README schema The documentation revamp (#2139) deleted docs/develop/README-nodes.md and with it the hand-written node catalog table this PR appended a row to. The catalog is now generated at /nodes from each node's own README.md, and docs/development/nodes/readme-schema.md fixes that README's shape: scripts/validate-node-readme.py --all gates the whole corpus in docs:validate, so a node README that predates the schema fails the build. Restructure this node's README to the schema rather than resurrecting the deleted path: - drop the Docusaurus frontmatter and <head> block (the docs build now owns the page's title and position), and lead with an H1 matching the directory name plus a summary paragraph; - lift the lane table out of "What it does" into its own ## Lanes section; - add the required ## Profiles section — auto, manual and the test-only ci_tiny profile, with the declared default named in the introduction; - replace the hand-written field table under ## Configuration, which duplicated the generated ## Schema table, with per-field usage guidance, and move the algorithm detail ("what is budgeted", window resolution, offline tiktoken) under ## Notes where the schema puts it. The budget-percentage guidance is corrected while it moves: the optimizer scales a set summing over 100 back to 100 and only warns below 90, so the earlier "keep the total at or below 100" advice described a constraint the code does not impose. No behaviour change; documentation only. The generated PARAMS region is carried over untouched. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: nihalnihalani <24360630+nihalnihalani@users.noreply.github.com> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
asclearuc
pushed a commit
that referenced
this pull request
Sep 15, 2026
…mented ingester protocol (zero engine changes) (#1612) * feat(sdk): rocketride.otelbridge — OpenTelemetry export over the documented ingester protocol (#1609) Maps the engine's monitor event stream to standard OTel telemetry with zero engine/server changes — the client-side refinement invited by tracking issue #616 after the engine-invasive #498 was archived. - mapper.py: stateful FlowSpanMapper reconstructs span trees from apaevt_flow — task root span, pipe span per pipe id, component child spans paired by component identity (per the observability doc's warning against stack-position pairing); trace.error -> span status ERROR + exception event; apaevt_sse -> span events; unclosed spans at end/shutdown flagged rocketride.span.unclosed=true; per-pipe state is bounded. MetricsMapper maps apaevt_status_update snapshots to OTel instruments (object counts, rates, cpu/gpu memory + peaks). - Semantic conventions per the July 2026 semantic-conventions-genai snapshot: gen_ai.provider.name (not the deprecated gen_ai.system), gen_ai.operation.name from the well-known list only, span names '{operation} {model}' style, INTERNAL span kind for in-process component executions. - Privacy by default: lane payloads / prompt / result content never reach attributes or events unless include_content is set (8KB truncation cap when it is); enforced by a sentinel-scan test across all exported attributes and events. - setup.py: TracerProvider/MeterProvider + OTLP http/protobuf exporters; ALL opentelemetry imports are lazy so the base package imports without the extra. 53 unit tests (InMemorySpanExporter/InMemoryMetricReader; importorskip so the base CI matrix stays green), including a full replay of the captured real-engine event fixture. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(cli): add 'rocketride otel' subcommand + optional [otel] extra (#1609) rocketride otel [--endpoint URL] [--protocol http|grpc] [--service-name] [--headers k=v,...] [--include-content] [--no-metrics] + the standard connection args. Subscribes via rrext_monitor with token '*' (the observability doc's recommended ingestion scope, types TASK|SUMMARY| FLOW|SSE), dispatches to the mappers, reconnects with capped backoff and resubscribes (subscriptions are per-connection and not durable), and flushes exporters on SIGINT/SIGTERM. Endpoint resolution honors the standard OTLP env contract: explicit --endpoint > OTEL_EXPORTER_OTLP_TRACES_ENDPOINT (verbatim) > OTEL_EXPORTER_OTLP_ENDPOINT (base URL + /v1/traces|/v1/metrics) > default. Packaging: opentelemetry-sdk + OTLP http exporter live behind an optional extra — the base install gains zero dependencies, and running the subcommand without the extra exits 2 with the exact "pip install 'rocketride[otel]'" hint before any connection attempt. 25 tests: config precedence matrix, dispatch/reconnect/resubscribe with a fake client, shutdown flush ordering, missing-extra exit path. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs: OpenTelemetry bridge guide with Jaeger, Langfuse, and Datadog recipes Setup guide covering the span model (task -> pipe -> component tree), the privacy default and --include-content, the FLOW-requires-trace-level callout, reconnect semantics, and copy-paste recipes with exact OTLP endpoints/auth for Jaeger all-in-one, Langfuse (/api/public/otel, Basic auth), and Datadog. Notes that Jaeger does not ingest OTLP metrics — pair it with --no-metrics or point metrics at a collector. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(sdk): handle partial OpenTelemetry installs — guard the HTTP exporter import CI exposed a real environment our matrix missed: opentelemetry-api/sdk present as another package's transitive dependency (tool_daytona pulls them in) while the OTLP HTTP exporter is absent. Two fixes: - setup.py: the http trace/metric exporter imports are now guarded like the grpc ones — a partial install raises OtelNotInstalledError with the pip install 'rocketride[otel]' hint (CLI exit 2) instead of a raw ModuleNotFoundError. Verified live in a reproduced partial venv. - test_otel_setup.py: importorskip targets the most specific module the tests exercise (opentelemetry.exporter.otlp.proto.http), so the module skips cleanly in CI's partial environment; plus a regression test that simulates the partial install via a None sys.modules entry and asserts the friendly error. Partial venv: 63 passed + 1 module skip; full extra: 79 passed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(sdk): OTel bridge review round — privacy-gate errors, run-id promotion, header precedence, shutdown ordering CodeRabbit review round on #1612, each finding verified before acting: - mapper: trace.error no longer bypasses the privacy gate — by default only the error signal exports (status ERROR, bare exception event, generic description); --include-content restores verbatim text, now subject to the same 8KB truncation strings previously skipped. The sentinel privacy test plants the sentinel in trace.error. - mapper: a run first seen via flow events (before the task 'running' snapshot) is now promoted from its project|source fallback key to the canonical run id when it arrives — previously a second run state was created (broken parentage) and the orphaned fallback state leaked permanently. Confirmed with a failing repro; guard test ensures two distinct canonical runs sharing (project, source) never merge. - config/setup: only --headers become explicit exporter headers; env headers are left to the SDK so OTEL_EXPORTER_OTLP_TRACES_HEADERS / _METRICS_HEADERS signal precedence works (it was silently defeated). Precedence matrix tests added. - setup: meter provider shutdown wrapped in finally so metrics flush even when tracer shutdown raises. - CLI: --trace-level none message now says flow tracing stays disabled; docs recipes pass collector auth via OTEL_EXPORTER_OTLP_HEADERS env instead of argv (shell-history hygiene); error-mapping doc wording matches the implementation (component leave events). Full extra: 90 passed. Partial install (api/sdk without exporter): 69 passed + clean skip. ruff clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(sdk): absorb stranded fallback run state when canonical state precedes it Follow-on to the run-promotion fix: an SSE event creates the canonical run state WITHOUT registering the (project, source) alias (SSE bodies carry no project/source per the protocol), so an id-less flow for the same run could still open a parallel fallback-keyed state — and because the canonical state already existed, the promotion branch was skipped, stranding the fallback state (kept alive forever by snapshot reconciliation) with duplicate spans. _get_run now reconciles whenever an event carries both the canonical id and project/source: no canonical state → promote (unchanged); canonical state exists → _absorb_fallback_run migrates non-colliding pipes (open spans re-stamped with the canonical run id), closes colliding pipes as rocketride.span.unclosed=true, and drops the fallback key. Only fallback-format keys are touched, so distinct canonical runs never merge (guard test unchanged and green). Both new regression tests failed before the fix. Full extra: 92 passed; partial install: 71 passed + expected skip; ruff clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(sdk): stamp OTel spans with the engine's body.eventTime The bridge was written against a monitor protocol that carried no event timestamps, so every span used the bridge's own clock at event arrival and the guide documented that as a limitation. Since #1661 the engine stamps `body.eventTime` (epoch seconds, set once at ingress by the run-log continuum) on every forwarded event, and derived flow events inherit their source event's stamp — so the limitation is no longer true and the documentation shipped with it was wrong against develop. Span start/end and span-event timestamps now come from `body.eventTime`, making durations engine-measured instead of bridge-observed: WebSocket transport latency and event-loop scheduling no longer inflate every component span. The fallback is exactly the previous behaviour. A body with no usable `eventTime` (an older engine, or a non-float/NaN/non-positive value) and every span closed outside an event — shutdown, seeded-snapshot reconciliation, tracked-run cap eviction — use `time.time_ns()`, the same clock the SDK would have used. Because a run can therefore mix both sources, each open span records its own start and a close is clamped to never precede it, so no span can report a negative duration. Metric points are unchanged: OpenTelemetry metric instruments take no explicit timestamp, so they remain arrival-timed. That is now stated rather than left implicit. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs(client-python): publish the OTel bridge guide and mirror its CLI entry Two co-location gaps (AGENTS.md:5-28) and one stale claim. The 369-line guide lived at docs/README-otel-bridge.md, and root docs/ is not in the docs-site glob (packages/docs/scripts/lib/gather.js:25), so the guide was GitHub-only. Moved to packages/client-python/docs/otel-bridge.md, the tree already mounted at develop/python (packages/client-python/scripts/ tasks.js:236) alongside analytics.md and log.md, so it is gathered to the site at /develop/python/otel-bridge. Its in-repo links become site routes to match the sibling pages; the three references to it were repointed. packages/client-python/docs/index.md is the site's Python page and mirrors the PyPI README's ## CLI block line for line, but had no `rocketride otel` row and none of the OpenTelemetry paragraph the README gained. Added both. The guide's ## Limitations still said span timestamps are arrival time because the wire carries none; the previous commit made spans engine-timed, so that bullet now describes what actually remains bridge-timed (spans closed outside an event, pre-continuum engines, and all metric points). The Python blocks in the guide are also reformatted: ruff formats Markdown code blocks as of 0.16, and CI's ruff-action is unpinned, so the file failed `ruff format --check` today at either path. Repo-wide check is clean again. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs(changelog): record the rocketride otel bridge under [Unreleased] CHANGELOG.md [Unreleased] ### Added is the convention for user-visible additions and is kept alphabetical by scope; a new CLI command and a new optional extra qualify. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs(client-python): name Grafana Tempo in the OTel backend recipes Issue #1609's acceptance list says "Jaeger/Tempo", and the guide covered Tempo only implicitly as "Grafana stacks". Tempo is the OTLP receiver a Grafana user actually points the bridge at, and it needs one non-obvious step (enable distributor.receivers.otlp) plus a separate metrics backend, so it is worth naming rather than leaving a reader to infer. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(sdk): address CodeRabbit review round on the rebased head Nine threads on the rebased head, all in the OTel bridge. Dispositions: - docs/otel-bridge.md:151 (Sensitive Data Exposure, CWE-522) — FIXED. The Langfuse and LangSmith recipes now prompt for the keys (printf + read -rs, spelled identically in bash and zsh) instead of embedding placeholders that operators replace with live secrets in a command line, and point at a secret manager as the better source. The base64 pipeline is now `printf '%s' ... | base64 | tr -d '\r\n'`: real Langfuse keys make the joined string 85 characters, past GNU base64's 76-character wrap, and command substitution strips only trailing newlines, so the embedded break would have produced an invalid Authorization header. - cli/commands/otel.py:207 (Sensitive Data Exposure, CWE-532) — FIXED. The startup line ran `config.endpoint` straight to stdout; an endpoint can carry userinfo credentials or a signed query string. It is now always passed through `redact_endpoint()`, which drops userinfo, query and fragment while keeping scheme/host/port and the ingest path operators actually verify. - cli/main.py:466, otelbridge/config.py:177, otelbridge/setup.py:123 (Cleartext Transmission, CWE-319) — FIXED, once, in one place. `validate_transport_security()` runs before any exporter exists (in `build_providers`, and again in the CLI so the failure is exit code 2 rather than a traceback): a credential-looking header name from `--headers` or any of the three OTEL_EXPORTER_OTLP_*HEADERS variables, plus a non-loopback cleartext endpoint for either signal, aborts startup. Header VALUES are never read or echoed — only names reach the message. Loopback stays exempt for local collectors; `--insecure` / ROCKETRIDE_OTEL_ALLOW_INSECURE=1 is the explicit opt-out and warns on stderr. - otelbridge/config.py:189 (CWE-201, redirect header forwarding) — FIXED. The OTLP/HTTP exporters now get a session that never follows redirects. `requests` strips Authorization only when the redirect changes host, so a 3xx replayed x-api-key to the target and even kept Authorization on an https -> http downgrade to the same host. - otelbridge/config.py:177 (endpoint precedence) — FIXED. `from_args_env` no longer pre-reads OTEL_EXPORTER_OTLP_ENDPOINT into an explicit exporter endpoint: doing so overrode the signal-specific _TRACES_ENDPOINT / _METRICS_ENDPOINT, inverting the OTLP spec order — the same defect already fixed for the header variables in c23b312. The SDK now resolves all three, and `effective_endpoint()` mirrors its order read-only for the startup line and the security check. - otelbridge/bridge.py:207 (signal ownership) — FIXED. `loop.add_signal_handler` replaces rather than chains, and asyncio exposes no way to restore the displaced callback, so the bridge no longer touches a signal the running loop already handles; `install_signal_handlers=False` turns registration off entirely for embedders. - otelbridge/mapper.py:611 (unbounded run.pipes) — FIXED. MAX_TRACKED_RUNS did not bound pipes: a run that keeps receiving events is re-inserted as most-recently-eventful and never evicted, while every begin (or event for a never-begun pipe) adds a _PipeState that only a matching end removes, and a reconnect does not replay missed ends. MAX_TRACKED_PIPES_PER_RUN now closes the least-recently-eventful pipe as unclosed, with recency updated on pipe activity rather than inferred from insertion order. - otelbridge/setup.py:202 (tracer leak) — FIXED. A failure anywhere in the metric half left the BatchSpanProcessor worker running with no handle to stop it, because `shutdown` had not been created yet. That path now shuts the tracer provider down before re-raising. Tests: 32 added (131 in tests/test_otel_*.py, was 99), covering stdout redaction, the refusal and its opt-out, env-var and signal-specific endpoint coverage, the non-redirecting session, the tracer teardown, signal ownership and the pipe cap's LRU behaviour. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * test(sdk): skip the loop signal-handler ownership test on Windows asyncio's ProactorEventLoop raises NotImplementedError from add_signal_handler, so the test cannot set up its precondition there. The bridge itself already tolerates that platform (the other signal tests pass on Windows); only the test needs the platform guard. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * docs(cli): document the otel --insecure flag in the site CLI reference The `otel` command grew an `--insecure` opt-out for the cleartext-credential guard, but the docs-site "Key flags" table was never updated to match. Users hitting exit code 2 from `rocketride otel --endpoint http://tempo:4318 --headers Authorization=...` had no way to learn why from the CLI reference — the flag was only documented in the client-python OTel bridge guide. Add the missing row, naming the `ROCKETRIDE_OTEL_ALLOW_INSECURE` env var alongside the flag so the site table matches otel-bridge.md and the argparse help text. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(sdk): declare the otel test dependency and stop orphaning bridge providers Two review findings on the `rocketride otel` bridge. 1. The opentelemetry dependency of the bridge tests was inherited, not declared. tests/test_otel_mapper.py and tests/test_otel_setup.py opened with pytest.importorskip('opentelemetry'), and nothing in the test path installs the 'otel' extra — the packages reached the engine interpreter only as a transitive dependency of the crewai node. Dropping or repinning that node would have turned ~1450 lines of mapper/setup coverage into a silent skip with CI still green. CI never pip-installs the package (pytest runs on the engine interpreter importing the package from the source checkout via tests/conftest.py), so the extras alone cannot fix this. The deps are added to packages/server/build-requirements.txt, which server:setup-test-deps installs through depends() under the merged constraints — the one install step that path goes through — and the 'dev'/'test' extras now pull 'rocketride[otel]' so a source checkout matches. Both importorskip guards become hard imports: a missing dependency now fails collection instead of skipping. 2. build_providers() always built both provider halves and adopted its shutdown only when shutdown_fn was None, so an embedder passing its own shutdown_fn left a TracerProvider and its BatchSpanProcessor worker thread with nothing to stop them — and the half the caller had already supplied via a factory was built, wired and discarded. build_providers() takes with_tracer / with_meter (defaulting to the old behaviour) and run_bridge asks only for the half it is missing; a caller-supplied shutdown_fn is now chained in front of the providers' shutdown — caller first, providers in a finally — instead of replacing it. The CLI path passes neither argument and is unaffected. Regression tests cover chain order, the provider shutdown surviving a raising caller, and which half is built for each factory combination; test_otel_bridge.py stays runnable without the extra. The embedded-use ownership rules are documented in the bridge doc. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * build(client-python): install the otel test deps in the test step, not the engine Review follow-up: the previous commit declared opentelemetry-sdk and opentelemetry-exporter-otlp-proto-http in packages/server/build-requirements.txt. That file is installed by server:setup-test-deps on every engine build, so every engine install carried the OTel SDK and the OTLP exporter whether or not anyone ran the bridge — which made the summary's "zero new base dependencies" true of the package but not of the engine. Move the cost to the step that needs it. A new client-python:setup-test-deps action installs packages/client-python/tests/requirements.txt into the engine interpreter through depends(), and runs only inside client-python:test, between the parallel build group and the test-server bracket: server:build -> [nodes:build | ai:build | client-python:build] -> client-python:setup-test-deps -> py-test-server:setup -> client-python:run-pytest -> teardown That ordering puts it after the node/ai requirement files land in the dist, so depends() resolves against the complete constraint set, and before the bracket, so a cold install does not hold the test server open. This mirrors the existing packages/ai/tests/requirements.txt pattern (packages/ai/scripts/tasks.js). packages/server/build-requirements.txt is restored byte-for-byte to its previous state. The 'rocketride[otel]' self-reference in the dev/test extras stays — that is the source-checkout path — and the hard imports in test_otel_mapper.py / test_otel_setup.py stay, so a missing dependency still fails collection instead of skipping. The tests no longer depend on the crewai node pulling opentelemetry in transitively either way. Docstrings in both test modules and the extras comment in pyproject.toml now point at the new file and step. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(client-python): surface the OTel bridge on the Python SDK overview Every other page under docs/public/python/ has a card in the SDK overview's "Explore the SDK" grid; the newly re-homed otel-bridge page did not, so it was reachable only from the sidebar and the CLI reference. Adds the card last in the grid, matching sidebar_position: 13. Uses LuRadar, which is not already taken by another card in this file and is present in the Lucide set react-icons v5 ships. This replaces the "OpenTelemetry export (`rocketride otel`)" section that used to live in docs/README-python-client.md and packages/client-python/docs/index.md — both deleted by the docs revamp (#2139). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(clients): declare otel-bridge as a Python-only site page The client-docs gate requires both folders to publish the same page set, with single-language extras declared in two places: PY_ONLY_PAGES/TS_ONLY_PAGES in scripts/validate-client-docs.py, and the Site page set section of the schema. docs/public/python/otel-bridge.md was added without either, so 'Docs site build' failed with: python: undeclared site page otel-bridge.md. The page is genuinely Python-only — the TypeScript SDK ships no OTel bridge (no otel surface under packages/client-typescript/src, no page in docs/public/typescript/) — so it belongs in PY_ONLY_PAGES rather than being mirrored. Declared at sidebar_position 13, matching the TypeScript-only database-sequelize extra. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * refactor(sdk): name the private SDK attribute the OTel bridge hooks The bridge receives monitor events by assigning its dispatcher to RocketRideClient._caller_on_event — the attribute EventMixin.__init__ creates from the public on_event= kwarg and EventMixin.on_event awaits for every DAP envelope. It assigns rather than passing on_event= because it is handed an already-built client and must restore the previous handler on exit. Nothing in the SDK's public surface pins that name, and a rename would fail silently: assigning an unknown attribute to a Python object succeeds, so the bridge would keep connecting, keep subscribing, keep logging "reconnected", and never export another span. Lift the string into CLIENT_EVENT_HOOK with the rationale next to it, so the coupling is a single named seam a test can assert on instead of a literal repeated across three call sites. No behaviour change. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * test(sdk): pin the OTel bridge against the real RocketRideClient The otel suite was mock-only past the mapper: test_otel_bridge.py and test_otel_cli.py each define their own FakeClient, and the provider tests monkeypatch build_providers, FlowSpanMapper and MetricsMapper out. That covers the run loop's branches but leaves the bridge's one point of coupling to the SDK untested — and a fake that defines _caller_on_event itself can never notice the SDK dropping it. Measured: renaming _caller_on_event to _caller_on_event_renamed across mixins/events.py and mixins/data.py — a change that stops the bridge exporting anything at all — leaves all 141 existing otel tests green. Add tests/test_otel_client_contract.py, which constructs a REAL RocketRideClient and stubs only the transport (call/is_connected); the SDK's add_monitor bookkeeping, EventMixin.on_event dispatch, the mappers and the OpenTelemetry provider pipeline all run for real: - three tests pinning CLIENT_EVENT_HOOK from different angles: the attribute exists in vars() on a real client, it is where the public on_event= kwarg lands, and the SDK's dispatch actually awaits it - run_bridge end to end on a real client, with events pushed through EventMixin.on_event rather than a fake's emit(), asserting routing, the unconsumed-event case, and hook restoration on shutdown - the wildcard subscription as the real SDK encodes it on the wire, plus the _monitor_keys entry the bridge relies on for replay after a reconnect - the 24-record wire fixture replayed to real exported spans, matching the 10-span forest test_otel_mapper.py gets by feeding the mappers directly, so the bridge provably adds, drops and reorders nothing Under the same simulated rename, six of these seven fail. Also add an opt-in live export against a real OTLP collector, gated on ROCKETRIDE_OTEL_TEST_ENDPOINT via the same skipif shape RocketRideClient_test.py uses for requires_llm. It starts no collector and changes no CI step; without the env var it simply skips. Note the new file in the client-python:setup-test-deps comment (it imports the OTel SDK at collection time) and cross-reference it from test_otel_bridge.py's docstring. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: nihalnihalani <24360630+nihalnihalani@users.noreply.github.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
dylan-savage
added a commit
that referenced
this pull request
Sep 15, 2026
Brings the Drizzle-over-pipes work up to date with develop, which has moved substantially since the last integration (merge base 210c9c7). Conflict resolutions: - packages/shell/contract/versions/v2.d.ts — took develop's. The branch's copy predates develop's `--regen` reset (branch v1 is dated 2026-08-11, older than develop's own v0 at 2026-08-18), so it belonged to a dead floor lineage. Recording DrizzleDatabaseLike now requires appending v3 via `shell:freeze`. - packages/shell/src/contract-check.generated.ts — took develop's; it is regenerated by the freeze. - packages/client-typescript/package.json — took develop's `client/`-prefixed dist paths and corrected the `./drizzle` export to match; it pointed at `dist/types/drizzle/…`, which the build no longer emits. - nodes/src/nodes/db_mysql/README.md — kept develop's rewritten Configuration intro and moved the new `row_mode` detail into the `execute` paragraph under "As a tool"; develop already documents `session_id` there. - packages/client-typescript/docs/guide/methods/database-sequelize.md — accepted develop's deletion (relocated by the docs revamp in #2139). The docs revamp also orphaned the Drizzle guide, which git did not flag because develop deleted its whole directory while the file was new there. Moved it to docs/public/typescript/database-drizzle.md, converted its frontmatter to the current convention (sidebar_position 14), and registered it as a TypeScript-only page in scripts/validate-client-docs.py, docs/development/clients/readme-schema.md and docs/public/typescript/index.mdx. Re-applied the Sequelize deprecation banner at its new path. scripts/validate-client-docs.py passes. The v3 shell freeze and the test suites are still outstanding. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NHWcvJ4P6y6gMChuoW1UMJ
This was referenced Sep 15, 2026
Merged
dylan-savage
added a commit
that referenced
this pull request
Sep 16, 2026
…2298) * docs(agents): restore the user task battery beside the agent docs it grades The 256-item acceptance sheet for the ROCKETRIDE_* context docs was dropped by the docs revamp (#2139). Bring it back at docs/agents/task-battery.md, outside context/ so the /client/docs bundle does not ship it, point its scoring-pass note at the post-revamp paths, and describe it in the README. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * docs: surface the Drizzle ORM driver in the agent bundle and SDK references The Drizzle-over-pipes driver (#1908) updated the public guide pages but not the agent-facing context, the pipeline-skill doc map, or the SDK reference tables, which still steered readers to the deprecated Sequelize binding. - ROCKETRIDE_typescript_API.md: add a `drizzle()` subsection, mark `sequelize()` deprecated, document `rowMode` on `query()` - ROCKETRIDE_DOC_MAP.md: link the Drizzle page - typescript/reference.md: add `drizzle` row and types, flag Sequelize as deprecated, add `rowMode` to the `query` signature - python/reference.md: point the cross-reference at Drizzle Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * docs: address review on the Drizzle agent-doc surfacing - Point the agent bundle at the absolute docs-site URL for the Drizzle guide; the bundle ships into user workspaces where repo paths do not resolve - `debug` is an npm package, not a Node built-in: name `util` and the `tty`/`util` pulled in via `debug` as the actual browser-bundle blockers (agent doc and public Sequelize guide) - Note the `unknown[][]` return shape under `rowMode: 'array'` and that `rowMode` is TypeScript-only Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
nihalnihalani
added a commit
to nihalnihalani/rocketride-server
that referenced
this pull request
Sep 18, 2026
… the python docs tree The Python-SDK half of this PR's documentation lived in docs/README-python-client.md. The documentation revamp (rocketride-org#2139, 92a1b7e) deleted that file wholesale as part of moving the client docs into docs/public/{python,typescript}/, and this branch's merge of develop took the deletion without relocating the pipe-diff content. The result was a regression against the state this PR was reviewed in: `rocketride diff` and the importable `rocketride.pipediff` package had no page anywhere under the new docs tree, and nothing in the Python client's own docs folder mentioned them. Add docs/public/python/pipe-diff.md as a Python-only site page (sidebar_position 14, the slot after otel-bridge at 13) carrying the material that used to live in the deleted README: install/quickstart, the flag table and exit codes, what counts as a change, the human/JSON/Markdown output modes, the CI recipe that points at the bundled composite action, and a reference of every name exported from `rocketride.pipediff` (engine, data model, reporters) plus troubleshooting and limitations. The three sample outputs are the real renderers' output, reproduced byte for byte. Wire it into the docs spine the same way otel-bridge is wired: - scripts/validate-client-docs.py: declare `pipe-diff` in PY_ONLY_PAGES. This is required — an undeclared page in one client folder and not the other fails the site-page-set parity check. - docs/development/clients/readme-schema.md: the schema doc enumerates the declared single-language extras that the validator constant mirrors, so it would otherwise name otel-bridge as Python's only extra. - docs/public/product/clients/index.md: the spine paragraph that lists what the Python package additionally ships would otherwise claim the OTel bridge is the only such surface. - docs/public/python/index.mdx: a side card next to the OTel bridge card. - docs/public/product/connect/cli.mdx: a pointer from the `diff` section to the in-depth guide. The CLI reference keeps its own flag table; the guide is the one that also documents the importable API. No behaviour change. Verification: `python3 scripts/validate-client-docs.py` ok (6 README sections, 13 shared site pages, symbol parity); `node scripts/build.js docs:test docs:build docs:check` exit 0 (250 pages staged, docs exports in sync); `ruff check` / `ruff format --check` clean on the touched .py file. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Merged
1 of 2 tasks
nihalnihalani
added a commit
to nihalnihalani/rocketride-server
that referenced
this pull request
Sep 18, 2026
CodeRabbit's r3980191483 asked for `--token`/`ROCKETRIDE_TOKEN` to be dropped from the eval documentation, since eval registers only `--uri` and `--apikey`. Both files it pointed at are gone (the rocketride-org#2139 docs revamp deleted docs/README-python-client.md and packages/client-python/docs/ index.md), and the re-homed `### eval` section lists no `--token` in its flag table, so the fix landed by relocation. What survives is the page's opening line, "Every command accepts connection options", above a table that lists `--token` and `--pipeline`. That sentence is pre-existing develop text shared with `validate` and is not rewritten here. Instead the eval section now states its own contract: only `--uri` and `--apikey`, with the reason (the pipeline comes from the spec, the task token is per run). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
nihalnihalani
added a commit
that referenced
this pull request
Sep 21, 2026
…ree) + PR-comment Action (#1607) * feat(sdk): rocketride.pipediff — semantic diff engine for .pipe files (#1606) A raw `git diff` of a `.pipe` file is dominated by canvas coordinate churn: every node carries a `ui` block whose `position.x`/`position.y` move whenever the node is nudged, plus a top-level `viewport` that shifts on pan or zoom. Those lines say nothing about behavior, yet they bury the changes that do — a swapped LLM provider, a re-tuned chunk size, a rewired retrieval step. `rocketride.pipediff` compares two parsed `.pipe` documents instead of their text. Components are matched by `id`; the result is grouped into nodes (added / removed / re-provisioned), edges (reconstructed from each component's `input[]` data lanes and `control[]` orchestration lanes), and config (deep-diffed into dotted paths such as `config.default.strlen`). Layout is collapsed to a single "changed" flag unless the caller asks for it, and JSON values are compared with JSON typing so `1` and `true` are never conflated. The package is import-only and does no I/O beyond reading the files it is given: `gitref` shells out to `git show <ref>:<file>` for the ref-vs-worktree case and maps every failure mode onto an explicit error, and `reporters` renders the same diff as human text, stable sorted JSON, or Markdown with `|` and backticks escaped so untrusted values cannot break a PR comment. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * feat(cli): add `rocketride diff` subcommand (#1606) Wires `rocketride.pipediff` onto the CLI as `rocketride diff`, the review half of the "pipelines as code" loop: `validate` says a `.pipe` is well-formed, `diff` says what changed in it. The subcommand is deliberately the odd one out in `setup_parser`: it is local only — it never contacts an engine or the network — so it does not take the shared `--uri`/`--apikey`/`--token` connection arguments every other command gets from `_add_connection_args`, and its `--json` is a plain format flag rather than the shared `--json [FILE]`. It compares two files positionally, or a single working-tree file against a git ref via `--git REF`. `--markdown` renders a PR-comment-friendly report, `--include-layout` opts the canvas churn back in, and `--exit-zero` turns a gating run into an informational one. Exit codes are the CI contract: 0 for no semantic changes (or any successful run under `--exit-zero`), 1 when semantic changes are found, and 2 for a usage error, an unreadable or unparseable file, or a bad git ref. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * feat(ci): add pipe-diff composite action — semantic `.pipe` diff as a sticky PR comment (#1606) Because `rocketride diff` is local and exits non-zero on change, it drops straight into a pull-request check. This composite action does the assembly: it installs the CLI, resolves the PR base commit (fetching it itself, so the default `fetch-depth: 1` checkout is fine), finds every changed `.pipe` file, diffs each against the base, and maintains one sticky comment so reviewers see "OpenAI → Anthropic, chunk size 512 → 1024, retrieval rewired through a reranker" instead of a wall of coordinate noise. Every run also writes the full report to the job summary, which is what keeps the action useful on a fork pull request: GitHub hands those jobs a read-only `GITHUB_TOKEN` regardless of `permissions:`, so the comment API returns 403 — the action warns instead of failing and the diff stays visible in the summary. The summary is written on the base-fetch failure path too, rather than leaving a run with no output at all. `install-from` exists because `rocketride diff` ships in a release after 1.3.0 and PyPI does not serve it yet; it points pip at a checkout or a pinned commit instead. Markdown assembled from file names and diff output is escaped, and the action distinguishes "no `.pipe` files changed" from "could not determine what changed" rather than reporting the reassuring answer blind. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs: document `rocketride diff` in the CLI reference and examples (#1606) Documents the new subcommand where the restructured docs tree puts CLI content — `docs/public/product/connect/cli.mdx` — as a `### diff` section alongside `### validate`, the other file-oriented, CI-friendly command. The section covers the flags, the exit-code contract, what does and does not count as a semantic change, all three output modes, and the `pipe-diff` action recipe including the fork-PR token caveat and the pre-release `install-from` workaround. Two surrounding statements needed amending rather than just appending: the Python-vs-TypeScript note now records that `diff` is the one Python-only command, and the "Connecting" preamble now excludes `diff` from the commands that take connection options, because it takes none. `examples/pipe-diff-example.md` puts a raw `git diff` next to `rocketride diff` on the same edit to `rag-pipeline.pipe`; the examples index gains a Guides entry pointing at it. Note for reviewers of the earlier revision: this content previously lived in `docs/README-python-client.md`, `packages/client-python/docs/index.md`, and `packages/docs/content-static/cli.mdx`. All three paths were removed by the documentation revamp, so the content is re-homed here instead of resurrecting them, and the example's cross-link now points at the published `/connect/cli#diff` anchor. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(client-python): ship rocketride_common when installing from the source tree (#1607) `pip install ./packages/client-python` from a clean checkout produced a `rocketride` binary that could not start: File ".../rocketride/cli/utils/env.py", line 29, in <module> from rocketride_common.env import ( ModuleNotFoundError: No module named 'rocketride_common' `cli/__init__.py` eagerly imports `.main`, `main.py` imports `.utils.env` at module scope, and that imports `rocketride_common` — so every subcommand, `diff` included, dies at import time. `rocketride_common` is a source library shared with the TypeScript client rather than a published distribution, so it cannot be added to `dependencies`. It lives at `packages/client-common/python/src` and the builder copies it into the staging tree (`client-python:wheel-source`) before the released wheel is built — which is why the published wheel has always been fine and only the source tree is broken. The source tree is also exactly what the new pipe-diff action's `install-from` input points at, in both documented forms (`./packages/client-python` and `git+https://…#subdirectory=packages/client-python`, where pip clones the whole repository), so the documented pre-release recipe could never work. Adding the sibling source root to `packages.find.where` makes setuptools find the shared package in the source-tree layout. setuptools skips a `where` entry whose directory does not exist, so the builder's staged layout — where `rocketride_common` is already inside `src/` and there is no sibling directory — is untouched. Verified end to end: from a clean `git archive` tree `rocketride diff --help` now exits 0, and rebuilding the staged layout through `python -m build` still yields an sdist and a wheel carrying both `rocketride` and `rocketride_common`, byte-identical in top-level contents to before. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(ci): diff .pipe files against the merge base, not the base branch tip (#1607) The action took `github.event.pull_request.base.sha` -- the tip of the base branch at event time -- and used it directly for both `git diff "${BASE_SHA}" HEAD` and `rocketride diff --git "${BASE_SHA}"`. That tip keeps moving as other pull requests merge, so every commit merged into the base since this branch forked was reported as a change made by THIS pull request, and reported backwards: those commits sit on the left-hand side of the diff, so a node someone else added shows up as a node this PR deleted. Reproduced in a scratch repository -- fork at C0, this branch adds C1 touching `a.pipe`, the base then gains C2 adding `b.pipe`: git diff --name-status $C2 HEAD -> M a.pipe / D b.pipe (wrong) git diff --name-status $(git merge-base $C2 HEAD) HEAD -> M a.pipe (right) The action now resolves `git merge-base "${BASE_SHA}" HEAD` once and feeds that to the changed-file enumeration, the deletion enumeration and every per-file `rocketride diff --git`. This is the comparison GitHub's own "Files changed" tab shows, and the same one the action's README already recommended in its manual recipe. Fetch depth had to change with it. The base commit was fetched with `--depth=1`, which leaves it with no ancestry for `merge-base` to walk, so that flag is gone. A default `fetch-depth: 1` checkout also leaves HEAD with no ancestry, so when the first `merge-base` attempt comes back empty the action deepens (`--unshallow`, falling back to `--deepen`) and retries; a workflow that already checks out with `fetch-depth: 0` never pays for that. Verified against a real depth-1 clone: the first attempt returns nothing, the deepen recovers, and the merge base resolves to the fork point. If the merge base still cannot be computed the step now fails with a report and an annotation pointing at `fetch-depth: 0`, rather than falling back to the base tip -- silently diffing the wrong thing is the defect being fixed. The action README and the CLI reference are updated to describe the merge-base comparison and the optional `fetch-depth: 0`. Verified: action.yml parses as YAML, `bash -n` and shellcheck pass on all four `run:` blocks. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * test(client-python): assert the Markdown table escaping without cmarkgfm (#1607) `test_gfm_renders_escaped_cells_back_to_the_original_value` was guarded by `@unittest.skipIf(cmarkgfm is None)` and `cmarkgfm` was in no dependency group, so the one test that decides whether `_md_cell`'s backslash handling is right never ran — not locally, not in CI. Declaring the dependency is not enough on its own: CI does not `pip install` this package's extras at all. `builder test` runs `client-python:run-pytest`, which invokes pytest through the bundled engine binary in its own isolated environment (packages/client-python/ scripts/tasks.js -> scripts/lib/pytest.js), so an entry under `[project.optional-dependencies] test` would still leave the test skipped on every CI run. So the coverage now comes from an assertion that needs no C extension. `_split_gfm_row` applies cmark-gfm's own table-row rule directly: a `|` splits the row only when no backslash precedes it, and a `|` preceded by a run of n backslashes is escaped and loses exactly one of them. The new `test_escaped_cells_survive_gfm_row_splitting` runs that over the same value corpus — which both tests now share — and asserts three columns and an exact round-trip to the original value. That rule was derived from cmark-gfm rather than from the spec prose, and then checked against it: comparing `_split_gfm_row` with `cmarkgfm` over several thousand random values built from the punctuation that shows up in pipeline config produced zero disagreements. Mutation-checked too — making `_md_cell` double the backslash instead of adding one fails six of the eight corpus values. The cmarkgfm test is kept as an oracle rather than deleted: it is the stronger check when the library is present, it now shares the corpus, and `cmarkgfm` is added to the `test` extra so `pip install .[test]` enables it instead of leaving the guard permanently dead. Its skip message says which check still covers the rules. Verified: 131 passed with cmarkgfm installed; 130 passed and 1 skipped (only the oracle) without it. Ruff check and format clean. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * ci: run the pipe-diff action on this repo and pin-check composite actions (#1607) Two gaps, both of which let the new action ship unexercised. Nothing referenced the action. No workflow under `.github/workflows/` mentioned pipe-diff, so the install step, the base resolution, the `rocketride diff` invocation and the comment step had never run as a GitHub Action — the first consumer to adopt it would have been the one to find out. `.github/workflows/pipe-diff.yml` runs it on `pull_request` against the repository's own 45 `.pipe` files, and the paths filter also fires on changes to the action, the diff engine and the CLI subcommand, so a change to any of them is exercised before it merges even when no pipeline changed. It is `pull_request`, not `pull_request_target`, for the reason the action's own README gives: a `pull_request_target` job checks out the base branch (nothing to diff) and holds a writable token, and this workflow installs the CLI from the checkout. On the `pull_request` event a fork's code runs with a read-only token and no secrets, and the report lands in the job summary — the documented fork behaviour. `install-from` points at the checkout because PyPI does not serve `rocketride diff` yet; the comment says to switch to `cli-version` once it does. Second, `scripts/verify-action-pins.mjs` only enumerated `.github/workflows/*.y(a)ml`, so the SHAs pinned inside composite actions were outside the pin guard entirely — both this PR's `.github/actions/pipe-diff/action.yml` and the pre-existing `.github/actions/validate-pipes/action.yml`. A composite action could carry a mismatched or unpinned `uses:` and CI would pass. The verifier now also enumerates `.github/actions/<name>/action.{yml,yaml}`, treating a missing directory as an empty list. Verified: the existing verifier unit tests still pass (7/7); the live check goes from 26 files to 29 and reports "Verified 87 action pins across 29 workflow and composite-action files" with no errors; and drifting the pipe-diff action's github-script comment from v7 to v6 is now caught with the correct file and line annotation, which it was not before. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * test(client-python): run the cmarkgfm escaping oracle in CI `test_gfm_renders_escaped_cells_back_to_the_original_value` is the test that decides how `_md_cell` escapes a pipe, but `cmarkgfm` was only in the `test` extra and CI never installs that extra — it runs pytest on the bundled engine interpreter against the source checkout. So on every CI run the decisive test skipped, and the gate was `test_escaped_cells_survive_gfm_row_splitting`, which resolves the same corpus with our own model of cmark-gfm's cell scanner. Code and test then shared one assumption: if the model were wrong, both would be wrong together and nothing would go red. The values are untrusted `.pipe` content rendered into a PR comment, so a cell that splits its row is exactly the failure the escaping exists to prevent. Add `cmarkgfm` to `packages/client-python/tests/requirements.txt`, which `client-python:setup-test-deps` installs into the engine interpreter through `depends()` — the mechanism #1612 added for the otel-bridge tests, so the wheel resolves under the same merged constraints as everything else and both packages stay on one route. The floor stays `>=2024.1.14`: 2024.11.20 publishes no macOS wheels, and a tighter pin would make the macOS leg build cmark-gfm from source. It resolves to 2025.10.22, which has wheels for every CI leg. The `pyproject.toml` comment that explained why the extra was not enough now names the route CI actually takes. The library-free check keeps its own docstring but no longer claims CI never installs the renderer. Verified with the engine interpreter, the same way CI will: depends(tests/requirements.txt) -> cmarkgfm==2025.10.22 (wheel) before: 1 passed, 1 skipped ("cmarkgfm is not installed") after: 2 passed, 16 subtests passed The oracle agrees with the scanner model on the whole corpus, so `_md_cell`'s rule is now checked against the renderer rather than against our model of it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * docs(client-python): document rocketride diff and the pipediff API in the python docs tree The Python-SDK half of this PR's documentation lived in docs/README-python-client.md. The documentation revamp (#2139, 92a1b7e) deleted that file wholesale as part of moving the client docs into docs/public/{python,typescript}/, and this branch's merge of develop took the deletion without relocating the pipe-diff content. The result was a regression against the state this PR was reviewed in: `rocketride diff` and the importable `rocketride.pipediff` package had no page anywhere under the new docs tree, and nothing in the Python client's own docs folder mentioned them. Add docs/public/python/pipe-diff.md as a Python-only site page (sidebar_position 14, the slot after otel-bridge at 13) carrying the material that used to live in the deleted README: install/quickstart, the flag table and exit codes, what counts as a change, the human/JSON/Markdown output modes, the CI recipe that points at the bundled composite action, and a reference of every name exported from `rocketride.pipediff` (engine, data model, reporters) plus troubleshooting and limitations. The three sample outputs are the real renderers' output, reproduced byte for byte. Wire it into the docs spine the same way otel-bridge is wired: - scripts/validate-client-docs.py: declare `pipe-diff` in PY_ONLY_PAGES. This is required — an undeclared page in one client folder and not the other fails the site-page-set parity check. - docs/development/clients/readme-schema.md: the schema doc enumerates the declared single-language extras that the validator constant mirrors, so it would otherwise name otel-bridge as Python's only extra. - docs/public/product/clients/index.md: the spine paragraph that lists what the Python package additionally ships would otherwise claim the OTel bridge is the only such surface. - docs/public/python/index.mdx: a side card next to the OTel bridge card. - docs/public/product/connect/cli.mdx: a pointer from the `diff` section to the in-depth guide. The CLI reference keeps its own flag table; the guide is the one that also documents the importable API. No behaviour change. Verification: `python3 scripts/validate-client-docs.py` ok (6 README sections, 13 shared site pages, symbol parity); `node scripts/build.js docs:test docs:build docs:check` exit 0 (250 pages staged, docs exports in sync); `ruff check` / `ruff format --check` clean on the touched .py file. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * docs(client-python): say the cmark-gfm oracle runs in CI, not that it is optional asclearuc asked for the cmark-gfm oracle to actually execute in CI rather than skip, so the backslash-escaping question is settled by the renderer instead of by our model of it. That was done by adding `cmarkgfm` to packages/client-python/tests/requirements.txt, which the `client-python:setup-test-deps` step installs into the engine interpreter the suite runs on. The prose around the test was never updated and still described the renderer as an optional local nicety: - `_md_cell`'s docstring said the round-trip was "cross-checked against ``cmarkgfm`` when that optional renderer is installed"; - the `skipIf` reason pointed only at `pip install .[test]`, so a reader of a skip line had no way to tell whether CI had skipped it too; - the library-free sibling test read as the primary check rather than the fallback. Rewrite all four strings (the guarded import comment, both test docstrings, and the skip reason) to state where the renderer comes from in CI and what a skip therefore means: a local interpreter without it, never the CI leg. The library-free `_split_gfm_row` check is named as the fallback over the same corpus; the renderer test is named as the one that decides the rule. Comments only -- no behaviour change, and the skip condition itself is unchanged. Verification: `./dist/server/engine -m pytest packages/client-python/tests/test_pipediff_reporters.py -q` -> "32 passed in 0.08s" (exit 0), with test_gfm_renders_escaped_cells_back_to_the_original_value running rather than skipping; `ruff check` / `ruff format --check` clean on both files. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * test(client-python): pin the top-level keys diff_pipes reads and ignores Author-side finding, not a reviewer ask. `diff_pipes` reads exactly three top-level keys of a `.pipe`: `components`, `version` and `viewport`. Every other top-level key the schema defines -- `project_id`, `isLocked`, and anything a later schema adds -- is ignored on purpose, because it is editor/session metadata rather than pipeline behaviour. That rule is load-bearing for the exit code (it is why opening a pipeline in the editor and saving it does not fail a CI gate) and it was implicit: nothing asserted it, and nothing stated it. A later contributor widening the read set by one key would turn every `project_id` rewrite into a gate failure, with no test to catch it. Pin it: - tests/test_pipediff_engine.py: `test_unknown_top_level_keys_are_ignored` changes `project_id`, `isLocked`, a `name`, and an unknown future key, and asserts an empty diff under both `include_layout=False` and `True`; `test_version_is_compared_but_viewport_only_under_include_layout` pins the other half -- `version` always counts, `viewport` is enumerated and counts only under the flag. - engine.py: one paragraph in the `diff_pipes` docstring stating the read set and naming the test. - docs/public/product/connect/cli.mdx and docs/public/python/pipe-diff.md: one bullet each in "What counts as a change", so the contract is stated where a user decides whether `diff` is safe to gate on. No behaviour change -- the tests characterize what the code already does, so they pass on the pre-change engine. Proof that they bite: with a temporary mutation adding `project_id` to the compared set, test_unknown_top_level_keys_are_ignored fails with "assert [NodeChange(i... d_changes=[])] == [] / Left contains one more item: NodeChange(id='__project__', kind='config', ...)" -- "1 failed, 61 deselected in 0.04s" (log impl-1607-toplevel-pin-mutation.log). The mutation was reverted before this commit. Verification: the four pipediff/CLI modules through the bundled engine -> "133 passed in 0.35s" (exit 0, was 131 before the two new tests); `ruff check` / `ruff format --check` clean on both .py files; `node scripts/build.js docs:test docs:build docs:check` exit 0; `python3 scripts/validate-client-docs.py` ok. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * docs(pipediff): pin note and two accuracy clauses on the Python guide The CI snippet on the Python guide carried no pin caveat, unlike its two siblings in cli.mdx and examples/pipe-diff-example.md, so a reader landing here was not told that `@develop` is a placeholder. Add the one sentence. Two accuracy clauses in the same page: the `--git` row now says a file that is absent in the ref also reports a version change from `null` (the CLI substitutes an empty pipeline), and the `resolve_git_ref` row names the remaining `PipeDiffError` case — contents at the ref that are not a valid pipeline. --------- Co-authored-by: nihalnihalani <24360630+nihalnihalani@users.noreply.github.com>
3 of 7 tasks
dk-rocketride
added a commit
that referenced
this pull request
Sep 23, 2026
* docs(agents): Drizzle in the agent bundle + restore the task battery (#2298)
* docs(agents): restore the user task battery beside the agent docs it grades
The 256-item acceptance sheet for the ROCKETRIDE_* context docs was dropped
by the docs revamp (#2139). Bring it back at docs/agents/task-battery.md,
outside context/ so the /client/docs bundle does not ship it, point its
scoring-pass note at the post-revamp paths, and describe it in the README.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* docs: surface the Drizzle ORM driver in the agent bundle and SDK references
The Drizzle-over-pipes driver (#1908) updated the public guide pages but
not the agent-facing context, the pipeline-skill doc map, or the SDK
reference tables, which still steered readers to the deprecated
Sequelize binding.
- ROCKETRIDE_typescript_API.md: add a `drizzle()` subsection, mark
`sequelize()` deprecated, document `rowMode` on `query()`
- ROCKETRIDE_DOC_MAP.md: link the Drizzle page
- typescript/reference.md: add `drizzle` row and types, flag Sequelize
as deprecated, add `rowMode` to the `query` signature
- python/reference.md: point the cross-reference at Drizzle
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* docs: address review on the Drizzle agent-doc surfacing
- Point the agent bundle at the absolute docs-site URL for the Drizzle
guide; the bundle ships into user workspaces where repo paths do not
resolve
- `debug` is an npm package, not a Node built-in: name `util` and the
`tty`/`util` pulled in via `debug` as the actual browser-bundle blockers
(agent doc and public Sequelize guide)
- Note the `unknown[][]` return shape under `rowMode: 'array'` and that
`rowMode` is TypeScript-only
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
* fix(nodes): stop four nodes passing raw input downstream (#2304)
After a Python lane handler returns normally, the engine also forwards the handler's input on the same lane. Only `preventDefault()` stops that. Four nodes returned normally, so downstream received the raw input next to the node's own output.
- `context_optimizer`: both exits of `writeQuestions` now end in `return self.preventDefault()`. Downstream no longer gets the untrimmed original next to the trimmed question.
- `summarization`: `writeText` and `writeTable` collect the text, then suppress the forward. Only the summary from `closing()` goes on.
- `tool_guild`: `writeText`, `writeQuestions` and `writeDocuments` collect, then suppress. The docstring claiming "overriding suppresses raw passthrough" is corrected.
- `tool_n8n`: the same for `writeText`, `writeQuestions`, `writeDocuments`, and every `writeImage` / `writeAudio` / `writeVideo` chunk.
- Missing docstrings added in the four node files.
Tests:
- `test_lane_forward_once_nodes.py`: delivery-count cases for every changed handler.
- New `summarization/test_summarization.py`: the node had no unit tests.
- `context_optimizer/test_all.py` and `test_tool_n8n.py`: direct handler calls now require the `PreventDefault` exception.
* fix(memory_persistent): prevent double-forward of questions and answers (#1682)
* fix(memory_persistent): prevent double-forward of questions and answers
* test(memory_persistent): assert preventDefault is called on explicit forwards
* docs(memory_persistent): note preventDefault() unreachability and test stub caveat
* fix(memory_persistent): cover all six exits with delivery-count tests
- Every exit of `writeQuestions` and `writeAnswers` now ends in `return self.preventDefault()`. The separate `return` after `preventDefault()` could never run, because `preventDefault()` raises.
- `test_node.py`: two tests for the invalid `session_id` paths, which no test reached before. Removing `preventDefault()` from either path now fails a test.
- `test_lane_forward_once_nodes.py`: six delivery-count cases, one per exit (no store, invalid session, normal path), under the engine's forward rule. On the normal path, only the enriched or stored copy reaches downstream, never the original.
- Docstrings in `IInstance.py` now document their arguments.
---------
Co-authored-by: Alexandru Sclearuc <asclearuc@gmail.com>
* fix(audio_player): accept stop()'s leftover sentinel in the drain test (#2306)
`stop()` queues its own `None` sentinel when it starts waiting before the audio callback has reached the decoder's. The callback stops at the first `None`, so the second one stays in the queue. `test_begin_write_end_still_drains_normally` still required an empty queue, so it failed whenever `stop()` won that race, which happens on a loaded CI runner.
- The drain test now accepts `[]` or `[None]` as the leftover. Unplayed audio still fails it.
- New `test_stop_waiting_before_the_callback_leaves_only_its_own_sentinel` forces the `stop()`-first order. It requires exactly `[None]` to be left, all audio played, and `stop()` returned.
- The shared setup moved into `_player_with_queued_audio()` and `_start_simulated_hardware()`.
* fix(ci): open the model-sync PR with the release-bot App token (#2312)
Mint a release-bot GitHub App token and pass it to create-pull-request so the weekly Sync LLM Models PR triggers CI on its own instead of waiting for a manual approve. Falls back to github.token if the token can't be minted. release-bot was granted Pull requests: write on the org install. Fixes #2311.
* feat(profiler): per-thread breakdown and a call tree that keeps real work (#2299)
* feat(ai): break the profiler report down by thread
stop() now also captures the session per yappi context, next to the flat
capture and before clear_stats(). threads() lists them busiest first with
their counts, and report_tree() takes an optional thread id; without one it
behaves as before. DAP: new rrext_cprofile_threads, plus an optional thread
argument on rrext_cprofile_report_tree, in the engine subprocess and the
task-server proxy.
Two yappi traps shape the capture:
- get_func_stats(ctx_id=...) tests the id for truthiness, so thread 0 would
silently get every thread's functions. The filter goes in as a dict.
- The default context name is the thread's class, so workers read 'Thread'
and engine workers '_DummyThread'. start() installs a callback reading
Thread.name from threading._active; current_thread() would deadlock when
the profiled call is _active_limbo_lock.acquire() (yappi #48). yappi keeps
the first name it gets, so the thread ident is reported as well.
Cost on a parse -> preprocessor pipeline over 12 and 36 documents: the
per-thread copy holds 2.1-3.5 MB against 0.5 MB flat (13-25 threads), and
the capture under _lock grows from ~117 ms to ~250-275 ms.
* perf(ai): cache _relativize_path
stop() calls it under _lock for every entry and callee of each capture:
about 5,900 calls per stop on a parse pipeline over 12 documents, since the
per-thread capture repeats the flat one's paths, over a few hundred
distinct paths. The result depends only on the path, so cache it.
On the same pipeline the capture under _lock drops from ~250-275 ms to
~160-170 ms, against ~117 ms before the per-thread capture existed. Memory
drops only ~10%: every entry of a path now shares one string, but the
per-entry dicts are most of the size.
* fix(ai): stop the call tree from pruning away real work
report_tree() dropped every node whose own time fell under min_pct, and with
it the whole subtree. Two kinds of node read light while real work hangs
below them: frames still running at stop() (thread entry points, executor
worker loops) read ~0 because yappi books time on return, and the event
loop step that resumes a coroutine is short while the coroutine is booked
its whole lifetime. On a parse pipeline this emptied the merged tree, the
event-loop thread and every executor thread started mid-session; threads
that predated the session survived only because their zero-time frames sat
at depths 0 and 1, which are never pruned.
A node is now pruned only when nothing reachable below it reaches the
threshold either. The threshold was also inflated: total_time summed ttot,
counting nested calls again (a 5 s session summed to 412 s). It is now the
time the threads ran, as yappi measures it per thread - the selected one,
or all summed - falling back to the sum of self times without thread data.
Same session after the fix: project code shows in every view at the
default threshold, and trees stay at 4-168 nodes, built in under 60 ms.
* feat(client): list profiled threads and fetch one thread's tree
Python: cprofile_threads() and a thread parameter on cprofile_report_tree().
TypeScript: cprofileThreads() and a trailing thread parameter on
cprofileReportTree(), with CProfileThreadInfo / CProfileThreadsResponse.
Thread 0 is a real id, so both clients test for absence, not truthiness.
The shell re-exports the SDK's types, so the two new ones are new shell-api
names and shell:check fails until they are frozen: this freezes shell-api
v4. Its delta over v3 is exactly these types and the two method signatures.
The contract test runs data through a task's pipeline and checks that no
thread carries yappi's class-based names, that one thread's tree totals that
thread's calls, and that an unknown id is refused, which proves the
argument reached the engine.
test-ui's API sweep and the SDK references list the new method; the Python
reference also claimed include_system defaults to False, which it never did.
* feat(profiler-ui): pick a thread to visualise
A Thread dropdown heads the viz controls: all threads (the old view) or one,
labelled with name, thread id and time, since names repeat. The views get
the same tree shape either way. The list is refetched with the report, a
selection missing from the new list falls back to all threads, and a server
without rrext_cprofile_threads just hides the dropdown. Start clears the
list, since ids are per session.
Bumps the app to 1.3.0: an OSS server reseeds a built-in app only when its
apps.json version changes, so an upgraded server would keep serving the
old profiler otherwise.
* fix(profiler-ui): prune the charts by subtree weight
The client-side cutoff compared a child's own cumtime with its parent's,
so a zero-time frame the server now keeps was dropped again at the root,
taking the work under it along. It now compares the largest cumtime in
each subtree, and runs before the depth limit so that weight covers the
whole subtree.
* fix(profiler-ui): clear the two type errors tsc reports
initConnectionStore took the shell's settings as Record<string, string>,
but they are Record<string, SettingValue>, and a port setting may well be
a number; the seeded host and port are now coerced to strings. The
sunburst filtered on x0/x1 before casting to HierarchyRectangularNode,
which the flame graph already does first. Neither changes behaviour: the
app build does not type-check, which is how both went unnoticed.
* fix: CodeRabbit
* fix(mcp): make the HTTP MCP server testable end to end (auth, routing, engine URI, tools) (#2315)
* fix(mcp): refuse query-string credentials on /mcp and never fall back to the shared client for authenticated callers
AuthMiddleware falls back to `?auth=` when no Authorization header is sent,
but auth.authorize() only read the header. `POST /mcp/?auth=<key>` therefore
passed the middleware, skipped every /mcp check (tk_/pk_/cd_ refusal, JWT
signature, issuer, audience) and ran tools on the server's shared engine
client.
- authorize() now refuses any request carrying an `auth` query parameter
(parsed as Starlette parses query_params), per the MCP authorization spec.
- Defence in depth: a request the middleware authenticated
(scope['state']['account']) that carries no header credential is refused
instead of served by the shared client. The credential-less dev bypass
(public route, no account) is unchanged.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(mcp): reserve /mcp for the MCP API — move the shell page to /mcp-server and route bare /mcp to the handler
The shell registered a public GET page at /mcp. The MCP endpoint is a
Starlette Mount, which only matches /mcp/..., so the page won the bare
path: POST /mcp answered 405 with no WWW-Authenticate challenge (spec
clients POST to the advertised resource .../mcp and never saw the OAuth
challenge), and AuthMiddleware treated every /mcp request as public.
add_route's duplicate check never saw the clash because the Mount goes
straight onto the router.
- shell: the MCP marketing page moves to /mcp-server (still unlisted).
- mcp: an exact, method-agnostic Route serves bare /mcp alongside the
Mount, so /mcp and /mcp/ reach handle_mcp identically (no 405/307).
- mcp: initModule refuses to start if any route already claims /mcp or
/mcp/*, or a public pattern matches the endpoint.
- mcp: MCP_DEV_NO_AUTH now opens exactly /mcp and /mcp/ (loopback only).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* feat(mcp): default the engine URI to the public MCP origin, or the local address on loopback binds
ROCKETRIDE_URI had no default, so on staging/prod every MCP tool call
raised "Missing engine URI". The URI is now resolved once in initModule
and injected into the config every engine client is built from, and it
also drives the widget CSP origin and the upload/dropper links:
1. explicit: config rocketride_uri / env ROCKETRIDE_URI, used as-is
2. loopback default: ws://127.0.0.1:<port> (ws://[::1]:<port>) so a
laptop engine never silently calls RocketRide's cloud
3. public default: the MCP_RESOURCE_IDENTIFIER origin, https->wss /
http->ws (default wss://api.rocketride.ai)
One startup log line names the resolved URI (userinfo/query stripped)
and the rule that chose it. ROCKETRIDE_AUTH/APIKEY handling is unchanged.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(mcp): only offer send_files on loopback-bound engines — it reads paths on the engine host
The SDK's send_files checks and opens each caller-supplied path on the
machine running the MCP module. On a deployed engine that is our server, so
any authenticated caller could probe file existence ("File not found") and
upload server files into their own pipeline.
initModule now decides local_engine = is_loopback_bind(bind_host) once and
passes it to build_mcp_server -> ToolRegistry(local_engine=...). Tools
registered local_engine_only (send_files) are neither listed nor dispatched
on a deployed engine: a call by name gets an Unavailable refusal that ignores
the client and arguments. The registry defaults to the deployed surface.
Loopback binds are unchanged.
Also corrects the send_files description (engine-host paths, not
store-relative), drops the stale "or filepath" from run_pipeline, and
updates the MCP docs and agent skill contract.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(mcp): send no SDK env from deployed engines so caller secrets keep precedence
WsEngineClient built RocketRideClient(uri, auth) with no env, so the SDK
loaded os.environ plus ./.env and use() forwarded every ROCKETRIDE_* var as
the run's env. The engine applies caller env last, over org/team/user
secrets, so on a deployed engine the server's own ROCKETRIDE_* vars
overrode a customer's same-named secrets for MCP-started runs.
initModule injects local_engine (the same loopback decision that gates
send_files) into config alongside the resolved URI; make_engine_client
builds the client with env={} unless it is set. Loopback binds keep the SDK
default. Explicit uri/auth are unaffected.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(mcp): rewrite deploy tools against the current SDK deploy API
The MCP deploy tools (#1880) were written against a Python SDK deploy API
that deploy-2 (#1764, 0dfdc8b44) had already removed. In production every
call failed: deploy_add passed a `schedule` kwarg add() no longer takes,
deploy_status/deploy_update called methods that no longer exist,
deploy_remove omitted the now-required team_id, and deploy_list counted
the keys of the {rows,total,page,pageSize} envelope. The suite stayed
green because FakeEngine and the seam tests mirrored the old API.
The seam (WsEngineClient) is now one keyword-argument passthrough per
DeployApi method, and the tools are thin wrappers in the
teams-as-environments model:
- deploy_add -> add (pipeline, comment, deploy_to)
- deploy_list -> list (team_id, paging, search, filters, sort); returns
deployments + count/total/page/pageSize
- deploy_status -> get (project_id, team_id)
- deploy_versions -> versions (project_id, paging)
- deploy_to_team -> deploy (project_id, version, team_id)
- deploy_set_schedule -> set_schedule (project_id, source_id, schedule,
team_id, ttl)
- deploy_enable / deploy_disable / deploy_remove -> enable / disable /
remove (project_id, team_id)
deploy_update is gone; its operations are deploy_add + deploy_to_team and
deploy_set_schedule. The rocketride://pipelines resource returns the
unwrapped envelope. Tool count: 33 (32 on a deployed engine).
Drift guard: test_engine.py calls every deploy_* seam method against a
recording fake and binds each recorded call to the real
rocketride.deploy.DeployApi signature (method must exist, arguments must
bind, every seam argument must arrive under the same name). It fails on
the old seam code. A second test pins the EngineClient protocol and the
conftest FakeEngineClient to the seam's deploy signatures.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(task_http): don't re-send engine env as caller env from task_Execute on deployed engines
task_Execute built RocketRideClient(uri, auth) with no env, so the SDK
snapshotted os.environ plus ./.env and use() forwarded every ROCKETRIDE_*
var as the run's caller env. The engine applies caller env last, over the
org/team/user secrets from get_merged_env, so on a deployed engine the
server's own ROCKETRIDE_* vars overrode a user's same-named secrets for
runs started via POST /task.
Apply the rule MCP already uses (76708650a): the client is built with
env={} unless the server's configured bind host is loopback
(ai.modules.mcp.auth.is_loopback_bind, falling back to the same
CONST_DEFAULT_WEB_HOST WebServer binds with). Loopback binds keep the SDK
default so a local developer's .env still applies. The MCP import is
deferred so loading task_http doesn't pull in the MCP package.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(mcp): close the credential-less auth hole and harden the MCP boundary (#2315)
Addresses the four review findings on PR #2315.
1. Log the resolved bind mode at startup. `is_loopback_bind` judges the
CONFIGURED host and `CONST_DEFAULT_WEB_HOST` is 'localhost', so a
deployment that never sets `host` silently runs as a local engine.
`is_loopback_bind` is unchanged (as the reviewer asked); instead the first
lines of the log now state the mode, the host it was judged from, and what
that grants (dev-bypass eligibility, `send_files`, caller-env forwarding),
with a WARNING when the bind is loopback only by fallback.
2. Refuse credential-less MCP requests. A public route pattern matching an
MCP descendant (`/mcp/{path}`) makes AuthMiddleware skip the request while
the Mount still forwards it to `handle_mcp`, where `authorize()` used to
return None and run tools on the shared, server-identity engine client.
Both ends are fixed: `authorize()` now refuses a request with no credential
unless the loopback dev bypass is genuinely active, and the startup guard
probes representative descendant paths through `is_public_route`, not just
`/mcp` and `/mcp/`. The bypass rule itself lives in one place --
`auth.dev_bypass_active` -- used by both `initModule` and `authorize`.
3. Require encrypted engine transport on a non-loopback bind. `handle_mcp`
hands the caller's credential to `WsEngineClient`, which sends it in the
first DAP `auth` message, so a `ws://`/`http://` engine URI (explicit or
derived from the MCP resource identifier) now fails boot with a message
naming the value and the variable it came from. Loopback binds keep
cleartext; the https staging/prod defaults are unaffected.
4. Reconcile MCP_TOOL_CONTRACT.md with the registrations. The tables listed
~23 of the 33 registered tools: `resolve_config` and `scaffold_node` were
undocumented and grouped capability rows hid individual tools. Every tool
now has its own row derived from its real schema and result, and the stale
`origin/develop @ eb67ddea` pin is replaced with what is accurate now.
Test-first: 27 new tests fail against the previous implementation. Three
existing tests encoded behaviour findings 2 and 3 deliberately change and were
updated; the WebServer-backed route tests now clear ROCKETRIDE_URI /
MCP_RESOURCE_IDENTIFIER after construction so a developer's dist/server/.env
no longer decides what they resolve.
717 passed, 0 skipped (was 683).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(mcp): key the startup guards on patterns and targets, not probes and binds (#2315)
Three follow-ups to the review findings on PR #2315.
1. The startup guard now inspects the registered public PATTERN STRINGS, not
just probe literals. Public entries are compiled with `compile_path`, so a
converter-typed pattern (`/mcp/{id:int}`, `/mcp/{x:uuid}`) narrows the regex
to values no fixed probe satisfies -- the guard missed it while
AuthMiddleware still skipped auth for every `/mcp/<digits>` the Mount
forwards. `_registered_public_patterns` reads `_public_paths` (real
WebServer) and `public` (test double), the same pair the dev bypass already
writes to, and degrades to nothing when neither is present.
`_claims_mcp_pattern` normalises (whitespace, trailing slashes) and refuses
only a whole-segment match, so `/mcp`, `/mcp/` and `/mcp/{...}` trip while
`/mcpx` and `/mcp-server` do not. Message shape and per-claimant naming are
unchanged.
2. The cleartext refusal now keys on the engine URI's TARGET HOST rather than
the bind. A credential sent to `ws://127.0.0.1:5565` never reaches a wire
anyone can tap, and the bind-keyed rule was a deployment hazard:
`WebServer.__init__` loads `dist/server/.env`, `.env.template` ships
`ROCKETRIDE_URI=http://localhost:5565`, and `docker/Dockerfile.engine`
copies `dist/server/` into the image, so a pod carrying that file would have
failed to boot. `auth.is_loopback_bind` judges a configured bind string
against exact literals and does not fit a URI netloc (userinfo, port, IPv6
brackets, the whole 127.0.0.0/8 range), so a small `_targets_loopback_host`
helper was added next to the rule instead of bending it; it still reuses
`auth.LOOPBACK_HOSTS` for the by-name allowlist. Non-loopback targets fail
boot as before, with the same redaction.
3. `MCP_TOOL_CONTRACT.md`'s `scaffold_node` row described only
`local_nodes/<name>/...`, so a contract-following client omitted the
`local_nodes/__init__.py` package marker `_scaffold_node` also returns and
the engine could not import `local_nodes.<name>`. The row now names the
marker and says to write every returned path; `docs/public/mcp/http/tools.md`
carried the same omission and is corrected too. The engine-module doc now
states the target-keyed transport rule.
Test-first for 1 and 2: 46 new tests fail against the previous implementation.
No existing test encoded the old bind-keyed rule (every refusal case already
used a non-loopback target), so none needed updating.
763 passed, 0 skipped (was 717).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(mcp): refuse a cleartext engine URI on every bind, not just a public one (#2315)
Closes the gap the previous commit left in `_resolve_engine_uri`. That commit
moved the cleartext refusal onto the engine's TARGET host
(`_targets_loopback_host`), but the explicit branch still guarded the call with
the BIND:
if explicit:
if not loopback:
_refuse_cleartext_engine_uri(...)
so a loopback-bound MCP server with `ROCKETRIDE_URI=ws://engine.remote:5565`
skipped the check entirely and sent the CALLER's credential over the network in
the clear -- `handle_mcp` binds it to the per-request engine client and
`WsEngineClient` puts it in the first DAP `auth` frame. The bind says nothing
about whether the wire is exposed; only the target does.
`_refuse_cleartext_engine_uri` now runs for every explicit URI, on any bind,
with the same `rocketride_uri` / `ROCKETRIDE_URI` source naming and the same
redaction. Loopback TARGETS stay allowed inside the helper, so the shipped
`dist/server/.env` (`ROCKETRIDE_URI=http://localhost:5565`) still boots. The
local `loopback` variable is gone: `auth.is_loopback_bind` is now read inline
where it genuinely selects the DEFAULT URI, so no bind-keyed condition is left
anywhere near the encryption rule. The `loopback default` branch resolves
`ws://127.0.0.1:<port>`, a loopback target, and the `public default` branch
already called the helper.
Docs and docstrings that stated the old bind-keyed rule now say the check keys
on the target host: `_resolve_engine_uri`, `initModule`,
`_refuse_cleartext_engine_uri`, and `docs/development/engine/mcp-module.md`.
`docs/public/mcp/http/self-hosting.md` never stated the rule at all even though
it fails an operator's boot, so its guard list now carries it.
Test-first: 8 new cases fail against the previous implementation (explicit
`ws://` and `http://` remote targets on `127.0.0.1` / `localhost` / `::1`
binds, from env and from config, redaction included), plus explicit `wss://`
accepted on any bind and the loopback-target cases extended over loopback
binds. Two existing tests used an explicit remote `ws://` URI incidentally --
one for dropper-link origins, one for `engine_origin` threading -- and now use
`wss://`; neither was asserting the transport rule.
794 passed, 0 skipped (was 763).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
* fix(ai): accept the single-component validation payload the node panel sends (#2346)
* fix(ai): accept the single-component validation payload the node panel sends
The node config panel validates one component on save and sends
`{version, component}` — the `IComponentValidatePayload` shape the shell
contract declares this endpoint accepts (shell/src/types/project.ts:308). The
engine validates pipelines and requires `components`, so every save answered
"'pipeline.components' must be an array", whatever the node.
The component now travels as a one-item `components` list, built before the
source is inferred so `resolve_implied_source` sees it. A config that already
carries `components` is untouched.
Reachable since #1573: until then `rrext_validate` failed on every call with
"'pipeline' is missing or invalid", so this payload never reached the
validator. The panel has sent this shape since #1843 (7 Aug).
Removing the four lines fails the two new tests, and the engine itself answers
the reported message:
validatePipeline({'pipeline': {'version': 1, 'component': c}}) -> "'pipeline.components' must be an array"
validatePipeline({'pipeline': {'version': 1, 'components': [c]}}) -> 0 errors
* docs(clients,ai): document the single-component validate payload
The endpoint now accepts {version, component} as well as a full pipeline, which
is a public-contract change: both SDK references say so, and on_rrext_validate's
Args: block does too — the inline comment explained it, but the docstring is
what a reader hits first.
Also states that a payload carrying both keys keeps 'components' and drops the
stray 'component', so the next reader does not have to derive that it is
deliberate.
* fix(crustdata): patch HTTP transport by object in tests (#2332)
Co-authored-by: Krish Garg <kg@g14-node1.tailb99b1b.ts.net>
* fix(notion): normalize malformed success responses (#2340)
Co-authored-by: Krish Garg <kg@g14-node1.tailb99b1b.ts.net>
* fix(notion): preserve query request status (#2339)
Co-authored-by: Krish Garg <kg@g14-node1.tailb99b1b.ts.net>
* fix(search-hybrid): isolate fallback identities (#2329)
* fix(search-hybrid): isolate fallback identities
Closes #2322
* test(search-hybrid): cover empty-ID fallback
Addresses CodeRabbit review on #2329
---------
Co-authored-by: Krish Garg <kg@g14-node1.tailb99b1b.ts.net>
* fix(notion): accept search pagination cursors (#2338)
* fix(notion): accept search pagination cursors
* test(notion): cover empty search cursors
---------
Co-authored-by: Krish Garg <kg@g14-node1.tailb99b1b.ts.net>
* fix(ai,nodes): surface database error text from execute and add refresh_schema tool (#2283)
* fix(ai,nodes): surface database error text from execute and add refresh_schema tool
Two gaps in the shared database node base, both scoped to stay out of the
way of the pull requests already open against this file.
_executeRawQuery caught every SQLAlchemyError, logged it, and returned
None, so execute() answered with the same 'check server logs for details'
string whether the statement had a typo, named a missing table, or was
refused for permissions. Its except clause now raises with the driver's own
message through IGlobal._format_db_error. Only that clause changes:
execute()'s `if result is None` branch becomes unreachable but is left in
place, with a comment saying so, because an open PR is rewriting that
region. Session-bound execute already re-raised the real exception, so the
two paths finally agree, and the max_execute_rows rollback is untouched.
IGlobal.db_schema is assigned exactly once, in beginGlobal, so get_schema
serves a task-start snapshot for the life of the task and a table created
through execute stays invisible until the pipeline restarts. The new
refresh_schema tool re-reflects and replaces it, returning the get_schema
shape plus a refreshed_at UTC timestamp so a caller can show how current
the schema it holds is. Reflection and publication run under a module-level
lock: concurrent callers would otherwise repeat the same full table walk
and race to publish the result. The tool is additive and lives entirely in
db_instance_base.py, since another open PR is adding to db_global_base.py.
It also fixes the natural-language path, which describes that same dict to
the LLM. get_schema's nested table formatter is lifted to a shared private
method so both tools render one shape.
Tests are a new module rather than an addition to test_db_base.py, which
two open PRs are both appending to. Eleven cases over an in-memory SQLite
engine cover the driver text reaching the caller, the gate still refusing
first, re-reflection seeing created and dropped tables, the timestamp
format, and four concurrent refreshes all returning.
Tool tables in the four node READMEs that inherit this surface list
refresh_schema and say in one sentence what get_schema's snapshot does not
show. services.json declares no tool list, so nothing to mirror there.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T4fLnswQF6yKQ6x7eWzSdd
* test(ai): give the execute-session IGlobal stub a real _format_db_error
execute()'s stateless failure path now formats the driver error through
IGlobal, but this module builds IGlobal as a SimpleNamespace carrying only
the four fields the transaction tests needed. Nothing failed, because no
test here drove a bad statement through that path — the next one to try
would have died with AttributeError instead of the RuntimeError it was
asserting.
The stub borrows the real DatabaseGlobalBase implementation rather than a
lambda, so the message a test sees is the message a caller gets. A test for
the stateless failure path comes with it, so the fixture stays honest:
removing the one line makes it fail with AttributeError.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T4fLnswQF6yKQ6x7eWzSdd
* refactor(ai): shrink the refresh_schema diff in db_instance_base
No behaviour change; this only narrows what the previous commit touched in
a file three open pull requests are editing.
get_schema's body is back to exactly what develop has, docstring aside: the
nested _format_table closure returns, and the _format_table static method
and _schemaPayload helper it had been lifted into are gone. Sharing them
was tidier, but it rewrote a method none of the open PRs needs to change,
and a few duplicated formatting lines cost less than that. That hunk is now
five added docstring lines instead of a 31-line rewrite.
refresh_schema moves to the end of the tool section, below dialect, so the
whole tool arrives as one appended block past the execute body PR #1908
rewrites rather than as an insertion ahead of it. It formats its own tables
inline, and the lock is now _REFLECT_LOCK.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T4fLnswQF6yKQ6x7eWzSdd
* docs(nodes): complete the agent tool contract in the db READMEs
Two review findings on the READMEs this PR touched, both in the section
the node README schema calls the agent-facing contract and holds to the
highest accuracy bar.
db_postgres listed four tool functions in the summary and in "As a tool",
then a separate table of three transaction tools — seven of the nine
@tool_function methods an agent actually sees on DatabaseInstanceBase.
`execute` appeared only as prose inside the transaction paragraph and
`dialect` only as the QuestionType.DIALECT lane behaviour, so an agent
author reading the inventory would not know either exists as a tool. The
count is now nine in both places, `dialect` joins the open table, and
`execute` joins what was the Transactions table, renamed "Raw SQL and
transactions" because a statement without a session_id is neither. That
table's gate sentence also said requests are "silently dropped" when
allow_execute is off; the four tools raise (db_instance_base.py:301,
339, 371), and it is the questions lane that logs and drops — the two
behaviours are now stated apart. The `execute` row records the driver
message and the rollback-on-overflow that this PR's own change produced,
rather than implying a truncated result set.
rocketride_sql named refresh_schema in its tool table but left its
contract out of the prose beneath, where that README already documents
every other non-obvious argument and return shape. It takes no arguments
(input_schema declares no properties) and returns {database, tables}
exactly as get_schema does, plus a refreshed_at UTC ISO-8601 timestamp
from datetime.now(timezone.utc).isoformat().
Docs only; no behaviour change. Both nodes still pass
scripts/validate-node-readme.py, and so does the whole corpus under
./builder docs:validate.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(ai): keep SQL statements and bind parameters out of execute error text
`_format_db_error` only produced a clean `Error <code>: <msg>` when the
driver put an integer first in `orig.args` — the pymysql and
clickhouse-driver shape. psycopg2 and sqlite3 put a string there, so both
fell through to `str(exc)`, which for a SQLAlchemy StatementError is the
full repr: the executed statement in `[SQL: ...]` and the values bound
into it in `[parameters: ...]`.
That string is no longer log-only. `_executeRawQuery` re-raises it as a
RuntimeError that the `execute` tool returns to its caller, and #2282
wires a user-facing Refresh/Run UI onto exactly that path, so a failed
INSERT would hand the caller back the row it tried to write.
Unwrap each driver shape explicitly and keep only the primary message:
the numeric-code branch as before; psycopg2's `diag.message_primary`
(its `str()` also carries a `LINE n:` echo of the statement, which
psycopg2 has already interpolated the bind values into); and `args[0]`
for sqlite3 and the generic DBAPI shape. Every branch then runs through
`_strip_statement_detail`, which truncates at the first `[SQL:`,
`[parameters:`, `LINE n:`, `DETAIL:`/`HINT:`/`CONTEXT:` or sqlalche.me
marker, so a driver shape not enumerated here still cannot leak the
tail. DETAIL is dropped along with the rest because PostgreSQL uses it
to restate the offending key values.
The helper is module-level rather than a method so `_format_db_error`
keeps working when tests bind it onto a stub IGlobal.
Tests cover the real sqlite3 driver end-to-end through `execute`, plus
the psycopg2 (with and without `diag`), sqlite3 and pymysql shapes and a
bare StatementError repr.
`test_format_db_error_falls_back_to_str_when_args_not_int_first` pinned
the old leaking fallback, so it is replaced by an assertion that the
driver message — not `str(exc)` — is what comes back.
`_strip_statement_detail`'s degenerate case gets the same treatment. A
message that is nothing but detail markers left no primary sentence, and
the old fallback returned the first line — which on that input is the
`[SQL: ...]` echo the function exists to remove. It now returns a
neutral constant, so there is no input on which the helper hands back the
statement.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(ai): refresh the answers-lane column map in refresh_schema
`refresh_schema` replaced `IGlobal.db_schema`, the database-wide
reflection the natural-language path describes to the LLM, and stopped
there. `IGlobal.schema` — the configured table's column map that
`_insertData` iterates to build every answers-lane INSERT — is a second
start-up snapshot, and it was left untouched, rebuilt only lazily when
falsy.
So the node was current on one path and stale on the other: after
running `ALTER TABLE ... ADD COLUMN` and calling the refresh tool, the
LLM path saw the new column while the insert lane kept writing the
start-up column set and silently dropped it. The four db node READMEs
document both the insert lane and "re-reads the schema from the
database", so the documented contract overstated the code.
Invalidate the map inside the same reflection lock. Emptying it re-arms
the lazy rebuild already at the top of `_insertData`, which reflects
through the same `_getTableSchema` call `beginGlobal` uses, so the next
insert sees what a freshly started node would. Emptying rather than
re-reflecting here keeps the call cheap for a node with no answers lane
wired, and makes the write a single atomic rebind: a concurrent
`_insertData` either reads the old map or finds it falsy and rebuilds,
never a half-built dict.
Invalidating the map is only safe once `_insertData` stops binding a
generated primary key, so that is fixed in the same commit.
`_createTableFromData` curates `IGlobal.schema` down to the data columns
for an auto-created table, because the `id` it prepends is generated by
the database; a map built by reflection carries `id` instead. Since
`_insertData` binds NULL for any schema column the rows do not supply,
swapping the curated map for a reflected one would have started binding
`id=None` on every later insert — tolerated by SQLite's rowid alias, a
not-null violation against the `id SERIAL NOT NULL` Postgres renders for
the same column. `_insertData` now drops primary-key columns that no row
in the batch supplies, decided once per batch so every mapping handed to
executemany keeps an identical key set. The two maps therefore produce
the same INSERT and refreshing cannot change answers-lane behaviour
mid-task. It also fixes the same NULL bind on the path that always
reflected: a table that already existed when `beginGlobal` ran.
Tests cover the refreshed insert picking up a new column, the
pre-existing stale behaviour it fixes, a dropped configured table leaving
a falsy map, and — by asserting on the compiled INSERT rather than on
what SQLite tolerates — that an auto-created table's insert still carries
the data columns only after a refresh, that a reflected primary key the
rows do not supply is omitted, that one they do supply is still bound,
and that a primary-key-only table falls back to binding it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* docs(nodes): describe the raw-SQL gate the db nodes actually have
The `Raw SQL and transactions` section said `allow_execute` is "the same
gate as `QuestionType.EXECUTE`" and contrasted the tools with "the
`questions` lane, which logs and drops the request". No SQL database
node implements that branch: `DatabaseInstanceBase.writeQuestions` does
not look at `question.type` at all, and `QuestionType` appears in
db_instance_base.py only at the import and in one unrelated
`Question(type=QuestionType.QUESTION, ...)` construction. The
dispatch-on-type behaviour belongs to the graph node
(`graph_instance_base.py` handles `DIALECT` and `EXECUTE`), and the
wording was carried across without being checked against this file.
Correct it wherever it appears in prose:
- db_postgres: the "Two special question types are handled on the
questions lane" bullets are removed — the lane has one behaviour,
translate-then-execute, and `dialect` / `execute` are the tool
functions that reach those behaviours. The gate sentence now says
what `allow_execute` gates (the four raw-SQL tools) and that it does
not change the questions lane, which never runs raw SQL.
- rocketride_sql: same two claims, in the config and Notes sections.
Also make the refresh_schema wording match the code now that the tool
invalidates the answers-lane column map as well as the database-wide
schema, so "re-reads the schema from the database" is true end to end
rather than only for the LLM path.
The same claim lived in the generated config tables, whose source of
truth is the `allow_execute` field description in each node's
services*.json. Those five descriptions now name the four tool functions
the flag actually gates, and the generated regions are regenerated from
them, so db_postgres/README.md no longer contradicts itself between its
prose and its parameter table. Only the SQL nodes are touched: the graph
nodes do dispatch on `QuestionType.EXECUTE`, so their copy is correct as
it stands.
Two smaller instances of the same transposition: the new db_postgres
sentence named `db_neo4j`, a retired path that redirects to
`graph_neo4j`, and db_clickhouse/IInstance.py said its dialect string is
"surfaced via QuestionType.DIALECT" when only the `dialect` tool reaches
it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(ai): apply the same execute error contract to the session path
A raw statement run with a session_id went through
TransactionRegistry.execute, which has no try/except, and the existing
`except Exception` arm in execute() re-raised the SQLAlchemy exception
unchanged. str() of that exception ends in `[SQL: ...]` and
`[parameters: ...]`, so the same tool behind the same allow_execute gate
returned the statement and its bound values to the caller purely because
a session id was passed, while the sessionless half stripped them.
Add a `SQLAlchemyError` arm above the existing `except Exception` (the
max-rows RuntimeError is not a SQLAlchemyError, so its wording and its
rollback are untouched): log the full exception, release the session, and
raise `RuntimeError('SQL execution failed: ' + _format_db_error(e))`.
The fix lives in execute() rather than tx_registry because the registry
has no IGlobal and is shared with other callers.
Session release moves into _releaseFailedSession, shared by both arms.
A rollback that itself fails is logged and swallowed: previously an
exception raised inside the cleanup replaced the error the caller was
being told about (a monkeypatched rollback surfaced as 'boom').
Also drop the dead `result is None` branch in execute() and correct
_executeRawQuery's annotation to `-> dict`: the method has returned the
shaped dict or raised since it started carrying the driver's message, so
the branch could only ever have produced the opaque 'check server logs
for details' string this work exists to replace, and the annotation
contradicted the comment above the branch. execute() is its only caller.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* refactor(ai): share one table formatter between get_schema and refresh_schema
refresh_schema rebuilt get_schema's nested _format_table line for line,
with `result` renamed to `entry`. Both tools promise callers the same
per-table shape -- their tool descriptions say so, and an LLM picks
between them on that sentence -- but the equality test compared the two
copies to each other, so it would have stayed green while a later change
added a key (an index list, a column comment, a nullable flag) to one
copy only.
Lift the formatter to a module-level _format_table and call it from
both; get_schema's body loses the nested definition, refresh_schema
loses the loop. The wire format is byte-identical, so the existing shape
tests pass untouched. The docstring paragraph calling the duplication
deliberate goes with it.
test_refresh_schema_matches_get_schema_shape now reflects a table with
both a primary key and a foreign key, so the two optional entries are
actually exercised on both sides, and _format_table gains a direct unit
test for the optional-key rules an equality check cannot see.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* fix(ai): make the lazy schema rebuild safe for concurrent inserts
refresh_schema empties IGlobal.schema to re-arm the rebuild at the top of
_insertData, which is what makes this reachable at runtime: a second
insert can now arrive while the first is reflecting.
_getTableSchema published `self.schema = {}` and then grew it column by
column, and _insertData checked, rebuilt and then took a reference to
that same dict without a lock. A concurrent insert could therefore see
the empty map (falling into the "no schema cached" branch, which binds
the item's raw keys and skips the generated-key handling), or a truthy
but half-built one (silently dropping columns the caller had supplied),
or a dict changing size while the per-row loop iterated it. A reflection
that failed midway left the half-built map published for the next insert
to believe.
_getTableSchema now builds into a local dict and publishes it with one
assignment, so a failure leaves the previous map untouched. _insertData
holds _REFLECT_LOCK across the falsy check, the rebuild and a snapshot
copy, and builds the batch from the snapshot; the lock is released before
the Table reflection and before the INSERT, neither of which reads the
map. The comment above refresh_schema's invalidation no longer claims
the single rebind makes the rebuild atomic.
Tests hold a rebuild open inside the real column walk rather than
simulating the window: without the lock the second insert stored NULL in
a column it had supplied, an overlapping refresh left a one-column map,
and a failed reflection published a partial one.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* fix(ai): decide generated primary keys per row, not per batch
The answers lane worked out which primary-key columns to leave unbound
once per batch, from the union of every row's keys. One row carrying
`id` therefore put `id` into every mapping, and the rows that omitted it
bound NULL into a key the database was supposed to generate. SQLite
tolerates that for a rowid alias; Postgres rejects the whole batch for an
`id SERIAL NOT NULL` column. The sole-primary-key table had the same
problem by construction: the `omit_columns = set()` fallback bound NULL
because omitting everything would have left an empty mapping.
Which primary keys the database generates is now read from the reflected
Table -- `autoincrement_column` (SQLAlchemy's own resolution, covering a
reflected `autoincrement=True` and the `'auto'` rule that recognises
SQLite's rowid alias), plus any primary-key column with a server_default,
an Identity, or an explicit autoincrement -- rather than assumed of every
primary key. Per row: a generated key the row omits is left out of that
row's mapping, a supplied value is bound, and an explicitly supplied None
is bound as NULL, so the three cases stay distinguishable. A primary key
the database does NOT generate and the row does not carry is a ValueError
naming the table, the columns and the row position, raised before the
transaction opens.
Rows are then executed as contiguous runs sharing a key set, all inside
one engine.begin(): contiguous rather than gathered, because generated
ids follow insertion order and regrouping would hand back ids in an order
the caller's rows never had, and one transaction because a batch must
stay all-or-nothing as a single executemany was. A run whose mapping is
empty gets a values-less `insert(table)` per row, which SQLAlchemy
renders as `DEFAULT VALUES` on PostgreSQL and SQLite and `() VALUES ()`
on MySQL.
test_insert_binds_the_primary_key_when_it_is_the_only_column pinned the
old fallback and is replaced by the default-values test, plus a
compile-only check of all three dialect renderings (no live PostgreSQL or
MySQL in this suite).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* test(ai): pin the caller-facing error policy for Postgres and MySQL
CodeRabbit's security finding asks for a generic message or a stable
category instead of the driver's own words. That is not adopted: the
maintainer's review endorses the primary-message contract precisely
because "no such column: foo" is what lets a caller fix their statement.
What was missing was the policy written down and held in place by tests.
_format_db_error's docstring now states the residual it accepts: a
driver's primary sentence may quote a value from the statement being
reported (Postgres `invalid input syntax for type integer: "abc"`, MySQL
`Duplicate entry 'x' for key 'users.email'`). That value was submitted by
the caller now reading the message, who also holds allow_execute; values
belonging to other rows appear only in DETAIL / HINT / CONTEXT, which are
stripped. Blanking quoted tokens generically is rejected because MySQL
quotes identifiers with single quotes too, so the redaction would delete
"which column" from the message that most needs it.
The tests build real DBAPIError wrappers via DBAPIError.instance, so
str(exc) genuinely carries the `[SQL: ...]` / `[parameters: ...]` tail and
the assertions bite: a Postgres duplicate-key whose DETAIL restates the
email, the same without .diag so the LINE echo is exercised, the MySQL
duplicate entry that documents the accepted residual, and the MySQL
unknown-column that documents why redaction is refused. One more drives
both shapes through BOTH execute paths and asserts the two messages are
identical -- against the pre-fix source the session-bound half raised the
raw DBAPIError with the bound 'ada@example.com' attached.
Mock-based by necessity: no live PostgreSQL or MySQL in this suite, so
the driver exception shapes are reproduced rather than provoked. No
behaviour change.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* docs(nodes): correct the lane, cache and error prose in the four SQL READMEs
All four checked against the node code and services.json rather than
against each other.
db_clickhouse declares only a `questions` lane (services.json
`lanes: {questions: [...]}`), so the sentence about schema refresh
reaching the `answers` insert lane described a lane the node does not
have; it is removed. db_mysql, db_postgres and rocketride_sql do declare
`answers`, so theirs stays — reworded, because refresh_schema
invalidates that column map rather than rebuilding it, and the rebuild
happens on the next insert.
"Return the schema reflected when the node started" was wrong in all
four after refresh_schema shipped: get_schema serves whatever snapshot
the node currently holds, which is the start-up reflection until a
refresh replaces it.
db_postgres said every question on the `questions` lane is translated to
SQL and executed. When the LLM reports the question is not a database
query (`isValid: false`), writeQuestions emits its prose answer and
nothing runs against the database, so the lane is described as the
natural-language path with only valid generated SQL executed.
Each README now states the execute error contract the code enforces: a
failure raises `SQL execution failed:` plus the driver's primary
message, identically with and without a session_id, with the statement
and bound parameters kept to the server log — and with the residual
named, since a driver's primary sentence can quote a value the caller
itself submitted.
scripts/validate-node-readme.py passes for all four nodes; no GENERATED
block touched.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* fix(ai): publish the auto-created table's column map in one assignment
_createTableFromData is the second writer of IGlobal.schema and still had
the publish-then-fill shape B4 removed from _getTableSchema: it created
the table, assigned `self.schema = {}`, then added one key per column,
all outside _REFLECT_LOCK. By the time that loop runs the table exists,
so a second first-batch can already be inside _insertData; it takes the
lock, finds a truthy one-column map, snapshots it and silently drops
every other column its row carried. Reproduced by pausing the real fill
loop: the second row landed as {'label': 'b', 'size': None,
'colour': None} for input that supplied all three.
The map is now built in a local dict and published in a single
assignment. The lock is NOT widened over table creation, so the
pre-existing double-create race is untouched and still out of scope.
Also seeds a stale map into a direct _getTableSchema failure test: the
existing rebuild-failure test starts from {}, which cannot tell "previous
map left untouched" from "map reset to {}". Against the pre-B4 source
that new test fails with the stale entry replaced by a partial one.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* fix(ai): cut the clickhouse stack trace and position echo from execute errors
The formatter's docstring claimed "pymysql / clickhouse-driver put
(errno, message) in .orig.args". Only pymysql does. clickhouse-driver's
DBAPI layer raises OperationalError(ServerException), so .orig.args holds
one exception object, every branch falls through to str(orig), and what
reached the caller was the driver's full string form: "Code: N.\n
DB::Exception: <message>. Stack trace:\n\n0. DB::Exception::Exception(...)
@ 0x..." — server internals the caller cannot act on — plus, for a syntax
error, the failing statement fragment quoted after "failed at position N
('...')", which is the same class of statement echo as psycopg2's
"LINE n:".
_DB_ERROR_DETAIL gains "Stack trace:" and "failed at position" as
markers, so both are cut by the stripper that already handles the other
drivers rather than by a new branch. Neither phrase can misfire on the
shapes the other drivers produce.
The docstring now states the real clickhouse-driver shape instead of
claiming the pymysql one. Two constructed tests cover it, labelled in
their docstrings as reproduced from the driver's documented behaviour:
clickhouse-driver is not installed in this engine, so the shape cannot be
provoked here.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* docs(ai): state the real execute error contract and schema-cache wording
Three corrections to text, no behaviour change.
The error-contract paragraph overclaimed. It said values belonging to
other rows appear only in DETAIL / HINT / CONTEXT, which are stripped.
Both halves of that are wrong: a real SQLite run returns
`SQL execution failed: near "'hunter2'": syntax error` — the statement
fragment the parser stopped on is in the primary sentence — and
PostgreSQL's `invalid input syntax for type integer: "<value>"` names a
row value an INSERT ... SELECT or a CAST merely touched, which can come
from another table. pymysql and psycopg2 interpolate binds client-side,
so such a fragment can carry a bound value too. The docstring now lists
what is removed (the SQLAlchemy tail, LINE n:, DETAIL/HINT/CONTEXT/
QUERY/STATEMENT, the ClickHouse stack trace and position echo) and what
is passed through, and rests the policy on the argument that actually
holds: the recipient has allow_execute and can already read the same data
with a SELECT, so the error channel does not widen read access. Two tests
(one per execute path) pin the syntax-error shape on a real SQLite run.
get_schema's docstring said the schema is "reflected once in beginGlobal"
and refresh_schema's tool description — the sentence an LLM reads when
choosing between the two tools — said get_schema serves the snapshot
reflected when the node started. Both became false the moment
refresh_schema shipped, and both contradicted the READMEs corrected for
exactly that in c258211d.
_generated_primary_keys gains a paragraph recording the SQLite-only
residuals of SQLAlchemy's 'auto' rule: INT/BIGINT/INTEGER-DESC primary
keys are treated as generated although SQLite does not alias them to the
rowid, and an INTEGER PRIMARY KEY that is also a foreign key is treated
as required although it is an alias. PostgreSQL and MySQL reflection set
autoincrement explicitly, so production nodes resolve exactly.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* docs(nodes): correct the error-contract and remaining schema wording
Two corrections, both to sentences this PR itself introduced or left
behind.
The error sentence in all four READMEs promised that "the statement text
and the bound parameter values are never part of it". That is not what
the code does: a real run returns `SQL execution failed: near
"'hunter2'": syntax error`, and PostgreSQL's `invalid input syntax for
type integer: "<value>"` names a value an INSERT ... SELECT or a CAST
read from another table. Each README now says what is actually removed
(SQLAlchemy's `[SQL:]`/`[parameters:]` echo, and per engine: PostgreSQL's
LINE n:/DETAIL/HINT/CONTEXT, ClickHouse's server stack trace and its
`failed at position` echo) and that the primary sentence is passed
through as the database wrote it, so it can name a value the statement
carried or touched — resting the policy on the argument that holds, that
reaching this tool requires Allow direct query execution and such a
caller can read the same data with a SELECT. The full text stays in the
server log. db_postgres moves that contract out of the already-long
`execute` table cell into prose below the table, as the other three do.
The intro sentences in db_clickhouse and db_mysql still said
"startup-reflected schema", the wording corrected everywhere else in
c258211d; both now describe the cached snapshot and name refresh_schema.
scripts/validate-node-readme.py passes for all four; no GENERATED block
touched.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* fix(ai): leave every database-owned column to the database on the insert lane
Two ways an explicit NULL still reached a column the database was meant
to fill, both silent because writeAnswers only logs, and one comment that
claimed neither could happen.
_generated_primary_keys skipped primary-key columns only, so any other
newly reflected column was bound NULL — and an explicit NULL overrides a
server default. The sequence the refresh tool invites: answers is
auto-created as (id, q, a) with the curated map {q, a}; an agent runs
ALTER TABLE answers ADD COLUMN created_at ... NOT NULL DEFAULT now(); the
tool description tells it to call refresh_schema, so the next batch
rebuilds the map by reflection, binds created_at = NULL, and the batch
that worked a moment earlier is dropped with one log line. Reproduced on
SQLite: NOT NULL constraint failed: answers.created_at, parameters
('how', 'like this', None). _insertData now also reads
generated_defaults — every column with a server_default or an identity —
off the reflected table and omits those the row does not supply.
An explicit null on a generated key was bound as NULL. On the execute
tool that is right: a person wrote the statement. On the insert lane the
caller is an upstream node, and an LLM node or a JSON mapper emits every
schema key, writing null for the ones it has no value for — the same
intent as omitting it. Postgres rejects that NULL for a SERIAL column;
MySQL and SQLite generate a value, so the failure is dialect-dependent as
well as silent. A supplied null on a generated primary key is now treated
as omitted; everywhere else it is still bound as given.
The comments at refresh_schema's invalidation and above the binding loop
said the invalidation cannot change answers-lane behaviour. This PR's own
test shows it can, and that half is the point of the tool. They now state
the guarantee that actually holds: the columns present only in the
reflected map are omitted rather than bound NULL, so the database still
fills in what it owns.
Also drops the unreachable "no schema cached" branch: after the locked
block IGlobal.schema is populated or the method has raised, so the
snapshot is always truthy. The empty-map half of the lock rationale went
with it; the half-built-map half, which the concurrency test proves,
stays.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* fix(ai): format driver errors the dialect did not wrap in a SQLAlchemy exception
The error contract this PR documents was not delivered on the ClickHouse
node at all. db_clickhouse connects with `clickhouse+native://`, which is
clickhouse-sqlalchemy's own connector; its Cursor.execute raises
`clickhouse_sqlalchemy.exceptions.DatabaseException`, a plain Exception
subclass carrying the driver's ServerException in `.orig`. SQLAlchemy
does not wrap a non-DBAPI exception, so `except SQLAlchemyError` never
fired on EITHER execute path, `_format_db_error` never ran, and the
caller got `Orig exception: Code: 62.\nSyntax error: failed at position
21 ('hunter2') ... Stack trace:\n\n0. DB::Exception::Exception(...)` —
raw, on both paths. The markers added in 27a04f68 were verified against
clickhouse-driver's own DBAPI layer, which this node does not use, so
they were unreachable for the node they were added for.
Both paths now handle a non-SQLAlchemy exception that carries a driver
`.orig`: log it and raise the same `SQL execution failed: <formatted>`
RuntimeError, releasing the session on the session path as before. Duck
-typed on `.orig` rather than importing clickhouse modules into the
shared base — a driver error carrying its original is precisely the shape
being handled, and the base must not depend on one node's driver.
Anything without `.orig` is re-raised untouched, so the max_execute_rows
RuntimeError keeps its wording and rollback and a programming error still
surfaces as itself.
_format_db_error also reads `.code` and `.message` off the original when
they are an int and a string: clickhouse-driver's ServerException puts
the code there rather than in args, so the numeric branch missed it and
the caller lost the code. ClickHouse now reads like pymysql —
`Error 60: Table default.widgets doesn't exist` — and the message still
runs through the stripper, so the stack trace and the `failed at
position` fragment inside it are cut.
The two existing clickhouse formatter tests are relabelled: they cover
clickhouse_driver.dbapi's shape, which this node never produces. New
tests cover the connector's real shape at the formatter and drive it
through both execute paths, asserting no stack trace, no statement
fragment, and a released session; two more assert that an exception
without `.orig` is not swallowed. All labelled as constructed shapes —
clickhouse-sqlalchemy is not installed in this engine.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* fix(ai): reject a malformed params argument instead of misreporting it
`to_sqlalchemy_text` turns `$n` into a bind by indexing `params[n - 1]`,
so a JSON object for `params` raised `KeyError: 0` and a short list an
`IndexError`, both from inside the rewriter. The sessionless path leaked
those bare; worse, on the session path the `KeyError` was caught by the
arm that maps a missing session id, so `execute` reported "unknown or
expired transaction session: <sid>" for a session that was still open and
still usable — the caller was told to abandon a live transaction because
they passed the wrong JSON type.
`execute` now validates before dispatching anything: `params` must be
absent or an array, and every `$n` the statement references must have a
value, with the offending placeholder named. tx_registry.py is untouched
— the check lives in `execute` and imports the placeholder pattern from
the module that owns it so it cannot drift from the rewriter it guards.
An empty list keeps its current meaning of "no binds", the same as
omitting `params`, because a statement carrying a literal `$n` inside a
string body relies on the rewriter leaving it alone.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* docs(nodes): document the insert-lane null rule and the clickhouse error layer
The three nodes that declare an `answers` lane now state, beside the
existing insert prose, what the code does with a value the row does not
carry: a column with a generated key or a DEFAULT is left out of the
statement so the database supplies it, rather than bound as an explicit
NULL that wou…
4 of 5 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This branch is the full documentation revamp, landed as one line of work: every hand-written doc consolidated into a single top-level
docs/tree, node and client docs brought under enforced schemas with builder gates, all node READMEs migrated, the Docusaurus site restyled and restructured on top of it, and (after review) a second pass that reshaped the tree around the approved layout, added user-facing Apps docs and an agent CLI page, and moved the schema checks into the builder. 637 files againstdevelop; see "Why 637 files" below.History: 39 commits on the PR line, the 29 pre-review commits kept as they map to the merged PRs below, then two develop merges and seven squashed follow-up commits (one per phase). Most of the original work was reviewed piecemeal through PRs merged into this branch (#1856, #1912, #1960–#1978, #2022, #2124, #2125, #2138). The review follow-ups (2026-09-09 to 2026-09-11) were done on the branch directly, then self-reviewed in six passes (docs content, docs pipeline, agent bundle, apps and packages, nodes and CI, tree-wide link check) with every finding fixed.
What's included
One docs tree with an export pipeline (#1856, #1969, follow-ups)
docs/, organized by audience:docs/public/(product + SDK docs;product/is the site spine and its paths are public URLs),docs/development/(contributor docs),docs/agents/context/(the agent-facingROCKETRIDE_*docs and the stubs, packed into the/client/docsbundle by./builder client-docs:agent; the only delivery path),docs/docusaurus/(the site package, moved here frompackages/docs/; site-only app pages underdocs/docusaurus/apps/<app>/).packages/client-*,packages/n8n-nodes,packages/chat-widget) are generated committed copies. Sources live indocs/public/<name>/README.md, refreshed by./builder docs:export, drift caught by./builder docs:check.absolutizeImageLinksinscripts/lib/markdown.js) rewrites them to raw GitHub URLs at the two copy sites only: the package exports and the VSIX README staging. The MCP stdio README keeps an absolute banner because it is both an export source and a site page.README.md,icon.svg, and assets live at the app root (apps/<app>/), matching what the scaffold emits; the VS Code extension owns its marketplace README and banner.Schemas, validators, and gates (#1912, #1971, #1973, follow-ups)
docs/development/nodes/readme-schema.md) and services.json contract, enforced byscripts/validate-node-readme.py; client docs schema enforced byscripts/validate-client-docs.py../builder docs:validateruns both validators across the whole corpus plus the node validator's own unit tests, blocking, insidedocs:test, which the docs workflow runs on every PR. The separate CIschemasjob is gone; the check lives in the builder.Node README migration, the whole catalog (#1970, #1974–#1978, #2022, #2124, #1960, follow-ups)
llm_glm, which arrived from develop and was migrated in the merge). The last seven stragglers and a new assembled README fortool_microsoft_365landed in the follow-ups.services*.jsondocumentationlink points at the node's own page on the docs site (175 files).ROCKETRIDE:GENERATED:PARAMSregions maintained bynodes:docs-generate.User-facing Apps docs and the agent CLI page (follow-ups)
ROCKETRIDE_CLI.md(therocketridecommand as the first choice for validate, run, store, deploy, and app work; the SDK docs' CLI sections became pointers) and the six stubs route to it.Content accuracy and restructuring
app createconnection, the developer-id gate, VS Code settings keys, and the tree-wide link check: 3 intentional leftovers out of ~840 links).docs/agents/skills/(docs(agents): land the pipeline-builder skill set in docs/agents/skills/ #2125), with its doc map repointed at the current site routes.The new site (#2138, follow-ups)
createRedirectsplus an explicit redirect list keep old URLs alive.metadataexport.Why 637 files
nodes/src/nodes/: 175services*.json(the documentation-link rewrite), 139 READMEs (the schema migration, including the newllm_glmnode from develop), 14 example bundles.packages/docs/→docs/docusaurus/anddocs/agents/*.md→docs/agents/context/.docs/public,docs/development,docs/agents), the generated package READMEs, the app listing files underapps/, two workflows, and two builder helpers.Verification (at e3931b6, after the second develop merge)
./builder docs:test(docs:validate: client-doc parity, node README corpus 139/139, validator unit tests; docs:unit),./builder docs:build,./builder docs:check: all green../builder client-docs:agent: bundle staged (17 files).Notes for reviewers
ROCKETRIDE:GENERATED:PARAMSblocks,packages/client-*/README.md,packages/chat-widget/README.md, and the pipeline reference all regenerate from their sources.main, as before this branch; images resolve oncedevelopis released tomain.🤖 Generated with Claude Code