Repository navigation
feat(ai,mcp): HTTP MCP server - #1880
Merged
Merged
Conversation
…stration Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…, load_pipeline Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…nitor Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
New HTTP-MCP tool that starts a pipeline and returns two self-contained, credential-embedded URLs for getting files in over a separate HTTP data channel (file bytes cannot ride the MCP tool call): - upload_url: programmatic multipart POST to /task/data?token=<tk_>&auth=<pk_> - dropper_url: /dropper?auth=<pk_>, a browser page for a human to drag-drop files Both carry the task's private token (tk_) and public auth key (pk_) read from the use() launch body, so no Authorization header is required. Also relaxes /task/data's required Authorization header to accept the credential via the ?auth= query (AuthMiddleware already supports it; the handler auths off request.state.account, so auth is unchanged). Adds a normalized base_url to the MCP engine seam. Live-verified: headerless multipart POST -> 200/ingested; browser drag-drop -> parsed text. Covers ingress Options 2 (pipe-then-dropper) and 3 (agent-direct-POST). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Extend tool_filesystem from a tool-only node into a dual-role tool + pipeline sink (classType ["tool","store"]). Six input lanes -- documents, text, table, image, audio, video -- normalize to a single `documents` metadata output. Each persists its payload to the account FileStore under `targetDir` (default `output/`), preserving the source object's original name (object-id fallback), deriving the extension from the name, mime type, or `.md`, and auto-suffixing on collision. Optional `emitUrl` attaches a time-limited signed download URL via FileStore.get_url (needs RR_SIGNING_KEY for the local backend). The sink reuses the existing allowWrite toggle and path whitelist -- it never bypasses them. New config: targetDir, emitUrl, urlExpiresIn. Emitted documents carry a valid DocMetadata (objectId/chunkId inherited from the source object). Verified with 20 unit tests, the full `nodes:test` contract suite, and a live end-to-end MCP pipeline (webhook -> parse -> tool_filesystem -> response) that wrote a parsed file into the store and emitted its reference. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ions
writeQuestions() now branches on QuestionType.EXECUTE before the LLM
translation path: it runs the raw query directly via _executeRawQuery,
gated by IGlobal.allow_execute and the is_sql_safe read-only whitelist,
and emits {rows, affected_rows, row_count} on the answers/table lanes.
db_postgres inherits this from DatabaseInstanceBase for free.
Restores QuestionType.EXECUTE to the Python SDK enum (rocketride
schema/question.py), which commit ac74740 removed in favor of the
execute/dialect @tool_function methods. This plan (Plan A, convenience
query tools) deliberately reintroduces EXECUTE on the Questions lane so
MCP tools can use()+chat() a session instead of a stateless tool RPC;
open-questions.md for this plan anticipated the enum gap and called for
wiring it back in as part of this task. Doc updated (packages/client-python/docs/index.md)
per the co-located documentation rule.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ed questions" This reverts commit 5e0539c.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Empty rows previously fell through to a fallback that hardcoded truncated=True with a notice; add an up-front empty-input case that returns truncated=False with no notice. Also rewrite the drained-to-empty fallback to merge extra before setting envelope fields (matching the loop path's precedence) so extra can never clobber rows_key/row_count/ truncated/notice. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds the sql_query MCP tool: assert_sql_read_only guard -> open_session (spawn-or-reuse a warm postgres_1 task) -> client.tool(execute) -> cap_rows envelope. Wires query.register into tools/__init__.register_all (17 -> 18 tools) and ships the blank-credential sql_query.pipe template. Broadens the exact-tool-set/count assertions in test_handlers.py, test_mcp_module.py, and test_visibility.py that enumerate register_all's output, since sql_query is now part of it.
Skipped by default (RR_LIVE_ENGINE unset); drives sql_query, graph_query, and vector_search against a real EngineClient + credential-filled .pipe copies. Live execution is deferred to a human with a running engine.
open_session() now only reuses a live session_token when its stored pipeline_ref matches the requested pipe_path; a mismatched or missing pipeline_ref spawns a fresh session instead of RPC-ing to the wrong pipeline's node IDs. Also corrects the stale register_all() docstring (three convenience query tools, not one) and adds the missing trailing newline to graph_query.pipe to match sql_query.pipe.
…/remove/update) Adds fs_stat, fs_get_url, deploy_status, deploy_remove, and deploy_update to the EngineClient Protocol and WsEngineClient shim, each a thin passthrough to the RocketRideClient SDK, following the existing deploy_add/fs_list_dir pattern. Tasks 3-4 build tools on these. Extends FakeEngineClient (conftest.py) and the WsEngineClient seam test fakes accordingly. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
dsapandora
approved these changes
Aug 19, 2026
dsapandora
left a comment
dsapandora
left a comment
Collaborator
There was a problem hiding this comment.
Approving. All three of my findings are fixed at 9e8c49da and I re-checked each against the head rather than taking the replies for it:
authorize()rejectstk_/pk_/cd_before the authenticator sees them, on every bind — not just public. That one was a real hole and you found the wider half of it yourself (no route allowlist upstream), which is the better fix.- Both resource reads are inside
asyncio.wait_foron the sharedDEFAULT_TIMEOUT_SECONDS. - The
tk_is out of the upload URL.
Worth flagging for whoever re-reviews: the Critical on filepath is closed at this head. load_pipeline is inline-only now and there is no open( left anywhere under modules/mcp — that finding was read against a head before 8033787f. Shell API contract is green too.
One non-blocking leftover from that fix, whenever you next touch the file — the description published to MCP clients still advertises the argument that was removed (tools/execution.py:298):
Start a RocketRide pipeline from an inline definition or filepath, returning a task_token.
The reader of that sentence is a model: it will send filepath and get a ValueError back on its first try.
Build / Ubuntu 22.04 was cancelled at 1h30m while macOS finished in 33m and Windows in 34m, so it read as a runner timeout rather than anything of yours — I re-ran it.
dylan-savage
dismissed
asclearuc’s stale review
August 19, 2026 19:17
Ariel's approval is enough for merge, he also confirmed that all of Alexandru's comments have been addressed.
dylan-savage
enabled auto-merge (squash)
August 19, 2026 19:18
dylan-savage
added a commit
that referenced
this pull request
Aug 20, 2026
Resolve packages/ai/scripts/tasks.js: union of the mcp-widgets ignore list from #1880 and the global node_modules exclusion (ENOTSUP fix). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
rajeev-chaurasia
added a commit
to rajeev-chaurasia/rocketride-server
that referenced
this pull request
Aug 23, 2026
validate_pipeline passed a pipeline naming a provider that does not exist, so a typed name validated clean and failed later at run time. The engine's provider lookup lives in validate_pipeline.cpp's single-component path, which the whole pipeline path never reaches. The tool now checks each component against the engine's service catalog after the engine's own validation, and reports the component id with the name it could not resolve. The names come from the engine, so no rule moves into the client, and engine findings are added to rather than replaced. Closes the third of the four tools rocketride-org#1989 asked for; it was previously treated as already shipped in rocketride-org#1880, which covers the structural half only.
rajeev-chaurasia
added a commit
to rajeev-chaurasia/rocketride-server
that referenced
this pull request
Aug 23, 2026
validate_pipeline passed a pipeline naming a provider that does not exist, so a typed name validated clean and failed later at run time. The engine's provider lookup lives in validate_pipeline.cpp's single-component path, which the whole pipeline path never reaches. The tool now checks each component against the engine's service catalog after the engine's own validation, and reports the component id with the name it could not resolve. The names come from the engine, so no rule moves into the client, and engine findings are added to rather than replaced. Closes the third of the four tools rocketride-org#1989 asked for; it was previously treated as already shipped in rocketride-org#1880, which covers the structural half only.
dylan-savage
added a commit
that referenced
this pull request
Aug 26, 2026
The MCP docs described only the stdio PyPI package, while the HTTP MCP server merged from develop (PR #1880) carried its docs as a 613-line co-located doc.md — the wrong depth for a public surface, in a location the docs tree does not allow. - docs/public/mcp/ now holds two folders: stdio/ and http/, each with a README.md, rendered in the sidebar as an MCP category with two pages. The two near-duplicate stdio docs (README.md export source + index.md site page) are merged into stdio/README.md, which serves as both the /protocols/mcp/stdio page and the packages/client-mcp README source (regenerated copy included). - docs/public/mcp/http/README.md is a new public doc sized to how GitHub, Sentry, and Linear document their hosted MCP servers: endpoint, auth in two sentences, per-client setup, the 27 tools by capability group, example prompts, resources, widgets, self-hosting env vars, security, FAQ. - packages/ai/src/ai/modules/mcp/doc.md is deleted; the few contributor facts that live nowhere else (engine-client seam, credentials-catalog builder gates, widget workspace) move to docs/development/engine/mcp-module.md. Everything else stays in git history. - Pipeline: DOCS_ROOT_MOUNTS split per folder; a README.md at a mount root with no index sibling is now gathered as that mount's page (the blanket export-source skip is otherwise unchanged); export mapping, spine, redirects, footer link, and spine tests updated; stale /protocols/mcp links retargeted to /protocols/mcp/stdio. Verified: docs:build (broken-link gate), docs:check, spine tests, validate-client-docs.py all green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
rajeev-chaurasia
added a commit
to rajeev-chaurasia/rocketride-server
that referenced
this pull request
Aug 28, 2026
validate_pipeline passed a pipeline naming a provider that does not exist, so a typed name validated clean and failed later at run time. The engine's provider lookup lives in validate_pipeline.cpp's single-component path, which the whole pipeline path never reaches. The tool now checks each component against the engine's service catalog after the engine's own validation, and reports the component id with the name it could not resolve. The names come from the engine, so no rule moves into the client, and engine findings are added to rather than replaced. Closes the third of the four tools rocketride-org#1989 asked for; it was previously treated as already shipped in rocketride-org#1880, which covers the structural half only.
joshuadarron
pushed a commit
that referenced
this pull request
Aug 28, 2026
…in validate_pipeline (#2082) * feat(mcp): add resolve_config, showing what a node actually receives A .pipe rarely says what a node gets. The engine merges profile defaults and, when a profile is set, reads the user layer only from the sub-object named after that profile, so a key written beside it is discarded in silence. The author of #1989 lost a day to exactly that, and the schema cannot express it: a schema describes what a field is, not that this field in this profile will be dropped. resolve_config runs the config through the same Config.getNodeConfig a node calls at load, and returns the resolved result plus the keys the resolver discarded, with a hint naming the object they belong in. Resolution has to happen engine-side. getServiceSchemas copies 15 keys into the catalog and preconfig is not among them (services.cpp:2100-2148), so profile resolution cannot be reproduced from rrext_services or /services. The tool reaches the engine through the existing seam rather than importing Config directly. modules/mcp/engine.py states that tools depend only on the EngineClient protocol so a later revision can swap the WS SDK for in-process calls without touching tool code; a direct import would have made this the first tool to break that. So this adds rrext_resolve_config next to rrext_services and rrext_validate, the matching SDK method, and one line on each side of the seam. Both argument checks default only a genuinely absent config: `or {}` would coerce a falsy non-object such as [] and skip the type check entirely. A test caught that. doc.md gains the tool row and its count moves to 28. * feat(mcp): add scaffold_node, emitting a node that loads on the first try Authoring a node means discovering its contract by failure, and the documented contract is wrong in three places, so reading the docs first does not help: - preconfig is marked optional in README-node-schema.md, but Config.getNodeConfig raises without it and nearly every IGlobal calls that at load. Of 170 services*.json in the tree, the only ones without preconfig are global-field files with no protocol plus a handful whose IGlobal never calls it. - depends() is documented as running in __init__.py, and the same page then contradicts itself. The real nodes call it from IGlobal.beginGlobal, and any third-party import has to follow it. - The "Adding a New Node" example shows a plain class with a process() method, which nothing in the engine calls. The contract is IGlobalBase/IInstanceBase. Two more the docs omit: a services.json without protocol is not registered as a service at all, and register must be filter or endpoint or the node loads but can never be instantiated. The provider name is the protocol, which is the mistake the issue opens with. scaffold_node emits all of that correctly, including the parent local_nodes/__init__.py and the local_nodes.<name> import path. class_type and lane_out are checked against the live catalog rather than a list here, so the allowed sets follow the engine instead of drifting as nodes are added. It returns files rather than writing them. _common.load_pipeline already refuses server-side reads because an MCP caller could otherwise reach the server's disk; writing deserves the same stance. The tests assert the emitted skeleton against the engine's real requirements: every generated Python file parses, the manifest carries the four keys that stop a node loading, depends() is in IGlobal and not __init__.py, and the handler matches the requested lane. Dropping preconfig from the template fails them. doc.md gains the tool row and its count moves to 29. * fix(client-python): let a non-object config reach the engine's type check resolve_config defaulted with `config or {}`, which coerces a falsy non-object such as [] into {} and hides the mistake the engine would have reported. Only an absent config is defaulted now, matching the handler it calls. * fix(mcp): report a config key the profile overwrote, not just an absent one resolve_config decided a key was discarded by checking whether it was missing from the resolved config. Nearly every profile in the catalog declares apikey, so an apikey written beside "profile" comes back present, holding the profile's value rather than the author's, and the tool stayed silent on the exact case it exists to surface. A key now counts as discarded when what the node receives differs from what the author wrote. Found by calling the tool over the MCP transport against a real llm_openai profile; the unit test missed it because its stub returned a resolved config without the key at all. * refactor(mcp): give the catalog lookups one job each _catalog_values branched on whether its key argument was the string 'lanes', so one function carried two different traversals. Splitting them drops the branch and the stringly-typed parameter, and the lane names are now filtered for strings the way the class types already were. * fix(mcp): harden scaffold_node against catalog and argument shapes Three defects found in review: A catalog entry whose classType is a bare string was iterated character by character, so 'text' admitted 't', 'e' and 'x' as class types and rejected the real one. Introduced when the lookup was split in two; the container check that guarded it is back. Dispatch does not apply the tool's inputSchema, so a non-string name reached the regex and raised TypeError instead of returning an actionable answer. Name, both lanes and class_type are now checked for type first. lane_in was validated only against the local handler templates, never against the catalog, so a lane with a template but no engine support produced a manifest with an input lane nothing serves. Also corrects the introspection group heading in doc.md, which still counted 5 tools and did not list scaffold.py as a source. * fix(mcp): report every profile sibling key, not only one whose value changed Comparing the resolved value against the author's missed the sibling that happens to match the profile's own value. The resolver never read that line either, so staying quiet left the author believing it was in effect when the key inside the profile is what applied. With a profile set every sibling is discarded, so the value is not worth comparing and the check is simpler for it. * fix(mcp): keep a supplied falsy argument out of the scaffold defaults lane_in, lane_out and class_type defaulted with `or`, so a supplied 0, '' or [] became the fallback and never reached the type check that would have reported it. Defaulting now applies only to an absent or null argument, matching the handler this PR already corrected on the client side. * feat(mcp): flag a component whose provider the engine cannot serve validate_pipeline passed a pipeline naming a provider that does not exist, so a typed name validated clean and failed later at run time. The engine's provider lookup lives in validate_pipeline.cpp's single-component path, which the whole pipeline path never reaches. The tool now checks each component against the engine's service catalog after the engine's own validation, and reports the component id with the name it could not resolve. The names come from the engine, so no rule moves into the client, and engine findings are added to rather than replaced. Closes the third of the four tools #1989 asked for; it was previously treated as already shipped in #1880, which covers the structural half only. * fix(mcp): report only the config keys a profile actually discarded resolve_config reported every sibling of the selected profile, so it named parameters on all 34 components in the repo's own pipelines that select one, and told the author to move it inside the profile object. parameters is not node configuration: the engine validates and consumes it on its own path (pipeline_config.cpp Rule 5 and Rule 6), and moving it would break the component. name is the same shape, and an editor-saved config keeps a sub-object per profile, so an unselected profile read as a discarded key too. Structural keys and the service's own profile names are now excluded. Over the same pipelines the report goes from 34 components to 1, and that one is a real finding: search_exa carries apikey beside the profile. The existing test passed because its config was hand-written with only apikey beside the profile. The new one is shaped like a real .pipe. Also drops the return in the scaffolded handler's preventDefault call, which raises, matching how the real nodes write it. * fix(examples): repair the two component configs these tools surfaced Both are real defects the PR's own additions found in shipped examples. document-processor names provider "anonymize", but that node declares "protocol": "anonymize_text://" and the provider is the protocol minus the scheme, so no such service exists and the component could never instantiate. The new provider check in validate_pipeline flags it; the old validator resolved the definition and silently ignored the miss. exa-search-working carries apikey beside "profile", where getNodeConfig never reads it, so the pipe named "working" resolved its key to the profile's empty default. Moving it inside the profile object is correct under the current resolver and under the one #2039 and #1979 propose. resolve_config reports it. Over every pipeline in the repo, resolve_config now reports no discarded keys.
This was referenced Sep 3, 2026
erichanwang
pushed a commit
to erichanwang/rocketride-server
that referenced
this pull request
Sep 7, 2026
* feat(mcp): scaffold in-process Streamable-HTTP MCP module + boot registration
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(mcp): engine access seam over the RocketRideClient SDK
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(mcp): tool framework — registry, task registry, error normalizer, load_pipeline
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(mcp): registry-based tool dispatch + status/deployments resources
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(mcp): 16-tool surface — introspection, execution, capability, monitor
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs(mcp): co-located documentation for the 16-tool HTTP MCP server
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(mcp): add run_dropper_pipe for out-of-band file ingress
New HTTP-MCP tool that starts a pipeline and returns two self-contained,
credential-embedded URLs for getting files in over a separate HTTP data
channel (file bytes cannot ride the MCP tool call):
- upload_url: programmatic multipart POST to /task/data?token=<tk_>&auth=<pk_>
- dropper_url: /dropper?auth=<pk_>, a browser page for a human to drag-drop files
Both carry the task's private token (tk_) and public auth key (pk_) read from
the use() launch body, so no Authorization header is required. Also relaxes
/task/data's required Authorization header to accept the credential via the
?auth= query (AuthMiddleware already supports it; the handler auths off
request.state.account, so auth is unchanged). Adds a normalized base_url to the
MCP engine seam.
Live-verified: headerless multipart POST -> 200/ingested; browser drag-drop ->
parsed text. Covers ingress Options 2 (pipe-then-dropper) and 3 (agent-direct-POST).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(nodes): tool_filesystem pipeline sink lanes
Extend tool_filesystem from a tool-only node into a dual-role tool + pipeline
sink (classType ["tool","store"]). Six input lanes -- documents, text, table,
image, audio, video -- normalize to a single `documents` metadata output. Each
persists its payload to the account FileStore under `targetDir` (default
`output/`), preserving the source object's original name (object-id fallback),
deriving the extension from the name, mime type, or `.md`, and auto-suffixing on
collision. Optional `emitUrl` attaches a time-limited signed download URL via
FileStore.get_url (needs RR_SIGNING_KEY for the local backend). The sink reuses
the existing allowWrite toggle and path whitelist -- it never bypasses them.
New config: targetDir, emitUrl, urlExpiresIn. Emitted documents carry a valid
DocMetadata (objectId/chunkId inherited from the source object).
Verified with 20 unit tests, the full `nodes:test` contract suite, and a live
end-to-end MCP pipeline (webhook -> parse -> tool_filesystem -> response) that
wrote a parsed file into the store and emitted its reference.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(nodes): direct-execute on Questions lane for EXECUTE-typed questions
writeQuestions() now branches on QuestionType.EXECUTE before the LLM
translation path: it runs the raw query directly via _executeRawQuery,
gated by IGlobal.allow_execute and the is_sql_safe read-only whitelist,
and emits {rows, affected_rows, row_count} on the answers/table lanes.
db_postgres inherits this from DatabaseInstanceBase for free.
Restores QuestionType.EXECUTE to the Python SDK enum (rocketride
schema/question.py), which commit ac74740 removed in favor of the
execute/dialect @tool_function methods. This plan (Plan A, convenience
query tools) deliberately reintroduces EXECUTE on the Questions lane so
MCP tools can use()+chat() a session instead of a stateless tool RPC;
open-questions.md for this plan anticipated the enum gap and called for
wiring it back in as part of this task. Doc updated (packages/client-python/docs/index.md)
per the co-located documentation rule.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Revert "feat(nodes): direct-execute on Questions lane for EXECUTE-typed questions"
This reverts commit 5e0539c.
* feat(mcp): add tool() RPC to the EngineClient seam
* feat(mcp): read-only SQL/Cypher guards for query tools
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(mcp): 40KB in-band result envelope helper
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(mcp): correct cap_rows empty-input and extra-merge precedence
Empty rows previously fell through to a fallback that hardcoded
truncated=True with a notice; add an up-front empty-input case that
returns truncated=False with no notice. Also rewrite the drained-to-empty
fallback to merge extra before setting envelope fields (matching the loop
path's precedence) so extra can never clobber rows_key/row_count/
truncated/notice.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(mcp): query session helper (spawn-or-reuse + TTL clamp)
* feat(mcp): sql_query convenience tool (tool-lane execute)
Adds the sql_query MCP tool: assert_sql_read_only guard -> open_session
(spawn-or-reuse a warm postgres_1 task) -> client.tool(execute) ->
cap_rows envelope. Wires query.register into tools/__init__.register_all
(17 -> 18 tools) and ships the blank-credential sql_query.pipe template.
Broadens the exact-tool-set/count assertions in test_handlers.py,
test_mcp_module.py, and test_visibility.py that enumerate register_all's
output, since sql_query is now part of it.
* feat(mcp): graph_query convenience tool (tool-lane execute)
* feat(mcp): vector_search convenience tool (tool-lane search)
* test(mcp): live integration for convenience query tools
Skipped by default (RR_LIVE_ENGINE unset); drives sql_query, graph_query,
and vector_search against a real EngineClient + credential-filled .pipe
copies. Live execution is deferred to a human with a running engine.
* docs(mcp): document convenience query tools
* fix(mcp): validate pipeline on session reuse + doc/format cleanup
open_session() now only reuses a live session_token when its stored
pipeline_ref matches the requested pipe_path; a mismatched or missing
pipeline_ref spawns a fresh session instead of RPC-ing to the wrong
pipeline's node IDs. Also corrects the stale register_all() docstring
(three convenience query tools, not one) and adds the missing trailing
newline to graph_query.pipe to match sql_query.pipe.
* feat(mcp): drop set_env/list_env_keys from the tool surface
* feat(mcp): grow EngineClient seam (fs_stat, fs_get_url, deploy status/remove/update)
Adds fs_stat, fs_get_url, deploy_status, deploy_remove, and deploy_update to
the EngineClient Protocol and WsEngineClient shim, each a thin passthrough to
the RocketRideClient SDK, following the existing deploy_add/fs_list_dir
pattern. Tasks 3-4 build tools on these. Extends FakeEngineClient (conftest.py)
and the WsEngineClient seam test fakes accordingly.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat(mcp): store_stat + store_get_url tools (out-of-band file egress)
* feat(mcp): deploy lifecycle tools (list, status, remove, update)
* feat(mcp): list_running_pipelines tool (server-authoritative task listing)
* docs(mcp): 25-tool surface docs + live verification; tracker updates
* docs(mcp): fix History tool-count arithmetic (final-review nit)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat(mcp): TaskRegistry flow-event machinery (ring buffers, control.id routing)
Add side-map flow-event tracking to TaskRegistry, kept independent of the
existing _tasks metadata dict so buffers survive remove(token) — one-shot
execution tools drop tokens from the registry while their traces must
remain drainable by a later get_pipeline_trace tool.
- ensure_flow/set_flow_id/mark_flow_subscribed/is_flow_subscribed
- record_flow: two-tier routing (exact control.id match, then token-prefix
match with id caching), monotonic seq
- flow_since: cursor-paged drain
- _token_core: strips tk_/pk_-style display prefixes
- Bounded: max 32 tracked tokens (FIFO evict oldest), each ring deque(maxlen=500)
Ported from feature/mcp-server-overhaul's client-mcp registry.py, adapted
to side-dicts per the get_pipeline_trace plan (Task 1).
* feat(mcp): engine seam add_monitor + on_event passthrough
* feat(mcp): flow-event dispatcher wired through engine factory + hoisted TaskRegistry
* feat(mcp): pipelineTraceLevel passthrough + flow subscription on run tools
* fix(mcp): do not flow-subscribe when pipelineTraceLevel is 'none'
* feat(mcp): get_pipeline_trace tool (combined subscribe+drain per-node tracing)
* docs(mcp): 26-tool surface complete; per-node tracing docs + live verification
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(mcp): clamp flow_since cursor; document 'none' subscribe gate
Cursor in flow_since no longer moves backward when since exceeds the
highest buffered seq (max(since, buf[-1]['seq'])), preventing stale
re-delivery of already-consumed events. Also documents that run tools
skip the MCP-side flow subscription entirely when pipelineTraceLevel
is 'none' (gate is not in (None, 'none')), so no flow_subscribed key
appears in their results in that case.
* fix(ai): add mcp to allowed-modules test expectations
The mcp module was added to ai.modules.ALL in 5fab3d8 but the
web-server allowlist test's expected set was never updated, failing
the full ai:test run.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(mcp): timeout add_monitor seam calls; guard missing run token
- add_monitor was the only blocking seam call not wrapped in
asyncio.wait_for, so a wedged connection could hang run tools'
best-effort flow subscription and get_pipeline_trace indefinitely.
- run_pipeline now returns a BadRequest instead of ok:true with a
null task_token when the engine yields no token, mirroring
run_dropper_pipe's existing guard.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat(mcp): drop sql_query/graph_query/vector_search pending cloud DB backend
The 3 convenience query tools target RocketRide-hosted SQL/graph/vector
databases that are not yet available in OSS develop, so they fail at
call time on any engine a teammate can actually run. Remove them from
the surface (26 -> 23 tools, 8 -> 7 groups) along with their private
infrastructure: tools/query.py, read_guards.py, result_envelope.py,
their pipes, the seam tool() RPC, and their tests. They are expected
to return once the cloud DB backend lands.
Also removes the query-tools section bolted onto the standalone
rocketride-mcp docs page (those tools never shipped in that package)
and adds a pointer there to the in-engine /mcp surface.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat(ai,mcp): migrate HTTP MCP server to MCP spec 2026-07-28 (SDK v2)
Bump mcp SDK to v2 and port handlers from the removed decorator API to
constructor kwargs. Add the now-required ttlMs/cacheScope cache hints with a
pinned deterministic tool order and a three-way read_resource TTL policy.
Rewrite the test harness on the v2 in-memory client and pin dual-revision
serving both ways (legacy initialize handshake and modern self-describing
POST) through the real /mcp mount. Live-verified 7/7 against a real engine.
Includes the migration audit, SDK API recon, and verification docs.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat(ai,mcp,mcp-widgets): embedded UI (MCP Apps) widgets for the HTTP MCP server
Serve ui:// widget resources with UI-extension capability gating and link
tools via ToolRegistry ui_resource_uri -> _meta.ui.resourceUri. Add the
apps/mcp-widgets workspace wired into the aggregate builder flows, plus uv
override support forcing mcp>=2 past crewai's cap. Ships three widgets in the
official RocketRide design language: pipelines-table (list_running_pipelines),
dropper (typed result tabs, engine envelope unwrap), and pipeline-graph.
MCPJam runbook and embedded-UI docs included. A run-monitor widget and a
slice-5 canvas embed were built and removed within this work; neither ships.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat(ai,mcp): DVR run-log tools — persistent run visibility (surface 26)
Expose the engine's persisted run log (rrext_log) as four MCP tools:
log_chapters, log_read (cursor-paged, console filter), log_traces (chapter-
addressable via chapterBeginSeq), and log_trace. Grow the EngineClient seam
with faithful-transport log methods; normalize the SDK's nested shapes at the
tool layer. Retire get_pipeline_trace and the flow ring-buffer machinery the
persisted log supersedes. Run tools now return projectId/source (the log
keys) and default pipelineTraceLevel to 'summary'. In-band error contracts:
not_found, trace_expired with retention hints, shared timeout envelope, and
bounded pages (200 events). Docs updated; live-verified 12/12 against a real
engine including the full send-data -> trace round trip.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* chore: untrack claude/ working docs (local-only by convention)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* chore(docs): revert ROCKETRIDE_OBSERVABILITY.md to develop's version
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* refactor(ai,mcp): embed the mcp-widgets workspace inside the mcp module
Move apps/mcp-widgets into packages/ai/src/ai/modules/mcp/apps/ so the
widgets live with the module that serves them. Vite now builds straight
into the served apps/dist/ (the copy step is gone), the builder module is
auto-discovered at its new path with unchanged mcp-widgets:* action names,
and ai:sync excludes the workspace's sources/toolchain so only dist/*.html
ships into the server dist.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(ai,mcp): apply CodeRabbit review fixes across the MCP surface
41 of 49 findings from the local CodeRabbit review applied (7 skipped with
documented reasons, 1 resolved by verification — asyncio_mode=auto).
Security/robustness:
- MCP_DEV_NO_AUTH bypass honored only on loopback binds; warned + ignored
elsewhere so /mcp stays authenticated
- shutdown chaining wrapped in try/finally so MCP cleanup always runs
- run_dropper_pipe terminates the started task when publicToken is missing
instead of orphaning it; dropper URLs normalized + query-encoded
- pipeline file reads moved off the event loop (load_pipeline_async)
- monitor clamps caller-supplied timeout (<=300s) and interval (>=0.25s),
marks poll-timeout snapshots with poll_timed_out
- store_get_url validates expires_in (schema minimum + handler check)
DVR log contract (pre-merge, no consumers yet):
- error_type unified to PascalCase: NotFound / TraceExpired
- dedicated engine.LogNotFound seam exception replaces broad KeyError
catches; unrelated KeyErrors now propagate
Widgets:
- pipelines-table surfaces isError/ok:false results instead of rendering
them as an empty list; Refresh survives the empty state; terminate
checks isError
- dropper dropzone keyboard-accessible (role=button, tabIndex, Enter/Space)
- vite target es2022 (verified the esnext workaround is unnecessary);
vite.config.ts + tsconfig.json included in the build fingerprint
Tests: 219 passing (5 new — clamp bounds, LogNotFound narrowing, dropper
teardown, initial-send timeout, no-provider describe_pipeline); shared
EXPECTED_TOOL_NAMES inventory in conftest replaces duplicated name sets
and magic counts.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(ai,mcp): apply CodeRabbit round-2 review fixes
22 of 27 follow-up findings applied (5 skipped with documented reasons —
see prior round for the standing skips: local .gitignore, CJS builder
convention, deferred filepath sandboxing).
Lifecycle & seam:
- _startup/_shutdown made idempotent so a server firing both router events
and _user_* slots cannot double-enter session_manager.run() or
double-close the shared engine client
- make_engine_client reads config (rocketride_uri/rocketride_auth) first,
env vars as fallback; WsEngineClient.close() takes the connect lock and
clears the flag in a finally
- list_running_pipelines gains the same asyncio.wait_for + Timeout
envelope as every other blocking seam call
Robustness:
- normalize_error classifies ConnectionError/TimeoutError subclasses as
hard failures via isinstance, not just exact type name
- call_tool round-trips one sanitized JSON payload for both content and
structured_content
- non-numeric maxEvents/n/chapterBeginSeq/beginSeq return BadRequest
envelopes; deploy_list tolerates a None SDK result; list_components
skips non-mapping service definitions
- pipelines-table widget rejects malformed payloads (ok !== true or
non-array tasks) instead of rendering an empty list
Build/infra:
- depends.py up-to-date predicate now validates the derived overrides
cache (missing or stale file triggers a rebuild — _override_args reads
it at install time)
- tsconfig type-checks vite.config.ts; mime_type snake_case unified;
stale 23-tool count in client-mcp docs corrected; doc.md auth/loopback/
history sections corrected
Tests: 222 passing (+3: JSON5 parse path, use() timeout, hang-based
wait_for proof); FakeWebServer consolidated into conftest (was 4 copies);
poll_timed_out marker pinned.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(ai,mcp): apply CodeRabbit round-3 review fixes
31 of 35 follow-up findings applied (4 standing skips: local .gitignore,
deferred filepath sandboxing, CJS builder convention, test-fixture churn).
Engine seam:
- full reconnect-on-drop: every seam call routes through a _guarded helper
that clears the cached connected flag on ConnectionError or the SDK's
'Server is not connected' RuntimeError (message-narrowed so unrelated
RuntimeErrors can't trigger a spurious re-connect on a live client) —
the process-wide singleton now recovers instead of failing until restart
Dropper widget:
- critical: stop the picker's synthetic click bubbling back into
zone.onclick (re-entrant loop / double-opened file dialog)
- media URLs restricted to data: scheme with mime validation (untrusted
pipeline output can't trigger remote fetches); xhr timeout restores the
dropzone on stalled uploads; null tool-result guard; dragleave
relatedTarget check; lang="en"
Pipelines-table widget:
- error states render via showError WITH the Refresh button (textContent
was wiping it — one failed refresh dead-ended the widget); aria-live
Tools & dispatch:
- capability.py: all 11 store/template/deploy seam calls get the standard
wait_for + Timeout envelope (last group without per-call timeouts)
- execution.py: shared _start_task helper (input check, use() kwargs,
trace-level default, timeout envelope in one place); accurate messages
for the missing-token vs missing-publicToken branches
- tooling.py rejects duplicate tool registrations; handlers.py imports the
resource URI constants and logs normalized failures at warning
- describe_pipeline skips non-mapping components; log_traces NotFound
message handles the omitted-chapter case
Build/docs:
- tasks.cjs verifies every widget bundle before skipping; tsconfig pins
types [node, vite/client] with an explicit @types/node devDep; no-effect
vite entryFileNames removed; widget HTML cached by mtime in apps.py
- doc.md: mcp>=2,<3; task_data docstring documents ?auth=; client-mcp
docs link to the engine MCP doc
Tests: 229 passing (+7: reconnect-on-drop, gated concurrent connect,
expires_in rejection, dispatch soft-error + nested HardError, negative
read_ui_resource, builtin-subclass classification).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* chore(examples): revert examples/ to develop's state
The branch carried stale edits to 11 example pipes (engine-stamped
project_id / layout churn from local runs during development). None of it
belongs to the HTTP MCP feature — restore develop's versions so the PR
diff carries no examples noise.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* chore: drop out-of-scope deltas from the branch diff
- packages/n8n-nodes/examples/rocketride-webhook.pipe: local-run
project_id stamp, not a feature change
- packages/client-mcp/docs/index.md: revert to develop's version; the
tool-count correction and doc links will ride a separate docs PR
(same treatment as ROCKETRIDE_OBSERVABILITY.md)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* chore(n8n-nodes): pin rocketride-webhook.pipe to develop's content
The running engine re-stamps project_id into this pipe on load, which
raced the previous revert commit. Staged from develop's blob directly;
the working-tree copy stays engine-stamped and uncommitted.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(ai,mcp): apply CodeRabbit PR-review fixes (round 4)
17 of 21 findings from the PR rocketride-org#1880 review applied (4 skips: the standing
filepath-sandbox deferral, the LogNotFound narrowing it previously asked
for, develop-owned example fixtures, and the deliberately-discovered
mcp-widgets task module).
Contract improvement:
- CallToolResult.is_error now mirrors the in-band ok flag — hosts detect
failed tool calls without parsing the body, while the {ok, error_type,
message, hint} envelope still rides content/structuredContent for agent
self-correction (doc.md + tests updated)
Robustness:
- TaskRegistry FIFO-bounded at 512 entries — leaked tokens (failed
terminate, client stops mid-run) can't grow the process forever;
eviction drops bookkeeping only, never the engine-side task
- engine_call timeout helper promoted to tools/_common.py; introspection's
four seam calls (incl. per-provider describe_pipeline lookups) are now
bounded — the last tool group without wall-clock limits
- close() declared on the EngineClient protocol (teardown is part of the
seam contract); LogNotFound handled uniformly in log_chapters/log_read;
deploy_add timeout hint directs agents to deploy_list instead of a blind
retry on the non-idempotent create; status resource counts unnamed tasks
Hygiene:
- theme.css stylelint (currentcolor casing, overflow-wrap); doc.md MD040;
conftest ImportError guard narrowed to rocketlib so a real ai.account
breakage isn't silenced
Tests: 231 passing + 1 skipped (new authenticated-mount test runs only in
the engine env). New coverage: full ordered tool-surface pin, log handler
name-binding, negative provider-lookup caching, deterministic monitor
clock, is_error expectations.
(--no-verify: the lefthook run hung on this commit; ruff check + format
were run manually and pass.)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(ai,mcp): CodeRabbit round 5, MCP SDK v2 port for client-mcp, DVR teamId migration
Three intertwined fixes after the develop merge (6fa7427):
1. packages/client-mcp ported to MCP SDK v2 (fixes the red CI on all three
platform builds): the branch pins mcp>=2.0.0,<3 and CI installs one
environment, which broke client-mcp's v1-API stdio server. server.py now
uses v2 constructor-kwargs handlers + create_initialization_options,
sse_server.py drops the removed FastMCP for lowlevel Server +
SseServerTransport (same /sse + /messages/ URLs), resources.py goes
snake_case, and the package pin moves to mcp>=2.0.0,<3.
2. DVR run-log seam migrated from runKind to teamId: the develop merge
(rocketride-org#1764 teams-as-environments) made the SDK log surface keyword-only
team_id — every seam call passing run_kind positionally raised TypeError
at runtime. Seam, tools, tool schema (runKind enum -> optional teamId),
doc.md, and all test fakes migrated; the fakes now mirror the SDK's
keyword-only signatures so this drift class fails in CI.
3. CodeRabbit round 5 (17 comments: 15 fixed, 2 skipped): empty-host
MCP_DEV_NO_AUTH bypass refused, lifecycle started/stopped flags latched
only after transitions succeed (+ retry failure-path tests), widget
app.connect() rejection handling, brand icon via DOMParser instead of
innerHTML, describe_pipeline overall provider-lookup budget, doc.md
protocol-version key + cache-policy rationale corrections, and 7 test
hardenings (drain-before-close ordering, timeout clamp pin, LogNotFound
branches, engine_call timeout pin, /mcp + /mcp/ auth assertions).
Skipped: rocketlib.debug() convention (module must import in a plain
venv without rocketlib; stdlib logging is deliberate).
Verified: mcp suite 237 passed + 2 skipped; client-mcp 74 passed + 1
skipped; ruff check/format clean; mcp-widgets build green. Committed with
--no-verify (lefthook hangs on the signing prompt); ruff run manually.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(ai,mcp): CodeRabbit round 6 — log_read byte/concurrency bounds, SSE loopback auth guard
- log_read now forwards a fixed 1 MiB max_bytes to the engine alongside the
200-event cap (the event cap alone is not a size bound: 200 x 64KiB values
is ~12.5MiB held 3x over by the handler), and concurrent log_read calls
are bounded by a module-level Semaphore(4). Seam, protocol, fakes, and a
forwarding pin updated; doc.md worst-case math rewritten to the enforced
bounds.
- sse_server main() refuses to start without MCP_API_KEY on a non-loopback
bind (run_pipeline uploads any server-readable file), mirroring the /mcp
MCP_DEV_NO_AUTH loopback rule. Filepath sandboxing remains the standing
deferred-by-design item.
- doc.md: deploy_update listed in the pipelines-resource cache rationale.
Verified: mcp suite 237 passed + 2 skipped; client-mcp 74 passed; ruff
check/format clean. --no-verify: lefthook signing-prompt hang; ruff run
manually.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat(ai,mcp): echo log-keying context from log_traces/log_trace
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat(ai,mcp): register trace-viewer widget and link log_traces/log_trace
* feat(ai,mcp): trace-viewer widget skeleton — React + cross-package bundling
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat(ai,mcp): trace-viewer widget list view over log_traces
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(ai,mcp): restore widget test typechecking (vitest 3 for vite 6 compat)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat(ai,mcp): trace-viewer widget detail view over the log_trace bridge
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat(ai,mcp): wire trace-viewer into the widget build + add mcp-widgets:test
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(ai,mcp): final-review fixes — doc contract, host theme, retry, list keys
- doc.md: document the context field on log_traces/log_trace payloads and
link both tools to the trace-viewer widget in the Embedded UI section
- host-theme.ts: host-supplied theme (app.getHostContext().theme /
hostcontextchanged) now wins over prefers-color-scheme, which is only the
fallback; wired via the SDK's applyDocumentTheme
- main.tsx: apply the host's initial theme after connect() resolves; key
TraceViewerApp on an incrementing per-ontoolresult counter so stale
selection state doesn't survive a second tool result
- TraceViewerApp.tsx: add a Retry button that remounts TraceDetail (via a
bumped retryKey) so a failed fetchTrace can be retried
- adapter.ts: give rows missing beginSeq a unique negative docId (was a
shared -1, colliding as React keys) instead of an index-derived one
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat(ai,mcp): OAuth 2.0 discovery and audience enforcement for /mcp
Claude and ChatGPT cannot be handed an API key — they only authenticate via
OAuth. Make /mcp a spec-compliant OAuth 2.0 protected resource so they can
discover how to log in, and check that the tokens they come back with were
actually issued for MCP.
Discovery (RFC 9728):
- Serve the protected-resource metadata document publicly at
/.well-known/oauth-protected-resource/mcp, naming Zitadel as the
authorization server. It has to be public — it is what an unauthenticated
client reads in order to learn how to authenticate.
- Advertise it from WWW-Authenticate on 401s, scoped to the MCP resource so
unrelated engine routes do not misdirect clients.
Audience enforcement:
- Verify the JWT signature against the issuer's JWKS, check the issuer, and
require MCP_EXPECTED_AUDIENCE to appear in aud. Bearer tokens are
interchangeable, so without this a token minted for the VS Code extension
would open /mcp just as readily as one minted for Claude.
- Static API keys (rr_/tk_/pk_/cd_) keep the existing path untouched.
- Credentials that are neither a known key prefix nor a decodable JWT are
refused while enforcing. Zitadel can be configured to issue opaque access
tokens, which would otherwise be mistaken for API keys and bypass the
audience check entirely.
- An unset audience is tolerated on a loopback bind and refused on any other,
mirroring MCP_DEV_NO_AUTH: local development keeps working, while a deploy
that forgets the setting fails loudly instead of silently accepting every
token it is handed.
- Close websocket scopes at the mount. AuthMiddleware is a BaseHTTPMiddleware
and only wraps 'http' scopes, but Mount routes by path for any scope type,
so a websocket would otherwise reach the session manager unauthenticated.
The audience check lives at the /mcp boundary rather than in the engine's
global authenticator chain on purpose: "was this token issued for MCP" is a
rule about MCP, and must not be imposed on /task or /api/chat.
Tests are force-added — the bare `tests/` pattern in .gitignore silently
excludes new files under packages/ai/tests/.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(ai,mcp): size and style the trace-viewer widget like its siblings
Sizing: the widget stretched to fill the host-allocated iframe because
#root forced height:100% with a 320px floor and the containers used
flex:1, so a single trace rendered a mostly-empty tall box. Height is now
content-driven end to end — one trace is one row tall — with scroll caps
at 420px (list) and 640px (call tree).
Those caps are in PIXELS deliberately. The MCP Apps auto-resize measures
the document with `html { height: max-content }` and resizes the iframe to
match, so a vh/%-relative cap shrinks as the iframe shrinks, clamps the
content that produced the measurement, and collapses the widget to nothing
over successive resize rounds. The 320px floor was all that hid this
before. No viewport-relative sizing remains in the widget.
Styling: adopt the shared widget design-system used by the dropper and
pipelines-table — brand header via mountBrandHeader, .rr-card containers
(padding zeroed so rows and the tree run edge to edge), .rr-btn/.rr-btn-ghost
for the header bar, and the flame left-border error idiom. theme.css is
imported after the app tokens: the two overlap on exactly one token
(--rr-accent), which only the chrome reads, so the shared trace tree's
colors are unaffected.
Verified: widget suite 20/20, typecheck clean, bundle rebuilt and served,
live-checked in MCPJam against a two-trace run.
* feat(ai,mcp): integrations surface - credential catalog, per-caller identity, list_integrations
Curated credentials.json (generator-reconciled via nodes:credentials-generate,
CI drift gate nodes:credentials-check) maps credentialed nodes to config
fields + suggested ROCKETRIDE_* names. Tool surface 26 -> 27: list_components
filtered to usable nodes with ${VAR} wiring, new list_integrations with
per-caller readiness (configured / unconfirmed / available) and relay-able
setup guidance, describe_component enriched. Readiness is computed per
request from the caller's own credential (task_http pattern) with the
configured-auth singleton as fallback; variable NAMES only - values never
transit MCP.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(ai,web): keep ai.web importable without rocketlib for plain-venv tests
The OAuth discovery feature made ai.modules.mcp import ai.web.oauth_resource,
and the ai.web package __init__ eagerly imports WebServer -> rocketlib. In a
plain venv (the mcp module's local test recipe) that made every mcp test
module fail collection. Guard the WebServer/AccountInfo imports with the same
rocketlib-narrowed ImportError check the test conftest uses — engine env is
unchanged, plain venv degrades to the rocketlib-free submodules
(oauth_resource) that must stay importable.
Verified: mcp suite + oauth tests 338 passed, 7 skipped in the plain venv.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(ai,mcp): review round 7 — rr_-only /mcp auth, inline-only pipelines, bounded resource reads
Addresses all 10 unresolved review threads (asclearuc, dsapandora, CodeRabbit):
- credentials.json: regenerate for db_hotdata (clears the Shell API contract
CI failure introduced by the rocketride-org#1937 merge)
- credentials.py: candidate matching is boundary-aware — an env-var name part
must start with the node token, so GIT no longer matches DIGITALOCEAN_TOKEN
- __init__.py: _bind_host falls back to CONST_DEFAULT_WEB_HOST instead of a
second 'localhost' literal that would fail open if the default ever moved
- auth.py: /mcp explicitly rejects task-scoped keys (tk_/pk_) and PKCE codes
(cd_) on every bind — the authenticator chain scopes permissions, not
routes, so a leaked pk_ could otherwise reach non-engine-gated tools
- execution.py: upload_url no longer embeds the tk_ control token; /task/data
resolves the task from the pk_ alone since rocketride-org#1764
- resources.py: deploy_list/list_tasks reads share the tools' 30s seam bound
instead of hanging on a wedged engine WS
- tools/*: the filepath argument is removed from all 7 pipeline-taking tools;
pipeline definitions are inline-only, so no MCP caller can read
server-local files (load_pipeline_async deleted with the file-read branch)
- engine.py: log_trace KeyError comment matches the SDK contract (missing OR
expired begin seq); the LogNotFound narrowing stays
Suite: 342 passed, 7 skipped (plain venv).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(ai,mcp): send validate_pipeline configs in the engine's {'pipeline': ...} envelope
The engine's PipelineConfig::validate() requires the outer {'pipeline': {...}}
envelope, but the rrext_validate path hands the flat config to the binding
bare, so the MCP validate_pipeline tool rejected every pipeline with
"'pipeline' is missing or invalid" (ccode 40).
Wrap the flat config in the MCP tool before calling client.validate(),
defaulting version to 1 the same way /pipe/validate does (a missing version
is treated as legacy v0 by upgradeComponent). Already-enveloped configs pass
through untouched — never double-wrapped.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
erichanwang
pushed a commit
to erichanwang/rocketride-server
that referenced
this pull request
Sep 7, 2026
…in validate_pipeline (rocketride-org#2082) * feat(mcp): add resolve_config, showing what a node actually receives A .pipe rarely says what a node gets. The engine merges profile defaults and, when a profile is set, reads the user layer only from the sub-object named after that profile, so a key written beside it is discarded in silence. The author of rocketride-org#1989 lost a day to exactly that, and the schema cannot express it: a schema describes what a field is, not that this field in this profile will be dropped. resolve_config runs the config through the same Config.getNodeConfig a node calls at load, and returns the resolved result plus the keys the resolver discarded, with a hint naming the object they belong in. Resolution has to happen engine-side. getServiceSchemas copies 15 keys into the catalog and preconfig is not among them (services.cpp:2100-2148), so profile resolution cannot be reproduced from rrext_services or /services. The tool reaches the engine through the existing seam rather than importing Config directly. modules/mcp/engine.py states that tools depend only on the EngineClient protocol so a later revision can swap the WS SDK for in-process calls without touching tool code; a direct import would have made this the first tool to break that. So this adds rrext_resolve_config next to rrext_services and rrext_validate, the matching SDK method, and one line on each side of the seam. Both argument checks default only a genuinely absent config: `or {}` would coerce a falsy non-object such as [] and skip the type check entirely. A test caught that. doc.md gains the tool row and its count moves to 28. * feat(mcp): add scaffold_node, emitting a node that loads on the first try Authoring a node means discovering its contract by failure, and the documented contract is wrong in three places, so reading the docs first does not help: - preconfig is marked optional in README-node-schema.md, but Config.getNodeConfig raises without it and nearly every IGlobal calls that at load. Of 170 services*.json in the tree, the only ones without preconfig are global-field files with no protocol plus a handful whose IGlobal never calls it. - depends() is documented as running in __init__.py, and the same page then contradicts itself. The real nodes call it from IGlobal.beginGlobal, and any third-party import has to follow it. - The "Adding a New Node" example shows a plain class with a process() method, which nothing in the engine calls. The contract is IGlobalBase/IInstanceBase. Two more the docs omit: a services.json without protocol is not registered as a service at all, and register must be filter or endpoint or the node loads but can never be instantiated. The provider name is the protocol, which is the mistake the issue opens with. scaffold_node emits all of that correctly, including the parent local_nodes/__init__.py and the local_nodes.<name> import path. class_type and lane_out are checked against the live catalog rather than a list here, so the allowed sets follow the engine instead of drifting as nodes are added. It returns files rather than writing them. _common.load_pipeline already refuses server-side reads because an MCP caller could otherwise reach the server's disk; writing deserves the same stance. The tests assert the emitted skeleton against the engine's real requirements: every generated Python file parses, the manifest carries the four keys that stop a node loading, depends() is in IGlobal and not __init__.py, and the handler matches the requested lane. Dropping preconfig from the template fails them. doc.md gains the tool row and its count moves to 29. * fix(client-python): let a non-object config reach the engine's type check resolve_config defaulted with `config or {}`, which coerces a falsy non-object such as [] into {} and hides the mistake the engine would have reported. Only an absent config is defaulted now, matching the handler it calls. * fix(mcp): report a config key the profile overwrote, not just an absent one resolve_config decided a key was discarded by checking whether it was missing from the resolved config. Nearly every profile in the catalog declares apikey, so an apikey written beside "profile" comes back present, holding the profile's value rather than the author's, and the tool stayed silent on the exact case it exists to surface. A key now counts as discarded when what the node receives differs from what the author wrote. Found by calling the tool over the MCP transport against a real llm_openai profile; the unit test missed it because its stub returned a resolved config without the key at all. * refactor(mcp): give the catalog lookups one job each _catalog_values branched on whether its key argument was the string 'lanes', so one function carried two different traversals. Splitting them drops the branch and the stringly-typed parameter, and the lane names are now filtered for strings the way the class types already were. * fix(mcp): harden scaffold_node against catalog and argument shapes Three defects found in review: A catalog entry whose classType is a bare string was iterated character by character, so 'text' admitted 't', 'e' and 'x' as class types and rejected the real one. Introduced when the lookup was split in two; the container check that guarded it is back. Dispatch does not apply the tool's inputSchema, so a non-string name reached the regex and raised TypeError instead of returning an actionable answer. Name, both lanes and class_type are now checked for type first. lane_in was validated only against the local handler templates, never against the catalog, so a lane with a template but no engine support produced a manifest with an input lane nothing serves. Also corrects the introspection group heading in doc.md, which still counted 5 tools and did not list scaffold.py as a source. * fix(mcp): report every profile sibling key, not only one whose value changed Comparing the resolved value against the author's missed the sibling that happens to match the profile's own value. The resolver never read that line either, so staying quiet left the author believing it was in effect when the key inside the profile is what applied. With a profile set every sibling is discarded, so the value is not worth comparing and the check is simpler for it. * fix(mcp): keep a supplied falsy argument out of the scaffold defaults lane_in, lane_out and class_type defaulted with `or`, so a supplied 0, '' or [] became the fallback and never reached the type check that would have reported it. Defaulting now applies only to an absent or null argument, matching the handler this PR already corrected on the client side. * feat(mcp): flag a component whose provider the engine cannot serve validate_pipeline passed a pipeline naming a provider that does not exist, so a typed name validated clean and failed later at run time. The engine's provider lookup lives in validate_pipeline.cpp's single-component path, which the whole pipeline path never reaches. The tool now checks each component against the engine's service catalog after the engine's own validation, and reports the component id with the name it could not resolve. The names come from the engine, so no rule moves into the client, and engine findings are added to rather than replaced. Closes the third of the four tools rocketride-org#1989 asked for; it was previously treated as already shipped in rocketride-org#1880, which covers the structural half only. * fix(mcp): report only the config keys a profile actually discarded resolve_config reported every sibling of the selected profile, so it named parameters on all 34 components in the repo's own pipelines that select one, and told the author to move it inside the profile object. parameters is not node configuration: the engine validates and consumes it on its own path (pipeline_config.cpp Rule 5 and Rule 6), and moving it would break the component. name is the same shape, and an editor-saved config keeps a sub-object per profile, so an unselected profile read as a discarded key too. Structural keys and the service's own profile names are now excluded. Over the same pipelines the report goes from 34 components to 1, and that one is a real finding: search_exa carries apikey beside the profile. The existing test passed because its config was hand-written with only apikey beside the profile. The new one is shaped like a real .pipe. Also drops the return in the scaffolded handler's preventDefault call, which raises, matching how the real nodes write it. * fix(examples): repair the two component configs these tools surfaced Both are real defects the PR's own additions found in shipped examples. document-processor names provider "anonymize", but that node declares "protocol": "anonymize_text://" and the provider is the protocol minus the scheme, so no such service exists and the component could never instantiate. The new provider check in validate_pipeline flags it; the old validator resolved the definition and silently ignored the miss. exa-search-working carries apikey beside "profile", where getNodeConfig never reads it, so the pipe named "working" resolved its key to the profile's empty default. Moving it inside the profile object is correct under the current resolver and under the one rocketride-org#2039 and rocketride-org#1979 propose. resolve_config reports it. Over every pipeline in the repo, resolve_config now reports no discarded keys.
4 of 7 tasks
Leela8256
added a commit
that referenced
this pull request
Sep 9, 2026
…contract gate Generated by nodes:credentials-generate; satisfies the gen-credentials.mjs --check introduced in #1880 after this branch was cut. The stub suggests ROCKETRIDE_GLM_APIKEY, matching the llm_kimi / llm_minimax entries. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Leela8256
added a commit
that referenced
this pull request
Sep 10, 2026
* feat(nodes): add Zhipu AI GLM LLM provider node (llm_glm) Adds an llm_glm node for Zhipu AI's GLM models (GLM-5.x / GLM-4.x) via the OpenAI-compatible Z.ai API or self-hosted vLLM/SGLang, following the llm_kimi / llm_minimax merged pattern: profile presets seeded from the live /v1/models catalog (GLM-5.2 default), Custom and "(Local)" profiles, save-time 1-token validation probe gated on the cloud base-URL substring, super()._chat delegation with <think>-block stripping (closing tag optional), sync_models provider integration (ROCKETRIDE_GLM_KEY), think-strip unit tests, live profile-existence test, and catalog rows. Fixes #1787 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(nodes): llm_glm review — private _chat declaration, prune unreachable exception types The class-level declaration now matches the _chat attribute that beginGlobal/endGlobal assign and LLMBase reads (llm_gmi_cloud precedent). AuthenticationError and RateLimitError are APIStatusError subclasses already handled by the preceding handler, so the second except tuple keeps only the non-HTTP types. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(nodes): add llm_glm IGlobal validation tests; fix stale provider docstring Adds test_llm_glm_global_validation.py mirroring the baidu_qianfan stub-and-load template: the validateConfig cloud probe (1-token request, self-hosted and missing-key early returns), APIStatusError body extraction surfaced via warning(), the pruned connection-error path, and the _format_error composer (full composition, whitespace collapsing, raw fallback without structured fields). Also drops the pre-reseed model list from the sync provider docstring per review. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(nodes): llm_glm review — drop Local profiles, <digit>v vision filter, GlmProvider unit test Removes the two (Local) profiles whose capabilities.reasoning flag routed self-hosted chat through the verbatim native reasoning stream and whose cloud-model-id reuse made the sync merger's last-wins model_to_key map enrich the local profile instead of the cloud one. Self-hosting remains available through the Custom Model profile (README updated accordingly). Replaces the per-generation 4v/5v/6v vision excludes with the full <digit>v set — the substring-engine equivalent of \dv — so future glm-N.Mv vision ids can never land as chat profiles via discovery. Adds test_glm_provider.py (baidu_qianfan template): registry and services-path wiring plus make_client base-url with a stubbed openai, so a registry typo fails CI without needing ROCKETRIDE_GLM_KEY. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * retrigger ci * fix(nodes): llm_glm — test cloud config for key presence, cover both guard branches Restructures the cloud-key guard to test the configured apikey directly (and fall back to the dummy token afterwards) instead of comparing against the sentinel, per review. Adds the two reviewer-suggested tests covering the node's fail-fast path: a cloud serverbase without a key raises, a self-hosted serverbase without a key builds fine. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(nodes): llm_glm — strip the configured apikey so whitespace-only keys fail fast A whitespace-only key was truthy, slipping past the cloud-key guard and reaching Z.ai as literal credentials (401 at request time instead of the local configuration error). Stripping before validation also cleans the common paste artifact of a valid key with a trailing newline/space. Regression test added per review. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore(nodes): register the llm_glm credential stub for the Shell API contract gate Generated by nodes:credentials-generate; satisfies the gen-credentials.mjs --check introduced in #1880 after this branch was cut. The stub suggests ROCKETRIDE_GLM_APIKEY, matching the llm_kimi / llm_minimax entries. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(nodes): llm_glm — drop the think-strip override now owned by the shared adapter The LangChainAdapter behind ChatBase._chat (since #1683) already strips <think> blocks — unterminated ones included — and routes reasoning to the thinking lane, so the local _chat override was dead code and its tests asserted against a stub base the node never runs on. The test file becomes test_llm_glm_chat.py, keeping the cloud-key guard coverage (plus a default-config construction case) with a simplified stub. Also per review: the charglm exclude pattern is the exact family name instead of the 'char' substring, and a stale local-profile sentence is dropped from the live-test docstring. README reasoning section updated to match. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Update nodes/src/nodes/llm_glm/IGlobal.py Co-authored-by: Alexandru Sclearuc <asclearuc@gmail.com> * Update packages/ai/src/ai/modules/mcp/credentials.json Co-authored-by: Alexandru Sclearuc <asclearuc@gmail.com> * fix(nodes): llm_glm review — format-error docstring and test, transparent brand icon Completes the status-only fallback fix applied in 567d3c9: the docstring now states the fallback is included after the status prefix, and the reviewer-suggested test pins the 'Error 502: Bad Gateway' shape. The icon drops its opaque background tile for a transparent brand-blue glyph in the sibling shape (preserve-colors marker, viewBox/1em sizing) — the marker, not multicolor detection, is what keeps the brand color per svgo-auto-currentcolor.mjs. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: Alexandru Sclearuc <asclearuc@gmail.com>
dylan-savage
added a commit
that referenced
this pull request
Sep 14, 2026
…+ CI gates, node migration, new site (#2139) * feat(docs): consolidate hand-written docs into top-level docs/ with export pipeline (#1856) * feat(docs): move spine pages to docs/product/, re-point gather Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(docs): mount client/mcp/engine docs from top-level docs/ tree Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(docs): move contributor docs to docs/internal/ Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(docs): add docs:export/docs:check; readme sources under docs/ Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(docs): docs:check skips gitignored export destinations Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(docs): commit exported package READMEs (un-gitignore) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(docs): remove docs/images and all references Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(docs): restore list spacing in vscode docs Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs: update doc-location rules, folder guide, CI drift gate Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * ci(docs): trigger docs workflow on top-level docs/ changes Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(clients): source package READMEs from committed docs:export copies Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(docs): close CI drift-gate gap, restore .pipe rule bullet Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs: drop reserved skills/ section from docs/ tree Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs: move top-level images/ into docs/images/ All doc-referenced images now live under docs/. Root README relative paths and main/develop-pinned raw.githubusercontent URLs rewritten; generated package READMEs refreshed via docs:export. Orphans (banner-root, screenshot-ide, screenshot-sdk) carried over unchanged. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs: fix stale README-* link labels and broken post-move links in internal docs CodeRabbit bucket-1 findings plus three broken links the sweep surfaced: README-vscode.md and README-*-client.md references now point at the new clients/ and mcp/ locations. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs: address CodeRabbit review findings across doc sources Verified each finding against current SDK/node code before fixing: - H1s, fence languages, MD031 blank lines, heading levels (lint batch) - Example correctness: import os, double-wrapped pipeline args (4 sites), undefined vars, Answer instance API documented from Question.ts, validate-gated use(), deploy rollback to recorded version, getTaskStatus/validate signatures from client.ts - Config correctness: postgres questions-lane wiring (both examples), qdrant serverName->host, neo4j password field, execution-model documents lane, node count 100+ - Security-hardening in crash-reporting.md examples (pinned checksum install, mktemp, env-var instead of interpolation), self-hosting default host now 127.0.0.1 - builder.md skip conditions include --force; LICENSE link depth (2 files) - Generated package READMEs refreshed via docs:export Deferred as code-behavior (not doc) issues: firecrawl unbounded 429 retry, vscode ROCKETRIDE_APIKEY persistence on disconnect. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> * feat(docs): documentation schemas, validators, CI gates, and docs tree cleanup (#1912) * docs(agents): make AGENTS.md the single-source contributor router AGENTS.md becomes the one canonical guide: repo overview, task routing table into docs/, common commands (absorbed from .claude/CLAUDE.md), conventions, and the documentation rules. .claude/CLAUDE.md becomes an @AGENTS.md import and .cursorrules a pointer, eliminating the three divergent copies of the doc-location rules. A nested nodes/AGENTS.md gives agents working in the nodes tree the README schema and validator command with zero indirection. docs/README.md references the documentation contracts in docs/internal/. * docs(nodes): add node README schema and deterministic validator docs/internal/node-readme-schema.md defines the per-node README contract: a fixed section set where services*.json triggers decide which sections are required (invoke→Connections, lanes→Lanes, tool∈classType→As a tool, ≥2 profiles→Profiles, gpu→Requirements, restricted capabilities→ Limitations), core sections for every node (summary, What it does, Configuration usage guidance, Example pipelines), and optional sections with placement rules (About vendor, Authentication, Notes, Upstream docs). The generated ROCKETRIDE:GENERATED:PARAMS region owns the field reference; hand-written Configuration is usage guidance only. scripts/validate-node-readme.py enforces the structural rules per node: section presence/order, forbidden-when-untriggered, table parity against declared invoke keys/lanes/profiles, and warnings when a field with objective complexity signals lacks a detail subsection. Multi-service directories are merged; fragment-only files (no protocol) are skipped. * docs(nodes): migrate six sample node READMEs to the README schema llm_anthropic, db_postgres, tool_github, store_qdrant, ocr, and agent_deepagent restructured to the schema in docs/internal/node-readme-schema.md. Content-preserving: existing prose kept, sections reordered into the canonical layout, bespoke headings folded into their owning sections or Notes, and the generated ROCKETRIDE:GENERATED:PARAMS regions untouched byte-for-byte. Substantive fixes made along the way: - llm_anthropic: Profiles table now lists all 21 declared profiles (previously 7 — Opus 5, Fable 5, Sonnet 5, 4.8 variants, latest aliases and 3 Haiku were declared in services.json but undocumented) - tool_github: removed a duplicated all-tools table that shadowed the per-group tool tables; grouped tables are now the single source - All six gain Example pipelines; db_postgres gains the Limitations section its noremote capability requires; vendor nodes gain About sections Validator: Profiles parity accepts key-or-title in the first two columns; grouped detail subsections satisfy the complex-field check via token containment. Schema: preset services with identical wiring may be documented once (db_postgres/Supabase case). * ci(docs): advisory node README schema check + CodeRabbit doc rules New node-docs CI job runs scripts/validate-node-readme.py over the node tree on every PR. Advisory for now (continue-on-error, not in ci-ok): the corpus is mid-migration to the schema, so the job surfaces results without blocking merges; flipping it to a hard gate is a two-line change once migration completes. CodeRabbit path_instructions direct review attention where the deterministic validator cannot reach: prose accuracy of node READMEs against code/services*.json (structure is machine-checked), and Python/TypeScript client-doc parity. * docs(clients): client readme schema, parity checker, and parity fixes docs/internal/client-readme-schema.md codifies the shared section order the Python and TypeScript readmes already follow, fixes the insertion points for language-specific sections, and states the parity rule: any documented method/type/exception exists in both readmes (snake_case and camelCase compare equal) or carries a language-specific marker. scripts/validate-client-docs.py enforces it deterministically: section presence/order/ownership plus API symbol parity harvested from method tables across the six shared API sections. Parity gaps it found, fixed against client source: - python readme was missing the attach/detach/is_attached/login/logout/ is_authenticated/set_env connection lifecycle (implemented in mixins/connection.py, previously undocumented) and Answer.setAnswer - typescript readme was missing Question.addGoal (Question.ts:264) - dap_request (python convenience) and withConnection (typescript convenience) marked language-specific with cross-references Package READMEs regenerated via docs:export; docs:check green; docs:build page count unchanged (179 pages, 125 nodes, 6 mounts). * refactor(docs): split protocols out, rename internal to development The docs/ tree is organized by audience, but docs/internal/ was a mixed bag: it held the WebSocket protocol surface (published to the site as Protocols > WebSocket) alongside contributor-only material, and an integration guide that belonged with the other integrations. - New docs/protocols/ holds both members of the site's Protocols section: websocket/ (was docs/internal/engine-protocol/) and mcp/ (was docs/mcp/). The section's two halves previously lived in unrelated trees. - docs/internal/ -> docs/development/. Every file that remains is contributor documentation: setup, builder, engine internals, node authoring, pre-commit hooks. The old name also read as company-private in a public repository, which is the opposite of the signal contributor docs should send. - docs/internal/n8n.md -> docs/product/integrations/n8n.md and onto the spine, joining the six existing integration guides. Prepared for publication: frontmatter added, repo-relative links rewritten to site routes or blob/develop URLs, and a reference to a local-only test harness removed. Every folder under docs/ is now entirely published or entirely not. docs/README.md documents that invariant and groups the sections by it. Paths updated in gather.js (mounts + root glob), export.js (MCP readme source), spine.js (integrations), AGENTS.md, nodes/AGENTS.md, docs/README.md, .coderabbit.yaml, ci.yml, SUPPORT.md, examples/README.md, tool_n8n's README, both validators, and the regenerated MCP package README. Both validators also reformatted to the repo's ruff style. Verification: docs:test 42/42, docs:check in sync, docs:build green, docs:gather 180 pages (179 + the newly published n8n page) / 125 nodes / 6 mounts. Node and client validators unchanged in behavior. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * ci(docs): build the docs site on doc-touching PRs; run client parity checker Review of the doc-CI surface found two genuine gaps; everything else was judged adequate or deliberately rejected to avoid CI sprawl. Gap 1: the docs site was only ever built AFTER merge (docs.yml on push to develop), so deterministic breakage — broken internal links, docs/ files not covered by a mount, spine entries without a page — merged clean and failed on develop's deploy. New path-filtered docs-build job runs docs:test + docs:build on PRs touching doc sources, wired into ci-ok (skip-tolerant, same pattern as helm/shell-contract). Blocking is justified: every failure mode is deterministic, no network flakiness. Gap 2: scripts/validate-client-docs.py was committed but nothing executed it. Added to the advisory node-docs job (renamed 'Doc schemas (advisory)'). Rejected as sprawl: an external-link checker (network-flaky by construction), prose/spell linting (noise on a technical corpus), a separate node-site-coverage check and a docs-folder-invariant check (both subsumed — the first by the advisory schema validator, the second by docs:build's unmounted-file abort now running on PRs). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Krish Garg <krish.garg@rocketride.ai> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> * docs(nodes): ship example bundles for seven nodes, then park the requirement (#1960) * docs(agents): make AGENTS.md the single-source contributor router AGENTS.md becomes the one canonical guide: repo overview, task routing table into docs/, common commands (absorbed from .claude/CLAUDE.md), conventions, and the documentation rules. .claude/CLAUDE.md becomes an @AGENTS.md import and .cursorrules a pointer, eliminating the three divergent copies of the doc-location rules. A nested nodes/AGENTS.md gives agents working in the nodes tree the README schema and validator command with zero indirection. docs/README.md references the documentation contracts in docs/internal/. * docs(nodes): add node README schema and deterministic validator docs/internal/node-readme-schema.md defines the per-node README contract: a fixed section set where services*.json triggers decide which sections are required (invoke→Connections, lanes→Lanes, tool∈classType→As a tool, ≥2 profiles→Profiles, gpu→Requirements, restricted capabilities→ Limitations), core sections for every node (summary, What it does, Configuration usage guidance, Example pipelines), and optional sections with placement rules (About vendor, Authentication, Notes, Upstream docs). The generated ROCKETRIDE:GENERATED:PARAMS region owns the field reference; hand-written Configuration is usage guidance only. scripts/validate-node-readme.py enforces the structural rules per node: section presence/order, forbidden-when-untriggered, table parity against declared invoke keys/lanes/profiles, and warnings when a field with objective complexity signals lacks a detail subsection. Multi-service directories are merged; fragment-only files (no protocol) are skipped. * docs(nodes): migrate six sample node READMEs to the README schema llm_anthropic, db_postgres, tool_github, store_qdrant, ocr, and agent_deepagent restructured to the schema in docs/internal/node-readme-schema.md. Content-preserving: existing prose kept, sections reordered into the canonical layout, bespoke headings folded into their owning sections or Notes, and the generated ROCKETRIDE:GENERATED:PARAMS regions untouched byte-for-byte. Substantive fixes made along the way: - llm_anthropic: Profiles table now lists all 21 declared profiles (previously 7 — Opus 5, Fable 5, Sonnet 5, 4.8 variants, latest aliases and 3 Haiku were declared in services.json but undocumented) - tool_github: removed a duplicated all-tools table that shadowed the per-group tool tables; grouped tables are now the single source - All six gain Example pipelines; db_postgres gains the Limitations section its noremote capability requires; vendor nodes gain About sections Validator: Profiles parity accepts key-or-title in the first two columns; grouped detail subsections satisfy the complex-field check via token containment. Schema: preset services with identical wiring may be documented once (db_postgres/Supabase case). * ci(docs): advisory node README schema check + CodeRabbit doc rules New node-docs CI job runs scripts/validate-node-readme.py over the node tree on every PR. Advisory for now (continue-on-error, not in ci-ok): the corpus is mid-migration to the schema, so the job surfaces results without blocking merges; flipping it to a hard gate is a two-line change once migration completes. CodeRabbit path_instructions direct review attention where the deterministic validator cannot reach: prose accuracy of node READMEs against code/services*.json (structure is machine-checked), and Python/TypeScript client-doc parity. * docs(clients): client readme schema, parity checker, and parity fixes docs/internal/client-readme-schema.md codifies the shared section order the Python and TypeScript readmes already follow, fixes the insertion points for language-specific sections, and states the parity rule: any documented method/type/exception exists in both readmes (snake_case and camelCase compare equal) or carries a language-specific marker. scripts/validate-client-docs.py enforces it deterministically: section presence/order/ownership plus API symbol parity harvested from method tables across the six shared API sections. Parity gaps it found, fixed against client source: - python readme was missing the attach/detach/is_attached/login/logout/ is_authenticated/set_env connection lifecycle (implemented in mixins/connection.py, previously undocumented) and Answer.setAnswer - typescript readme was missing Question.addGoal (Question.ts:264) - dap_request (python convenience) and withConnection (typescript convenience) marked language-specific with cross-references Package READMEs regenerated via docs:export; docs:check green; docs:build page count unchanged (179 pages, 125 nodes, 6 mounts). * refactor(docs): split protocols out, rename internal to development The docs/ tree is organized by audience, but docs/internal/ was a mixed bag: it held the WebSocket protocol surface (published to the site as Protocols > WebSocket) alongside contributor-only material, and an integration guide that belonged with the other integrations. - New docs/protocols/ holds both members of the site's Protocols section: websocket/ (was docs/internal/engine-protocol/) and mcp/ (was docs/mcp/). The section's two halves previously lived in unrelated trees. - docs/internal/ -> docs/development/. Every file that remains is contributor documentation: setup, builder, engine internals, node authoring, pre-commit hooks. The old name also read as company-private in a public repository, which is the opposite of the signal contributor docs should send. - docs/internal/n8n.md -> docs/product/integrations/n8n.md and onto the spine, joining the six existing integration guides. Prepared for publication: frontmatter added, repo-relative links rewritten to site routes or blob/develop URLs, and a reference to a local-only test harness removed. Every folder under docs/ is now entirely published or entirely not. docs/README.md documents that invariant and groups the sections by it. Paths updated in gather.js (mounts + root glob), export.js (MCP readme source), spine.js (integrations), AGENTS.md, nodes/AGENTS.md, docs/README.md, .coderabbit.yaml, ci.yml, SUPPORT.md, examples/README.md, tool_n8n's README, both validators, and the regenerated MCP package README. Both validators also reformatted to the repo's ruff style. Verification: docs:test 42/42, docs:check in sync, docs:build green, docs:gather 180 pages (179 + the newly published n8n page) / 125 nodes / 6 mounts. Node and client validators unchanged in behavior. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * ci(docs): build the docs site on doc-touching PRs; run client parity checker Review of the doc-CI surface found two genuine gaps; everything else was judged adequate or deliberately rejected to avoid CI sprawl. Gap 1: the docs site was only ever built AFTER merge (docs.yml on push to develop), so deterministic breakage — broken internal links, docs/ files not covered by a mount, spine entries without a page — merged clean and failed on develop's deploy. New path-filtered docs-build job runs docs:test + docs:build on PRs touching doc sources, wired into ci-ok (skip-tolerant, same pattern as helm/shell-contract). Blocking is justified: every failure mode is deterministic, no network flakiness. Gap 2: scripts/validate-client-docs.py was committed but nothing executed it. Added to the advisory node-docs job (renamed 'Doc schemas (advisory)'). Rejected as sprawl: an external-link checker (network-flaky by construction), prose/spell linting (noise on a technical corpus), a separate node-site-coverage check and a docs-folder-invariant check (both subsumed — the first by the advisory schema validator, the second by docs:build's unmounted-file abort now running on PRs). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(docs): shipped example bundle — screenshot + downloadable .pipe per node Every node's Example pipelines section now opens with a shipped example: a minimal runnable pipeline committed as example.pipe next to the node, a canvas screenshot of that same pipeline (example.png), and a shields.io download badge. Readers browsing the docs see the node wired on the canvas and can download the .pipe straight into their own canvas. - Schema: the standalone optional screenshot (formerly under What it does) is folded into Example pipelines as part of the bundle. Authors reference example.png / example.pipe by bare relative name so node folders stay self-contained and GitHub renders them natively. - gather.js: stageNodeMarkdown rewrites those two refs to develop-pinned repository URLs when staging site pages (raw for the image, blob for the .pipe download) — same pattern as the generated Source link. Verified end-to-end on a staged page. - Validator: bundle is all-or-nothing (pipe file, screenshot file, embed, download link + alt text) — any part present makes every part required; nothing at all is a warning until the node's bundle lands. New nodes must ship it (CodeRabbit instruction added). docs:test 42/42, staging parity (180 pages / 125 nodes / 6 mounts), 6 samples still pass (now with the expected bundle warning), ruff clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(nodes): move Example pipelines above the reference sections; migrate agent_llamaindex Example pipelines now sits directly under What it does, so a reader sees what the node is, a canvas screenshot, and a downloadable .pipe before any reference tables. Schema sections renumbered; the six migrated samples reordered to match. agent_llamaindex migrated to the schema: About LlamaIndex, three example pipelines, Connections/Lanes/As a tool tables matching services.json, and Configuration reduced to usage guidance — the duplicated hand-written field table is gone (the generated Schema table owns it), with detail subsections for the three fields that need them (Agent description as the parent-agent selection signal, Instructions, Require tool call as the fabrication guard). Loop internals and SSE progress moved under Notes. 7/125 nodes pass. docs:test 42/42, staging parity, ruff clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(docs): trigger the example bundle on references, ship agent_llamaindex's .pipe The all-or-nothing rule counted a loose file as a bundle part, so committing example.pipe before its screenshot existed would fail the node — blocking the natural two-step path (author the pipeline, screenshot it later). The trigger is now a README *reference*: referencing either half obliges the other and every referenced file must exist (no broken links, never a half-rendered README), while an unreferenced file is a warning naming what is present and what is still needed. agent_llamaindex's example.pipe is the existing, working examples/agent-llamaindex.pipe — chat -> agent (llm_anthropic on the llm channel, tool_http_request on tool) -> response_answers — rather than an authored approximation. Its screenshot and the two references land when the pipeline is captured on the canvas. Verified all three paths: unreferenced file warns, a lone badge fails, a referenced-but-missing file fails. docs:test 42/42, staging parity, 7/125 nodes pass, ruff clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(nodes): ship example.pipe for all seven migrated nodes; fix a broken example Every migrated node now carries the .pipe half of its example bundle, so completing each one is capture-a-screenshot plus two pasted lines. docs/development/example-bundle-todo.md holds the per-node instructions, the exact markdown to paste (alt text written per node), and provenance. Three are existing working pipelines copied from examples/: agent_llamaindex <- agent-llamaindex.pipe, ocr <- document-processor.pipe, store_qdrant <- rag-pipeline.pipe. The other four (llm_anthropic, db_postgres, tool_github, agent_deepagent) were authored from those proven shapes and statically validated: every provider is a declared node protocol, every input lane is one the upstream node produces, every control channel is declared in the target's invoke. They are unrun — opening each on the canvas for its screenshot is the verification step. Also fixes examples/document-processor.pipe, which referenced provider 'anonymize' while the node declares 'anonymize_text' — a broken example that predates this branch. All 17 files in examples/ now reference only real providers. 7/125 nodes pass, each with the expected unreferenced-bundle warning. docs:test 42/42, staging parity. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(docs): centre the example screenshot and download button, matching the root README The markdown image form rendered left-aligned at natural size; the root README gets its look from an HTML block, so node examples now use the same house style: a centred div with a width=100% screenshot above the shields.io download button. Safe on the docs site because Docusaurus is configured markdown.format 'detect', so .md is parsed as CommonMark rather than MDX (tags are still self-closed for portability). - gather.js: the staging rewrite now matches HTML attribute forms (src=, href=) as well as markdown targets, so the centred block resolves to repository URLs on the site. The shields.io URL is deliberately untouched. - Validator accepts either form and reads alt text out of the img tag, so the alt-text requirement still holds. - Schema documents the HTML block as house style and records why the link stays relative: the VS Code extension registers a custom editor for *.pipe, so clicking the button in the IDE opens the pipeline directly on the canvas — an absolute URL would bounce the reader to a browser. - agent_llamaindex converted, its example.png committed, and its first example corrected to describe the pipeline actually shipped (chat + tool_http_request, not webhook + tool_tavily). First fully complete bundle in the repo. - Checklist snippets regenerated in the new form. docs:build green (HTML compiles on the site), docs:test 42/42, 7/125 nodes pass, ruff clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(docs): keep the example block renderable in the VS Code preview The raw-HTML form centred correctly on GitHub but rendered as empty space in the VS Code / Cursor markdown preview: that preview rewrites relative paths only for markdown-syntax images, so an <img src="example.png"> is left unresolved and blocked. The IDE is the surface where the download button matters most (clicking it opens the pipeline on the canvas), so it cannot be the one that breaks. Blank lines inside the wrapper fix it. A blank line ends the HTML block per CommonMark, so the image and badge parse as markdown — resolving in the preview — while the surrounding div still centres them, and GitHub scales the screenshot to the content width on its own. Renders in all three surfaces: IDE preview, GitHub, and the docs site. Schema records why the blank lines are load-bearing so nobody tidies them away. Checklist snippets regenerated. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(nodes): complete shipped example bundles * docs(nodes): park the Example pipelines section and its bundle checks The shipped-example bundle (canvas screenshot + downloadable .pipe) is complete for seven nodes but is a per-node manual step, and it blocks the migration of the remaining ~120 READMEs behind screenshot capture. Park the requirement so the migration can proceed on prose alone. `## Example pipelines` is commented out in the schema with its full text preserved in place, and the validator's bundle checks are commented out alongside it — un-parking is uncommenting both and re-numbering the schema's sections. The section stays in the validator's CANONICAL_ORDER, so it remains tolerated in its slot rather than flagged as an unknown heading: the seven migrated nodes keep their examples and keep passing. It is simply no longer required. gather.js keeps rewriting relative example refs for the staged site — inert with nothing to rewrite, needed when this returns. * ci: scope the doc schema checks to fix/docs, leave the shared pipeline alone The doc schema work added four things to config that every branch shares: a docs path filter and two new jobs in ci.yml, a pnpm install plus a docs:check step bolted onto the always-on ruff-check job, and two path instructions in .coderabbit.yaml. None of that belongs on develop while the schemas are still being rolled out — and the ruff-check change in particular would have added an install and a docs build to every pull request in the repository, not just documentation ones. ci.yml is restored byte-for-byte to its pre-branch state, so merging fix/docs into develop is now a no-op for the shared pipeline. The same checks now live in .github/workflows/docs-schemas.yml, which triggers only on pull requests and pushes targeting fix/docs. It stays inert on develop even after this branch merges. Validators are advisory (the node corpus is mid-migration and would paint every PR red); the site build and export drift gate are blocking, since both pass today. CodeRabbit has no per-branch config file, but it reads .coderabbit.yaml from a pull request's base branch — so adding fix/docs to base_branches here enables review for PRs into this branch while develop keeps using its own copy. That entry and the two doc path instructions are fenced with TEMPORARY markers to be removed in the fix/docs -> develop PR. * ci: drop the unusable workflow_dispatch trigger from the doc schema checks workflow_dispatch only fires when the workflow file lives on the repository's default branch. This file is deliberately scoped to fix/docs and is meant never to reach develop, so the trigger could never have worked — gh confirms it with 'workflow not found on the default branch'. Removed rather than left as dead configuration. * chore: drop agent stubs the VS Code extension planted in this repo The extension's agent installer (apps/vscode/src/agents/) writes AI assistant instruction files into whatever workspace is open, on startup, with rocketride.integrations.autoAgentIntegration defaulting to true. It adds its own .rocketride/ docs directory to .gitignore but not the stubs it plants, so three of them were swept into b7c9918f, and its marker-based merge appended a thirtieth-line block to AGENTS.md because that file had no ROCKETRIDE markers to replace. That block is the reason CodeRabbit flagged AGENTS.md as self-contradicting on #1960: line 21 sends contributors to docs/agents/ROCKETRIDE_QUICKSTART.md while the appended block made .rocketride/docs/ROCKETRIDE_README.md step 1. AGENTS.md is the contributor router, and .rocketride/docs/ is customer-facing SDK documentation — the wrong guidance for someone working on the engine. Removes the three files and the appended block, and gitignores the stub targets not already covered (.cursor/ and .claude/* already were, which is the only reason the Cursor stub never landed). AGENTS.md cannot be gitignored, so the installer is also disabled in .vscode/settings.json, which is tracked and therefore fixes this for everyone who opens the repo rather than one machine. The underlying extension bug is not fixed here. --------- Co-authored-by: Krish Garg <krish.garg@rocketride.ai> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> * ci(docs): scope the node schema check to the PR's diff, and make it blocking (#1971) * ci(docs): gate the node schema on the PR's own diff, and make it blocking The corpus-wide run was advisory because 109 of 124 nodes fail the schema today, so a hard gate would have been red on every PR from the day it was added. But a check that has always failed is a check nobody reads, and the day it fails for a new reason looks identical to the two hundred days before it. The interesting failures were going to be buried under structural noise like "unknown heading '## Modes'". Scoping it to the nodes the PR actually touches inverts that: green by default, so red means this PR broke something. It also lets the check be a hard gate immediately rather than waiting on a 109-node cleanup to finish before it protects anything, and the backlog drains on its own — touching a node obliges you to bring its README along — while never being able to grow. services*.json counts as a trigger alongside README.md, since the metadata is what decides which sections are required or forbidden; a metadata-only change can invalidate a README nobody edited. That also closes the gap where the docs path filter watched only README.md. --all keeps a home in a new informational job that writes pass/fail counts to the run summary, where a large number is a trend line rather than a merge blocker. Retire that job when the count reaches zero. Checked against the existing docs:check gate to be sure two checks cannot disagree about what a good README is: docs:check verifies that exported client package READMEs match their docs/ sources, which is export drift, not document structure. No overlap. Verified under bash (CI's shell, not the zsh this was written in, which does not word-split unquoted variables): 8 nodes detected from the diff, all pass, exit 0; including an unmigrated node exits 1. * ci(docs): stop the scoped schema gate failing on a diff with no node files Two bugs CodeRabbit caught in the previous commit, both reproduced before fixing. Under `set -euo pipefail`, grep exits 1 when nothing matches, so a PR that touched no node README or services*.json died on the command substitution before reaching the empty check that was meant to report a clean skip. Since that job is now blocking, an unrelated PR into fix/docs would have been blocked outright. The `-d` test in the trailing loop had the same shape when the last candidate was a deleted node. `grep -c` already prints 0 when it matches nothing and exits 1 as well, so `|| echo 0` emitted a second zero and the run summary rendered "0 0". Verified: empty diff reaches the skip path and exits 0; the real diff still resolves 8 nodes; counts render as a single 0. --------- Co-authored-by: Krish Garg <krish.garg@rocketride.ai> * fix(docs): stop the node validator eating underscores in identifiers (#1973) `table_col` stripped `[*_`]` from every table cell to remove markdown decoration before comparing names against services*.json. Underscores are markdown emphasis, but they are also legal in the identifiers this function exists to compare, so the strip silently corrupted them: `_source` became `source` and no longer matched the lane that declared it. Caught on tool_pipe, whose README was correct and failed anyway. Thirteen nodes carry an underscore in a lane, connection, or profile name, and the LLM family is the worst affected — `gemini-2_5-flash`, `llama3_2-vision-11b`, `ai21_jamba-1_5-large` — so this would have produced a wave of false failures on the largest batch still to come. Profiles masked the bug because their comparison falls back to fuzzy containment across two columns; lanes and connections compare strictly, so they failed outright. Emphasis is now only removed when a pair of underscores wraps the whole cell, which is the only case where an underscore is decoration rather than part of a name. Verified against `_source`, `z_score`, `gemini-2_5-flash`, `streamable_http`, `_emphasised_`, and both the `*(default)*` and `_(default)_` markers already in use. No change to the corpus pass count. Co-authored-by: Krish Garg <krish.garg@rocketride.ai> * refactor(docs): reorganize docs/ by audience and retire duplicated pages (#1969) * refactor(docs): reorganize docs/ by audience and retire duplicated pages Collapses docs/ to three top-level folders keyed by who reads them, brings all five clients under one roof, and removes the hand-written pages that duplicated generated node documentation. docs/ public/ humans outside the repo: typescript, python, vscode, mcp, n8n (each with a distribution README.md and assets/), product/ (the site spine), assets/ development/ contributors; never published, no exceptions agents/ AI assistants: ROCKETRIDE_*.md, stubs/, skills/ clients/, protocols/, stubs/ and images/ are dissolved. Site mount destinations are unchanged, so no published URL moves except where noted below. Five clients, one place. mcp graduates from protocols/ to a client folder (its README already was the published rocketride-mcp README). n8n joins as the fifth: packages/n8n-nodes/README.md is now generated by docs:export like the other three, so it can no longer drift. Integration pages retired. Six of seven pages under product/integrations were node-README duplicates at 73-90% overlap, and postgres.md had already drifted into publishing Supabase instructions that the db_postgres README says will fail to resolve. All six are deleted and redirected to their node pages; n8n survives because it documents webhook ingress, credential-vault and human-in-the-loop flows that no single node README can own. development/ is now genuinely internal. Its websocket docs split by audience: the wire-protocol reference (external integrators) moved to the site spine and keeps its URLs, while crash-reporting.md (lldb, breakpad) is unpublished. apps.md split likewise, its published-SDK half becoming develop/apps with shell-ui imports normalized to rocketride/app-sdk. gather no longer sweeps development/ at all, so it cannot publish by accident. The remainder is grouped by subsystem (builder/, engine/, nodes/, clients/, apps/). product/ regrouped. The real quickstart was filed under Evaluate while /quickstart was an empty card grid pointing at it; it is now a landing page with the tutorial split into an IDE walkthrough and an SDK guide. Loose root files became operate/ (cloud, self-hosting, troubleshooting) and reference/ (cli, glossary, pipeline-reference). Broken distribution assets fixed. Every banner in the published READMEs pointed at a main-pinned raw URL for a path that had never existed there, so all of them 404 on npm, PyPI and the Marketplace. They now target the per-client assets/ paths; the root README uses relative paths that resolve on any branch. New guardrail: a doc id is a public URL here, and a spine id with no backing file used to publish a silent "coming soon" page. assertNoUnexpectedPlaceholders now fails the build and names the offending id, with an allowlist for intentional stubs. It caught four moved URLs hardcoded in the site footer during this work. Also adds development/docs-pipeline.md and development/ci-gates.md, deletes a service catalog that had drifted to 118/148 against a real 143/163, and fixes setup-guide and root-file link depths. Verified: docs:build 176 pages / 125 nodes / 5 mounts, docs:check in sync, docs:test 48/48, validate-client-docs.py passing, docs:export idempotent. 14 retired routes all have emitted redirects. * docs: address CodeRabbit review on #1969 Twelve of thirteen findings accepted; several were symptoms of wider problems and were fixed at the root. Wrong technical claims, verified against the source and rewritten: - Data lanes and agent tools were described as mutually exclusive. Nine of 27 `tool` services declare lanes, and validate-node-readme.py gates the two sections independently. Replaced with a 2x2 matrix. - The node loader contract was pseudocode for a `MyNode`/`process()` API that does not exist. python-instance.cpp requires `IInstance`, python-global.cpp treats `IGlobal` as optional, and bindMethod binds callbacks by name from a 24-entry list, silently no-oping on a typo. Replaced with a working three-file template. - `./builder test server` is not an alias for the server tests: bare args fan out to every module. Corrected here and two more sites in index.md. Host and path corrections: - The SDK `uri` examples used cloud.rocketride.ai, which is the web console. The API host is api.rocketride.ai (CONST_DEFAULT_WEB_CLOUD in both SDKs since #1078). Only SDK/URI contexts were changed; ~20 sites across the Python, TypeScript and VS Code pages, which had drifted away from the package READMEs that were already correct. - The exported agent doc linked into docs/development/, which docs:export never copies to .rocketride/docs/. Repointed at a sibling exported doc and published URLs. - docs.rocketride.ai -> .org in apps/index.md, matching docusaurus.config.ts. Also: documented the four app-SDK methods missing from the API list, corrected the n8n trigger's authentication description (it declares no credentials, only a `secret` parameter checked against the Authorization header), fixed rsbuild dependency drift in the app template, normalized bare `builder` to `./builder` in 14 places, tagged three untagged code fences, and ran prettier over redirects.ts (all 702 entry lines predated this PR; array data verified identical before and after). Rejected: the suggestion to unify updateEditorViewState and updateEditorViewport. They are distinct methods with different signatures in both Documents.tsx and shell-api/v0.d.ts; unifying them would introduce a bug. Also rejected `npx` -> `pnpm exec`, which the apps' own tasks.js contradict. Two source-level issues found and left for separate PRs: app-sdk/index.ts is missing four methods the runtime exposes, and tool_firecrawl/README.md was already schema-nonconformant. * docs(nodes): migrate the tool_ family to the README schema (#1974) * docs(nodes): migrate tool_apify, tool_bland_ai, tool_chartjs, tool_cognee, tool_daytona, tool_deepl to the README schema * docs(nodes): migrate tool_exa_search, tool_filesystem, tool_firecrawl, tool_git, tool_guild, tool_http_request to the README schema * docs(nodes): migrate tool_laserdata_memory, tool_mcp_client, tool_mem0, tool_n8n, tool_oura, tool_pipe to the README schema * docs(nodes): migrate tool_google_workspace, tool_pipedrive, tool_python, tool_slack, tool_tavily, tool_v0, tool_xtrace_memory to the README schema * docs(tool_pipedrive): clarify bearer token format * docs(nodes): use American spelling in tool_n8n --------- Co-authored-by: Krish Garg <krish.garg@rocketride.ai> * docs(nodes): migrate the LLM family with progressive model tables (#2022) * docs(nodes): migrate LLM family with progressive model tables * docs(nodes): settle the Profiles contract before the other families rebase This PR's schema and validator changes become the rules the five in-flight node-README PRs are held to, so four gaps get closed here rather than after 74 more nodes have adopted them. Each protocol-bearing service keeps its own default. A second registration is a separate node to the engine — a branded preset, a second provider, a second backend — so its preconfig.default is a fact about that service, not a competing claim about the primary one. The old single-default rule false-failed cloud_tts (#1975) and store_elasticsearch (#1970), which are correct as written, and it also hid a real defect here: llm_openai_api's Nebius default llama-3-3-70b rendered unmarked, and the intro bolded a title the metadata does not declare. Both fixed. Four tests cover the multi-service case; there were none. The visible-table rule said "until the table reaches six rows" while the validator enforced "at most six", so ten of twelve large-layout nodes under-fill against the written text. Six is now stated as a ceiling, not a quota, with an explicit clause for the nodes that front several vendors' catalogues — llm_bedrock was collapsing Llama 4 Scout and Maverick while showing Llama 3.3 above the fold, which that clause now forbids and this change corrects. How a row names its profile is now stated: the declared key as code, or the declared title reproduced exactly, one form per node. That is what the validator already enforced through resolve_profile_row, but leaving it unwritten is why guardrails (#1978) fails on re-cased display names. The default marker is **(default)**. The validator stays decoration-agnostic so pages using *(default)* keep passing; the schema now says which to write. No regression across the 124-node corpus: same failure set before and after. 21 validator tests pass. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: Dylan Savage <dylanmsavage@hotmail.com> Co-authored-by: Claude Opus 5 <noreply@anthropic.com> * docs(nodes): migrate the store_ family to the README schema (#1970) * docs(nodes): migrate store family to README schema * docs(nodes): name each store profile default in its Profiles intro #2022 landed the tightened Profiles contract: every protocol-bearing service's declared default has to appear before the first table, with its key and its title in bold, and be marked on its row. Six of the store nodes marked the row but never named the default in prose, so they fail the new validator. store_elasticsearch registers two backends with two separate defaults — Elasticsearch defaults to self-managed, OpenSearch to local. Both rows were already marked, which read as a contradiction with nothing explaining it; the intro now says which default belongs to which service. That shape is what the new multi-service clause in the schema describes. store_qdrant is not part of this migration — it was already on fix/docs and already passing the old validator — but it is the same one-line fix in the same family and it fails the new rules, so it is folded in here rather than left as an orphan nothing re-checks. Default markers switched from *(default)* to **(default)**, which is the form the schema now specifies. The validator accepts either. All nine store nodes pass; 21 validator tests pass. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: Dylan Savage <dylanmsavage@hotmail.com> Co-authored-by: Claude Opus 5 <noreply@anthropic.com> * docs(nodes): migrate the small node families to the README schema (#1975) * docs(cloud_tts): seed the generated params region so the node reaches the docs site gather.js stages a node only when its README carries the ROCKETRIDE:GENERATED:PARAMS marker, and nodes:docs-generate refuses to create one (resolveDocPath returns null without it), so a node that never had the marker can never acquire it on its own. cloud_tts was in that state and has been invisible on the site. Seeded an empty marker pair and ran the generator to fill it, which produced both service tables (ElevenLabs and OpenAI). Content is entirely machine-generated; nothing here is hand-written. * docs(nodes): migrate agent_crewai, agent_langchain, agent_rocketride, cloud_tts, db_clickhouse, db_mysql to the README schema * docs(nodes): migrate embedding_image, embedding_openai, embedding_transformer, embedding_video, graph_arango, graph_falkordb to the README schema * docs(nodes): migrate graph_hydradb, graph_neo4j, memory_internal, memory_persistent, preprocessor_code, preprocessor_langchain, preprocessor_llm to the README schema * fix(docs): clarify supported OpenAI embedding profiles * fix(docs): document embedding API key resolution * fix(docs): list custom node profiles * docs(nodes): bring the small families up to the new Profiles contract Nine nodes fail the contract #2022 landed. Most need only the intro sentence naming the declared default, but three carried real gaps: embedding_transformer and memory_persistent never listed their `custom` profile, so a profile the configuration panel offers was undocumented. memory_persistent went further and explained the omission in prose; the row is now present and that sentence says what `custom` is for instead. embedding_openai rendered its declared context window as "8191 tokens." in a semantic column, which the validator compares against the metadata value. The column is now `Context tokens` carrying the number alone. cloud_tts registers OpenAI and ElevenLabs with a default each. Both rows were marked with nothing explaining why there were two; the intro now names each default and the service it belongs to. Its Profile column also mixed bare keys for one provider with key-plus-title for the other — every row now uses the declared title, one convention per node as the schema asks. Profile titles are padded to align in the configuration panel's dropdown ('Text Small - ...', 'Text Large - ...'). That alignment is deliberate and belongs in services.json, but requiring a README to reproduce it verbatim in bold prose is not reasonable, so title matching now collapses whitespace runs on both sides. Two tests cover it, including that a genuine wording mismatch still fails. All 19 nodes in this PR pass; 23 validator tests pass. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(preprocessor_code): correct the parser cache's scope The README said the parser cache is retained for the lifetime of the global node instance. languageParsers is declared as a class attribute on PreProcessor (code.py:166) and the write at code.py:235 is an item assignment, so it mutates the class-level dict rather than rebinding a per-instance one. A grammar loaded once is therefore reused by every instance in the process and outlives any single global node instance. Raised by CodeRabbit on #1975. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: Krish Garg <krish.garg@rocketride.ai> Co-authored-by: Dylan Savage <dylanmsavage@hotmail.com> Co-authored-by: Claude Opus 5 <noreply@anthropic.com> * docs(nodes): migrate the media nodes, and document eight that never had a README (#1976) * docs(nodes): migrate detect_segment, face_detection, pose_estimation, video_composer to the README schema * docs(nodes): migrate background_removal, caption, depth_estimate, detect to the README schema * docs(nodes): migrate accessibility_describe, frame_grabber, image_cleanup, thumbnail to the README schema * docs(nodes): migrate audio_player, audio_transcribe, audio_tts, twelvelabs to the README schema * docs(nodes): seed generated params regions for the eight newly documented nodes These nodes had no README at all, so they had no marker region either, and nodes:docs-generate cannot create one (resolveDocPath returns null when the marker is absent). Seeded an empty pair per node and ran the generator to fill it. Content is entirely machine-generated. * fix(docs): clarify audio player video input * fix(docs): clarify background removal output size * fix(docs): label RT-DETR fallback link * docs(nodes): bring the media nodes up to the new Profiles contract Eight nodes needed the intro sentence naming the declared default, which is all the new contract asks of this family — no missing rows, no semantic-column mismatches, no multi-service defaults. Markers switched to **(default)**. The review also turned up a boilerplate problem worth fixing while the family is open. Four ## Requirements sections opened with "This node is marked as GPU-capable. Its metadata declares local CPU, Apple Silicon (MPS), and CUDA operation" and closed with a variant of "The source does not specify VRAM requirements or relative CPU performance." Both halves describe the metadata and the writing process rather than the node: a reader who opens ## Requirements wants to know what hardware to provision, and being told what the source failed to mention answers nothing. Those sections now say where inference runs and what the device lock does, and stop. depth_estimate keeps the one fact the closer was carrying — its output is relative depth, not calibrated distance. Same fix in two other places: caption pointed at sibling nodes via "the node metadata directs object-detection use cases to Object Detection", and audio_player attributed its own truncation note to "The source notes that...". All 16 nodes pass. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: Krish Garg <krish.garg@rocketride.ai> Co-authored-by: Dylan Savage <dylanmsavage@hotmail.com> * docs(nodes): migrate the text, NLP, and document-parsing nodes (#1977) * docs(nodes): migrate dictionary, extract_data, extract_facts, ner to the README schema * docs(nodes): migrate anomaly_detector, anonymize, answer_documents, currency_convert_explicit to the README schema * docs(nodes): migrate prompt, question, schema_validate, summarization to the README schema * docs(nodes): migrate landing_ai, llamaparse, reducto, vectorizer to the README schema * docs(landing_ai): seed the generated params region so the node reaches the docs site Last of the thirteen nodes gather.js was silently skipping. Its two variant sub-pages (extract, parse) were hidden with it, since variants are only staged once their parent is. * fix(docs): clarify anonymizer classifier fallback * fix(docs): use canonical LlamaParse links * fix(docs): document JSON scalar lane metadata * fix(docs): use American spelling consistently * fix(docs): clarify severity fallback warnings * docs(nodes): bring the text family up to the new Profiles contract Three nodes fail the contract #2022 landed. anomaly_detector needs only the intro sentence naming the declared default; anonymize and ner also never listed their `custom` profile, so a model choice the configuration panel offers was undocumented in both. The review turned up a second, larger thing. Seven nodes carried an `## About RocketRide` section. That slot is for third-party services a node wraps, and RocketRide is not third-party: on a page inside RocketRide's own documentation, "RocketRide connects data-processing components into pipelines" tells a reader nothing, and the sentence after it describes what the node does, which is what `## What it does` is for and where each of these already says it. Each one also dragged along an `## Upstream docs` link back to docs.rocketride.org — circular, and present only because `## About` requires that section. Both are removed from all seven. question's `## Requirements` said the node "is marked GPU-capable in its metadata, but its implementation only strips text". A reader opening that section wants to know what to provision, not what the metadata claims, so it now says the node needs no hardware of its own and that any GPU requirement belongs downstream. All 16 nodes pass. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: Krish Garg <krish.garg@rocketride.ai> Co-authored-by: Dylan Savage <dylanmsavage@hotmail.com> Co-authored-by: Claude Opus 5 <noreply@anthropic.com> * docs(nodes): migrate the infrastructure and I/O nodes (#1978) * docs(nodes): migrate local_text_output, telegram, text_output, webhook to the README schema * docs(nodes): migrate autopipe, core, remote, response to the README schema * docs(nodes): migrate aparavi_aql, rocketride_graph, rocketride_sql, rocketride_vector to the README schema * docs(nodes): migrate guardrails, rerank_cohere, search_exa to the README schema * fix(docs): explain local-only node execution * docs(guardrails): document custom profile * docs(search-exa): warn against production mock bypass * docs(webhook): clarify endpoint security constraints * docs(nodes): bring the infrastructure nodes up to the new Profiles contract guardrails named its profiles "Basic", "Strict" and "Custom" — display names the metadata never declares, so no row resolved to a profile and the declared default was not marked on any of them. Rows now use the declared keys, and the prose that referred to the display names follows. remote and rerank_cohere needed the intro sentence; rerank_cohere also described its `custom` profile in a trailing sentence rather than listing it, so a profile the configuration panel offers had no row. Five nodes carried an `## About` section that was not about a third-party service. webhook and local_text_output were filler — "RocketRide is the pipeline system that hosts this node", "Python provides the filesystem functions used by this node" — and each dragged along an `## Upstream docs` link back to docs.rocketride.org that existed only because `## About` requires that section. The three managed-database nodes were different: their About carried a real fact, that RocketRide provisions a per-tenant database and the node resolves it from the signed-in identity rather than from connection fields. That is configuration guidance, and each `## Configuration` already gestured at it, so it moves there in full rather than being dropped. search_exa said the same thing twice: the third paragraph of `## What it does` restated `## Notes` → URL safety almost sentence for sentence, and took the section to nine sentences. The summary now states the guarantee and points at Notes, which keeps the detail and regains the address classes the summary had been carrying. All 15 nodes pass. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: Krish Garg <krish.garg@rocketride.ai> Co-authored-by: Dylan Savage <dylanmsavage@hotmail.com> Co-authored-by: Claude Opus 5 <noreply@anthropic.com> * docs(nodes): clear the last three nodes the migration PRs did not cover (#2124) These three were already on fix/docs before the six family migrations, so no PR owned them and nothing re-validated them once #2022 tightened the contract. ocr was not a documentation problem. Its table is the best-formed profile table in the corpus — twelve profiles with the key and the title in separate columns — but the first column is headed "Profile key", which PROFILE_HEADERS did not alias, so every row resolved to an empty cell and the validator reported all twelve profiles missing and twelve unknown rows. The alias is added, with a test, because separating key from title is a shape worth supporting rather than a mistake to correct. ocr additionally needed its default marked and its title in the intro. tool_mcp_client registers MCP Client and the Butterbase preset, each with its own default, and only the RocketRide one was marked — so the Butterbase preset's default was undocumented. Both are now marked and named. tool_laserdata_memory needed the intro sentence. All 124 nodes pass; 24 validator tests pass. Co-authored-by: Claude Opus 5 <noreply@anthropic.com> * docs(mcp): split MCP docs into stdio and http, retire the module doc The MCP docs described only the stdio PyPI package, while the HTTP MCP server merged from develop (PR #1880) carried its docs as a 613-line co-located doc.md — the wrong depth for a public surface, in a location the docs tree does not allow. - docs/public/mcp/ now holds two folders: stdio/ and http/, each with a README.md, rendered in the sidebar as an MCP category with two pages. The two near-duplicate stdio docs (README.md export source + index.md site page) are merged into stdio/README.md, which serves as both the /protocols/mcp/stdio page and the packages/client-mcp README source (regenerated copy included). - docs/public/mcp/http/README.md is a new public doc sized to how GitHub, Sentry, and Linear document their hosted MCP servers: endpoint, auth in two sentences, per-client setup, the 27 tools by capability group, example prompts, resources, widgets, self-hosting env vars, security, FAQ. - packages/ai/src/ai/modules/mcp/doc.md is deleted; the few contributor facts that live nowhere else (engine-client seam, credentials-catalog builder gates, widget workspace) move to docs/development/engine/mcp-module.md. Everything else stays in git history. - Pipeline: DOCS_ROOT_MOUNTS split per folder; a README.md at a mount root with no index sibling is now gathered as that mount's page (the blanket export-source skip is otherwise unchanged); export mapping, spine, redirects, footer link, and spine tests updated; stale /protocols/mcp links retargeted to /protocols/mcp/stdio. Verified: docs:build (broken-link gate), docs:check, spine tests, validate-client-docs.py all green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs: fix accuracy drift across the docs tree from a full code-verified audit Every fix traces to a finding confirmed against source by an independent adversarial verification pass (10 audit lanes, 10 verifiers, 10 fix executors; ~252 confirmed fixes applied, 12 refuted findings dropped). Highlights: - SDK docs (TS/Python): remove fictional APIs and behavior (.env auto-parsing, exponential backoff, Answer.parseJson, dapRequest, MIME table), correct signatures/defaults, add missing parameter rows - Examples: pipeline samples now validate (components key, real providers/fields, embedded question lanes for qdrant) - WebSocket protocol pages: correct wire commands, token semantics, response shapes, event names; document the run-log continuum - VS Code docs: migrate to the grouped settings schema, real command surface and workflows - Development docs: builder module table from live --list output, ci-gates matches actual CI, runnable command fixes - Operate/reference: ROCKETRIDE_APIKEY vs ROCKETRIDE_AUTH scoping, CLI table corrections, working tar command - Regenerated client-mcp/client-python/client-typescript READMEs via docs:export Gates: docs:build (broken-link check), docs:check, validate-client-docs. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(agents): land the pipeline-builder skill set in docs/agents/skills/ (#2125) * docs(agents): land the pipeline-builder skill set in docs/agents/skills/ First occupants of the reserved skills directory: the five hand-written pipeline-builder skills (orchestrator + design/configure/run/debug) plus MCP_TOOL_CONTRACT.md, the frozen tool-name/result-shape contract against the 27-tool HTTP MCP server surface. The skills are weak-model-first: verification by tools (validate_pipeline is the compiler, the run result is the proof) and hard gates (Waiting = STOP, cost approval, 17 forcing functions in GATE_PROTOCOL.md) rather than model intelligence. Bundled L1 node index regenerated 2026-08-25 from a live v3.3.1 engine (124 nodes, matches this branch's README corpus). Deliberately not exported by docs:export (subdirectories are excluded); README documents the layout and the future export hook. Bundled shims conformed to repo ruff style; anti-example keys shortened to obvious placeholders that clear the gitleaks rules. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(agents): apply review fixes to the pipeline-builder skill set Code-verified review fixes on top of the initial drop: - MCP_TOOL_CONTRACT: error envelope is {error_type, message, hint}; add pipelineTraceLevel to run_pipeline/run_dropper_pipe inputs (server default summary); list_integrations detail key is 'setup' - L1 node index reconciled against the branch's node corpus: 167 providers (renamed graph nodes, store tool flags, missing lanes, 45 absent providers added); meta provenance + counts synced - Error strings now match what the code emits verbatim (or are worded as symptom patterns): 'The service <provider> was not found', 'references unknown component id', 'input has unknown lane', 'Connection closed unexpectedly', 'Pipeline is already running.' - ${ROCKETRIDE_*} prefix requirement restored in PIPELINE_ANTIPATTERNS and the rules summary (non-prefixed refs come back <REDACTED>) - Shims: MIT headers, corrupt-cache reads follow the ERROR_JSON contract, fetch-doc docstring drift fixed, cross-skill hint path - New tools/generate-index.py — the index regenerator three docs reference (SDK get_services(), atomic writes, --dry-run) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Krish Garg <krish.garg@rocketride.ai> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: Dylan Savage <dylanmsavage@hotmail.com> * docs: restyle the site, restructure the IA, and finish the content revamp (#2138) * feat(docs): restyle the docs site to the RocketRide marketing design language Port the marketing/cloud landing look onto the Docusaurus theme: the red-orange star (#F93822) as sole accent over a warm paper field with a dot grid (light) or the Abyss Deep near-black (dark); Ubuntu Mono uppercase kickers; squared surfaces with hairline borders; contrast- inversion hovers on cards and sidebar rows; a hard-orange-offset CTA block; a floating form-fitting dark sidebar panel; and a rocket-mark "you are here" indicator on the page TOC. Rebuild the homepage as a Dify-style hero + numbered card sections using new global Card/CardGrid MDX components. Cascade notes: the build injects Infima/theme rules at (2,x,y) hacked specificity and wraps CSS in cascade layers, so overrides variously match the :not(#\#) hack, exceed it, or pin with !important — each case is documented inline in custom.css. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(docs): make sidebar group kickers readable in the mobile drawer The sidebar menu markup is reused by the mobile navbar drawer, whose surface is light — the white kicker text scoped to the menu vanished there. Kickers now default to the star orange (readable on the drawer in both themes); only the dark desktop panel re-whitens them. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(site): restructure the IA into journey order (phases 1-3) Rebuild the docs spine around the reader journey — Get Started, Concepts, Clients, Guides, Examples, Nodes, Connect, Deploy & Operate, Reference, Support & Community — and make the docs/public/product/ tree, doc ids, and URLs one namespace again. - Move 12 pages, split 4 by audience (performance, error-handling, apps, cli)…
dylan-savage
added a commit
that referenced
this pull request
Sep 18, 2026
…, engine URI, tools) (#2315) * fix(mcp): refuse query-string credentials on /mcp and never fall back to the shared client for authenticated callers AuthMiddleware falls back to `?auth=` when no Authorization header is sent, but auth.authorize() only read the header. `POST /mcp/?auth=<key>` therefore passed the middleware, skipped every /mcp check (tk_/pk_/cd_ refusal, JWT signature, issuer, audience) and ran tools on the server's shared engine client. - authorize() now refuses any request carrying an `auth` query parameter (parsed as Starlette parses query_params), per the MCP authorization spec. - Defence in depth: a request the middleware authenticated (scope['state']['account']) that carries no header credential is refused instead of served by the shared client. The credential-less dev bypass (public route, no account) is unchanged. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(mcp): reserve /mcp for the MCP API — move the shell page to /mcp-server and route bare /mcp to the handler The shell registered a public GET page at /mcp. The MCP endpoint is a Starlette Mount, which only matches /mcp/..., so the page won the bare path: POST /mcp answered 405 with no WWW-Authenticate challenge (spec clients POST to the advertised resource .../mcp and never saw the OAuth challenge), and AuthMiddleware treated every /mcp request as public. add_route's duplicate check never saw the clash because the Mount goes straight onto the router. - shell: the MCP marketing page moves to /mcp-server (still unlisted). - mcp: an exact, method-agnostic Route serves bare /mcp alongside the Mount, so /mcp and /mcp/ reach handle_mcp identically (no 405/307). - mcp: initModule refuses to start if any route already claims /mcp or /mcp/*, or a public pattern matches the endpoint. - mcp: MCP_DEV_NO_AUTH now opens exactly /mcp and /mcp/ (loopback only). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * feat(mcp): default the engine URI to the public MCP origin, or the local address on loopback binds ROCKETRIDE_URI had no default, so on staging/prod every MCP tool call raised "Missing engine URI". The URI is now resolved once in initModule and injected into the config every engine client is built from, and it also drives the widget CSP origin and the upload/dropper links: 1. explicit: config rocketride_uri / env ROCKETRIDE_URI, used as-is 2. loopback default: ws://127.0.0.1:<port> (ws://[::1]:<port>) so a laptop engine never silently calls RocketRide's cloud 3. public default: the MCP_RESOURCE_IDENTIFIER origin, https->wss / http->ws (default wss://api.rocketride.ai) One startup log line names the resolved URI (userinfo/query stripped) and the rule that chose it. ROCKETRIDE_AUTH/APIKEY handling is unchanged. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(mcp): only offer send_files on loopback-bound engines — it reads paths on the engine host The SDK's send_files checks and opens each caller-supplied path on the machine running the MCP module. On a deployed engine that is our server, so any authenticated caller could probe file existence ("File not found") and upload server files into their own pipeline. initModule now decides local_engine = is_loopback_bind(bind_host) once and passes it to build_mcp_server -> ToolRegistry(local_engine=...). Tools registered local_engine_only (send_files) are neither listed nor dispatched on a deployed engine: a call by name gets an Unavailable refusal that ignores the client and arguments. The registry defaults to the deployed surface. Loopback binds are unchanged. Also corrects the send_files description (engine-host paths, not store-relative), drops the stale "or filepath" from run_pipeline, and updates the MCP docs and agent skill contract. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(mcp): send no SDK env from deployed engines so caller secrets keep precedence WsEngineClient built RocketRideClient(uri, auth) with no env, so the SDK loaded os.environ plus ./.env and use() forwarded every ROCKETRIDE_* var as the run's env. The engine applies caller env last, over org/team/user secrets, so on a deployed engine the server's own ROCKETRIDE_* vars overrode a customer's same-named secrets for MCP-started runs. initModule injects local_engine (the same loopback decision that gates send_files) into config alongside the resolved URI; make_engine_client builds the client with env={} unless it is set. Loopback binds keep the SDK default. Explicit uri/auth are unaffected. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(mcp): rewrite deploy tools against the current SDK deploy API The MCP deploy tools (#1880) were written against a Python SDK deploy API that deploy-2 (#1764, 0dfdc8b) had already removed. In production every call failed: deploy_add passed a `schedule` kwarg add() no longer takes, deploy_status/deploy_update called methods that no longer exist, deploy_remove omitted the now-required team_id, and deploy_list counted the keys of the {rows,total,page,pageSize} envelope. The suite stayed green because FakeEngine and the seam tests mirrored the old API. The seam (WsEngineClient) is now one keyword-argument passthrough per DeployApi method, and the tools are thin wrappers in the teams-as-environments model: - deploy_add -> add (pipeline, comment, deploy_to) - deploy_list -> list (team_id, paging, search, filters, sort); returns deployments + count/total/page/pageSize - deploy_status -> get (project_id, team_id) - deploy_versions -> versions (project_id, paging) - deploy_to_team -> deploy (project_id, version, team_id) - deploy_set_schedule -> set_schedule (project_id, source_id, schedule, team_id, ttl) - deploy_enable / deploy_disable / deploy_remove -> enable / disable / remove (project_id, team_id) deploy_update is gone; its operations are deploy_add + deploy_to_team and deploy_set_schedule. The rocketride://pipelines resource returns the unwrapped envelope. Tool count: 33 (32 on a deployed engine). Drift guard: test_engine.py calls every deploy_* seam method against a recording fake and binds each recorded call to the real rocketride.deploy.DeployApi signature (method must exist, arguments must bind, every seam argument must arrive under the same name). It fails on the old seam code. A second test pins the EngineClient protocol and the conftest FakeEngineClient to the seam's deploy signatures. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(task_http): don't re-send engine env as caller env from task_Execute on deployed engines task_Execute built RocketRideClient(uri, auth) with no env, so the SDK snapshotted os.environ plus ./.env and use() forwarded every ROCKETRIDE_* var as the run's caller env. The engine applies caller env last, over the org/team/user secrets from get_merged_env, so on a deployed engine the server's own ROCKETRIDE_* vars overrode a user's same-named secrets for runs started via POST /task. Apply the rule MCP already uses (7670865): the client is built with env={} unless the server's configured bind host is loopback (ai.modules.mcp.auth.is_loopback_bind, falling back to the same CONST_DEFAULT_WEB_HOST WebServer binds with). Loopback binds keep the SDK default so a local developer's .env still applies. The MCP import is deferred so loading task_http doesn't pull in the MCP package. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(mcp): close the credential-less auth hole and harden the MCP boundary (#2315) Addresses the four review findings on PR #2315. 1. Log the resolved bind mode at startup. `is_loopback_bind` judges the CONFIGURED host and `CONST_DEFAULT_WEB_HOST` is 'localhost', so a deployment that never sets `host` silently runs as a local engine. `is_loopback_bind` is unchanged (as the reviewer asked); instead the first lines of the log now state the mode, the host it was judged from, and what that grants (dev-bypass eligibility, `send_files`, caller-env forwarding), with a WARNING when the bind is loopback only by fallback. 2. Refuse credential-less MCP requests. A public route pattern matching an MCP descendant (`/mcp/{path}`) makes AuthMiddleware skip the request while the Mount still forwards it to `handle_mcp`, where `authorize()` used to return None and run tools on the shared, server-identity engine client. Both ends are fixed: `authorize()` now refuses a request with no credential unless the loopback dev bypass is genuinely active, and the startup guard probes representative descendant paths through `is_public_route`, not just `/mcp` and `/mcp/`. The bypass rule itself lives in one place -- `auth.dev_bypass_active` -- used by both `initModule` and `authorize`. 3. Require encrypted engine transport on a non-loopback bind. `handle_mcp` hands the caller's credential to `WsEngineClient`, which sends it in the first DAP `auth` message, so a `ws://`/`http://` engine URI (explicit or derived from the MCP resource identifier) now fails boot with a message naming the value and the variable it came from. Loopback binds keep cleartext; the https staging/prod defaults are unaffected. 4. Reconcile MCP_TOOL_CONTRACT.md with the registrations. The tables listed ~23 of the 33 registered tools: `resolve_config` and `scaffold_node` were undocumented and grouped capability rows hid individual tools. Every tool now has its own row derived from its real schema and result, and the stale `origin/develop @ eb67dde` pin is replaced with what is accurate now. Test-first: 27 new tests fail against the previous implementation. Three existing tests encoded behaviour findings 2 and 3 deliberately change and were updated; the WebServer-backed route tests now clear ROCKETRIDE_URI / MCP_RESOURCE_IDENTIFIER after construction so a developer's dist/server/.env no longer decides what they resolve. 717 passed, 0 skipped (was 683). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(mcp): key the startup guards on patterns and targets, not probes and binds (#2315) Three follow-ups to the review findings on PR #2315. 1. The startup guard now inspects the registered public PATTERN STRINGS, not just probe literals. Public entries are compiled with `compile_path`, so a converter-typed pattern (`/mcp/{id:int}`, `/mcp/{x:uuid}`) narrows the regex to values no fixed probe satisfies -- the guard missed it while AuthMiddleware still skipped auth for every `/mcp/<digits>` the Mount forwards. `_registered_public_patterns` reads `_public_paths` (real WebServer) and `public` (test double), the same pair the dev bypass already writes to, and degrades to nothing when neither is present. `_claims_mcp_pattern` normalises (whitespace, trailing slashes) and refuses only a whole-segment match, so `/mcp`, `/mcp/` and `/mcp/{...}` trip while `/mcpx` and `/mcp-server` do not. Message shape and per-claimant naming are unchanged. 2. The cleartext refusal now keys on the engine URI's TARGET HOST rather than the bind. A credential sent to `ws://127.0.0.1:5565` never reaches a wire anyone can tap, and the bind-keyed rule was a deployment hazard: `WebServer.__init__` loads `dist/server/.env`, `.env.template` ships `ROCKETRIDE_URI=http://localhost:5565`, and `docker/Dockerfile.engine` copies `dist/server/` into the image, so a pod carrying that file would have failed to boot. `auth.is_loopback_bind` judges a configured bind string against exact literals and does not fit a URI netloc (userinfo, port, IPv6 brackets, the whole 127.0.0.0/8 range), so a small `_targets_loopback_host` helper was added next to the rule instead of bending it; it still reuses `auth.LOOPBACK_HOSTS` for the by-name allowlist. Non-loopback targets fail boot as before, with the same redaction. 3. `MCP_TOOL_CONTRACT.md`'s `scaffold_node` row described only `local_nodes/<name>/...`, so a contract-following client omitted the `local_nodes/__init__.py` package marker `_scaffold_node` also returns and the engine could not import `local_nodes.<name>`. The row now names the marker and says to write every returned path; `docs/public/mcp/http/tools.md` carried the same omission and is corrected too. The engine-module doc now states the target-keyed transport rule. Test-first for 1 and 2: 46 new tests fail against the previous implementation. No existing test encoded the old bind-keyed rule (every refusal case already used a non-loopback target), so none needed updating. 763 passed, 0 skipped (was 717). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(mcp): refuse a cleartext engine URI on every bind, not just a public one (#2315) Closes the gap the previous commit left in `_resolve_engine_uri`. That commit moved the cleartext refusal onto the engine's TARGET host (`_targets_loopback_host`), but the explicit branch still guarded the call with the BIND: if explicit: if not loopback: _refuse_cleartext_engine_uri(...) so a loopback-bound MCP server with `ROCKETRIDE_URI=ws://engine.remote:5565` skipped the check entirely and sent the CALLER's credential over the network in the clear -- `handle_mcp` binds it to the per-request engine client and `WsEngineClient` puts it in the first DAP `auth` frame. The bind says nothing about whether the wire is exposed; only the target does. `_refuse_cleartext_engine_uri` now runs for every explicit URI, on any bind, with the same `rocketride_uri` / `ROCKETRIDE_URI` source naming and the same redaction. Loopback TARGETS stay allowed inside the helper, so the shipped `dist/server/.env` (`ROCKETRIDE_URI=http://localhost:5565`) still boots. The local `loopback` variable is gone: `auth.is_loopback_bind` is now read inline where it genuinely selects the DEFAULT URI, so no bind-keyed condition is left anywhere near the encryption rule. The `loopback default` branch resolves `ws://127.0.0.1:<port>`, a loopback target, and the `public default` branch already called the helper. Docs and docstrings that stated the old bind-keyed rule now say the check keys on the target host: `_resolve_engine_uri`, `initModule`, `_refuse_cleartext_engine_uri`, and `docs/development/engine/mcp-module.md`. `docs/public/mcp/http/self-hosting.md` never stated the rule at all even though it fails an operator's boot, so its guard list now carries it. Test-first: 8 new cases fail against the previous implementation (explicit `ws://` and `http://` remote targets on `127.0.0.1` / `localhost` / `::1` binds, from env and from config, redaction included), plus explicit `wss://` accepted on any bind and the loopback-target cases extended over loopback binds. Two existing tests used an explicit remote `ws://` URI incidentally -- one for dropper-link origins, one for `engine_origin` threading -- and now use `wss://`; neither was asserting the transport rule. 794 passed, 0 skipped (was 763). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
dylan-savage
added a commit
that referenced
this pull request
Sep 18, 2026
…, engine URI, tools) (#2315) (#2354) * fix(mcp): refuse query-string credentials on /mcp and never fall back to the shared client for authenticated callers AuthMiddleware falls back to `?auth=` when no Authorization header is sent, but auth.authorize() only read the header. `POST /mcp/?auth=<key>` therefore passed the middleware, skipped every /mcp check (tk_/pk_/cd_ refusal, JWT signature, issuer, audience) and ran tools on the server's shared engine client. - authorize() now refuses any request carrying an `auth` query parameter (parsed as Starlette parses query_params), per the MCP authorization spec. - Defence in depth: a request the middleware authenticated (scope['state']['account']) that carries no header credential is refused instead of served by the shared client. The credential-less dev bypass (public route, no account) is unchanged. * fix(mcp): reserve /mcp for the MCP API — move the shell page to /mcp-server and route bare /mcp to the handler The shell registered a public GET page at /mcp. The MCP endpoint is a Starlette Mount, which only matches /mcp/..., so the page won the bare path: POST /mcp answered 405 with no WWW-Authenticate challenge (spec clients POST to the advertised resource .../mcp and never saw the OAuth challenge), and AuthMiddleware treated every /mcp request as public. add_route's duplicate check never saw the clash because the Mount goes straight onto the router. - shell: the MCP marketing page moves to /mcp-server (still unlisted). - mcp: an exact, method-agnostic Route serves bare /mcp alongside the Mount, so /mcp and /mcp/ reach handle_mcp identically (no 405/307). - mcp: initModule refuses to start if any route already claims /mcp or /mcp/*, or a public pattern matches the endpoint. - mcp: MCP_DEV_NO_AUTH now opens exactly /mcp and /mcp/ (loopback only). * feat(mcp): default the engine URI to the public MCP origin, or the local address on loopback binds ROCKETRIDE_URI had no default, so on staging/prod every MCP tool call raised "Missing engine URI". The URI is now resolved once in initModule and injected into the config every engine client is built from, and it also drives the widget CSP origin and the upload/dropper links: 1. explicit: config rocketride_uri / env ROCKETRIDE_URI, used as-is 2. loopback default: ws://127.0.0.1:<port> (ws://[::1]:<port>) so a laptop engine never silently calls RocketRide's cloud 3. public default: the MCP_RESOURCE_IDENTIFIER origin, https->wss / http->ws (default wss://api.rocketride.ai) One startup log line names the resolved URI (userinfo/query stripped) and the rule that chose it. ROCKETRIDE_AUTH/APIKEY handling is unchanged. * fix(mcp): only offer send_files on loopback-bound engines — it reads paths on the engine host The SDK's send_files checks and opens each caller-supplied path on the machine running the MCP module. On a deployed engine that is our server, so any authenticated caller could probe file existence ("File not found") and upload server files into their own pipeline. initModule now decides local_engine = is_loopback_bind(bind_host) once and passes it to build_mcp_server -> ToolRegistry(local_engine=...). Tools registered local_engine_only (send_files) are neither listed nor dispatched on a deployed engine: a call by name gets an Unavailable refusal that ignores the client and arguments. The registry defaults to the deployed surface. Loopback binds are unchanged. Also corrects the send_files description (engine-host paths, not store-relative), drops the stale "or filepath" from run_pipeline, and updates the MCP docs and agent skill contract. * fix(mcp): send no SDK env from deployed engines so caller secrets keep precedence WsEngineClient built RocketRideClient(uri, auth) with no env, so the SDK loaded os.environ plus ./.env and use() forwarded every ROCKETRIDE_* var as the run's env. The engine applies caller env last, over org/team/user secrets, so on a deployed engine the server's own ROCKETRIDE_* vars overrode a customer's same-named secrets for MCP-started runs. initModule injects local_engine (the same loopback decision that gates send_files) into config alongside the resolved URI; make_engine_client builds the client with env={} unless it is set. Loopback binds keep the SDK default. Explicit uri/auth are unaffected. * fix(mcp): rewrite deploy tools against the current SDK deploy API The MCP deploy tools (#1880) were written against a Python SDK deploy API that deploy-2 (#1764, 0dfdc8b) had already removed. In production every call failed: deploy_add passed a `schedule` kwarg add() no longer takes, deploy_status/deploy_update called methods that no longer exist, deploy_remove omitted the now-required team_id, and deploy_list counted the keys of the {rows,total,page,pageSize} envelope. The suite stayed green because FakeEngine and the seam tests mirrored the old API. The seam (WsEngineClient) is now one keyword-argument passthrough per DeployApi method, and the tools are thin wrappers in the teams-as-environments model: - deploy_add -> add (pipeline, comment, deploy_to) - deploy_list -> list (team_id, paging, search, filters, sort); returns deployments + count/total/page/pageSize - deploy_status -> get (project_id, team_id) - deploy_versions -> versions (project_id, paging) - deploy_to_team -> deploy (project_id, version, team_id) - deploy_set_schedule -> set_schedule (project_id, source_id, schedule, team_id, ttl) - deploy_enable / deploy_disable / deploy_remove -> enable / disable / remove (project_id, team_id) deploy_update is gone; its operations are deploy_add + deploy_to_team and deploy_set_schedule. The rocketride://pipelines resource returns the unwrapped envelope. Tool count: 33 (32 on a deployed engine). Drift guard: test_engine.py calls every deploy_* seam method against a recording fake and binds each recorded call to the real rocketride.deploy.DeployApi signature (method must exist, arguments must bind, every seam argument must arrive under the same name). It fails on the old seam code. A second test pins the EngineClient protocol and the conftest FakeEngineClient to the seam's deploy signatures. * fix(task_http): don't re-send engine env as caller env from task_Execute on deployed engines task_Execute built RocketRideClient(uri, auth) with no env, so the SDK snapshotted os.environ plus ./.env and use() forwarded every ROCKETRIDE_* var as the run's caller env. The engine applies caller env last, over the org/team/user secrets from get_merged_env, so on a deployed engine the server's own ROCKETRIDE_* vars overrode a user's same-named secrets for runs started via POST /task. Apply the rule MCP already uses (7670865): the client is built with env={} unless the server's configured bind host is loopback (ai.modules.mcp.auth.is_loopback_bind, falling back to the same CONST_DEFAULT_WEB_HOST WebServer binds with). Loopback binds keep the SDK default so a local developer's .env still applies. The MCP import is deferred so loading task_http doesn't pull in the MCP package. * fix(mcp): close the credential-less auth hole and harden the MCP boundary (#2315) Addresses the four review findings on PR #2315. 1. Log the resolved bind mode at startup. `is_loopback_bind` judges the CONFIGURED host and `CONST_DEFAULT_WEB_HOST` is 'localhost', so a deployment that never sets `host` silently runs as a local engine. `is_loopback_bind` is unchanged (as the reviewer asked); instead the first lines of the log now state the mode, the host it was judged from, and what that grants (dev-bypass eligibility, `send_files`, caller-env forwarding), with a WARNING when the bind is loopback only by fallback. 2. Refuse credential-less MCP requests. A public route pattern matching an MCP descendant (`/mcp/{path}`) makes AuthMiddleware skip the request while the Mount still forwards it to `handle_mcp`, where `authorize()` used to return None and run tools on the shared, server-identity engine client. Both ends are fixed: `authorize()` now refuses a request with no credential unless the loopback dev bypass is genuinely active, and the startup guard probes representative descendant paths through `is_public_route`, not just `/mcp` and `/mcp/`. The bypass rule itself lives in one place -- `auth.dev_bypass_active` -- used by both `initModule` and `authorize`. 3. Require encrypted engine transport on a non-loopback bind. `handle_mcp` hands the caller's credential to `WsEngineClient`, which sends it in the first DAP `auth` message, so a `ws://`/`http://` engine URI (explicit or derived from the MCP resource identifier) now fails boot with a message naming the value and the variable it came from. Loopback binds keep cleartext; the https staging/prod defaults are unaffected. 4. Reconcile MCP_TOOL_CONTRACT.md with the registrations. The tables listed ~23 of the 33 registered tools: `resolve_config` and `scaffold_node` were undocumented and grouped capability rows hid individual tools. Every tool now has its own row derived from its real schema and result, and the stale `origin/develop @ eb67dde` pin is replaced with what is accurate now. Test-first: 27 new tests fail against the previous implementation. Three existing tests encoded behaviour findings 2 and 3 deliberately change and were updated; the WebServer-backed route tests now clear ROCKETRIDE_URI / MCP_RESOURCE_IDENTIFIER after construction so a developer's dist/server/.env no longer decides what they resolve. 717 passed, 0 skipped (was 683). * fix(mcp): key the startup guards on patterns and targets, not probes and binds (#2315) Three follow-ups to the review findings on PR #2315. 1. The startup guard now inspects the registered public PATTERN STRINGS, not just probe literals. Public entries are compiled with `compile_path`, so a converter-typed pattern (`/mcp/{id:int}`, `/mcp/{x:uuid}`) narrows the regex to values no fixed probe satisfies -- the guard missed it while AuthMiddleware still skipped auth for every `/mcp/<digits>` the Mount forwards. `_registered_public_patterns` reads `_public_paths` (real WebServer) and `public` (test double), the same pair the dev bypass already writes to, and degrades to nothing when neither is present. `_claims_mcp_pattern` normalises (whitespace, trailing slashes) and refuses only a whole-segment match, so `/mcp`, `/mcp/` and `/mcp/{...}` trip while `/mcpx` and `/mcp-server` do not. Message shape and per-claimant naming are unchanged. 2. The cleartext refusal now keys on the engine URI's TARGET HOST rather than the bind. A credential sent to `ws://127.0.0.1:5565` never reaches a wire anyone can tap, and the bind-keyed rule was a deployment hazard: `WebServer.__init__` loads `dist/server/.env`, `.env.template` ships `ROCKETRIDE_URI=http://localhost:5565`, and `docker/Dockerfile.engine` copies `dist/server/` into the image, so a pod carrying that file would have failed to boot. `auth.is_loopback_bind` judges a configured bind string against exact literals and does not fit a URI netloc (userinfo, port, IPv6 brackets, the whole 127.0.0.0/8 range), so a small `_targets_loopback_host` helper was added next to the rule instead of bending it; it still reuses `auth.LOOPBACK_HOSTS` for the by-name allowlist. Non-loopback targets fail boot as before, with the same redaction. 3. `MCP_TOOL_CONTRACT.md`'s `scaffold_node` row described only `local_nodes/<name>/...`, so a contract-following client omitted the `local_nodes/__init__.py` package marker `_scaffold_node` also returns and the engine could not import `local_nodes.<name>`. The row now names the marker and says to write every returned path; `docs/public/mcp/http/tools.md` carried the same omission and is corrected too. The engine-module doc now states the target-keyed transport rule. Test-first for 1 and 2: 46 new tests fail against the previous implementation. No existing test encoded the old bind-keyed rule (every refusal case already used a non-loopback target), so none needed updating. 763 passed, 0 skipped (was 717). * fix(mcp): refuse a cleartext engine URI on every bind, not just a public one (#2315) Closes the gap the previous commit left in `_resolve_engine_uri`. That commit moved the cleartext refusal onto the engine's TARGET host (`_targets_loopback_host`), but the explicit branch still guarded the call with the BIND: if explicit: if not loopback: _refuse_cleartext_engine_uri(...) so a loopback-bound MCP server with `ROCKETRIDE_URI=ws://engine.remote:5565` skipped the check entirely and sent the CALLER's credential over the network in the clear -- `handle_mcp` binds it to the per-request engine client and `WsEngineClient` puts it in the first DAP `auth` frame. The bind says nothing about whether the wire is exposed; only the target does. `_refuse_cleartext_engine_uri` now runs for every explicit URI, on any bind, with the same `rocketride_uri` / `ROCKETRIDE_URI` source naming and the same redaction. Loopback TARGETS stay allowed inside the helper, so the shipped `dist/server/.env` (`ROCKETRIDE_URI=http://localhost:5565`) still boots. The local `loopback` variable is gone: `auth.is_loopback_bind` is now read inline where it genuinely selects the DEFAULT URI, so no bind-keyed condition is left anywhere near the encryption rule. The `loopback default` branch resolves `ws://127.0.0.1:<port>`, a loopback target, and the `public default` branch already called the helper. Docs and docstrings that stated the old bind-keyed rule now say the check keys on the target host: `_resolve_engine_uri`, `initModule`, `_refuse_cleartext_engine_uri`, and `docs/development/engine/mcp-module.md`. `docs/public/mcp/http/self-hosting.md` never stated the rule at all even though it fails an operator's boot, so its guard list now carries it. Test-first: 8 new cases fail against the previous implementation (explicit `ws://` and `http://` remote targets on `127.0.0.1` / `localhost` / `::1` binds, from env and from config, redaction included), plus explicit `wss://` accepted on any bind and the loopback-target cases extended over loopback binds. Two existing tests used an explicit remote `ws://` URI incidentally -- one for dropper-link origins, one for `engine_origin` threading -- and now use `wss://`; neither was asserting the transport rule. 794 passed, 0 skipped (was 763). --------- (cherry picked from commit 0d12d30) Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
dk-rocketride
added a commit
that referenced
this pull request
Sep 23, 2026
* docs(agents): Drizzle in the agent bundle + restore the task battery (#2298)
* docs(agents): restore the user task battery beside the agent docs it grades
The 256-item acceptance sheet for the ROCKETRIDE_* context docs was dropped
by the docs revamp (#2139). Bring it back at docs/agents/task-battery.md,
outside context/ so the /client/docs bundle does not ship it, point its
scoring-pass note at the post-revamp paths, and describe it in the README.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* docs: surface the Drizzle ORM driver in the agent bundle and SDK references
The Drizzle-over-pipes driver (#1908) updated the public guide pages but
not the agent-facing context, the pipeline-skill doc map, or the SDK
reference tables, which still steered readers to the deprecated
Sequelize binding.
- ROCKETRIDE_typescript_API.md: add a `drizzle()` subsection, mark
`sequelize()` deprecated, document `rowMode` on `query()`
- ROCKETRIDE_DOC_MAP.md: link the Drizzle page
- typescript/reference.md: add `drizzle` row and types, flag Sequelize
as deprecated, add `rowMode` to the `query` signature
- python/reference.md: point the cross-reference at Drizzle
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* docs: address review on the Drizzle agent-doc surfacing
- Point the agent bundle at the absolute docs-site URL for the Drizzle
guide; the bundle ships into user workspaces where repo paths do not
resolve
- `debug` is an npm package, not a Node built-in: name `util` and the
`tty`/`util` pulled in via `debug` as the actual browser-bundle blockers
(agent doc and public Sequelize guide)
- Note the `unknown[][]` return shape under `rowMode: 'array'` and that
`rowMode` is TypeScript-only
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
* fix(nodes): stop four nodes passing raw input downstream (#2304)
After a Python lane handler returns normally, the engine also forwards the handler's input on the same lane. Only `preventDefault()` stops that. Four nodes returned normally, so downstream received the raw input next to the node's own output.
- `context_optimizer`: both exits of `writeQuestions` now end in `return self.preventDefault()`. Downstream no longer gets the untrimmed original next to the trimmed question.
- `summarization`: `writeText` and `writeTable` collect the text, then suppress the forward. Only the summary from `closing()` goes on.
- `tool_guild`: `writeText`, `writeQuestions` and `writeDocuments` collect, then suppress. The docstring claiming "overriding suppresses raw passthrough" is corrected.
- `tool_n8n`: the same for `writeText`, `writeQuestions`, `writeDocuments`, and every `writeImage` / `writeAudio` / `writeVideo` chunk.
- Missing docstrings added in the four node files.
Tests:
- `test_lane_forward_once_nodes.py`: delivery-count cases for every changed handler.
- New `summarization/test_summarization.py`: the node had no unit tests.
- `context_optimizer/test_all.py` and `test_tool_n8n.py`: direct handler calls now require the `PreventDefault` exception.
* fix(memory_persistent): prevent double-forward of questions and answers (#1682)
* fix(memory_persistent): prevent double-forward of questions and answers
* test(memory_persistent): assert preventDefault is called on explicit forwards
* docs(memory_persistent): note preventDefault() unreachability and test stub caveat
* fix(memory_persistent): cover all six exits with delivery-count tests
- Every exit of `writeQuestions` and `writeAnswers` now ends in `return self.preventDefault()`. The separate `return` after `preventDefault()` could never run, because `preventDefault()` raises.
- `test_node.py`: two tests for the invalid `session_id` paths, which no test reached before. Removing `preventDefault()` from either path now fails a test.
- `test_lane_forward_once_nodes.py`: six delivery-count cases, one per exit (no store, invalid session, normal path), under the engine's forward rule. On the normal path, only the enriched or stored copy reaches downstream, never the original.
- Docstrings in `IInstance.py` now document their arguments.
---------
Co-authored-by: Alexandru Sclearuc <asclearuc@gmail.com>
* fix(audio_player): accept stop()'s leftover sentinel in the drain test (#2306)
`stop()` queues its own `None` sentinel when it starts waiting before the audio callback has reached the decoder's. The callback stops at the first `None`, so the second one stays in the queue. `test_begin_write_end_still_drains_normally` still required an empty queue, so it failed whenever `stop()` won that race, which happens on a loaded CI runner.
- The drain test now accepts `[]` or `[None]` as the leftover. Unplayed audio still fails it.
- New `test_stop_waiting_before_the_callback_leaves_only_its_own_sentinel` forces the `stop()`-first order. It requires exactly `[None]` to be left, all audio played, and `stop()` returned.
- The shared setup moved into `_player_with_queued_audio()` and `_start_simulated_hardware()`.
* fix(ci): open the model-sync PR with the release-bot App token (#2312)
Mint a release-bot GitHub App token and pass it to create-pull-request so the weekly Sync LLM Models PR triggers CI on its own instead of waiting for a manual approve. Falls back to github.token if the token can't be minted. release-bot was granted Pull requests: write on the org install. Fixes #2311.
* feat(profiler): per-thread breakdown and a call tree that keeps real work (#2299)
* feat(ai): break the profiler report down by thread
stop() now also captures the session per yappi context, next to the flat
capture and before clear_stats(). threads() lists them busiest first with
their counts, and report_tree() takes an optional thread id; without one it
behaves as before. DAP: new rrext_cprofile_threads, plus an optional thread
argument on rrext_cprofile_report_tree, in the engine subprocess and the
task-server proxy.
Two yappi traps shape the capture:
- get_func_stats(ctx_id=...) tests the id for truthiness, so thread 0 would
silently get every thread's functions. The filter goes in as a dict.
- The default context name is the thread's class, so workers read 'Thread'
and engine workers '_DummyThread'. start() installs a callback reading
Thread.name from threading._active; current_thread() would deadlock when
the profiled call is _active_limbo_lock.acquire() (yappi #48). yappi keeps
the first name it gets, so the thread ident is reported as well.
Cost on a parse -> preprocessor pipeline over 12 and 36 documents: the
per-thread copy holds 2.1-3.5 MB against 0.5 MB flat (13-25 threads), and
the capture under _lock grows from ~117 ms to ~250-275 ms.
* perf(ai): cache _relativize_path
stop() calls it under _lock for every entry and callee of each capture:
about 5,900 calls per stop on a parse pipeline over 12 documents, since the
per-thread capture repeats the flat one's paths, over a few hundred
distinct paths. The result depends only on the path, so cache it.
On the same pipeline the capture under _lock drops from ~250-275 ms to
~160-170 ms, against ~117 ms before the per-thread capture existed. Memory
drops only ~10%: every entry of a path now shares one string, but the
per-entry dicts are most of the size.
* fix(ai): stop the call tree from pruning away real work
report_tree() dropped every node whose own time fell under min_pct, and with
it the whole subtree. Two kinds of node read light while real work hangs
below them: frames still running at stop() (thread entry points, executor
worker loops) read ~0 because yappi books time on return, and the event
loop step that resumes a coroutine is short while the coroutine is booked
its whole lifetime. On a parse pipeline this emptied the merged tree, the
event-loop thread and every executor thread started mid-session; threads
that predated the session survived only because their zero-time frames sat
at depths 0 and 1, which are never pruned.
A node is now pruned only when nothing reachable below it reaches the
threshold either. The threshold was also inflated: total_time summed ttot,
counting nested calls again (a 5 s session summed to 412 s). It is now the
time the threads ran, as yappi measures it per thread - the selected one,
or all summed - falling back to the sum of self times without thread data.
Same session after the fix: project code shows in every view at the
default threshold, and trees stay at 4-168 nodes, built in under 60 ms.
* feat(client): list profiled threads and fetch one thread's tree
Python: cprofile_threads() and a thread parameter on cprofile_report_tree().
TypeScript: cprofileThreads() and a trailing thread parameter on
cprofileReportTree(), with CProfileThreadInfo / CProfileThreadsResponse.
Thread 0 is a real id, so both clients test for absence, not truthiness.
The shell re-exports the SDK's types, so the two new ones are new shell-api
names and shell:check fails until they are frozen: this freezes shell-api
v4. Its delta over v3 is exactly these types and the two method signatures.
The contract test runs data through a task's pipeline and checks that no
thread carries yappi's class-based names, that one thread's tree totals that
thread's calls, and that an unknown id is refused, which proves the
argument reached the engine.
test-ui's API sweep and the SDK references list the new method; the Python
reference also claimed include_system defaults to False, which it never did.
* feat(profiler-ui): pick a thread to visualise
A Thread dropdown heads the viz controls: all threads (the old view) or one,
labelled with name, thread id and time, since names repeat. The views get
the same tree shape either way. The list is refetched with the report, a
selection missing from the new list falls back to all threads, and a server
without rrext_cprofile_threads just hides the dropdown. Start clears the
list, since ids are per session.
Bumps the app to 1.3.0: an OSS server reseeds a built-in app only when its
apps.json version changes, so an upgraded server would keep serving the
old profiler otherwise.
* fix(profiler-ui): prune the charts by subtree weight
The client-side cutoff compared a child's own cumtime with its parent's,
so a zero-time frame the server now keeps was dropped again at the root,
taking the work under it along. It now compares the largest cumtime in
each subtree, and runs before the depth limit so that weight covers the
whole subtree.
* fix(profiler-ui): clear the two type errors tsc reports
initConnectionStore took the shell's settings as Record<string, string>,
but they are Record<string, SettingValue>, and a port setting may well be
a number; the seeded host and port are now coerced to strings. The
sunburst filtered on x0/x1 before casting to HierarchyRectangularNode,
which the flame graph already does first. Neither changes behaviour: the
app build does not type-check, which is how both went unnoticed.
* fix: CodeRabbit
* fix(mcp): make the HTTP MCP server testable end to end (auth, routing, engine URI, tools) (#2315)
* fix(mcp): refuse query-string credentials on /mcp and never fall back to the shared client for authenticated callers
AuthMiddleware falls back to `?auth=` when no Authorization header is sent,
but auth.authorize() only read the header. `POST /mcp/?auth=<key>` therefore
passed the middleware, skipped every /mcp check (tk_/pk_/cd_ refusal, JWT
signature, issuer, audience) and ran tools on the server's shared engine
client.
- authorize() now refuses any request carrying an `auth` query parameter
(parsed as Starlette parses query_params), per the MCP authorization spec.
- Defence in depth: a request the middleware authenticated
(scope['state']['account']) that carries no header credential is refused
instead of served by the shared client. The credential-less dev bypass
(public route, no account) is unchanged.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(mcp): reserve /mcp for the MCP API — move the shell page to /mcp-server and route bare /mcp to the handler
The shell registered a public GET page at /mcp. The MCP endpoint is a
Starlette Mount, which only matches /mcp/..., so the page won the bare
path: POST /mcp answered 405 with no WWW-Authenticate challenge (spec
clients POST to the advertised resource .../mcp and never saw the OAuth
challenge), and AuthMiddleware treated every /mcp request as public.
add_route's duplicate check never saw the clash because the Mount goes
straight onto the router.
- shell: the MCP marketing page moves to /mcp-server (still unlisted).
- mcp: an exact, method-agnostic Route serves bare /mcp alongside the
Mount, so /mcp and /mcp/ reach handle_mcp identically (no 405/307).
- mcp: initModule refuses to start if any route already claims /mcp or
/mcp/*, or a public pattern matches the endpoint.
- mcp: MCP_DEV_NO_AUTH now opens exactly /mcp and /mcp/ (loopback only).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* feat(mcp): default the engine URI to the public MCP origin, or the local address on loopback binds
ROCKETRIDE_URI had no default, so on staging/prod every MCP tool call
raised "Missing engine URI". The URI is now resolved once in initModule
and injected into the config every engine client is built from, and it
also drives the widget CSP origin and the upload/dropper links:
1. explicit: config rocketride_uri / env ROCKETRIDE_URI, used as-is
2. loopback default: ws://127.0.0.1:<port> (ws://[::1]:<port>) so a
laptop engine never silently calls RocketRide's cloud
3. public default: the MCP_RESOURCE_IDENTIFIER origin, https->wss /
http->ws (default wss://api.rocketride.ai)
One startup log line names the resolved URI (userinfo/query stripped)
and the rule that chose it. ROCKETRIDE_AUTH/APIKEY handling is unchanged.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(mcp): only offer send_files on loopback-bound engines — it reads paths on the engine host
The SDK's send_files checks and opens each caller-supplied path on the
machine running the MCP module. On a deployed engine that is our server, so
any authenticated caller could probe file existence ("File not found") and
upload server files into their own pipeline.
initModule now decides local_engine = is_loopback_bind(bind_host) once and
passes it to build_mcp_server -> ToolRegistry(local_engine=...). Tools
registered local_engine_only (send_files) are neither listed nor dispatched
on a deployed engine: a call by name gets an Unavailable refusal that ignores
the client and arguments. The registry defaults to the deployed surface.
Loopback binds are unchanged.
Also corrects the send_files description (engine-host paths, not
store-relative), drops the stale "or filepath" from run_pipeline, and
updates the MCP docs and agent skill contract.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(mcp): send no SDK env from deployed engines so caller secrets keep precedence
WsEngineClient built RocketRideClient(uri, auth) with no env, so the SDK
loaded os.environ plus ./.env and use() forwarded every ROCKETRIDE_* var as
the run's env. The engine applies caller env last, over org/team/user
secrets, so on a deployed engine the server's own ROCKETRIDE_* vars
overrode a customer's same-named secrets for MCP-started runs.
initModule injects local_engine (the same loopback decision that gates
send_files) into config alongside the resolved URI; make_engine_client
builds the client with env={} unless it is set. Loopback binds keep the SDK
default. Explicit uri/auth are unaffected.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(mcp): rewrite deploy tools against the current SDK deploy API
The MCP deploy tools (#1880) were written against a Python SDK deploy API
that deploy-2 (#1764, 0dfdc8b44) had already removed. In production every
call failed: deploy_add passed a `schedule` kwarg add() no longer takes,
deploy_status/deploy_update called methods that no longer exist,
deploy_remove omitted the now-required team_id, and deploy_list counted
the keys of the {rows,total,page,pageSize} envelope. The suite stayed
green because FakeEngine and the seam tests mirrored the old API.
The seam (WsEngineClient) is now one keyword-argument passthrough per
DeployApi method, and the tools are thin wrappers in the
teams-as-environments model:
- deploy_add -> add (pipeline, comment, deploy_to)
- deploy_list -> list (team_id, paging, search, filters, sort); returns
deployments + count/total/page/pageSize
- deploy_status -> get (project_id, team_id)
- deploy_versions -> versions (project_id, paging)
- deploy_to_team -> deploy (project_id, version, team_id)
- deploy_set_schedule -> set_schedule (project_id, source_id, schedule,
team_id, ttl)
- deploy_enable / deploy_disable / deploy_remove -> enable / disable /
remove (project_id, team_id)
deploy_update is gone; its operations are deploy_add + deploy_to_team and
deploy_set_schedule. The rocketride://pipelines resource returns the
unwrapped envelope. Tool count: 33 (32 on a deployed engine).
Drift guard: test_engine.py calls every deploy_* seam method against a
recording fake and binds each recorded call to the real
rocketride.deploy.DeployApi signature (method must exist, arguments must
bind, every seam argument must arrive under the same name). It fails on
the old seam code. A second test pins the EngineClient protocol and the
conftest FakeEngineClient to the seam's deploy signatures.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(task_http): don't re-send engine env as caller env from task_Execute on deployed engines
task_Execute built RocketRideClient(uri, auth) with no env, so the SDK
snapshotted os.environ plus ./.env and use() forwarded every ROCKETRIDE_*
var as the run's caller env. The engine applies caller env last, over the
org/team/user secrets from get_merged_env, so on a deployed engine the
server's own ROCKETRIDE_* vars overrode a user's same-named secrets for
runs started via POST /task.
Apply the rule MCP already uses (76708650a): the client is built with
env={} unless the server's configured bind host is loopback
(ai.modules.mcp.auth.is_loopback_bind, falling back to the same
CONST_DEFAULT_WEB_HOST WebServer binds with). Loopback binds keep the SDK
default so a local developer's .env still applies. The MCP import is
deferred so loading task_http doesn't pull in the MCP package.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(mcp): close the credential-less auth hole and harden the MCP boundary (#2315)
Addresses the four review findings on PR #2315.
1. Log the resolved bind mode at startup. `is_loopback_bind` judges the
CONFIGURED host and `CONST_DEFAULT_WEB_HOST` is 'localhost', so a
deployment that never sets `host` silently runs as a local engine.
`is_loopback_bind` is unchanged (as the reviewer asked); instead the first
lines of the log now state the mode, the host it was judged from, and what
that grants (dev-bypass eligibility, `send_files`, caller-env forwarding),
with a WARNING when the bind is loopback only by fallback.
2. Refuse credential-less MCP requests. A public route pattern matching an
MCP descendant (`/mcp/{path}`) makes AuthMiddleware skip the request while
the Mount still forwards it to `handle_mcp`, where `authorize()` used to
return None and run tools on the shared, server-identity engine client.
Both ends are fixed: `authorize()` now refuses a request with no credential
unless the loopback dev bypass is genuinely active, and the startup guard
probes representative descendant paths through `is_public_route`, not just
`/mcp` and `/mcp/`. The bypass rule itself lives in one place --
`auth.dev_bypass_active` -- used by both `initModule` and `authorize`.
3. Require encrypted engine transport on a non-loopback bind. `handle_mcp`
hands the caller's credential to `WsEngineClient`, which sends it in the
first DAP `auth` message, so a `ws://`/`http://` engine URI (explicit or
derived from the MCP resource identifier) now fails boot with a message
naming the value and the variable it came from. Loopback binds keep
cleartext; the https staging/prod defaults are unaffected.
4. Reconcile MCP_TOOL_CONTRACT.md with the registrations. The tables listed
~23 of the 33 registered tools: `resolve_config` and `scaffold_node` were
undocumented and grouped capability rows hid individual tools. Every tool
now has its own row derived from its real schema and result, and the stale
`origin/develop @ eb67ddea` pin is replaced with what is accurate now.
Test-first: 27 new tests fail against the previous implementation. Three
existing tests encoded behaviour findings 2 and 3 deliberately change and were
updated; the WebServer-backed route tests now clear ROCKETRIDE_URI /
MCP_RESOURCE_IDENTIFIER after construction so a developer's dist/server/.env
no longer decides what they resolve.
717 passed, 0 skipped (was 683).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(mcp): key the startup guards on patterns and targets, not probes and binds (#2315)
Three follow-ups to the review findings on PR #2315.
1. The startup guard now inspects the registered public PATTERN STRINGS, not
just probe literals. Public entries are compiled with `compile_path`, so a
converter-typed pattern (`/mcp/{id:int}`, `/mcp/{x:uuid}`) narrows the regex
to values no fixed probe satisfies -- the guard missed it while
AuthMiddleware still skipped auth for every `/mcp/<digits>` the Mount
forwards. `_registered_public_patterns` reads `_public_paths` (real
WebServer) and `public` (test double), the same pair the dev bypass already
writes to, and degrades to nothing when neither is present.
`_claims_mcp_pattern` normalises (whitespace, trailing slashes) and refuses
only a whole-segment match, so `/mcp`, `/mcp/` and `/mcp/{...}` trip while
`/mcpx` and `/mcp-server` do not. Message shape and per-claimant naming are
unchanged.
2. The cleartext refusal now keys on the engine URI's TARGET HOST rather than
the bind. A credential sent to `ws://127.0.0.1:5565` never reaches a wire
anyone can tap, and the bind-keyed rule was a deployment hazard:
`WebServer.__init__` loads `dist/server/.env`, `.env.template` ships
`ROCKETRIDE_URI=http://localhost:5565`, and `docker/Dockerfile.engine`
copies `dist/server/` into the image, so a pod carrying that file would have
failed to boot. `auth.is_loopback_bind` judges a configured bind string
against exact literals and does not fit a URI netloc (userinfo, port, IPv6
brackets, the whole 127.0.0.0/8 range), so a small `_targets_loopback_host`
helper was added next to the rule instead of bending it; it still reuses
`auth.LOOPBACK_HOSTS` for the by-name allowlist. Non-loopback targets fail
boot as before, with the same redaction.
3. `MCP_TOOL_CONTRACT.md`'s `scaffold_node` row described only
`local_nodes/<name>/...`, so a contract-following client omitted the
`local_nodes/__init__.py` package marker `_scaffold_node` also returns and
the engine could not import `local_nodes.<name>`. The row now names the
marker and says to write every returned path; `docs/public/mcp/http/tools.md`
carried the same omission and is corrected too. The engine-module doc now
states the target-keyed transport rule.
Test-first for 1 and 2: 46 new tests fail against the previous implementation.
No existing test encoded the old bind-keyed rule (every refusal case already
used a non-loopback target), so none needed updating.
763 passed, 0 skipped (was 717).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(mcp): refuse a cleartext engine URI on every bind, not just a public one (#2315)
Closes the gap the previous commit left in `_resolve_engine_uri`. That commit
moved the cleartext refusal onto the engine's TARGET host
(`_targets_loopback_host`), but the explicit branch still guarded the call with
the BIND:
if explicit:
if not loopback:
_refuse_cleartext_engine_uri(...)
so a loopback-bound MCP server with `ROCKETRIDE_URI=ws://engine.remote:5565`
skipped the check entirely and sent the CALLER's credential over the network in
the clear -- `handle_mcp` binds it to the per-request engine client and
`WsEngineClient` puts it in the first DAP `auth` frame. The bind says nothing
about whether the wire is exposed; only the target does.
`_refuse_cleartext_engine_uri` now runs for every explicit URI, on any bind,
with the same `rocketride_uri` / `ROCKETRIDE_URI` source naming and the same
redaction. Loopback TARGETS stay allowed inside the helper, so the shipped
`dist/server/.env` (`ROCKETRIDE_URI=http://localhost:5565`) still boots. The
local `loopback` variable is gone: `auth.is_loopback_bind` is now read inline
where it genuinely selects the DEFAULT URI, so no bind-keyed condition is left
anywhere near the encryption rule. The `loopback default` branch resolves
`ws://127.0.0.1:<port>`, a loopback target, and the `public default` branch
already called the helper.
Docs and docstrings that stated the old bind-keyed rule now say the check keys
on the target host: `_resolve_engine_uri`, `initModule`,
`_refuse_cleartext_engine_uri`, and `docs/development/engine/mcp-module.md`.
`docs/public/mcp/http/self-hosting.md` never stated the rule at all even though
it fails an operator's boot, so its guard list now carries it.
Test-first: 8 new cases fail against the previous implementation (explicit
`ws://` and `http://` remote targets on `127.0.0.1` / `localhost` / `::1`
binds, from env and from config, redaction included), plus explicit `wss://`
accepted on any bind and the loopback-target cases extended over loopback
binds. Two existing tests used an explicit remote `ws://` URI incidentally --
one for dropper-link origins, one for `engine_origin` threading -- and now use
`wss://`; neither was asserting the transport rule.
794 passed, 0 skipped (was 763).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
* fix(ai): accept the single-component validation payload the node panel sends (#2346)
* fix(ai): accept the single-component validation payload the node panel sends
The node config panel validates one component on save and sends
`{version, component}` — the `IComponentValidatePayload` shape the shell
contract declares this endpoint accepts (shell/src/types/project.ts:308). The
engine validates pipelines and requires `components`, so every save answered
"'pipeline.components' must be an array", whatever the node.
The component now travels as a one-item `components` list, built before the
source is inferred so `resolve_implied_source` sees it. A config that already
carries `components` is untouched.
Reachable since #1573: until then `rrext_validate` failed on every call with
"'pipeline' is missing or invalid", so this payload never reached the
validator. The panel has sent this shape since #1843 (7 Aug).
Removing the four lines fails the two new tests, and the engine itself answers
the reported message:
validatePipeline({'pipeline': {'version': 1, 'component': c}}) -> "'pipeline.components' must be an array"
validatePipeline({'pipeline': {'version': 1, 'components': [c]}}) -> 0 errors
* docs(clients,ai): document the single-component validate payload
The endpoint now accepts {version, component} as well as a full pipeline, which
is a public-contract change: both SDK references say so, and on_rrext_validate's
Args: block does too — the inline comment explained it, but the docstring is
what a reader hits first.
Also states that a payload carrying both keys keeps 'components' and drops the
stray 'component', so the next reader does not have to derive that it is
deliberate.
* fix(crustdata): patch HTTP transport by object in tests (#2332)
Co-authored-by: Krish Garg <kg@g14-node1.tailb99b1b.ts.net>
* fix(notion): normalize malformed success responses (#2340)
Co-authored-by: Krish Garg <kg@g14-node1.tailb99b1b.ts.net>
* fix(notion): preserve query request status (#2339)
Co-authored-by: Krish Garg <kg@g14-node1.tailb99b1b.ts.net>
* fix(search-hybrid): isolate fallback identities (#2329)
* fix(search-hybrid): isolate fallback identities
Closes #2322
* test(search-hybrid): cover empty-ID fallback
Addresses CodeRabbit review on #2329
---------
Co-authored-by: Krish Garg <kg@g14-node1.tailb99b1b.ts.net>
* fix(notion): accept search pagination cursors (#2338)
* fix(notion): accept search pagination cursors
* test(notion): cover empty search cursors
---------
Co-authored-by: Krish Garg <kg@g14-node1.tailb99b1b.ts.net>
* fix(ai,nodes): surface database error text from execute and add refresh_schema tool (#2283)
* fix(ai,nodes): surface database error text from execute and add refresh_schema tool
Two gaps in the shared database node base, both scoped to stay out of the
way of the pull requests already open against this file.
_executeRawQuery caught every SQLAlchemyError, logged it, and returned
None, so execute() answered with the same 'check server logs for details'
string whether the statement had a typo, named a missing table, or was
refused for permissions. Its except clause now raises with the driver's own
message through IGlobal._format_db_error. Only that clause changes:
execute()'s `if result is None` branch becomes unreachable but is left in
place, with a comment saying so, because an open PR is rewriting that
region. Session-bound execute already re-raised the real exception, so the
two paths finally agree, and the max_execute_rows rollback is untouched.
IGlobal.db_schema is assigned exactly once, in beginGlobal, so get_schema
serves a task-start snapshot for the life of the task and a table created
through execute stays invisible until the pipeline restarts. The new
refresh_schema tool re-reflects and replaces it, returning the get_schema
shape plus a refreshed_at UTC timestamp so a caller can show how current
the schema it holds is. Reflection and publication run under a module-level
lock: concurrent callers would otherwise repeat the same full table walk
and race to publish the result. The tool is additive and lives entirely in
db_instance_base.py, since another open PR is adding to db_global_base.py.
It also fixes the natural-language path, which describes that same dict to
the LLM. get_schema's nested table formatter is lifted to a shared private
method so both tools render one shape.
Tests are a new module rather than an addition to test_db_base.py, which
two open PRs are both appending to. Eleven cases over an in-memory SQLite
engine cover the driver text reaching the caller, the gate still refusing
first, re-reflection seeing created and dropped tables, the timestamp
format, and four concurrent refreshes all returning.
Tool tables in the four node READMEs that inherit this surface list
refresh_schema and say in one sentence what get_schema's snapshot does not
show. services.json declares no tool list, so nothing to mirror there.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T4fLnswQF6yKQ6x7eWzSdd
* test(ai): give the execute-session IGlobal stub a real _format_db_error
execute()'s stateless failure path now formats the driver error through
IGlobal, but this module builds IGlobal as a SimpleNamespace carrying only
the four fields the transaction tests needed. Nothing failed, because no
test here drove a bad statement through that path — the next one to try
would have died with AttributeError instead of the RuntimeError it was
asserting.
The stub borrows the real DatabaseGlobalBase implementation rather than a
lambda, so the message a test sees is the message a caller gets. A test for
the stateless failure path comes with it, so the fixture stays honest:
removing the one line makes it fail with AttributeError.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T4fLnswQF6yKQ6x7eWzSdd
* refactor(ai): shrink the refresh_schema diff in db_instance_base
No behaviour change; this only narrows what the previous commit touched in
a file three open pull requests are editing.
get_schema's body is back to exactly what develop has, docstring aside: the
nested _format_table closure returns, and the _format_table static method
and _schemaPayload helper it had been lifted into are gone. Sharing them
was tidier, but it rewrote a method none of the open PRs needs to change,
and a few duplicated formatting lines cost less than that. That hunk is now
five added docstring lines instead of a 31-line rewrite.
refresh_schema moves to the end of the tool section, below dialect, so the
whole tool arrives as one appended block past the execute body PR #1908
rewrites rather than as an insertion ahead of it. It formats its own tables
inline, and the lock is now _REFLECT_LOCK.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T4fLnswQF6yKQ6x7eWzSdd
* docs(nodes): complete the agent tool contract in the db READMEs
Two review findings on the READMEs this PR touched, both in the section
the node README schema calls the agent-facing contract and holds to the
highest accuracy bar.
db_postgres listed four tool functions in the summary and in "As a tool",
then a separate table of three transaction tools — seven of the nine
@tool_function methods an agent actually sees on DatabaseInstanceBase.
`execute` appeared only as prose inside the transaction paragraph and
`dialect` only as the QuestionType.DIALECT lane behaviour, so an agent
author reading the inventory would not know either exists as a tool. The
count is now nine in both places, `dialect` joins the open table, and
`execute` joins what was the Transactions table, renamed "Raw SQL and
transactions" because a statement without a session_id is neither. That
table's gate sentence also said requests are "silently dropped" when
allow_execute is off; the four tools raise (db_instance_base.py:301,
339, 371), and it is the questions lane that logs and drops — the two
behaviours are now stated apart. The `execute` row records the driver
message and the rollback-on-overflow that this PR's own change produced,
rather than implying a truncated result set.
rocketride_sql named refresh_schema in its tool table but left its
contract out of the prose beneath, where that README already documents
every other non-obvious argument and return shape. It takes no arguments
(input_schema declares no properties) and returns {database, tables}
exactly as get_schema does, plus a refreshed_at UTC ISO-8601 timestamp
from datetime.now(timezone.utc).isoformat().
Docs only; no behaviour change. Both nodes still pass
scripts/validate-node-readme.py, and so does the whole corpus under
./builder docs:validate.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(ai): keep SQL statements and bind parameters out of execute error text
`_format_db_error` only produced a clean `Error <code>: <msg>` when the
driver put an integer first in `orig.args` — the pymysql and
clickhouse-driver shape. psycopg2 and sqlite3 put a string there, so both
fell through to `str(exc)`, which for a SQLAlchemy StatementError is the
full repr: the executed statement in `[SQL: ...]` and the values bound
into it in `[parameters: ...]`.
That string is no longer log-only. `_executeRawQuery` re-raises it as a
RuntimeError that the `execute` tool returns to its caller, and #2282
wires a user-facing Refresh/Run UI onto exactly that path, so a failed
INSERT would hand the caller back the row it tried to write.
Unwrap each driver shape explicitly and keep only the primary message:
the numeric-code branch as before; psycopg2's `diag.message_primary`
(its `str()` also carries a `LINE n:` echo of the statement, which
psycopg2 has already interpolated the bind values into); and `args[0]`
for sqlite3 and the generic DBAPI shape. Every branch then runs through
`_strip_statement_detail`, which truncates at the first `[SQL:`,
`[parameters:`, `LINE n:`, `DETAIL:`/`HINT:`/`CONTEXT:` or sqlalche.me
marker, so a driver shape not enumerated here still cannot leak the
tail. DETAIL is dropped along with the rest because PostgreSQL uses it
to restate the offending key values.
The helper is module-level rather than a method so `_format_db_error`
keeps working when tests bind it onto a stub IGlobal.
Tests cover the real sqlite3 driver end-to-end through `execute`, plus
the psycopg2 (with and without `diag`), sqlite3 and pymysql shapes and a
bare StatementError repr.
`test_format_db_error_falls_back_to_str_when_args_not_int_first` pinned
the old leaking fallback, so it is replaced by an assertion that the
driver message — not `str(exc)` — is what comes back.
`_strip_statement_detail`'s degenerate case gets the same treatment. A
message that is nothing but detail markers left no primary sentence, and
the old fallback returned the first line — which on that input is the
`[SQL: ...]` echo the function exists to remove. It now returns a
neutral constant, so there is no input on which the helper hands back the
statement.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(ai): refresh the answers-lane column map in refresh_schema
`refresh_schema` replaced `IGlobal.db_schema`, the database-wide
reflection the natural-language path describes to the LLM, and stopped
there. `IGlobal.schema` — the configured table's column map that
`_insertData` iterates to build every answers-lane INSERT — is a second
start-up snapshot, and it was left untouched, rebuilt only lazily when
falsy.
So the node was current on one path and stale on the other: after
running `ALTER TABLE ... ADD COLUMN` and calling the refresh tool, the
LLM path saw the new column while the insert lane kept writing the
start-up column set and silently dropped it. The four db node READMEs
document both the insert lane and "re-reads the schema from the
database", so the documented contract overstated the code.
Invalidate the map inside the same reflection lock. Emptying it re-arms
the lazy rebuild already at the top of `_insertData`, which reflects
through the same `_getTableSchema` call `beginGlobal` uses, so the next
insert sees what a freshly started node would. Emptying rather than
re-reflecting here keeps the call cheap for a node with no answers lane
wired, and makes the write a single atomic rebind: a concurrent
`_insertData` either reads the old map or finds it falsy and rebuilds,
never a half-built dict.
Invalidating the map is only safe once `_insertData` stops binding a
generated primary key, so that is fixed in the same commit.
`_createTableFromData` curates `IGlobal.schema` down to the data columns
for an auto-created table, because the `id` it prepends is generated by
the database; a map built by reflection carries `id` instead. Since
`_insertData` binds NULL for any schema column the rows do not supply,
swapping the curated map for a reflected one would have started binding
`id=None` on every later insert — tolerated by SQLite's rowid alias, a
not-null violation against the `id SERIAL NOT NULL` Postgres renders for
the same column. `_insertData` now drops primary-key columns that no row
in the batch supplies, decided once per batch so every mapping handed to
executemany keeps an identical key set. The two maps therefore produce
the same INSERT and refreshing cannot change answers-lane behaviour
mid-task. It also fixes the same NULL bind on the path that always
reflected: a table that already existed when `beginGlobal` ran.
Tests cover the refreshed insert picking up a new column, the
pre-existing stale behaviour it fixes, a dropped configured table leaving
a falsy map, and — by asserting on the compiled INSERT rather than on
what SQLite tolerates — that an auto-created table's insert still carries
the data columns only after a refresh, that a reflected primary key the
rows do not supply is omitted, that one they do supply is still bound,
and that a primary-key-only table falls back to binding it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* docs(nodes): describe the raw-SQL gate the db nodes actually have
The `Raw SQL and transactions` section said `allow_execute` is "the same
gate as `QuestionType.EXECUTE`" and contrasted the tools with "the
`questions` lane, which logs and drops the request". No SQL database
node implements that branch: `DatabaseInstanceBase.writeQuestions` does
not look at `question.type` at all, and `QuestionType` appears in
db_instance_base.py only at the import and in one unrelated
`Question(type=QuestionType.QUESTION, ...)` construction. The
dispatch-on-type behaviour belongs to the graph node
(`graph_instance_base.py` handles `DIALECT` and `EXECUTE`), and the
wording was carried across without being checked against this file.
Correct it wherever it appears in prose:
- db_postgres: the "Two special question types are handled on the
questions lane" bullets are removed — the lane has one behaviour,
translate-then-execute, and `dialect` / `execute` are the tool
functions that reach those behaviours. The gate sentence now says
what `allow_execute` gates (the four raw-SQL tools) and that it does
not change the questions lane, which never runs raw SQL.
- rocketride_sql: same two claims, in the config and Notes sections.
Also make the refresh_schema wording match the code now that the tool
invalidates the answers-lane column map as well as the database-wide
schema, so "re-reads the schema from the database" is true end to end
rather than only for the LLM path.
The same claim lived in the generated config tables, whose source of
truth is the `allow_execute` field description in each node's
services*.json. Those five descriptions now name the four tool functions
the flag actually gates, and the generated regions are regenerated from
them, so db_postgres/README.md no longer contradicts itself between its
prose and its parameter table. Only the SQL nodes are touched: the graph
nodes do dispatch on `QuestionType.EXECUTE`, so their copy is correct as
it stands.
Two smaller instances of the same transposition: the new db_postgres
sentence named `db_neo4j`, a retired path that redirects to
`graph_neo4j`, and db_clickhouse/IInstance.py said its dialect string is
"surfaced via QuestionType.DIALECT" when only the `dialect` tool reaches
it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(ai): apply the same execute error contract to the session path
A raw statement run with a session_id went through
TransactionRegistry.execute, which has no try/except, and the existing
`except Exception` arm in execute() re-raised the SQLAlchemy exception
unchanged. str() of that exception ends in `[SQL: ...]` and
`[parameters: ...]`, so the same tool behind the same allow_execute gate
returned the statement and its bound values to the caller purely because
a session id was passed, while the sessionless half stripped them.
Add a `SQLAlchemyError` arm above the existing `except Exception` (the
max-rows RuntimeError is not a SQLAlchemyError, so its wording and its
rollback are untouched): log the full exception, release the session, and
raise `RuntimeError('SQL execution failed: ' + _format_db_error(e))`.
The fix lives in execute() rather than tx_registry because the registry
has no IGlobal and is shared with other callers.
Session release moves into _releaseFailedSession, shared by both arms.
A rollback that itself fails is logged and swallowed: previously an
exception raised inside the cleanup replaced the error the caller was
being told about (a monkeypatched rollback surfaced as 'boom').
Also drop the dead `result is None` branch in execute() and correct
_executeRawQuery's annotation to `-> dict`: the method has returned the
shaped dict or raised since it started carrying the driver's message, so
the branch could only ever have produced the opaque 'check server logs
for details' string this work exists to replace, and the annotation
contradicted the comment above the branch. execute() is its only caller.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* refactor(ai): share one table formatter between get_schema and refresh_schema
refresh_schema rebuilt get_schema's nested _format_table line for line,
with `result` renamed to `entry`. Both tools promise callers the same
per-table shape -- their tool descriptions say so, and an LLM picks
between them on that sentence -- but the equality test compared the two
copies to each other, so it would have stayed green while a later change
added a key (an index list, a column comment, a nullable flag) to one
copy only.
Lift the formatter to a module-level _format_table and call it from
both; get_schema's body loses the nested definition, refresh_schema
loses the loop. The wire format is byte-identical, so the existing shape
tests pass untouched. The docstring paragraph calling the duplication
deliberate goes with it.
test_refresh_schema_matches_get_schema_shape now reflects a table with
both a primary key and a foreign key, so the two optional entries are
actually exercised on both sides, and _format_table gains a direct unit
test for the optional-key rules an equality check cannot see.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* fix(ai): make the lazy schema rebuild safe for concurrent inserts
refresh_schema empties IGlobal.schema to re-arm the rebuild at the top of
_insertData, which is what makes this reachable at runtime: a second
insert can now arrive while the first is reflecting.
_getTableSchema published `self.schema = {}` and then grew it column by
column, and _insertData checked, rebuilt and then took a reference to
that same dict without a lock. A concurrent insert could therefore see
the empty map (falling into the "no schema cached" branch, which binds
the item's raw keys and skips the generated-key handling), or a truthy
but half-built one (silently dropping columns the caller had supplied),
or a dict changing size while the per-row loop iterated it. A reflection
that failed midway left the half-built map published for the next insert
to believe.
_getTableSchema now builds into a local dict and publishes it with one
assignment, so a failure leaves the previous map untouched. _insertData
holds _REFLECT_LOCK across the falsy check, the rebuild and a snapshot
copy, and builds the batch from the snapshot; the lock is released before
the Table reflection and before the INSERT, neither of which reads the
map. The comment above refresh_schema's invalidation no longer claims
the single rebind makes the rebuild atomic.
Tests hold a rebuild open inside the real column walk rather than
simulating the window: without the lock the second insert stored NULL in
a column it had supplied, an overlapping refresh left a one-column map,
and a failed reflection published a partial one.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* fix(ai): decide generated primary keys per row, not per batch
The answers lane worked out which primary-key columns to leave unbound
once per batch, from the union of every row's keys. One row carrying
`id` therefore put `id` into every mapping, and the rows that omitted it
bound NULL into a key the database was supposed to generate. SQLite
tolerates that for a rowid alias; Postgres rejects the whole batch for an
`id SERIAL NOT NULL` column. The sole-primary-key table had the same
problem by construction: the `omit_columns = set()` fallback bound NULL
because omitting everything would have left an empty mapping.
Which primary keys the database generates is now read from the reflected
Table -- `autoincrement_column` (SQLAlchemy's own resolution, covering a
reflected `autoincrement=True` and the `'auto'` rule that recognises
SQLite's rowid alias), plus any primary-key column with a server_default,
an Identity, or an explicit autoincrement -- rather than assumed of every
primary key. Per row: a generated key the row omits is left out of that
row's mapping, a supplied value is bound, and an explicitly supplied None
is bound as NULL, so the three cases stay distinguishable. A primary key
the database does NOT generate and the row does not carry is a ValueError
naming the table, the columns and the row position, raised before the
transaction opens.
Rows are then executed as contiguous runs sharing a key set, all inside
one engine.begin(): contiguous rather than gathered, because generated
ids follow insertion order and regrouping would hand back ids in an order
the caller's rows never had, and one transaction because a batch must
stay all-or-nothing as a single executemany was. A run whose mapping is
empty gets a values-less `insert(table)` per row, which SQLAlchemy
renders as `DEFAULT VALUES` on PostgreSQL and SQLite and `() VALUES ()`
on MySQL.
test_insert_binds_the_primary_key_when_it_is_the_only_column pinned the
old fallback and is replaced by the default-values test, plus a
compile-only check of all three dialect renderings (no live PostgreSQL or
MySQL in this suite).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* test(ai): pin the caller-facing error policy for Postgres and MySQL
CodeRabbit's security finding asks for a generic message or a stable
category instead of the driver's own words. That is not adopted: the
maintainer's review endorses the primary-message contract precisely
because "no such column: foo" is what lets a caller fix their statement.
What was missing was the policy written down and held in place by tests.
_format_db_error's docstring now states the residual it accepts: a
driver's primary sentence may quote a value from the statement being
reported (Postgres `invalid input syntax for type integer: "abc"`, MySQL
`Duplicate entry 'x' for key 'users.email'`). That value was submitted by
the caller now reading the message, who also holds allow_execute; values
belonging to other rows appear only in DETAIL / HINT / CONTEXT, which are
stripped. Blanking quoted tokens generically is rejected because MySQL
quotes identifiers with single quotes too, so the redaction would delete
"which column" from the message that most needs it.
The tests build real DBAPIError wrappers via DBAPIError.instance, so
str(exc) genuinely carries the `[SQL: ...]` / `[parameters: ...]` tail and
the assertions bite: a Postgres duplicate-key whose DETAIL restates the
email, the same without .diag so the LINE echo is exercised, the MySQL
duplicate entry that documents the accepted residual, and the MySQL
unknown-column that documents why redaction is refused. One more drives
both shapes through BOTH execute paths and asserts the two messages are
identical -- against the pre-fix source the session-bound half raised the
raw DBAPIError with the bound 'ada@example.com' attached.
Mock-based by necessity: no live PostgreSQL or MySQL in this suite, so
the driver exception shapes are reproduced rather than provoked. No
behaviour change.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* docs(nodes): correct the lane, cache and error prose in the four SQL READMEs
All four checked against the node code and services.json rather than
against each other.
db_clickhouse declares only a `questions` lane (services.json
`lanes: {questions: [...]}`), so the sentence about schema refresh
reaching the `answers` insert lane described a lane the node does not
have; it is removed. db_mysql, db_postgres and rocketride_sql do declare
`answers`, so theirs stays — reworded, because refresh_schema
invalidates that column map rather than rebuilding it, and the rebuild
happens on the next insert.
"Return the schema reflected when the node started" was wrong in all
four after refresh_schema shipped: get_schema serves whatever snapshot
the node currently holds, which is the start-up reflection until a
refresh replaces it.
db_postgres said every question on the `questions` lane is translated to
SQL and executed. When the LLM reports the question is not a database
query (`isValid: false`), writeQuestions emits its prose answer and
nothing runs against the database, so the lane is described as the
natural-language path with only valid generated SQL executed.
Each README now states the execute error contract the code enforces: a
failure raises `SQL execution failed:` plus the driver's primary
message, identically with and without a session_id, with the statement
and bound parameters kept to the server log — and with the residual
named, since a driver's primary sentence can quote a value the caller
itself submitted.
scripts/validate-node-readme.py passes for all four nodes; no GENERATED
block touched.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* fix(ai): publish the auto-created table's column map in one assignment
_createTableFromData is the second writer of IGlobal.schema and still had
the publish-then-fill shape B4 removed from _getTableSchema: it created
the table, assigned `self.schema = {}`, then added one key per column,
all outside _REFLECT_LOCK. By the time that loop runs the table exists,
so a second first-batch can already be inside _insertData; it takes the
lock, finds a truthy one-column map, snapshots it and silently drops
every other column its row carried. Reproduced by pausing the real fill
loop: the second row landed as {'label': 'b', 'size': None,
'colour': None} for input that supplied all three.
The map is now built in a local dict and published in a single
assignment. The lock is NOT widened over table creation, so the
pre-existing double-create race is untouched and still out of scope.
Also seeds a stale map into a direct _getTableSchema failure test: the
existing rebuild-failure test starts from {}, which cannot tell "previous
map left untouched" from "map reset to {}". Against the pre-B4 source
that new test fails with the stale entry replaced by a partial one.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* fix(ai): cut the clickhouse stack trace and position echo from execute errors
The formatter's docstring claimed "pymysql / clickhouse-driver put
(errno, message) in .orig.args". Only pymysql does. clickhouse-driver's
DBAPI layer raises OperationalError(ServerException), so .orig.args holds
one exception object, every branch falls through to str(orig), and what
reached the caller was the driver's full string form: "Code: N.\n
DB::Exception: <message>. Stack trace:\n\n0. DB::Exception::Exception(...)
@ 0x..." — server internals the caller cannot act on — plus, for a syntax
error, the failing statement fragment quoted after "failed at position N
('...')", which is the same class of statement echo as psycopg2's
"LINE n:".
_DB_ERROR_DETAIL gains "Stack trace:" and "failed at position" as
markers, so both are cut by the stripper that already handles the other
drivers rather than by a new branch. Neither phrase can misfire on the
shapes the other drivers produce.
The docstring now states the real clickhouse-driver shape instead of
claiming the pymysql one. Two constructed tests cover it, labelled in
their docstrings as reproduced from the driver's documented behaviour:
clickhouse-driver is not installed in this engine, so the shape cannot be
provoked here.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* docs(ai): state the real execute error contract and schema-cache wording
Three corrections to text, no behaviour change.
The error-contract paragraph overclaimed. It said values belonging to
other rows appear only in DETAIL / HINT / CONTEXT, which are stripped.
Both halves of that are wrong: a real SQLite run returns
`SQL execution failed: near "'hunter2'": syntax error` — the statement
fragment the parser stopped on is in the primary sentence — and
PostgreSQL's `invalid input syntax for type integer: "<value>"` names a
row value an INSERT ... SELECT or a CAST merely touched, which can come
from another table. pymysql and psycopg2 interpolate binds client-side,
so such a fragment can carry a bound value too. The docstring now lists
what is removed (the SQLAlchemy tail, LINE n:, DETAIL/HINT/CONTEXT/
QUERY/STATEMENT, the ClickHouse stack trace and position echo) and what
is passed through, and rests the policy on the argument that actually
holds: the recipient has allow_execute and can already read the same data
with a SELECT, so the error channel does not widen read access. Two tests
(one per execute path) pin the syntax-error shape on a real SQLite run.
get_schema's docstring said the schema is "reflected once in beginGlobal"
and refresh_schema's tool description — the sentence an LLM reads when
choosing between the two tools — said get_schema serves the snapshot
reflected when the node started. Both became false the moment
refresh_schema shipped, and both contradicted the READMEs corrected for
exactly that in c258211d.
_generated_primary_keys gains a paragraph recording the SQLite-only
residuals of SQLAlchemy's 'auto' rule: INT/BIGINT/INTEGER-DESC primary
keys are treated as generated although SQLite does not alias them to the
rowid, and an INTEGER PRIMARY KEY that is also a foreign key is treated
as required although it is an alias. PostgreSQL and MySQL reflection set
autoincrement explicitly, so production nodes resolve exactly.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* docs(nodes): correct the error-contract and remaining schema wording
Two corrections, both to sentences this PR itself introduced or left
behind.
The error sentence in all four READMEs promised that "the statement text
and the bound parameter values are never part of it". That is not what
the code does: a real run returns `SQL execution failed: near
"'hunter2'": syntax error`, and PostgreSQL's `invalid input syntax for
type integer: "<value>"` names a value an INSERT ... SELECT or a CAST
read from another table. Each README now says what is actually removed
(SQLAlchemy's `[SQL:]`/`[parameters:]` echo, and per engine: PostgreSQL's
LINE n:/DETAIL/HINT/CONTEXT, ClickHouse's server stack trace and its
`failed at position` echo) and that the primary sentence is passed
through as the database wrote it, so it can name a value the statement
carried or touched — resting the policy on the argument that holds, that
reaching this tool requires Allow direct query execution and such a
caller can read the same data with a SELECT. The full text stays in the
server log. db_postgres moves that contract out of the already-long
`execute` table cell into prose below the table, as the other three do.
The intro sentences in db_clickhouse and db_mysql still said
"startup-reflected schema", the wording corrected everywhere else in
c258211d; both now describe the cached snapshot and name refresh_schema.
scripts/validate-node-readme.py passes for all four; no GENERATED block
touched.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* fix(ai): leave every database-owned column to the database on the insert lane
Two ways an explicit NULL still reached a column the database was meant
to fill, both silent because writeAnswers only logs, and one comment that
claimed neither could happen.
_generated_primary_keys skipped primary-key columns only, so any other
newly reflected column was bound NULL — and an explicit NULL overrides a
server default. The sequence the refresh tool invites: answers is
auto-created as (id, q, a) with the curated map {q, a}; an agent runs
ALTER TABLE answers ADD COLUMN created_at ... NOT NULL DEFAULT now(); the
tool description tells it to call refresh_schema, so the next batch
rebuilds the map by reflection, binds created_at = NULL, and the batch
that worked a moment earlier is dropped with one log line. Reproduced on
SQLite: NOT NULL constraint failed: answers.created_at, parameters
('how', 'like this', None). _insertData now also reads
generated_defaults — every column with a server_default or an identity —
off the reflected table and omits those the row does not supply.
An explicit null on a generated key was bound as NULL. On the execute
tool that is right: a person wrote the statement. On the insert lane the
caller is an upstream node, and an LLM node or a JSON mapper emits every
schema key, writing null for the ones it has no value for — the same
intent as omitting it. Postgres rejects that NULL for a SERIAL column;
MySQL and SQLite generate a value, so the failure is dialect-dependent as
well as silent. A supplied null on a generated primary key is now treated
as omitted; everywhere else it is still bound as given.
The comments at refresh_schema's invalidation and above the binding loop
said the invalidation cannot change answers-lane behaviour. This PR's own
test shows it can, and that half is the point of the tool. They now state
the guarantee that actually holds: the columns present only in the
reflected map are omitted rather than bound NULL, so the database still
fills in what it owns.
Also drops the unreachable "no schema cached" branch: after the locked
block IGlobal.schema is populated or the method has raised, so the
snapshot is always truthy. The empty-map half of the lock rationale went
with it; the half-built-map half, which the concurrency test proves,
stays.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* fix(ai): format driver errors the dialect did not wrap in a SQLAlchemy exception
The error contract this PR documents was not delivered on the ClickHouse
node at all. db_clickhouse connects with `clickhouse+native://`, which is
clickhouse-sqlalchemy's own connector; its Cursor.execute raises
`clickhouse_sqlalchemy.exceptions.DatabaseException`, a plain Exception
subclass carrying the driver's ServerException in `.orig`. SQLAlchemy
does not wrap a non-DBAPI exception, so `except SQLAlchemyError` never
fired on EITHER execute path, `_format_db_error` never ran, and the
caller got `Orig exception: Code: 62.\nSyntax error: failed at position
21 ('hunter2') ... Stack trace:\n\n0. DB::Exception::Exception(...)` —
raw, on both paths. The markers added in 27a04f68 were verified against
clickhouse-driver's own DBAPI layer, which this node does not use, so
they were unreachable for the node they were added for.
Both paths now handle a non-SQLAlchemy exception that carries a driver
`.orig`: log it and raise the same `SQL execution failed: <formatted>`
RuntimeError, releasing the session on the session path as before. Duck
-typed on `.orig` rather than importing clickhouse modules into the
shared base — a driver error carrying its original is precisely the shape
being handled, and the base must not depend on one node's driver.
Anything without `.orig` is re-raised untouched, so the max_execute_rows
RuntimeError keeps its wording and rollback and a programming error still
surfaces as itself.
_format_db_error also reads `.code` and `.message` off the original when
they are an int and a string: clickhouse-driver's ServerException puts
the code there rather than in args, so the numeric branch missed it and
the caller lost the code. ClickHouse now reads like pymysql —
`Error 60: Table default.widgets doesn't exist` — and the message still
runs through the stripper, so the stack trace and the `failed at
position` fragment inside it are cut.
The two existing clickhouse formatter tests are relabelled: they cover
clickhouse_driver.dbapi's shape, which this node never produces. New
tests cover the connector's real shape at the formatter and drive it
through both execute paths, asserting no stack trace, no statement
fragment, and a released session; two more assert that an exception
without `.orig` is not swallowed. All labelled as constructed shapes —
clickhouse-sqlalchemy is not installed in this engine.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* fix(ai): reject a malformed params argument instead of misreporting it
`to_sqlalchemy_text` turns `$n` into a bind by indexing `params[n - 1]`,
so a JSON object for `params` raised `KeyError: 0` and a short list an
`IndexError`, both from inside the rewriter. The sessionless path leaked
those bare; worse, on the session path the `KeyError` was caught by the
arm that maps a missing session id, so `execute` reported "unknown or
expired transaction session: <sid>" for a session that was still open and
still usable — the caller was told to abandon a live transaction because
they passed the wrong JSON type.
`execute` now validates before dispatching anything: `params` must be
absent or an array, and every `$n` the statement references must have a
value, with the offending placeholder named. tx_registry.py is untouched
— the check lives in `execute` and imports the placeholder pattern from
the module that owns it so it cannot drift from the rewriter it guards.
An empty list keeps its current meaning of "no binds", the same as
omitting `params`, because a statement carrying a literal `$n` inside a
string body relies on the rewriter leaving it alone.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* docs(nodes): document the insert-lane null rule and the clickhouse error layer
The three nodes that declare an `answers` lane now state, beside the
existing insert prose, what the code does with a value the row does not
carry: a column with a generated key or a DEFAULT is left out of the
statement so the database supplies it, rather than bound as an explicit
NULL that wou…
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
In-process HTTP MCP server for the RocketRide engine
This branch adds a Streamable-HTTP MCP server that runs inside the engine process, mounted at
/mcpon the engine's own web port. Any MCP client — Claude Desktop/Code, Cursor, MCPJam, or a headless agent — can connect straight to the running engine and author, execute, and observe pipelines over one endpoint, with no separate bridge process, no stdio wrapper, and no session state.Protocol
2026-07-28, fully stateless —StreamableHTTPSessionManager(stateless=True), noMcp-Session-Id, one JSON-RPC request in / one response out.2025-11-25and earlier) via the SDK's per-request routing — 2025-era hosts work unchanged, pinned bytest_dual_revision.py.ttl_ms/cache_scope, SEP-2549) on every listable/readable result.The 26-tool surface
list_components,describe_component,validate_pipeline,describe_pipelinerun_pipeline,run_dropper_pipe,send_data,terminate,send_filesstore_read/list/stat/get_url,save_template,load_template,deploy_add/list/status/remove/updatemonitor,list_running_pipelineslog_chapters,log_read,log_traces,log_tracePlus 2 resources (
rocketride://status,rocketride://pipelines).Design decisions worth knowing when reviewing:
validate_pipelineruns the engine's own validation; component metadata comes from liveget_services. No client-side rule copies.use()returns the task token as the single run identity; a server-ownedTaskRegistrytracks live tokens;run_dropper_pipereturns a self-contained upload URL for out-of-band file ingress (bytes can't ride a tool call).(projectId, source, runKind), addressable across restarts, gated bypipelineTraceLevel(run tools default'summary'). This retired the live-onlyget_pipeline_traceand its in-memory event-buffer machinery.{ok: false, error_type, message, hint}(PascalCase vocabulary:BadRequest,Timeout,NotFound,TraceExpired) so agents can fix their call; hard failures (connection/auth) surface as real MCP tool errors. Every blocking seam call isasyncio.wait_for-bounded.EngineClientprotocol +WsEngineClient): tool code depends only on the protocol; the v0 impl wraps the WS SDK with lazy singleton connect and reconnect-on-transport-drop. A later revision can swap in direct in-process calls without touching tool code.MCP Apps (embedded UI)
Two single-file HTML widgets served as
ui://resources for hosts that support theio.modelcontextprotocol/uiextension (hosts without it just see the JSON results):list_running_pipelineswith refresh/terminate actionsrun_dropper_pipe's upload URL, with typed result tabs (text/tables/images/audio/video/…)The vite workspace lives inside the module (
packages/ai/src/ai/modules/mcp/apps/), builds straight into the serveddist/, andai:syncships only the built bundles — toolchain, sources, and node_modules never reach the server dist. Builder module:mcp-widgets:build.Auth
/mcpsits behind the engine'sAuthMiddlewarelike every other route. The only override is theMCP_DEV_NO_AUTH=1/mcp_dev_no_authdev bypass, which is enforced loopback-only (ignored with a warning on any non-loopback bind). No OAuth/MCP auth negotiation yet — tracked as follow-up. Known, documented limitation:filepatharguments are not sandboxed (deferred to the dropper-ingress seam, which will own path resolution for all callers).Testing
packages/ai/tests/ai/modules/mcp/: 229 passing (1 pre-existing env-wall failure intest_module_registration—rocketlibimport outside the engine env, same asdevelopbaseline). Suite covers dispatch, dual-revision, cache policy, every tool group, the seam, widgets' served resources, and the error contracts.:5565over a real MCP connection: all 4 DVR tools (cursor paging, chapter addressing,NotFound/TraceExpiredpaths) and the full demo flow; widgets render in MCPJam.npx tsc --noEmitclean (widget workspace),ruffclean,./builder mcp-widgets:build && ./builder ai:buildclean.Docs
packages/ai/src/ai/modules/mcp/doc.mdis the module's full reference (protocol, boot, all 26 tools, security, history).packages/client-mcp/docs/index.mdpoints stdio/SSE users at this surface as the maintained HTTP path.Summary by CodeRabbit
New Features
Bug Fixes
Documentation
Note for node authors: the MCP integrations surface ships a committed credential catalog at
packages/ai/src/ai/modules/mcp/credentials.json. Adding or changing a node withsecurecredential fields requires regenerating it — run./builder nodes:credentials-generate(also chained intonodes:build) and commit the result;nodes:credentials-checkenforces this in CI.