Repository navigation
fix(server): dispatch pipeline lifecycle in topological order, including control sub-pipelines - #1669
Conversation
🤖 Internal: Discord sync markerAuto-managed by the Discord notification workflow. Stores the linked Discord message ID and forum thread ID. Do not edit or delete. |
📝 WalkthroughWalkthroughThe engine now computes dependency-ordered lifecycle regions, binds ChangesPipeline lifecycle ordering
Estimated code review effort: 4 (Complex) | ~45 minutes Suggested reviewers: Sequence Diagram(s)sequenceDiagram
participant PipelineInput
participant IServiceEndpoint
participant LifecycleRegions
participant ToolPipe
participant ResponseLane
PipelineInput->>IServiceEndpoint: buildConnections
IServiceEndpoint->>LifecycleRegions: compute dependency-ordered regions
LifecycleRegions-->>IServiceEndpoint: return head and control orders
IServiceEndpoint->>ToolPipe: dispatch open, closing, and close
ToolPipe-->>ResponseLane: write flushed response
IServiceEndpoint->>ResponseLane: read response after flush
🚥 Pre-merge checks | ✅ 3 | ❌ 2❌ Failed checks (2 warnings)
✅ Passed checks (3 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
joshuadarron
left a comment
There was a problem hiding this comment.
Overview
Solid fix for a real bug. Lifecycle (open/closing/close) was dispatched in .pipe declaration order via per-parent binder chains, so a join node could flush before an upstream branch emitted its flush-time data. This binds the source-reachable nodes' lifecycle flat onto the pipe head in topological order — closing/close upstream-first, open reversed — leaving control/invoke sub-pipelines on their per-parent bindings.
I verified the two claims the whole design rests on, and both hold:
opendownstream-first matches existing semantics.IPythonInstanceBase::opencallsParent::open(→binder.open→ children) before invoking the Python node's ownopen(), so children already opened first under the old chain. ReversinglifecycleOrderpreserves that.closing/closeupstream-first matches existing semantics.IPythonInstanceBase::closing/closerun the Python method first, thenParent::→ binder → children. Parent-before-child, same as the new flat order.
Also checked:
tool_pipeis the only production consumer ofself.instance.open()/close()(nodes/src/nodes/tool_pipe/IInstance.py:98,106). It is wired to agents throughcontrol/invoke, never as a datainputtarget, so neither it nor its sub-pipeline nodes are source-reachable — the exclusion works, and its per-parent bindings survive.packages/ai/src/ai/modules/data/data_conn.py:504is the pipe head, unaffected.- The
Ec::PreventDefault-no-longer-prunes note is accurate: everypreventDefault()call innodes/is in a data lane (writeText,writeImage,writeDocuments,renderObject), none inopen/closing/close. pipeStack[0]is always the pipe filter (buildPipeStackpushes it first) andinstanceStackis 1:1 withpipeStack, solifecycleOrderindices can't collide with the head.bindFiltershas exactly one caller, so the rewritten doc comment ("called when we create the initial filter stack") is correct.- No double-binding: the skip is keyed on
toIndex, so a head-managed target is bound exactly once, on the head. computeLifecycleOrderis deterministic and allocation-safe —unordered_mapelement references survive rehash, sofor (int to : adj[node])is fine even thoughadj[node]may insert.
Docs are updated per the co-located rule, CI is green on all three platforms, and the reasoning for skipping the E2E (a naive diamond with a response join is a false-green) is sound.
Two things I'd like addressed before merge — both small.
Requested changes
1. Binder::getBound() is dead code
packages/server/engine-lib/engLib/store/core/binder.cpp:59 + headers/binder.hpp:178
The accessor is added to the public Binder API, documented as "Used by tests to assert lifecycle lane contents/order" — and has zero callers. The only new test (lifecycle_order.cpp) exercises the pure computeLifecycleOrder function and never touches a Binder.
This is public surface added for a consumer that doesn't exist, which .claude/rules/architecture.md explicitly rules out ("Every file and folder exists because something depends on it right now"). Either use it (see #2) or drop it.
2. Nothing tests the actual bind order
All 10 new tests cover the ordering algorithm. Nothing covers the part of bindFilters that consumes it, which is where a silent regression would live:
closing/closebound onfilters[0]in forward order,openin reverse- head-managed targets skip the per-parent lifecycle bind
- non-reachable (control/invoke) targets keep it
That last one is the tool_pipe contract. If someone later drops the headManaged guard, computeLifecycleOrder still passes all 10 tests and tool_pipe breaks. getBound() is exactly the hook needed to assert this — please add the test it was written for, so the accessor earns its place.
Non-blocking nits
- Bounds-check asymmetry (
endpoint.pipes.cpp:145-159): theclosing/closeloop validatestoIndexagainstfilters.size(); the reversedopenloop doesn't. It's safe only because loop 1 validates every entry before loop 2 runs — an implicit coupling that breaks the moment someone reorders or early-exits. Validate the wholelifecycleOrderonce up front (or inbuildConnections, againstpipeStack.size()) and drop the in-loop check. toIndex >= filters.size()comparesinttosize_t. Consistent with the pre-existing checks in this file and builds are clean, so noting only for consistency if you touch it per the point above.headManagedallocation: the comment atstack.cppsays the order is precomputed "so bindFilters stays allocation-free per pipe", butbindFiltersnow builds anunordered_setper pipe stack.lifecycleOrderis small — a linearstd::findwould beat the hash set here anyway. Either way, the comment overstates it.- Kahn loop is O(N²) (
stack.cpp): the inner scan restarts from the head ofnodeOrderafter each emit. Irrelevant at pipeline sizes and it runs once per endpoint, so fine — just flagging it's quadratic by construction, not by accident. A rolling start index would make it linear-ish if you ever care. if (inDegree[to] > 0) inDegree[to]--;: with deduped edges and seeded in-degrees this guard can't fire. It converts a would-be logic error into a silently wrong order. An assert (or nothing) says more.edgesduplicates the information already inadj; the in-degree pass could walkadjdirectly and drop the extra vector.
Question
The mixed reachability node included after reachable parent test locks in a behavior change worth confirming: a node fed by both a source-reachable parent and a control/invoke node becomes head-managed, so its invoke node's self.instance.open()/close() no longer reaches it.
Concretely — a response node shared between the main pipeline and a tool_pipe sub-branch would stop being opened/closed per tool invocation. The tool_pipe README says to cap each branch with its own response node, so this is probably out of contract, but it's a wiring a user can produce on the canvas. Is that the intended trade-off? If so, worth a line in ROCKETRIDE_PIPELINE_RULES.md next to the new lifecycle guarantee.
Also: the ./builder test checkbox is still unchecked in the description.
Pipeline open/closing/close were dispatched in `.pipe` canvas order, not dependency order, so a join node could be flushed before an upstream branch emitted its flush-time data — silently dropping it. Bind the source-reachable nodes' lifecycle onto the pipe head in topological order instead: closing/close upstream-first (a node flushes only after all of its upstream parents), open reversed. Control/invoke sub-pipelines are left untouched — they keep their per-parent binding so their invoke node (e.g. `tool_pipe`) can still open/close them. The order is computed once by `computeLifecycleOrder` (reachability from the source, then a Kahn topological sort over the reachable set; a cycle in the reachable region is rejected). Adds unit tests for the ordering and documents the guarantee. Closes #1650
Generalize the open/closing/close dependency-order guarantee to a control node's inline sub-pipeline (e.g. tool_pipe): it is flushed and closed in dependency order per invocation, to any nesting depth, so a diamond sub-pipeline returns the merged output of all branches. tool_pipe now drives closing() then close(). Reject at pipeline open the wirings that break exclusive sub-pipeline ownership: sharing a sub-pipeline node with the main flow, a second start, or another control node; and a data-fed invoke node. Adds unit tests and updates pipeline-rules/observability docs and examples.
Report each node as its service title plus component id, e.g. "Pipeline Tool" (pipe_tool_1), instead of a pipeStack index, so a rejected wiring points at what the author sees on the canvas. Document the exclusive-ownership rule in the tool_pipe README and add a common-mistakes entry covering the three rejections. Examples switch to agent_crewai and lane-specific response nodes.
411ec05 to
8eb14ff
Compare
Remove `Binder::getBound`: public API added with no caller. Range-check the lifecycle indices once up front rather than inside one of the bind loops, so every walk is covered whatever order they run in, and correct the comment that claimed bindFilters was allocation-free per pipe. Add an end-to-end test for the closing order: a diamond of buffer-and-flush nodes whose join is declared before its branches, so dispatching in declaration order would flush the join early and drop a branch's output.
Thanks for the thorough review. The two semantics checks you did ( Please note the PR grew after your review. You reviewed
This invalidates one of your conclusions. You wrote that Requested changes1.
|
There was a problem hiding this comment.
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
packages/server/engine-lib/engLib/store/stack.cpp (1)
332-345: 🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick winReject every data-fed control root with a sub-pipeline.
Lines 337-345 use
ownerOfto infer whether the control root is data-fed, butownerOfonly records nodes already claimed by lifecycle regions. A sub-pipeline root fed by an unowned node is accepted, then receives a per-parent lifecycle binding inbindFilterswhile its descendants are also flat-bound from the invocation root, allowing duplicate lifecycle execution and dropped/duplicated flush output.Track incoming data targets directly and reject
region.firstwhenever it appears as a connection target.The PR objective explicitly requires rejecting control roots that own a region and are also data-fed.
Proposed fix
+ std::unordered_set<int> dataFed; + for (const auto &conn : connections) + dataFed.insert(std::get<1>(conn)); + for (const auto ®ion : regions.control) { - auto it = ownerOf.find(region.first); - if (it != ownerOf.end()) + if (dataFed.count(region.first)) return APERR(Ec::InvalidParam, "Control node", nameOf(region.first), - "drives a sub-pipeline and is also data-fed (owned by", - nameOf(it->second), - "); an invoked node with a sub-pipeline must not take a " - "data input"); + "drives a sub-pipeline and is also data-fed; an " + "invoked node with a sub-pipeline must not take a " + "data input"); }🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/server/engine-lib/engLib/store/stack.cpp` around lines 332 - 345, Update the control-root validation in the region-checking loop to use the incoming data-connection targets rather than ownerOf, since unowned data-fed nodes are currently missed. Track or query those targets directly and reject every region.first that is a data connection target, while preserving the existing InvalidParam error behavior and message context.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Outside diff comments:
In `@packages/server/engine-lib/engLib/store/stack.cpp`:
- Around line 332-345: Update the control-root validation in the region-checking
loop to use the incoming data-connection targets rather than ownerOf, since
unowned data-fed nodes are currently missed. Track or query those targets
directly and reject every region.first that is a data connection target, while
preserving the existing InvalidParam error behavior and message context.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 34b3ff4e-5fa8-4ea3-bf6d-752a3be6d2e4
📒 Files selected for processing (3)
nodes/test/test_lifecycle_order.pypackages/server/engine-lib/engLib/store/core/endpoint/endpoint.pipes.cpppackages/server/engine-lib/engLib/store/stack.cpp
There was a problem hiding this comment.
solid work — the flat-per-region binding is the right model, and I verified it end to end: Binder::callMethods dispatches methodMap[lane] in bind order, and IServiceFilterInstance::open/closing/close are pre-order, so with region-owned nodes no longer bound per-parent there's no double dispatch. The controls.clear() in buildConnections and the try/finally in tool_pipe are nice catches too.
* fix(server): bind Python instance.closing() to cb_closing (#1667)
`IServiceFilterInstance`'s `closing` method was bound to `cb_close` — the same
callback as `close` — so `self.instance.closing()` walked the `close` lane
(teardown) instead of the `closing`/flush lane. The correct callback,
`cb_closing`, was implemented but bound to nothing, i.e. dead code.
Bind `closing` to `cb_closing` so a node can drive flush on its downstream.
No behavior change today (no node calls `instance.closing()`), but it removes a
correctness landmine and unblocks control-node sub-pipeline flush (e.g.
`tool_pipe`).
Closes #1666
* feat(nodes): tool_filesystem pipeline-sink lanes (#1651)
* feat(nodes): tool_filesystem sink lanes + filename handling
Split the tool_filesystem node changes out of the feat/http-mcp branch so
they can land independently of the HTTP MCP module. Adds pipeline-sink
behavior (write pipeline output to the scoped FileStore) and filename
handling, with unit tests that stub the engine/ai layer.
No changes to the FileStore/account package — the node consumes the
existing FileStore API only.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(nodes): address CodeRabbit review on tool_filesystem sink
- IGlobal._sink_config: clamp urlExpiresIn lower bound too — non-positive
values now fall back to the 3600s default instead of passing through.
- services.json: add minimum/maximum (1..3600) bounds to filesystem.urlExpiresIn.
- test_sink_naming: cover negative/zero urlExpiresIn -> default.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* refactor(nodes)!: rename tool_filesystem node to filesystem
The node is no longer just an agent tool — it now also acts as a pipeline
sink (input/output lanes). Drop the tool_ prefix and rename the node to
`filesystem`, which aligns with its existing title ("File System"),
file-system.svg icon, and filesystem.* config namespace (no config-key churn).
BREAKING CHANGE: the node protocol/identity changes from `tool_filesystem://`
to `filesystem://` and the module path from `nodes.tool_filesystem` to
`nodes.filesystem`. Pipelines referencing the old identity must update.
- git mv nodes/src/nodes/tool_filesystem -> nodes/src/nodes/filesystem
- git mv nodes/test/tool_filesystem -> nodes/test/filesystem
- services.json: protocol + path updated; description now covers both the
agent-tool and pipeline-sink roles.
- Internal refs, log prefixes, README, and test module paths updated.
Generated docs under packages/docs/static (gitignored) regenerate from the
node source. A few unrelated files carry stale prose mentions of the old
name (rocketride_db.py, test_tool_tavily.py, test_tool_args.py) — left
untouched here to keep this change node-scoped.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(nodes): address review feedback on tool_filesystem sink; revert rename
Reverts the tool_filesystem -> filesystem rename. Renaming the node changes the
provider identity stored in saved .pipe files, which needs the engine-side
migration hop the graph rename got. That is out of scope here, so the rename
will land as its own PR together with the hop. This PR is purely additive on
top of develop again, contained to nodes/.
Review fixes applied:
- Per-lane filename resolution. Each lane owns its rule instead of one
precedence chain where the source extension always won: text/table are
markdown so they store .md; documents keep the source extension (else .txt);
media derive from mime, then source extension, then .bin. Mime->extension now
handles compound subtypes (image/svg+xml -> .svg, not .svg+xml) via an
override map plus a suffix-stripping fallback.
- Streamed media. image/audio/video chunks stream straight to the store via
open_write/write_chunk/close_write, so memory is bounded regardless of file
size. The handle opens lazily on the first non-empty chunk, so an empty
stream writes no file, and a fresh BEGIN discards a half-written prior
stream instead of leaking it into the next object's file.
- Deterministic, collision-free store keys (targetDir + <objectId>/ + name,
with an index for multi-document objects). Removes the stat-probe loop, which
also closes the non-atomic write race and the unbounded-loop concern.
- Emission correctness: DocMetadata(self, ...) so objectId/nodeId/permissionId/
signature are inherited, a distinct chunkId per reference so vector stores
keyed on objectId+chunkId no longer overwrite each other, and
preventDefault() on every lane so downstream receives the emitted references
rather than the originals as well.
- Path whitelist is validated before anything touches the store, so a rejected
path no longer probes for existence.
- Config robustness: non-numeric urlExpiresIn falls back to the default instead
of raising out of beginGlobal; endGlobal resets all three sink fields.
- classType reordered to ["store", "tool"] to match the codebase convention.
Order is not behavioural: the engine keys a controller map by the classType
string and all UI reads are membership tests, so nothing depends on position.
- Tests rewritten and de-rigged: named objects now exercise the real lane
semantics (report.pdf through the text lane must store .md), plus streaming
media, empty-stream, abort-on-new-BEGIN, chunkId, preventDefault, and the
previously missing writeVideo case. 36 pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(nodes): capped collision probe, per-lane extensions, per-object reset for tool_filesystem sink
Review feedback from dsapandora and asclearuc on PR #1651:
- collision handling: keep clean names (output/report.txt), append _1, _2, ...
only on real collision, bounded by MAX_COLLISION_SUFFIX (100) then raise;
every candidate is whitelist-checked before any stat probe
- documents lane always stores .txt: page_content is parsed text, so the
stored extension stays truthful about the bytes (same rule as text/table .md)
- _ext_from_mime rejects unmapped vnd.* subtypes instead of emitting junk
- open() per-object hook: aborts media streams the prior object left
unfinished (closes stale handle, deletes partial file) and restarts chunkIds
- _run_async docstring names the sink lane handlers as a supported caller
- tests: 47 passing (blank/unmappable mime fallbacks, conflicting mime,
emitUrl on emitted Doc metadata, open() reset, whitelist-before-probe)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* test(nodes): make tool_filesystem sink tests environment-proof
Under ./builder nodes:test the suite runs in the engine python with
pytest-xdist, where two things differ from the bare venv: other node
suites install bare rocketlib stubs (some with an empty IInstanceBase),
and the real pydantic ai.common.schema is importable. Whichever test
file ran first in a worker decided which base class our node inherited,
producing 12 AttributeError failures in CI (no preventDefault on the
foreign stub; no IEndpoint for the real DocMetadata).
- judge rocketlib.IInstanceBase by capability (has preventDefault), not
presence, so foreign empty stubs are replaced while the real class and
our own stub are kept
- wire concrete currentObject fields and IEndpoint.endpoint.jobConfig so
the real pydantic DocMetadata constructs cleanly under the builder
- pin the PREVENT_DEFAULT sentinel at instance level so tests behave the
same whether the stub base (returns) or the real base (raises) won
Verified: bare venv (47 passed), poisoned ordering with tool_mem0 first
(59 passed), engine python with real ai + client-python on the path
(47 passed).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* test(nodes): keep tool_filesystem importable under minimal rocketlib stubs
The Entry import added for the open() hook broke test_read_size_cap.py
at collection: its minimal rocketlib stub predates Entry. Entry is
annotation-only, so import it under TYPE_CHECKING (PEP 563 lazy
annotations make the runtime import unnecessary).
Also complete that stub with rocketlib.warning: importing the real
ai.common.utils fresh under the stub always needed it transitively
(tool_args), and only ever passed via module-cache ordering luck in the
xdist worker.
Verified: read_size_cap alone in a fresh engine python (19 passed), the
whole test/tool_filesystem dir under engine python with the real ai +
client-python on the path (66 passed), bare venv with poisoned stub
ordering (59 passed).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* fix(llm_anthropic): disable extended thinking at the node as an interim stop-gap (#1681)
Enabling extended thinking makes Anthropic reasoning models return content as typed
blocks (thinking + text) that the agent / expectJson path cannot consume, which caused
the production incident (#1658). Gate thinking activation behind a node-level constant
instead of flipping capabilities.reasoning, since that flag is stamped by
tools/sync_models and re-stamps to true on the next sync. Flip the constant back to
re-enable once thinking is normalized behind the adapter (#1679). The flatten in #1659
stays as defense-in-depth for that point.
Fixes #1680
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(endpoints): unique stable per-pipe endpoint URLs ({host}/{type}/{project_id}/{source}) (#1557)
* fix(endpoints): unique stable per-pipe endpoint URLs
Advertised chat/dropper/webhook URLs now embed the task identity as
{host}/{type}/{project_id}/{source}; additive path routes accept data
POSTs and resolve the task from the path. Legacy routes and ?auth=
resolution unchanged.
Fixes rocketride-ai/rocketride-saas#262
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(task-http): only treat missing tasks as token-resolution fallback
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(task): guard identity placeholders for partially-initialized tasks
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Krish Garg <krishgarg@Krishs-MacBook-Pro.local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* feat(ai): every subprocess exposes /task/data via a shared web server (#912)
* test(ai): characterize lazy-target contract for modules/data
Add `test_lazy_target.py` (15 tests) pinning the contract that
the shared-subprocess-web-server work needs from `modules/data`:
- `initModule` tolerates `state.target` missing or `None` at load time.
- `DataServer._target` reads `state.target` lazily on each access
(picks up writes made by source nodes after the module is loaded).
- `DataConn.__init__` accepts `target=None` without dereferencing;
thread count falls back to the existing default (4).
- `DataConn._require_target()` returns the target when set, or raises
a controlled `RuntimeError` (not the bare `AttributeError` from
`None.putPipe()`) when no source has registered one.
These tests fail against the current production code. The next commit
makes them pass.
* refactor(ai): tolerate `state.target=None` in modules/data
Make `modules/data` safe to eager-load before any source node has
registered a target. Needed so the shared subprocess web server (in a
follow-up commit to `node.py`) can expose `/task/data` from the moment
the subprocess starts, regardless of pipeline shape.
Changes:
- `__init__.py`: `initModule` no longer reads `state.target` eagerly.
`DataServer` now receives the `server` reference and reads the target
lazily via property.
- `data_server.py`: `DataServer.__init__(server, **kwargs)` stores the
server reference; new `_target` `@property` returns
`getattr(server.app.state, 'target', None)` on each access, so source
nodes that write `state.target` later are seen automatically.
- `data_conn.py`: `__init__` guards `target.taskConfig.get(...)` for
`target=None` (defaults to 4 threads); new `_require_target()` helper
raises a controlled `RuntimeError` instead of the bare
`AttributeError` from `None.putPipe()`. The 3 deref sites
(`_cleanup_pipe`, `open_sync` allocate, `open_sync` error path) now
route through it.
Behavior for source-bearing pipelines (webhook, telegram) is unchanged:
they still set `state.target` before any data flows, so the lazy read
returns the same target and the `_require_target` guard passes.
Test coverage: 42/42 in `tests/ai/modules/data/` (27 pre-existing + 15
new from `test_lazy_target.py`); 370/370+2 skipped across
`tests/ai/modules/`. No regressions.
* feat(ai): shared web server in node.py for every subprocess
Bootstrap a FastAPI/Uvicorn WebServer in `node.py` whenever EaaS spawns
the subprocess with `--data_port=N` (which it always does, see
`task_engine.py:1558-1565`). The server runs on the existing
`server_loop` daemon thread alongside `processArguments`, so EaaS can
reach every subprocess on `ws://localhost:N/task/data` for DAP traffic
regardless of pipeline shape — including agentic / sourceless
pipelines that previously had no listener at all.
Changes:
- `_setup_shared_web_server()`: parses `--data_host`/`--data_port` from
`sys.argv`; returns `(None, None)` if absent (legacy / test
invocations). Otherwise constructs the WebServer, calls
`.use('data')` to register `/task/data` (which carries
`rrext_cprofile_*` DAP commands as of PR #803), schedules `serve()`
on `server_loop` via `asyncio.run_coroutine_threadsafe`, and blocks
on a `threading.Event` set by an `on_startup` lifespan callback so
the listener is guaranteed up before `processArguments` runs. A 10s
timeout (`_SHARED_SERVER_STARTUP_TIMEOUT_SECONDS`) safety-nets a
stuck startup.
- `_teardown_shared_web_server()`: called from `run()`'s `finally`
block. No-op when no server was constructed. Otherwise calls
`server.stop()` (flips uvicorn's `should_exit` flag) and awaits the
`serve()` future with a 5s budget. Exception-safe — failures here
log via `debug` and never mask the original `processArguments`
error.
- `run()`: wires the helpers in between `_start_event_loop()` and the
`rocketlib` import; tears down in `finally` before
`_stop_event_loop()`. Exposes `shared_web_server` as a module-level
attribute (via `global`) so source nodes can discover it later
through `from ai.node import shared_web_server`.
Tests (`tests/ai/test_node.py`, 14 cases): argv parsing, WebServer
construction, `.use('data')` call, `serve()` scheduling on
`server_loop`, the startup-event handshake (both the happy path and
the timeout-without-signal path), and the teardown contract including
exception swallowing.
Intermediate state: Phases 3 and 4 (webhook and telegram source-node
refactors) still pending; until then, webhook/telegram pipelines
launched via EaaS will fail with `EADDRINUSE` because the source
nodes' own `_run()` still tries to bind the same `data_port`. Agentic
pipelines work correctly and gain DAP reachability today.
Test coverage: 14/14 in `tests/ai/test_node.py`; 941/941+2 across
`tests/ai/`. No regressions.
* fix(ai): mirror `shared_web_server` from `__main__` to `ai.node`
`engine.exe` spawns `node.py` as a script, so Python loads the file
as `sys.modules['__main__']`. The `global shared_web_server` write
in `run()` therefore lands on `__main__.shared_web_server`. Source
nodes later do `from ai import node as ai_node` and Python imports
the same file AGAIN as `sys.modules['ai.node']` — a separate module
with `shared_web_server = None`. Source nodes saw `None`, fell back
to the legacy self-hosted path, and the data plane was torn between
two competing WebServers (target set on one `app.state`, read from
another).
Mirror the assignment (and the `None` reset in `finally`) into
`sys.modules['ai.node']` so the standard import path returns the
live value. Runs once on the main thread before `processArguments`
spawns C-engine threads; GIL guarantees atomicity, no lock needed.
* feat(nodes): webhook + telegram use shared subprocess web server
Both source nodes drop their own `WebServer` + `.run()` and instead
discover `ai.node.shared_web_server` (bootstrapped by `node.py` from
the `--data_port` flag EaaS passes to every subprocess). They set
`state.target = self.target`, register their routes/modules on the
shared server, and block on a `threading.Event` to preserve the C
engine's `scanObjects()` contract (lifetime == pipeline lifetime).
webhook:
- shared-server path uses sync lifespan helpers
(`_emit_ready_status_sync` / `_emit_shutdown_status_sync`) so we
don't spin up a one-shot `asyncio.run` loop that closes before
the C engine's threads can use it.
telegram:
- `_startup` / `_shutdown` do real async work (aiohttp session,
polling task), so they're scheduled on `server_loop` via
`asyncio.run_coroutine_threadsafe(...).result()` instead of the
webhook's sync-helper trick.
- webhook-mode adds `/telegram/webhook` POST on the shared server.
- polling-mode unchanged behaviourally.
Legacy `_run_legacy_self_hosted_server()` retained for direct CLI
invocations without `--data_port`; emits a `debug(...)` marker on
entry so production logs make it obvious if it's ever exercised
(EaaS always passes `--data_port`, so it shouldn't be).
Tests:
- nodes/test/webhook/test_endpoint.py — 9 unit tests, both paths
- nodes/test/telegram/test_endpoint.py — 12 unit tests, both modes
and both paths
* fix(ai): alias `ai.node` to `__main__` instead of re-importing
`import ai.node` from inside the `__main__`-loaded `node.py` triggered
a fresh module evaluation — creating a second `server_loop` daemon
thread. Source nodes then saw `ai.node.server_loop` (loop B) while
`WebServer.serve()` was scheduled on `__main__.server_loop` (loop A),
breaking cross-loop scheduling.
Replace the `import` with `sys.modules.setdefault('ai.node',
sys.modules[__name__])` — pure dict aliasing, no second evaluation,
both names point at the same module object.
* fix: correct installation of dependencies
* fix(ai): fail fast on shared `serve()` death; fix test_node race
Production: if `uvicorn.serve()` raises during bind (port collision,
permission denied), `startup_ready` never fires and the 10s timeout
elapses silently, publishing a dead `shared_web_server`. Check
`future.done()` after the wait and call `future.result()` to re-raise.
Test: `test_setup_blocks_until_on_startup_fires` assumed the fake
WebServer had captured `on_startup` after a timed `join()` — racy on
slow CI. Add a `threading.Event` set by the fake server so the test
waits on a signal rather than a guess.
* fix(nodes,ai): propagate telegram `_startup` failures; cover serve()-died branch
- `telegram/IEndpoint.py`: re-raise after the `_startup` try/except so
startup failures (network, TLS, aiohttp) surface to the engine instead
of leaving the thread stuck on `_shutdown_event.wait()`.
- `tests/ai/test_node.py`: add regression test for the fail-fast branch
in `_setup_shared_web_server` — pre-failed future whose `.result()`
raises must propagate.
* fix(telegram): raise on `_startup` soft-fail paths (missing token, webhook fail)
`_startup` previously returned normally for fatal startup states like
a missing bot token or a failed webhook registration, leaving the
source thread stuck on `_shutdown_event.wait()` with a non-functional
bot. The `raise` in commit `1463bd2b` only fires when `_startup` itself
raises — it can't help the silent-return paths.
Replace `return` with `raise RuntimeError(...)` after `monitorStatus`
for both fatal paths so `_run`'s existing try/except re-raises out of
the source-node thread. Add two unit tests that exercise `_startup`
directly.
* fix(ai): make `DataConn._target` lazy to close race
* test(telegram): stub `aiohttp` for webhook-setup-fail test
`_startup` does `import aiohttp` + `aiohttp.ClientSession()` before reaching
the webhook check the test wants to exercise. In production this works
because `IGlobal.beginGlobal()` ran `depends(requirements.txt)` first.
Unit tests bypass that lifecycle, and the broader test framework
intentionally mocks `depends` to a `MagicMock` in `test_contracts.py`'s
`mock_engine_libs` fixture — so we can't even rely on `beginGlobal()`
as a test setup. The test doesn't actually need real aiohttp (the
session is never used because `_setup_webhook` is stubbed), so stub
`sys.modules['aiohttp']` via `monkeypatch.setitem` for the test's
duration. Same pattern `test_contracts.py:579-580` uses for engine deps.
* test(ai): disconnect `DataConn` in async lazy_target tests
CodeRabbit flagged that 5 async tests construct `DataConn` (which spawns
`_monitor_task` via `asyncio.create_task`) but never call `disconnect()`,
leaking pending tasks at teardown. Wrap each `DataConn` use in
`try/finally` and `await conn.disconnect()` so the monitor task is
cancelled cleanly. Production code unchanged.
* refactor(ai,nodes): post-PR cleanups around shared subprocess web server
- `node.py`: extract `_parse_data_host_port` helper — single source of
truth for `--data_host`/`--data_port` parsing across the shared-server
bootstrap and the webhook/telegram legacy fallback paths.
- `node.py`: magic numbers.
- `webhook/IEndpoint.py`: plain sync `_startup` / `_shutdown`.
- Sweep over inline comments added during the PR — keep only the
load-bearing "WHY" (race windows, ordering constraints, gotchas);
drop obvious narration of what the code does.
No production behaviour change. Tests: 980 ai + 31 webhook/telegram
all pass locally.
* refactor(ai,nodes): drop legacy fallback + defer DataConn semaphore init
- node.py: `_setup_shared_web_server` raises on missing `--data_port`
(EaaS supplies it unconditionally; the legacy "no shared server"
path is dead).
- webhook/telegram: delete `_run_legacy_self_hosted_server` and the
`if shared is None` branch in `_run()`.
- data_conn.py: defer semaphore + threadCount init to first `_open`
via `_ensure_pipe_sem` — folds the race-window wait with the
threadCount read so the user's config is no longer silently ignored.
- Tests updated; legacy-path tests deleted.
* fix(ai): lazy-init pipe semaphore in DataConn tool handler
The standalone client.tool() path opens no pipe, so `_open` never runs
`_ensure_pipe_sem`, leaving `_pipe_sem` None. Route the borrowed-pipe
acquire through `_ensure_pipe_sem`, matching `_open`.
* fix(ai,nodes): make shared web server optional; guard source-node access
_setup_shared_web_server() returns (None, None) instead of raising when
--data_port is absent, so a subprocess with no data plane (e.g. a
catalog-mode scan) runs rather than failing at startup. This realigns the
code with the contract its own test docstring already documented.
webhook/telegram resolve the server through require_shared_web_server(),
which raises a named error citing the node and --data_port — replacing a
bare 'NoneType' AttributeError surfacing out of scanObjects.
Correct the webhook/telegram READMEs, which still described each node
creating its own FastAPI server on a default port 5567; both untrue since
the server moved into node.py.
* fix(ai): bind shared /task/data channel to 127.0.0.1 not localhost
The name `localhost` resolves to both 127.0.0.1 (IPv4) and ::1 (IPv6),
so asyncio binds one socket per address family and a client can dial
the wrong one, producing intermittent ECONNREFUSED. Bind and dial the
shared subprocess web server's data channel on 127.0.0.1 explicitly,
matching the engine control-port convention.
* test(ai): cover standalone tool call's lazy semaphore init
`_tool` borrows a pipe instead of opening one, so `_open` — the only
caller of `_ensure_pipe_sem` — never runs on that path. The existing
tests exercised `_ensure_pipe_sem` directly and so missed the caller
that skipped it, letting a `None` semaphore reach `.acquire()`.
Both tests fail with "'NoneType' object has no attribute 'acquire'"
when the lazy init is removed from the borrowed path.
* feat(database): Sequelize ORM over pipes + server-side DB transactions (#1467)
* feat(database): server-side multi-statement transaction sessions
Add a TransactionRegistry that holds one pooled DB connection open per session_id (bounded, idle-reaped, connection-leak-guarded), wire it into the DatabaseGlobalBase lifecycle, and expose begin/commit/rollback tool functions plus a session-aware execute with server-side $n parameter binding. All are gated by allow_execute. Includes SQLite-backed unit tests and db_postgres/db_mysql doc updates.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(database): SDK transaction methods + Sequelize ORM (client.database.sequelize)
TS and Python clients gain beginTransaction/commit/rollback and query() session/params. Add a pg-compatible shim injected as Sequelize v6's dialectModule (transaction-control SQL drives the pipe's begin/commit/rollback), a createSequelize() factory, and client.database.sequelize({ Sequelize, ... }) that injects the Sequelize class so the core client uses type-only imports and stays browser-bundle-safe (sequelize is an optional peerDependency). Includes unit + integration tests and the SDK doc.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* test(database): end-to-end transaction + Sequelize smoke tests
Gated (ROCKETRIDE_E2E=1) end-to-end tests exercising the full client -> pipe -> engine -> Postgres round-trip: multi-statement transaction isolation/commit/rollback via the raw tool functions and via client.database.sequelize(). Verified green against a live engine + Postgres.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(database): address PR review — per-session tx locks, savepoint interception, test/doc coverage
- tx_registry: per-session locking so different sessions run concurrently; the idle reaper skips in-flight sessions (was one global lock held across conn.execute()).
- pg-shim: anchor the transaction-control regex so `ROLLBACK TO SAVEPOINT` (nested rollback) is forwarded as an ordinary query rather than intercepted as a top-level ROLLBACK.
- e2e: close the Sequelize instance in a finally block so each test releases its RocketRide session.
- Tests: session-invalidated-after-rollback, multi-digit ($1..$11) placeholder guard, reaper-skips-in-flight, tool-not-called on validation failure, ROLLBACK TO SAVEPOINT pass-through.
- db_postgres/db_mysql READMEs: document the session-aware query(session_id/sessionId, params) API.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* style(database): ruff-format db_instance_base.py (CI ruff 0.15.20 parity)
* fix(database): roll back on max_execute_rows overflow in stateless execute
_executeRawQuery() logged and returned None inside engine.begin() when a
SELECT/RETURNING overflowed max_execute_rows, so a write still committed while
execute() reported failure (duplicate-retry risk). Raise inside the transaction
so it rolls back — mirroring the session path. Adds a regression test.
Reported by CodeRabbit.
* fix(database): roll back session on a failed session-bound execute
A session-bound execute that raises (e.g. max_execute_rows overflow) left the
held connection pinned until idle-reaping, with the aborted statement still
committable. Roll the session back before re-raising to release the connection
and discard the statement. Adds a regression test.
Reported by CodeRabbit.
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(shell-ui): surface dashboard fetch failures instead of loading forever (#1673)
* fix(shell-ui): surface dashboard fetch failures instead of loading forever
_fetchDashboard() caught every error and only console.log'd it, leaving _data
null. Consumers render their loading state whenever data is null, so a failed
fetch was indistinguishable from a slow one and the view sat on "Loading
dashboard data..." indefinitely.
That is what the "Dashboard hangs on Loading" report was (rocketride-saas #373):
the request was returning Permission 'task.monitor' denied and the UI simply
never said so.
Track the failure as first-class state, clear it on success, phrase permission
denials in plain language, and expose it as `error` on the hook so views can
render it. Adding a field is backward compatible — the existing consumer
destructures a subset and is unaffected.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015nTVr6jfSFYm1GppxbjghP
* fix(shell-ui): don't double up the denial wording; clear data on denial only
Both of @anandray's points.
**Doubled wording.** The raw message already says "permission denied" in
exactly the case the friendly sentence exists to explain, so the two
concatenated into "You do not have permission to view the dashboard. Permission
denied". Replaced rather than appended; the verbatim text still goes to the
console line directly above for anyone debugging.
**_data retention — answering rather than confirming.** The retention was
intentional for transient failures and should stay: a poll that blips must
leave the last good numbers on screen with an error beside them, not blank the
dashboard every time the network hiccups. Going empty on each failed poll
would be its own bug.
But it was wrong to apply that uniformly. A denial is not a blip — it means
this user is not entitled to these numbers, and continuing to show a previous
session's data under an "access denied" banner is both confusing and the wrong
default for something access-scoped. So `_data` is now cleared on denial only,
and the split is spelled out in the comment so the next reader does not have to
guess which half was deliberate.
Note the existing 104 tsc errors in apps/shell-ui are unrelated (unbuilt
workspace deps); count is identical with and without this change, and none are
in this file. CI does not typecheck shell-ui, so this was checked locally.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015nTVr6jfSFYm1GppxbjghP
* fix(shell-ui): classify a denial from the structured error, not the prose
Two unresolved CodeRabbit threads. The second is the substantive one and it
breaks this PR on its own terms.
Denial detection was /denied|permission|forbidden|403/ over err.message. But
getDashboard() goes through call(), so a structured DAP failure arrives as
DAPException with the detail in `dapResult` — the human-readable message is not
guaranteed to carry it. And even on the text path the pattern had 403 but not
401 and not "unauthorized", so a plain `Unauthorized` fell through as a
transient blip.
Either way the result is the branch failing to fire for exactly the case it
exists for: a real denial would keep rendering the previous session's numbers
under an "access denied" banner, which is the behaviour this PR is meant to
remove.
Now reads dapResult.status / statusCode / code first and falls back to the
text, with 401 and unauthori[sz]ed added. Checked the classification against
both: Unauthorized, unauthorised, Not authorized, 403 Forbidden, bare 401 and
"failed with status 401" all classify as denied; timeout and network error stay
transient, so cached data still survives a blip.
Also corrected the snapshot comment, which claimed the object is replaced "only
when data or events change" — it is replaced on error too.
Related: #1673.
* fix(shell-ui): treat symbolic DAP denial codes as denials
CodeRabbit caught a real hole in this PR. `Number()` over dapResult.code turns
a symbolic 'UNAUTHORIZED' or 'FORBIDDEN' into NaN, so neither status check
fired, and the prose regex only ever saw `message`. With a generic human
message — which is exactly the case this branch was written for, since the
structured detail is not guaranteed to reach the text — `denied` stayed false.
The consequence is the specific bug this PR is meant to kill: `_data` is
deliberately not cleared on failure so a network blip leaves the last good
numbers on screen, so a missed denial keeps rendering the previous session's
figures under a vague error instead of "you do not have permission".
Now matches the code as a string as well. The pattern is lifted to a named
const rather than duplicated, and carries no `g` flag, so reusing it across two
.test() calls has no lastIndex statefulness.
Verified with tsc against apps/shell-ui/tsconfig.json: no errors in this file.
(The remaining TS2307 "Cannot find module 'rocketride'" errors are the known
unbuilt client-typescript dist/types artifact, not this change — and worth
saying out loud that CI would not have caught a type error here either, since
nothing typechecks apps/** on a PR.)
Refs #1673.
---------
Co-authored-by: Dmitrii Kataraev <dmitrii.kataraev@rocketride.ai>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* fix(auth): log why has_permission denies instead of swallowing silently (#1670)
* fix(auth): log why has_permission denies instead of swallowing silently
resolve_team_permissions() raises PermissionError when the session's
defaultTeam is not present in account_info.organization (AccountInfo carries
a single org). has_permission() caught that and returned a bare False, so the
caller only ever saw "Permission 'x' denied" with no trace of the real cause —
an unresolvable default team. That silent swallow is why saas #373 (admin
Dashboard hanging on "Loading Dashboard Data" with Permission 'task.monitor'
denied, for a user who genuinely held the grant) took two days to diagnose.
Behaviour is unchanged — still denies — but the reason is now in the log.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015nTVr6jfSFYm1GppxbjghP
* fix(auth): log the denial where we can actually see it (stdlib, not rocketlib)
@anandray asked me to verify rocketlib.debug is enabled in prod before relying
on it. Checked against the running cloud ALB, and the answer kills the original
approach twice over:
1. debug() is gated on the engine trace level and prod runs without one:
`./engine ./ai/eaas.py --saas --host=0.0.0.0 --port=5565 --modelserver=...`.
Corroborated in the logs — `[account] upsert_user` fires on every login and
appears zero times.
2. Bumping to warning() does not fix it either, which is what I would have
shipped on the first suggestion. rocketlib output goes to the engine JOB
log, which is delivered to the connected client, not to container stdout.
A 5000-line sample of the prod ALB log is 100% uvicorn access lines:
1808 GET / 200
681 GET /version 200
237 connection open
216 WebSocket /task/service [accepted]
Nothing from the engine at all, and no engine log file in the pod. So this
PR as written produced no server-side observability whatsoever — and
denying a client, then reporting the reason only to that same client, is
not observability in any case.
Switched to stdlib logging, which those uvicorn lines prove reaches stdout and
therefore CloudWatch. Volume is bounded by actual denials, and a flood of them
is itself the signal — that is exactly the shape #373 had.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015nTVr6jfSFYm1GppxbjghP
* test(ai): assert the denial log says who, which team and what failed
CodeRabbit's point on this PR, and it is the same defect I have been catching
in other people's work all week: the existing test asserts only that
has_permission returns False, which passes just as happily with the logging
deleted. The log is the entire point of the change, and nothing was holding it.
The new test asserts the record carries the team, the user, the requested
permission and the original PermissionError text — so removing the line, or
dropping a field from it, fails.
Note the suggestion said to mock task_conn.debug; that is stale. This PR moved
the call to stdlib logging precisely because rocketlib's debug() never reaches
container stdout in the cloud, so the test uses caplog against the module
logger instead.
HONESTY ON VERIFICATION: I could not run this. The suite needs the built
engine — pytest walks fastapi -> uvicorn -> dotenv -> `rocketride` itself, at
which point pip starts trying to fetch our own published package over the local
source, and that is the wrong direction. Assertions were checked statically
against the f-string in task_conn.py: 'team-1', 'user-1', 'task.control' and
the exception text all appear in it.
And CI will not run it either. os-matrix.yml is the only workflow invoking
`./builder test`, and it fires on workflow_dispatch or a monthly cron — so
packages/ai/tests gates nothing on a PR. That is the third instance today of
tests existing with nothing running them (see saas #400 / #401). Filing
separately rather than widening this PR.
Related: #1670.
* test(ai): pin the denial log to one WARNING from this module
Second CodeRabbit finding on this PR, and it is the same criticism I made of
the test I was replacing — mine was also too loose to fail properly.
Joining every captured record meant an unrelated log could satisfy the field
assertions, and the test would keep passing if the denial were downgraded to
debug or raised to error. Both matter here specifically: the whole reason this
PR exists is that rocketlib's debug() never reaches container stdout in the
cloud, so a level change is exactly the regression that would make the message
invisible again while the test stayed green.
Now filters to records from task_conn's own logger at WARNING and asserts
exactly one, so a wrong logger, a wrong level, a missing record or a duplicate
all fail — and the failure message prints what was actually captured.
Refs #1670.
---------
Co-authored-by: Dmitrii Kataraev <dmitrii.kataraev@rocketride.ai>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* fix(server): dispatch pipeline lifecycle in topological order, including control sub-pipelines (#1669)
* fix(server): dispatch pipeline lifecycle in topological order
Pipeline open/closing/close were dispatched in `.pipe` canvas order, not
dependency order, so a join node could be flushed before an upstream branch
emitted its flush-time data — silently dropping it.
Bind the source-reachable nodes' lifecycle onto the pipe head in topological
order instead: closing/close upstream-first (a node flushes only after all of
its upstream parents), open reversed. Control/invoke sub-pipelines are left
untouched — they keep their per-parent binding so their invoke node (e.g.
`tool_pipe`) can still open/close them.
The order is computed once by `computeLifecycleOrder` (reachability from the
source, then a Kahn topological sort over the reachable set; a cycle in the
reachable region is rejected). Adds unit tests for the ordering and documents
the guarantee.
Closes #1650
* fix(server): extend topological lifecycle to control sub-pipelines
Generalize the open/closing/close dependency-order guarantee to a control node's
inline sub-pipeline (e.g. tool_pipe): it is flushed and closed in dependency order
per invocation, to any nesting depth, so a diamond sub-pipeline returns the merged
output of all branches. tool_pipe now drives closing() then close().
Reject at pipeline open the wirings that break exclusive sub-pipeline ownership:
sharing a sub-pipeline node with the main flow, a second start, or another control
node; and a data-fed invoke node.
Adds unit tests and updates pipeline-rules/observability docs and examples.
* fix(server): name nodes in sub-pipeline ownership errors
Report each node as its service title plus component id, e.g.
"Pipeline Tool" (pipe_tool_1), instead of a pipeStack index, so a rejected
wiring points at what the author sees on the canvas.
Document the exclusive-ownership rule in the tool_pipe README and add a
common-mistakes entry covering the three rejections. Examples switch to
agent_crewai and lane-specific response nodes.
* fix(server): address review feedback and cover the closing order
Remove `Binder::getBound`: public API added with no caller.
Range-check the lifecycle indices once up front rather than inside one of the
bind loops, so every walk is covered whatever order they run in, and correct
the comment that claimed bindFilters was allocation-free per pipe.
Add an end-to-end test for the closing order: a diamond of buffer-and-flush
nodes whose join is declared before its branches, so dispatching in declaration
order would flush the join early and drop a branch's output.
* fix(deps): restore a resolvable onnxruntime pin — 1.20.1 is gone from PyPI (#1725)
The -gpu build of 1.20.1 is absent from the index (not yanked; the release
key itself is gone), so every dependency resolve that misses uv's cache dies
with "no version of onnxruntime-gpu==1.20.1 ... requirements are
unsatisfiable". The CPU build of the same version survives, which is what
let the split sit broken unnoticed.
Move all five consumers to 1.22.0 together. It has wheels for -gpu on
win/linux and for the CPU build on macOS-arm, which 1.20.1 no longer does.
Note that the obvious one-patch bump is a trap: 1.20.2 exists only for -gpu,
so pinning it would break macOS.
Also correct the comment that justified the pin. It claimed rtmlib needs
exactly this version; rtmlib declares onnxruntime unpinned, as does gliner,
and faster-whisper asks for <2,>=1.14. The pin exists so the ~200 MB runtime
installs once across consumers — the number is ours to choose, provided all
five files agree.
Verified with the four consuming nodes, opted in via ROCKETRIDE_INCLUDE_SKIP
(real installs and model downloads, not just resolution):
pose_estimation (rtmlib), anonymize, ner (gliner), audio_transcribe
(faster-whisper) — all pass.
Fixes #1723
* chore(models): sync LLM model lists (#1634)
Co-authored-by: asclearuc <29624014+asclearuc@users.noreply.github.com>
* feat(nodes): add Guild.ai tool node (#1689)
Dual data+tool node running Guild.ai agents from a pipeline step or as an agent tool.
Mirrors tool_n8n (dual-node structure) and tool_v0 (IGlobal/test conventions, raise-not-return errors).
- Pipeline face: lane input runs the configured Guild agent; the answer flows downstream.
- Tool face: run_agent / get_session / get_session_events to delegate to and inspect sessions.
- HTTP Basic trigger-key auth; per-run session budget; POSTs never retried; identifiers sanitised.
- Response shapes verified end-to-end against a live Guild workspace, pinned as regression fixtures.
Closes #1688
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(docs): normalize Discord invite links to a single code (#1746)
The repo referenced three different Discord invites. Standardize every
reference on discord.gg/PMXrtenMsY (the code already used in the
top-level README and client readmes):
- discord.gg/9hr3tdZmEG (docs site + co-located docs) -> PMXrtenMsY
- discord.gg/rocketride (hello-ui placeholder) -> PMXrtenMsY
CI webhook URLs (discord.com/api/webhooks/...) are unrelated and left
untouched.
Co-authored-by: Mithilesh Gaurihar <mithileshgaurihar@Mithileshs-MacBook-Pro.local>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* fix(agent): optional require_tool_call runtime fabrication guard (#1490)
* fix(agent): optional require_tool_call runtime fabrication guard
No runtime guard asserted that an agent actually invoked a tool, so a weak
planning model could narrate a multi-step tool chain in prose instead of
calling the tools — producing a plausible but ungrounded answer. The
"must call >=1 tool" rule existed only as a prompt instruction, never as code.
Enforce it once in the shared AgentBase so it covers all five agent
frameworks (langchain, llamaindex, rocketride, crewai, deepagent):
- AgentContext gains a run-scoped `invoked_tools` tally (a mutable field on
the otherwise-frozen context; compare=False keeps the context hashable, and
list.append is thread-safe under the GIL).
- call_tool — the single choke point every driver routes real tool calls
through — records each invocation (memory.peek and other local reads bypass
it and correctly do not count).
- run_agent raises ToolCallRequiredError -> a distinct
RocketRide.agent.guard.v1 error answer when require_tool_call is on and no
tool ran; meta.tool_calls now reports the count on success, guard, and error.
- The crewai-manager and deepagent sub-contexts thread the parent's tally so
tools a sub-agent invokes count toward the parent run's guard.
Exposed as an off-by-default `require_tool_call` boolean on the six top-level
agent service files, read with parse_bool (a stringified "false" leaves it
off, rather than tripping the guard), with README notes + regenerated schema
tables.
Tests (packages/ai/tests/ai/common/agent/ + test_config_shapes.py): AgentBase
guard unit tests, config resolution (flat/nested/default shapes), and
end-to-end integration driving the real RocketRide wave loop and the real
LangChain create_agent loop — fabrication trips the guard, grounding passes,
memory.peek-only trips, and parallel tools in one wave are all counted.
Closes #1403
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(agent): version guard code in docs + retain raw trace on guard answer
Addresses PR review feedback:
- The Fabrication-guard notes in all five agent READMEs referenced the
unversioned `RocketRide.agent.guard`; the runtime emits
`RocketRide.agent.guard.v1` (matching the existing `.raw.v1` / `.error.v1`
identifiers). Use the versioned id consistently.
- The guard-trip answer now also carries the framework's raw output as a
`RocketRide.agent.raw.v1` stack entry (like the success path), so an operator
investigating a tripped guard can inspect exactly what was narrated. `raw` is
bound before the try, so this is safe on every path.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: kgarg2468 <181051779+kgarg2468@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(chroma): validate server version and surface silent index failures (#1494)
* fix(chroma): validate server version and surface silent index failures
Chroma silently indexed nothing against incompatible/old servers: the real
error (KeyError('_type')) was swallowed into a transient debug() trace, and
the user only saw a cryptic "'NoneType' object has no attribute 'delete'"
crash with "0 done · 1 error" while downstream chat hallucinated.
- Validate the Chroma server version on connect (floor: major >= 1) and wrap
the eager client construction so an incompatible/old server yields an
actionable message instead of a cryptic one.
- _createCollection now re-raises the real cause instead of swallowing it;
flush() guards a missing collection with a clear message (defense in depth).
- Harden _doesCollectionExist to accept both list_collections() shapes (names
and Collection objects), fixing a latent always-empty-search bug on newer
clients.
- Pin chromadb-client>=1.0.0 to match the supported server floor.
Fixes #1405
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(chroma): make server-version check robust to thin client; floor 0.6
The initial connect-time version check broke the working path in CI:
- The node installs the thin `chromadb-client`, whose HttpClient may lack
get_version. Calling it inside the connect try mis-reported the AttributeError
as "server too old". Probe defensively via _getServerVersion() so an absent or
failing get_version is treated as unknown, never a connection error.
- The >= 1.0 floor rejected servers that work (the project's own test infra runs
chromadb/chroma:0.6.3). Lower the floor to 0.6.0 (the v2 tenant/database API
boundary), compared on major.minor.
- Revert the chromadb-client>=1.0.0 pin: the incompatibility is server-side and
the pin was ineffective in CI.
Refs #1405
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: kgarg2468 <181051779+kgarg2468@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(storage): joined filesystem, in-store authorization, task-file identity, and rocketlib.getTask (#1686)
* feat(storage): joined filesystem, in-store authorization, task-file identity, and rocketlib.getTask
Foundation for team-scoped deployments: deployments will belong to teams,
so storage, permissions, and subprocess identity all had to stop assuming
"everything belongs to the calling user". This lands that groundwork;
deployment records and team run logs build on it next.
WHY each piece exists:
* In-store authorization — file_store.py IS the security boundary.
Storage I/O happens from many places (rrext_store, rrext_log, fetch,
the run-log writer, deployments). Checking permissions in command
handlers meant every new caller could forget one — the "forgot to
check" bug class. Now every path of every operation authorizes INSIDE
the store, at the single point where paths resolve, against the
RequestContext bound at construction. Command handlers do zero storage
permission checks.
* RequestContext identity (copied verbatim from feat/alb, with MERGE
NOTEs). feat/alb generalizes per-request identity for every handler;
adopting the identical shape now makes that merge a dedupe instead of
a rework. The internal()/engine() factories make the entire trusted
surface greppable: every privileged caller is a
RequestContext.internal(...) call site.
* Store singleton + RR_STORE_URL (STORE_URL hard-fails). A server has
exactly one store, so Store.instance()/Store.file_store(ctx) replaces
the multi-step create()/get_file_store dance that invited divergent
configuration. The legacy env name fails loudly instead of falling
back, because a silent fallback would never get migrated. Providers
became packages (store_providers/filesystem/, memory/). Write locks
and open handles moved to the shared Store so per-connection FileStore
instances still exclude each other on one physical path.
* Joined filesystem, grammar v3: '@' is a virtual root listing User/,
Team/, Org/ so a file browser can walk user, team, and org storage as
ONE tree with ordinary path joining — while plain paths stay "my
files" for nodes and SDK code (simple mode). Bare segments are display
NAMES that resolve ONLY through the caller's own session membership
(a foreign org's names are inexpressible by construction) and
round-trip byte-exact: the path you browse to is the path you open.
=<id> is the only cross-boundary spelling (sys.admin support access).
Unknown, foreign, permission-less, and ambiguous references all deny
with ONE identical message so nothing leaks existence. Names starting
with '=' are uncreatable at scope tops so id references can never be
shadowed. Case-exact and alias-free: one location, one path string.
Normalization runs BEFORE parsing, killing the \@\Team\... bypass
family (fuzz-tested).
* System-owned trees: .logs and .deployments are engine-written and
served by their domain APIs (rrext_log today, rrext_deploy next).
Raw file access could corrupt them or expose data the user must not
see, so through the file API they are invisible in listings and denied
on access for every session identity except sys.admin (who may do
anything). Domain APIs act through the internal identity and keep
their own task.monitor/task.control gates at the command layer.
* Three live cross-team permission holes closed: on_execute accepted an
arbitrary teamId and merged that team's secrets into the task env
BEFORE any membership check (secret exfiltration); on_restart
restarted token-addressed tasks with only a defaultTeam check; the
cprofile proxy could drive another team's engine subprocess. All three
now resolve permissions against the ADDRESSED team/task first
(verify_team_permission / get_task), with uniform denials.
* Task-file identity + storage anchor: subprocess identity rides the
0600 task file ('identity' block) — never the environment, whose
ROCKETRIDE_* namespace is caller-influenced by design; the
ROCKETRIDE_CLIENT_ID env var is gone. The new 'storage' block gives
nodes chroot semantics: dev runs anchor at the owner's whole tree
(unchanged behavior), deploy runs at teams/<teamId>/files/tasks/
<projectId>, so node-level paths are IDENTICAL in both modes, a
deployed task needs no user, and concurrent deployments cannot fight
over working storage. The engine context rejects every '@' spelling
and system tree regardless of anchor — LLM-generated paths cannot
leave the sandbox.
* rocketlib.getTask() C++ binding: the engine already parses the task
file, so python re-reading argv would duplicate parsing and couple to
the launch contract. ITask::execute() publishes the task JSON around
the begin/end window — before beginTask, because endpoint (and node
global) construction happens inside it — and clears it on every exit
path. The store layer resolves identity+anchor itself
(Store.engine_file_store()), so tool_filesystem carries zero jobConfig
plumbing and future consumers get task data for free. Single-threaded
by contract (main task thread only), hence no lock.
* explorer-ui roots its listing at '@' to present the joined tree.
1572 server tests green (suite grew ~90 tests: authorization matrix,
grammar fuzz, system-tree policy, anchor chroot, cprofile proxy).
Engine rebuilt on both dists; rocketlib.getTask verified live.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(storage): resolve PR #1686 review findings — fail-closed handles, universal root guards, real org resolution
Addresses the 14 CodeRabbit review threads:
Security/correctness:
- file_store: _get_handle requires an EXACT conn_id match. The handle
registry became Store-wide in this PR, so the lenient empty-conn_id
check would have let any default RequestContext reach another
connection's live handle. Fail closed.
- file_store: the rename/delete root guards no longer exempt the caller's
own namespace — rename('') resolved to the account root and turned the
copy+delete loop into a whole-account move. delete/rename/rmdir now
reject every root uniformly.
- store: a session-shaped ctx (account present, not internal/engine) with
an empty userId can no longer anchor a FileStore via explicit client_id;
that combination (e.g. a task-scoped AccountInfo built from an empty
control.userId) would unlock an arbitrary user's namespace. Account-LESS
contexts stay constructible because resolve_scope already denies their
every operation with 'Not authenticated'.
- task_conn/cmd_task/cmd_debug: new resolve_org_for_team() — members
resolve via their own membership; callers that pass the team check
without membership (sys.admin, internal) resolve the target team's REAL
org through the account backend, and an unresolvable team is denied with
the uniform message. Previously such callers registered the task with
orgId='' travelling in the task file as trusted identity.
- task_engine: _storage_root() validates through validate_storage_root at
launch, so a malformed client-supplied project_id fails with a clear
launch-time error instead of tool_filesystem disabling itself mid-run
inside the subprocess.
Minor:
- explorer-ui: handleMove/handleUpload carry the '@' mount through an
empty targetDir — the tree is rooted at '@', so bare names silently
targeted the plain user store.
- tool_filesystem: IGlobal docstring, README handwritten sections, and the
services.json pathWhitelist description describe the task storage anchor
(dev = user tree, deploy = team task subtree) instead of the universal
user-root claim. The generated README block regenerates on develop
(nodes:docs-generate is branch-gated by design).
- file_store: stale usage examples fixed (Store.file_store(ctx), no
connection_id params); handle-cap comment updated to string conn_ids.
- task.hpp/task.cpp: currentTask()'s main-task-thread-only access model is
stated on the public API — publish, reads, and clear are strictly
sequential on the single main task thread, so the value is
unsynchronized by design.
- store/file_store: per-handle force-close teardown deduped into
Store._teardown_handle(); FileStore._force_close_handle and
close_all_handles delegate so the sequences cannot drift.
- task_server: dropped the outer sys.admin short-circuit duplicated inside
resolve_task_permissions.
Tests: real resolve_org_for_team bound into the cmd_task/cmd_debug stubs,
get_merged_env patched where execution reaches the secret merge, dead
unowned FileStore construction removed, 16 inline FileStore(Store(istore),
...) constructions replaced with a shared make_file_store() helper, and
the sys.admin bypass test now asserts the resolver-side grant.
1572 passed / 122 skipped; ruff clean; explorer-ui rebuilt.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(storage): close the empty-orgId fast path; take develop's services.json description
resolve_org_for_team returned org.get('id', '') straight from the
membership loop, so a membership record whose org carries no id skipped
the empty-org guard the function exists to enforce. Both paths now feed
one guard: membership resolves the id (falling through to the account
backend when it is empty), and an unresolvable org denies with the
uniform no-permissions message.
tool_filesystem/services.json keeps develop's (#1651) description text
verbatim as the conflict resolution for the rebase onto develop; the
sink code's storage wording will be reconciled with the task-anchor
model when this PR merges.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(storage): resolve PR #1686 round-2 review findings — one normalization, sigil reservation, membership-based own-tree access
- file_store: normalize_path extracted to module level as THE wire-path
normalization; FileStore._validate_path delegates to it. cmd_store's
system-tree filter now normalizes with this same routine AND resolves the
listing from the same normalized path — deciding on the raw spelling while
resolving the normalized one let '@//User'-family spellings reach the user
root with .logs/.deployments unfiltered.
- file_store: enforce the documented '@' reservation at scope tops alongside
'=' — a scoped rest could smuggle in an '@'-leading name inside the
grammar room the docs already reserved.
- file_store: the caller's OWN tree no longer hinges on the defaultTeam
pointer (an unset or stale defaultTeam denied a user their own storage);
the file-storage permission grants when ANY membership carries it.
- file_store: rmdir root guard made universal (plain `not rest`) to match
delete/rename — the kind-conditional form was exactly the guard a future
normalization change could quietly defeat.
- store: double-checked locking on the Store singleton — two threads racing
instance() could each build a Store with its own _shared_handles /
_shared_write_locks registry, silently defeating the cross-instance write
exclusion this PR introduces (engine-subprocess node init is the realistic
race).
- task_engine: anonymous dev runs (client_id='' — OSS/standalone) carry NO
storage anchor instead of failing the launch: identity.userId rides empty
too, so engine_file_store() yields None and the storage tools disable
themselves — the same degradable posture as the run-log writer; the shared
'users//files' prefix is never composed.
- models: RequestContext.account_info docs now name engine-subprocess
contexts as the second legitimate None case.
- tests: pin the universal own-root guard (delete/rename of ''), the
normalization-bypass listing matrix ('@//User', '@/./User', '\@\User',
'/'), the dev-run no-anchor posture, STORE_URL/RR_STORE_URL isolation via
monkeypatch.delenv, and mirror Store.file_store's real signature
(root=None) in the cmd_task stub.
1636 server tests green (./builder ai:run-pytest --verbose).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(storage): audit sys.admin =id crossings; pin internal ordering; document resolver contract split (PR #1686)
Addresses the three non-blocking notes from the authorization review:
- file_store: sys.admin '=id' resolutions are the ONE place resolve_scope
crosses the storage boundary instead of enforcing it (the platform support
capability). Each crossing — foreign team/org/user id — now emits a
server-side audit trace (actor, conn, target) via rocketlib.debug. Member
'=id' resolutions cross nothing and are NOT audited. Trace-only: the wire
response still cannot distinguish a crossing from an ordinary resolve, so
the uniform-denial / no-oracle posture is preserved.
- file_store: comment the internal-identity branch as deliberately ordered
BEFORE the system-tree gate (intent, not fall-through) — the run-log writer
and domain APIs write .logs/.deployments through that identity;
test_internal_identity_writes_logs is now also the ordering pin.
- models: banner both resolve_task_permissions (RETURNS []) and
resolve_team_permissions (RAISES) with their opposite no-membership
contracts. The rename to carry the difference in the name (get_* vs
require_*) is deferred to the feat/alb dedupe, where both branches touch
these functions — renaming here would fight that dedupe.
- tests: TestSysAdminCrossingAudit pins that foreign-id crossings audit and
member resolutions don't (monkeypatches the hook, since the sink is native
rocketlib.debug).
757 account+task tests green (./builder ai:run-pytest path).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(storage): resolve PR #1686 round-3 review — sign resolved fetch paths, live Store.reset, single-parse resolver
- get_url now signs the RESOLVED physical store path instead of the wire
spelling. The fetch handler re-resolved the claim under an internal
identity, which has no name dictionary and no org context — so signed
URLs for '@/Team/<name>' and '@/Org' paths died with an unhandled
PermissionError (HTTP 500). Authorization already ran at issuance under
the session identity; the signed JWT is the capability, so the handler
now serves the claim verbatim through backend._get_full_path (which
guards traversal) with no scope re-resolution at all.
FileStore.resolve() existed solely for that handler re-resolution and
lost its only caller — removed rather than left as dead public surface.
New TestSignedFetchUrls pins the reviewer's failure matrix (plain,
@/User, @/Team by name, @/Team by id, @/Org), the resolve-free serve
path, and that issuance still gates (@/Org without org.admin cannot
mint a URL).
- Store.reset() is now actually wired: an autouse fixture in the ai tests
conftest drops the process singleton around EVERY test, so no test can
pin the RR_STORE_URL (plus _shared_handles/_shared_write_locks) that
happened to be live when the first singleton-reaching test ran —
singleton state was order-dependent and stateful before this.
Also corrected the two stale STORE_URL docstrings in store.py to
RR_STORE_URL (the legacy name hard-fails at create()).
- resolve_scope now returns (kind, scope_root, rest): it already had the
kind in hand from its own parse_scope call, so _resolve no longer
re-parses every path just to recover it — one parse per operation, and
no "cannot fail" reasoning left for readers to re-verify.
- _store_fs_stat's virtual-mount check now runs normalize_path — the same
ONE-normalization rule _store_fs_list_dir documents and follows —
instead of .strip('/'), so mount detection and store resolution can
never disagree about a spelling, and traversal raises instead of being
stripped.
Full ai suite: 1646 passed, 122 skipped.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* chore(deps): align @types/react to ~18.3.x across the workspace
The root package.json pinned EXACT @types/react 18.2.79 / react-dom 18.2.25
while shared-ui had moved to ~18.3.29, so pnpm materialized TWO peer-variant
instances of every @types/react-peer package (react-markdown and friends) —
duplicated store copies and forked type-checking between apps. One types
version ends that: root, shell-ui, chat-ui, and dropper-ui now declare the
same ~18.3.29 / ~18.3.7 ranges shared-ui already uses. Types-only change;
chat-ui and dropper-ui type-check clean on the new majors, and the
pre-existing shell-ui contract-check delta is unchanged (verified against a
stashed baseline).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* chore(deps): land the @types/react package.json pins (completes 4b12d763)
4b12d763 was meant to carry these pins but landed only the lockfile
reshuffle (a stash-pop failure dropped the package.json edits before the
commit — message and content disagreed). This commit lands the actual
alignment: root, shell-ui, chat-ui, dropper-ui at ~18.3.29/~18.3.7;
vscode's floor raised to the resolved ^18.3.31; shared-ui's tilde floor
follows to ~18.3.31. One @types/react instance in the lock (zero 18.2
refs), so @types/react-peer packages (react-markdown et al.) no longer
fork into duplicate peer-variant copies.…
Summary
open/closing/close) was dispatched in.pipecanvas /input[]order, not dependency order, so a join node could be flushed before anupstream branch emitted its flush-time data — silently dropping it (the admin
chat.pipeincident: embeddings never reached the vector store).closing/closeupstream-first (a node flushes only after ALL its upstream parents),openreversed. The pipe head owns the source region; each control/invoke node (e.g.tool_pipe) owns the sub-pipeline it reaches and drives it the same way, to anynesting depth. Orders are computed once by
computeLifecycleOrder(reachability + Kahntopo sort) and partitioned by
computeLifecycleRegions.tool_pipenow drivesclosing()beforeclose(), so buffer-and-flush nodes in itssub-pipeline emit before the tool reads its result. A diamond sub-pipeline returns the
merged output of all branches instead of dropping one.
a sub-pipeline shared with the main flow or a second start; a node reached by two
control nodes; an invoke node that is itself data-fed.
Behaviour notes for reviewers:
Policy: a node is flushed and closed only after every node that feeds it, and
opened before them. In the diamond
A -> J,B -> J:closing/close:A,B, thenJ— the join is lastopen:J, thenB,A— the join is first, so it is ready before anyone writes to itThis matches today's chain semantics, so existing linear pipelines are unchanged.
Control/invoke sub-pipelines are no longer left on per-parent bindings — this is the
scope change versus the first review. Their nodes move onto the invoke node's own lanes.
The three rejections above are new hard failures at
beginEndpoint/client.use().Normal
agent -> tool_pipe -> sub-pipelinewiring is unaffected; invoking the sametool from two agents stays valid.
Rejection messages name nodes by service title and component id, e.g.
Control node "Pipeline Tool" (pipe_tool_1) reaches node "Prompt" (sub_mid) ....An error in a native node's
closingnow aborts the remaining flush walk (Python nodesalready did; unifies semantics).
Ec::PreventDefaultfrom a lifecycle callback no longer prunes a subtree (no productionnode uses it there).
Trace
apaevt_flow: lifecycle frames appear flat under the pipe head, and under eachcontrol node for its sub-pipeline (pairs stay balanced; no protocol change).
buildConnectionsnow clears thecontrolstable (stale rows could survive a retriedbeginEndpoint).Type
fix
Testing
store::lifecycleOrder— 22 sections / 31 assertions (per-region ordering)store::lifecycleRegions— 12 sections / 20 assertions (partition + all threeownership rejections, nested invoke chains, multi-start topologies)
nodes/test/test_lifecycle_order.py— end-to-end guard for theclosingorder: adiamond of buffer-and-flush nodes whose join is declared before its branches, so
declaration-order dispatch would flush it early and drop a branch. No LLM or mock.
store::linkages(38) andPipelineConfig(42) green, noregressions. Runtime verified against a live engine with instrumented sub-pipeline
nodes: each node receives
open/closing/closeexactly once per invocation andthe join flushes last, both single-level and two-level nested.
./builder testpassesDocs & examples
docs/agents/ROCKETRIDE_PIPELINE_RULES.md— lifecycle guarantee extended to controlsub-pipelines, plus the three wiring constraints
docs/agents/ROCKETRIDE_COMMON_MISTAKES.md— new entry for sharing a sub-pipelinenodes/src/nodes/tool_pipe/README.md— exclusive-ownership rulepackages/server/docs/observability.md— how sub-pipeline lifecycle frames appearexamples/tool-pipe-diamond.pipe,examples/tool-pipe-nested.pipe, andexamples/incorrect/(five rejected wirings + README)Checklist
Linked Issue
Fixes #1650
Summary by CodeRabbit
New Features
Bug Fixes
Documentation