ROSClaw

中文版

ROSClaw is a robot-agent runtime with a first-class CMU ROS1 mobility application. In this repository it turns natural language, CLI commands, and MCP tool calls into guarded high-level robot actions, records replayable evidence, and keeps robot embodiment, world semantics, safety, evaluation, and recovery logic in one Python package.

What Is In This Repository

Area	What it does	Main code
CMU Mobility app	Natural-language navigation, exploration control, observation capture, visual target navigation, chat/IM control, dashboard, and metrics for the CMU ROS1 ARE + ARiADNE2 stack.	src/rosclaw/apps
Runtime core	Event bus, durable run spine, lifecycle, provider routing, guard pipeline, memory, practice recording, and graceful degradation when optional backends are missing.	src/rosclaw/core, src/rosclaw/run
Agent OS control plane	Governed ActionEnvelope execution, RBAC, policy decisions, approval interrupts, audit logs, kill switches, context bundles, prompt registry, and memory policy.	src/rosclaw/action, src/rosclaw/governance, src/rosclaw/context
Agent interfaces	CLI, FastAPI /api/v1, stdio MCP server, HTTP MCP facade, and versioned CMU/runtime/context tool names for local agents.	src/rosclaw/cli.py, src/rosclaw/api, src/rosclaw/mcp
e-URDF robot profiles	Robot identity, embodiment, safety, semantic, capability, benchmark, and simulation metadata loaded from the local zoo.	e-urdf-zoo, src/rosclaw/runtime
Mission and world model	Offline mission planning, recoverable MissionGraph execution, BehaviorTree.CPP XML import/export for review, semantic place grounding, taught places/objects, dynamic entities, and scene graph snapshots.	src/rosclaw/mission, src/rosclaw/worldmodel
Practice and evaluation	Standardized task artifacts, metrics, traces, dry-run records, dataset export, and benchmark hooks.	src/rosclaw/practice, src/rosclaw/data
KNOW/HOW/AUTO/Darwin	Knowledge lookup, heuristic recovery, self-evolution control plane, and multi-seed evaluation pressure.	src/rosclaw/know, src/rosclaw/how, src/rosclaw/auto, src/rosclaw/darwin

Main Path: Natural Language To ROS1 Mobility

The active application path is CMU Mobility. ROSClaw parses a constrained mobility command, checks it against a safety policy, writes an artifact record, and either proposes the action in dry-run mode or publishes high-level ROS topics into the CMU ARE stack.

flowchart LR
    user[Human or agent] --> iface[CLI / MCP / Feishu]
    iface --> parse[Deterministic parser + optional LLM fallback]
    parse --> safety[CMU safety policy]
    safety --> bridge[CMU ARE bridge]
    bridge --> ros[/ROS1 topics/]
    ros --> sim[vehicle_simulator + local_planner + ARiADNE2]
    sim --> traces[odom / path / camera / map / point cloud]
    traces --> artifacts[episode artifacts + dashboard + eval]

Loading

ROSClaw deliberately publishes high-level control topics instead of taking over raw velocity control. Raw direct cmd_vel requests are blocked by the CMU safety layer.

Direction	Topics
Publish	/way_point, /speed, /stop, /rosclaw/exploration_control
Observe/cache	/state_estimation, /cmd_vel, /path, /camera/image, /grid_map, /registered_scan, /terrain_map
Readiness checks	pose, registered scan, and terrain map before live navigation by default

Try It Without ROS

Most inspection, planning, safety, dry-run, and artifact features work without ROS1. Install the Python package, then use the repository-local command examples below. If you installed the console script globally, replace ./rosclaw with rosclaw.

python3 -m pip install -e ".[dev]"

./rosclaw --help
./rosclaw cmu --help
./rosclaw cmu check --json
./rosclaw cmu go inspection_a --dry-run --json --no-llm
./rosclaw cmu explore start --dry-run --json
./rosclaw cmu observe --dry-run --json
./rosclaw cmu visual-nav red_door --dry-run --json
./rosclaw cmu eval --output-dir practice_data/app_runs --limit 5 --json

Useful offline commands beyond the CMU app:

./rosclaw robot list
./rosclaw robot inspect mock_mobile_base
./rosclaw robot validate mock_mobile_base
./rosclaw mission plan inspection_a --json
./rosclaw mission graph-plan inspection_a --json
./rosclaw mission run inspection_a --dry-run --json
./rosclaw mission bt export inspection_a --json
./rosclaw worldmodel query inspection --json
./rosclaw provider list
./rosclaw runtime backends
./rosclaw sandbox list-worlds
./rosclaw know search velocity --robot-id mock_mobile_base
./rosclaw auto status
./rosclaw practice list
./rosclaw memory status
./rosclaw memory search dry --json
./rosclaw run submit --action='{"instruction":"dry"}' --json
./rosclaw context build --instruction dry --json

Mission OS: Recoverable Agent Task Graphs

The Mission OS is the Agent/Mission orchestration layer for mobile tasks. It does not replace Nav2 or BehaviorTree.CPP inside ROS; instead it gives the Python agent a safe, auditable task graph before any command reaches the mock or CMU navigator.

Natural-language mission planning still returns a linear MissionSpec for compatibility. The graph path now produces a MissionGraph with a root sequence node and typed action children by default. Imported or hand-authored graphs can also use:

Node type	Behavior
action	Calls the existing mission step executor and navigator adapters.
sequence	Runs children in order and stops at the first failure.
fallback	Tries children until one succeeds, then marks the rest skipped.
recovery	Runs a primary child, executes recovery children after failure, and retries the primary with a capped attempt count.

Every graph is validated before execution. The validator rejects unknown nodes, duplicate IDs, invalid child references, structural/dependency cycles, excessive depth or size, retry counts above the configured cap, and action kinds outside the existing MissionStepKind allowlist.

The current BehaviorTree.CPP integration is deliberately interoperability-only:

• MissionGraph -> BTCPP XML export is for review, datasets, and alignment with tools such as BTGenBot.
• BTCPP XML -> MissionGraph import only accepts an allowlisted subset: Sequence, Fallback, RecoveryNode, and high-level ROSClaw mission actions.
• Imported XML is validated and can be dry-run; XML is not used as a live robot execution kernel.

CLI support includes mission graph-plan, mission graph-validate, mission graph-run, and mission bt export/import. The examples above are file-free commands that run offline; validation and import commands accept JSON/XML paths or - for stdin.

Enterprise Agent OS Slice

ROSClaw now has a single governed action entrypoint for CLI, API, and MCP paths. External callers submit an ActionEnvelope v1; the runtime creates or resumes an AgentRun, checkpoints a RunGraph node, then enforces:

flowchart LR
    action[ActionEnvelope v1] --> auth[AuthContext / RBAC]
    auth --> kill[KillSwitch global / robot / run / action]
    kill --> policy[PolicyEngine + CMU safety evidence]
    policy --> approval{Approval required?}
    approval -- yes --> interrupt[Durable RunGraph interrupt]
    approval -- no --> execute[Tool / provider / robot handler]
    execute --> context[Context snapshot + memory/artifact refs]
    context --> audit[Append-only audit event]

Loading

Key public contracts are versioned under schemas:

Contract	Purpose
ActionEnvelope v1	Canonical action submission format across CLI, API, MCP, and future IM adapters.
ActionResult v1	Governed result with run/step IDs, policy decision, approval ID, artifact refs, and error state.
DecisionRecord	Unified enterprise policy/safety decision record.
LLMContextBundle v1	Debuggable context bundle with task, robot, policy, tools, memory, knowledge, artifact refs, redactions, and token budget.

Stable API routes are served by src/rosclaw/api/enterprise_server.py:

Route	Role
/api/v1/actions	Submit a governed action.
/api/v1/context/build	Build and optionally persist an LLM context snapshot.
/api/v1/runs, /api/v1/approvals, /api/v1/audit	Inspect run lifecycle, durable approval interrupts, and audit events.
/api/v1/policies, /api/v1/kill-switch, /api/v1/metrics	Policy evaluation, scoped emergency stops, and operational metrics.

Production authentication is token-based by default. Local development can explicitly opt out with ROSCLAW_AUTH_DISABLED=1.

Run With CMU ROS1 ARE

The live simulator path is packaged through Docker Compose. The main service builds a ROS Noetic image, prepares the catkin workspace from bundled simulator/planner packages, launches the selected CMU world, and can also start ARiADNE2, RViz, noVNC, camera/lidar/model spawns, and the artifact dashboard.

docker compose up --build rosclaw
docker compose --profile shell run --rm rosclaw-shell
docker compose --profile dashboard up cmu-dashboard

Inside the shell or a configured host environment:

./rosclaw cmu launch --world campus --headless --ariadne2
./rosclaw cmu dashboard --no-ros
./rosclaw cmu chat
./rosclaw cmu im --adapter feishu

Key environment knobs are read by Docker and the CLI:

Variable group	Examples	Purpose
CMU runtime	CMU_ARE_WORLD, CMU_OUTPUT_DIR, CMU_SPEED, CMU_NAV_TIMEOUT, CMU_NAV_TOLERANCE	World, artifact path, speed, timeout, and goal tolerance.
Visualization	HEADLESS, USE_RVIZ, RVIZ_PROFILE, CMU_GUI_BACKEND, CMU_NOVNC_PORT	Headless/Gazebo/RViz/noVNC behavior.
LLM and VLM	DEEPSEEK_API_KEY, OPENAI_API_KEY, OPENAI_VISION_API_KEY, OPENAI_VISION_MODEL	Optional parser/chat/vision providers. Deterministic parsing works without keys.
Remote IM	FEISHU_APP_ID, FEISHU_APP_SECRET, FEISHU_VERIFICATION_TOKEN	Feishu webhook bridge credentials.

CMU Mobility Features

Feature	Behavior
Place navigation	Named places and aliases come from docker/ros1/places.campus.yaml, including home, entrance_a, inspection_a, inspection_b, staging_area, tunnel_entrance, and upper_open_area_hint.
Relative motion	Chinese and English relative commands are parsed into map-frame waypoint requests.
Exploration	start, pause, resume, and stop publish exploration control for ARiADNE2.
Observation	Captures pose, camera image, grid map, trajectory, registered scan, terrain map, and a normalized observation record when ROS topics are available.
Visual navigation	Builds short-step target proposals from camera evidence and VLM decisions; high-risk visual motion requires confirmation by default.
Chat task manager	Handles multi-step tasks, progress messages, preemption, semantic grounding, visual snapshots/search, and cancellation.
Feishu bridge	Exposes the same task manager through a webhook adapter with allowlists, command prefixes, text replies, and image replies.
Dashboard	FastAPI UI for artifact search, pin/archive/trash, comparison, diagnostics, CSV/JSON export, trajectories, and image serving.

Evidence-First Artifacts

Every CMU command can write a standardized episode directory. Dry-run episodes contain the parsed intent and safety decision; live episodes add topic traces, observations, visual decisions, and measured navigation outcomes.

The artifact contract includes:

Record	Meaning
summary	Human-readable command result and source task summary.
episode	Normalized command, observation, safety, visual, metric, and metadata bundle.
metrics	success, timeout, duration, path length, final error, SPL-like score, intervention counts, visual metrics, and exploration duration.
commands	JSONL command timeline.
observations	JSONL snapshots and final poses.
safety decisions	JSONL safety checks, approval requirements, risk score, and policy version.
task events	Optional chat/exploration/visual-navigation progress timeline.

Robot Profiles

The local e-URDF zoo contains eight complete profile directories. Each directory is validated from required profile files for robot identity, safety, semantics, capabilities, and benchmarks, with optional URDF/MJCF/assets.

Directory	Robot ID	Robot	Vendor	DOF
crazyflie_2	crazyflie_2_1	Crazyflie 2.1	Bitcraze	6
fetch_robot	fetch_robot	Fetch	Fetch Robotics	8
franka_panda	franka_emika_panda	Panda	Franka Emika	7
g1	unitree_g1	G1	Unitree	23
mock_mobile_base	mock_mobile_base	Mock Mobile Base	ROSClaw Mock	2
skydio_x2	skydio_x2	Skydio X2	Skydio	6
unitree_go2	unitree_go2	Go2	Unitree	12
ur5e	universal_robots_ur5e	UR5e	Universal Robots	6

The mock mobile base is the offline-friendly profile used by many commands. It exposes navigation, PID move, and inspection capabilities with a low-risk bounding-box safety profile.

Agent And MCP Surface

The default MCP server is intentionally CMU-focused and safe for local agents. It exposes versioned tools such as:

Tool	Purpose
rosclaw.cmu.check.v1	Detect Docker/ROS1/CMU ARE availability.
rosclaw.cmu.go.v1	Parse and optionally execute a natural-language navigation request.
rosclaw.cmu.explore.v1	Dry-run or control exploration.
rosclaw.cmu.observe.v1	Capture or dry-run an observation artifact.
rosclaw.cmu.visual_nav.v1	Create a guarded visual navigation proposal.
rosclaw.cmu.eval.v1	Evaluate CMU episode artifacts.
rosclaw.run.list.v1, rosclaw.run.get.v1	Inspect durable AgentRuns.
rosclaw.approval.list.v1, rosclaw.approval.decide.v1	Manage pending approvals.
rosclaw.action.submit.v1	Submit a governed action through AgentActionRuntime.
rosclaw.context.build.v1	Build an LLMContextBundle with memory/source references.
rosclaw.memory.search.v1	Search memory with score, confidence, and redaction metadata.

The lower-level MCPHub also registers local integration tools for Mission OS, including rosclaw.mission.plan.v1, rosclaw.mission.run.v1, rosclaw.mission.graph.plan.v1, rosclaw.mission.graph.validate.v1, rosclaw.mission.bt.export.v1, and rosclaw.mission.bt.import.v1. The minimal stdio server keeps those legacy hub tools hidden by default; set ROSCLAW_MCP_INCLUDE_LEGACY_HUB_TOOLS=1 when a local agent needs that broader development surface.

mcp.json runs the stdio server from src/rosclaw/mcp/minimal_server.py. The HTTP facade in src/rosclaw/mcp/http_server.py provides health, capability, tool-list, resource-read, and tool-call endpoints.

Design Highlights

Design choice	Why it matters
Canonical event bus	Runtime modules communicate through normalized rosclaw.* topics with trace IDs, wildcard subscriptions, in-memory history, and dead-letter capture.
Guarded high-level control	CMU Mobility publishes waypoint/speed/stop/exploration topics and blocks raw velocity control requests.
Recoverable Mission graph	MissionGraph adds sequence, fallback, recovery, retry caps, skipped status, validation, dry-run, and BTCPP XML review without handing live control to generated XML.
Governed action runtime	CLI/API/MCP actions converge on Auth, KillSwitch, Policy, Approval, RunGraph, Audit, and artifact/memory recording.
Durable run spine	AgentRuns, steps, safety decisions, approvals, spans, RunGraph checkpoints, and artifacts are stored through file, SQLite, or composite stores.
Context engineering	ContextBuilder assembles task, robot, policy, tools, memory, knowledge graph, and artifacts into a versioned context snapshot.
Offline-first workflow	Dry-run CMU commands, mock mission planning, world model queries, robot inspection, providers, sandbox worlds, and KNOW/HOW checks run without ROS.
Physical DNA	e-URDF profiles keep robot embodiment, safety, semantics, capabilities, benchmark metadata, and simulation links together.
Evidence before learning	Practice artifacts become evaluation reports and dataset exports instead of disappearing into logs.
Recovery loop	KNOW supplies fast symptom matches; HOW turns failures into heuristic recovery plans; AUTO/Darwin add experiment and regression pressure.

Development

python3 -m pytest tests/test_readme_commands.py tests/test_docs_assets.py
python3 -m pytest tests/test_cmu_mobility_contract.py tests/test_cmu_are_bridge.py tests/test_mcp_minimal_server.py
python3 -m ruff check src tests
make check

The project targets Python 3.10 to 3.12. Core dependencies are declared in pyproject.toml, with optional developer tooling under the dev extra. make check runs the full pytest suite, ruff, the executable mypy baseline for platform-core modules, README command checks, and schema contract tests.

Current Boundaries

Boundary	Practical implication
Live CMU mobility needs ROS1	cmu go without --dry-run, live observation, chat execution, and exploration control require the Docker/ROS Noetic CMU ARE stack or equivalent ROS1 topics.
Providers are optional	Deterministic parsers and mock providers work offline; LLM/VLM behavior needs configured API keys and compatible OpenAI-style endpoints.
Some planes are research adapters	AUTO, Darwin, policy, benchmark, and sandbox commands expose useful contracts and mock paths, while hardware-grade execution depends on configured backends.
Mission behavior trees are agent-level	The Python MissionGraph executor orchestrates high-level mission actions. BehaviorTree.CPP XML is imported/exported for review and offline workflows, not executed as the live ROS behavior-tree runtime.
Safety is policy enforcement, not a physical guarantee	It constrains requests before publishing high-level topics. Simulator, planner, sensors, and environment still decide real-world behavior.

License

MIT. See LICENSE.