RLM: tools, sub_tools, root_tools

[snimu]

Description

Adds explicit tool splitting for RLMEnv and enables root‑REPL tools, with fixed tools and shared infrastructure.

New tool parameters

• tools: shared by root and sub‑LLMs
• root_tools: root‑REPL only
• sub_tools: sub‑LLM only

Shared tools are appended to both root/sub lists, preserving order. Tool ordering is:

1. fixed tools (e.g. llm_batch)
2. tools
3. role‑specific tools (root_tools / sub_tools)

Each list is de‑duplicated by name; collisions across different callables raise a clear error. Fixed tools are reserved and cannot be overridden.

Root tools

• Root tools run only inside the REPL (Python functions), not via OpenAI tool calls.
• Sandbox execution routes root tool calls through a new host RPC endpoint; args/results are serialized.
• Serialization reuses the existing SerializerRegistry / data_serializers (no separate serializer set).
• Local execution bypasses serializers and uses pickle.

Fixed tool

• llm_batch is now a fixed, non‑overridable root tool implemented on the host (not in the worker script).

Type of Change

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• Documentation update
• Test improvement

Testing

• All existing tests pass when running uv run pytest locally.
• New tests have been added to cover the changes

Checklist

• My code follows the style guidelines of this project as outlined in AGENTS.md
• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• Any dependent changes have been merged and published

Additional Notes

---

Note

Adds root/shared/sub tool model to RLMEnv with host-implemented fixed root tool llm_batch, RPC plumbing, and serialization.

• Introduces tools (shared), root_tools (REPL-only), sub_tools (sub-LLM) with deterministic ordering, per-list dedupe, and reserved name checks; exposes only sub tools as OpenAI tools
• Implements host-side root tool execution via new interception route /rollout/{id}/v1/rlm/tools; worker proxies calls and (sandbox) serializes args/results using the existing SerializerRegistry (local uses pickle)
• Adds root tool docs (Root REPL Tools) and updates sub tool docs (Sub-LLM Tools) in system prompt; stores rlm_shared_tools, rlm_root_tools, rlm_sub_tools in state
• Tracks root tool metrics (total and per-tool) in RLMMonitorRubric
• Refactors/extends worker script and env startup to pass root tool names/serializers; factors helper utilities for tool merging/deduping
• Extends data serialization utils to carry serializer source/function; build_custom_serializer returns specs for sandbox use
• Updates docs (environments and experimental README) and adds comprehensive tests for tool split, ordering, serialization, RPC, and prompts

^{Written by Cursor Bugbot for commit db4f88a. This will update automatically on new commits. Configure here.}

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{Bugbot Autofix is OFF. To automatically fix reported issues with Cloud Agents, enable Autofix in the Cursor dashboard.}