RLMEnv: Make sub-LLM calls work for training

[snimu]

Description

Align sub‑LLM requests with interleaved rollouts; tighten local RLM worker shutdown

Context
Sub‑LLM requests were hitting the wrong endpoint when interleaved rollouts are enabled, causing vLLM 400s and token accounting issues. Also, local code execution left child processes alive on teardown.

What changed

• Sub‑LLM calls now follow the same interleaved path as the main model: tokenize + /chat/completions/tokens when interleaved_rollouts=True, otherwise standard chat completions.
• Sub‑LLM sampling args are normalized for token‑prompt mode to match main‑model behavior.
• Local RLM worker is launched in its own process group and terminated by group on teardown to avoid orphaned children.

Type of Change

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• Documentation update
• Test improvement

Testing

• All existing tests pass when running uv run pytest locally.
• New tests have been added to cover the changes

Checklist

• My code follows the style guidelines of this project as outlined in AGENTS.md
• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• Any dependent changes have been merged and published

Additional Notes

---

Note

Aligns sub‑LLM requests with main-model interleaved flow and improves local worker robustness.

• Sub‑LLM API: when interleaved_rollouts=True, tokenize via tokenize_vllm and POST to /chat/completions/tokens with normalized sampling args; otherwise use chat.completions.create. Removes logprobs fallback logic and adds message content normalization.
• Execution/local: launch local worker with start_new_session=True and terminate via os.killpg(.., SIGTERM/SIGKILL) to avoid orphaned children.
• Interception/typing: pass state into sub‑LLM paths, use ChatMessages types, and record sub‑LLM turns/metrics in trajectory steps.
• Tests: add coverage for interleaved request path, env var export, new-session start, and process-group kill; adjust fixtures to avoid SIGTERM handler registration.

^{Written by Cursor Bugbot for commit abd8127. This will update automatically on new commits. Configure here.}

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{Bugbot Autofix is OFF. To automatically fix reported issues with Cloud Agents, enable Autofix in the Cursor dashboard.}

[cursor]

Cursor Bugbot has reviewed your changes and found 2 potential issues.

^{Bugbot Autofix is OFF. To automatically fix reported issues with Cloud Agents, enable Autofix in the Cursor dashboard.}