[BREAKING][misc] feat: reorganize Local/Remote rollout docs and switch dataset support to parquet/jsonl/csv

[JoyboyBrian]

What

• Reorganized documentation into a clearer IA:
  • split rollout docs into docs/local-rollout/* and docs/remote-rollout/*
  • moved shared topics to top-level docs (docs/datasets.md, docs/test-mode.md, docs/eval-mode.md)
  • added new reference pages (docs/configuration.md, docs/troubleshooting.md)
• Rewrote README.md for a two-mode onboarding flow (Local Rollout vs Remote Rollout), with badges/logo and links to dedicated example repos.
• Updated supporting docs and examples (docs/cli.md, examples/README.md, SECURITY.md) for consistency and clarity.
• Updated dataset handling in eval/test workflows:
  • added CSV support in DatasetReader
  • removed .json dataset support
  • standardized supported formats to .parquet (recommended), .jsonl, .csv
• Synced CLI help text for dataset flags in eval/test commands.
• Expanded dataset unit tests with CSV coverage and migrated JSON-based tests to JSONL.
• Added explicit Python classifiers (3.10, 3.11, 3.12, 3.13) in pyproject.toml.

Why

• The previous docs structure mixed concepts across local and remote workflows, making onboarding and navigation harder.
• Users need a mode-first path from install → setup → test/eval.
• CSV support lowers adoption friction for common tabular workflows, while standardizing formats simplifies validation and maintenance.
• Aligning code behavior, CLI help, and docs reduces confusion and support overhead.

Type of Change

• Bug fix (non-breaking change that fixes an issue)
• New feature (non-breaking change that adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)
• Refactor (code change that neither fixes a bug nor adds a feature)
• Documentation update
• CI/build configuration change

How to Test

• uv run pytest tests/unit/rollout/eval/common/test_dataset.py
• uv run pytest
• uv run ruff check .
• uv run ruff format --check .
• uv run pyright osmosis_ai/

Manual sanity checks:

• Verify .csv datasets work with osmosis test and osmosis eval.
• Verify .json datasets now fail with DatasetParseError.
• Verify key docs links from README.md and docs/README.md.

Checklist

• PR title follows [module] type: description format
• Appropriate labels added (e.g. enhancement, bug, breaking)
• ruff check . and ruff format --check . pass
• pyright osmosis_ai/ passes
• pytest passes (new tests added if applicable)
• Public API changes are documented
• No secrets or credentials included

---

Summary by cubic

Reorganized docs into Local vs Remote rollout guides with a simpler README and clearer onboarding. Standardized datasets to Parquet, JSONL, and CSV (dropped .json), updated CLI/help/tests, and improved agent loop docs; the base loop now emits structured tool messages.

• New Features

  • Split docs into local-rollout/* and remote-rollout/*; added CLI, configuration, datasets, test-mode, eval-mode, and troubleshooting; removed redundant sections.
  • Simplified README with logo/badges, two-mode onboarding, and links to example repos.
  • Standardized datasets: .parquet (recommended), .jsonl, .csv; strict CSV header validation with clear errors; updated CLI help; added CSV tests and migrated JSON tests to JSONL.
  • Clarified Remote Rollout agent loop docs and improved docstrings; base loop parses tool args and emits structured "tool" messages with latency tracking.
  • Added Python 3.10–3.13 classifiers; simplified CODEOWNERS.

• Migration

  • Replace any .json datasets with .jsonl, .parquet, or .csv.
  • Ensure CSV files include required headers and matching columns.
  • Update links to the new docs structure; verify with osmosis test and osmosis eval.

^{Written for commit 7123f53. Summary will update on new commits.}

[cubic-dev-ai]

3 issues found across 29 files

Prompt for AI agents (all issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="osmosis_ai/rollout/eval/common/dataset.py">

<violation number="1" location="osmosis_ai/rollout/eval/common/dataset.py:172">
P2: CSV rows with more columns than the header can produce a None key in DictReader, which will crash in _validate_row (k.lower()) with an AttributeError instead of a dataset parse/validation error. Guard against None keys or raise a DatasetParseError when extra columns are present.</violation>
</file>

<file name="docs/remote-rollout/overview.md">

<violation number="1" location="docs/remote-rollout/overview.md:62">
P3: The example now defines three methods (`get_tools`, `execute_tool`, `run`) but the surrounding text still says to implement two methods. Update the narrative to match the new example so readers aren’t misled.</violation>
</file>

<file name="docs/remote-rollout/deployment.md">

<violation number="1" location="docs/remote-rollout/deployment.md:66">
P2: The Compose healthcheck uses `curl`, but the provided Dockerfile examples are based on `python:3.11-slim` and do not install curl. In the documented setup this healthcheck will fail, so use a dependency-free check (e.g., Python) or note the need to install curl.</violation>
</file>

_{Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.}

[cubic-dev-ai]

2 issues found across 10 files (changes from recent commits).

Prompt for AI agents (all issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="docs/configuration.md">

<violation number="1" location="docs/configuration.md:45">
P3: This line implies the settings classes always auto-load environment variables and `.env` files, but the code only does that when `pydantic-settings` is installed; otherwise they fall back to `BaseModel` without env loading. Please qualify this statement so it matches actual behavior.</violation>
</file>

<file name="docs/troubleshooting.md">

<violation number="1" location="docs/troubleshooting.md:416">
P3: The new `--log` flag description conflicts with the unchanged `{debug_dir}` path example below, so the doc now references two different directory names. Update the path placeholder to match `--log` (e.g., `{log_dir}`) or adjust the wording so they’re consistent.</violation>
</file>

_{Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.}

[cubic-dev-ai]

3 issues found across 33 files

Prompt for AI agents (all issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="docs/test-mode.md">

<violation number="1" location="docs/test-mode.md:31">
P2: The programmatic example uses `async with`/`await` at top level, which is invalid Python outside an `async def`. Wrap this in an async function and call it with `asyncio.run(...)` so users can run the snippet.</violation>
</file>

<file name="docs/local-rollout/reward-functions.md">

<violation number="1" location="docs/local-rollout/reward-functions.md:7">
P2: This overstates the `@osmosis_reward` requirements. The decorator only enforces the classic signature when the first parameter is `solution_str`; other signatures are accepted without validation, so saying all functions must match this exact signature is misleading.</violation>

<violation number="2" location="docs/local-rollout/reward-functions.md:23">
P3: `**kwargs` is not required by `@osmosis_reward`; the decorator only drops `data_source` when the function can’t accept it. This line misleads users into adding `**kwargs` unnecessarily.</violation>
</file>

_{Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.}