Skip to content

Add vector env benchmarking suite - #197

Open
ParamThakkar123 wants to merge 1 commit into
mainfrom
codex/create-benchmarking-scripts-for-speedup
Open

ParamThakkar123 wants to merge 1 commit into
mainfrom
codex/create-benchmarking-scripts-for-speedup

Conversation

@ParamThakkar123

Copy link
Copy Markdown
Owner

Motivation

  • Provide a lightweight, dependency-light way to measure throughput and speedup of TorchVectorizedEnv versus a single-threaded loop and to collect cProfile bottleneck data.
  • Make it easy to run quick synthetic benchmarks for CI and to profile framework overhead before using real environments.

Description

  • Add a CLI benchmarking script scripts/benchmark_vector_env_speed.py that implements a synthetic image environment, single-threaded and vectorized benchmarks, and optional cProfile reporting (binary .prof and top cumulative text output).
  • Add a smoke test world_models/benchmarks/tests/test_vector_env_speed.py that validates the synthetic benchmark functions and is guarded with pytest.importorskip("torch") to avoid failures where PyTorch is unavailable.
  • Document usage and profiling examples in docs/source/benchmarks.md and world_models/benchmarks/README.md so the benchmark is discoverable and reproducible.

Testing

  • Run python -m py_compile scripts/benchmark_vector_env_speed.py world_models/benchmarks/tests/test_vector_env_speed.py which succeeded to validate basic syntax and typing.
  • Run python -m pytest world_models/benchmarks/tests/test_vector_env_speed.py which executed and skipped the tests in this environment because torch was not installed, as intended by the importorskip guard.
  • Verified the new script is importable and the benchmark helpers (benchmark_single_threaded, benchmark_vectorized, run_benchmark) produce ThroughputResult/BenchmarkReport shapes in local smoke runs when torch is available.

Codex Task

@github-actions github-actions Bot added the documentation Improvements or additions to documentation label Jun 16, 2026
@github-actions

Copy link
Copy Markdown
Contributor
PR Preview Action v1.8.1

QR code for preview link

🚀 View preview at
https://ParamThakkar123.github.io/torchwm/pr-preview/pr-197/

Built to branch gh-pages at 2026-06-16 12:20 UTC.
Preview will be ready when the GitHub Pages deployment is complete.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

codex documentation Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant