Use threading for sandbox client

[mikasenghaas]

Description

Implements threading utils that allow creating per-thread storage (e.g. use different clients for each thread in a thread pool executor), and a wrapper class ThreadedAsyncSandboxClient which delegates all calls via sandbox clients to a thread pool, each with their local client. This drastically lowers the error rate when training in PRIME-RL because it unblocks the main orchestrator event loop. Basically makes the difference between being trainable or not.

Type of Change

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• Documentation update
• Test improvement

Testing

• All existing tests pass when running uv run pytest locally.
• New tests have been added to cover the changes

Checklist

• My code follows the style guidelines of this project as outlined in AGENTS.md
• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• Any dependent changes have been merged and published

Additional Notes

---

Note

Introduce a ThreadPool-backed ThreadedAsyncSandboxClient with per-thread event loops/clients and replace AsyncSandboxClient usage in SandboxEnv, adding configurable limits and teardown.

• Env (verifiers/envs/sandbox_env.py):
  • Replace AsyncSandboxClient with new ThreadedAsyncSandboxClient using ThreadPoolExecutor and per-thread client/event loop.
  • Add configurable client params: sandbox_client_max_workers, sandbox_client_max_connections, sandbox_client_max_keepalive_connections.
  • Add @vf.teardown method teardown_sandbox_client to shut down the executor; keep sync bulk-delete teardown for remaining sandboxes.
• Utils (verifiers/utils/thread_utils.py):
  • New thread-local helpers: get_or_create_thread_attr and get_or_create_thread_loop for per-thread resource management.

^{Written by Cursor Bugbot for commit 32e50c3. This will update automatically on new commits. Configure here.}

I was wrong

[cursor]

Bug: Thread-local storage key collision ignores different client configurations

The hardcoded key "sandbox_client" in get_or_create_thread_attr causes all ThreadedAsyncSandboxClient instances to share the same thread-local AsyncSandboxClient. When multiple instances are created with different client_kwargs (e.g., different max_connections values), the first instance to make a call on a thread creates the client with its config, and subsequent instances silently reuse that same client, ignoring their own configurations. The factory kwargs are only used on creation, not on retrieval.

Additional Locations (1)

• verifiers/utils/thread_utils.py#L17-L21