add Client Pool

[hallerite]

Description

Type of Change

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• Documentation update
• Test improvement

Testing

• All existing tests pass when running uv run pytest locally.
• New tests have been added to cover the changes

Checklist

• My code follows the style guidelines of this project as outlined in AGENTS.md
• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• Any dependent changes have been merged and published

Additional Notes

---

Note

Medium Risk
Touches core evaluation execution, endpoint resolution, and worker request/timeout behavior; misconfiguration or resolution bugs could change which endpoints receive traffic or affect long-running eval stability.

Overview
Adds multi-endpoint “client pool” support by extending ClientConfig with endpoint_configs/client_idx and introducing resolve_client_config(s) to select and inherit per-endpoint settings (with cycle detection).

Updates eval execution to round-robin across multiple endpoints (and keep grouped-scoring rollouts for the same example on a single endpoint), while ensuring locally created clients are closed on success, resume, and partial setup failures; server/client worker paths now resolve endpoint configs and apply per-client request timeouts.

Migrates endpoint registries to a preferred TOML format (configs/endpoints.toml) while keeping .py support, changes Endpoints to map ids to lists of variants, adds endpoint_id support in eval TOML configs, removes support for api_base_url lists, and updates docs/config samples and the default --endpoints-path accordingly.

^{Written by Cursor Bugbot for commit e103f3c. This will update automatically on new commits. Configure here.}

[cursor]

Cursor Bugbot has reviewed your changes and found 3 potential issues.

^{Bugbot Autofix is OFF. To automatically fix reported issues with Cloud Agents, enable Autofix in the Cursor dashboard.}