Skip to content

Comments

Add MCPEnv to envs.experimental#655

Merged
willccbb merged 13 commits intomainfrom
will/mcp-to-experimental
Dec 22, 2025
Merged

Add MCPEnv to envs.experimental#655
willccbb merged 13 commits intomainfrom
will/mcp-to-experimental

Conversation

@willccbb
Copy link
Member

@willccbb willccbb commented Dec 22, 2025

Description

Type of Change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation update
  • Test improvement

Testing

  • All existing tests pass when running uv run pytest locally.
  • New tests have been added to cover the changes

Checklist

  • My code follows the style guidelines of this project as outlined in AGENTS.md
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • Any dependent changes have been merged and published

Additional Notes


Note

Add new mcp-search-env built on verifiers.envs.experimental.MCPEnv, remove old in-repo mcp_env, and update deps/tests accordingly.

  • Environments:
    • Add environments/mcp_search_env/ with mcp_search_env.py and README; uses verifiers.envs.experimental.MCPEnv and requires EXA_API_KEY.
    • Remove legacy in-repo MCP implementation (environments/mcp_env/ incl. mcp_env.py, mcp_server_connection.py, mcp_tool_wrapper.py, models.py) and associated output artifacts under outputs/evals/.
  • Dependencies:
    • Update root pyproject.toml deps (add math-verify, prime-sandboxes, pydantic, etc.) and include mcp in envs optional extras; add environments/mcp_search_env/pyproject.toml with verifiers>=0.1.8.post2 and mcp>=1.0.0.
  • Tests:
    • Adjust tests/test_envs.py to skip mcp_search_env (requires EXA_API_KEY) and remove skip for mcp_env.
  • Repo housekeeping:
    • Tweak .gitignore to ignore root __pycache__/.

Written by Cursor Bugbot for commit 6d7bc97. This will update automatically on new commits. Configure here.

@willccbb willccbb marked this pull request as ready for review December 22, 2025 05:23
@willccbb willccbb requested a review from cdreetz December 22, 2025 05:25
@willccbb willccbb changed the title Will/mcp to experimental Add MCPEnv to envs.experimental Dec 22, 2025
from datasets import Dataset

import verifiers as vf
from verifiers.envs.experimental.mcp_env import MCPEnv
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unused python-dotenv dependency breaks .env file loading

The old mcp_env.py called load_dotenv() to automatically load environment variables from .env files, but the new mcp_search_env.py does not import or use python-dotenv despite listing it as a dependency. Users who previously relied on .env files for EXA_API_KEY will now receive a RuntimeError even with a valid .env file present. Either the dependency should be removed if .env loading is not intended, or load_dotenv() should be called before checking os.environ.get("EXA_API_KEY").

Additional Locations (1)

Fix in Cursor Fix in Web

@willccbb willccbb merged commit a74d0d4 into main Dec 22, 2025
6 checks passed
@mikasenghaas mikasenghaas mentioned this pull request Dec 22, 2025
13 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant