feat: Add HTTP support for local MCP servers

[devin-ai-integration]

feat: Add HTTP support for local MCP servers

Summary

This PR adds support for http:// URLs in MCP server references to enable local MCP server connections during development. Previously, CrewAI only accepted https:// URLs for external MCP servers, which prevented developers from using local MCP servers running on http://localhost or other local HTTP endpoints.

Changes:

• Updated validate_mcps() in base_agent.py to accept http://, https://, and crewai-amp: URLs
• Updated _get_mcp_tools_from_string() in agent/core.py to route http:// URLs to _get_external_mcp_tools()
• Added warning log when http:// scheme is detected (security awareness for non-TLS connections)
• Updated field descriptions and docstrings to document http:// support for local development
• Added comprehensive test suite covering validation, routing, config support, fragment filtering, and warning logs

Example Usage:

agent = Agent(
    role="Data Collector",
    goal="Collect data using MCP tools",
    backstory="...",
    mcps=["http://localhost:7365/mcp#diff_general_info"]  # Now works!
)

Review & Testing Checklist for Human

• Run the new test suite - I couldn't run tests locally due to environment issues. Please run uv run pytest lib/crewai/tests/mcp/test_http_mcp_support.py -vv to verify all tests pass
• Test with a real local MCP server - Spin up a local MCP server on http://localhost:PORT/mcp and verify an agent can connect and retrieve tools
• Verify warning log appears - When using http:// URLs, confirm the warning message appears in logs mentioning "local development only" and recommending https:// for production
• Test backward compatibility - Verify existing https:// and crewai-amp: URLs still work correctly
• Check error messages - Try an invalid URL like ftp://example.com/mcp and verify the error message lists all three supported schemes (http://, https://, crewai-amp:)

Notes

• The underlying HTTPTransport and MCPToolWrapper already supported HTTP URLs without validation, so this PR primarily removes the artificial restriction in the validator and routing logic
• Security warning is logged at the WARNING level when http:// is detected to make developers aware of the non-TLS connection
• All tests use mocking to avoid network calls (required for CI's --block-network constraint)

Link to Devin run: https://app.devin.ai/sessions/28a42c2337524ee78df7d245e28e3ade
Requested by: João (joao@crewai.com)

Fixes #3876

[devin-ai-integration]

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

• Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
• Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

• Disable automatic comment and CI monitoring

[lorenzejay]

addressed here - #3850