Python: DevUI Agent Discovery Issue: External Tool Imports Cause "No valid entity found" Error

[hellices]

Environment

• Agent Framework Version: agent-framework (pre-release)
• Python Version: 3.12
• OS: Linux
• DevUI Command: uv run devui ./agents --port 8080

Problem Description

When creating agents with tools imported from external modules (outside of agent.py), DevUI fails to discover the agent and returns a 500 Internal Server Error with the message:

[ERROR] Error getting entity info for webscraper_agent: No valid entity found in /home/inhwanhwang/python/nl2sql-maf/backend/agents/webscraper_agent

Error Traceback

Traceback (most recent call last):
  File "/home/inhwanhwang/python/nl2sql-maf/backend/.venv/lib/python3.12/site-packages/agent_framework_devui/_executor.py", line 176, in execute_entity
    entity_obj = await self.entity_discovery.load_entity(entity_id)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/inhwanhwang/python/nl2sql-maf/backend/.venv/lib/python3.12/site-packages/agent_framework_devui/_discovery.py", line 106, in load_entity
    entity_obj = await self._load_directory_entity(entity_id, entity_info)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/inhwanhwang/python/nl2sql-maf/backend/.venv/lib/python3.12/site-packages/agent_framework_devui/_discovery.py", line 172, in _load_directory_entity
    raise ValueError(f"No valid entity found in {dir_path}")
ValueError: No valid entity found in /home/inhwanhwang/python/nl2sql-maf/backend/agents/webscraper_agent

Steps to Reproduce

❌ Failing Configuration (External Tool Import)

File Structure:

backend/
  agents/
    webscraper_agent/
      __init__.py        # from .agent import agent
      agent.py
  tools/
    progressive_data_sources.py  # Contains fetch_web_content

agents/webscraper_agent/agent.py:

from agent_framework.azure import AzureOpenAIResponsesClient
from azure.identity import AzureCliCredential
from tools.progressive_data_sources import fetch_web_content  # External import

def create_agent():
    client = AzureOpenAIResponsesClient(
        credential=AzureCliCredential(),
    )
    return client.create_agent(
        instructions="...",
        name="WebScraperAgent",
        tools=[fetch_web_content],  # ❌ Tool from external module
    )

agent = create_agent()

Result:

INFO: 127.0.0.1:35446 - "GET /v1/entities/webscraper_agent/info HTTP/1.1" 500 Internal Server Error
[ERROR] Error getting entity info for webscraper_agent: No valid entity found

✅ Working Configuration (Tool Defined Inline)

agents/instruction_agent/agent.py:

from agent_framework.azure import AzureOpenAIResponsesClient
from azure.identity import AzureCliCredential

def create_agent():
    return AzureOpenAIResponsesClient(
        credential=AzureCliCredential(),
    ).create_agent(
        instructions="...",
        name="InstructionAgent",
        # No tools, or tools defined in same file
    )

agent = create_agent()

Result:

INFO: 127.0.0.1:35438 - "GET /v1/entities/instruction_agent/info HTTP/1.1" 200 OK

Verification

The agent can be imported successfully from Python:

$ cd backend
$ uv run python -c "from agents.webscraper_agent import agent; print('Agent:', agent.name)"
Agent: WebScraperAgent  # ✅ Works fine

But DevUI cannot discover it:

$ uv run devui ./agents --port 8080
# Accessing http://localhost:8080/v1/entities/webscraper_agent/info
# Returns 500 Error

Expected Behavior

DevUI should be able to discover and load agents regardless of where their tools are defined, as long as:

1. The agent module is importable
2. The agent has a module-level agent variable
3. The __init__.py properly exports the agent

Actual Behavior

DevUI's entity discovery fails when tools are imported from external modules, even though:

• The agent can be imported successfully via Python
• The agent structure follows the same pattern as working agents
• All required files (__init__.py, agent.py) are present

Workaround

Move all tool definitions into agent.py:

# agents/webscraper_agent/agent.py
from agent_framework.azure import AzureOpenAIResponsesClient
from azure.identity import AzureCliCredential
import asyncio
import requests
from typing import Annotated
from bs4 import BeautifulSoup
from markdownify import markdownify as md

# Tool defined in same file ✅
async def fetch_web_content(
    url: Annotated[str, "The URL of the website to fetch"],
) -> str:
    """Fetch static HTML content from a website."""
    # ... implementation ...
    pass

def create_agent():
    client = AzureOpenAIResponsesClient(
        credential=AzureCliCredential(),
    )
    return client.create_agent(
        instructions="...",
        name="WebScraperAgent",
        tools=[fetch_web_content],  # ✅ Tool in same file
    )

agent = create_agent()

This workaround works but has significant drawbacks:

• Code duplication when multiple agents need the same tool
• Poor maintainability when tools need to be updated
• Violates DRY principles and standard Python modular design
• Bloated agent files when agents use multiple complex tools

Impact

This limitation severely impacts:

1. Code organization - Cannot follow standard Python project structure
2. Code reusability - Cannot share tools across multiple agents
3. Maintainability - Must duplicate tool code for each agent
4. Testing - Cannot test tools independently from agents
5. Development experience - Unexpected behavior with no clear error message

Suggested Fix

DevUI's _discovery.py entity loading logic should:

1. Support agents with tools imported from external modules
2. Follow the same import resolution as Python's standard import system
3. Provide clearer error messages when entity discovery fails

Additional Context

• All agents follow identical structure
• Agents with no tools or inline tools work fine
• Agents with external tool imports fail consistently
• Python imports work correctly outside of DevUI
• No clear documentation about this limitation

Related Files

• agent_framework_devui/_discovery.py:172 - Where the error is raised
• agent_framework_devui/_executor.py:176 - Entity loading call
• agent_framework_devui/_server.py:282 - Error logging

---

Is this a known limitation or a bug? If it's intended behavior, it should be clearly documented. If it's a bug, it would greatly improve the development experience to fix it.

[victordibia]

Looking into this! Thanks for the issue @hellices

[victordibia]

Some updates here ..

• DevUI has been updated to return dependency error more clearly (rather than simply module not found)
• Readme updated as below

---

If your agents import tools or utilities from sibling directories (e.g., from tools.helpers import my_tool), you must set PYTHONPATH to include the parent directory:

# Project structure:
# backend/
# ├── agents/
# │   └── my_agent/
# │       └── agent.py    # contains: from tools.helpers import my_tool
# └── tools/
#     └── helpers.py

# Run from project root with PYTHONPATH
cd backend
PYTHONPATH=. devui ./agents --port 8080

Without PYTHONPATH, Python cannot find modules in sibling directories and DevUI will report an import error.

I have updated the readme for devui