fix(search): add database_name to search results and metadata to prevent LLM hallucination

[rafmacalaba]

Summary

LLMs were hallucinating human-readable names for opaque database_id codes
returned by search and data tools. For example, WB_GS was being presented
to users as "World Bank – Global Statistics" when the correct name is
"Gender Statistics". This happened because no grounded label was ever
surfaced in tool responses — the LLM was expanding the abbreviation by guessing.

This patch makes the correct database name available in every tool response
that returns a database_id, so the LLM always has the authoritative label
and never needs to infer it.

Root Cause

The DATABASES registry (id → name mapping) already existed in
mcp_server/resources.py as a static MCP resource (data360://databases),
but it was only readable if the client explicitly fetched that resource.
Tool responses (search_indicators, get_metadata, get_data) never
included the corresponding label, leaving the LLM with no grounded value
to cite.

Changes

src/data360/constants.py [NEW]

• Dependency-free module holding the DATABASES registry and a pre-built
  DB_NAME_LOOKUP dict (database_id → name).
• Single source of truth — both api.py and mcp_server/resources.py now
  import from here instead of duplicating the list.

src/data360/models.py

• EnrichedIndicator: added database_name: str | None field with a docstring
  that explicitly instructs the LLM to use this value when presenting data and
  never to guess from database_id.

src/data360/api.py

• _enrich_search_results(): populates database_name via _DB_NAME_LOOKUP
  for every indicator returned by data360_search_indicators.
• get_metadata(): injects database_name into indicator_metadata (which
  is also surfaced through get_data's metadata field). The key is always
  retained even when select_fields restricts other fields.

src/data360/mcp_server/resources.py

• Removed duplicate DATABASES definition; now imports from constants.py.

tests/test_api.py

• TestDatabaseNameInSearch: three tests covering known id resolution
  (WB_GS → "Gender Statistics"), unknown id → None (not a guessed
  string), and a coverage sweep across all registered databases.
• TestDatabaseNameInMetadata: three tests covering get_metadata injection,
  survival through select_fields filtering, and unknown id → None.

Testing

pytest tests/test_api.py -q   # 35 passed
pytest tests/ -q --ignore=tests/test_pr41_compliance.py  # 242 passed

Checklist

• The correct fix is applied (grounded label in response, not prompt-only)
• Single source of truth for the database registry (constants.py)
• database_name is None for unknown ids — no invented fallback
• database_name survives select_fields filtering in get_metadata
• All existing tests pass
• New tests added for both the search and metadata paths
• Pre-commit hooks pass

[Copilot]

Pull request overview

This PR aims to reduce LLM hallucination by surfacing authoritative, human-readable database labels (database_name) alongside database_id in tool responses, and centralizing the database registry so the mapping is consistent across the API and MCP server.

Changes:

• Added a shared DATABASES registry + DB_NAME_LOOKUP, and used it to populate database_name in search results and metadata responses.
• Extended the API/models to support richer search workflows (multi-query / query groups) and added a new data360_analyze_development_topic MCP tool.
• Added/updated tests to cover database name injection and the new behaviors.

Reviewed changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated 11 comments.

Show a summary per file

File	Description
src/data360/constants.py	New centralized database registry and lookup map.
src/data360/models.py	Adds database_name, multi-query models, and typing updates.
src/data360/api.py	Populates database_name; adds multi-query search flow; adds topic analysis tool; adjusts get_data behavior/logging.
src/data360/mcp_server/resources.py	Imports centralized DATABASES instead of duplicating.
src/data360/mcp_server/tools.py	Exposes data360_analyze_development_topic as an MCP tool.
src/data360/mcp_server/_server_definition.py	Wires optional OpenAI sampling handler for server-side sampling fallback.
tests/test_api.py	Adds tests for database_name in search/metadata plus resilience behaviors.
tests/test_multi_query_search.py	New comprehensive tests for multi-query search behavior.
tests/test_analyze_topic.py	New tests for analyze_development_topic.
docs/overview.md	Documents the new analyze tool.
README.md	Documents optional OpenAI API key for sampling fallback.

Comments suppressed due to low confidence (1)

src/data360/api.py:1267

• The smart time default logic still sets start_year = current_year - 19 (last 20 years) even though the docstring and surrounding comments now state the default is the last 5 years. Update the calculation to match the documented behavior (ideally reuse a single constant, e.g. _DEFAULT_SUMMARY_YEARS).

    # Smart time defaults: if no time range specified, default to last 5 years
    if start_year is None and end_year is None:
        from datetime import datetime  # noqa: PLC0415

        current_year = datetime.now().year
        end_year = current_year
        start_year = current_year - 19  # Last 20 years
        _logger.info(f"Smart default: Applied time range {start_year}-{end_year}")

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[rafmacalaba]

@avsolatorio add patch fix for dataset/database name

[rafmacalaba]

Thanks @avsolatorio! I have replaced the hardcoded partial list with a hybrid DatabaseManager that relies on a complete, script-generated 161-dataset JSON file fallback (databases.json) to eliminate any 'onset empty' edge cases if the API is dropped. It updates cleanly using a dynamically fetched TTL payload, completely removing code-level dependency on static constants.

[avsolatorio]

lgtm! Thanks!