Metamorphic Testing for LLM

[AmileshN]

Purpose

Ensure CityAgent produces consistent results under semantic-preserving input transformations.

Tools

• pytest
• Parameterized tests(come up with this)

Tasks

• Implement metamorphic tests for:
  • Case-insensitivity
  • Order-independence
  • Acronyms (e.g., Dr vs Drive)
  • Synonyms
• Compare structured outputs and citations, not exact wording

Definition of Done

• Tests exist under tests/metamorphic/
• Each invariant has at least one test case
• Assertions validate semantic equivalence
• Tests pass