Goal: Repeat the ICD-11 E2E evaluation workflow on a real-world country-level dataset (rather than the curated sample dataset used in March), testing the pipeline at greater scale and linguistic/clinical diversity. Results inform the WHO ICD-11 connectathon preparation in May.

Demo Driver: Joe

Steps:

1. Obtain and prepare the country-level input dataset (e.g. a national concept dictionary or local mapping dataset from a partner — MSF, Arogya Sri Lanka, or similar)
2. Pre-process dataset as CSV for OCL Mapper input
3. Start a new project in OCL Mapper:
   • Target repo: WHO / ICD-11-WHO
   • Algorithms: CIEL Bridge, WHO ICD-11 automatch, LLM-as-terminologist
   • Reranker + AI Assistant enabled
4. Run Auto-match across all algorithms
5. Export results and evaluate:
   • Re-ranked score distributions
   • AI Assistant recommendation quality
   • Cases where algorithms agree/disagree
   • Human expert spot-check on high-confidence and low-confidence cases
6. Document findings in an evaluation summary
7. Identify issues to resolve before the May connectathon