Collect human preference datasets for less-resourced languages and specific sectors,
while raising awareness about model diversity, bias, and environmental impact.
Built by the French government, now growing into new languages and sectors.
🇫🇷 French platform · 🇩🇰 Danish platform
|
Launched in October 2024 by DINUM and the French Ministry of Culture to address the lack of French-language preference data for LLM training nd evaluation. Since launch: 700,000+ prompts, 250,000+ preference votes, 450,000+ visitors. One of the largest non-English human preference datasets available. All data published openly on Hugging Face:comparIA/comparia-fr-arena We published a pre-print to dive deep into the project's strategy in France. |
Compar:IA on the France 2 evening news, used in the classroom to teach students about AI models, bias, and environmental impact. |
|
Most LLMs underperform outside English. Compar:IA collects the preference data needed to close this gap. Already live in French and Danish, and planning launches in Sweden, Estonia and Lithuania. |
Generic benchmarks miss domain-specific needs. A sector arena reveals which models handle specialised language best. Healthcare, legal, education, public admin, agriculture... |
Run your own arena, evaluate models on your real-world tasks, and contribute data back to the commons. Governments, universities, hospitals, companies, NGOs... |
|
Teach citizens and professionals about model diversity, bias, and environmental cost. Already used in schools and training sessions. 
|
Produce instruction and preference data in less-ressourced languages. 
|
Data feeds into new model training, leaderboards, use case mappings, and other research topics. 
|
The platform is fully open source, self-hostable, and customizable: choose your models, translate the interface, adapt prompt suggestions, add your logo. We can host it for you or help you set it up yourself.
Whatever your situation, reach out first and we'll figure out the best path together.
📬 contact@comparia.beta.gouv.fr
Compar:IA is a digital common. Whether you can offer funding, code, translations, or simply ideas, there is a place for you.
💰 Financially. Compar:IA has been funded by DINUM and the French Ministry of Culture, with European support from ALT-EDIC. We are actively looking for new partners and funders to sustain the infrastructure, expand to new languages, and keep the project independent. contact@comparia.beta.gouv.fr
💻 In code. The entire platform is open source and we welcome contributions of all sizes: bug fixes, new features, translations, documentation. Come build with us. GitHub repository
💬 In discussions. Share your ideas, flag issues, or just ask questions on GitHub Discussions. We want to hear from you. GitHub Discussions
Any other way. Partnerships, academic collaborations, media coverage, spreading the word: every contribution matters. Reach out and let's talk. Contact us
- Authentication (🇪🇺 ALT-EDIC, 🇫🇷 DINUM)
- Style control #532 (🇫🇷 Ministry of Culture)
- Prompt moderation #542 (🇫🇷 Ministry of Culture)
- Improved model cards (🇫🇷 Ministry of Culture)
- Live use-case mapping (🇪🇺 ALT-EDIC, 🇫🇷 DINUM)
- Message history (🇪🇺 ALT-EDIC, 🇫🇷 DINUM)
- Socio-demographic data collection (🇪🇺 ALT-EDIC, 🇫🇷 DINUM)
- Back-office management (🇪🇺 ALT-EDIC, 🇫🇷 DINUM)
- New voting system (🇪🇺 ALT-EDIC, 🇫🇷 Ministry of Culture)
- Web search (🇪🇺 ALT-EDIC)
- Separation of all platforms into separate instances (🇪🇺 ALT-EDIC, 🇫🇷 DINUM)
- Ranking consolidation and internationalization (🇪🇺 ALT-EDIC, 🇫🇷 DINUM)
- Language/platform-specific model support (🇪🇺 ALT-EDIC, 🇫🇷 DINUM)
- Gradio → FastAPI migration (🇫🇷 Ministry of Culture, 🇫🇷 DINUM, 🇪🇺 ALT-EDIC)
- EcoLogits update (🇪🇺 ALT-EDIC, 🇫🇷 DINUM)
- Dataset publishing pipeline v1 (🇫🇷 DINUM, 🇫🇷 Ministry of Culture)
- Leaderboard v1 (🇫🇷 DINUM, 🇫🇷 Ministry of Culture, in collaboration with 🇫🇷 PEReN)
- Archived models (🇫🇷 DINUM, 🇫🇷 Ministry of Culture)
- Blog section (🇫🇷 DINUM, 🇫🇷 Ministry of Culture)
- Internationalization foundations (🇫🇷 DINUM, 🇫🇷 Ministry of Culture)
- compar:IA v1 (🇫🇷 DINUM, 🇫🇷 Ministry of Culture)
👉 Full technical roadmap on GitHub
The platform is fully open source and self-hostable.
Self-host with Docker (single server, automatic HTTPS via Caddy): see DOCKER_INSTALL.md
Local development (basic):
cp .env.example .env # Configure environment variables
make install # Install all dependencies
source .env
make dev-backend # Backend on http://localhost:8008
make dev-frontend # Frontend on http://localhost:5173For the full setup guide (instances, KeePass, Docker, testing, models, i18n, architecture), see CONTRIBUTING.md.
