Customized OpenCompass API

This repository provides an example of how to use a customized API model within OpenCompass.

Setup Instructions

1. Configure API Settings Set your API_BASE and KEY in the following files:

   • my_api.py
   • ./mymode/default.py

2. Integrate Your API Model

   • Copy my_api.py to <your_package_path>/opencompass/models.

   • Add the following import statement to <your_package_path>/opencompass/models/__init__.py

     from .my_api import MyModelAPI

3. Copy Configuration Files Copy the ./mymode directory to <your_package_path>/opencompass/configs/models.

4. Run Your Process Execute your process with a command like:

   opencompass example_gsm8k.py

References

• OpenCompass GitHub Repository: OpenCompass is an LLM evaluation platform supporting a wide range of models (Llama3, Mistral, InternLM2, GPT-4, LLaMa2, Qwen, GLM, Claude, etc.) across 100+ datasets.
• OpenCompass Documentation: Comprehensive documentation for OpenCompass 0.5.0.