Skip to content

feat(benchmarks): add MiniMax provider arm (MiniMax-M3, MiniMax-M2.7) - #656

Open
octo-patch wants to merge 1 commit into
DietrichGebert:mainfrom
octo-patch:octo/20260731-provider-add-recvqf0qrY9sUo
Open

feat(benchmarks): add MiniMax provider arm (MiniMax-M3, MiniMax-M2.7)#656
octo-patch wants to merge 1 commit into
DietrichGebert:mainfrom
octo-patch:octo/20260731-provider-add-recvqf0qrY9sUo

Conversation

@octo-patch

Copy link
Copy Markdown

Reason: provider-add — add MiniMax benchmark coverage (MiniMax-M3, MiniMax-M2.7) via the OpenAI-compatible endpoint without changing the skill's prompt arms.

Changes

  • Add benchmarks/promptfooconfig.minimax.yaml: a new promptfoo eval config that runs the existing baseline vs ponytail arms against MiniMax-M3 and MiniMax-M2.7 over the MiniMax OpenAI-compatible endpoint (https://api.minimax.io/v1). It reuses the repo's existing arms (arms/baseline.js, arms/ponytail.js) and the existing LOC + correctness gates (loc.js, correctness.js) on the same five everyday tasks as the other provider configs.
  • Add MINIMAX_API_KEY to .env.example so the key is documented alongside the existing ANTHROPIC_API_KEY placeholder.
  • Document the MiniMax run in benchmarks/README.md (a ### MiniMax reproduce section matching the Claude/Gemini/OpenAI sections).

The provider is wired as openai:chat:<model> with apiBaseUrl and apiKeyEnvar, so the API key stays in .env and no provider-specific promptfoo plugin is needed. No skill or prompt arm is touched; the new config is additive and mirrors the existing per-provider configs.

Checks run

  • node --test tests/*.test.js — 82 passed, 0 failed.
  • node --test benchmarks/correctness.test.js benchmarks/loc.test.js — passed.
  • YAML parsed and validated with PyYAML (providers + tests shape correct).
  • Secret scan on the added diff: clean.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant