Skip to content

farm: model discovery surfaces expired 'free promotion' models (401) — add entitlement pre-check #93

Description

@SUaDtL

Found during the docs-site-mvp /ca:sprint --farm run (2026-06-18).

OpenCode Zen's /models catalog lists models the API key is not entitled to. minimax-m3-free and qwen3.6-plus-free are listed but /chat/completions returns HTTP 401 with "Free promotion has ended for ...". The canary cannot distinguish this from a model-quality failure and burns full attempts/timeouts on dead candidates (one earlier 7-model canary ran for minutes before being killed).

Workaround used: a fast direct screen (one tiny /chat/completions per candidate, 35s timeout) to drop 401 models before the real canary; big-pickle survived and passed.

Perma-fix candidates:

  • Before the canary, do a cheap 1-token /chat/completions entitlement probe per candidate; drop any that return 401 with a clear "promotion ended / not entitled" note.
  • Surface 401 distinctly from capability failure in canary-report.json.
  • Cap per-candidate canary wall-clock so one slow/dead model can't dominate the probe.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestfeature-forgePreview-feature feedback (Feature Forge)

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions