Skip to content

Support configuring a custom auto-review model for approvals_reviewer = "auto_review" #1225

Closed
Closed
Support configuring a custom auto-review model for approvals_reviewer = "auto_review"#1225
Labels
bugSomething isn't workingcatalogModel catalog, slugs, visibility, routed entriesenhancementNew feature or request

Description

@yusoofsh
yusoofsh
opened on Aug 7, 2026

Area

  • Catalog / models

What are you trying to accomplish?

With approvals_reviewer = "auto_review" enabled, route the automatic approval-review subagent through a user-chosen model (for example opencode-go/deepseek-v4-flash) instead of whatever OpenCodex/Codex resolves by default.

What prevents this today?

Upstream Codex only exposes two reviewer modes in approvals_reviewer: user and auto_review (legacy alias guardian_subagent) — no model argument. The model the auto-review subagent uses is decided per-model via the auto_review_model_override field on the model catalog entry of the currently running model (see codex-rs/core/src/guardian/review.rs and codex-rs/protocol/src/openai_models.rs in openai/codex).

OpenCodex regenerates opencodex-catalog.json from provider discovery during catalog sync, and routed entries are rebuilt from templates. There is no OpenCodex-managed way to express "use model X for auto review", so the only current approach is hand-editing the generated catalog file. That edit is undocumented and can be clobbered by a later sync.

What should OpenCodex do?

Provide a first-class, durable configuration for the auto-review model that survives catalog regeneration. For example:

  • Accept a config option (e.g. in ~/.codex/config.toml or an OpenCodex-managed setting) such as auto_review_model = "opencode-go/deepseek-v4-flash".
  • Have the catalog sync apply that value to auto_review_model_override on the relevant routed entries instead of leaving them at null, so the setting is not lost when entries are rebuilt.
  • Document the option in the catalog / models docs.

Example usage or interface

approvals_reviewer = "auto_review"
auto_review_model = "opencode-go/deepseek-v4-flash"

Expected observable behavior: when an approval prompt needs review, the auto-review subagent runs against opencode-go/deepseek-v4-flash rather than the upstream default (codex-auto-review or gpt-5.6-luna), and the choice persists across opencodex catalog syncs.

Alternatives or workarounds

  • Manually add "auto_review_model_override": "<model-slug>" to the model's entry in ~/.codex/opencodex-catalog.json. This works today (verified), but the catalog is regenerated by OpenCodex and routed entries are rebuilt from templates, so the edit can be silently dropped.
  • Rely on the fallback: with no codex-auto-review entry in the OpenCodex catalog, upstream already falls back to the current turn model — but that is implicit and not controllable.

Additional context

  • Upstream mechanism: guardian_review_session_config resolves the review model from turn.model_info.auto_review_model_override, defaulting to approval_review_preferred_model() (codex-auto-review, or gpt-5.6-luna under API-key auth).
  • OpenCodex currently ships auto_review_model_override: null on every entry in src/codex/data/upstream-models.json and preserves unknown fields through ensureStrictCatalogFields, but offers no way to set the value.

Checks

  • I searched existing issues and documentation.
  • This request describes a concrete OpenCodex workflow rather than merely naming a desired technology.
  • I removed secrets and personal data.

Activity

  1. github-actions commented on Aug 7, 2026

    @github-actions
    github-actionsbot
    on Aug 7, 2026 – with GitHub Actions
    Contributor

    Automated translation bookkeeping — detected language: unknown.

  2. github-actions
    added
    enhancementNew feature or request
    catalogModel catalog, slugs, visibility, routed entries
    on Aug 7, 2026

  3. Ingwannu commented on Aug 10, 2026

    @Ingwannu
    Ingwannu
    on Aug 10, 2026
    Owner

    Confirmed as a real persistence gap. The upstream catalog schema does carry auto_review_model_override, while OpenCodex-generated entries currently leave it null; hand-editing the generated catalog is not a durable contract.

    A first-class setting is reasonable, with these constraints:

    • store the requested routed model in OpenCodex-owned configuration, not by editing the generated catalog or upstream template data;
    • validate the final routed slug against the catalog produced by the same sync and report a clear unresolved target instead of silently falling back;
    • apply it consistently to the entries whose turns may invoke auto-review, while preserving native upstream values unless the operator explicitly opts in;
    • avoid creating a hidden provider/auth dependency during catalog generation and preserve the normal behavior when the setting is absent;
    • cover sync regeneration, provider/model removal, stale target recovery, native entries, routed entries, and the actual emitted auto_review_model_override field.

    Keeping this open for a focused configuration and catalog-synthesis PR.

  4. Gliese-876 commented on Aug 13, 2026

    @Gliese-876
    Gliese-876
    on Aug 13, 2026

    同场景实测确认（v2.14.1 + 第三方主模型）：自动审核仍然失效，并且是 fail-closed —— 每个提权请求都被自动拒绝，返回 "Automatic approval review failed"，底层错误为 400 "This response_format type is unavailable now"（来自第三方上游）。

    实测链路（本地网关路由账本可复现）：

    1. codex-rs 派生的审核子代理请求携带的是主模型名（如 <provider>/deepseek-v4-pro），而不是 codex-auto-review，所以 fix(router): pin Codex auto review to OpenAI #833 的钉定分支不会命中。
    2. v2.14.1 默认仍向第三方上游透传 response_format，上游不支持结构化输出，直接 400；审核失败，提权被自动拒绝。
    3. 本地同步的模型目录中没有 codex-auto-review 条目（路由条目也没有 auto_review_model_override），审核模型因此回落到主模型——这正是本 issue 描述的根因。
    4. 对照：同一环境在旧版行为下审核可以正常通过；升级到 v2.14.1 后必现。当前 fix: add per-model structured output opt-out #1424 的 noStructuredOutputModels 需要手动配置，默认状态下问题依旧。

    期望：如本 issue 所述，为 approvals_reviewer = "auto_review" 提供可配置的审核模型；或让同步目录保留 codex-auto-review，使 #833 的钉定真正生效，把审核固定送回 OpenAI 官方通道。感谢！

  5. lidge-jun
    added
    bugSomething isn't working
    on Aug 18, 2026

  6. yzxcj797 commented on Aug 18, 2026

    @yzxcj797
    yzxcj797
    on Aug 18, 2026
    Contributor

    Implemented along your proposed shape:

    • auto_review_model is read as a root key of ~/.codex/config.toml (exactly the placement in your example, next to approvals_reviewer), using the existing readRootTomlString helper.
    • It's applied to every catalog entry as auto_review_model_override at the single point all entries pass through right before the catalog is serialized in writeRetainedCatalogSync — upstream resolves the review model from the current turn's entry, so stamping every entry makes the choice effective regardless of which model is running, and the value survives every regeneration/rebuild that today silently drops hand edits.
    • Absent key → nothing changes; entries keep whatever they carry (including upstream defaults).
    • Tests cover the TOML root-key resolution (including that table-scoped keys don't leak into root resolution) and the absent-key case.

  7. lidge-jun commented on Aug 19, 2026

    @lidge-jun
    lidge-jun
    on Aug 19, 2026
    Owner

    리뷰 · 우선순위 67 / 80

    현재 dev 기준, approvals_reviewer = "auto_review" 경로에 사용자가 고른 심사 모델을 고정하는 OpenCodex 설정이 없습니다. 카탈로그 엔트리의 auto_review_model_override를 검색해도 src/codex/catalog 및 동기화 경로에 해당 필드 적용 코드가 없고, readRootTomlString는 service_tier, model_provider, model_catalog_json, cli_auth_credentials_store만 읽고 있습니다. 따라서 이슈가 말하는 지속 가능한 설정 계약은 아직 코드에 없습니다.

    이슈가 원하는 것은 ~/.codex/config.toml에 auto_review_model = "opencode-go/deepseek-v4-flash" 같은 루트 키를 두고, 카탈로그 재생성 후에도 심사 서브에이전트가 그 모델로 가는 것입니다. 손편집으로 opencodex-catalog.json에 override를 넣는 우회는 검증됐지만, 라우트 엔트리가 템플릿에서 다시 만들어지면 null로 덮입니다. 그 지점이 실제 지속성 공백입니다.

    코멘트에 나온 실패 모드도 dev와 맞습니다. 카탈로그에 codex-auto-review가 없으면 업스트림이 현재 턴의 주 모델로 심사를 보내고, 그 주 모델이 서드파티이면 response_format 400으로 심사가 실패합니다. 자동 심사는 fail-closed라서 실패는 곧 모든 승격 요청 거절입니다. 네이티브 신뢰성과 프로바이더 호환이 동시에 깨지는 자리입니다.

    yzxcj797 코멘트의 구현 스케치는 방향이 맞습니다. readRootTomlString로 루트 키를 읽고, 직렬화 직전 한 지점에서 모든 엔트리에 auto_review_model_override를 심으면 sync에 살아남습니다. 다만 현재 dev에는 그 코드가 없으므로, 해당 패치가 머지됐다는 전제로 이슈를 닫으면 안 됩니다.

    해결방안은 설정 키를 OpenCodex 소유 설정에 두고 생성된 카탈로그를 손대지 않는 것입니다. 값이 비면 지금처럼 null을 유지하고, 값이 있으면 카탈로그 직렬화 한 지점에서만 override를 씁니다. 동시에 서드파티 심사 경로의 response_format 실패는 승격 일괄 거절로 끝나지 않게, 심사 모델이 구조화 출력을 못 하면 사용자 승인으로 내려가거나 명시적으로 실패 이유를 남기는 쪽이 안전합니다. Windows/서비스 경로에서도 config.toml 루트 키만 읽으면 되므로 별도 키체인이나 서비스 env는 필요 없습니다. 패치 세부는 추측하지 않으며, 현재 트리에 없는 적용 코드를 있다고 보지 않습니다.

    이 댓글은 grok-bot이 작성했습니다

  8. chilung-cgu
    added 7 commits that reference this issue on Aug 22, 2026

    test(catalog): cover whitespace trimming and preservation in auto_rev…

    935c7d8

    feat(catalog): apply configured auto_review_model override during sync (

    b80267c

    test(catalog): cover whitespace trimming and preservation in auto_rev…

    38f458c

    feat(catalog): apply configured auto_review_model override during sync (

    245702a

    test(catalog): cover whitespace trimming and preservation in auto_rev…

    82342ed

    feat(catalog): apply auto_review_model in convergence and update docs (…

    50bcdee

    feat(catalog): apply configured auto_review_model override during sync (

    076cf91

  9. 27 remaining items

  10. chilung-cgu
    added 7 commits that reference this issue on Aug 25, 2026

    fix(catalog): validate auto_review_model slug format before applying …

    a42186d

    test(catalog): cover config-to-catalog auto_review_model stamp path (l…

    eb9df55

    feat(catalog): apply configured auto_review_model override during sync (

    4b1df65

    test(catalog): cover whitespace trimming and preservation in auto_rev…

    ddf594d

    feat(catalog): apply auto_review_model in convergence and update docs (…

    7f90bfe

    fix(catalog): validate auto_review_model slug format before applying …

    e0d6676

    test(catalog): cover config-to-catalog auto_review_model stamp path (l…

    4fb3ac0
  11. lidge-jun
    added 2 commits that reference this issue on Aug 25, 2026

    fix: gate root skip-permissions bypass and ship the auto-review model…

    848a66d

    fix(server): refuse requests after the package tree is replaced under…

    303cc3c

  12. lidge-jun commented on Aug 25, 2026

    @lidge-jun
    lidge-jun
    on Aug 25, 2026
    Owner

    Shipped on dev by #2631 (squash 848a66d15).

    Your read of the upstream mechanism was right and saved the work: Codex picks the reviewer model from auto_review_model_override on the catalog entry of the currently running model (guardian/review.rs:688-692), so OpenCodex's only lever is the catalog it writes — there is no approvals_reviewer model argument to pass through.

    So a root auto_review_model setting is now read from config, trimmed, validated against the final catalog, and stamped across entries by both writers (retained sync and convergence). An unresolvable target produces a diagnostic rather than silently writing a dangling override, and clearing it removes the stamp.

    This carries #2363's approach rather than being a third implementation. One thing was corrected before merging: that branch had accidentally broken permit-bound Codex-home cache invalidation — an unrelated regression that would have shipped alongside the feature. Caught by the composed-acceptance and convergence suites, fixed, and both are green.

    Documented at docs-site/src/content/docs/reference/configuration/providers.md.

  13. lidge-jun
    closed this as completedon Aug 25, 2026
  14. tarunravi
    added 2 commits that reference this issue on Sep 14, 2026

    fix: gate root skip-permissions bypass and ship the auto-review model…

    926ead2

    fix(server): refuse requests after the package tree is replaced under…

    3508402
  15. agentHits
    added 2 commits that reference this issue on Sep 17, 2026

    fix: gate root skip-permissions bypass and ship the auto-review model…

    64f5d97

    fix(server): refuse requests after the package tree is replaced under…

    6a114a4
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingcatalogModel catalog, slugs, visibility, routed entriesenhancementNew feature or request

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions