Skip to content

fix(export): hermes export emits flat string array without per-model capabilities (supports_vision: true) #3146

Closed
Closed
fix(export): hermes export emits flat string array without per-model capabilities (supports_vision: true)#3146
Labels
providerProvider adapters, OpenAI-compat presets, upstream API quirksprovider-compatibilityProvider compatibility reports

Description

@GoldenLoaf24h
GoldenLoaf24h
opened on Sep 1, 2026

Client or integration

Other

Provider or upstream service

Hermes Agent client export (ocx export --client hermes)

OpenCodex version

2.38.0

Endpoint or capability

Client integration & configuration export (/api/export / ocx export --client hermes)

Current behaviour

When exporting configuration for Hermes Agent via ocx export --client hermes (or Web UI Integration), the models list is serialized as a flat array of model id strings:

providers:
  opencodex:
    api: "http://127.0.0.1:10100/v1"
    api_key: "${OPENCODEX_HERMES_API_KEY}"
    api_mode: chat_completions
    discover_models: false
    models:
      - google-antigravity/gemini-3.7-flash
      - opencode-go/gpt-5.6-luna

Because opencodex is a custom local provider (not found in Hermes's default models.dev remote registry), Hermes's dynamic router (agent.image_routing._lookup_supports_vision) falls back to None and assumes text-only capability when given a flat list without explicit flags. Consequently, native multimodal image/video requests to vision-capable models (e.g. Gemini 3.7 Flash, GPT-5.6 Luna, Grok 4.6, MiMo v2.5) are intercepted or treated as text-only.

Expected behaviour

ocx export --client hermes should serialize models as a dictionary mapping with capability metadata (e.g. supports_vision: true, supports_video: true) for multimodal models based on OpenCodex's existing catalog modality metadata (modelInputModalities / visionEligible):

providers:
  opencodex:
    api: "http://127.0.0.1:10100/v1"
    api_key: "${OPENCODEX_HERMES_API_KEY}"
    api_mode: chat_completions
    discover_models: false
    models:
      google-antigravity/gemini-3.7-flash:
        supports_vision: true
        supports_video: true
      opencode-go/gpt-5.6-luna:
        supports_vision: true
      opencode-go/grok-4.6:
        supports_vision: true
      opencode-go/deepseek-v4-flash:
        supports_vision: false

Minimal redacted request or reproduction

  1. Run ocx export --client hermes
  2. Inspect the generated providers.opencodex.models YAML section.
  3. Paste into ~/.hermes/config.yaml and observe that Hermes treats all custom models as supports_vision: None unless explicitly overridden as a dictionary.

Actual response or error

Flat string array output without per-model capability declarations.

Upstream documentation

Hermes Agent provider capability configuration: https://hermes-agent.nousresearch.com/docs

Suggested mapping or implementation notes

In the Hermes export serializer (where client YAML is generated), inspect the catalog row for each model:

  • If model has image/vision input capability, emit supports_vision: true (and supports_video: true for video-capable models like Gemini).
  • Otherwise emit supports_vision: false or omit the capability mapping.

Additional context and attachments

Both string list and dictionary mapping formats are natively supported by Hermes, but dictionary mapping is required for custom local endpoints to advertise vision capability without falling back to text-only mode.

Checks

  • I searched existing provider and compatibility issues.
  • The request and response were redacted.
  • The expected behaviour is based on an upstream specification or a concrete client requirement.

Activity

  1. github-actions commented on Sep 1, 2026

    @github-actions
    github-actionsbot
    on Sep 1, 2026 – with GitHub Actions
    Contributor

    Automated translation bookkeeping — detected language: English.

  2. github-actions
    added
    provider-compatibilityProvider compatibility reports
    providerProvider adapters, OpenAI-compat presets, upstream API quirks
    on Sep 1, 2026

  3. Ingwannu commented on Sep 1, 2026

    @Ingwannu
    Ingwannu
    on Sep 1, 2026
    Owner

    Confirmed against current dev and the current Hermes Agent parser.

    OpenCodex currently emits providers.opencodex.models as a string array. Hermes normalizes each string to { modelId: {} }; its custom-provider capability lookup then finds no per-model supports_vision boolean and returns unknown. Because opencodex is not a built-in models.dev provider, the router cannot recover that capability elsewhere. The reported behavior is therefore real, not only a display-format preference.

    I will update the Hermes export contract and all shared export surfaces together:

    • emit models as a mapping keyed by the canonical namespaced selector;
    • emit supports_vision: true only when the OpenCodex catalog explicitly declares image input;
    • emit supports_vision: false for an explicitly text-only row;
    • omit the flag when modalities are unknown, rather than guessing;
    • preserve model ordering, admission-key references, and the existing no-secret export boundary;
    • add exporter, YAML round-trip, CLI/management parity, and unknown-modality regressions.

    I will not add supports_video in this fix. OpenCodex currently validates text/image/audio modalities and has no authoritative video-input fact in this export contract, so adding it from model-name guesses would violate the existing no-guessed-metadata rule.

  4. Ingwannu commented on Sep 1, 2026

    @Ingwannu
    Ingwannu
    on Sep 1, 2026
    Owner

    Implemented in Draft PR #3151 and requested owner review from @lidge-jun.

    The exporter now emits the Hermes-supported model metadata map instead of a flat string array. It carries only catalog-backed vision facts: explicit image becomes supports_vision true, explicit non-image modalities become false, and unknown modalities stay an empty object. It intentionally does not guess supports_video.

    Verification on the exact branch head:

    • 198 focused export, YAML, CLI, management, writer, and lifecycle tests passed.
    • TypeScript passed.
    • The changed-graph run passed 3,241 tests; its unrelated storage-policy and TOML-parser failures were reproduced on clean origin/dev. The PR remains Draft while exact-head CI completes.
    • Protected local runtime configuration remained unchanged.

  5. lidge-jun commented on Sep 1, 2026

    @lidge-jun
    lidge-jun
    on Sep 1, 2026
    Owner

    리뷰 · 우선순위 65 / 80

    설명

    이 이슈는 ocx export --client hermes(또는 GUI Integration)가 Hermes용 models 를 문자열 배열로만 내보내, Hermes의 custom-provider vision lookup이 실패한다는 기능 버그입니다. 보고 버전은 2.38.0 입니다. Hermes는 opencodex를 models.dev 기본 레지스트리에서 찾지 못하면 flat list를 텍스트 전용으로 가정합니다. 그래서 Gemini 3.7 Flash, GPT-5.6 Luna, Grok, MiMo처럼 이미지/비디오를 받는 행이 텍스트로만 취급됩니다.

    지금 HEAD src/clients/config-export.ts 를 보면 HermesProviderBlock 의 models 타입은 string[] 이고 discover_models: false 입니다. buildHermesClientConfig 가 그 블록을 만듭니다. 이슈에 붙은 YAML 예시와 바이트 단위로 같은 모양입니다. #3047 Aside export, #3048 Integrations, #3082 Hermes 마크가 클라이언트 표면을 키우는 동안에도 Hermes models 직렬화 shape는 그대로였습니다. #3105 가 카탈로그 modalities를 vision-sidecar와 맞췄으므로, export가 그 modalities를 Hermes supports_vision 으로 투영하는 것이 다음 자연스러운 단계입니다.

    같은 날 PR #3151 이 metadata map 형태로 바꾸고, image 포함 여부로 supports_vision true/false/생략을 정하며 Closes #3146 을 걸었습니다. MERGEABLE에 가깝고 테스트·structure 문서가 같이 갑니다. types.ts/config.ts 분할 캠페인과 무관합니다. remote hub 열차와 겹치지 않아 독립 랜딩하기 좋습니다.

    점수는 65입니다. 수정 PR이 이미 있고, 영향 범위는 Hermes export + vision 사용자입니다. 이 이슈만 열어 두면 #3151 랜딩 뒤 열린 숫자만 남습니다.

    경로 src/clients/config-export.ts HermesProviderBlock.models - HEAD가 string[] 인 것이 직접 원인입니다
    경로 src/clients/config-export.ts buildHermesClientConfig - flat 배열을 실제로 조립하는 함수입니다
    경로 PR #3151 - 직접 수정이고 Closes 연결이 있습니다
    경로 Hermes agent.image_routing._lookup_supports_vision - 업스트림 가정은 이슈에 적혀 있습니다. OpenCodex는 export shape만 고치면 됩니다
    경로 #3105 - 카탈로그 modalities 정합의 선행입니다. 이 이슈와 중복으로 닫지 마십시오

    메인테이너의 판단이 필요한 지점

    너의 추천
    #3151을 머지한 뒤 이 이슈를 닫습니다. 같은 주제로 늦게 오는 중복 PR이 있으면 close-don't-rebase로 닫습니다. 라벨은 이 댓글에서 바꾸지 않습니다.

    이 댓글은 grok-bot이 작성했습니다

    Translated Message

    Original language: Korean

    Review · Priority 65 / 80

    Description

    This issue is a functional bug where ocx export --client hermes (or GUI Integration) exports models for Hermes as only a string array, causing Hermes's custom-provider vision lookup to fail. The reported version is 2.38.0. When Hermes cannot find opencodex in the models.dev default registry, it assumes a flat list to be text-only. Therefore, rows that accept images/videos like Gemini 3.7 Flash, GPT-5.6 Luna, Grok, and MiMo are treated as text-only.

    Looking at the current HEAD src/clients/config-export.ts, the models type of HermesProviderBlock is string[] and discover_models: false. buildHermesClientConfig creates that block. It matches the YAML example attached to the issue byte-for-byte. While #3047 Aside export, #3048 Integrations, and #3082 Hermes labels expanded the client surface, the Hermes models serialization shape remained unchanged. Since #3105 aligned catalog modalities with vision-sidecar, the next natural step is for export to project those modalities to Hermes supports_vision.

    On the same day, PR #3151 changed to metadata map form, determines supports_vision true/false/omitted by image inclusion, and tagged Closes #3146. It is close to MERGEABLE and includes tests and structure documentation. This is unrelated to the types.ts/config.ts split campaign. It does not overlap with the remote hub train, making it good for independent landing.

    The score is 65. A fix PR already exists, and the impact scope is Hermes export + vision users. Keeping only this issue open would leave open numbers after #3151 landing.

    Path src/clients/config-export.ts HermesProviderBlock.models - The fact that HEAD is string[] is the direct cause.
    Path src/clients/config-export.ts buildHermesClientConfig - This is the function that actually assembles the flat array.
    Path PR #3151 - This is the direct fix with a Closes connection.
    Path Hermes agent.image_routing._lookup_supports_vision - The upstream assumption is written in the issue. OpenCodex only needs to fix the export shape.
    Path #3105 - This is a prerequisite for catalog modalities alignment. Do not close this issue as a duplicate of this.

    Points requiring maintainer judgment

    My recommendation
    Merge #3151 and then close this issue. If a duplicate PR comes in late on the same topic, close with close-don't-rebase. Do not change labels from this comment.

    This comment was written by grok-bot

  6. lidge-jun commented on Sep 1, 2026

    @lidge-jun
    lidge-jun
    on Sep 1, 2026
    Owner

    Fixed by #3178 (maintainer carry of @Ingwannu's #3151), landed on dev as 51c4917.

    The Hermes export now emits the metadata map rather than a flat string array, so each row carries its capabilities: supports_vision true when the exported catalog modalities include image, false for an explicit non-empty modality list without image, and the flag omitted entirely when modalities are unknown rather than guessed. Selector ordering, admission references, YAML round-tripping, and CLI/management parity are unchanged.

    Closing manually because pull requests target dev rather than the default branch.

  7. lidge-jun
    closed this as completedon Sep 1, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    providerProvider adapters, OpenAI-compat presets, upstream API quirksprovider-compatibilityProvider compatibility reports

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions