Skip to content

Declare modalities so image input actually reaches the model#1

Open
untilhamza wants to merge 1 commit into
t94j0:mainfrom
untilhamza:fix/image-modalities
Open

Declare modalities so image input actually reaches the model#1
untilhamza wants to merge 1 commit into
t94j0:mainfrom
untilhamza:fix/image-modalities

Conversation

@untilhamza

Copy link
Copy Markdown

The bug

Attaching an image to a Kimi model in OpenCode does nothing — the model replies that it can't read images, even though k3 and both coding models support vision.

toConfigModel sets attachment, but OpenCode gates what it will actually send on modalities.input, which defaults to text-only. The attachment is dropped before the request is built, so the model never sees it.

This is easy to miss because the two disagree in the same output:

$ opencode models kimi-code --verbose
  "capabilities": {
    "attachment": true,       # plugin set this
    "input": {
      "text": true,
      "image": false,         # ...but OpenCode gates on this

I confirmed this isn't upstream-side — Kimi's /models reports supports_image_in: true for all three models, and POSTing an image_url part directly to api.kimi.com/coding/v1/chat/completions returns a correct reading of the image. It's purely a registration gap.

Changes

  • toConfigModel — map supports_image_in / supports_video_in into modalities.input alongside the existing attachment flag.
  • STATIC_MODELS — the baseline omitted both flags, so on a cold cache (fresh install, or logged out) every model registered as text-only regardless of the first fix. Added them to match what /models reports.
  • Tests — cover the modality mapping, the text-only fallback, and output modality.

Verification

Against opencode 1.18.3, with no user config override:

  • input.image resolves true from the cache path and the static-baseline path (tested by moving models.json aside)
  • k3 correctly reads an attached PNG that it previously refused
  • npm run ci clean — lint, format, typecheck, 38/38 tests

Note on the video modality

I mapped supports_video_in faithfully since /models reports it and video is a valid literal in OpenCode's schema, but I've only exercised the image path. Happy to drop video to image-only if you'd rather not advertise something untested.

`toConfigModel` set `attachment`, but OpenCode gates what it will actually
send on `modalities.input`, which defaults to text-only. An image-capable
model therefore had its attachments dropped before the request was built,
and replied that it could not read images — while `opencode models
kimi-code --verbose` reported `attachment: true` alongside `input.image:
false`, making the mismatch easy to miss.

Kimi's /models reports supports_image_in and supports_video_in for k3 and
both coding models, and the API accepts image parts on the subscription
endpoint — so this was purely a registration gap.

- map supports_image_in/supports_video_in into modalities.input
- fix STATIC_MODELS, which omitted both flags: on a cold cache (fresh
  install, or logged out) every model registered as text-only regardless
- cover both in unit tests

Verified against opencode 1.18.3: `input.image` now resolves true from
both the cache and the static baseline, and k3 reads an attached PNG.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant