Skip to content

Onboarding: offline Dia tests + guide updates for quant predicates, KV cache sizing, and batch contracts - #909

Open
mm65x wants to merge 1 commit into
Blaizzy:mainfrom
mm65x:model-onboarding-hardening
Open

Onboarding: offline Dia tests + guide updates for quant predicates, KV cache sizing, and batch contracts#909
mm65x wants to merge 1 commit into
Blaizzy:mainfrom
mm65x:model-onboarding-hardening

Conversation

@mm65x

@mm65x mm65x commented Aug 24, 2026

Copy link
Copy Markdown
Contributor

Summary

Hardens the new-model onboarding path based on three failure modes we hit while porting models recently:

  1. Silently broken converted checkpoints when blanket quantization corrupts embeddings tied to output heads or sensitive paths (fixed for fish_qwen3_omni in fix(fish_speech): prevent fast_ path and embeddings from quantizing to fix convert script #905). The adding-a-model guide now documents model_quant_predicate with examples.
  2. KV cache overruns when a prompt is prefilled before decoding and the cache was sized only for the generated tokens (fixed for dia in fix(dia): allocate kv cache to fit prefill + generation length #904). The guide now warns to size caches for prefill + generation combined.
  3. Batch result contract violations: consumers expect exactly one result per input even when an item produces no audio (fixed for qwen3_tts in fix(qwen3_tts): yield silent audio instead of dropping empty batch items #907). The guide now says to yield silence rather than skip.

Changes

  • mlx_audio/tts/tests/test_dia.py (new): first offline tests for Dia — tiny random-weight config, no network/downloads; covers config parsing, generation without ref_audio, and the KVCache capacity contract.
  • docs/contributing/adding-a-model.md: new "Prevent Broken Conversions" section (model_quant_predicate), KV cache sizing warning, offline-testing guidance replacing the download-based test example, batch contract note, extended checklist.

Companion PRs carrying their own regression tests: #904, #905.

@lucasnewman

Copy link
Copy Markdown
Collaborator

@mm65x Can you sign your commits and force push so we can clear the merge requirements?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants