Commit d6a3072
committed
fix(gallery): raise arch-router scoring output cap via parallel:64
Scoring decodes the whole prompt+candidate in a single llama_decode and
reads one logit row per candidate token. The vendored llama.cpp server
caps causal output rows at n_parallel, so the default of 1 aborts with
GGML_ASSERT(n_outputs_max <= cparams.n_outputs_max) on multi-token route
labels. Set options: [parallel:64] on both arch-router quant entries to
lift the cap; kv_unified (the grpc-server default) keeps the full context
per sequence, so this does not split the KV cache.
Assisted-by: claude-code:claude-opus-4-8 [Claude Code]1 parent 8290c3f commit d6a3072
1 file changed
Lines changed: 11 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
23985 | 23985 | | |
23986 | 23986 | | |
23987 | 23987 | | |
| 23988 | + | |
| 23989 | + | |
| 23990 | + | |
| 23991 | + | |
| 23992 | + | |
| 23993 | + | |
| 23994 | + | |
| 23995 | + | |
23988 | 23996 | | |
23989 | 23997 | | |
23990 | 23998 | | |
| |||
23996 | 24004 | | |
23997 | 24005 | | |
23998 | 24006 | | |
| 24007 | + | |
| 24008 | + | |
| 24009 | + | |
23999 | 24010 | | |
24000 | 24011 | | |
24001 | 24012 | | |
| |||
0 commit comments