You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
JinjaParser crashes server on unsupported chat_template statement keywords (e.g. {% macro %}) #273
Found while running OllamaBenchmarks against Bonsai-27B: JinjaParser (the chat-template Jinja2-subset interpreter, per docs/TOKENIZERS.md) only handles if/for/set statement keywords. Any other keyword — e.g. {% macro %}, which Qwen3-style chat templates use — throws an unhandled exception during parsing.
Impact
ServerStartup.LoadModel does not catch this exception, so the entire server process crashes before the model loads for any GGUF whose declared tokenizer.chat_template uses a macro (or any other statement type JinjaParser doesn't support) — not a graceful per-request failure, a full startup crash.
Repro
Serve prism-ml/Ternary-Bonsai-27B-gguf (or any GGUF with a {% macro %}-based chat template) via dotllm serve. Server crashes on load.
Workaround used (not a fix)
The benchmarking session served a byte-length-identical patched copy of the GGUF with a reduced if/for/set-only ChatML template, bypassing the actual template.
Suggested fix direction
Two independent improvements, likely both worth doing:
JinjaParser should support (or at minimum gracefully skip/no-op) {% macro %} blocks rather than throwing on an unrecognized keyword — real-world Qwen/Qwen3-family templates commonly use macros.
ServerStartup.LoadModel should catch template-parsing failures and either fail loudly with a clear, actionable error message (rather than an unhandled crash) or fall back to a safe default template, rather than taking down the whole process.
Fixed and merged to dev. Real macro definition + call-site expansion added to the Jinja evaluator (not a stub), plus ServerStartup.LoadModel now catches JinjaException and falls back gracefully instead of crashing. Real-world verified against the actual Mach-1-Additive-35B chat_template.jinja with exact-output assertions. 158 tests pass, no regressions in the broader CPU suite.
Activity
fix(tokenizers,server): add Jinja {% macro %} support + graceful chat…
jamesburton commentedon Aug 7, 2026
Fixed and merged to dev. Real macro definition + call-site expansion added to the Jinja evaluator (not a stub), plus ServerStartup.LoadModel now catches JinjaException and falls back gracefully instead of crashing. Real-world verified against the actual Mach-1-Additive-35B chat_template.jinja with exact-output assertions. 158 tests pass, no regressions in the broader CPU suite.
merge: Jinja {% macro %} support + graceful chat_template failure int…
fix(server): route CUDA hybrid architectures to their own KV-cache (#274
merge: reconcile with origin/dev (#265/#273 landed) before #274 verif…
merge: fix KV-cache dispatch OOM for hybrid CUDA models into dev (#274)