Skip to content

JinjaParser crashes server on unsupported chat_template statement keywords (e.g. {% macro %}) #273

Closed

Description

@jamesburton

Summary

Found while running OllamaBenchmarks against Bonsai-27B: JinjaParser (the chat-template Jinja2-subset interpreter, per docs/TOKENIZERS.md) only handles if/for/set statement keywords. Any other keyword — e.g. {% macro %}, which Qwen3-style chat templates use — throws an unhandled exception during parsing.

Impact

ServerStartup.LoadModel does not catch this exception, so the entire server process crashes before the model loads for any GGUF whose declared tokenizer.chat_template uses a macro (or any other statement type JinjaParser doesn't support) — not a graceful per-request failure, a full startup crash.

Repro

Serve prism-ml/Ternary-Bonsai-27B-gguf (or any GGUF with a {% macro %}-based chat template) via dotllm serve. Server crashes on load.

Workaround used (not a fix)

The benchmarking session served a byte-length-identical patched copy of the GGUF with a reduced if/for/set-only ChatML template, bypassing the actual template.

Suggested fix direction

Two independent improvements, likely both worth doing:

  1. JinjaParser should support (or at minimum gracefully skip/no-op) {% macro %} blocks rather than throwing on an unrecognized keyword — real-world Qwen/Qwen3-family templates commonly use macros.
  2. ServerStartup.LoadModel should catch template-parsing failures and either fail loudly with a clear, actionable error message (rather than an unhandled crash) or fall back to a safe default template, rather than taking down the whole process.

References

  • docs/TOKENIZERS.md (chat template / Jinja2-subset interpreter docs)

Activity

  1. jamesburton commented on Aug 7, 2026

    @jamesburton
    OwnerAuthor

    Fixed and merged to dev. Real macro definition + call-site expansion added to the Jinja evaluator (not a stub), plus ServerStartup.LoadModel now catches JinjaException and falls back gracefully instead of crashing. Real-world verified against the actual Mach-1-Additive-35B chat_template.jinja with exact-output assertions. 158 tests pass, no regressions in the broader CPU suite.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions