Closed
Description
This llama.cpp PR fixed a random problem that I've observed in the past but never been able to reproduce:
The problem happens because the string definition in the JSON grammar doesn't exclude emitting raw \n, \r, etc. A model can thus produce a string split into multiple lines, which is not valid JSON.
File llama_cpp/llama_grammar.py needs the string grammar definition fixed in several places:
- At the JSON_GBNF definition and others that have a string rule, like JSON_ARR_GBNF, etc.
- At the string definition inside PRIMITIVE_RULES.
It's a single line change in the above PR.
Regards
Metadata
Metadata
Assignees
Labels
No labels