Skip to content

Commit 7a626ba

Browse files
committed
Sync llama : improve sep token handling
1 parent bf90063 commit 7a626ba

File tree

1 file changed

+10
-0
lines changed

1 file changed

+10
-0
lines changed

llama_cpp/llama_cpp.py

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2807,6 +2807,16 @@ def llama_vocab_get_add_eos(vocab: llama_vocab_p, /) -> bool:
28072807
...
28082808

28092809

2810+
# LLAMA_API bool llama_vocab_get_add_sep(const struct llama_vocab * vocab);
2811+
@ctypes_function(
2812+
"llama_vocab_get_add_sep",
2813+
[llama_vocab_p_ctypes],
2814+
ctypes.c_bool,
2815+
)
2816+
def llama_vocab_get_add_sep(vocab: llama_vocab_p, /) -> bool:
2817+
...
2818+
2819+
28102820
# LLAMA_API llama_token llama_vocab_fim_pre(const struct llama_vocab * vocab);
28112821
@ctypes_function(
28122822
"llama_vocab_fim_pre",

0 commit comments

Comments
 (0)