Skip to content

Commit 97c0958

Browse files
authored
server : clarify some items in the readme (#5957)
* server : clarify some items in the readme * server : fix typo
1 parent fb215c3 commit 97c0958

File tree

1 file changed

+6
-2
lines changed

1 file changed

+6
-2
lines changed

examples/server/README.md

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -195,7 +195,11 @@ node index.js
195195

196196
*Options:*
197197

198-
`prompt`: Provide the prompt for this completion as a string or as an array of strings or numbers representing tokens. Internally, the prompt is compared to the previous completion and only the "unseen" suffix is evaluated. If the prompt is a string or an array with the first element given as a string, a `bos` token is inserted in the front like `main` does.
198+
`prompt`: Provide the prompt for this completion as a string or as an array of strings or numbers representing tokens. Internally, if `cache_prompt` is `true`, the prompt is compared to the previous completion and only the "unseen" suffix is evaluated. A `BOS` token is inserted at the start, if all of the following conditions are true:
199+
200+
- The prompt is a string or an array with the first element given as a string
201+
- The model's `tokenizer.ggml.add_bos_token` metadata is `true`
202+
- The system prompt is empty
199203

200204
`temperature`: Adjust the randomness of the generated text (default: 0.8).
201205

@@ -308,7 +312,7 @@ Notice that each `probs` is an array of length `n_probs`.
308312

309313
`content`: Set the text to tokenize.
310314

311-
Note that the special `BOS` token is not added in front of the text and also a space character is not inserted automatically as it is for `/completion`.
315+
Note that a special `BOS` token is never inserted.
312316

313317
- **POST** `/detokenize`: Convert tokens to text.
314318

0 commit comments

Comments
 (0)