Skip to content

Commit

Permalink
server : clarify some items in the readme (#5957)
Browse files Browse the repository at this point in the history
* server : clarify some items in the readme

* server : fix typo
  • Loading branch information
ggerganov authored Mar 9, 2024
1 parent fb215c3 commit 97c0958
Showing 1 changed file with 6 additions and 2 deletions.
8 changes: 6 additions & 2 deletions examples/server/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -195,7 +195,11 @@ node index.js

*Options:*

`prompt`: Provide the prompt for this completion as a string or as an array of strings or numbers representing tokens. Internally, the prompt is compared to the previous completion and only the "unseen" suffix is evaluated. If the prompt is a string or an array with the first element given as a string, a `bos` token is inserted in the front like `main` does.
`prompt`: Provide the prompt for this completion as a string or as an array of strings or numbers representing tokens. Internally, if `cache_prompt` is `true`, the prompt is compared to the previous completion and only the "unseen" suffix is evaluated. A `BOS` token is inserted at the start, if all of the following conditions are true:

- The prompt is a string or an array with the first element given as a string
- The model's `tokenizer.ggml.add_bos_token` metadata is `true`
- The system prompt is empty

`temperature`: Adjust the randomness of the generated text (default: 0.8).

Expand Down Expand Up @@ -308,7 +312,7 @@ Notice that each `probs` is an array of length `n_probs`.

`content`: Set the text to tokenize.

Note that the special `BOS` token is not added in front of the text and also a space character is not inserted automatically as it is for `/completion`.
Note that a special `BOS` token is never inserted.

- **POST** `/detokenize`: Convert tokens to text.

Expand Down

0 comments on commit 97c0958

Please sign in to comment.