Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

server : clarify some items in the readme #5957

Merged
merged 2 commits into from
Mar 9, 2024
Merged

server : clarify some items in the readme #5957

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
e41add2
server : clarify some items in the readme
ggerganov Mar 9, 2024
2995880
server : fix typo
ggerganov Mar 9, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 6 additions & 2 deletions examples/server/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -195,7 +195,11 @@ node index.js

*Options:*

`prompt`: Provide the prompt for this completion as a string or as an array of strings or numbers representing tokens. Internally, the prompt is compared to the previous completion and only the "unseen" suffix is evaluated. If the prompt is a string or an array with the first element given as a string, a `bos` token is inserted in the front like `main` does.
`prompt`: Provide the prompt for this completion as a string or as an array of strings or numbers representing tokens. Internally, if `cache_prompt` is `true`, the prompt is compared to the previous completion and only the "unseen" suffix is evaluated. A `BOS` token is inserted at the start, if all of the following conditions are true:

- The prompt is a string or an array with the first element given as a string
- The model's `tokenizer.ggml.add_bos_token` metadata is `true`
- The system prompt is empty

`temperature`: Adjust the randomness of the generated text (default: 0.8).

Expand Down Expand Up @@ -308,7 +312,7 @@ Notice that each `probs` is an array of length `n_probs`.

`content`: Set the text to tokenize.

Note that the special `BOS` token is not added in front of the text and also a space character is not inserted automatically as it is for `/completion`.
Note that a special `BOS` token is never inserted.

- **POST** `/detokenize`: Convert tokens to text.

Expand Down
Loading