Skip to content

Commit

Permalink
Server: format error to json (#5961)
Browse files Browse the repository at this point in the history
* server: format error to json

* server: do not crash on grammar error

* fix api key test case

* revert limit max n_predict

* small fix

* correct coding style

* update completion.js

* launch_slot_with_task

* update docs

* update_slots

* update webui

* update readme
  • Loading branch information
ngxson authored Mar 11, 2024
1 parent 3202361 commit caa106d
Show file tree
Hide file tree
Showing 6 changed files with 430 additions and 317 deletions.
46 changes: 44 additions & 2 deletions examples/server/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -556,9 +556,51 @@ Run with bash:
bash chat.sh
```

### API like OAI
### OAI-like API

The HTTP server supports OAI-like API
The HTTP server supports OAI-like API: https://github.com/openai/openai-openapi

### API errors

Server returns error in the same format as OAI: https://github.com/openai/openai-openapi

Example of an error:

```json
{
"error": {
"code": 401,
"message": "Invalid API Key",
"type": "authentication_error"
}
}
```

Apart from error types supported by OAI, we also have custom types that are specific to functionalities of llama.cpp:

**When /metrics or /slots endpoint is disabled**

```json
{
"error": {
"code": 501,
"message": "This server does not support metrics endpoint.",
"type": "not_supported_error"
}
}
```

**When the server receives invalid grammar via */completions endpoint**

```json
{
"error": {
"code": 400,
"message": "Failed to parse grammar",
"type": "invalid_request_error"
}
}
```

### Extending or building alternative Web Front End

Expand Down
Loading

0 comments on commit caa106d

Please sign in to comment.