Skip to content

Commit

Permalink
Server: reorganize some http logic (#5939)
Browse files Browse the repository at this point in the history
* refactor static file handler

* use set_pre_routing_handler for validate_api_key

* merge embedding handlers

* correct http verb for endpoints

* fix embedding response

* fix test case CORS Options

* fix code style
  • Loading branch information
ngxson authored Mar 9, 2024
1 parent e1fa956 commit 950ba1a
Show file tree
Hide file tree
Showing 4 changed files with 335 additions and 314 deletions.
4 changes: 2 additions & 2 deletions examples/server/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ see https://github.com/ggerganov/llama.cpp/issues/1437
- `-to N`, `--timeout N`: Server read/write timeout in seconds. Default `600`.
- `--host`: Set the hostname or ip address to listen. Default `127.0.0.1`.
- `--port`: Set the port to listen. Default: `8080`.
- `--path`: path from which to serve static files (default examples/server/public)
- `--path`: path from which to serve static files (default: disabled)
- `--api-key`: Set an api key for request authorization. By default the server responds to every request. With an api key set, the requests must have the Authorization header set with the api key as Bearer token. May be used multiple times to enable multiple valid keys.
- `--api-key-file`: path to file containing api keys delimited by new lines. If set, requests must include one of the keys for access. May be used in conjunction with `--api-key`'s.
- `--embedding`: Enable embedding extraction, Default: disabled.
Expand Down Expand Up @@ -558,7 +558,7 @@ The HTTP server supports OAI-like API

### Extending or building alternative Web Front End

The default location for the static files is `examples/server/public`. You can extend the front end by running the server binary with `--path` set to `./your-directory` and importing `/completion.js` to get access to the llamaComplete() method.
You can extend the front end by running the server binary with `--path` set to `./your-directory` and importing `/completion.js` to get access to the llamaComplete() method.

Read the documentation in `/completion.js` to see convenient ways to access llama.

Expand Down
Loading

0 comments on commit 950ba1a

Please sign in to comment.