Description
Prerequisites
Please answer the following questions for yourself before submitting an issue.
- I am running the latest code. Development is very rapid so there are no tagged versions as of now.
- I carefully followed the README.md.
- I searched using keywords relevant to my issue to make sure that I am creating a new issue that is not already open (or closed).
- I reviewed the Discussions, and have a new bug or useful enhancement to share.
Expected Behavior
I expect to be able to send either credentialed or uncredentialed responses without a difference in behavior in the server.
read here on MDN about "credentials" in the heading "Functional overview"
Current Behavior
When sending a credentialed request , the llama.cpp server does not respond with the correct CORS preflight response.
Environment and Context
You can use this super simple llama client to test out the behavior. Write the url in the input field, for instance I have http://localhost:8083/v1/chat/completions) and press the "send with auth" button to see that it does not work. Open browser console to inspect that the OPTIONS flight request fails.
The almost identical request that does not use a credentialed request works as expected.
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>llama.cpp server CORS</title>
</head>
<body>
<button class="without">Send without auth</button>
<button class="with">Send with auth</button>
<input type="text" class="url" placeholder="localhost:8080/v1/completion">
<pre>
<code class="out">
</code>
</pre>
<script type="text/javascript">
const $out = document.querySelector(".out");
const $btn_without = document.querySelector(".without");
const $btn_with = document.querySelector(".with");
const $url = document.querySelector(".url");
$btn_with.addEventListener("click", async function () {
const res = await fetch($url.value, {
method: "POST",
headers: {
Authorization: "no key",
"Content-Type": "application/json",
},
body: JSON.stringify({
messages: [{role: "user", content: "What is the color of the sky?"}]
})
});
$out.innerText = JSON.stringify(JSON.parse(await res.text()), null, 4);
});
$btn_without.addEventListener("click", async function () {
const res = await fetch($url.value, {
method: "POST",
headers: {
"Content-Type": "application/json",
},
body: JSON.stringify({
messages: [{role: "user", content: "What is the color of the sky?"}]
})
});
$out.innerText = JSON.stringify(JSON.parse(await res.text()), null, 4);
});
</script>
</body>
</html>
Activity