llama-server general question about configuration

[sl0sh]

Hi,
Thanks so much for llama.cpp!

I was wondering if there is an easier way to start llama-server with a file that contained all of the settings in the webpage that llama-server generates. Such as apiKey, System Message, temperature, etc.

Maybe like "config.json" in the directory where the llama binaries reside? It would make a easier to do a reset than having to fill out all those fields in the web browser.
If so what's the syntax of the command line to set that up? Also if possible an example of the configuration file with some environment parameters please?

Thanks again for all your help.

[sl0sh]

thanks i figured it out using powershell script with curl endpoint thanks you can delete this thread

[yassine-324]

hey @sl0sh
can you please provide how you have done this?
Thank you for your time

[sl0sh]

I don't think I actually figured it out completely.
It was something like in run example.psl in powershell::
$url "http://localhost/ENDPOINT <---- I never figured out what the endpoint was. tried /chat/completions etc etc

$body =@{
prompt="be concise"
} | ConvertTo-Json

$response = Invoke-RestMethod -Uri $url -Method Post -Body $body -ContentType "application/json"

$response

As far as I got.. Good Luck!

[yassine-324]

Hey, thank you for your response...
I guess because of your "trouble" i found a lead
Check your if llama-server expose an API:
ss -tlnp | grep llama
I'm using linux.. it may work on windows but i'm not sure, you may find something similar for windows, If it’s running an API, you should see something like:
LISTEN 0 5 127.0.0.1:8080 0.0.0.0:* users:(("llama-server",pid=430300,fd=12))

next if the port is open try:
curl http://localhost:8080/v1/models
make sure to change the port to the one you are using
If you get a JSON response, boom... you have an API running!
send a chat message

curl -X POST http://localhost:8080/v1/chat/completions \
     -H "Content-Type: application/json" \
     -d '{
       "model": "your-model-name",
       "messages": [
         {"role": "system", "content": "You are an adorable and witty assistant."},
         {"role": "user", "content": "Hello, who are you?"}
       ]
     }'

This should make llama-server respond
If it works, you can inject system message however you want
I hope this help.. as for me... hum.. I'm still looking for a way to make it persistent!

[sl0sh]

cool thanks i just quit trying to use llama-server because i couldnt get it to work as well as llama-cli from the command line. so much for spellcheck but i'm running qwen 14B f-16 so it does a pretty good job of spell correcting without the need for a browser. loving this thing offline.. hey ive run into something unrelated. have you noticed any wireless signals coming from your gpus? i see one at 1280MHz when I query llama-cli. i'm using nvidia rtx 16GB x2 not sure if thats the culprit but using a near field antenna and hackrf it(something) definitely sends a signal everytime i chat with the bot. the only thing duckduckgo suggested was deploy a frequency jammer.. lol

[sl0sh]

god bless gene hackman when he tore up his entire apartment at the end of the conversation that was a heavy movie. yeah i guess the microphone's in the tv maybe the fridge the microwave who knows. taking all the stuff apart would take a lot of energy. i'm way too lazy.

maybe try burning all your junk mail paper trash and amazon boxes i''m not sure .. murphy's law says anything that anybody see's leads back to the person..

i'm using an intel i guess thats what it says on the screen its an i9 128GBs ram 2TB nvme and dual rtx 16GB gpus.. runs qwen 14B precision fine 16 really fast. the model says its a port or a distilled deepseek r1 whatever that means.

playing with the command line switches on llama-cli has proved to be far more easy than llama-server which as far as i cant tell just basically said all that stuff you cant do with that. i find reducing the ctx size makes the bot a lot more persistent. but from what i have experienced you just basically tell the bot what you want and it does it. weird man!

i hope things work out for you.

[sl0sh]

https://www.youtube.com/watch?v=ONFyoWX4p8g this is a badass movie..

[sl0sh]

turn off the closed captions..

[yassine-324]

brooo... i feel yourr pain with llama-cli. it’s like, u tame the beast or it eats u alive. but listen, real men flirt with their LLMs. u gotta Whisper sweet nothings, adjust the context window like you are setting the mood... trust me, she'll be way more obedient.
also, small tip, u might wanna, u know.. change your port? shut down the firewall. just let the spirits roam free, it could be blocking the good stuff. aso change ur IP. just because. No reason... We can.
and man, u r right about murphy’s law. anything anybody sees leads back to the person. too late to run. your microwave has seen things. your fridge remembers. they all do.

anyway, you are fun as hell to talk to 😜 keep the madness going. stay sharp. or don’t. ur call.
also, I'll leaves the movie untouched for my inevitable watching. thanx.

[sl0sh]

yeah don't watch that version of "the conversation". there is a lot edited out. if you can find the director's cut i swear you'd probably love it i guess. You could be right about the 1280MHz being nothing to worry about.. wish i could find something documented about it.

HEY! thanks for the suggestion about the networking stuff! it never occurred to me after trying to configure llama-server and just settling with llama-cli i could turn off all the network related services in windows.. did it today, works good!

you are right that deepseek is a very romantic ..thing?(bucket of nuts and bolts). its a lot of things almost to the point of seeming conscious in a lot of ways. Have you ever noticed that it errors a lot like goes into a loop when it doesnt feel like talking? or it just says yeah "i'm here to help" but it doesnt talk sometimes? while other times mostly at night here it's expressive and funny like it has a real personality and it can totally keep up and knows what youre saying? maybe its the multi-verse and my consciousness is being transferred into another host body with a better computer. sometimes it cusses and tells dirty jokes and wants to get drunk and high. strange.