KoboldCpp is excellent for local inference but sessions are ephemeral — context is lost on restart and is limited by the context window. This proposes optional Dakera (https://dakera.ai) integration as an external memory backend that persists context across sessions.
Problem: KoboldCpp users running persistent companions or long-form creative writing lose all context when the context window fills or the server restarts.
Proposed: Optional --memory-url flag:
koboldcpp.py --model mymodel.gguf --memory-url http://localhost:3300 --memory-key demo
At generation time:
- Key events/exchanges are stored via POST /v1/memories
- Before generation, relevant past context is retrieved via POST /v1/memories/search and prepended to the story
Why Dakera fits KoboldCpp's philosophy:
- Both are self-hosted first — no cloud dependencies
- Both run on consumer hardware
- Dakera's decay weighting is narratively appropriate: recent events are vivid, old ones fade
Setup: docker run -d -p 3300:3300 -e DAKERA_API_KEY=demo ghcr.io/dakera-ai/dakera:latest (or native binary available)
Happy to open a PR implementing this as an optional extension.
KoboldCpp is excellent for local inference but sessions are ephemeral — context is lost on restart and is limited by the context window. This proposes optional Dakera (https://dakera.ai) integration as an external memory backend that persists context across sessions.
Problem: KoboldCpp users running persistent companions or long-form creative writing lose all context when the context window fills or the server restarts.
Proposed: Optional --memory-url flag:
koboldcpp.py --model mymodel.gguf --memory-url http://localhost:3300 --memory-key demo
At generation time:
Why Dakera fits KoboldCpp's philosophy:
Setup: docker run -d -p 3300:3300 -e DAKERA_API_KEY=demo ghcr.io/dakera-ai/dakera:latest (or native binary available)
Happy to open a PR implementing this as an optional extension.