Skip to content

server : fix default draft model parameters#10586

Merged
ggerganov merged 3 commits into
masterfrom
gg/server-force-draft-kv-f16
Dec 3, 2024
Merged

server : fix default draft model parameters#10586
ggerganov merged 3 commits into
masterfrom
gg/server-force-draft-kv-f16

Conversation

@ggerganov

@ggerganov ggerganov commented Nov 29, 2024

Copy link
Copy Markdown
Member

ref #10552

  • Use F16 KV cache for the draft model
  • Set draft context equal to slot context
  • Do not speculate during prompt processing

@ggerganov ggerganov marked this pull request as ready for review December 3, 2024 08:47
@ggerganov ggerganov changed the title server : force F16 KV cache for the draft model server : fix default draft model parameters Dec 3, 2024
@ggerganov ggerganov merged commit 70b98fa into master Dec 3, 2024
@ggerganov ggerganov deleted the gg/server-force-draft-kv-f16 branch December 3, 2024 09:20
arthw pushed a commit to arthw/llama.cpp that referenced this pull request Dec 20, 2024
* server : force F16 KV cache for the draft model

ggml-ci

* server : fix draft params

ggml-ci

* server : various params fixes

ggml-ci
Seunghhon pushed a commit to Seunghhon/llama.cpp that referenced this pull request Apr 26, 2026
* server : force F16 KV cache for the draft model

ggml-ci

* server : fix draft params

ggml-ci

* server : various params fixes

ggml-ci
ljubomirj pushed a commit to ljubomirj/llama.cpp that referenced this pull request May 6, 2026
* server : force F16 KV cache for the draft model

ggml-ci

* server : fix draft params

ggml-ci

* server : various params fixes

ggml-ci
my-other-github-account pushed a commit to my-other-github-account/llama.cpp that referenced this pull request May 15, 2026
* server : force F16 KV cache for the draft model

ggml-ci

* server : fix draft params

ggml-ci

* server : various params fixes

ggml-ci
my-other-github-account pushed a commit to my-other-github-account/llama.cpp that referenced this pull request May 15, 2026
* server : force F16 KV cache for the draft model

ggml-ci

* server : fix draft params

ggml-ci

* server : various params fixes

ggml-ci
phibya pushed a commit to ziee-ai/llama.cpp that referenced this pull request May 29, 2026
* server : force F16 KV cache for the draft model

ggml-ci

* server : fix draft params

ggml-ci

* server : various params fixes

ggml-ci
AlexiAlp pushed a commit to minghaop/llama.cpp that referenced this pull request Jun 2, 2026
* server : force F16 KV cache for the draft model

ggml-ci

* server : fix draft params

ggml-ci

* server : various params fixes

ggml-ci
AlexiAlp pushed a commit to minghaop/llama.cpp that referenced this pull request Jun 2, 2026
* server : force F16 KV cache for the draft model

ggml-ci

* server : fix draft params

ggml-ci

* server : various params fixes

ggml-ci
fukuro-kun pushed a commit to fukuro-kun/fukuro-llama-cpp-turboquant that referenced this pull request Jul 5, 2026
* server : force F16 KV cache for the draft model

ggml-ci

* server : fix draft params

ggml-ci

* server : various params fixes

ggml-ci
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant