Skip to content

Const ref pair #302

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 11 commits into from
Aug 15, 2024
Merged

Const ref pair #302

merged 11 commits into from
Aug 15, 2024

Conversation

@Nexesenex
Copy link
Owner

No description provided.

GermanAizek and others added 11 commits May 13, 2024 20:07
@GermanAizek
Added const reference for std::pair<> and std::tuple<> more 16 bytes:
ced5bfe 
- std::pair<llama_ngram, llama_ngram_cache_part> (72 bytes -> 8 bytes)
- std::tuple<std::string, float> (40 bytes -> 8 bytes)
@GermanAizek
Merge branch 'ggerganov:master' into const-ref-pair
ce4a390
@GermanAizek
Added const reference for std::pair<> and std::tuple<> more 16 bytes:
f2e4d92 
- std::pair<llama_ngram, llama_ngram_cache_part> (72 bytes -> 8 bytes)
- std::tuple<std::string, float> (40 bytes -> 8 bytes)
@0cc4m
Vulkan Optimizations and Fixes (ggml-org#8959)
5fd89a7 
* Optimize Vulkan REPEAT performance

* Use Vulkan GLSL fused multiply-add instruction where possible

* Add GGML_VULKAN_PERF option to output performance data per operator

* Rework and fix Vulkan descriptor set and descriptor pool handling

* Fix float32 concat f16 shader validation error

* Add Vulkan GROUP_NORM eps parameter

* Fix validation error with transfer queue memory barrier flags

* Remove trailing whitespaces
@jpodivin
server : init stop and error fields of the result struct (ggml-org#9026)
234b306 
Signed-off-by: Jiri Podivin <jpodivin@redhat.com>
@ggerganov
ci : disable bench workflow (ggml-org#9010)
d5492f0
@Exploder98
llama : add pre-tokenizer regexes for BLOOM and gpt3-finnish (ggml-or…
6bda7ce 
…g#8850)
@kylo5aby
common : remove duplicate function llama_should_add_bos_token (ggml-o…
4af8420 
…rg#8778)
@snowyu
server : fix duplicated n_predict key in the generation_settings (ggm…
37501d9 
…l-org#8994)
@gtygo
retrieval : fix memory leak in retrieval query handling (ggml-org#8955)
4b9afbb 
* retrieval

* Reuse querybatch to reduce frequent memory allocation

* delete unused white space
@GermanAizek
Merge branch 'master' into const-ref-pair
2793b86
@Nexesenex Nexesenex merged commit 6d504c4 into Nexesenex:lcpp_pr_ngram_cache Aug 15, 2024
Nexesenex added a commit that referenced this pull request May 21, 2025
@Nexesenex @ikawrakow
Quantization improvements #295 and #302, GGML part only
ee4c57b 
Co-Authored-By: Kawrakow <iwankawrakow@gmail.com>
Nexesenex added a commit that referenced this pull request May 21, 2025
@Nexesenex @ikawrakow
Quantization improvements #295 and #302, GGML part only
5281487 
Co-Authored-By: Kawrakow <iwankawrakow@gmail.com>
Nexesenex added a commit that referenced this pull request May 21, 2025
@Nexesenex @ikawrakow
Quantization improvements #295 and #302, GGML part only
337c22d 
Co-Authored-By: Kawrakow <iwankawrakow@gmail.com>
Nexesenex pushed a commit that referenced this pull request May 24, 2025
@ikawrakow
Quantization improvements (2) (#302)
190e786 
* iq3_k: slightly better quantization

Not much of a difference for most models, but this change
avoids what it looks like a catastrophic failure for DeepSeek-Lite
(PPL is now 7.041 vs 7.314 on main).

* Small improvement for type-1 quants

---------

Co-authored-by: Iwan Kawrakow <iwan.kawrakow@gmail.com>
Nexesenex added a commit that referenced this pull request Aug 8, 2025
@Nexesenex @ikawrakow
Quantization improvements #295 and #302, GGML part only
c93034c 
Co-Authored-By: Kawrakow <iwankawrakow@gmail.com>
Nexesenex added a commit that referenced this pull request Aug 8, 2025
@Nexesenex @ikawrakow
Quantization improvements #295 and #302, GGML part only
25863af 
Co-Authored-By: Kawrakow <iwankawrakow@gmail.com>
Nexesenex added a commit that referenced this pull request Aug 8, 2025
@Nexesenex @ikawrakow
Quantization improvements #295 and #302, GGML part only
4c5a3c9 
Co-Authored-By: Kawrakow <iwankawrakow@gmail.com>
Nexesenex added a commit that referenced this pull request Aug 8, 2025
@Nexesenex @ikawrakow
Quantization improvements #295 and #302, GGML part only
ff7f31e 
Co-Authored-By: Kawrakow <iwankawrakow@gmail.com>
Nexesenex added a commit that referenced this pull request Aug 9, 2025
@Nexesenex @ikawrakow
Quantization improvements #295 and #302, GGML part only
3f8ad23 
Co-Authored-By: Kawrakow <iwankawrakow@gmail.com>
Nexesenex added a commit that referenced this pull request Aug 9, 2025
@Nexesenex @ikawrakow
Quantization improvements #295 and #302, GGML part only
7fdaac2 
Co-Authored-By: Kawrakow <iwankawrakow@gmail.com>
Nexesenex added a commit that referenced this pull request Aug 10, 2025
@Nexesenex @ikawrakow
Quantization improvements #295 and #302, GGML part only
3a8622d 
Co-Authored-By: Kawrakow <iwankawrakow@gmail.com>
Nexesenex added a commit that referenced this pull request Aug 11, 2025
@Nexesenex @ikawrakow
Quantization improvements #295 and #302, GGML part only
89fbb9f 
Co-Authored-By: Kawrakow <iwankawrakow@gmail.com>
Nexesenex added a commit that referenced this pull request Oct 5, 2025
@Nexesenex @ikawrakow
Quantization improvements #295 and #302, GGML part only
c7f21b9 
Co-Authored-By: Kawrakow <iwankawrakow@gmail.com>
Nexesenex added a commit that referenced this pull request Oct 7, 2025
@Nexesenex @ikawrakow
Quantization improvements #295 and #302, GGML part only
1974bdf 
Co-Authored-By: Kawrakow <iwankawrakow@gmail.com>
Nexesenex added a commit that referenced this pull request Oct 7, 2025
@Nexesenex @ikawrakow
Quantization improvements #295 and #302, GGML part only
c83c50b 
Co-Authored-By: Kawrakow <iwankawrakow@gmail.com>
Nexesenex added a commit that referenced this pull request Oct 9, 2025
@Nexesenex @ikawrakow
Quantization improvements #295 and #302, GGML part only
a53d6af 
Co-Authored-By: Kawrakow <iwankawrakow@gmail.com>
Nexesenex added a commit that referenced this pull request Oct 9, 2025
@Nexesenex @ikawrakow
Quantization improvements #295 and #302, GGML part only
0e7f878 
Co-Authored-By: Kawrakow <iwankawrakow@gmail.com>
Nexesenex added a commit that referenced this pull request Oct 11, 2025
@Nexesenex @ikawrakow
Quantization improvements #295 and #302, GGML part only
9d5362c 
Co-Authored-By: Kawrakow <iwankawrakow@gmail.com>
Nexesenex added a commit that referenced this pull request Oct 11, 2025
@Nexesenex @ikawrakow
Quantization improvements #295 and #302, GGML part only
b0c0d01 
Co-Authored-By: Kawrakow <iwankawrakow@gmail.com>
Nexesenex added a commit that referenced this pull request Oct 11, 2025
@Nexesenex @ikawrakow
Quantization improvements #295 and #302, GGML part only
0a6734f 
Co-Authored-By: Kawrakow <iwankawrakow@gmail.com>
Nexesenex added a commit that referenced this pull request Oct 11, 2025
@Nexesenex @ikawrakow
Quantization improvements #295 and #302, GGML part only
7b5c99e 
Co-Authored-By: Kawrakow <iwankawrakow@gmail.com>
Nexesenex added a commit that referenced this pull request Oct 12, 2025
@Nexesenex @ikawrakow
Quantization improvements #295 and #302, GGML part only
297d5db 
Co-Authored-By: Kawrakow <iwankawrakow@gmail.com>
Nexesenex added a commit that referenced this pull request Oct 13, 2025
@Nexesenex @ikawrakow
Quantization improvements #295 and #302, GGML part only
6ae7fe5 
Co-Authored-By: Kawrakow <iwankawrakow@gmail.com>
Nexesenex added a commit that referenced this pull request Oct 13, 2025
@Nexesenex @ikawrakow
Quantization improvements #295 and #302, GGML part only
57a9471 
Co-Authored-By: Kawrakow <iwankawrakow@gmail.com>
Nexesenex added a commit that referenced this pull request Oct 16, 2025
@Nexesenex @ikawrakow
Quantization improvements #295 and #302, GGML part only
ef904f2 
Co-Authored-By: Kawrakow <iwankawrakow@gmail.com>
Nexesenex added a commit that referenced this pull request Oct 16, 2025
@Nexesenex @ikawrakow
Quantization improvements #295 and #302, GGML part only
2ca6148 
Co-Authored-By: Kawrakow <iwankawrakow@gmail.com>
Nexesenex added a commit that referenced this pull request Oct 18, 2025
@Nexesenex @ikawrakow
Quantization improvements #295 and #302, GGML part only
b486263 
Co-Authored-By: Kawrakow <iwankawrakow@gmail.com>
Nexesenex added a commit that referenced this pull request Oct 19, 2025
@Nexesenex @ikawrakow
Quantization improvements #295 and #302, GGML part only
e627d3a 
Co-Authored-By: Kawrakow <iwankawrakow@gmail.com>
Nexesenex added a commit that referenced this pull request Oct 20, 2025
@Nexesenex @ikawrakow
Quantization improvements #295 and #302, GGML part only
0dde5b5 
Co-Authored-By: Kawrakow <iwankawrakow@gmail.com>
Nexesenex added a commit that referenced this pull request Oct 21, 2025
@Nexesenex @ikawrakow
Quantization improvements #295 and #302, GGML part only
f5c0291 
Co-Authored-By: Kawrakow <iwankawrakow@gmail.com>
Nexesenex added a commit that referenced this pull request Oct 21, 2025
@Nexesenex @ikawrakow
Quantization improvements #295 and #302, GGML part only
00819be 
Co-Authored-By: Kawrakow <iwankawrakow@gmail.com>
Nexesenex added a commit that referenced this pull request Oct 21, 2025
@Nexesenex @ikawrakow
Quantization improvements #295 and #302, GGML part only
d58d030 
Co-Authored-By: Kawrakow <iwankawrakow@gmail.com>
Nexesenex added a commit that referenced this pull request Oct 22, 2025
@Nexesenex @ikawrakow
Quantization improvements #295 and #302, GGML part only
9795834 
Co-Authored-By: Kawrakow <iwankawrakow@gmail.com>
Nexesenex added a commit that referenced this pull request Oct 22, 2025
@Nexesenex @ikawrakow
Quantization improvements #295 and #302, GGML part only
6f7389d 
Co-Authored-By: Kawrakow <iwankawrakow@gmail.com>
Nexesenex added a commit that referenced this pull request Oct 22, 2025
@Nexesenex @ikawrakow
Quantization improvements #295 and #302, GGML part only
4e33205 
Co-Authored-By: Kawrakow <iwankawrakow@gmail.com>
Nexesenex added a commit that referenced this pull request Oct 23, 2025
@Nexesenex @ikawrakow
Quantization improvements #295 and #302, GGML part only
c29bd84 
Co-Authored-By: Kawrakow <iwankawrakow@gmail.com>
Nexesenex added a commit that referenced this pull request Oct 24, 2025
@Nexesenex @ikawrakow
Quantization improvements #295 and #302, GGML part only
3a758ee 
Co-Authored-By: Kawrakow <iwankawrakow@gmail.com>
Nexesenex added a commit that referenced this pull request Oct 25, 2025
@Nexesenex @ikawrakow
Quantization improvements #295 and #302, GGML part only
5224ca3 
Co-Authored-By: Kawrakow <iwankawrakow@gmail.com>
Nexesenex added a commit that referenced this pull request Oct 25, 2025
@Nexesenex @ikawrakow
Quantization improvements #295 and #302, GGML part only
124e87c 
Co-Authored-By: Kawrakow <iwankawrakow@gmail.com>
Nexesenex added a commit that referenced this pull request Oct 25, 2025
@Nexesenex @ikawrakow
Quantization improvements #295 and #302, GGML part only
d9e3206 
Co-Authored-By: Kawrakow <iwankawrakow@gmail.com>
Nexesenex added a commit that referenced this pull request Oct 27, 2025
@Nexesenex @ikawrakow
Quantization improvements #295 and #302, GGML part only
dc45167 
Co-Authored-By: Kawrakow <iwankawrakow@gmail.com>
Nexesenex added a commit that referenced this pull request Oct 28, 2025
@Nexesenex @ikawrakow
Quantization improvements #295 and #302, GGML part only
d418b1f 
Co-Authored-By: Kawrakow <iwankawrakow@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

devops examples ggml python server Vulkan

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

9 participants

@Nexesenex @GermanAizek @0cc4m @jpodivin @ggerganov @Exploder98 @kylo5aby @snowyu @gtygo