Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix embeddings when using CUDA #3657

Merged
merged 1 commit into from
Oct 17, 2023
Merged

fix embeddings when using CUDA #3657

merged 1 commit into from
Oct 17, 2023

Conversation

slaren
Copy link
Collaborator

@slaren slaren commented Oct 17, 2023

Fixes #3625

@ggerganov ggerganov mentioned this pull request Oct 17, 2023
Closed
@slaren slaren merged commit cb33f43 into master Oct 17, 2023
33 checks passed
@slaren slaren deleted the fix-cuda-embeddings branch October 17, 2023 20:24
@ggerganov ggerganov mentioned this pull request Oct 17, 2023
Closed
9 tasks
joelkuiper added a commit to vortext/llama.cpp that referenced this pull request Oct 19, 2023
@joelkuiper
Merge branch 'master' of github.com:ggerganov/llama.cpp
384fc72 
* 'master' of github.com:ggerganov/llama.cpp:
  fix embeddings when using CUDA (ggerganov#3657)
  llama : avoid fprintf in favor of LLAMA_LOG (ggerganov#3538)
  readme : update hot-topics & models, detail windows release in usage (ggerganov#3615)
  CLBlast: Fix temporary buffer size for f16 conversion (wsize)
  train-text-from-scratch : fix assert failure in ggml-alloc (ggerganov#3618)
  editorconfig : remove trailing spaces
  server : documentation of JSON return value of /completion endpoint (ggerganov#3632)
  save-load-state : fix example + add ci test (ggerganov#3655)
  readme : add Aquila2 links (ggerganov#3610)
  tokenizer : special token handling (ggerganov#3538)
  k-quants : fix quantization ranges (ggerganov#3646)
  llava : fix tokenization to not add bos between image embeddings and user prompt (ggerganov#3645)
  MPT : support GQA for replit-code-v1.5 (ggerganov#3627)
  Honor -ngl option for Cuda offloading in llava (ggerganov#3621)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ggerganov ggerganov ggerganov approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Bug: Invalid Embeddings if GPU offloaded (CUDA)
2 participants
@slaren @ggerganov