Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

November Binary Update #962

Open
wants to merge 7 commits into
base: master
Choose a base branch
from

Conversation

martindevans
Copy link
Member

@martindevans martindevans commented Oct 26, 2024

Updated to 958367bf530d943a902afa1ce1c342476098576b, built with https://github.com/SciSharp/LLamaSharp/actions/runs/11509292954

This is a much smaller update that the last one. Only notable changes are:

  • Some changes in the fill-in-middle (FIM) token functions (no change to C# API)
  • New FIM tokens exposed, not sure what these are for so no useful comments, if anyone knows a tip would be appreciated!
  • Removed softmax sampler. It's deprecated upstream, so it's best to remove it quickly on our end since that's a new API no one will be depending on yet.
  • Removed tail free sampler. Also deprecated upstream (llama : remove Tail-Free sampling ggerganov/llama.cpp#10071)

Testing:

  • Windows CPU
  • Windows CUDA
  • Windows Vulkan
  • Linux CPU
  • Linux CUDA
  • Linux Vulkan
  • MacOS CPU
  • MacOS Metal

@m0nsky
Copy link
Contributor

m0nsky commented Oct 26, 2024

Unit tests passed on Windows CUDA.

Test application is running fine on:

  • Windows CPU
  • Windows CUDA
  • Windows Vulkan
  • Linux CPU
  • Linux CUDA
  • Linux Vulkan

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants