-
Notifications
You must be signed in to change notification settings - Fork 10.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix swift package + add example swift implementation of batched api #3562
Conversation
I think we need figure out why CI is still passing after #3548 merged, it seems like the UPDATE: Since it includes ggml-backend.h it will build successfully in CI, so that should be expected behavior. However, undefined symbol error will occur when referencing externally if not add |
Derp something is wrong with my swift string parsing, doesn't work for emojis |
Thanks for the contribution! Is this example going to be built by the CI? |
1b3b907
to
4b78c4c
Compare
rebased master + added the build step to the root makefile when running on macOS so the CI will pick it up. |
Ok, I'll add a patched GitHub action with those changes |
cf00916
to
44e5a51
Compare
…ed & build it in ci to validate SPM is correctly configured for dependencies
44e5a51
to
cb56fb1
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good! 👍
…example * 'master' of github.com:ggerganov/llama.cpp: (34 commits) examples: support LLaVA v1.5 (multimodal model) (ggml-org#3436) docs : fix typo GOMP_CPU_AFFINITY (ggml-org#3597) cmake : fix add_compile_options on macOS typo : it is `--n-gpu-layers` not `--gpu-layers` (ggml-org#3592) ci : check if there is enough VRAM (ggml-org#3596) server : add completion mode (no chat) (ggml-org#3582) prompts : add mnemonics.txt server : fix kv cache management (ggml-org#3588) main : fix session loading bug (ggml-org#3400) server : add parameter -tb N, --threads-batch N (ggml-org#3584) common : fix mirostat state when using multiple sequences (ggml-org#3543) batched : add bench tool (ggml-org#3545) examples : add batched.swift + improve CI for swift (ggml-org#3562) Add MPT model to supported models in README.md (ggml-org#3574) Minor improvements in GPT2 tokenizer (ggml-org#3567) readme : add bloom (ggml-org#3570) llm : add bloom models (ggml-org#3553) swift : improvements and fixes (ggml-org#3564) llm : add MPT support (ggml-org#3417) infill. : fix tokenization (ggml-org#3508) ...
hey there, thanks for an amazing repo!
the swift package stopped compiling for me and it was tough to debug because the automated tests here are passing, so thought it made sense to add an example use in swift to surface the compiler error and make it trackable as part of the repo. plus, a few people have asked for swift examples with the batch api.
the compiler bug is fixed by including
ggml-backed.c
in the SPM sources, so that's the change inPackage.swift
. you can verify it's needed if you like by commenting out the line and runningmake
inexamples/swift
.i wrote a rough line for line copy of
examples/batched.cpp
in swift, though i don't know c++ very well so relied heavily on copilot.. seems to produce the same output but ymmv :)