changelog : `libllama` API

[ggerganov]

Overview

This is a list of changes to the public interface of the llama library. Collaborators are encouraged to edit this post in order to reflect important changes to the API that end up merged into the master branch.

If you are building a 3rd party project that relies on libllama, it is recommended to follow this issue and check it before upgrading to new versions.

See also:

• Changelog for llama-server REST API

Recent API changes (most recent at the top)

version	PR	desc
b4599	#9639	Added llama_sampler_init_grammar_lazy to support lazy grammars w/ trigger words & tokens
b4524	#11016	Add name parameter to llama_model_chat_template (uses default template if NULL)
b4501	#11262	Remove rpc_servers from llama_model and llama_model_params
b4464	#11110	Add llama_vocab and rename various structs and calls
b4424	#11063	Update llama_model API naming
b4357	#10784	Remove llama_model_get_tensor()
b4337	#10803	Change llama_sampler_init_penalties()
b4282	#10446	Removed support for Q4_0_N_M model files in favor of automatic repacking of Q4_0
b4167	#10497	Add devices to llama_model_params
b3948	#9897	Deprecate softmax sampler and update dist sampler`
b3988	#10071	Remove Tail-Free sampling
b3943	#9745	Removed all_pos_0, all_pos_1, all_seq_id from llama_batch
b3908	#9798	Update FIM-related API
b3841	#9510	Add LLAMA_POOLING_TYPE_RANK
b3774	#9512	Add llama_n_head()
b3750	#9355	Add llama_perf API + param to disable internal profiling
b3749	#9445	Add llama_sampler_chain_remove()
b3681	#9294	Major changes to the sampling API (see PR for more info)
b3651	#8980	Add LLAMA_VOCAB_TYPE_RWKV enum value
b3644	#8672	Add llama_threadpool API + change uint32_t -> int32_t
b3614	#8526	Add llama_model_is_recurrent

For older changes, use:

git log --oneline -p b3614 -- include/llama.h

(For collaborators) To link between PR number vs Build number:

git log --oneline | tail -r | nl

Upcoming API changes

• TBD

[ggerganov]

#9355 restores the functionality for getting performance measurements from within libllama (which was removed in #9294) via a new llama_perf API. The llama_context_params is extended with a new bool no_perf parameter that can be used to disable the internal timings during libllama compute.

[ddh0]

Looks like llama_model_get_tensor was removed from the API but that change was not documented here

[ggerganov]

Looks like llama_model_get_tensor was removed from the API but that change was not documented here

I didn't expect that this function is being used by anyone, so I skipped updating the changelog. It's updated now.

Btw, what do you use this call for?

[ddh0]

I didn't expect that this function is being used by anyone, so I skipped updating the changelog. It's updated now.

Btw, what do you use this call for?

I don't use it personally but the function was included in my Python code, I started to get ctypes "symbol not found" errors and I had to do some digging to figure out why. No worries!

[ggerganov]

Significant updates to the public API - added struct llama_vocab and applied multiple naming changes for better consistency: #11110. Update of user code should be relatively easy - mostly changing functions to use the new names and calling llama_model_get_vocab(model) where the old API required llama_model and the new API requires llama_vocab.