Embedding failure in current version: 'llama_model_params' object has no attribute 'embedding'

[pkreissel]

Prerequisites

Please answer the following questions for yourself before submitting an issue.

• I am running the latest code. Development is very rapid so there are no tagged versions as of now.
• I carefully followed the README.md.
• I searched using keywords relevant to my issue to make sure that I am creating a new issue that is not already open (or closed).
• I reviewed the Discussions, and have a new bug or useful enhancement to share.

Expected Behavior

llm.embed(text) should output embedding

Current Behavior

I get this error:
AttributeError: 'llama_model_params' object has no attribute 'embedding'

Environment and Context

I think the error is related to this recent change: ggml-org/llama.cpp#3301

• Physical (or virtual) hardware you are using, e.g. for Linux:

MacOS M1 architecture

• Operating System, e.g. for Linux:

MacOS 14.0

• SDK version, e.g. for Linux:

$ python3 3.11
$ make 3.81
$ g++ 15

Failure Information (for bugs)

AttributeError: 'llama_model_params' object has no attribute 'embedding'

Steps to Reproduce

I use this, but the error seems independent: https://github.com/rhohndorf/Auto-Llama-cpp

1. Load llm
2. call llm.embed with text
3. Get error

llama-cpp-python was cloned and installed today.

Failure Logs

File "/Users/.../Auto-Llama-cpp-main/scripts/memory/base.py", line 8, in get_ada_embedding
return llm.embed(text)
^^^^^^^^^^^^^^^
File "/opt/homebrew/lib/python3.11/site-packages/llama_cpp/llama.py", line 898, in embed
return list(map(float, self.create_embedding(input)["data"][0]["embedding"]))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/lib/python3.11/site-packages/llama_cpp/llama.py", line 844, in create_embedding
if self.model_params.embedding == False:
^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'llama_model_params' object has no attribute 'embedding'
ggml_metal_free: deallocating

[darxkies]

Changing model_params to context_params in the llama.py:844 seems to fix the problem.

[abetlen]

@pkreissel should be fixed now in v0.2.10, let me know if that works now!

[darxkies]

v0.2.10 fixed the bug.