Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Working dummy example for custom LLM endpoint integration #436

Open
borhenryk opened this issue Jan 29, 2024 · 1 comment
Open

Working dummy example for custom LLM endpoint integration #436

borhenryk opened this issue Jan 29, 2024 · 1 comment
Labels
usage How to use `spacy-llm`

Comments

@borhenryk
Copy link

How would a working dummy example work for the scenario of an own LLM like in this README.md https://github.com/explosion/spacy-llm/blob/main/usage_examples/README.md when running it with the proposed config file and adding the dummy RandomClassification.v1 I get this error:

Exception has occurred: AssertionError
exception: no description

from spacy_llm.registry import registry
import random
from typing import Iterable

@registry.llm_models("RandomClassification.v1")
def random_textcat(labels: str):
    labels = labels.split(",")
    def _classify(prompts: Iterable[str]) -> Iterable[str]:
        for _ in prompts:
            yield random.choice(labels)

    return _classify
...
[components.llm.task]
@llm_tasks = "spacy.TextCat.v1"
labels = LABEL1,LABEL2,LABEL3


[components.llm.model]
@llm_models = "RandomClassification.v1"
labels = ${components.llm.task.labels}  # Make sure to use the same label
@rmitsch rmitsch added the usage How to use `spacy-llm` label Feb 1, 2024
@rmitsch
Copy link
Collaborator

rmitsch commented Feb 1, 2024

Hi @borhenryk, can you provide your config and the entire script?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
usage How to use `spacy-llm`
Projects
None yet
Development

No branches or pull requests

2 participants