Skip to content

Commit

Permalink
Add documentation for model dispatch
Browse files Browse the repository at this point in the history
  • Loading branch information
rlouf committed Jul 30, 2024
1 parent 038a70a commit d2eeaf9
Showing 1 changed file with 23 additions and 0 deletions.
23 changes: 23 additions & 0 deletions docs/reference/dispatch.md
Original file line number Diff line number Diff line change
@@ -1 +1,24 @@
# Model-based prompt dispatching


Different models often require different prompts to achieve a given task. They are, in essence, not different prompts in the sense that they are supposed to perform the same operation. In the same way we use `functools.singledispatch` to dispatch a functionality on the type of the first
argument, it can be useful to dispatch the prompt on the model that is being used.

`prompts` provides a way to dispatch the prompt on the model:


```python
import prompts


@prompts.template
def a_simple_prompt(query: str):
"""<s>{{ query }}</s>"""

@a_simple_prompt.register("google/gemma-2-9b")
def a_simple_prompt_gemma(query: str):
"""<bos>{{ query }}<eos>"""
```

!!! note
Choosing BOS and EOS based on the model is better achieved by using [special variables](special_tokens.md).

0 comments on commit d2eeaf9

Please sign in to comment.