Skip to content

Commit

Permalink
fix hyperlink
Browse files Browse the repository at this point in the history
  • Loading branch information
cmacdonald authored Nov 10, 2022
1 parent 7e97dde commit 10d0a45
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion docs/terrier-indexing.rst
Original file line number Diff line number Diff line change
Expand Up @@ -231,7 +231,7 @@ Sometimes you want more fine-grained control over the tokenisation directly with
{'docno' : 'd2', 'toks' : {'a' : 2, '##2' : 1}}
])

This allows tokenisation using, for instance, the `HuggingFace tokenizers <https://huggingface.co/docs/transformers/fast_tokenizers>_`::
This allows tokenisation using, for instance, the `HuggingFace tokenizers <https://huggingface.co/docs/transformers/fast_tokenizers>`_::

iter_indexer = pt.IterDictIndexer("./pretokindex", meta={'docno': 20}, threads=1, pretokenised=True)
from transformers import AutoTokenizer
Expand Down

0 comments on commit 10d0a45

Please sign in to comment.