We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
There was an error while loading. Please reload this page.
1 parent da1cc3b commit 57eb8d7Copy full SHA for 57eb8d7
docs/source-doc-builder/api/normalizers.mdx
@@ -2,9 +2,9 @@
2
3
<tokenizerslangcontent>
4
<python>
5
-## BertNormalizer
+## ByteLevel
6
7
-[[autodoc]] tokenizers.normalizers.BertNormalizer
+[[autodoc]] tokenizers.normalizers.ByteLevel
8
9
## Lowercase
10
@@ -53,11 +53,16 @@
53
## StripAccents
54
55
[[autodoc]] tokenizers.normalizers.StripAccents
56
+
57
+## BertNormalizer
58
59
+[[autodoc]] tokenizers.normalizers.BertNormalizer
60
61
</python>
62
<rust>
63
The Rust API Reference is available directly on the [Docs.rs](https://docs.rs/tokenizers/latest/tokenizers/) website.
64
</rust>
65
<node>
66
The node API has not been documented yet.
67
</node>
-</tokenizerslangcontent>
68
+</tokenizerslangcontent>
0 commit comments