---
updated-dependencies:
- dependency-name: on-headers
  dependency-version: 1.1.0
  dependency-type: indirect
- dependency-name: compression
  dependency-version: 1.8.1
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

…r for WebAssembly Compatibility (#1758)

* Add from_bytes and read_bytes method to WordPiece

* Change wordpiece  method return value

---------

Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>

* update

* update

* updates

* up

* oikay

* use stream input

* nice all test pass?

* fmt

* dev

* rename

* simplify a hell lot

* proper testing

* fix inti

* fix test

* nits

* make clippy happy now

* fmt fml

* remove the prints

* fix gate

Update broken wikitext-103 and tokenizers-pipeline links

Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>

* add async bindings

* update based on review!

* us hf internal testing for testing

* reduce burden for the CI

* asyn is not necessarily fast

* remove comments

---------

Co-authored-by: Arthur <arthur.zucker@gmail.com>
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>

This reverts commit b43d8d7.

* chore(trainers): add __init__ to fix python type check errors

* restore

* chore(trainer): add and improve trainer signature

* clean fix

* chore(fmt): fix cargo fmt error

---------

Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>

* Test hfh 1.0 rc0

* Tokenizers works on both 0.x and 1.x versions