-
Notifications
You must be signed in to change notification settings - Fork 10
Closed
Description
Thanks for the port of ernie layout to torch!
I have two question regarding ernie-layoutx-base-uncased conversion:
- How can I obtain the corresponding tokenizer files (as e.g. given in
Norm/ERNIE-Layout-Pytorch)? - Do you know if
ErnieLayoutTokenizercan be replaced by a fast tokenizer? E.g. use xlnet tokenizer (that uses sentencepiece tokenizer, as well) instead. For my current pipeline, I need some attributes/methods that are currently not supported byErnieLayoutTokenizer.
Metadata
Metadata
Assignees
Labels
No labels