Skip to content

Latest commit

 

History

History

extra_tokens

Folders and files

NameName
Last commit message
Last commit date

parent directory

..
 
 
 
 
 
 
 
 

Extra tokens fies

Just some files to store the extra tokens I want to use for training multilingual Phi-1.5 models.


日語常用漢字表.csv

Source link

Common kanji characters in Japanese.


教育部4808個常用字.csv

Source link

Common characters in Traditional Chinese.