!sed -i "113s/.*/ byte_fallback=False,/" ./llama2.c/tinystories.py
Vocabulary size is smaller than required_chars. 100 vs 105.
!cd llama2.c && python tinystories.py train_vocab --vocab_size=105
!cd llama2.c/data && tar -czvf tok105.tar.gz tok105
tok105
Folders and files
Name | Name | Last commit date | ||
---|---|---|---|---|
parent directory.. | ||||