Should the tokenizer be refined if the model is finetuned on new language dataset?

[Pandede]

I've finetuned the model around 30k samples on Thai and Vietnamese language.
After 5 hours of training, the training metrices start to converge and result in poor performance (top-5 accuracy around 0.34),

I've followed the instruction about finetuning the model and nothing should go wrong.

Thus, I try to investigate the root cause and observe this:

from fish_speech.tokenizer import FishTokenizer, BOS_TOKEN, EOS_TOKEN

tokenizer.encode(f"{BOS_TOKEN}你好，世界！{EOS_TOKEN}") # [100000, 8403, 259, 2090, 7563, 100001]
tokenizer.encode(f"{BOS_TOKEN}ภาษาไทย{EOS_TOKEN}") # [100000, 88518, 160, 88518, 178, 88518, 169, 88518, 178, 224, 4382, 88518, 151, 88518, 162, 100001]

When encoding Chinese characters, everything seems fine, 4 characters to 4 tokens. While encoding the Thai characters, something goes wrong, 7 characters to 14 tokens. Would this introduce any issue?