Does the Heuristic Query Matching Crawling Strategies work on other languages than English?

[Pencoding1]

Hello everyone,

I am wondering if standard query matching strategies are effective for languages other than English or those commonly discussed here—specifically Vietnamese.

I am investigating heuristic-based crawling strategies, as my research suggests that current models struggle with Vietnamese due to its inherent complexity (e.g., diacritics/accents and compound word structures where multiple syllables form a single semantic unit).

For example:

Học Sinh = Student
but the algorithm could see the word "Học Sinh" as two seperate words: "Học" and "Sinh" which are also has meaning of "study" and "biology" ("Sinh" in "Sinh Học") or "animal" ("Sinh" in "sinh vật")

Could you suggest alternative ways to improve query scoring algorithms? Additionally, am I mistaken in my assumptions—does BM25 actually perform well for Vietnamese in practice?

[Pencoding1]

For more clear that I know that with these complicated query I should use Embedding strategies, however I would like to see light-weight approaches first because I pior my crawler works as fast as possible.