sqlitefts-python provides binding for tokenizer of SQLite Full-Text search(FTS3/4). It allows you to write tokenizers in Python.
SQLite has Full-Text search feature FTS3/FTS4 along with some predefined tokenizers. It is easy to use and has enough functionality. Python has a built-in SQLite module, so that it is easy to use and deploy. You don't need anything else to full-text search.
But... Predefined tokenizers are not enough for some languages including Japanese. Also it is not easy to write own tokenizers. With this modules, you can write tokenizers in Python without any extra modules and C compiler.
- FTS3/4 enabled SQLite
This software is released under the MIT License, see LICENSE.