Store transliterations in database

[harjot1singh]

Summary

We need to also store transliterations in the database for primary content!

This does depend on Gurmukhi Utils being updated to support the transliterations

Approach

No response

[harjot1singh]

Based on analysis (thank you Cursor):

Current data:
141,264 lines in database
Average Gurmukhi line: ~20-30 characters
Average English transliteration: ~25-35 characters
Storage calculation for 1M lines:
Per line:
Gurmukhi: ~25 chars × 3 bytes (UTF-8) = 75 bytes
English transliteration: ~30 chars × 1 byte (ASCII) = 30 bytes
Total per line: ~105 bytes
For 1M lines:
Raw transliteration data: 1,000,000 × 30 bytes = 30 MB
With SQLite overhead (~20%): ~36 MB
With indexing overhead (~10%): ~40 MB
Comparison:
Current DB has 141K lines
1M lines = 7x current size
Transliteration storage = negligible (40MB vs GB+ for full DB)
Verdict: Storage cost is minimal. 40MB for 1M transliterations is peanuts compared to the performance benefits. Worth storing.