-
-
Notifications
You must be signed in to change notification settings - Fork 5
Added support for spelling correction in our bindings. #130
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #130 +/- ##
============================================
+ Coverage 94.29% 94.41% +0.12%
- Complexity 258 262 +4
============================================
Files 49 51 +2
Lines 368 376 +8
Branches 3 3
============================================
+ Hits 347 355 +8
Misses 18 18
Partials 3 3 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
0901bc8 to
ba6e721
Compare
188c415 to
ff8366d
Compare
|
@kelson42 For the terminating due to uncaught exception of type Xapian::InvalidArgumentError
2025-11-07 18:30:24.573 15528-15528 libc org.kiwix.kiwixmobile A Fatal signal 6 (SIGABRT), code -1 (SI_QUEUE) in tid 15528 (wix.kiwixmobile), pid 15528 (wix.kiwixmobile)
2025-11-07 18:30:25.098 15745-15745 DEBUG crash_dump64 A Cmdline: org.kiwix.kiwixmobile
2025-11-07 18:30:25.098 15745-15745 DEBUG crash_dump64 A pid: 15528, tid: 15528, name: wix.kiwixmobile >>> org.kiwix.kiwixmobile <<<
2025-11-07 18:30:25.098 15745-15745 DEBUG crash_dump64 A #01 pc 00000000000a0210 /data/app/~~wFNDBzWVSZG5nNE4yU9tyg==/org.kiwix.kiwixmobile-rdEwlJuELnkO-KnMI9v9-w==/base.apk!libc++_shared.so (offset 0xbf0000) (BuildId: ea306390624df9242cca21aeec76a0427683bd72)
2025-11-07 18:30:25.098 15745-15745 DEBUG crash_dump64 A #02 pc 000000000009eec8 /data/app/~~wFNDBzWVSZG5nNE4yU9tyg==/org.kiwix.kiwixmobile-rdEwlJuELnkO-KnMI9v9-w==/base.apk!libc++_shared.so (offset 0xbf0000) (BuildId: ea306390624df9242cca21aeec76a0427683bd72)
2025-11-07 18:30:25.098 15745-15745 DEBUG crash_dump64 A #03 pc 000000000009f374 /data/app/~~wFNDBzWVSZG5nNE4yU9tyg==/org.kiwix.kiwixmobile-rdEwlJuELnkO-KnMI9v9-w==/base.apk!libc++_shared.so (offset 0xbf0000) (BuildId: ea306390624df9242cca21aeec76a0427683bd72)
2025-11-07 18:30:25.098 15745-15745 DEBUG crash_dump64 A #04 pc 00000000000bae6c /data/app/~~wFNDBzWVSZG5nNE4yU9tyg==/org.kiwix.kiwixmobile-rdEwlJuELnkO-KnMI9v9-w==/base.apk!libc++_shared.so (offset 0xbf0000) (__cxa_rethrow+220) (BuildId: ea306390624df9242cca21aeec76a0427683bd72)
2025-11-07 18:30:25.098 15745-15745 DEBUG crash_dump64 A #05 pc 00000000006c660c /data/app/~~wFNDBzWVSZG5nNE4yU9tyg==/org.kiwix.kiwixmobile-rdEwlJuELnkO-KnMI9v9-w==/base.apk!libkiwix.so (offset 0x165c000)
2025-11-07 18:30:25.098 15745-15745 DEBUG crash_dump64 A #06 pc 00000000006c7ef8 /data/app/~~wFNDBzWVSZG5nNE4yU9tyg==/org.kiwix.kiwixmobile-rdEwlJuELnkO-KnMI9v9-w==/base.apk!libkiwix.so (offset 0x165c000)
2025-11-07 18:30:25.098 15745-15745 DEBUG crash_dump64 A #07 pc 00000000006c7ddc /data/app/~~wFNDBzWVSZG5nNE4yU9tyg==/org.kiwix.kiwixmobile-rdEwlJuELnkO-KnMI9v9-w==/base.apk!libkiwix.so (offset 0x165c000)
2025-11-07 18:30:25.098 15745-15745 DEBUG crash_dump64 A #08 pc 0000000000656cb4 /data/app/~~wFNDBzWVSZG5nNE4yU9tyg==/org.kiwix.kiwixmobile-rdEwlJuELnkO-KnMI9v9-w==/base.apk!libkiwix.so (offset 0x165c000) (Xapian::WritableDatabase::commit()+96)
2025-11-07 18:30:25.098 15745-15745 DEBUG crash_dump64 A #09 pc 000000000064959c /data/app/~~wFNDBzWVSZG5nNE4yU9tyg==/org.kiwix.kiwixmobile-rdEwlJuELnkO-KnMI9v9-w==/base.apk!libkiwix.so (offset 0x165c000)
2025-11-07 18:30:25.098 15745-15745 DEBUG crash_dump64 A #10 pc 0000000000649164 /data/app/~~wFNDBzWVSZG5nNE4yU9tyg==/org.kiwix.kiwixmobile-rdEwlJuELnkO-KnMI9v9-w==/base.apk!libkiwix.so (offset 0x165c000)
2025-11-07 18:30:25.098 15745-15745 DEBUG crash_dump64 A #11 pc 000000000064907c /data/app/~~wFNDBzWVSZG5nNE4yU9tyg==/org.kiwix.kiwixmobile-rdEwlJuELnkO-KnMI9v9-w==/base.apk!libkiwix.so (offset 0x165c000) (kiwix::SpellingsDB::SpellingsDB(zim::Archive const&, std::__ndk1::__fs::filesystem::path)+60)
2025-11-07 18:30:25.098 15745-15745 DEBUG crash_dump64 A #12 pc 000000000002ec90 /data/app/~~wFNDBzWVSZG5nNE4yU9tyg==/org.kiwix.kiwixmobile-rdEwlJuELnkO-KnMI9v9-w==/base.apk!libkiwix_wrapper.so (offset 0xb3c000) (void std::__ndk1::allocator<kiwix::SpellingsDB>::construct[abi:ne180000]<kiwix::SpellingsDB, zim::Archive&, std::__ndk1::basic_string<char, std::__ndk1::char_traits<char>, std::__ndk1::allocator<char>>&>(kiwix::SpellingsDB*, zim::Archive&, std::__ndk1::basic_string<char, std::__ndk1::char_traits<char>, std::__ndk1::allocator<char>>&)+96) (BuildId: 82d512a6b8c471e83e6644edeafe6ee1d9ad9b07)
2025-11-07 18:30:25.098 15745-15745 DEBUG crash_dump64 A #13 pc 000000000002dd6c /data/app/~~wFNDBzWVSZG5nNE4yU9tyg==/org.kiwix.kiwixmobile-rdEwlJuELnkO-KnMI9v9-w==/base.apk!libkiwix_wrapper.so (offset 0xb3c000) (Java_org_kiwix_libkiwix_SpellingsDB_setNativeSpellingsDB+284) (BuildId: 82d512a6b8c471e83e6644edeafe6ee1d9ad9b07)
2025-11-07 18:30:25.098 15745-15745 DEBUG crash_dump64 A #24 pc 00000000000134f0 <anonymous:7c0e6c7000> (org.kiwix.kiwixmobile.core.search.viewmodel.SearchViewModel.prepareSpellingsDatabase+0)
2025-11-07 18:30:25.098 15745-15745 DEBUG crash_dump64 A #29 pc 00000000000127d0 <anonymous:7c0e6c7000> (org.kiwix.kiwixmobile.core.search.viewmodel.SearchViewModel.access$prepareSpellingsDatabase+0)
2025-11-07 18:30:25.098 15745-15745 DEBUG crash_dump64 A #34 pc 0000000000011aec <anonymous:7c0e6c7000> (org.kiwix.kiwixmobile.core.search.viewmodel.SearchViewModel$4.invokeSuspend+0)
2025-11-07 18:30:25.099 15745-15745 DEBUG crash_dump64 A #79 pc 0000000000013934 <anonymous:7c0e6c7000> (org.kiwix.kiwixmobile.core.search.viewmodel.SearchViewModel.<init>+0)
2025-11-07 18:30:25.099 15745-15745 DEBUG crash_dump64 A #84 pc 000000000001266c <anonymous:7c0e6c7000> (org.kiwix.kiwixmobile.core.search.viewmodel.SearchViewModel_Factory.newInstance+0)
2025-11-07 18:30:25.099 15745-15745 DEBUG crash_dump64 A #89 pc 0000000000012600 <anonymous:7c0e6c7000> (org.kiwix.kiwixmobile.core.search.viewmodel.SearchViewModel_Factory.get+0)
2025-11-07 18:30:25.099 15745-15745 DEBUG crash_dump64 A #94 pc 00000000000125d0 <anonymous:7c0e6c7000> (org.kiwix.kiwixmobile.core.search.viewmodel.SearchViewModel_Factory.get+0)
2025-11-07 18:30:25.099 15745-15745 DEBUG crash_dump64 A #99 pc 000000000000fa74 <anonymous:7c0a629000> (org.kiwix.kiwixmobile.core.ViewModelFactory.create+0)
2025-11-07 18:30:25.099 15745-15745 DEBUG crash_dump64 A #139 pc 00000000000d0d00 <anonymous:7922bea000> (org.kiwix.kiwixmobile.core.search.SearchFragment.searchViewModel_delegate$lambda$0+0)
2025-11-07 18:30:25.099 15745-15745 DEBUG crash_dump64 A #144 pc 00000000000d0cac <anonymous:7922bea000> (org.kiwix.kiwixmobile.core.search.SearchFragment.$r8$lambda$PTNMLSrZaC0QOxYFFooM4hB50IE+0)
2025-11-07 18:30:25.099 15745-15745 DEBUG crash_dump64 A #149 pc 00000000000cd404 <anonymous:7922bea000> (org.kiwix.kiwixmobile.core.search.SearchFragment$$ExternalSyntheticLambda11.invoke+0)
2025-11-07 18:30:25.099 15745-15745 DEBUG crash_dump64 A #159 pc 00000000000d0cc8 <anonymous:7922bea000> (org.kiwix.kiwixmobile.core.search.SearchFragment.getSearchViewModel+0)
2025-11-07 18:30:25.099 15745-15745 DEBUG crash_dump64 A #164 pc 00000000000d15a4 <anonymous:7922bea000> (org.kiwix.kiwixmobile.core.search.SearchFragment.onViewCreated+0)2nd crash terminating due to uncaught exception of type Xapian::DatabaseCreateError
2025-11-07 18:20:01.205 14747-14747 libc org.kiwix.kiwixmobile A Fatal signal 6 (SIGABRT), code -1 (SI_QUEUE) in tid 14747 (wix.kiwixmobile), pid 14747 (wix.kiwixmobile)
2025-11-07 18:20:01.794 14911-14911 DEBUG pid-14911 A Cmdline: org.kiwix.kiwixmobile
2025-11-07 18:20:01.794 14911-14911 DEBUG pid-14911 A pid: 14747, tid: 14747, name: wix.kiwixmobile >>> org.kiwix.kiwixmobile <<<
2025-11-07 18:20:01.794 14911-14911 DEBUG pid-14911 A #01 pc 00000000000a0210 /data/app/~~wFNDBzWVSZG5nNE4yU9tyg==/org.kiwix.kiwixmobile-rdEwlJuELnkO-KnMI9v9-w==/base.apk!libc++_shared.so (offset 0xbf0000) (BuildId: ea306390624df9242cca21aeec76a0427683bd72)
2025-11-07 18:20:01.794 14911-14911 DEBUG pid-14911 A #02 pc 000000000009eec8 /data/app/~~wFNDBzWVSZG5nNE4yU9tyg==/org.kiwix.kiwixmobile-rdEwlJuELnkO-KnMI9v9-w==/base.apk!libc++_shared.so (offset 0xbf0000) (BuildId: ea306390624df9242cca21aeec76a0427683bd72)
2025-11-07 18:20:01.794 14911-14911 DEBUG pid-14911 A #03 pc 000000000009f374 /data/app/~~wFNDBzWVSZG5nNE4yU9tyg==/org.kiwix.kiwixmobile-rdEwlJuELnkO-KnMI9v9-w==/base.apk!libc++_shared.so (offset 0xbf0000) (BuildId: ea306390624df9242cca21aeec76a0427683bd72)
2025-11-07 18:20:01.794 14911-14911 DEBUG pid-14911 A #04 pc 00000000000babc8 /data/app/~~wFNDBzWVSZG5nNE4yU9tyg==/org.kiwix.kiwixmobile-rdEwlJuELnkO-KnMI9v9-w==/base.apk!libc++_shared.so (offset 0xbf0000) (BuildId: ea306390624df9242cca21aeec76a0427683bd72)
2025-11-07 18:20:01.794 14911-14911 DEBUG pid-14911 A #05 pc 00000000000bab3c /data/app/~~wFNDBzWVSZG5nNE4yU9tyg==/org.kiwix.kiwixmobile-rdEwlJuELnkO-KnMI9v9-w==/base.apk!libc++_shared.so (offset 0xbf0000) (__cxa_throw+128) (BuildId: ea306390624df9242cca21aeec76a0427683bd72)
2025-11-07 18:20:01.794 14911-14911 DEBUG pid-14911 A #06 pc 00000000006c43ac /data/app/~~wFNDBzWVSZG5nNE4yU9tyg==/org.kiwix.kiwixmobile-rdEwlJuELnkO-KnMI9v9-w==/base.apk!libkiwix.so (offset 0x165c000)
2025-11-07 18:20:01.794 14911-14911 DEBUG pid-14911 A #07 pc 00000000006c7b60 /data/app/~~wFNDBzWVSZG5nNE4yU9tyg==/org.kiwix.kiwixmobile-rdEwlJuELnkO-KnMI9v9-w==/base.apk!libkiwix.so (offset 0x165c000)
2025-11-07 18:20:01.794 14911-14911 DEBUG pid-14911 A #08 pc 00000000006ac2e4 /data/app/~~wFNDBzWVSZG5nNE4yU9tyg==/org.kiwix.kiwixmobile-rdEwlJuELnkO-KnMI9v9-w==/base.apk!libkiwix.so (offset 0x165c000) (Xapian::WritableDatabase::WritableDatabase(std::__ndk1::basic_string<char, std::__ndk1::char_traits<char>, std::__ndk1::allocator<char>> const&, int, int)+1496)
2025-11-07 18:20:01.794 14911-14911 DEBUG pid-14911 A #09 pc 00000000006494ec /data/app/~~wFNDBzWVSZG5nNE4yU9tyg==/org.kiwix.kiwixmobile-rdEwlJuELnkO-KnMI9v9-w==/base.apk!libkiwix.so (offset 0x165c000)
2025-11-07 18:20:01.794 14911-14911 DEBUG pid-14911 A #10 pc 0000000000649164 /data/app/~~wFNDBzWVSZG5nNE4yU9tyg==/org.kiwix.kiwixmobile-rdEwlJuELnkO-KnMI9v9-w==/base.apk!libkiwix.so (offset 0x165c000)
2025-11-07 18:20:01.794 14911-14911 DEBUG pid-14911 A #11 pc 000000000064907c /data/app/~~wFNDBzWVSZG5nNE4yU9tyg==/org.kiwix.kiwixmobile-rdEwlJuELnkO-KnMI9v9-w==/base.apk!libkiwix.so (offset 0x165c000) (kiwix::SpellingsDB::SpellingsDB(zim::Archive const&, std::__ndk1::__fs::filesystem::path)+60)
2025-11-07 18:20:01.794 14911-14911 DEBUG pid-14911 A #12 pc 000000000002ec90 /data/app/~~wFNDBzWVSZG5nNE4yU9tyg==/org.kiwix.kiwixmobile-rdEwlJuELnkO-KnMI9v9-w==/base.apk!libkiwix_wrapper.so (offset 0xb3c000) (void std::__ndk1::allocator<kiwix::SpellingsDB>::construct[abi:ne180000]<kiwix::SpellingsDB, zim::Archive&, std::__ndk1::basic_string<char, std::__ndk1::char_traits<char>, std::__ndk1::allocator<char>>&>(kiwix::SpellingsDB*, zim::Archive&, std::__ndk1::basic_string<char, std::__ndk1::char_traits<char>, std::__ndk1::allocator<char>>&)+96) (BuildId: 82d512a6b8c471e83e6644edeafe6ee1d9ad9b07)
2025-11-07 18:20:01.794 14911-14911 DEBUG pid-14911 A #13 pc 000000000002dd6c /data/app/~~wFNDBzWVSZG5nNE4yU9tyg==/org.kiwix.kiwixmobile-rdEwlJuELnkO-KnMI9v9-w==/base.apk!libkiwix_wrapper.so (offset 0xb3c000) (Java_org_kiwix_libkiwix_SpellingsDB_setNativeSpellingsDB+284) (BuildId: 82d512a6b8c471e83e6644edeafe6ee1d9ad9b07)
2025-11-07 18:20:01.794 14911-14911 DEBUG pid-14911 A #24 pc 00000000000134f0 <anonymous:7c0e6c7000> (org.kiwix.kiwixmobile.core.search.viewmodel.SearchViewModel.prepareSpellingsDatabase+0)
2025-11-07 18:20:01.794 14911-14911 DEBUG pid-14911 A #29 pc 00000000000127d0 <anonymous:7c0e6c7000> (org.kiwix.kiwixmobile.core.search.viewmodel.SearchViewModel.access$prepareSpellingsDatabase+0)
2025-11-07 18:20:01.794 14911-14911 DEBUG pid-14911 A #34 pc 0000000000011aec <anonymous:7c0e6c7000> (org.kiwix.kiwixmobile.core.search.viewmodel.SearchViewModel$4.invokeSuspend+0)
2025-11-07 18:20:01.794 14911-14911 DEBUG pid-14911 A #84 pc 0000000000013934 <anonymous:7c0e6c7000> (org.kiwix.kiwixmobile.core.search.viewmodel.SearchViewModel.<init>+0)
2025-11-07 18:20:01.794 14911-14911 DEBUG pid-14911 A #89 pc 000000000001266c <anonymous:7c0e6c7000> (org.kiwix.kiwixmobile.core.search.viewmodel.SearchViewModel_Factory.newInstance+0)
2025-11-07 18:20:01.794 14911-14911 DEBUG pid-14911 A #94 pc 0000000000012600 <anonymous:7c0e6c7000> (org.kiwix.kiwixmobile.core.search.viewmodel.SearchViewModel_Factory.get+0)
2025-11-07 18:20:01.794 14911-14911 DEBUG pid-14911 A #99 pc 00000000000125d0 <anonymous:7c0e6c7000> (org.kiwix.kiwixmobile.core.search.viewmodel.SearchViewModel_Factory.get+0)
2025-11-07 18:20:01.794 14911-14911 DEBUG pid-14911 A #104 pc 000000000000fa74 <anonymous:7c0a629000> (org.kiwix.kiwixmobile.core.ViewModelFactory.create+0)
2025-11-07 18:20:01.794 14911-14911 DEBUG pid-14911 A #144 pc 00000000000d0d00 <anonymous:7922ceb000> (org.kiwix.kiwixmobile.core.search.SearchFragment.searchViewModel_delegate$lambda$0+0)
2025-11-07 18:20:01.794 14911-14911 DEBUG pid-14911 A #149 pc 00000000000d0cac <anonymous:7922ceb000> (org.kiwix.kiwixmobile.core.search.SearchFragment.$r8$lambda$PTNMLSrZaC0QOxYFFooM4hB50IE+0)
2025-11-07 18:20:01.795 14911-14911 DEBUG pid-14911 A #154 pc 00000000000cd404 <anonymous:7922ceb000> (org.kiwix.kiwixmobile.core.search.SearchFragment$$ExternalSyntheticLambda11.invoke+0)
2025-11-07 18:20:01.795 14911-14911 DEBUG pid-14911 A #164 pc 00000000000d0cc8 <anonymous:7922ceb000> (org.kiwix.kiwixmobile.core.search.SearchFragment.getSearchViewModel+0)
2025-11-07 18:20:01.795 14911-14911 DEBUG pid-14911 A #169 pc 00000000000d15a4 <anonymous:7922ceb000> (org.kiwix.kiwixmobile.core.search.SearchFragment.onViewCreated+0)So to cactching these errors i have palced a fix in 8dad899 so after adding this code the native crash stops with this error Attempting to create spelling DB with path: /storage/emulated/0/Android/data/org.kiwix.kiwixmobile/cache
2025-11-07 19:22:49.229 20799-20799 kiwix org.kiwix.kiwixmobile E Failed to initialize SpellingsDB: Unknown native exception occurred while creating SpellingsDB
java.lang.Exception: Unknown native exception occurred while creating SpellingsDBBut this error showing our custom message written in the bindings not the actual one. @veloman-yunkan Can we improve this error handling(To show the proper error message)? Do you have any idea about this? |
|
When does it crash? At indexation time? Are you sure the index is properly written? Is the place you want to write the index the right place? Is the casche dire properly existing and is writtable? |
@kelson42 It is crashing when creating the It seems at that time the issue with the directory since i was using the same cache dir that used for loadiung the asset(That was removing when the reader screen is deatteched from window so due to that a native crash occures because the dir was not exist). But after that, i have provided it dedicated app-specific for this(Removing it properly after use). Then these native errors was gone. But, if there is some type of error while creating the Failed to initialize SpellingsDB: Unknown native exception occurred while creating SpellingsDB
java.lang.Exception: Unknown native exception occurred while creating SpellingsDB |
|
@MohitMaliFtechiz Please don't optimize anything at first. This is only to be used for dwds, anything else should not use use this feature for the moment. Not kiwix and no other custom app. You will have to create an option in the custom app json to make this configurable, again here look at kiwix apple. You don't have to make it smart. The first time you get the db instance, if not existing it will create it. So it will be slow and will maybe take 30s. This is why it is important to do that in background thread. Any other tiem you will get the db it will be quick as the index will already have been created. Once you have the db instance, it should be attached to the book so you can reuse it each time needed without having to ask to recreate it each time. All of this is explained in the issue I have recommend you to read. |
* Created `spellingdb.cpp`, and `SpellingsDB` files for this. * Added test cases for spelling corrections. * Fixed: CodeFactor issue.
8dad899 to
43616f9
Compare
@kelson42 Thank you for pointing out all the details. Yes, i have read this on the apple ticket, and i have created the the DB on IO thread so that app behavious normally.
Yes, i have configure the DB object like this so that it can easily resusable without recreating.
@kelson42 I have reverted the change. |
| TestSpellingsDB testSpellingsDB = new TestSpellingsDB(archive, cacheDir.getAbsolutePath()); | ||
| String[] spellingCorrections = testSpellingsDB.getSpellingCorrections("Test ZIM fileeee", 1); | ||
| assertEquals("Test ZIM file", spellingCorrections[0]); | ||
| String[] spellingCorrections1 = testSpellingsDB.getSpellingCorrections("Test ZIM f", 1); | ||
| assertEquals("Test ZIM file", spellingCorrections1[0]); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A fraud implementation of getSpellingCorrections() always returning the string "Test ZIM file" packaged into a single element array will pass this test. In fact, it may return an array containing more than one element and the test will still pass.
Tests should be written in a way ruling out such temporary implementations (acceptable in TDD approach). At a minimum, you should cover the case when no spelling correction is returned.
See kiwix/libkiwix#1230.
spellingdb.cpp, andSpellingsDBfiles for this.