Failing diacriticization tests

[ronaldtse]

There were two diacriticization tests that are currently failing (and I commented them out).

@AhMohsen46 can you help check?

  1) Rababa::Diacritizer diacriticizes # گيله پسمير الجديد 34
     Failure/Error: expect(diacritizer.diacritize_text(source)).to eq target
     
       expected: "# گيَلِهُ پسُمِيْرٌ الجَدِيدُ 34"
            got: "# گيَلْهُ پسَمِيرٌ الجَدِيدَ 34"
     
       (compared using ==)
     # ./spec/rababa/diacritizer_spec.rb:49:in `block (3 levels) in <top (required)>'

  2) Rababa::Diacritizer diacriticizes 26 سبتمبر العقبة
     Failure/Error: expect(diacritizer.diacritize_text(source)).to eq target
     
       expected: "26 سَبْتَمْبَرِ العَقَبَة"
            got: "26 سَبْتَمْبَرَ العَقْبَة"
     
       (compared using ==)
     # ./spec/rababa/diacritizer_spec.rb:49:in `block (3 levels) in <top (required)>'

Finished in 12.58 seconds (files took 0.36641 seconds to load)
6 examples, 2 failures

[gilgameshjw]

ran first example:
"# گيَلِهُ پسُمِيْرٌ الجَدِيدُ 34"

second one:
"26 سَبْتَمْبَرِ العَقَبَة"

Why is your output different?

I have attached the code for replication. Just change name .txt -> .rb to run.
run_reconcile.txt

[ronaldtse]

@gilgameshjw you can see the tests failing in this PR: #26 . Please help directly fix that branch if possible. Thanks!

e.g. failure: https://github.com/interscript/rababa/pull/26/checks?check_run_id=3236987149

[ronaldtse]

From @gilgameshjw for @AhMohsen46 :

are these correct diacritizations?:
26 سَبْتَمْبَرِ العَقَبَةِ
# گيَلِهُ پسُمِيرٌ الجَدِيدُ 34

Python and Ruby give different results. Which is worrying. Might be misalignment of vectors or ONNX...

[ronaldtse]

Closing for now, we have a pending PR #26.