Add support for other numeral systems.

[noviluni]

At this moment, the main number-parser goal is to return the number equivalences from different languages, but only when those words are representing the number using the "decimal numeral system" (https://en.wikipedia.org/wiki/Decimal).

However, there are some numeral systems that don't rely on the decimal numeral system and uses other structures. That's the case of the Roman Numeral System (https://en.wikipedia.org/wiki/Numeral_system) or the Chinese/Japanese/Korean/Vietnamese Numeral System (https://en.wikipedia.org/wiki/Chinese_numerals and https://en.wikipedia.org/wiki/Suzhou_numerals).

We could probably add support for them in a future version, as they will probably need another kind of parser.

For more on this, you can also check this: https://en.wikipedia.org/wiki/Numeral_system

[noviluni]

Here ("Using neither" section): https://en.wikipedia.org/wiki/Long_and_short_scales#Using_neither there is a list of different numeral systems.

[noviluni]

BTW, this resource has a lot of useful information: https://www.languagesandnumbers.com/site-map/en/

[AmPhIbIaN26]

@noviluni I had asked Adrian about this issue he advised me that I could add features in number parser, so would you recommend to take this issue for my GSoC 2021 prposal?

[AmPhIbIaN26]

@noviluni i have decided to take this for my GSoC proposal, I understood how one can wrote roman numerals in English alphabet and how one would implement it, I wanted to ask about Chinese/Japanese/Korean/Vietnamese numeral system. What I think is that it would parse the number which are input in symbols form the respective languages, or do I do it with the help of Unicode?

[Gallaecio]

What I think is that it would parse the number which are input in symbols form the respective languages, or do I do it with the help of Unicode?

I did not understand this question. To me “symbols form” and “Unicode” are basically the same here, the input would be the symbols as a Python string.

[AmPhIbIaN26]

I meant to ask was should i be using "零" or "U+96F6", cause i tried some test code in python and it did print "零" as it is.
I assumed if one was to use this feature and take input from user or any other application which uses number parser they would use it in "零" form, so that it would be directly handled instead of being converted to Unicode first.

[noviluni]

Hi @AmPhIbIaN26

thanks for showing interest in fixing this, yes, I think this could be feasible for a GSoC proposal.

I'm not sure about your last question. In Python 3, unicode is enabled by default, so you don't need to handle it. 零 is unicode the same way than "U+96F6". I don't think you need to convert anything.

[AmPhIbIaN26]

Ohh ok thanks, I'll work on it

[AmPhIbIaN26]

@noviluni I am doing a bit of research on this topic first, so do you only want conversion of numbers to int or a readable string. Or maybe could you give me examples of what kinds of input do you want it to take and what should it return.

[AmPhIbIaN26]

i looked up on how to parse roman numerals, it can be added to the current parser, but for the case of these other numerals for Chinese Japanese and other languages should I create a new parser??

[Gallaecio]

do you only want conversion of numbers to int or a readable string.

If you mean whether the goal is to support new numerical systems in parse_number or parse, I would say that ideally both. And also parse_ordinal and parse_fraction if relevant.

That said, maybe it makes sense to prioritize parse, the one that replaces the numbers in-place in the strings, as I imagine that is the method more prone to be used by the likes of Dateparser and price-parser.

maybe could you give me examples of what kinds of input do you want it to take and what should it return.

It’s hard to give specific examples without some knowledge of those other numeral systems, but for Roman numbers I imagine something like:

>>> parse('Built in MDCCLXXVI')
'Built in 1776'
>>> parse_number('MDCCLXXVI')
1776

i looked up on how to parse roman numerals, it can be added to the current parser, but for the case of these other numerals for Chinese Japanese and other languages should I create a new parser??

I’m not very familiar with the internals of number-parser, but I would aim for it to be possible for users to limit which numeral systems are considered by number-parser in a given call. For example, allow users to use number-parser functions limiting them to decimal numbers, roman numbers, or any combination or numeral systems.

So I would say that, ideally, the parsers should be as independent as possible.

[AmPhIbIaN26]

So I add roman to parser and then make a new parser for the other languages, right?

[Gallaecio]

So I add roman to parser and then make a new parser for the other languages, right?

That’s the opposite of what I meant, but I’m also starting to think that I may be misunderstanding what you mean here by “parser”.

How would each of the options (extending the existing parser with Roman number support vs adding a separate parser for Roman number support) look like for users, API-wise? Are you talking about creating separate user-level functions for other numeral systems, as opposed to have the existing functions like parse and parse_number to support them?