Support for Pypy3?

[noviluni]

I tried to add the Pypy3 env to tox and Travis and the tests were failing because the regex package doesn't support Pypy/Pypy3 officially.

Should we add support for Pypy3? should we try to remove the dependency on the regex package (I tried in the past and it's not as simple as it seems)?

Feedback welcomed

[lopuhin]

I think pypy build passes in master https://travis-ci.org/github/scrapinghub/dateparser/jobs/708282459 and from regex bugtracker it seems it works on pypy3 (although there may be some issues with complex unicode handling), so to me it seems that removing regex package is not the best option.

[noviluni]

Hi @lopuhin Thanks for your answer

Pypy was passing because when testing we were installing a pinned version of the regex package from the requirements.txt file, but it didn't work with the last regex version. There was an issue about this but it wasn't investigated so far: #565, and after removing the requirements.txt file we found the same issue, so I pinned a working regex version in the setup.py file to allow the pipelines to pass: 'regex==2019.01.24; platform.python_implementation == "PyPy"', (added in: #740)

As we are releasing a new version without Python 2 support we will remove support for Pypy too. I tried adding the Pypy3 env but the pipelines are failing. You can see more in the first comment here: #746 As you can see is not just a test failing.

I would love to get a PR fixing those issues, however, I've never seen any issue requesting for pypy3 support, so:

• Do you think is required / really useful to add Pypy3 support?
• It's possible to fix it?

About removing the regex package, I know it's not trivial (and the performance could be affected), but I found some projects that stoped using dateparser because of regex:

• Docker Build & CI Refinements python-discord/bot#444 (first comment)
• karlicoss/promnesia@1daff43
• etc.

And it seems that it's because it has to be compiled from the sources when installing it and that's something to deal with in some scenarios. This fact and the pypy issues could be enough to at least consider if it's viable to remove the dependency or not.

I don't have too much experience on any of these matters, so if you have any further insight please, share it with me :)

[lopuhin]

Thanks for investigating and clarifying the state of the pypy3 support @noviluni. I think relying on community interest wrt pypy3 support is good here, I don't have any extra info about potential pypy3 users.

but I found some projects that stoped using dateparser because of regex:

Thanks for insight, I was not aware of this.

There is some discussion of making regex optional in #446 - at first sight it does not look impossible but requires quite a bit of work. Investigating this more looks promising to me, but pypy3 support looks less important than users which can't use regex due to other reasons, based on feedback.

[noviluni]

We tried some things (#746) and it seems that we won't be supporting PyPy 3 because of the incompatibilities with the regex package. I don't discard to add it in the future, but based on the current community interest (https://github.com/scrapinghub/dateparser/issues?q=is%3Aissue+pypy) it seems that we can focus on other affairs and more demanded features.

Thank you @lopuhin for your feedback 🙂 , I will probably return on the regex dependency issue in the future, but at this moment it seems really utopic for me to be able to make the regex package optional.