Introduce poetry

[alecandido]

Switch build tool and dependency management to poetry, and we other things:

• update testing workflow
• update deploy workflow
• add task runner
• add dynamic version
• update docs building

[codecov]

Codecov Report

Merging #82 (b307a1f) into develop (8d0809d) will not change coverage.
The diff coverage is 100.00%.

@@            Coverage Diff            @@
##           develop       #82   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files           31        32    +1     
  Lines         2370      2372    +2     
=========================================
+ Hits          2370      2372    +2     

Flag	Coverage Δ
unittests	100.00% <100.00%> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
src/eko/strong_coupling.py	100.00% <ø> (ø)
src/eko/__init__.py	100.00% <100.00%> (ø)
src/eko/output.py	100.00% <100.00%> (ø)
src/eko/version.py	100.00% <100.00%> (ø)

[alecandido]

Deploy workflow is upgraded, but not yet tested (we need a pre-release).

The update is sufficiently small not to justify a dedicated setup with test.pypi.org, if it'll break, I'll fix it in a point release.

[alecandido]

It's relatively ready to merge, but if someone can review this and try to figure out if there was anything else depending on setuptools, or any more dead code, it would be appreciated :)

[alecandido]

I'm updating the docs section, but as per yadism there is the usual issue with rtd, documented in the following issues:

• usage on readthedocs.org python-poetry/poetry#754
• Support Poetry readthedocs/readthedocs.org#4912

The workaround we apply for the time being is that one coming from this comment readthedocs/readthedocs.org#4912 (comment) and above.

Not elegant (as the comment itself and preceding says), but it's currently a limitation of rtd, i.e.:

• we would like to have rtd take into account also dev-dependencies (those dependencies that are not needed by package users)
• since dev-deps are not a pip feature, and poetry is supported by rtd only through pip and PEP 517, they are completely ignored
• instead extras are a pip feature, installed like pip install package[extra] (but they are a different feature, because are dependencies needed by the end user, but not for all use cases), and so we avoid making them main dependencies by making them extra, and tell rtd to install extras of docs group (and duplicating them as dev-deps, since they are actually needed in development)

[felixhekhorn]

• do you understand the codecov complaint https://www.codefactor.io/repository/github/n3pdf/eko/pull/82 ? we do not import eko in there, which I guess would be the equivalent (PS: and a local pylint run does not bring the issue)
• except for the install.py business, I guess, this is good for merging

Unfortunately there are still the known issues with poetry

• a fresh install (even with all packages cached) still takes around 5 mins
• we're waiting eagerly for v1.2 to get editable ...

Nevertheless, I guess we want to keep going, since we also gain some significant features ...

[felixhekhorn]

Actually, there is another issue and I don't know how serious this actually is:

https://github.com/N3PDF/eko/blob/d374b52c45865300e9c776c769f2517e715e03e0/benchmarks/setup.py#L18

this does pull eko from PyPI and does NOT use the local version

[alecandido]

• do you understand the codecov complaint https://www.codefactor.io/repository/github/n3pdf/eko/pull/82 ? we do not import eko in there, which I guess would be the equivalent

I guess this pylint doesn't like the concept of exposing, or it's simply a bug in the pylint implementation itself (since from . means import from myself, but in the package sense, and not as a module, so maybe it's simply confused).

(PS: and a local pylint run does not bring the issue)

This has to be related to the pylint version, I guess...

Unfortunately there are still the known issues with poetry

It's even worse, I need tens of minutes to upgrade dependencies, when not more than an hour.
Nevertheless, poetry is the most actively maintained dependency manager in the python ecosystem, at the moment, and it offers significant advantages over bare virtualenv use, so it's still worth for me, and we hope issues will be solved in the near future.

If you wish, you're free to test Pipenv and compare the two ;) (it's the only other relevant alternative)

[alecandido]

Actually, there is another issue and I don't know how serious this actually is:

https://github.com/N3PDF/eko/blob/d374b52c45865300e9c776c769f2517e715e03e0/benchmarks/setup.py#L18

this does pull eko from PyPI and does NOT use the local version

Here I simply didn't touch anything, yet...

We're in a complicated situation, since ekomark depends on eko, but at the same time is a development tool for eko.

For yadmark I already sketched a layout, that to me seemed the cleanest: the benchmark suite is a devdep of the package, since it's actually a part of it, and it's not shipped to the end user.

Things will change with ekobox, because at that point ekomark will be shipped.
When the time will come, I would simply move ekomark in src/ekomark, as another package we're developing, and keep everything else in benchmark (like runners). So they are simply living side by side.
Unfortunately, we'll have some issues in packaging as well python-poetry/poetry#936, so we might consider shipping them in a unique bundle...

[felixhekhorn]

• Okay, but then let's move ekomark also to Poetry (and do the same fix, i.e., skip the dependency) - could you please take care of this @alecandido ?
• and that we will move ekomark at some point, I guess we already agreed upon ...
• at this point, benchmarks will indeed contain nothing but actual implementations of ekomark (=runners) and their associated data

[alecandido]

• Okay, but then let's move ekomark also to Poetry (and do the same fix, i.e., skip the dependency) - could you please take care of this @alecandido ?

Okay, I'll copy yadmark (but it will have same issues).

• and that we will move ekomark at some point, I guess we already agreed upon ...

Yes, but as said before poetry has no support for monorepo, so we might be forced to move to "monopackage"...

Note: we're not the only one looking for monorepo, and there are already some tools for the task. However, I made my proposal, let's see if anyone will be interested...

[felixhekhorn]

let's

• adjust the README
• merge
• immediately after setup monorepo structure

[alecandido]

Wait, it's being tricky to release banana, gimme a while to fix it...

[alecandido]

Please @felixhekhorn and @giacomomagni review this, and I'll merge.