Add support for Python 3.8 #1679

cburgdorf · 2020-04-17T14:23:55Z

What was wrong?

When we get rid of Python 3.6 support we should make sure we support Python 3.8 by testing against it in CI.

How was it fixed?

Add CI jobs for Python 3.8

To-Do

Clean up commit history

Add entry to the release notes

Cute Animal Picture

pipermerriam

Doesn't have to happen in this PR (and might be a great "Good First Issues") but we should look into using the native contextlib.asynccontextmanager and any other 3.7 things that we've had to shim back into 3.6.

cburgdorf · 2020-04-22T14:52:39Z

@ChihChengLiang do you have any idea what's preventing the milagro-bls-binding from getting properly installed under a Python 3.8 environment?

cburgdorf · 2020-04-22T14:53:22Z

@pipermerriam No you're right, there was a huge chunk of things that we could get rid off. I updated the PR.

ChihChengLiang · 2020-04-23T12:28:22Z

Hi @cburgdorf, I think I missed some release of milagro-bls-binding binding for 3.8 version. Just fixed it now, would you like to give it another try?

cburgdorf · 2020-04-24T09:29:42Z

@ChihChengLiang Thanks for fixing the Python 3.8 builds! Unfortunately, it seems like the API has changed substantially enough to throw me off 😅 For example, sign has no domain anymore, and verify_multiple is gone. Would you mind handling the upgrade to the latest version in a standalone PR?

ChihChengLiang · 2020-04-26T09:05:22Z

@cburgdorf I'm terribly sorry for saying it's ready for the upgrade and I'm so sorry for your time on trying to upgrade that 🙈, I totally forgot the breaking API change. The API change would require a lengthy PR because it touches eth2 code and eth2-fixture tests.

Here are some band-aids for that.

The quickest possible way is to drop the eth2-extra requirement in CI, the tests would fall back to using py-ecc as the bls library.
Meanwhile, I'll try to release an old API compatible py 3.8 release of milagro-bls-binding, under version 0.1.x. If it works, it makes existing Trinity codebase works. I'll yank out the 0.2.0, 0.3.0, and 0.4.0, as those versions are using the new API. The new API will be released under 2.0.x. I'll update in this thread when it's done.

I'm not able to commit to the upgrade of Trinity codebase to the new API yet, due to the other ongoing tasks. But happy to do it a while later if no one else is on it.

ChihChengLiang · 2020-04-26T11:28:26Z

@cburgdorf I just released milagro-bls-binding==0.1.4 for py3.8 and old API support. The installation in Trinity looks fine and CI tests depend on that pass here. Hope this time works 😂.

cburgdorf · 2020-04-27T08:21:04Z

@ChihChengLiang Awesome! That fixes it. Also, don't worry too much, I haven't wasted much time on the issue and it isn't urgent. Thank you very much for cutting a new release 🙏

cburgdorf · 2020-04-28T15:06:56Z

@carver When you feel ready to ditch Python 3.6 we can merge this to trade it against Python 3.8 support

cburgdorf · 2020-05-25T13:54:21Z

@carver Are we good to trade Python 3.6 against Python 3.8 support by now?

carver · 2020-05-26T17:43:34Z

@carver Are we good to trade Python 3.6 against Python 3.8 support by now?

What is the urgency to drop 3.6 again? Can't we add 3.8 support without dropping 3.6?

cburgdorf · 2020-05-26T17:57:18Z

We can get rid of a bunch of Python 3.6 shims. I also suspect there may be some other features we could use that do not have shims available.
Python 3.6 does not support pickleing of generic classes (Something that our pending eth/66 implementation uses but which I suspect might also be useful for various lahja events e.g. eventbus.stream(Something[PayloadX]))
Adding Python 3.8 support next to Python 3.6 would also increase our CI times.

carver

Okay, I guess it's been long enough since the LTS came out. I still haven't upgraded but 3.7 is of course available through the package manager.

setup.py

tox.ini

carver · 2020-05-27T16:54:26Z

tox.ini

+    py{37,38}-{eth1-core,p2p,p2p-trio,integration,lightchain_integration,eth2-core,eth2-fixtures,eth2-integration,eth1-components,eth2-utils,eth2-trio}
+    py37-long_run_integration
+    py37-rpc-blockchain
+    py38-rpc-state-{frontier,homestead,tangerine_whistle,spurious_dragon,byzantium,constantinople,petersburg,istanbul}


Looks like this one got missed, I expected: py37

veox · 2020-05-29T19:56:55Z

So this added proper Python 3.8 support (for running the node, etc.), or does it only make CI pass?

Because if the former, then the newsfragments doesn't reflect it, and the fact will not be in the release notes.

cburgdorf · 2020-05-29T20:38:10Z

@veox oh you're right. I somehow missed to mention Python 3.8 support. I should work but afaik there are still some quirks with Python 3.8 that need to be resolved.

carver · 2020-05-29T20:58:35Z

We can add the release note when we upgrade to a py-evm version that includes ethereum/py-evm#1940

cburgdorf force-pushed the christoph/feat/py38 branch 3 times, most recently from 5fea2a6 to 6d3b3ee Compare April 21, 2020 12:51

pipermerriam reviewed Apr 21, 2020

View reviewed changes

cburgdorf force-pushed the christoph/feat/py38 branch 2 times, most recently from 188c310 to 395f953 Compare April 22, 2020 14:39

cburgdorf force-pushed the christoph/feat/py38 branch 2 times, most recently from 6d5b05e to e600f60 Compare April 27, 2020 08:18

cburgdorf force-pushed the christoph/feat/py38 branch from e600f60 to 483539b Compare April 28, 2020 14:38

cburgdorf requested a review from carver April 28, 2020 15:05

cburgdorf added 3 commits May 25, 2020 15:38

Remove support for Python 3.6

7c0dea9

Add support for Python 3.8

ade078f

Drop shims for Python 3.6 support

f4f82ec

cburgdorf force-pushed the christoph/feat/py38 branch from 483539b to f4f82ec Compare May 25, 2020 13:42

carver approved these changes May 26, 2020

View reviewed changes

setup.py Show resolved Hide resolved

tox.ini Outdated Show resolved Hide resolved

cburgdorf force-pushed the christoph/feat/py38 branch 2 times, most recently from 598fe68 to 61fd1c1 Compare May 27, 2020 08:42

PR feedback

873619f

cburgdorf force-pushed the christoph/feat/py38 branch from 61fd1c1 to 873619f Compare May 27, 2020 08:56

cburgdorf mentioned this pull request May 27, 2020

Activate rpc-state-istanbul CI Job #1748

Closed

cburgdorf merged commit 1f31660 into ethereum:master May 27, 2020

carver reviewed May 27, 2020

View reviewed changes

veox mentioned this pull request May 29, 2020

[WIP] [BLOCKED] Add Python 3.8 testing ethereum/py-evm#1885

Closed

11 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add support for Python 3.8 #1679

Add support for Python 3.8 #1679

cburgdorf commented Apr 17, 2020 •

edited

Loading

pipermerriam left a comment

cburgdorf commented Apr 22, 2020

cburgdorf commented Apr 22, 2020

ChihChengLiang commented Apr 23, 2020

cburgdorf commented Apr 24, 2020

ChihChengLiang commented Apr 26, 2020

ChihChengLiang commented Apr 26, 2020

cburgdorf commented Apr 27, 2020

cburgdorf commented Apr 28, 2020

cburgdorf commented May 25, 2020

carver commented May 26, 2020

cburgdorf commented May 26, 2020 •

edited

Loading

carver left a comment

carver May 27, 2020

veox commented May 29, 2020

cburgdorf commented May 29, 2020

carver commented May 29, 2020

Add support for Python 3.8 #1679

Add support for Python 3.8 #1679

Conversation

cburgdorf commented Apr 17, 2020 • edited Loading

What was wrong?

How was it fixed?

To-Do

Cute Animal Picture

pipermerriam left a comment

Choose a reason for hiding this comment

cburgdorf commented Apr 22, 2020

cburgdorf commented Apr 22, 2020

ChihChengLiang commented Apr 23, 2020

cburgdorf commented Apr 24, 2020

ChihChengLiang commented Apr 26, 2020

ChihChengLiang commented Apr 26, 2020

cburgdorf commented Apr 27, 2020

cburgdorf commented Apr 28, 2020

cburgdorf commented May 25, 2020

carver commented May 26, 2020

cburgdorf commented May 26, 2020 • edited Loading

carver left a comment

Choose a reason for hiding this comment

carver May 27, 2020

Choose a reason for hiding this comment

veox commented May 29, 2020

cburgdorf commented May 29, 2020

carver commented May 29, 2020

cburgdorf commented Apr 17, 2020 •

edited

Loading

cburgdorf commented May 26, 2020 •

edited

Loading