tests: update to match `py-evm`: don't skip Constantinople, add Petersburg, use `--fork`; break out long-running categories #608

veox · 2019-05-14T13:13:00Z

What was wrong?

Tests for network=='Constantinople' are being skipped.

The pinned git submodule is for https://github.com/ethereum/tests/tree/420f443477caa8516f1f9ee8122fafc3415c0f34 (same as tag v6.0.0-beta.2, which is not even latest), whereas py-evm has https://github.com/ethereum/tests/tree/6b85703b568f4456582a00665d8a3e5c3b20b484 (untagged commit slightly after v6.0.0-beta.3, which itself does not include ConstantinopleFix - which why we're using something "slightly after").

How was it fixed?

Skip network=='Istanbul' (leave placeholder).
Use same fixtures commit as py-evm.
Add petersburg jobs to CI.
Use pytest --fork arg (copy-paste! from py-evm).
Address test failures (the usual disparity on "synthetic state").
Address long CI run times:
- break out categories into their own CI jobs:
  - zero-knowledge;
  - SSTORE;
- switch these break-out categories to use Python 3.7 (a tiny bit more performant than 3.6).

To-Do

Clean up commit history

Add entry to the release notes PR: Release notes for v0.1.0-alpha.24 #445 (comment)

Cute Animal Picture

Source: Felicia the ferret of Fermilab; via Atlas Obscura

veox · 2019-05-14T13:43:22Z

Failure is in the usual "incorrect upstream" disagreement.

veox · 2019-05-14T15:00:27Z

Enabled ConstantinopleFix, too, out of curiosity; but no actual CI job configured for that, yet).

EDIT: Oh, wait; we're pinned to a version of fixtures that doesn't have ConstantinopleFix yet. 😆

veox · 2019-05-14T16:05:05Z

tests/json-fixtures-over-rpc/test_rpc_fixtures.py

@@ -115,6 +115,24 @@
    # The result is in conflict with the yellow-paper:
    # * https://github.com/ethereum/py-evm/pull/1224#issuecomment-418800369
    ('GeneralStateTests/stRevertTest/RevertInCreateInInit_d0g0v0.json', 'RevertInCreateInInit_d0g0v0_Byzantium'),  # noqa: E501
+    ('GeneralStateTests/stRevertTest/RevertInCreateInInit_d0g0v0.json', 'RevertInCreateInInit_d0g0v0_Constantinople'),  # noqa: E501


This INCORRECT_UPSTREAM_TESTS blob is the same as the one in py-evm. It matches 1:1 the particular py-evm version that trinity is pinned to.

Wonder if we should make it more accessible in py-evm so it can be imported here instead of copy-pasting.

Added a TODO comment so this is not forgotten.

Are you up for making the changes and/or making an issue in py-evm to expose this API?

I'll make issues, as you outlined in:

Yeah, to avoid committing the TODO's to master, how about an issue in trinity that links to an issue in py-evm which lists the methods to expose in the API.

Use test-fixture fork-selection helpers from py-evm when they become available #621

Parts of the test suite could be exposed for downstream use py-evm#1781

tests/json-fixtures-over-rpc/test_rpc_fixtures.py

carver · 2019-05-14T19:37:18Z

Skip network=='ConstantinopleFix' instead.

Wait, shouldn't we be running Constantinople(Fix) tests? I thought everything was passing now.

veox · 2019-05-15T10:37:52Z

We've not even pinned the git submodule for fixtures that has ConstantinopleFix. That happened in py-evm only. So it is being tested, just not in this repo.

The latest state of this PR skips network=='Istanbul', ~~which is equally unrepresented in the fixtures~~ purely as a placeholder.

Updated PR summary with current state of affairs.

tox.ini

veox · 2019-05-16T11:48:04Z

Constantinople and Petersburg CI runs take >30 minutes on my free CircleCI account. Adding entries to SLOW_TESTS...

veox · 2019-05-16T12:01:25Z

tests/json-fixtures-over-rpc/test_rpc_fixtures.py

-    'Call1024PreCalls_d0g0v0_EIP150',
-    'Call1024PreCalls_d0g0v0_Byzantium',
-    'Call1024PreCalls_d0g0v0_EIP150',
-    'Call1024PreCalls_d0g0v0_EIP158',


The above six lines are duplicates of the first three lines in the list: therefore removed.

veox · 2019-05-16T13:33:08Z

tests/json-fixtures-over-rpc/test_rpc_fixtures.py

-    'ForkStressTest_Byzantium',
+    'Create2Recursive_d0g0v0',
+    'Create2Recursive_d0g1v0',
+    'ForkStressTest',


This test is slow across all forks - however, removal of fork qualifier means that any fixture that has this string will be marked SLOW.

Is this acceptable?

veox · 2019-05-16T13:37:17Z

PR ready for review, I guess. (Test runs might still take very long, though.)

carver · 2019-05-16T17:12:55Z

Hm, yeah, 30min is pretty long. Byzantium only takes 6min:

============ 4698 passed, 4751 skipped, 1 xfailed in 373.23 seconds ============

Constantinople:

========== 10323 passed, 10384 skipped, 5 xfailed in 1920.96 seconds ===========

I just checked the native runs in py-evm, and the total test counts look about right. (wow 2x more tests!) Interestingly, Constantinople is less than 2x slower in native, but >5x slower over RPC.

There aren't anymore really long individual tests left, so I don't see an obvious way to quickly shorten the long run. :( It should help a bit when we release & import the latest py-evm. But since most of the time seems to be RPC overhead, I guess it won't help that much.

carver

Yeah, to avoid committing the TODO's to master, how about an issue in trinity that links to an issue in py-evm which lists the methods to expose in the API.

30+ minute test jobs make me sad, but I'm not sure where to start on optimizing those yet. My first guess is that it's somewhere in lahja, since I don't remember RPC adding that much overhead before it was handled by events. cc @cburgdorf @lithp

carver · 2019-05-16T17:20:59Z

tests/json-fixtures-over-rpc/test_rpc_fixtures.py

@@ -115,6 +115,24 @@
    # The result is in conflict with the yellow-paper:
    # * https://github.com/ethereum/py-evm/pull/1224#issuecomment-418800369
    ('GeneralStateTests/stRevertTest/RevertInCreateInInit_d0g0v0.json', 'RevertInCreateInInit_d0g0v0_Byzantium'),  # noqa: E501
+    ('GeneralStateTests/stRevertTest/RevertInCreateInInit_d0g0v0.json', 'RevertInCreateInInit_d0g0v0_Constantinople'),  # noqa: E501


Are you up for making the changes and/or making an issue in py-evm to expose this API?

cburgdorf · 2019-05-16T18:20:31Z

My first guess is that it's somewhere in lahja, since I don't remember RPC adding that much overhead before it was handled by events. cc @cburgdorf @lithp

@carver If I'm not mistaken, the only thing that the JSON-RPC server uses the event bus for is to make requests to the PeerPool (e.g. to get the current peer count) but that shouldn't come into play here I guess. Worth taking a closer look into this.

carver · 2019-05-16T18:27:49Z

Yeah, you're right I was conflating two things: the event bus, and rpc running as an isolated plugin. I should have said the latter. So the overhead would be about sending the chain requests over multiprocessing.

cburgdorf · 2019-05-16T18:41:05Z

So the overhead would be about sending the chain requests over multiprocessing

Mmh..that should not have changed the tests. It's always been that way that the RPC tests would connect via IPC and make the request. In fact, the RPC tests do not even know about the plugin. They just spin up the IPCServer directly (nothing changed about that).

trinity/tests/conftest.py

Lines 298 to 320 in a37f26c

    
           @pytest.mark.asyncio 
        
           @pytest.fixture 
        
           async def ipc_server( 
        
                   monkeypatch, 
        
                   event_bus, 
        
                   jsonrpc_ipc_pipe_path, 
        
                   event_loop, 
        
                   chain_with_block_validation): 
        
               """ 
        
               This fixture runs a single RPC server over IPC over 
        
               the course of all tests. It yields the IPC server only for monkeypatching purposes 
        
               """ 
        
               rpc = RPCServer( 
        
                   initialize_eth1_modules(chain_with_block_validation, event_bus) 
        
               ) 
        
               ipc_server = IPCServer(rpc, jsonrpc_ipc_pipe_path, loop=event_loop) 
        
               asyncio.ensure_future(ipc_server.run(), loop=event_loop) 
        
               try: 
        
                   yield ipc_server 
        
               finally: 
        
                   await ipc_server.cancel()

carver · 2019-05-16T18:51:45Z

So the overhead would be about sending the chain requests over multiprocessing

Mmh..that should not have changed the tests. It's always been that way that the RPC tests would connect via IPC and make the request. In fact, the RPC tests do not even know about the plugin. They just spin up the IPCServer directly (nothing changed about that).

trinity/tests/conftest.py

Lines 298 to 320 in a37f26c

@pytest.mark.asyncio

@pytest.fixture

async def ipc_server(

monkeypatch,

event_bus,

jsonrpc_ipc_pipe_path,

event_loop,

chain_with_block_validation):

"""

This fixture runs a single RPC server over IPC over

the course of all tests. It yields the IPC server only for monkeypatching purposes

"""

rpc = RPCServer(

initialize_eth1_modules(chain_with_block_validation, event_bus)

)

ipc_server = IPCServer(rpc, jsonrpc_ipc_pipe_path, loop=event_loop)

asyncio.ensure_future(ipc_server.run(), loop=event_loop)

try:

yield ipc_server

finally:

await ipc_server.cancel()

I'm talking about the time before RPCServer was moved to a BaseIsolatedPlugin and just ran in the main process. Quite a while ago, now.

Edit: last Aug according to ethereum/py-evm@1006923

Edit 2: Okay, so not as big effect as I expected from ethereum/py-evm#1212 -- the previous PR to merge was ethereum/py-evm#1297

embedded
- py36-native-state-byzantium: 1:31
- py36-rpc-state-byzantium: 4:32 (~3x)
isolated plugin
- py36-native-state-byzantium: 2:00
- py36-rpc-state-byzantium: 6:40 (~3.3x)

So something is still missing about why the rpc gap widened from 3x to 5x since then...

cburgdorf · 2019-05-16T19:09:47Z

I'm talking about the time before RPCServer was moved to a BaseIsolatedPlugin and just ran in the main process. Quite a while ago, now.

Yes, but what I mean is: From the perspective of these tests, nothing has changed when the RPCServer moved into an BaseIsolatedPlugin. Afaik, the tests do not spin up a Trinity instance but instead just manually instantiate the RPCServer. In fact, I think the RPCServer in the tests just runs in the same process all the other tests run.

carver · 2019-05-16T19:12:23Z

I'm talking about the time before RPCServer was moved to a BaseIsolatedPlugin and just ran in the main process. Quite a while ago, now.

Yes, but what I mean is: From the perspective of these tests, nothing has changed when the RPCServer moved into an BaseIsolatedPlugin. Afaik, the tests do not spin up a Trinity instance but instead just manually instantiate the RPCServer. In fact, I think the RPCServer in the tests just runs in the same process all the other tests run.

Ah right, which also explains why there wasn't much of a hit from ethereum/py-evm#1212

veox · 2019-05-17T12:21:52Z

30+ minute test jobs make me sad, but I'm not sure where to start on optimizing those yet.

We could switch those CI jobs to run on Python 3.7 instead of 3.6...

That said, it might not be that big of a deal for someone not using CircleCI's free tier.

Another approach is to run some classes of tests separately, similar to how stQuadraticComplexityTest are already done; candidates:

ZeroKnowledge (>1300 items, many last 0.15 seconds each; easily 200 seconds total);
recursive and stack exhaustion (~100 (orly?) items, up to a few seconds each; easily 120 seconds total).

I've made a separate branch to test out the latter hypothesis.

EDIT:

ZK tests took 5:33 to run (06:11 for entire CI job);
25:07 for Constantinople with ZK tests ignored via pytest -k;
24:55 for Petersburg with ZK tests ignored via pytest -k.

So, a matching reduction from the previous 30+ minutes. It's a viable strategy...

A yet another approach would be to avoid MiningChain in pytest, as these tests have to be set up so they can be marked skip (and therefore have to be torn down afterwards). Haven't measured how much of an effect this has on the overall run duration, though.

BTW, could it be that MiningChain is only a remnant of the testing that had to be done in py-evm, and it's a leftover in trinity? Or is this something that is used in Hive?..

Testing hypothesis as separate PR #623.

carver · 2019-05-17T21:46:17Z

30+ minute test jobs make me sad, but I'm not sure where to start on optimizing those yet.

We could switch those CI jobs to run on Python 3.7 instead of 3.6...

👍

That said, it might not be that big of a deal for someone not using CircleCI's free tier.

Eh, max(len(job) for job in ci_jobs) still matters, even with high concurrency. So I like your next approach...

Another approach is to run some classes of tests separately, similar to how stQuadraticComplexityTest are already done; candidates:

ZeroKnowledge (>1300 items, many last 0.15 seconds each; easily 200 seconds total);

recursive and stack exhaustion (~100 (orly?) items, up to a few seconds each; easily 120 seconds total).

I've made a separate branch to test out the latter hypothesis.

EDIT:

ZK tests took 5:33 to run (06:11 for entire CI job);

25:07 for Constantinople with ZK tests ignored via pytest -k;

24:55 for Petersburg with ZK tests ignored via pytest -k.

So, a matching reduction from the previous 30+ minutes. It's a viable strategy...

Yeah, I'm definitely in favor of this direction.

veox · 2019-05-19T13:44:43Z

Identified stSStoreTest as the largest offender. On a check-everything local pytest tests run, fixtures from that dir made up 5722 of 10000 slowest test runs.

Breaking them out further reduced Petersburg job run time to 04:10 (4:41 job total).

... But now the sstore CI job takes 19:20 to run (20:03 job total): it runs the tests for all forks. :/

PR #623 is extremely likely to help with this: setting up the tests also takes very long:

========================== slowest 50 test durations ===========================
1.31s setup    tests/json-fixtures-over-rpc/test_rpc_fixtures.py::test_rpc_against_fixtures[/home/circleci/repo/fixtures/BlockchainTests/GeneralStateTests/stSStoreTest/sstore_combinations_initial2_d656g0v0.json:sstore_combinations_initial2_d656g0v0_Constantinople:Constantinople-MiningChain]
1.28s setup    tests/json-fixtures-over-rpc/test_rpc_fixtures.py::test_rpc_against_fixtures[/home/circleci/repo/fixtures/BlockchainTests/GeneralStateTests/stSStoreTest/sstore_combinations_initial2_d384g0v0.json:sstore_combinations_initial2_d384g0v0_Constantinople:Constantinople-Chain]
1.28s setup    tests/json-fixtures-over-rpc/test_rpc_fixtures.py::test_rpc_against_fixtures[/home/circleci/repo/fixtures/BlockchainTests/GeneralStateTests/stSStoreTest/sstore_combinations_initial2_d680g0v0.json:sstore_combinations_initial2_d680g0v0_ConstantinopleFix:ConstantinopleFix-MiningChain]
1.28s setup    tests/json-fixtures-over-rpc/test_rpc_fixtures.py::test_rpc_against_fixtures[/home/circleci/repo/fixtures/BlockchainTests/GeneralStateTests/stSStoreTest/sstore_combinations_initial2_d466g0v0.json:sstore_combinations_initial2_d466g0v0_Constantinople:Constantinople-MiningChain]
1.27s setup    tests/json-fixtures-over-rpc/test_rpc_fixtures.py::test_rpc_against_fixtures[/home/circleci/repo/fixtures/BlockchainTests/GeneralStateTests/stSStoreTest/sstore_combinations_initial2_d976g0v0.json:sstore_combinations_initial2_d976g0v0_ConstantinopleFix:ConstantinopleFix-MiningChain]
1.25s setup    tests/json-fixtures-over-rpc/test_rpc_fixtures.py::test_rpc_against_fixtures[/home/circleci/repo/fixtures/BlockchainTests/GeneralStateTests/stSStoreTest/sstore_combinations_initial2_d643g0v0.json:sstore_combinations_initial2_d643g0v0_ConstantinopleFix:ConstantinopleFix-Chain]

Didn't check (yet) the test fixtures themselves as to why.

veox · 2019-05-21T09:58:31Z

Rebased on master to get PR #623; sstore job now takes 8-9 minutes (jobs 2675, 2684, 2685):

================== 11253 passed, 7 xfailed in 525.26 seconds ===================
================== 11253 passed, 7 xfailed in 506.88 seconds ===================
================== 11253 passed, 7 xfailed in 516.59 seconds ===================

After switching break-out categories to Python 3.7, sstore job takes 7-8 minutes (jobs 2716, 2718, 2719):

================== 11253 passed, 7 xfailed in 466.06 seconds ===================
================== 11253 passed, 7 xfailed in 436.75 seconds ===================
================== 11253 passed, 7 xfailed in 487.96 seconds ===================

I'm out of good ideas, but maybe that's fast enough?..

EDIT: py3{6,7}-eth2-fixtures takes that long, too, so - yeah, we're good.

…TS and `xfail`s. The `xfail`s are a 1:1 copy of what's in py-evm's pinned commit; SLOW_TESTS are also very similar...

Mostly a copy-paste from `py-evm` codebase; relevant PRs that introduced the functionality there: ethereum/py-evm#1470 ethereum/py-evm#1577

…icates.

… job.

veox · 2019-05-21T11:57:45Z

Cleaned up commits; unmarking WIP; ready for review.

pipermerriam

👍 to

Thanks for the cleanup

pipermerriam · 2019-05-21T12:50:11Z

tests/json-fixtures-over-rpc/test_rpc_fixtures.py

@@ -188,13 +205,33 @@ def blockchain_fixture_mark_fn(fixture_path, fixture_name):
        return pytest.mark.xfail(reason="Listed in INCORRECT_UPSTREAM_TESTS.")


+def generate_ignore_fn_for_fork(passed_fork):
+    if passed_fork:
+        passed_fork = passed_fork.lower()


nit we'd normally do this as normalized_fork = passed_fork.lower() to avoid destroying the initial data.

I'll fix that up real quick; but note it's a copy-paste from https://github.com/ethereum/py-evm/blob/ddefa2fbad378aad6562239eb087e8eb27794fc1/tests/json-fixtures/test_blockchain.py#L231.

I'll admit we aren't always perfectly consistent about these things. Thanks for the cleanup.

pipermerriam · 2019-05-21T13:16:07Z

will merge when green. Thanks a bunch.

veox force-pushed the dont-skip-constantinople-tests branch from dfd036b to 4addf50 Compare May 14, 2019 14:58

veox force-pushed the dont-skip-constantinople-tests branch from 4addf50 to c7cc127 Compare May 14, 2019 15:13

veox commented May 14, 2019

View reviewed changes

tests/json-fixtures-over-rpc/test_rpc_fixtures.py Outdated Show resolved Hide resolved

veox changed the title ~~[WIP] tests: don't skip fixtures for network 'Constantinople'~~ [WIP] tests: update to match py-evm (don't skip fixtures for Constantinople, add fixtures for Petersburg) May 15, 2019

veox force-pushed the dont-skip-constantinople-tests branch from c49dfe1 to 56ea99f Compare May 15, 2019 14:11

veox commented May 15, 2019

View reviewed changes

tox.ini Outdated Show resolved Hide resolved

veox force-pushed the dont-skip-constantinople-tests branch from 845cd46 to eb5220a Compare May 16, 2019 11:01

veox mentioned this pull request May 16, 2019

Release notes for v0.1.0-alpha.24 #445

Merged

veox commented May 16, 2019

View reviewed changes

veox force-pushed the dont-skip-constantinople-tests branch 2 times, most recently from 260613c to 75aa66c Compare May 16, 2019 13:30

veox commented May 16, 2019

View reviewed changes

veox changed the title ~~[WIP] tests: update to match py-evm (don't skip fixtures for Constantinople, add fixtures for Petersburg)~~ tests: update to match py-evm (don't skip fixtures for Constantinople, add fixtures for Petersburg) May 16, 2019

carver approved these changes May 16, 2019

View reviewed changes

This was referenced May 17, 2019

Parts of the test suite could be exposed for downstream use ethereum/py-evm#1781

Open

Use test-fixture fork-selection helpers from py-evm when they become available #621

Closed

veox mentioned this pull request May 17, 2019

tests: Don't consider MiningChain in tests #623

Merged

2 tasks

veox changed the title ~~tests: update to match py-evm (don't skip fixtures for Constantinople, add fixtures for Petersburg)~~ [WIP] tests: update to match py-evm (don't skip Constantinople, add Petersburg) May 19, 2019

veox force-pushed the dont-skip-constantinople-tests branch 2 times, most recently from 0c27da1 to f516371 Compare May 19, 2019 13:16

veox force-pushed the dont-skip-constantinople-tests branch from 265723b to 3f5514d Compare May 20, 2019 20:18

veox added 7 commits May 21, 2019 14:52

fixtures: update git submodule to same commit as in py-evm.

1146f83

tests: don't skip fixtures for 'Constantinople{,fix}' + mark SLOW_TES…

13e4f48

…TS and `xfail`s. The `xfail`s are a 1:1 copy of what's in py-evm's pinned commit; SLOW_TESTS are also very similar...

ci: add petersburg CI job.

137e287

ci/tests: ignore tests not matching --fork arg given to pytest.

712f16e

Mostly a copy-paste from `py-evm` codebase; relevant PRs that introduced the functionality there: ethereum/py-evm#1470 ethereum/py-evm#1577

tests: update SLOW_TESTS now that more forks are tested + remove dupl…

adf43f6

…icates.

ci: break out zeroKnowledge{,2} and sstore tests into separate CI…

020778e

… job.

ci: run rpc-state-{quadratic,sstore,zero_knowledge} in py37 env.

61a25fc

veox force-pushed the dont-skip-constantinople-tests branch from 6eec26c to 61a25fc Compare May 21, 2019 11:56

veox changed the title ~~[WIP] tests: update to match py-evm (don't skip Constantinople, add Petersburg)~~ [WIP] tests: update to match py-evm: don't skip Constantinople, add Petersburg, use --fork, break out long-running categories May 21, 2019

veox changed the title ~~[WIP] tests: update to match py-evm: don't skip Constantinople, add Petersburg, use --fork, break out long-running categories~~ tests: update to match py-evm: don't skip Constantinople, add Petersburg, use --fork, break out long-running categories May 21, 2019

veox changed the title ~~tests: update to match py-evm: don't skip Constantinople, add Petersburg, use --fork, break out long-running categories~~ tests: update to match py-evm: don't skip Constantinople, add Petersburg, use --fork; break out long-running categories May 21, 2019

pipermerriam approved these changes May 21, 2019

View reviewed changes

tests: minor, don't re-write passed_fork.

c3fe964

pipermerriam merged commit c3778a5 into ethereum:master May 21, 2019

carver added a commit that referenced this pull request May 21, 2019

Add release notes for #608 and #623

3dfb794

veox deleted the dont-skip-constantinople-tests branch May 21, 2019 19:42

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

tests: update to match `py-evm`: don't skip Constantinople, add Petersburg, use `--fork`; break out long-running categories #608

tests: update to match `py-evm`: don't skip Constantinople, add Petersburg, use `--fork`; break out long-running categories #608

veox commented May 14, 2019 •

edited

Loading

veox commented May 14, 2019 •

edited

Loading

veox commented May 14, 2019 •

edited

Loading

veox May 14, 2019 •

edited

Loading

carver May 16, 2019

veox May 17, 2019 •

edited

Loading

carver commented May 14, 2019

veox commented May 15, 2019 •

edited

Loading

veox commented May 16, 2019 •

edited

Loading

veox May 16, 2019 •

edited

Loading

veox May 16, 2019 •

edited

Loading

veox commented May 16, 2019

carver commented May 16, 2019 •

edited

Loading

carver left a comment

carver May 16, 2019

cburgdorf commented May 16, 2019 •

edited

Loading

carver commented May 16, 2019

cburgdorf commented May 16, 2019

carver commented May 16, 2019 •

edited

Loading

cburgdorf commented May 16, 2019

carver commented May 16, 2019

veox commented May 17, 2019 •

edited

Loading

carver commented May 17, 2019

veox commented May 19, 2019 •

edited

Loading

veox commented May 21, 2019 •

edited

Loading

veox commented May 21, 2019

pipermerriam left a comment

pipermerriam May 21, 2019

veox May 21, 2019

pipermerriam May 21, 2019

pipermerriam commented May 21, 2019

tests: update to match py-evm: don't skip Constantinople, add Petersburg, use --fork; break out long-running categories #608

tests: update to match py-evm: don't skip Constantinople, add Petersburg, use --fork; break out long-running categories #608

Conversation

veox commented May 14, 2019 • edited Loading

What was wrong?

How was it fixed?

To-Do

Cute Animal Picture

veox commented May 14, 2019 • edited Loading

veox commented May 14, 2019 • edited Loading

veox May 14, 2019 • edited Loading

Choose a reason for hiding this comment

carver May 16, 2019

Choose a reason for hiding this comment

veox May 17, 2019 • edited Loading

Choose a reason for hiding this comment

carver commented May 14, 2019

veox commented May 15, 2019 • edited Loading

veox commented May 16, 2019 • edited Loading

veox May 16, 2019 • edited Loading

Choose a reason for hiding this comment

veox May 16, 2019 • edited Loading

Choose a reason for hiding this comment

veox commented May 16, 2019

carver commented May 16, 2019 • edited Loading

carver left a comment

Choose a reason for hiding this comment

carver May 16, 2019

Choose a reason for hiding this comment

cburgdorf commented May 16, 2019 • edited Loading

carver commented May 16, 2019

cburgdorf commented May 16, 2019

carver commented May 16, 2019 • edited Loading

cburgdorf commented May 16, 2019

carver commented May 16, 2019

veox commented May 17, 2019 • edited Loading

carver commented May 17, 2019

veox commented May 19, 2019 • edited Loading

veox commented May 21, 2019 • edited Loading

veox commented May 21, 2019

pipermerriam left a comment

Choose a reason for hiding this comment

pipermerriam May 21, 2019

Choose a reason for hiding this comment

veox May 21, 2019

Choose a reason for hiding this comment

pipermerriam May 21, 2019

Choose a reason for hiding this comment

pipermerriam commented May 21, 2019

tests: update to match `py-evm`: don't skip Constantinople, add Petersburg, use `--fork`; break out long-running categories #608

tests: update to match `py-evm`: don't skip Constantinople, add Petersburg, use `--fork`; break out long-running categories #608

veox commented May 14, 2019 •

edited

Loading

veox commented May 14, 2019 •

edited

Loading

veox commented May 14, 2019 •

edited

Loading

veox May 14, 2019 •

edited

Loading

veox May 17, 2019 •

edited

Loading

veox commented May 15, 2019 •

edited

Loading

veox commented May 16, 2019 •

edited

Loading

veox May 16, 2019 •

edited

Loading

veox May 16, 2019 •

edited

Loading

carver commented May 16, 2019 •

edited

Loading

cburgdorf commented May 16, 2019 •

edited

Loading

carver commented May 16, 2019 •

edited

Loading

veox commented May 17, 2019 •

edited

Loading

veox commented May 19, 2019 •

edited

Loading

veox commented May 21, 2019 •

edited

Loading