Skip to content

Conversation

marun
Copy link
Contributor

@marun marun commented Jul 11, 2023

Why this should be merged

This change updates the e2e test suite to use the new testnet fixture introduced in #1700. The new fixture is simpler and maintained in tree to better enable e2e test development.

As part of this change, the fixture's funded keys are allocated at-most-once to tests to avoid the following problems:

  • reusing the same pre-defined set of funded addresses (e.g. the EWOQ key) precludes proper test isolation
  • iterating against a persistent network with a limited set of fixed addresses is more likely to result in failure due to funds exhaustion
  • the wallet's lack of concurrency precludes tests using a common set of keys from executing in parallel

How this works

  • The use of ANR in the e2e suite is replaced by the testnet fixture
  • Tests are updated to retrieve funded keys at runtime
  • Keys are served via a simple http server to allow at-most-once allocation even when ginkgo is running specs in parallel

How this was tested

  • ./scripts/tests.e2e.sh was updated to run tests in parallel and in random order. This revealed the requirement to tag the virtuous tests as Serial to avoid skipping on conflict with other tests.
  • ./scripts/tests.e2e.persistent.sh was updated to run tests.e2e.sh against a persistent network

TODO

@marun marun force-pushed the e2e-address-allocation branch 8 times, most recently from f8d41ab to 6b9a223 Compare July 11, 2023 18:51
@marun marun force-pushed the e2e-address-allocation branch from 6b9a223 to 282ed56 Compare July 12, 2023 15:37
@marun marun force-pushed the e2e-address-allocation branch 4 times, most recently from c32cd2c to a47f45a Compare July 20, 2023 05:44
@marun marun changed the title e2e: Improve test isolation with per-test funded key allocation e2e: Switch e2e to use new testnet fixture Jul 20, 2023
@marun marun changed the title e2e: Switch e2e to use new testnet fixture e2e: Switch to testnet fixture Jul 20, 2023
@marun marun force-pushed the e2e-address-allocation branch 12 times, most recently from 5c01cfa to e10b764 Compare July 25, 2023 00:47
@marun marun force-pushed the e2e-address-allocation branch from 04eecaa to bb20c3e Compare August 8, 2023 16:40
Copy link
Contributor

@hexfusion hexfusion left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good few minor nits

@marun marun force-pushed the e2e-address-allocation branch from 7c85f13 to 19ccc50 Compare August 8, 2023 23:39
@hexfusion hexfusion requested a review from felipemadero August 9, 2023 18:27
for {
_, reserved := constants.NetworkIDToNetworkName[networkID]
if reserved {
networkID++
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice catch

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All @darioush

@marun marun removed request for gyuho and abi87 August 14, 2023 20:39
Comment on lines +62 to +64
WithTimeout(e2e.DefaultTimeout).
WithPolling(e2e.DefaultPollingInterval).
Should(gomega.BeTrue(), "The cluster is generating ongoing blocks. Is this test being run in parallel?")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I spent like 20m understanding how this works lol.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I'm not a huge fan of how ginkgo does this but I wanted to use ginkgo for consistency. I'm writing new e2e tests with testify, and at some point we can rewrite existing tests to also use it.

@StephenButtolph StephenButtolph merged commit 09a9f44 into dev Aug 15, 2023
@StephenButtolph StephenButtolph deleted the e2e-address-allocation branch August 15, 2023 00:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
testing This primarily focuses on testing
Projects
No open projects
Archived in project
Development

Successfully merging this pull request may close these issues.

5 participants