Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FLIP 204 - Smart-contract Defines target duration and end time for epochs #5062

Closed
wants to merge 8 commits into from

Conversation

jordanschalm and others added 6 commits November 10, 2023 10:40
* add TargetEndTime to epoch models

* update service event parsing

* fix some typos and godocs

* update mocks

* lint

* lint

* pin new version of core-contracts

- adds TargetEndTime field
- pinned to branch - needs to be updated when release is created

* update expected state commitment

* update another state commitment constant

* update another state commit constant

* represent target end time as uint64

* lint / mocks
* unit tests, integration tests using buildjet-4vcpu runner

* listTargetPackages() returns map of CI runners

* test for custom runners

* generateTestMatrix() stores CI runner

* more testing with custom CI runners

* CI test - 2 unit tests running buildjet runners

* increase engine unit tests to 8 vCPUs

* resource manager full load test (8 vCPUs)

* network/test tests unquarantined, increased runners

* increased to 16 vCPUs

* increased engine tests to 16 vCPUs

* insecure module uses buildjet 4 vcpu runner

* bft framework integration test using buildjet

* engine tests split up to 6 jobs with default runners

* added back all integration tests

* upgraded runners for some integration tests, engine/execution

* dummy commit to kick off CI

* split up TestScriptExecutionAndGetAccounts into 3 tests

* updated runners for Epoch Cohort2, Access Integration Tests

* AN integration tests split up into 3 cohorts (skip unit, other integration tests)

* docker build using GitHub cache

* remove saving Docker image locally

* cache v3, saving local tar before caching, 8vcpu

* add 11 docker images to docker-images.tar, 16 vcpu

* docker save multiline fix; skip localnet-test

* docker build increase to 32 vcpu

* buildjet cache instead of GitHub cache

* lint fix

* activate more integration tests with default runners, using cached Docker

* docker-build downsized to 16 vcpu

* integration test without relic build

* put back relic build for integration tests

* put back all remaining integration tests using BuildJet runners

* integration tests reverted to ubuntu-latest runners

* re-activated unit-test, unit-test-modules jobs

* added (failing) test for sub-sub packages

* sub-sub packages support

* sub-sub package test enhanced

* updated other tests to support new sub-sub packages

* split up engine/execution unit tests into 3 jobs

* engine/execution/ingestion upgraded to 4 vcpu

* engine/execution/ingestion:buildjet-8vcpu

* network/test, network/p2p increased to 8 vcpu; added remaining network tests

* BFT Framework, Epoch Cohort 2 - increased to 4 vcpu

* upgraded runners for Access Cohort 1, Epoch Cohort 1, 2

* epoch cohort2 upgraded to 16 vcpu

* BFT Framework upgrade to 8 vcpu, epoch cohort2 removed flaky test, downgrade to 4 vcpu

* quarantined flaky test - TestEpochJoinAndLeaveVN

* quarantined flaky test - TestSealingAndVerificationPassThrough

* lint fix

* network/test split up into network/test/cohort1, network/test/cohort2

* network/p2p split up into 4 subpackages

network/p2p/connection network/p2p/scoring network/p2p/p2pnode network/p2p

* extracted network/alsp into separate job

* extracted module/dkg into separate job

* engine/execution/ingestion dowgraded to stock runner

* added engine parent package as separate job

* added storage parent package as separate job

* added state package as separate job

* removed localnet-test job

* engine/execution/ingestion upgraded to 2 vcpu

* network/p2p/p2pnode upgraded to 2 vcpu

* module, engine upgraded to 2 vcpu

* network/p2p/p2pnode upgraded to 4 vcpu

* network/test/cohort2 upgraded to 4 vcpu

* engine upgraded to 4 vcpu

* engine/execution/ingestion upgraded to 4 vcpu, network/test/cohort1 upgraded to 2 vcpu

* network/test/cohort1 upgraded to 4 vcpu

* engine/execution/ingestion upgraded to 8 vcpu

* network/test/cohort1 upgraded to 8 vcpu

* BFT (Protocol) upgraded to 4 vcpu

* network/test/cohort1 upgraded to 16 vcpu

* BFT (Protocol), Epoch Cohort1 upgraded to 8 vcpu

* noop push to kick off CI

* Module (integration) upgraded to 4 vcpu

* TestUnicastRateLimit_Messages flaky test quarantined

* noop push to kick off CI

* buildjet/cache@v3 => actions/cache@v3

* test retries increased to 5, timeout increased to 35 mins

* module job increased to 4 vcpu

* unquarantined epoch flaky test after it was fixed

#4975

* unlink flaky test monitor workflow from running when ci changes

* clean up

* lint fix

* add TargetDuration to models, conversion

* bump core-contracts version

got this error:
```
go: module github.com/onflow/flow-core-contracts@9e8417b found (v0.0.0-20231120143830-9e8417b56122), but does not contain package github.com/onflow/flow-core-contracts/lib/go/templates
```

Maybe because templates did not change since previous version?

* update state commitment constants in tests

* tidy

* update mocks

* Update state/protocol/inmem/convert.go

Co-authored-by: Alexander Hentschel <alex.hentschel@axiomzen.co>

---------

Co-authored-by: Misha <15269764+gomisha@users.noreply.github.com>
Co-authored-by: Alexander Hentschel <alex.hentschel@axiomzen.co>
…e`, `TargetDuration` (#5023)

* unit tests, integration tests using buildjet-4vcpu runner

* listTargetPackages() returns map of CI runners

* test for custom runners

* generateTestMatrix() stores CI runner

* more testing with custom CI runners

* CI test - 2 unit tests running buildjet runners

* increase engine unit tests to 8 vCPUs

* resource manager full load test (8 vCPUs)

* network/test tests unquarantined, increased runners

* increased to 16 vCPUs

* increased engine tests to 16 vCPUs

* insecure module uses buildjet 4 vcpu runner

* bft framework integration test using buildjet

* engine tests split up to 6 jobs with default runners

* added back all integration tests

* upgraded runners for some integration tests, engine/execution

* dummy commit to kick off CI

* split up TestScriptExecutionAndGetAccounts into 3 tests

* updated runners for Epoch Cohort2, Access Integration Tests

* AN integration tests split up into 3 cohorts (skip unit, other integration tests)

* docker build using GitHub cache

* remove saving Docker image locally

* cache v3, saving local tar before caching, 8vcpu

* add 11 docker images to docker-images.tar, 16 vcpu

* docker save multiline fix; skip localnet-test

* docker build increase to 32 vcpu

* buildjet cache instead of GitHub cache

* lint fix

* activate more integration tests with default runners, using cached Docker

* docker-build downsized to 16 vcpu

* integration test without relic build

* put back relic build for integration tests

* put back all remaining integration tests using BuildJet runners

* integration tests reverted to ubuntu-latest runners

* re-activated unit-test, unit-test-modules jobs

* added (failing) test for sub-sub packages

* sub-sub packages support

* sub-sub package test enhanced

* updated other tests to support new sub-sub packages

* split up engine/execution unit tests into 3 jobs

* engine/execution/ingestion upgraded to 4 vcpu

* engine/execution/ingestion:buildjet-8vcpu

* network/test, network/p2p increased to 8 vcpu; added remaining network tests

* BFT Framework, Epoch Cohort 2 - increased to 4 vcpu

* upgraded runners for Access Cohort 1, Epoch Cohort 1, 2

* epoch cohort2 upgraded to 16 vcpu

* BFT Framework upgrade to 8 vcpu, epoch cohort2 removed flaky test, downgrade to 4 vcpu

* quarantined flaky test - TestEpochJoinAndLeaveVN

* quarantined flaky test - TestSealingAndVerificationPassThrough

* lint fix

* network/test split up into network/test/cohort1, network/test/cohort2

* network/p2p split up into 4 subpackages

network/p2p/connection network/p2p/scoring network/p2p/p2pnode network/p2p

* extracted network/alsp into separate job

* extracted module/dkg into separate job

* engine/execution/ingestion dowgraded to stock runner

* added engine parent package as separate job

* added storage parent package as separate job

* added state package as separate job

* removed localnet-test job

* engine/execution/ingestion upgraded to 2 vcpu

* network/p2p/p2pnode upgraded to 2 vcpu

* module, engine upgraded to 2 vcpu

* network/p2p/p2pnode upgraded to 4 vcpu

* network/test/cohort2 upgraded to 4 vcpu

* engine upgraded to 4 vcpu

* engine/execution/ingestion upgraded to 4 vcpu, network/test/cohort1 upgraded to 2 vcpu

* network/test/cohort1 upgraded to 4 vcpu

* engine/execution/ingestion upgraded to 8 vcpu

* network/test/cohort1 upgraded to 8 vcpu

* BFT (Protocol) upgraded to 4 vcpu

* network/test/cohort1 upgraded to 16 vcpu

* BFT (Protocol), Epoch Cohort1 upgraded to 8 vcpu

* noop push to kick off CI

* Module (integration) upgraded to 4 vcpu

* TestUnicastRateLimit_Messages flaky test quarantined

* noop push to kick off CI

* buildjet/cache@v3 => actions/cache@v3

* test retries increased to 5, timeout increased to 35 mins

* module job increased to 4 vcpu

* unquarantined epoch flaky test after it was fixed

#4975

* unlink flaky test monitor workflow from running when ci changes

* clean up

* lint fix

* add TargetEndTime to epoch models

* update service event parsing

* fix some typos and godocs

* update mocks

* lint

* lint

* pin new version of core-contracts

- adds TargetEndTime field
- pinned to branch - needs to be updated when release is created

* update expected state commitment

* update another state commitment constant

* update another state commit constant

* use TargetEndTime in cruisectl

* represent target end time as uint64

* lint / mocks

* use unix time in block time controller

* add todod for epoch fallback flag

* wip

* wip - begin adding duration

* add TargetDuration to models, conversion

* bump core-contracts version

got this error:
```
go: module github.com/onflow/flow-core-contracts@9e8417b found (v0.0.0-20231120143830-9e8417b56122), but does not contain package github.com/onflow/flow-core-contracts/lib/go/templates
```

Maybe because templates did not change since previous version?

* update state commitment constants in tests

* tidy

* update mocks

* track target duration in block time ctl

* remove transition_time

* update tests

* fix last 2 tests

- add helper functions for concisely converting between various
  time/duration types.
- use ns-level precision in view measurement, primarily so that we can
  verify this implementation is consistent with the Python simulation.
  For real-world, second-level precision is likely fine.

* Update consensus/hotstuff/cruisectl/block_time_controller.go

Co-authored-by: Alexander Hentschel <alex.hentschel@axiomzen.co>

* Update consensus/hotstuff/cruisectl/block_time_controller_test.go

* Update consensus/hotstuff/cruisectl/block_time_controller_test.go

* lint

* Apply suggestions from code review

Co-authored-by: Alexander Hentschel <alex.hentschel@axiomzen.co>

* add docs

* add docs for config setter/getters

* gofmt

---------

Co-authored-by: Misha <15269764+gomisha@users.noreply.github.com>
Co-authored-by: Alexander Hentschel <alex.hentschel@axiomzen.co>
* Update cadence to v0.42.5

* use random string

* update godoc

* add in memory register store

* update comments

* add comment and update GetUpdatedRegister

* refactor IsErrPruned

* update tests

* prefix with tests InMemoryRegister

* use t.Run

* Apply suggestions from code review

Co-authored-by: Peter Argue <89119817+peterargue@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Peter Argue <89119817+peterargue@users.noreply.github.com>

* address review comments

* Apply suggestions from code review

Co-authored-by: Gregor G. <75445744+sideninja@users.noreply.github.com>

* fix linter

* fix linter

* fix lint

* use RegisterEntries

* add chainhash to integration

* add chainhash to insecure

* add chainhash to integration

* adds duration to the startup log

* update mocks

* adds TestNewSubscriptionRecordCache

* adds TestGetSubscribedTopics

* adds TestDuplicateTopics

* adds test move update cycle

* adds TestSubscriptionValidator_Integration

* adds TestMoveUpdateCycleWithDifferentPeers

* update emulator version

* updates mocks

* fixes build errors in insecure package

* adds TestSubscriptionProvider_GetSubscribedTopics_SkippingUnknownPeers

* minor fixes

* add comment explainin idProvider mock expectations

* use unittest random string

* add comment about continue statement

- remove obsolete continue statement

* Update fixtures.go

* small fixes

* fixes merge conflicts

* remove unnecessary variable declaration

* add support for testnet chain

* generalize support for all chains

* lint fix

* restoore gating EVM setup behind feature flag

* lint fix

* consolidates PeerIdFixtureB with PeerIdFixture

* adds lint and tidy to insecure package

* lint fix

* Revert "lint fix"

This reverts commit 9b1a4cc.

* lint fix

* revert the account not found error on withdraw

* define direct call types

* update docs

* remove unnecessary event types

* .

* add more contract types for testing

* fix tests

* skip test, requires funding (implemented in follow up PR)

* unskip test

* check error

* • extended compile-time checks for telemetry consumer enforcing that all happy-path interfaces are implemented
• extended subscriptions of telemetry consumer to receive missing events

* reduced notifications consumer interface in PaceMaker, as it only emits events from `hotstuff.ParticipantConsumer`

* update tests

* Refactor event emmision code

* Update network/p2p/unicast/README.MD

Co-authored-by: Khalil Claybon <khalil.claybon@dapperlabs.com>

* expose more test utility

* lint

* add register store

* update LastFinalizedAndExecutedHeight

* convert storage.ErrNotFound to nil

* fix lint

* fixes the logger

* moving ready to the select-case

* moving ready to select-case

* revises the error by update loop to make it irrecoverable

* clean up benchmarking

* adds select case for startup of subscription provider

* Update network/p2p/scoring/subscription_provider.go

Co-authored-by: Khalil Claybon <khalil.claybon@dapperlabs.com>

* adds documentation to subscription record cache

* fixes a bug with subscription record id

* adds more documentation for current cycle

* fix epoch event docs

* [Access] Handle script canceled and timeout errors

* [Access] Make script exec configurable

* Make computation limit configurable on ENs

* set limit setting once

* undo whitespace changes

* add storehouse loader

* add loaders

* add comments update log

* add comments

* update loader

* add extending block snapshot

* fix tests

* fix lint

* update interface

* update tests

* fix lint

* update committer

* update mocks

* update committer tests

* refactor collector

* update mocks

* update bootstrap

* fix noop committer

* fix computer_test

* fix tests

* refactor test

* update committer tests

* fix tests

* Update engine/access/rpc/backend/backend_scripts_test.go

Co-authored-by: Gregor G. <75445744+sideninja@users.noreply.github.com>

* fix committer tests

* add comment

* update tests

* Added implementation for execution api engine

* Added tests for execution api engine

* Fixed issues based on comments

* Added accidentally removed code

* Updated implementations following protobuf changes,updated mocks

* Linted

* [Access] Update websockets event streaming to return JSON-CDC encoded events

* Added tests for sentinel errors

* Update engine/execution/rpc/engine.go

Co-authored-by: Peter Argue <89119817+peterargue@users.noreply.github.com>

* update NewStorageSnapshot

* update mocks

* update tests

* refactor script engine

* update mocks

* fix tests

* fix tests

* remove scripts engine tests

* update errors

* update tests

* refactor tests

* test CreateSnapshot

* add executing block snapshot

* reuse convert functions

* add block_end_snapshot tests

* remove changes

* Apply suggestions from code review

Co-authored-by: Peter Argue <89119817+peterargue@users.noreply.github.com>

* add todo

* Update engine/execution/storehouse/block_end_snapshot_test.go

Co-authored-by: Peter Argue <89119817+peterargue@users.noreply.github.com>

* Update engine/execution/storehouse/block_end_snapshot_test.go

Co-authored-by: Peter Argue <89119817+peterargue@users.noreply.github.com>

* fix lint

* update comment

* add back interface

* update getFromStorage

* handle not found case

* add IsBlockExcuted

* update state commmitment by block id

* update mocks

* fix lint

* update unexecuted_loader

* fix ingestion tests

* fix stop control

* use IsParentExecuted

* fix loader tests

* Fix bug in reencoding and update tests

* update core-contracts version and state commitment constants

This is just to get tests to pass, all these changes will be overwritten
with the update to version v0.15.0 in #5027

---------

Co-authored-by: Yahya Hassanzadeh <yhassanzadeh13@ku.edu.tr>
Co-authored-by: Supun Setunga <supun.setunga@gmail.com>
Co-authored-by: Khalil Claybon <khalil.claybon@dapperlabs.com>
Co-authored-by: Gregor G <75445744+sideninja@users.noreply.github.com>
Co-authored-by: Misha <15269764+gomisha@users.noreply.github.com>
Co-authored-by: Leo Zhang (zhangchiqing) <zhangchiqing@gmail.com>
Co-authored-by: Peter Argue <89119817+peterargue@users.noreply.github.com>
Co-authored-by: Bastian Müller <bastian@turbolent.com>
Co-authored-by: Gregor Gololicic <gregor.gololicic@dapperlabs.com>
Co-authored-by: Ramtin M. Seraj <ramtinms@users.noreply.github.com>
Co-authored-by: ramtinms <ramtin.seraj@dapperlabs.com>
Co-authored-by: Yahya Hassanzadeh, Ph.D <yhassanzadeh@ieee.org>
Co-authored-by: Khalil Claybon <kclaybon1@gmail.com>
Co-authored-by: Alexander Hentschel <alex.hentschel@axiomzen.co>
Co-authored-by: Janez Podhostnik <67895329+janezpodhostnik@users.noreply.github.com>
Co-authored-by: Janez Podhostnik <janez.podhostnik@gmail.com>
Co-authored-by: Jan Bernatik <jan.bernatik@dapperlabs.com>
Co-authored-by: Andrii <andriy.dyachuk95@gmail.com>
Co-authored-by: Yurii Oleksyshyn <yuraolex@gmail.com>
Co-authored-by: Kan Zhang <kan@dapperlabs.com>
…ss (#5027)

* lint fix

* consolidates PeerIdFixtureB with PeerIdFixture

* adds lint and tidy to insecure package

* lint fix

* Revert "lint fix"

This reverts commit 9b1a4cc.

* lint fix

* revert the account not found error on withdraw

* define direct call types

* update docs

* remove unnecessary event types

* .

* add more contract types for testing

* fix tests

* skip test, requires funding (implemented in follow up PR)

* unskip test

* check error

* • extended compile-time checks for telemetry consumer enforcing that all happy-path interfaces are implemented
• extended subscriptions of telemetry consumer to receive missing events

* reduced notifications consumer interface in PaceMaker, as it only emits events from `hotstuff.ParticipantConsumer`

* update tests

* update another state commit constant

* Refactor event emmision code

* use TargetEndTime in cruisectl

* Update network/p2p/unicast/README.MD

Co-authored-by: Khalil Claybon <khalil.claybon@dapperlabs.com>

* expose more test utility

* lint

* add TargetEndTime to epoch setup fixture

* add register store

* update LastFinalizedAndExecutedHeight

* convert storage.ErrNotFound to nil

* fix lint

* fixes the logger

* add epoch timing config flags to finalize cmd

* moving ready to the select-case

* moving ready to select-case

* revises the error by update loop to make it irrecoverable

* clean up benchmarking

* adds select case for startup of subscription provider

* Update network/p2p/scoring/subscription_provider.go

Co-authored-by: Khalil Claybon <khalil.claybon@dapperlabs.com>

* adds documentation to subscription record cache

* fixes a bug with subscription record id

* adds more documentation for current cycle

* represent target end time as uint64

* lint / mocks

* fix epoch event docs

* use unix time in block time controller

* add todod for epoch fallback flag

* [Access] Handle script canceled and timeout errors

* update fixtures

* validation of timing config flags

* set target end time

* add tests for epoch timing validation

* change cruise-ctl-enabled to default true

* disable cruise-ctl in integration tests

* tweak wording in comment

* remove heuristic code

* remove unneeded config field and flag

* [Access] Make script exec configurable

* Make computation limit configurable on ENs

* wip

* set limit setting once

* undo whitespace changes

* add storehouse loader

* add loaders

* add comments update log

* add comments

* update loader

* add extending block snapshot

* fix tests

* fix lint

* update interface

* update tests

* fix lint

* update committer

* update mocks

* update committer tests

* refactor collector

* update mocks

* update bootstrap

* fix noop committer

* fix computer_test

* fix tests

* refactor test

* update committer tests

* fix tests

* wip - begin adding duration

* add TargetDuration to models, conversion

* bump core-contracts version

got this error:
```
go: module github.com/onflow/flow-core-contracts@9e8417b found (v0.0.0-20231120143830-9e8417b56122), but does not contain package github.com/onflow/flow-core-contracts/lib/go/templates
```

Maybe because templates did not change since previous version?

* update state commitment constants in tests

* tidy

* update mocks

* track target duration in block time ctl

* remove transition_time

* update tests

* Update engine/access/rpc/backend/backend_scripts_test.go

Co-authored-by: Gregor G. <75445744+sideninja@users.noreply.github.com>

* fix last 2 tests

- add helper functions for concisely converting between various
  time/duration types.
- use ns-level precision in view measurement, primarily so that we can
  verify this implementation is consistent with the Python simulation.
  For real-world, second-level precision is likely fine.

* Update consensus/hotstuff/cruisectl/block_time_controller.go

Co-authored-by: Alexander Hentschel <alex.hentschel@axiomzen.co>

* Update consensus/hotstuff/cruisectl/block_time_controller_test.go

* Update consensus/hotstuff/cruisectl/block_time_controller_test.go

* lint

* add target duration

* fix committer tests

* add comment

* more descriptive util names

* update tests

* Added implementation for execution api engine

* Added tests for execution api engine

* pass through root block in exe state bootstrap

* remove hard-coded genesis in Bootstrap

The bootstrap was previously hard-coded to use the genesis block, even
if a differing block was generated.

* setup equality checks new fields

* update conversion and add tests

* Fixed issues based on comments

* Added accidentally removed code

* Updated implementations following protobuf changes,updated mocks

* Linted

* [Access] Update websockets event streaming to return JSON-CDC encoded events

* Added tests for sentinel errors

* Update engine/execution/rpc/engine.go

Co-authored-by: Peter Argue <89119817+peterargue@users.noreply.github.com>

* update dependencies

- add explicit import for problematic package

* lint

* conditionally default to hard-coded genesis

* update NewStorageSnapshot

* update mocks

* update tests

* refactor script engine

* update mocks

* fix tests

* fix tests

* remove scripts engine tests

* update errors

* update tests

* refactor tests

* test CreateSnapshot

* add executing block snapshot

* reuse convert functions

* add block_end_snapshot tests

* remove changes

* Apply suggestions from code review

Co-authored-by: Peter Argue <89119817+peterargue@users.noreply.github.com>

* add todo

* Update engine/execution/storehouse/block_end_snapshot_test.go

Co-authored-by: Peter Argue <89119817+peterargue@users.noreply.github.com>

* Update engine/execution/storehouse/block_end_snapshot_test.go

Co-authored-by: Peter Argue <89119817+peterargue@users.noreply.github.com>

* fix lint

* update comment

* add back interface

* update getFromStorage

* handle not found case

* add IsBlockExcuted

* update state commmitment by block id

* update mocks

* fix lint

* update unexecuted_loader

* fix ingestion tests

* fix stop control

* use IsParentExecuted

* fix loader tests

* Fix bug in reencoding and update tests

* Apply suggestions from code review

Co-authored-by: Alexander Hentschel <alex.hentschel@axiomzen.co>

* add docs

* add docs for config setter/getters

* Apply suggestions from code review

Co-authored-by: Alexander Hentschel <alex.hentschel@axiomzen.co>

* update docs

* address review feedback

* gofmt

* address review feedback

* correct comment

* add default timing flag to bn2

* lint

* bump flow-go crypto version to v0.24.10

* update tests

* state commitment constants

* update flow-emu version

* update core-contracts version and state commitment constants

This is just to get tests to pass, all these changes will be overwritten
with the update to version v0.15.0 in #5027

* make tidy

* update test

---------

Co-authored-by: Yahya Hassanzadeh <yhassanzadeh13@ku.edu.tr>
Co-authored-by: Yahya Hassanzadeh, Ph.D <yhassanzadeh@ieee.org>
Co-authored-by: Bastian Müller <bastian@turbolent.com>
Co-authored-by: ramtinms <ramtin.seraj@dapperlabs.com>
Co-authored-by: Alexander Hentschel <alex.hentschel@axiomzen.co>
Co-authored-by: Janez Podhostnik <67895329+janezpodhostnik@users.noreply.github.com>
Co-authored-by: Janez Podhostnik <janez.podhostnik@gmail.com>
Co-authored-by: Ramtin M. Seraj <ramtinms@users.noreply.github.com>
Co-authored-by: Khalil Claybon <khalil.claybon@dapperlabs.com>
Co-authored-by: Leo Zhang (zhangchiqing) <zhangchiqing@gmail.com>
Co-authored-by: Gregor G <75445744+sideninja@users.noreply.github.com>
Co-authored-by: Jan Bernatik <jan.bernatik@dapperlabs.com>
Co-authored-by: Peter Argue <89119817+peterargue@users.noreply.github.com>
Co-authored-by: Andrii <andriy.dyachuk95@gmail.com>
Co-authored-by: Yurii Oleksyshyn <yuraolex@gmail.com>
Co-authored-by: Kan Zhang <kan@dapperlabs.com>
@codecov-commenter
Copy link

codecov-commenter commented Nov 27, 2023

Codecov Report

Attention: 658 lines in your changes are missing coverage. Please review.

Comparison is base (cafb1a6) 56.02% compared to head (0b64668) 56.23%.
Report is 574 commits behind head on master.

Files Patch % Lines
engine/access/apiproxy/access_api_proxy.go 2.38% 82 Missing ⚠️
module/metrics/execution.go 0.00% 71 Missing ⚠️
fvm/evm/stdlib/contract.go 82.70% 33 Missing and 27 partials ⚠️
engine/execution/storehouse/register_store.go 70.42% 32 Missing and 10 partials ⚠️
engine/execution/rpc/engine.go 66.66% 33 Missing and 4 partials ⚠️
network/netconf/flags.go 0.00% 34 Missing ⚠️
...sensus/hotstuff/cruisectl/block_time_controller.go 54.23% 19 Missing and 8 partials ⚠️
engine/execution/storehouse/register_engine.go 0.00% 26 Missing ⚠️
fvm/bootstrap.go 26.92% 18 Missing and 1 partial ⚠️
engine/access/rpc/backend/backend_transactions.go 61.70% 13 Missing and 5 partials ⚠️
... and 32 more
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #5062      +/-   ##
==========================================
+ Coverage   56.02%   56.23%   +0.21%     
==========================================
  Files         965      976      +11     
  Lines       89671    91261    +1590     
==========================================
+ Hits        50241    51325    +1084     
- Misses      35678    36089     +411     
- Partials     3752     3847      +95     
Flag Coverage Δ
unittests 56.23% <63.94%> (+0.21%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

* prefix with tests InMemoryRegister

* use t.Run

* Apply suggestions from code review

Co-authored-by: Peter Argue <89119817+peterargue@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Peter Argue <89119817+peterargue@users.noreply.github.com>

* address review comments

* Apply suggestions from code review

Co-authored-by: Gregor G. <75445744+sideninja@users.noreply.github.com>

* fix linter

* fix linter

* fix lint

* use RegisterEntries

* add chainhash to integration

* add chainhash to insecure

* add chainhash to integration

* adds duration to the startup log

* update mocks

* adds TestNewSubscriptionRecordCache

* adds TestGetSubscribedTopics

* adds TestDuplicateTopics

* adds test move update cycle

* adds TestSubscriptionValidator_Integration

* adds TestMoveUpdateCycleWithDifferentPeers

* update emulator version

* updates mocks

* fixes build errors in insecure package

* adds TestSubscriptionProvider_GetSubscribedTopics_SkippingUnknownPeers

* minor fixes

* add comment explainin idProvider mock expectations

* use unittest random string

* add comment about continue statement

- remove obsolete continue statement

* Update fixtures.go

* Added error filtering for CB to pass some gRPC errors.

* Removed unnecessary code

* Removed unnecessary code

* small fixes

* fixes merge conflicts

* remove unnecessary variable declaration

* add support for testnet chain

* generalize support for all chains

* lint fix

* restoore gating EVM setup behind feature flag

* lint fix

* consolidates PeerIdFixtureB with PeerIdFixture

* adds lint and tidy to insecure package

* lint fix

* Revert "lint fix"

This reverts commit 9b1a4cc.

* lint fix

* revert the account not found error on withdraw

* define direct call types

* update docs

* remove unnecessary event types

* .

* add more contract types for testing

* fix tests

* skip test, requires funding (implemented in follow up PR)

* unskip test

* check error

* • extended compile-time checks for telemetry consumer enforcing that all happy-path interfaces are implemented
• extended subscriptions of telemetry consumer to receive missing events

* reduced notifications consumer interface in PaceMaker, as it only emits events from `hotstuff.ParticipantConsumer`

* update tests

* Refactor event emmision code

* Update network/p2p/unicast/README.MD

Co-authored-by: Khalil Claybon <khalil.claybon@dapperlabs.com>

* expose more test utility

* lint

* add register store

* update LastFinalizedAndExecutedHeight

* convert storage.ErrNotFound to nil

* fix lint

* fixes the logger

* moving ready to the select-case

* moving ready to select-case

* revises the error by update loop to make it irrecoverable

* clean up benchmarking

* adds select case for startup of subscription provider

* Update network/p2p/scoring/subscription_provider.go

Co-authored-by: Khalil Claybon <khalil.claybon@dapperlabs.com>

* adds documentation to subscription record cache

* fixes a bug with subscription record id

* adds more documentation for current cycle

* fix epoch event docs

* [Access] Handle script canceled and timeout errors

* [Access] Make script exec configurable

* Make computation limit configurable on ENs

* set limit setting once

* undo whitespace changes

* add storehouse loader

* add loaders

* add comments update log

* add comments

* update loader

* add extending block snapshot

* fix tests

* fix lint

* update interface

* update tests

* fix lint

* update committer

* update mocks

* update committer tests

* refactor collector

* update mocks

* update bootstrap

* fix noop committer

* fix computer_test

* fix tests

* refactor test

* update committer tests

* fix tests

* Update engine/access/rpc/backend/backend_scripts_test.go

Co-authored-by: Gregor G. <75445744+sideninja@users.noreply.github.com>

* fix committer tests

* add comment

* update tests

* Added implementation for execution api engine

* Added tests for execution api engine

* Fixed issues based on comments

* Added accidentally removed code

* Updated implementations following protobuf changes,updated mocks

* Linted

* [Access] Update websockets event streaming to return JSON-CDC encoded events

* Added tests for sentinel errors

* Update engine/execution/rpc/engine.go

Co-authored-by: Peter Argue <89119817+peterargue@users.noreply.github.com>

* update NewStorageSnapshot

* update mocks

* update tests

* refactor script engine

* update mocks

* fix tests

* fix tests

* remove scripts engine tests

* update errors

* update tests

* refactor tests

* test CreateSnapshot

* add executing block snapshot

* reuse convert functions

* add block_end_snapshot tests

* remove changes

* Apply suggestions from code review

Co-authored-by: Peter Argue <89119817+peterargue@users.noreply.github.com>

* add todo

* Update engine/execution/storehouse/block_end_snapshot_test.go

Co-authored-by: Peter Argue <89119817+peterargue@users.noreply.github.com>

* Update engine/execution/storehouse/block_end_snapshot_test.go

Co-authored-by: Peter Argue <89119817+peterargue@users.noreply.github.com>

* fix lint

* update comment

* add back interface

* update getFromStorage

* handle not found case

* add IsBlockExcuted

* update state commmitment by block id

* update mocks

* fix lint

* update unexecuted_loader

* fix ingestion tests

* fix stop control

* use IsParentExecuted

* fix loader tests

* Fix bug in reencoding and update tests

* Added test to emulate errors treated as success for cb

* Replaced check with switch

---------

Co-authored-by: Leo Zhang (zhangchiqing) <zhangchiqing@gmail.com>
Co-authored-by: Peter Argue <89119817+peterargue@users.noreply.github.com>
Co-authored-by: Gregor G. <75445744+sideninja@users.noreply.github.com>
Co-authored-by: Bastian Müller <bastian@turbolent.com>
Co-authored-by: Gregor Gololicic <gregor.gololicic@dapperlabs.com>
Co-authored-by: Yahya Hassanzadeh <yhassanzadeh13@ku.edu.tr>
Co-authored-by: Ramtin M. Seraj <ramtinms@users.noreply.github.com>
Co-authored-by: ramtinms <ramtin.seraj@dapperlabs.com>
Co-authored-by: Yahya Hassanzadeh, Ph.D <yhassanzadeh@ieee.org>
Co-authored-by: Khalil Claybon <khalil.claybon@dapperlabs.com>
Co-authored-by: Khalil Claybon <kclaybon1@gmail.com>
Co-authored-by: Andrii Slisarchuk <andriyslisarchuk@gmail.com>
Co-authored-by: Andrii Slisarchuk <Guitarheroua@users.noreply.github.com>
Co-authored-by: Alexander Hentschel <alex.hentschel@axiomzen.co>
Co-authored-by: Janez Podhostnik <67895329+janezpodhostnik@users.noreply.github.com>
Co-authored-by: Janez Podhostnik <janez.podhostnik@gmail.com>
Co-authored-by: Jan Bernatik <jan.bernatik@dapperlabs.com>
Co-authored-by: Andrii <andriy.dyachuk95@gmail.com>
Co-authored-by: Yurii Oleksyshyn <yuraolex@gmail.com>
Co-authored-by: Kan Zhang <kan@dapperlabs.com>
* define direct call types

* update docs

* remove unnecessary event types

* .

* add more contract types for testing

* fix tests

* skip test, requires funding (implemented in follow up PR)

* unskip test

* check error

* • extended compile-time checks for telemetry consumer enforcing that all happy-path interfaces are implemented
• extended subscriptions of telemetry consumer to receive missing events

* reduced notifications consumer interface in PaceMaker, as it only emits events from `hotstuff.ParticipantConsumer`

* update tests

* Refactor event emmision code

* moves fixtures from insecure to p2ptest

* adds gossipsub message fixture function

* refactors test fixtures

* creates GossipSubRpcFixture

* adds documentation

* adds godoc

* Update network/p2p/unicast/README.MD

Co-authored-by: Khalil Claybon <khalil.claybon@dapperlabs.com>

* expose more test utility

* lint

* add register store

* update LastFinalizedAndExecutedHeight

* convert storage.ErrNotFound to nil

* fix lint

* fixes the logger

* moving ready to the select-case

* moving ready to select-case

* revises the error by update loop to make it irrecoverable

* clean up benchmarking

* adds select case for startup of subscription provider

* Update network/p2p/scoring/subscription_provider.go

Co-authored-by: Khalil Claybon <khalil.claybon@dapperlabs.com>

* adds documentation to subscription record cache

* fixes a bug with subscription record id

* adds more documentation for current cycle

* untangled Register API handler

* fix epoch event docs

* chnages per suggestions

* remove mock

* [Access] Handle script canceled and timeout errors

* [Access] Make script exec configurable

* Make computation limit configurable on ENs

* set limit setting once

* undo whitespace changes

* add storehouse loader

* add loaders

* add comments update log

* add comments

* update loader

* add extending block snapshot

* fix tests

* fix lint

* update interface

* update tests

* fix lint

* update committer

* update mocks

* update committer tests

* refactor collector

* update mocks

* update bootstrap

* fix noop committer

* fix computer_test

* fix tests

* refactor test

* update committer tests

* fix tests

* Changed input parameters for new methods

* Updated according to comments

* Update engine/access/rpc/backend/backend_scripts_test.go

Co-authored-by: Gregor G. <75445744+sideninja@users.noreply.github.com>

* fix committer tests

* add comment

* update tests

* Added implementation for execution api engine

* Added tests for execution api engine

* Added implementation for method for returning tx result

* Fixed issues based on comments

* Added accidentally removed code

* Updated implementations following protobuf changes,updated mocks

* Linted

* [Access] Update websockets event streaming to return JSON-CDC encoded events

* Added tests for sentinel errors

* Update engine/execution/rpc/engine.go

Co-authored-by: Peter Argue <89119817+peterargue@users.noreply.github.com>

* Generated mocks, added missing argument to method:

* fix per suggestion

* cleanup

* fix mocks

* lint

* update NewStorageSnapshot

* update mocks

* update tests

* refactor script engine

* update mocks

* fix tests

* fix tests

* remove scripts engine tests

* update errors

* update tests

* refactor tests

* test CreateSnapshot

* add executing block snapshot

* reuse convert functions

* add block_end_snapshot tests

* remove changes

* Apply suggestions from code review

Co-authored-by: Peter Argue <89119817+peterargue@users.noreply.github.com>

* add todo

* Update engine/execution/storehouse/block_end_snapshot_test.go

Co-authored-by: Peter Argue <89119817+peterargue@users.noreply.github.com>

* Update engine/execution/storehouse/block_end_snapshot_test.go

Co-authored-by: Peter Argue <89119817+peterargue@users.noreply.github.com>

* fix lint

* update comment

* add back interface

* update getFromStorage

* handle not found case

* add IsBlockExcuted

* update state commmitment by block id

* update mocks

* fix lint

* update unexecuted_loader

* fix ingestion tests

* fix stop control

* use IsParentExecuted

* chnages to error handling

* Added test

* fix loader tests

* Fix bug in reencoding and update tests

* Fixd remarks

* Added test to emulate errors treated as success for cb

* Added test cases for checking wrong input

* Merged and Linted

* Returned back apiTimeout and used it in connection factory

* Updated test

* Linted

* remove index reporter for registersAsync

* Added parameter blockID to GetSystemTransaction, so it will be more clear that it's return system tx for block, generated mocks, fixed test

* Updated existing functional test to check upstream failover

* use atomic pointer

* Updated documentaion according to comments

* Upgraded godoc

* Replaced check with switch

* Added event with payload to test decoding, linted

* Removed replace and added current version of flow proto

* Replace flow-emulator for integration tests

* changes per suggestions

* correct compareAndSwap

* make tidy

* Fixed errors

* Added more comments for tests

* changed version of flow-emulator

* Removed comments

* modifies the flow-emulator version pinned to match replace statement

See:
- onflow/flow-emulator#514
- #5049

---------

Co-authored-by: ramtinms <ramtin.seraj@dapperlabs.com>
Co-authored-by: Bastian Müller <bastian@turbolent.com>
Co-authored-by: Alexander Hentschel <alex.hentschel@axiomzen.co>
Co-authored-by: Andrii Slisarchuk <Guitarheroua@users.noreply.github.com>
Co-authored-by: Janez Podhostnik <67895329+janezpodhostnik@users.noreply.github.com>
Co-authored-by: Janez Podhostnik <janez.podhostnik@gmail.com>
Co-authored-by: Ramtin M. Seraj <ramtinms@users.noreply.github.com>
Co-authored-by: Yahya Hassanzadeh, Ph.D <yhassanzadeh@ieee.org>
Co-authored-by: Yahya Hassanzadeh <yhassanzadeh13@ku.edu.tr>
Co-authored-by: Khalil Claybon <khalil.claybon@dapperlabs.com>
Co-authored-by: Leo Zhang (zhangchiqing) <zhangchiqing@gmail.com>
Co-authored-by: Gregor G <75445744+sideninja@users.noreply.github.com>
Co-authored-by: Jan Bernatik <jan.bernatik@dapperlabs.com>
Co-authored-by: Amlandeep Bhadra <amlandeep1912@gmail.com>
Co-authored-by: Peter Argue <89119817+peterargue@users.noreply.github.com>
Co-authored-by: Andrii <andriy.dyachuk95@gmail.com>
Co-authored-by: UlyanaAndrukhiv <u.andrukhiv@gmail.com>
Co-authored-by: Yurii Oleksyshyn <yuraolex@gmail.com>
Co-authored-by: Amlandeep Bhadra <koko1123@users.noreply.github.com>
Co-authored-by: Andrii Slisarchuk <andriyslisarchuk@gmail.com>
Co-authored-by: Kan Zhang <kan@dapperlabs.com>
@jordanschalm
Copy link
Member Author

Closing in favour of #5071

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants