Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BFT Testing] Reducing load of BFT tests and improving test packages #4652

Merged
merged 50 commits into from
Aug 31, 2023

Conversation

yhassanzadeh13
Copy link
Contributor

@yhassanzadeh13 yhassanzadeh13 commented Aug 22, 2023

This PR introduces essential corrections to enhance the stability of the BFT integration tests. The improvements are as follows:

  1. Optimizing Network Size: The network size for individual tests has been reduced to the minimum required, shrinking unnecessary complexity and resource usage.
  2. Bug Fix and Test for Configuration Filter: A bug within the configuration filter method has been fixed, and a corresponding test has been written to ensure its correct functionality.
  3. Reducing BFT Test Load: The load imposed by the BFT tests has been minimized, enhancing the efficiency and performance of the testing process.
  4. Enhanced Message Assessment: A "require eventually" strategy has been employed instead of immediate evaluations for received messages.

This PR also adds readme for individual BFT test packages.

This PR also temporarily quarantines the following tests, which will be un-quarantined once we split the network CI job on unit tests into smaller jobs:

  1. TestStubEngineTestSuite/TestDuplicateMessageDifferentChan_Publish
  2. MeshEngineTestSuite/TestMaxMessageSize_Multicast
  3. MeshEngineTestSuite/TestMaxMessageSize_Publish
  4. EchoEngineTestSuite/TestDuplicateMessageParallel_Publish
  5. EchoEngineTestSuite/TestDuplicateMessageDifferentChan_Multicast
  6. EchoEngineTestSuite/TestEchoMultiMsgAsync_Multicast
  7. EchoEngineTestSuite/TestDuplicateMessageParallel_Multicast

@codecov-commenter
Copy link

codecov-commenter commented Aug 22, 2023

Codecov Report

Patch coverage: 70.61% and project coverage change: -1.22% ⚠️

Comparison is base (2528190) 56.25% compared to head (26f70c3) 55.03%.
Report is 944 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #4652      +/-   ##
==========================================
- Coverage   56.25%   55.03%   -1.22%     
==========================================
  Files         653      819     +166     
  Lines       64699    78838   +14139     
==========================================
+ Hits        36396    43389    +6993     
- Misses      25362    32024    +6662     
- Partials     2941     3425     +484     
Flag Coverage Δ
unittests 55.03% <70.61%> (-1.22%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Changed Coverage Δ
cmd/bootstrap/transit/cmd/utils.go 31.54% <ø> (ø)
cmd/execution_builder.go 0.00% <0.00%> (ø)
cmd/execution_config.go 0.00% <0.00%> (ø)
engine/access/rest/middleware/metrics.go 0.00% <0.00%> (ø)
engine/access/rest/request/request.go 0.00% <0.00%> (ø)
engine/access/rest/routes/execution_result.go 56.75% <0.00%> (ø)
engine/access/rpc/backend/backend_network.go 34.11% <0.00%> (ø)
engine/access/state_stream/engine.go 0.00% <0.00%> (ø)
engine/common/rpc/convert/blocks.go 78.76% <ø> (ø)
engine/execution/state/state.go 33.82% <0.00%> (-1.57%) ⬇️
... and 68 more

... and 182 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@yhassanzadeh13 yhassanzadeh13 marked this pull request as ready for review August 23, 2023 19:11
@yhassanzadeh13 yhassanzadeh13 added this pull request to the merge queue Aug 31, 2023
Merged via the queue into master with commit bc25e34 Aug 31, 2023
36 checks passed
@yhassanzadeh13 yhassanzadeh13 deleted the yahya/6857-bft-testing-ci-fix-2 branch August 31, 2023 15:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants