Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Mutations Testing] for PR: Feat/nakamoto block push #4877 #4881

Open
ASuciuX opened this issue Jun 12, 2024 · 3 comments · Fixed by #4918
Open

[Mutations Testing] for PR: Feat/nakamoto block push #4877 #4881

ASuciuX opened this issue Jun 12, 2024 · 3 comments · Fixed by #4918

Comments

@ASuciuX
Copy link
Contributor

ASuciuX commented Jun 12, 2024

Is your feature request related to a problem? Please describe.
Mutations Testing works by modifying the body of the function to a default return.
eg. if i have a sum function which returns int -> the mutant would replace it with return 0

The problem with http request/response functions is that they are waiting for that request/response, but it was erased from the function as it only contains the return value. So the mutant tries to run the unit tests and timeouts.

Describe the solution you'd like
These functions cannot be tested with mutants so best approach is to skip them by using skip-headers for functions or files.

Resources

@ASuciuX
Copy link
Contributor Author

ASuciuX commented Jun 12, 2024

Timeout Mutants

These are the initial timed out mutants as each of the workflow jobs stopped after 360 minutes. Probably after skipping these, some new functions would be highlighted.

  • stackslib/src/net/p2p.rs:1353:9: replace PeerNetwork::dispatch_request -> Result<(), net_error> with Ok(()) in 202.5s build + 2865.0s test
  • stackslib/src/net/relay.rs:2416:9: replace Relayer::process_new_epoch2_blocks -> (u64, u64, u64) with (0, 0, 1) in 193.4s build + 2869.2s test
  • stackslib/src/net/unsolicited.rs:57:9: replace PeerNetwork::check_peer_authenticated -> Option with None in 193.2s build + 2865.0s test
  • stackslib/src/net/unsolicited.rs:254:9: replace PeerNetwork::buffer_data_message -> bool with false in 71.8s build + 2865.1s test
  • stackslib/src/net/relay.rs:2416:9: replace Relayer::process_new_epoch2_blocks -> (u64, u64, u64) with (0, 1, 0) in 194.7s build + 2832.3s test
  • stackslib/src/net/relay.rs:2495:9: replace Relayer::relay_epoch3_blocks with () in 197.7s build + 2828.1s test
  • stackslib/src/net/relay.rs:1928:9: replace Relayer::process_new_nakamoto_blocks -> Result<(Vec, Vec), net_error> with Ok((vec![], vec![])) in 214.1s build + 2868.9s test
  • stackslib/src/net/relay.rs:2416:9: replace Relayer::process_new_epoch2_blocks -> (u64, u64, u64) with (0, 1, 1) in 210.8s build + 2869.2s test
  • stackslib/src/net/relay.rs:2591:9: replace Relayer::process_new_epoch3_blocks -> u64 with 0 in 208.9s build + 2868.9s test
  • stackslib/src/net/relay.rs:2416:9: replace Relayer::process_new_epoch2_blocks -> (u64, u64, u64) with (1, 0, 0) in 208.4s build + 2824.3s test
  • stackslib/src/net/relay.rs:2591:9: replace Relayer::process_new_epoch3_blocks -> u64 with 1 in 245.4s build + 2823.6s test
  • stackslib/src/net/relay.rs:950:9: replace Relayer::process_downloaded_nakamoto_blocks -> Result<Vec, chainstate_error> with Ok(vec![]) in 184.4s build + 2845.9s test
  • stackslib/src/net/relay.rs:2416:9: replace Relayer::process_new_epoch2_blocks -> (u64, u64, u64) with (1, 0, 1) in 197.5s build + 2846.4s test
  • stackslib/src/net/relay.rs:2638:9: replace Relayer::process_new_transactions -> Vec with vec![] in 194.9s build + 2842.3s test
  • stackslib/src/net/relay.rs:2416:9: replace Relayer::process_new_epoch2_blocks -> (u64, u64, u64) with (1, 1, 0) in 197.1s build + 2778.3s test
  • stackslib/src/net/unsolicited.rs:748:9: replace PeerNetwork::find_nakamoto_block_reward_cycle -> (Option, bool) with (None, true) in 156.6s build + 2774.1s test
  • stackslib/src/net/unsolicited.rs:843:9: replace PeerNetwork::inner_handle_unsolicited_NakamotoBlocksData -> bool with true in 126.9s build + 2774.1s test
  • stackslib/src/net/codec.rs:359:9: replace ::consensus_serialize -> Result<(), codec_error> with Ok(()) in 167.1s build + 2830.0s test
  • stackslib/src/net/relay.rs:1542:9: replace Relayer::process_pushed_nakamoto_blocks -> Result<(Vec, Vec), net_error> with Ok((vec![], vec![])) in 201.8s build + 2830.1s test
  • stackslib/src/net/relay.rs:2416:9: replace Relayer::process_new_epoch2_blocks -> (u64, u64, u64) with (1, 1, 1) in 157.6s build + 2834.3s test
  • stackslib/src/net/unsolicited.rs:174:9: replace PeerNetwork::can_buffer_data_message -> bool with false in 192.5s build + 2830.1s test
  • stackslib/src/net/relay.rs:2416:9: replace Relayer::process_new_epoch2_blocks -> (u64, u64, u64) with (0, 0, 0) in 195.6s build + 2865.1s test
  • stackslib/src/net/relay.rs:2468:9: replace Relayer::get_last_n_sortitions -> Result<Vec, chainstate_error> with Ok(vec![]) in 196.5s build + 2861.1s test
  • stackslib/src/net/unsolicited.rs:748:9: replace PeerNetwork::find_nakamoto_block_reward_cycle -> (Option, bool) with (Some(0), true) in 155.8s build + 2861.1s test
  • stackslib/src/net/unsolicited.rs:922:9: replace PeerNetwork::handle_unsolicited_message -> (bool, bool) with (true, false) in 165.4s build + 2822.1s test
  • stackslib/src/net/unsolicited.rs:715:9: replace PeerNetwork::check_nakamoto_block_signer_signature -> bool with false in 153.2s build + 2818.1s test
  • stackslib/src/net/unsolicited.rs:922:9: replace PeerNetwork::handle_unsolicited_message -> (bool, bool) with (false, false) in 165.5s build + 2818.1s test
  • stackslib/src/net/unsolicited.rs:804:9: replace PeerNetwork::is_nakamoto_block_bufferable -> bool with true in 162.8s build + 2770.1s test
  • stackslib/src/net/unsolicited.rs:1022:9: replace PeerNetwork::handle_unsolicited_messages -> HashMap<NeighborKey, Vec> with HashMap::new() in 197.4s build + 2770.1s test
  • stackslib/src/net/unsolicited.rs:804:9: replace PeerNetwork::is_nakamoto_block_bufferable -> bool with false in 157.7s build + 2825.0s test
  • stackslib/src/net/unsolicited.rs:889:9: replace PeerNetwork::handle_unsolicited_NakamotoBlocksData -> bool with false in 160.7s build + 2835.1s test

@ASuciuX
Copy link
Contributor Author

ASuciuX commented Jun 12, 2024

Missed Mutants

These are the initial missed out mutants as each of the workflow jobs stopped after 360 minutes. They highlight these functions have cases not tested. Probably after skipping the timeouts mutants, other mutants would appear as missed/timeout/unviable.

  • stackslib/src/net/relay.rs:524:9: replace Relayer::validate_blocks_push -> Result<(), net_error> with Ok(()) in 205.8s build + 1906.7s test
  • stackslib/src/net/mod.rs:445:9: replace ::cause -> Option<&dyn error::Error> with None in 205.6s build + 1835.5s test
  • stackslib/src/net/codec.rs:969:9: replace StacksMessageType::get_message_name -> &'static str with "" in 131.7s build + 1881.1s test
  • stackslib/src/net/relay.rs:564:9: replace Relayer::validate_nakamoto_blocks_push -> Result<(), net_error> with Ok(()) in 190.8s build + 1869.1s test
  • stackslib/src/net/unsolicited.rs:289:9: replace PeerNetwork::need_block_or_microblock_stream -> Result<bool, NetError> with Ok(true) in 195.7s build + 1849.5s test
  • stackslib/src/net/unsolicited.rs:663:9: replace PeerNetwork::handle_unsolicited_MicroblocksData -> bool with true in 158.8s build + 1883.8s test
  • stackslib/src/net/chat.rs:2251:9: replace ConversationP2P::validate_nakamoto_block_push -> Result<Option, net_error> with Ok(None) in 58.1s build + 1911.0s test
  • stackslib/src/net/codec.rs:969:9: replace StacksMessageType::get_message_name -> &'static str with "xyzzy" in 136.3s build + 1907.5s test
  • stackslib/src/net/unsolicited.rs:93:9: replace PeerNetwork::handle_unsolicited_inv_update_epoch2x -> Result<Option, NetError> with Ok(None) in 178.5s build + 1909.3s test
  • stackslib/src/net/unsolicited.rs:289:9: replace PeerNetwork::need_block_or_microblock_stream -> Result<bool, NetError> with Ok(false) in 160.6s build + 1901.2s test
  • stackslib/src/net/mod.rs:1545:9: replace NetworkResult::has_nakamoto_blocks -> bool with true in 39.7s build + 1879.5s test
  • stackslib/src/net/codec.rs:1004:9: replace StacksMessageType::get_message_description -> String with String::new() in 207.6s build + 1934.5s test
  • stackslib/src/net/relay.rs:187:9: replace ::get_id -> String with String::new() in 202.2s build + 1896.4s test
  • stackslib/src/net/unsolicited.rs:93:9: replace PeerNetwork::handle_unsolicited_inv_update_epoch2x -> Result<Option, NetError> with Ok(Some(0)) in 197.7s build + 1898.8s test
  • stackslib/src/net/unsolicited.rs:333:9: replace PeerNetwork::handle_unsolicited_BlocksAvailable -> bool with true in 166.8s build + 1835.1s test
  • stackslib/src/net/unsolicited.rs:712:9: replace PeerNetwork::check_nakamoto_block_signer_signature -> bool with true in 172.5s build + 1838.3s test
  • stackslib/src/net/mod.rs:1545:9: replace NetworkResult::has_nakamoto_blocks -> bool with false in 34.7s build + 1881.8s test
  • stackslib/src/net/codec.rs:1004:9: replace StacksMessageType::get_message_description -> String with "xyzzy".into() in 179.4s build + 1893.5s test
  • stackslib/src/net/relay.rs:187:9: replace ::get_id -> String with "xyzzy".into() in 175.4s build + 1879.3s test
  • stackslib/src/net/unsolicited.rs:93:9: replace PeerNetwork::handle_unsolicited_inv_update_epoch2x -> Result<Option, NetError> with Ok(Some(1)) in 155.4s build + 1876.6s test
  • stackslib/src/net/unsolicited.rs:447:9: replace PeerNetwork::handle_unsolicited_MicroblocksAvailable -> bool with true in 141.7s build + 1857.7s test
  • stackslib/src/net/relay.rs:2351:9: replace Relayer::relay_epoch2_blocks with () in 192.3s build + 1839.8s test
  • stackslib/src/net/unsolicited.rs:447:9: replace PeerNetwork::handle_unsolicited_MicroblocksAvailable -> bool with false in 136.7s build + 1879.6s test
  • stackslib/src/net/relay.rs:3166:9: replace PeerNetwork::update_relayer_stats -> () with () in 199.0s build + 1903.7s test
  • stackslib/src/net/unsolicited.rs:547:9: replace PeerNetwork::handle_unsolicited_BlocksData -> bool with true in 164.0s build + 1919.2s test
  • testnet/stacks-node/src/nakamoto_node/miner.rs:592:9: replace BlockMinerThread::broadcast -> Result<(), ChainstateError> with Ok(()) in 17.2s build + 159.6s test
  • stackslib/src/net/codec.rs:970:9: replace StacksMessageType::get_message_name -> &'static str with "" in 132.9s build + 1856.1s test
  • stackslib/src/net/unsolicited.rs:292:9: replace PeerNetwork::need_block_or_microblock_stream -> Result<bool, NetError> with Ok(true) in 163.3s build + 1865.9s test
  • stackslib/src/net/codec.rs:970:9: replace StacksMessageType::get_message_name -> &'static str with "xyzzy" in 131.3s build + 1871.0s test
  • stackslib/src/net/unsolicited.rs:292:9: replace PeerNetwork::need_block_or_microblock_stream -> Result<bool, NetError> with Ok(false) in 162.5s build + 1870.7s test
  • stackslib/src/net/unsolicited.rs:666:9: replace PeerNetwork::handle_unsolicited_MicroblocksData -> bool with false
  • stackslib/src/net/codec.rs:1005:9: replace StacksMessageType::get_message_description -> String with String::new() in 180.3s build + 1868.1s test
  • stackslib/src/net/relay.rs:2354:9: replace Relayer::relay_epoch2_blocks with () in 195.2s build + 1861.1s test
  • stackslib/src/net/unsolicited.rs:336:9: replace PeerNetwork::handle_unsolicited_BlocksAvailable -> bool with true in 193.6s build + 1851.5s test
  • stackslib/src/net/unsolicited.rs:715:9: replace PeerNetwork::check_nakamoto_block_signer_signature -> bool with true in 154.2s build + 1861.0s test
  • stackslib/src/net/codec.rs:1005:9: replace StacksMessageType::get_message_description -> String with "xyzzy".into() in 175.2s build + 1855.8s test
  • stackslib/src/net/unsolicited.rs:336:9: replace PeerNetwork::handle_unsolicited_BlocksAvailable -> bool with false in 202.4s build + 1866.2s test
  • stackslib/src/net/relay.rs:3174:9: replace PeerNetwork::update_relayer_stats -> () with () in 210.5s build + 1897.8s test
  • stackslib/src/net/unsolicited.rs:450:9: replace PeerNetwork::handle_unsolicited_MicroblocksAvailable -> bool with true in 205.2s build + 1879.6s test
  • stackslib/src/net/unsolicited.rs:94:9: replace PeerNetwork::handle_unsolicited_inv_update_epoch2x -> Result<Option, NetError> with Ok(None) in 189.6s build + 1872.7s test
  • stackslib/src/net/unsolicited.rs:450:9: replace PeerNetwork::handle_unsolicited_MicroblocksAvailable -> bool with false in 138.1s build + 1928.7s test
  • stackslib/src/net/unsolicited.rs:94:9: replace PeerNetwork::handle_unsolicited_inv_update_epoch2x -> Result<Option, NetError> with Ok(Some(0)) in 177.5s build + 1848.6s test
  • stackslib/src/net/unsolicited.rs:550:9: replace PeerNetwork::handle_unsolicited_BlocksData -> bool with true in 141.1s build + 1888.1s test
  • stackslib/src/net/unsolicited.rs:94:9: replace PeerNetwork::handle_unsolicited_inv_update_epoch2x -> Result<Option, NetError> with Ok(Some(1)) in 196.7s build + 1886.1s test
  • stackslib/src/net/unsolicited.rs:550:9: replace PeerNetwork::handle_unsolicited_BlocksData -> bool with false in 141.9s build + 1879.6s test

@ASuciuX ASuciuX changed the title [Mutations Testing] Skip Timeout mutants related to http request/response [Mutations Testing] for PR: Feat/nakamoto block push #4877 Jun 12, 2024
@ASuciuX ASuciuX linked a pull request Jun 26, 2024 that will close this issue
@jcnelson
Copy link
Member

jcnelson commented Aug 5, 2024

Thanks for gathering these all together. I'll annotate these functions in a follow-on PR. From a cursory read, I think most if not all of these can be skipped since they pertain to best-effort replication protocols which are not guaranteed to succeed in the first place.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: Status: 🆕 New
Development

Successfully merging a pull request may close this issue.

2 participants