feature(collator): cumulative stats

[drmick]

No description provided.

[codecov]

Codecov Report

Attention: Patch coverage is 76.65929% with 211 lines in your changes missing coverage. Please review.

Project coverage is 47.90%. Comparing base (4118d3b) to head (cd9e4c5).
Report is 16 commits behind head on feat/imp-sync-long-tail-version.

Files with missing lines	Patch %	Lines
collator/src/types/processed_upto.rs	43.85%	32 Missing ⚠️
...r/src/collator/messages_reader/internals_reader.rs	81.69%	18 Missing and 8 partials ⚠️
collator/src/collator/do_collate/mod.rs	45.00%	22 Missing ⚠️
collator/src/collator/mod.rs	0.00%	21 Missing ⚠️
block-util/src/queue/proto.rs	34.61%	17 Missing ⚠️
collator/src/internal_queue/types.rs	39.28%	17 Missing ⚠️
collator/src/collator/do_collate/prepare.rs	0.00%	15 Missing ⚠️
collator/src/collator/do_collate/finalize.rs	0.00%	14 Missing ⚠️
collator/src/types.rs	0.00%	10 Missing ⚠️
collator/src/queue_adapter.rs	60.00%	8 Missing ⚠️
... and 11 more

Additional details and impacted files

@@                         Coverage Diff                         @@
##           feat/imp-sync-long-tail-version     #636      +/-   ##
===================================================================
+ Coverage                            46.92%   47.90%   +0.97%     
===================================================================
  Files                                  282      282              
  Lines                                53857    54184     +327     
  Branches                             53857    54184     +327     
===================================================================
+ Hits                                 25273    25956     +683     
+ Misses                               27276    26904     -372     
- Partials                              1308     1324      +16     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[Copilot]

Pull Request Overview

This PR implements cumulative statistics for the collator component, refactoring statistics-related methods from load_statistics to load_diff_statistics and introducing a cumulative aggregation mechanism. Key changes include adding support for per-partition and overall statistics aggregation, removing the redundant min_message field from StatKey serialization/deserialization, and updating tests and adapter methods to use the new APIs.

Reviewed Changes

Copilot reviewed 22 out of 22 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
storage/src/store/internal_queue/snapshot.rs	Updates iterator bounds and API signatures for diff-based stats collection.
storage/src/store/internal_queue/model.rs	Removes the unused min_message field from StatKey.
collator/*	Updates tests, adapter, and state logic to use load_diff_statistics and incorporate cumulative statistics.
block-util/src/queue/proto.rs	Adjusts QueueKey update logic when new_lt reaches its maximum.
collator/src/collator/do_collate/mod.rs	Adds new cumulative statistics parameters and introduces a panic on unsupported PrevBlockRef variant.

Comments suppressed due to low confidence (1)

storage/src/store/internal_queue/snapshot.rs:175

• The change to use only the prefix slice ([..StatKey::PREFIX_SIZE]) for setting the lower bound may narrow the iterator range more than intended. Please verify that this behavior correctly captures all required keys compared to using the full byte sequence.

read_config.set_iterate_lower_bound(&from_bytes[..StatKey::PREFIX_SIZE]);