You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
bc01e96 fmt: re-order include accorindgly to clang formatter (Konstantin Akimov)
8a68ec6 fix: remove duplicated check for hash existance in map, avoid hash recalculation (Konstantin Akimov)
b75e640 refactor: add gsl::not_null annotation for cached_sml and CalcCbTxMerkleRootMNList (Konstantin Akimov)
d0a2791 test: add comprehensive unit tests for SML caching mechanism (pasta)
d61f44e refactor: centralize SML cache invalidation logic (pasta)
53190a5 refactor: enhance thread safety for SML caching with mutex protection (pasta)
83c667e refactor: update comment for SML cache population (pasta)
63031da refactor: drop dependency SML on DMN (Konstantin Akimov)
7bed080 perf: cache shared_ptr to SML instead of SML list (Konstantin Akimov)
3b4a1a0 feat: new GetSML method for CDeterministicMNList (Konstantin Akimov)
bbd8731 fmt: apply clang-format suggestions (Konstantin Akimov)
c2e053f refactor: move diff related code from evo/simplifiedmns to new module evo/smldiff (Konstantin Akimov)
e8949c8 refactor: move CSimplifiedMNListDiffs::ToJson to evo/core_write (Konstantin Akimov)
Pull request description:
## Issue being fixed or feature implemented
During block validation, each block requires SML (Simplified MN List) to be constructed from scratch: convert ~3000 DeterministicMN + DMNState objects to CSimplifiedMNListEntry; sort them by protx and compare with previously calculated SML.
In case if SML is changed, Dash Core re-calculates Merkle Root hash. On practice, changes in SML list happens only once in tens (or even hundreds) of blocks.
But it is a heavy process which takes up to 20% of CPU-time during block validation.
## What was done?
Instead of re-calculation SML list for each block, let's try to keep this calculation cached and reset this cache every time when list of masternodes is actually changed.
The main idea of this cache is to add a `shared_ptr<CSimplifiedMNList>` to each object `CDeterministicMNList`. Its value is set when `CDeterministicMNManager::ProcessBlock` is called and can be copied every time, when new list is re-created.
Because `ProcessBlock` is called consequently if and only if a new block is connected we don't waste creation of SML when we don't need it.
This cache to SML is invalidated every time, when `AddMN` or `RemoveMN` is called. Calls `UpdateMN` do not reset this cache if produced CSimplifiedMNListEntry is not changed (see implementation for details).
Though, indirect calls of AddMN/RemoveMN/UpdateMN (such as calls happened inside ApplyDiff) also invalidates cache.
## Side notes about implementation
This PR inverts dependency of `evo/simplifiedmns -> evo/deterministicmns` to `evo/deterministicmns -> evo/simplifiedmns`.
This change caused explosion in amount of new circular dependencies of `llmq/* <-> evo/determnisticmns`; to prevent it a new module `evo/smldiff` has been introduced.
Also `CSimplifiedMNListDiffs::ToJson` and `CSimplifiedMNList::ToJson` has been moved to `evo/core_write`.
## How Has This Been Tested?
Tested by invalidation + reconsider 7000 blocks (~2 weeks of blocks).
This PR speeds up blocks validation for 15% compare to current develop.
Drastically improved performance `CalcCbTxMerkleRootMNList` (which is part of `CheckCbTxMerkleRoots`) and `CSimplifiedMNList::CSimplifiedMNList` is replaced by `CDeterministicMNList::to_sml` which is almost instant (see perf screenshots).
PR times:
```
[bench] - m_dmnman: 2.99ms [5.65s]
[bench] - CalcCbTxMerkleRootMNList: 0.32ms [0.40s]
[bench] - CachedGetQcHashesQcIndexedHashes: 0.69ms [6.52s]
[bench] - Loop: 0.02ms [0.82s]
[bench] - ComputeMerkleRoot: 0.05ms [0.32s]
[bench] - CalcCbTxMerkleRootQuorums: 0.78ms [7.68s]
[bench] - CheckCbTxMerkleRoots: 1.11ms [8.10s]
[bench] - ProcessSpecialTxsInBlock: 7.49ms [63.82s (8.97ms/blk)]
[bench] - Connect 59 transactions: 7.74ms (0.131ms/tx, 0.067ms/txin) [65.66s (9.23ms/blk)]
[bench] - Verify 116 txins: 7.75ms (0.067ms/txin) [67.62s (9.51ms/blk)]
[bench] - Dash specific: 0.14ms [0.94s (0.13ms/blk)]
[bench] - Write undo data: 0.00ms [0.01s (0.00ms/blk)]
[bench] - Index writing: 0.01ms [0.01s (0.00ms/blk)]
[bench] - Connect total: 8.34ms [69.41s (9.76ms/blk)]
[bench] - Connect block: 9.43ms [71.67s (10.08ms/blk)]
```
<img width="644" alt="image" src="https://github.com/user-attachments/assets/547591cb-9207-402c-a24f-bbf2237275ee" />
develop times:
```
[bench] - m_dmnman: 1.43ms [6.34s]
[bench] - CalcCbTxMerkleRootMNList: 0.36ms [2.40s]
[bench] - CachedGetQcHashesQcIndexedHashes: 0.70ms [7.13s]
[bench] - Loop: 0.02ms [0.82s]
[bench] - ComputeMerkleRoot: 0.05ms [0.33s]
[bench] - CalcCbTxMerkleRootQuorums: 0.79ms [8.32s]
[bench] - CheckCbTxMerkleRoots: 3.16ms [19.32s]
[bench] - ProcessSpecialTxsInBlock: 8.20ms [75.66s (10.64ms/blk)]
[bench] - Connect 59 transactions: 8.54ms (0.145ms/tx, 0.074ms/txin) [77.68s (10.92ms/blk)]
[bench] - Verify 116 txins: 8.55ms (0.074ms/txin) [79.71s (11.21ms/blk)]
[bench] - Dash specific: 0.30ms [1.34s (0.19ms/blk)]
[bench] - Write undo data: 0.00ms [0.01s (0.00ms/blk)]
[bench] - Index writing: 0.01ms [0.01s (0.00ms/blk)]
[bench] - Connect total: 9.10ms [82.17s (11.55ms/blk)]
[bench] - Connect block: 10.07ms [84.77s (11.92ms/blk)]
```
<img width="644" alt="image" src="https://github.com/user-attachments/assets/e3552ab3-04ba-457d-a14d-f4f46b55df6d" />
## Breaking Changes
N/A
## Checklist:
- [x] I have performed a self-review of my own code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have added or updated relevant unit/integration/functional/e2e tests
- [ ] I have made corresponding changes to the documentation
- [x] I have assigned this pull request to a milestone
ACKs for top commit:
UdjinM6:
light ACK bc01e96
Tree-SHA512: e1b5e08a4c4ede7f7ec886dacb684f39d962a486c3e48afd48fb67da930798cb14b3a55b27d10e0ade6113d352d103b5cbba37cabb2e49f60f1d40593827425b
0 commit comments