Skip to content

Commit 7149b9d

Browse files
Merge #6738: perf: use cache for SML
bc01e96 fmt: re-order include accorindgly to clang formatter (Konstantin Akimov) 8a68ec6 fix: remove duplicated check for hash existance in map, avoid hash recalculation (Konstantin Akimov) b75e640 refactor: add gsl::not_null annotation for cached_sml and CalcCbTxMerkleRootMNList (Konstantin Akimov) d0a2791 test: add comprehensive unit tests for SML caching mechanism (pasta) d61f44e refactor: centralize SML cache invalidation logic (pasta) 53190a5 refactor: enhance thread safety for SML caching with mutex protection (pasta) 83c667e refactor: update comment for SML cache population (pasta) 63031da refactor: drop dependency SML on DMN (Konstantin Akimov) 7bed080 perf: cache shared_ptr to SML instead of SML list (Konstantin Akimov) 3b4a1a0 feat: new GetSML method for CDeterministicMNList (Konstantin Akimov) bbd8731 fmt: apply clang-format suggestions (Konstantin Akimov) c2e053f refactor: move diff related code from evo/simplifiedmns to new module evo/smldiff (Konstantin Akimov) e8949c8 refactor: move CSimplifiedMNListDiffs::ToJson to evo/core_write (Konstantin Akimov) Pull request description: ## Issue being fixed or feature implemented During block validation, each block requires SML (Simplified MN List) to be constructed from scratch: convert ~3000 DeterministicMN + DMNState objects to CSimplifiedMNListEntry; sort them by protx and compare with previously calculated SML. In case if SML is changed, Dash Core re-calculates Merkle Root hash. On practice, changes in SML list happens only once in tens (or even hundreds) of blocks. But it is a heavy process which takes up to 20% of CPU-time during block validation. ## What was done? Instead of re-calculation SML list for each block, let's try to keep this calculation cached and reset this cache every time when list of masternodes is actually changed. The main idea of this cache is to add a `shared_ptr<CSimplifiedMNList>` to each object `CDeterministicMNList`. Its value is set when `CDeterministicMNManager::ProcessBlock` is called and can be copied every time, when new list is re-created. Because `ProcessBlock` is called consequently if and only if a new block is connected we don't waste creation of SML when we don't need it. This cache to SML is invalidated every time, when `AddMN` or `RemoveMN` is called. Calls `UpdateMN` do not reset this cache if produced CSimplifiedMNListEntry is not changed (see implementation for details). Though, indirect calls of AddMN/RemoveMN/UpdateMN (such as calls happened inside ApplyDiff) also invalidates cache. ## Side notes about implementation This PR inverts dependency of `evo/simplifiedmns -> evo/deterministicmns` to `evo/deterministicmns -> evo/simplifiedmns`. This change caused explosion in amount of new circular dependencies of `llmq/* <-> evo/determnisticmns`; to prevent it a new module `evo/smldiff` has been introduced. Also `CSimplifiedMNListDiffs::ToJson` and `CSimplifiedMNList::ToJson` has been moved to `evo/core_write`. ## How Has This Been Tested? Tested by invalidation + reconsider 7000 blocks (~2 weeks of blocks). This PR speeds up blocks validation for 15% compare to current develop. Drastically improved performance `CalcCbTxMerkleRootMNList` (which is part of `CheckCbTxMerkleRoots`) and `CSimplifiedMNList::CSimplifiedMNList` is replaced by `CDeterministicMNList::to_sml` which is almost instant (see perf screenshots). PR times: ``` [bench] - m_dmnman: 2.99ms [5.65s] [bench] - CalcCbTxMerkleRootMNList: 0.32ms [0.40s] [bench] - CachedGetQcHashesQcIndexedHashes: 0.69ms [6.52s] [bench] - Loop: 0.02ms [0.82s] [bench] - ComputeMerkleRoot: 0.05ms [0.32s] [bench] - CalcCbTxMerkleRootQuorums: 0.78ms [7.68s] [bench] - CheckCbTxMerkleRoots: 1.11ms [8.10s] [bench] - ProcessSpecialTxsInBlock: 7.49ms [63.82s (8.97ms/blk)] [bench] - Connect 59 transactions: 7.74ms (0.131ms/tx, 0.067ms/txin) [65.66s (9.23ms/blk)] [bench] - Verify 116 txins: 7.75ms (0.067ms/txin) [67.62s (9.51ms/blk)] [bench] - Dash specific: 0.14ms [0.94s (0.13ms/blk)] [bench] - Write undo data: 0.00ms [0.01s (0.00ms/blk)] [bench] - Index writing: 0.01ms [0.01s (0.00ms/blk)] [bench] - Connect total: 8.34ms [69.41s (9.76ms/blk)] [bench] - Connect block: 9.43ms [71.67s (10.08ms/blk)] ``` <img width="644" alt="image" src="https://github.com/user-attachments/assets/547591cb-9207-402c-a24f-bbf2237275ee" /> develop times: ``` [bench] - m_dmnman: 1.43ms [6.34s] [bench] - CalcCbTxMerkleRootMNList: 0.36ms [2.40s] [bench] - CachedGetQcHashesQcIndexedHashes: 0.70ms [7.13s] [bench] - Loop: 0.02ms [0.82s] [bench] - ComputeMerkleRoot: 0.05ms [0.33s] [bench] - CalcCbTxMerkleRootQuorums: 0.79ms [8.32s] [bench] - CheckCbTxMerkleRoots: 3.16ms [19.32s] [bench] - ProcessSpecialTxsInBlock: 8.20ms [75.66s (10.64ms/blk)] [bench] - Connect 59 transactions: 8.54ms (0.145ms/tx, 0.074ms/txin) [77.68s (10.92ms/blk)] [bench] - Verify 116 txins: 8.55ms (0.074ms/txin) [79.71s (11.21ms/blk)] [bench] - Dash specific: 0.30ms [1.34s (0.19ms/blk)] [bench] - Write undo data: 0.00ms [0.01s (0.00ms/blk)] [bench] - Index writing: 0.01ms [0.01s (0.00ms/blk)] [bench] - Connect total: 9.10ms [82.17s (11.55ms/blk)] [bench] - Connect block: 10.07ms [84.77s (11.92ms/blk)] ``` <img width="644" alt="image" src="https://github.com/user-attachments/assets/e3552ab3-04ba-457d-a14d-f4f46b55df6d" /> ## Breaking Changes N/A ## Checklist: - [x] I have performed a self-review of my own code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have added or updated relevant unit/integration/functional/e2e tests - [ ] I have made corresponding changes to the documentation - [x] I have assigned this pull request to a milestone ACKs for top commit: UdjinM6: light ACK bc01e96 Tree-SHA512: e1b5e08a4c4ede7f7ec886dacb684f39d962a486c3e48afd48fb67da930798cb14b3a55b27d10e0ade6113d352d103b5cbba37cabb2e49f60f1d40593827425b
2 parents 996cc4c + bc01e96 commit 7149b9d

19 files changed

+683
-450
lines changed

src/Makefile.am

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -195,6 +195,7 @@ BITCOIN_CORE_H = \
195195
evo/netinfo.h \
196196
evo/providertx.h \
197197
evo/simplifiedmns.h \
198+
evo/smldiff.h \
198199
evo/specialtx.h \
199200
evo/specialtxman.h \
200201
dsnotificationinterface.h \
@@ -472,6 +473,7 @@ libbitcoin_node_a_SOURCES = \
472473
evo/mnhftx.cpp \
473474
evo/providertx.cpp \
474475
evo/simplifiedmns.cpp \
476+
evo/smldiff.cpp \
475477
evo/specialtx.cpp \
476478
evo/specialtxman.cpp \
477479
flatfile.cpp \

src/evo/cbtx.h

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,9 @@
1414
class BlockValidationState;
1515
class CBlock;
1616
class CBlockIndex;
17+
class CDeterministicMNList;
1718
class TxValidationState;
19+
1820
namespace llmq {
1921
class CQuorumBlockProcessor;
2022
}// namespace llmq

src/evo/core_write.cpp

Lines changed: 96 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,8 @@
88
#include <evo/mnhftx.h>
99
#include <evo/netinfo.h>
1010
#include <evo/providertx.h>
11+
#include <evo/simplifiedmns.h>
12+
#include <evo/smldiff.h>
1113
#include <llmq/commitment.h>
1214

1315
#include <univalue.h>
@@ -149,3 +151,97 @@
149151
ret.pushKV("commitment", commitment.ToJson());
150152
return ret;
151153
}
154+
155+
[[nodiscard]] UniValue CSimplifiedMNListEntry::ToJson(bool extended) const
156+
{
157+
UniValue obj(UniValue::VOBJ);
158+
obj.pushKV("nVersion", nVersion);
159+
obj.pushKV("nType", ToUnderlying(nType));
160+
obj.pushKV("proRegTxHash", proRegTxHash.ToString());
161+
obj.pushKV("confirmedHash", confirmedHash.ToString());
162+
if (IsServiceDeprecatedRPCEnabled()) {
163+
obj.pushKV("service", netInfo->GetPrimary().ToStringAddrPort());
164+
}
165+
obj.pushKV("addresses", netInfo->ToJson());
166+
obj.pushKV("pubKeyOperator", pubKeyOperator.ToString());
167+
obj.pushKV("votingAddress", EncodeDestination(PKHash(keyIDVoting)));
168+
obj.pushKV("isValid", isValid);
169+
if (nType == MnType::Evo) {
170+
obj.pushKV("platformHTTPPort", platformHTTPPort);
171+
obj.pushKV("platformNodeID", platformNodeID.ToString());
172+
}
173+
174+
if (extended) {
175+
CTxDestination dest;
176+
if (ExtractDestination(scriptPayout, dest)) {
177+
obj.pushKV("payoutAddress", EncodeDestination(dest));
178+
}
179+
if (ExtractDestination(scriptOperatorPayout, dest)) {
180+
obj.pushKV("operatorPayoutAddress", EncodeDestination(dest));
181+
}
182+
}
183+
return obj;
184+
}
185+
186+
[[nodiscard]] UniValue CSimplifiedMNListDiff::ToJson(bool extended) const
187+
{
188+
UniValue obj(UniValue::VOBJ);
189+
190+
obj.pushKV("nVersion", nVersion);
191+
obj.pushKV("baseBlockHash", baseBlockHash.ToString());
192+
obj.pushKV("blockHash", blockHash.ToString());
193+
194+
CDataStream ssCbTxMerkleTree(SER_NETWORK, PROTOCOL_VERSION);
195+
ssCbTxMerkleTree << cbTxMerkleTree;
196+
obj.pushKV("cbTxMerkleTree", HexStr(ssCbTxMerkleTree));
197+
198+
obj.pushKV("cbTx", EncodeHexTx(*cbTx));
199+
200+
UniValue deletedMNsArr(UniValue::VARR);
201+
for (const auto& h : deletedMNs) {
202+
deletedMNsArr.push_back(h.ToString());
203+
}
204+
obj.pushKV("deletedMNs", deletedMNsArr);
205+
206+
UniValue mnListArr(UniValue::VARR);
207+
for (const auto& e : mnList) {
208+
mnListArr.push_back(e.ToJson(extended));
209+
}
210+
obj.pushKV("mnList", mnListArr);
211+
212+
UniValue deletedQuorumsArr(UniValue::VARR);
213+
for (const auto& e : deletedQuorums) {
214+
UniValue eObj(UniValue::VOBJ);
215+
eObj.pushKV("llmqType", e.first);
216+
eObj.pushKV("quorumHash", e.second.ToString());
217+
deletedQuorumsArr.push_back(eObj);
218+
}
219+
obj.pushKV("deletedQuorums", deletedQuorumsArr);
220+
221+
UniValue newQuorumsArr(UniValue::VARR);
222+
for (const auto& e : newQuorums) {
223+
newQuorumsArr.push_back(e.ToJson());
224+
}
225+
obj.pushKV("newQuorums", newQuorumsArr);
226+
227+
// Do not assert special tx type here since this can be called prior to DIP0003 activation
228+
if (const auto opt_cbTxPayload = GetTxPayload<CCbTx>(*cbTx, /*assert_type=*/false)) {
229+
obj.pushKV("merkleRootMNList", opt_cbTxPayload->merkleRootMNList.ToString());
230+
if (opt_cbTxPayload->nVersion >= CCbTx::Version::MERKLE_ROOT_QUORUMS) {
231+
obj.pushKV("merkleRootQuorums", opt_cbTxPayload->merkleRootQuorums.ToString());
232+
}
233+
}
234+
235+
UniValue quorumsCLSigsArr(UniValue::VARR);
236+
for (const auto& [signature, quorumsIndexes] : quorumsCLSigs) {
237+
UniValue j(UniValue::VOBJ);
238+
UniValue idxArr(UniValue::VARR);
239+
for (const auto& idx : quorumsIndexes) {
240+
idxArr.push_back(idx);
241+
}
242+
j.pushKV(signature.ToString(), idxArr);
243+
quorumsCLSigsArr.push_back(j);
244+
}
245+
obj.pushKV("quorumsCLSigs", quorumsCLSigsArr);
246+
return obj;
247+
}

src/evo/deterministicmns.cpp

Lines changed: 37 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,7 @@
77
#include <evo/dmnstate.h>
88
#include <evo/evodb.h>
99
#include <evo/providertx.h>
10+
#include <evo/simplifiedmns.h>
1011
#include <evo/specialtx.h>
1112
#include <index/txindex.h>
1213

@@ -36,6 +37,14 @@ uint64_t CDeterministicMN::GetInternalId() const
3637
return internalId;
3738
}
3839

40+
CSimplifiedMNListEntry CDeterministicMN::to_sml_entry() const
41+
{
42+
const CDeterministicMNState& state{*pdmnState};
43+
return CSimplifiedMNListEntry(proTxHash, state.confirmedHash, state.netInfo, state.pubKeyOperator,
44+
state.keyIDVoting, !state.IsBanned(), state.platformHTTPPort, state.platformNodeID,
45+
state.scriptPayout, state.scriptOperatorPayout, state.nVersion, nType);
46+
}
47+
3948
std::string CDeterministicMN::ToString() const
4049
{
4150
return strprintf("CDeterministicMN(proTxHash=%s, collateralOutpoint=%s, nOperatorReward=%f, state=%s", proTxHash.ToString(), collateralOutpoint.ToStringShort(), (double)nOperatorReward / 100, pdmnState->ToString());
@@ -258,6 +267,22 @@ std::vector<CDeterministicMNCPtr> CDeterministicMNList::GetProjectedMNPayees(gsl
258267
return result;
259268
}
260269

270+
gsl::not_null<std::shared_ptr<const CSimplifiedMNList>> CDeterministicMNList::to_sml() const
271+
{
272+
LOCK(m_cached_sml_mutex);
273+
if (!m_cached_sml) {
274+
std::vector<std::unique_ptr<CSimplifiedMNListEntry>> sml_entries;
275+
sml_entries.reserve(mnMap.size());
276+
277+
ForEachMN(false, [&sml_entries](auto& dmn) {
278+
sml_entries.emplace_back(std::make_unique<CSimplifiedMNListEntry>(dmn.to_sml_entry()));
279+
});
280+
m_cached_sml = std::make_shared<CSimplifiedMNList>(std::move(sml_entries));
281+
}
282+
283+
return m_cached_sml;
284+
}
285+
261286
int CDeterministicMNList::CalcMaxPoSePenalty() const
262287
{
263288
// Maximum PoSe penalty is dynamic and equals the number of registered MNs
@@ -443,6 +468,7 @@ void CDeterministicMNList::AddMN(const CDeterministicMNCPtr& dmn, bool fBumpTota
443468

444469
mnMap = mnMap.set(dmn->proTxHash, dmn);
445470
mnInternalIdMap = mnInternalIdMap.set(dmn->GetInternalId(), dmn->proTxHash);
471+
InvalidateSMLCache();
446472
if (fBumpTotalCount) {
447473
// nTotalRegisteredCount acts more like a checkpoint, not as a limit,
448474
nTotalRegisteredCount = std::max(dmn->GetInternalId() + 1, (uint64_t)nTotalRegisteredCount);
@@ -514,6 +540,10 @@ void CDeterministicMNList::UpdateMN(const CDeterministicMN& oldDmn, const std::s
514540

515541
dmn->pdmnState = pdmnState;
516542
mnMap = mnMap.set(oldDmn.proTxHash, dmn);
543+
LOCK(m_cached_sml_mutex);
544+
if (m_cached_sml && oldDmn.to_sml_entry() != dmn->to_sml_entry()) {
545+
m_cached_sml = nullptr;
546+
}
517547
}
518548

519549
void CDeterministicMNList::UpdateMN(const uint256& proTxHash, const std::shared_ptr<const CDeterministicMNState>& pdmnState)
@@ -585,6 +615,7 @@ void CDeterministicMNList::RemoveMN(const uint256& proTxHash)
585615

586616
mnMap = mnMap.erase(proTxHash);
587617
mnInternalIdMap = mnInternalIdMap.erase(dmn->GetInternalId());
618+
InvalidateSMLCache();
588619
}
589620

590621
bool CDeterministicMNManager::ProcessBlock(const CBlock& block, gsl::not_null<const CBlockIndex*> pindex,
@@ -604,6 +635,8 @@ bool CDeterministicMNManager::ProcessBlock(const CBlock& block, gsl::not_null<co
604635
int nHeight = pindex->nHeight;
605636

606637
try {
638+
newList.to_sml(); // to populate the SML cache
639+
607640
LOCK(cs);
608641

609642
oldList = GetListForBlockInternal(pindex->pprev);
@@ -619,6 +652,7 @@ bool CDeterministicMNManager::ProcessBlock(const CBlock& block, gsl::not_null<co
619652

620653
diff.nHeight = pindex->nHeight;
621654
mnListDiffsCache.emplace(pindex->GetBlockHash(), diff);
655+
mnListsCache.emplace(newList.GetBlockHash(), newList);
622656
} catch (const std::exception& e) {
623657
LogPrintf("CDeterministicMNManager::%s -- internal error: %s\n", __func__, e.what());
624658
return state.Invalid(BlockValidationResult::BLOCK_CONSENSUS, "failed-dmn-block");
@@ -751,8 +785,8 @@ CDeterministicMNList CDeterministicMNManager::GetListForBlockInternal(gsl::not_n
751785

752786
if (tipIndex) {
753787
// always keep a snapshot for the tip
754-
if (snapshot.GetBlockHash() == tipIndex->GetBlockHash()) {
755-
mnListsCache.emplace(snapshot.GetBlockHash(), snapshot);
788+
if (const auto snapshot_hash = snapshot.GetBlockHash(); snapshot_hash == tipIndex->GetBlockHash()) {
789+
mnListsCache.emplace(snapshot_hash, snapshot);
756790
} else {
757791
// keep snapshots for yet alive quorums
758792
if (ranges::any_of(Params().GetConsensus().llmqs,
@@ -762,7 +796,7 @@ CDeterministicMNList CDeterministicMNManager::GetListForBlockInternal(gsl::not_n
762796
(snapshot.GetHeight() + params.dkgInterval * (params.keepOldConnections + 1) >=
763797
tipIndex->nHeight);
764798
})) {
765-
mnListsCache.emplace(snapshot.GetBlockHash(), snapshot);
799+
mnListsCache.emplace(snapshot_hash, snapshot);
766800
}
767801
}
768802
}

src/evo/deterministicmns.h

Lines changed: 56 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,8 @@ class CBlock;
3030
class CBlockIndex;
3131
class CCoinsViewCache;
3232
class CEvoDB;
33+
class CSimplifiedMNList;
34+
class CSimplifiedMNListEntry;
3335
class TxValidationState;
3436

3537
extern RecursiveMutex cs_main;
@@ -79,6 +81,7 @@ class CDeterministicMN
7981

8082
[[nodiscard]] uint64_t GetInternalId() const;
8183

84+
[[nodiscard]] CSimplifiedMNListEntry to_sml_entry() const;
8285
[[nodiscard]] std::string ToString() const;
8386
[[nodiscard]] UniValue ToJson() const;
8487
};
@@ -145,6 +148,22 @@ class CDeterministicMNList
145148
// we keep track of this as checking for duplicates would otherwise be painfully slow
146149
MnUniquePropertyMap mnUniquePropertyMap;
147150

151+
// This SML could be null
152+
// This cache is used to improve performance and meant to be reused
153+
// for multiple CDeterministicMNList until mnMap is actually changed.
154+
// Calls of AddMN, RemoveMN and (in some cases) UpdateMN reset this cache;
155+
// it happens also for indirect calls such as ApplyDiff
156+
// Thread safety: Protected by its own mutex for thread-safe access
157+
mutable Mutex m_cached_sml_mutex;
158+
mutable std::shared_ptr<const CSimplifiedMNList> m_cached_sml GUARDED_BY(m_cached_sml_mutex);
159+
160+
// Private helper method to invalidate SML cache
161+
void InvalidateSMLCache()
162+
{
163+
LOCK(m_cached_sml_mutex);
164+
m_cached_sml = nullptr;
165+
}
166+
148167
public:
149168
CDeterministicMNList() = default;
150169
explicit CDeterministicMNList(const uint256& _blockHash, int _height, uint32_t _totalRegisteredCount) :
@@ -155,6 +174,36 @@ class CDeterministicMNList
155174
assert(nHeight >= 0);
156175
}
157176

177+
// Copy constructor
178+
CDeterministicMNList(const CDeterministicMNList& other) :
179+
blockHash(other.blockHash),
180+
nHeight(other.nHeight),
181+
nTotalRegisteredCount(other.nTotalRegisteredCount),
182+
mnMap(other.mnMap),
183+
mnInternalIdMap(other.mnInternalIdMap),
184+
mnUniquePropertyMap(other.mnUniquePropertyMap)
185+
{
186+
LOCK(other.m_cached_sml_mutex);
187+
m_cached_sml = other.m_cached_sml;
188+
}
189+
190+
// Assignment operator
191+
CDeterministicMNList& operator=(const CDeterministicMNList& other)
192+
{
193+
if (this != &other) {
194+
blockHash = other.blockHash;
195+
nHeight = other.nHeight;
196+
nTotalRegisteredCount = other.nTotalRegisteredCount;
197+
mnMap = other.mnMap;
198+
mnInternalIdMap = other.mnInternalIdMap;
199+
mnUniquePropertyMap = other.mnUniquePropertyMap;
200+
201+
LOCK2(m_cached_sml_mutex, other.m_cached_sml_mutex);
202+
m_cached_sml = other.m_cached_sml;
203+
}
204+
return *this;
205+
}
206+
158207
template <typename Stream, typename Operation>
159208
inline void SerializationOpBase(Stream& s, Operation ser_action)
160209
{
@@ -195,6 +244,7 @@ class CDeterministicMNList
195244
mnMap = MnMap();
196245
mnUniquePropertyMap = MnUniquePropertyMap();
197246
mnInternalIdMap = MnInternalIdMap();
247+
InvalidateSMLCache();
198248
}
199249

200250
[[nodiscard]] size_t GetAllMNsCount() const
@@ -314,6 +364,12 @@ class CDeterministicMNList
314364
*/
315365
[[nodiscard]] std::vector<CDeterministicMNCPtr> GetProjectedMNPayees(gsl::not_null<const CBlockIndex* const> pindexPrev, int nCount = std::numeric_limits<int>::max()) const;
316366

367+
/**
368+
* Calculates CSimplifiedMNList for current list and cache it
369+
* Thread safety: Uses internal mutex for thread-safe cache access
370+
*/
371+
gsl::not_null<std::shared_ptr<const CSimplifiedMNList>> to_sml() const;
372+
317373
/**
318374
* Calculates the maximum penalty which is allowed at the height of this MN list. It is dynamic and might change
319375
* for every block.

0 commit comments

Comments
 (0)