Whom do I thank for 87% RAM usage improvement? #138
Comments
taoeffect
changed the title
|
@domob1812 ? @JeremyRand thinks it might be you:
Maybe you could comment a bit if it was? Is this a "real fix" or just because of the checkpoint? (In other words, can I expect the RAM usage to start climbing back, and if so, how difficult is it to "really fix" it?) |
|
@taoeffect, domob did the name index optimization, I think phelix did the checkpoint. I don't know which of them had a larger effect. I can tell you that with libcoin, the checkpoint cut RAM usage during sync in half (compared against no checkpoints at all). |
|
I think the RAM reduction was mostly due to my patch that removed the auxpow from CBlockIndex objects. Those (the "block headers") are kept in memory for each known block, and the auxpow is not really necessary for the function they have there - but it took up most of the space. So removing it cut down usage quite a lot. The total effect may be related to other improvements as well, though. There've been quite a few in recent times - and I'm working on more (although mostly related to disk) for Huntercoin, which will be ported back to Namecoin as well. |
|
Cool, thanks for explanation @domob1812. I'm still not clear though on whether this is a "real fix" or not. Will ram usage just go back to the massive 300MB+ that it was the farther we get from the most recent checkpoint? Or is the memory management done more intelligently now to prevent that? |
|
It has nothing to do with checkpoints. All known block headers are kept in memory, and of course, this amount will grow over time - although much more slowly now than before. I think that without the auxpow, a block header is only a couple of bytes (80?). So keeping in memory even 1,000,000 blocks in the far future shouldn't be a big problem. This is the same for Bitcoin, BTW. |
|
Oh ok, thanks @domob1812 for the clarification, it's starting to make sense now, and it sounds like this is more of a "real fix" then. |
bccaf86 Merge pull request namecoin#150 2a53a47 Merge pull request namecoin#151 5f5a31f Merge pull request namecoin#149 3907277 Merge pull request namecoin#142 a3e0611 Enable tests in x86 travis builds 45da235 x86 builder 8bb0e93 Merge pull request namecoin#155 971fe81 build: fix openssl detection for cross builds f22d73e Explicitly access %0..%2 as 64-bit so we use the right registers for x32 ABI e66d4d6 Avoid the stack in assembly and use explicit registers cf7b2b4 Fix ECDSA message hashes to 32 bytes 056ad31 Really compile with -O3 by default 74ad63a Merge pull request namecoin#146 9000458 Merge pull request namecoin#145 1f46b00 build: fix __builtin_expect detection for clang aaba2e0 Merge pull request namecoin#136 8a0775c Merge pull request namecoin#144 ee1eaa7 Merge pull request namecoin#141 c88e2b8 Compile with -O3 by default 6558a26 Make the benchmarks print out stats 000bdf6 Rename bench_verify to bench_recovery 7c6fed2 Add a few more additional tests. 992e03b travis: add clang to the test matrix b43b79a Merge pull request namecoin#143 e06a924 Include time.h header for time(). 8d11164 Add some additional tests. 3545627 Merge pull request namecoin#118 6a9901e Merge pull request namecoin#137 376b28b Merge pull request namecoin#128 1728806 Merge pull request namecoin#138 a5759c5 Check return value of malloc 39bd94d Variable time normalize ad86bdf Merge pull request namecoin#140 54b768c Another redundant secp256k1_fe_normalize 69dcaab Merge pull request #139 1c29f2e Remove redundant secp256k1_fe_normalize from secp256k1_gej_add_ge_var. 2b9388b Remove unused secp256k1_fe_inv_all f461b76 Allocate precomputation arrays on the heap b2c9681 Make {mul,sqr}_inner use the same argument order as {mul,sqr} 6793505 Convert YASM code into inline assembly f048615 Rewrite field assembly to match the C version 3ce74b1 Tweak precomputed table size for G git-subtree-dir: src/secp256k1 git-subtree-split: bccaf86caa9c44166e5a66600b742c516e03c3f0
WRT
namecoind, this appears to have happened in the past few weeks (or months) in the namecoinq branch. Which commit did it? It went from >300MB to 90MB. Thank you to whomever did it!The text was updated successfully, but these errors were encountered: