This repository was archived by the owner on Nov 6, 2020. It is now read-only.
Fix OOM when restoring a Snapshot #9088
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ngotchac
added
A0-pleasereview 🤓
ordian
approved these changes
Jul 11, 2018
ordian
added
A8-looksgood 🦄
5chdn
pushed a commit
that referenced
this pull request
Jul 11, 2018
5chdn
added a commit
that referenced
this pull request
Jul 17, 2018
* parity-version: stabelize 1.11 * parity-version: bump stable to 1.11.7 * Don't fetch snapshot chunks at random (#9088) * Offload cull to IoWorker. * Limit the number of transactions in pending set (#8777) * Unordered iterator. * Use unordered and limited set if full not required. * Split timeout work into smaller timers. * Avoid collecting all pending transactions when mining * Remove println. * Use priority ordering in eth-filter. * Fix ethcore-miner tests and tx propagation. * Review grumbles addressed. * Add test for unordered not populating the cache. * Fix ethcore tests. * Fix light tests. * Fix ethcore-sync tests. * Fix RPC tests. * Make sure to produce full blocks. * Update hidapi, fixes #7542 (#9108) * docker: add cmake dependency (#9111) * Fix miner tests. * Revert "Make sure to produce full blocks." This reverts commit b12d592. * Update light client hardcoded headers (#9098) * Insert Kovan hardcoded headers until #7690241 * Insert Kovan hardcoded headers until block 7690241 * Insert Ropsten hardcoded headers until #3612673 * Insert Mainnet hardcoded headers until block 5941249 * Make sure to produce full blocks. (#9115) * Insert ETC (classic) hardcoded headers until block #6170625 (#9121) * fix verification in ethcore-sync collect_blocks (#9135) * `evm bench` fix broken dependencies (#9134) * `evm bench` use valid dependencies Benchmarks of the `evm` used stale versions of a couple a crates that this commit fixes! * fix warnings
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
4 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes #8825
The issue was that snapshot chunks were fetched at random, meaning that if an account would be divided into multiple chunks (because it was too big to fit in one), the extracted data would sit in memory until the last chunk of the account is fetched and processed; thus causing increasing memory usage until everything can be flushed in DB, and this would trigger an OOM.
This fix is thus quite simple : fetch the chunks in the right order.