Skip to content
This repository has been archived by the owner on Dec 5, 2024. It is now read-only.

Feature/db upgrade #986

Merged
merged 124 commits into from
Mar 20, 2018
Merged

Feature/db upgrade #986

Changes from 2 commits
Commits
Show all changes
124 commits
Select commit Hold shift + click to select a range
d2ca1a7
Switch persistent db engine to RocksDB
mkalinin Nov 30, 2017
596f275
Stop using BloomedSource; RocksDB supports bloom filters out of the box
mkalinin Nov 30, 2017
27146aa
Implement key compositor layer for the nodes of contract storage tries
mkalinin Dec 8, 2017
641214c
Allow encode method to handle case of nondirty Node instantiated by a…
bendyarm Sep 20, 2017
4e53fba
fix many bugs in RLP
Oct 14, 2017
ae44a65
Increase coverage for .config.net to 100%
Oct 24, 2017
2672c97
Increase coverage for .config
Oct 24, 2017
dad0548
Verify that Py/Go Snappy encoded files can be decoded
Nov 6, 2017
4dd92f8
Fix word misspelled
sasaxie Nov 9, 2017
175c7cf
Update README.md
frazjp65 Nov 14, 2017
a4e1af3
Upgraded to Java 8
zilm13 Nov 30, 2017
c250b37
Fix circleci tests with Hive supporting EthereumJ (#974)
zilm13 Dec 13, 2017
d904c69
Fixed several RLP bugs
zilm13 Dec 14, 2017
39c7032
Removed negative numbers RLP failure, Note added
zilm13 Dec 14, 2017
80a2c8c
RLP class enhanced according to review
zilm13 Dec 15, 2017
c690343
Drop peer if it sends message with framed snappy compression
zilm13 Dec 17, 2017
08989f5
Ignore jenv
mkalinin Dec 20, 2017
e452818
Introduce new pruning aproach and get rid of CountingBytesSource
mkalinin Dec 25, 2017
296c791
Merge branch 'develop' into feature/db-upgrade
mkalinin Dec 25, 2017
cef9fea
Add PruneEntrySource to flush manager
mkalinin Dec 26, 2017
0690458
Fix inline docs in ethereumj.conf
mkalinin Dec 26, 2017
af33451
Get rid of wildcards and unused entities in import
mkalinin Dec 26, 2017
c56f71d
Add newline in .gitignore
mkalinin Dec 26, 2017
cf75595
Fix pruning counting in PruningWindow
mkalinin Dec 26, 2017
bcaba03
Adjust pruning tests to new pruning approach
mkalinin Dec 26, 2017
7f93f63
Apply node key composition to storage and code in FastSync
mkalinin Dec 27, 2017
067f734
Cosmetic fix in NodeKeyCompositor error message
mkalinin Dec 28, 2017
1a47888
Apply node key composition to storage and code in FastSync
mkalinin Dec 28, 2017
e249c31
Merge branch 'feature/db-upgrade' of github.com:ethereum/ethereumj in…
zilm13 Dec 28, 2017
b022715
Apply node key composition to storage and code in FastSync
mkalinin Dec 28, 2017
3207d7e
Added long running test to check DB consistency when application is k…
zilm13 Dec 29, 2017
af88127
Merge branch 'feature/db-upgrade' of github.com:ethereum/ethereumj in…
zilm13 Dec 29, 2017
d3aa307
Added comment how to run test with Gradle
zilm13 Jan 1, 2018
e3ab515
Test stream processing refactored
zilm13 Jan 6, 2018
66edd11
Merge branch 'develop' into feature/db-upgrade
mkalinin Jan 8, 2018
fe9b732
Forward system properties defined for gradle to test task
mkalinin Jan 11, 2018
4454f4d
Add live fast noprune config for tests
mkalinin Jan 11, 2018
a94fc11
Fix states collision in FastSyncSanity test
mkalinin Jan 12, 2018
d18ee2f
Redesign pruning
mkalinin Jan 12, 2018
d414704
FastSyncSanityTest: stop checking sync state if it has already been p…
mkalinin Jan 13, 2018
9ad001e
FastSyncSanityTest: log nodes and headers validation
mkalinin Jan 13, 2018
c966a70
State validation: run storage traversals in separate threads
mkalinin Jan 16, 2018
d4ea3fa
Separate fork management from main chain pruning
mkalinin Jan 17, 2018
4dac181
Break down pruning into certain rules which are triggered by PruneMan…
mkalinin Jan 19, 2018
83ad362
Cosmetic fixes in pruning flow inline docs
mkalinin Jan 19, 2018
022185d
Polish pruning rules and documentation
mkalinin Jan 22, 2018
7b2e3a5
Redesign pruning, revision number 4
mkalinin Jan 27, 2018
78d9f94
Fix trie: old kvNode wasn't deleted when kvNode merge happens
mkalinin Jan 27, 2018
f22e4b5
Get rid of time tracing in PruneManager
mkalinin Jan 27, 2018
6286b63
Polish segment size calculation in PruneManager
mkalinin Jan 27, 2018
9c0744c
Increase max size of prune segment
mkalinin Jan 29, 2018
315cf52
Do not instantiate Pruner if pruning is disabled
mkalinin Jan 29, 2018
77e2fae
Pruner: fix fork duplication by checking whether block already included
mkalinin Jan 30, 2018
5c374f8
Get rid of pruneCacheSize config attribute
mkalinin Jan 30, 2018
0c71d87
Add backup/restore feature to RocksDB db source
mkalinin Jan 30, 2018
cba202d
Add option to backup state fetched by fast sync
mkalinin Jan 30, 2018
559939e
Add storage traversal option
mkalinin Jan 31, 2018
9fbb9f4
Fix NodeDataMessage encoding: remove superfluous RLP boxing
mkalinin Jan 31, 2018
8813d4b
Put error in log when trie node can't be resolved
mkalinin Jan 31, 2018
31a0c0e
Cover Segment with tests
mkalinin Feb 5, 2018
e4cb058
Cosmetic fix in Pruner logging
mkalinin Feb 5, 2018
472756a
Update PruneTest checks to new pruning algorithm
mkalinin Feb 6, 2018
44ce0e5
Fix PruneManager: segment block was also included in upcoming blocks
mkalinin Feb 6, 2018
18213d2
Handle runtime error in VM tracing
mkalinin Feb 7, 2018
ce4c4db
Log tx.hash for Top level call invocations
mkalinin Feb 8, 2018
35146dc
Add a log trace for call to precompiles with supplying data
mkalinin Feb 8, 2018
6b4caf2
Polish log message for precompile call
mkalinin Feb 8, 2018
f679e55
Make HashUtil.sha256 thread safe; it fixes RC in Sha256 precompile
mkalinin Feb 8, 2018
ecb3ce1
Supply fast sync with option to skip bodies and receipts downloading
mkalinin Feb 9, 2018
3dd79c4
Move pending state processing out of event bus
mkalinin Feb 12, 2018
a6bfb01
Fix typo in ethereumj.conf
mkalinin Feb 12, 2018
fec6189
Log pending state processing time
mkalinin Feb 13, 2018
64a94c0
Log tx number in pending state processing
mkalinin Feb 13, 2018
f1930a9
Log pending state time in seconds
mkalinin Feb 13, 2018
0c3ddf9
Revert "Move pending state processing out of event bus"
mkalinin Feb 14, 2018
112debf
Do not clean up fast sync data if blocks loader is enabled
mkalinin Feb 14, 2018
af684ce
Implement CountingQuotientFilter: a QuotientFilter with collisions re…
mkalinin Feb 16, 2018
0aefeb2
Prototyping fuzzy pruning: emulator mode
mkalinin Feb 16, 2018
44fb120
Prototyping fuzzy pruning: improve forks reverting; fix stats
mkalinin Feb 17, 2018
fdd71c1
Switch segment pruning to fuzzy algorithm
mkalinin Feb 20, 2018
cef1852
Add filter load stats to pruning
mkalinin Feb 20, 2018
3a2d6e9
Dispose FastSync downloaders if tasks are done
mkalinin Feb 21, 2018
b1219c2
Queue new blocks only if processing is close to the chain top
mkalinin Feb 21, 2018
e0f5cd9
Fix Segment: manage complex forks cases
mkalinin Feb 22, 2018
1e2dc72
Fix fork pruning: do not revert nodes inserted in the main chain
mkalinin Feb 22, 2018
03e3148
Cache Tx.rawHash instead of Tx.rlpRaw to reduce memory consumption
mkalinin Feb 23, 2018
b2adf4b
Tune ByteArrayEstimator
mkalinin Feb 23, 2018
81548db
Limit BlockBodiesDownloader queue in attempt to reduce memory consump…
mkalinin Feb 23, 2018
7c381d5
Add memory estimators for block abstractions
mkalinin Feb 23, 2018
e40dec1
Reduce FastSyncDownloader queue limit to the number of requested blocks
mkalinin Feb 26, 2018
3bd44ec
Improve memory size limitation of SyncManager queue
mkalinin Feb 26, 2018
15e49fe
Revert "Improve memory size limitation of SyncManager queue"
mkalinin Mar 1, 2018
e05de9f
Revert "Limit BlockBodiesDownloader queue in attempt to reduce memory…
mkalinin Mar 1, 2018
ce9479c
Revert "Queue new blocks only if processing is close to the chain top"
mkalinin Mar 1, 2018
5ec11a8
Add a name of downloader to sync log messages
mkalinin Mar 1, 2018
bfa4cf7
Limit queues in SyncManager and BlockBodiesDownloader by memory cap
mkalinin Mar 2, 2018
7f818cd
Reduce threshold for too distant new blocks in SyncManager
mkalinin Mar 2, 2018
a535504
Track header queue size in BlockBodiesDownloader
mkalinin Mar 2, 2018
b54f7e2
Get rid of LogInfo.rlpEncoded cause this field is unused
mkalinin Mar 2, 2018
32af0e7
Limit ReceiptsDownloader memory consumption
mkalinin Mar 2, 2018
fa37a5d
Fix logger args in BlockDownloader
mkalinin Mar 3, 2018
b9f793d
Fix ReceiptsDownloader mem estimation
mkalinin Mar 3, 2018
af8eae6
Wait for receipts being stored before requesting a new batch
mkalinin Mar 5, 2018
5dd5e7b
Improve ReceiptsDownloader queue management
mkalinin Mar 6, 2018
1b4a114
Improve total headers calculation in SyncManager
mkalinin Mar 6, 2018
55882fa
Log amount of memory consumed by BlockQueue
mkalinin Mar 6, 2018
614324a
Log BlockBodiesDownloader queue size in bytes
mkalinin Mar 7, 2018
10eb1c4
Reduce locked time in ReceiptsDownloader thread
mkalinin Mar 7, 2018
e0ff85f
Fix overflows in QuotientFilter constructor
mkalinin Mar 8, 2018
87db9cc
Distance main chain from forks in state pruning
mkalinin Mar 9, 2018
f3a70b2
Add prune tests checking two step mode
mkalinin Mar 10, 2018
545b204
Change official RockDB build to our custom build with more portable W…
Nashatyrev Mar 11, 2018
cefa88d
Improve FastSync state logging in corner cases to avoid users' being …
mkalinin Mar 12, 2018
daa117d
Improve receipts downloading by implementing adaptive request size
mkalinin Mar 12, 2018
99cbf0a
Merge branch 'feature/db-upgrade' of github.com:/ethereum/ethereumj i…
mkalinin Mar 12, 2018
de8e94c
Add a note about pruning mem footprint to the config
mkalinin Mar 12, 2018
7f9f0d2
Inject Blockchain via SyncPool.init(); fixes probable circular refere…
mkalinin Mar 13, 2018
dfb57d9
Do not exit on block conflict in tests
mkalinin Mar 14, 2018
c81deba
Explicitly set blockchain config in GitHubTransactionTest
mkalinin Mar 14, 2018
06c9798
Merge branch 'develop' into feature/db-upgrade
mkalinin Mar 16, 2018
3575605
Merge branch 'develop' into feature/db-upgrade
mkalinin Mar 16, 2018
e8402bb
Update -Xss4m with -Xss2m for network launches
mkalinin Mar 19, 2018
b994ef1
Turn off checksum verification in db reads
mkalinin Mar 20, 2018
356300c
Set RocksDB block cache size to 32m (default is 8m)
mkalinin Mar 20, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion ethereumj-core/build.gradle
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -117,7 +117,7 @@ dependencies {

compile "org.ethereum:leveldbjni-all:1.18.3" // native leveldb components

compile "org.rocksdb:rocksdbjni:5.8.6" // RocksDB Java API
compile "org.ethereum:rocksdbjni:5.9.2" // RocksDB Java API

compile "org.ethereum:solcJ-all:0.4.8" // Solidity Compiler win/mac/linux binaries

Expand Down