Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Import export trie log #6363

Merged
merged 51 commits into from
Jan 22, 2024
Merged
Changes from 1 commit
Commits
Show all changes
51 commits
Select commit Hold shift + click to select a range
16c0a49
Add x-trie-log subcommand for one-off backlog prune
siladu Nov 20, 2023
7dd4928
long -> int
siladu Nov 20, 2023
bf2b098
Removed banned method
gfukushima Dec 12, 2023
e67ae51
Preload process stream in parallel
gfukushima Dec 12, 2023
9b4e0c9
Drop unwanted trielogs and keep reatain layers only
gfukushima Dec 14, 2023
0b9fe83
Add output to user and cleanup refactor
gfukushima Dec 15, 2023
426848e
small tweak to display cf that had reference dropped by RocksDbSegmen…
gfukushima Dec 15, 2023
7401b59
spotless
gfukushima Dec 15, 2023
1b7fb72
Fix classes that changed package
gfukushima Dec 15, 2023
11e6b05
spotless
gfukushima Dec 15, 2023
f2d01e2
Code review
gfukushima Dec 15, 2023
04f1aaa
Only clear DB when we have the exact amount of trie logs we want in m…
gfukushima Dec 15, 2023
2f01c5a
Trielogs stream to and from file to avoid possibly OOM
gfukushima Dec 18, 2023
56e4c8e
Process trie logs in chunks to avoid OOM
gfukushima Dec 18, 2023
78561b0
save and read in batches to handle edge cases
gfukushima Dec 19, 2023
42c72cf
save and read files to/from database dir
gfukushima Dec 20, 2023
9961fc2
Merge branch 'main' into x-trie-log-subcommand-2
gfukushima Dec 20, 2023
9389540
add unit tests and PR review fixes
gfukushima Dec 21, 2023
e3d4fbc
Merge branch 'main' into x-trie-log-subcommand-2
gfukushima Dec 21, 2023
c7144fe
spdx
gfukushima Dec 21, 2023
20b0ba5
Fix unit tests directory creation and deletion
gfukushima Dec 21, 2023
586ab25
rename Xbonsai-trie-log-pruning-enabled to Xbonsai-limit-trie-logs-en…
gfukushima Jan 4, 2024
67e6f3d
Import and export trie log subcommands
gfukushima Jan 4, 2024
b9640e5
PR review
gfukushima Jan 4, 2024
3bc1878
spotless
gfukushima Jan 4, 2024
d47ddf5
fix path resolver and added unit tests
gfukushima Jan 7, 2024
999edb6
Merge branch 'main' into import-export-trie-log
gfukushima Jan 8, 2024
1699fe4
fix unit test
gfukushima Jan 8, 2024
e679cb3
Merge remote-tracking branch 'origin/import-export-trie-log' into imp…
gfukushima Jan 8, 2024
5d3b4f2
fix unit test
gfukushima Jan 8, 2024
f839b75
Merge branch 'main' into import-export-trie-log
gfukushima Jan 8, 2024
0caa4cf
Add import and export to list of subcommands under --x-trie-log
gfukushima Jan 8, 2024
2d5d31d
Merge remote-tracking branch 'origin/import-export-trie-log' into imp…
gfukushima Jan 8, 2024
37df23e
Remove static from setup method
gfukushima Jan 8, 2024
5ce1800
change option name and fix descriptions
gfukushima Jan 8, 2024
98423dc
Merge branch 'main' into import-export-trie-log
gfukushima Jan 8, 2024
cf3a5e6
Fix subcommands descriptions
gfukushima Jan 9, 2024
c759bba
Merge branch 'main' into import-export-trie-log
gfukushima Jan 9, 2024
5fb9413
Remove old flag and move commands const into Unstable
gfukushima Jan 12, 2024
087c54b
Allow list of block hashes to passed as well as a file to be generate…
gfukushima Jan 12, 2024
4b033a3
Merge remote-tracking branch 'origin/import-export-trie-log' into imp…
gfukushima Jan 12, 2024
3a89ac3
Allow list of block hashes to passed as well as a file to be generate…
gfukushima Jan 12, 2024
9be7d13
Fix broken test when replaced the old option
gfukushima Jan 12, 2024
75d1c3b
import and export using rlp
jframe Jan 19, 2024
5eb4cda
Merge branch 'main' into import-export-trie-log
jframe Jan 19, 2024
5425075
tests for exporting and importing multiple trielogs
jframe Jan 19, 2024
2de0b19
Merge branch 'import-export-trie-log' of github.com:gfukushima/besu i…
jframe Jan 19, 2024
248a776
Merge branch 'main' into import-export-trie-log
jframe Jan 19, 2024
55d653e
fix build
jframe Jan 19, 2024
c9d38f0
Merge branch 'main' into import-export-trie-log
jframe Jan 19, 2024
b9d7620
Merge branch 'main' into import-export-trie-log
jframe Jan 22, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
spotless
Signed-off-by: Gabriel Fukushima <gabrielfukushima@gmail.com>
  • Loading branch information
gfukushima committed Jan 4, 2024
commit 3bc1878424b2e91fc7e4c91804e4ed67c6acbe16
Original file line number Diff line number Diff line change
Expand Up @@ -45,8 +45,7 @@ public class DataStorageOptions implements CLIOptions<DataStorageConfiguration>
private static final String BONSAI_TRIE_LOG_PRUNING_ENABLED =
"--Xbonsai-trie-log-pruning-enabled";

private static final String BONSAI_LIMIT_TRIE_LOGS_ENABLED =
"--Xbonsai-limit-trie-logs-enabled";
private static final String BONSAI_LIMIT_TRIE_LOGS_ENABLED = "--Xbonsai-limit-trie-logs-enabled";

// Use Bonsai DB
@Option(
Expand Down