Combine Cel2 migration scripts#148

Merged

alecps merged 78 commits intocelo6from

cel2-migration-script

Jun 25, 2024

Commits on Jun 12, 2024

Initial script to play with celo DB history migration

mcortesi
authored and
alecps
committed
Can Read All the headers

authored and
alecps
committed
Adds new command to migrate ancients db

mcortesi
authored and
alecps
committed
Adds comment

mcortesi
authored and
alecps
committed
Adds extension methods for transformation

mcortesi
authored and
alecps
committed
Implements Transform CeloBody

mcortesi
authored and
alecps
committed
Adds impl that runs steps in a concurrent pipeline

mcortesi
authored and
alecps
committed
Adds transformHead, verify hashing works

mcortesi
authored and
alecps
committed
add migration for non-frozen blocks

alecps
authored and
alecps
committed
copy over entire db and modify in place, works with op-geth at piersy/minimal-data-migration

alecps
authored and
alecps
committed
remove unecessary copying, cleanup code

alecps
authored and
alecps
committed
close and reopen DBs

alecps
authored and
alecps
committed
migrate newdb in place

alecps
authored and
alecps
committed
saving progress

authored and
alecps
committed
Refactor code to improve database migration process

mcortesi
authored and
alecps
committed
better logging

mcortesi
authored and
alecps
committed
refactor: inline parMigrateAncientRange

mcortesi
authored and
alecps
committed
Remove frozen blocks from nonAncient DB

mcortesi
authored and
alecps
committed
check hash matches on nonAncients migration

mcortesi
authored and
alecps
committed
clean up branch

mcortesi
authored and
alecps
committed
decode into new types

alecps
authored and
alecps
committed
fix transformHeader

mcortesi
authored and
alecps
committed
make old freezer not readonly so that .meta files are created

alecps
authored and
alecps
committed
add configurable memory limit

alecps
authored and
alecps
committed
add comment about memory

alecps
authored and
alecps
committed
Added celo-dbmigrate Makefile target

jcortejoso
authored and
alecps
committed
Added dockerfile for celo-dbmigrate and celo-migrate tools

jcortejoso
authored and
alecps
committed
Workflow for running cel2-migration-tool

jcortejoso
authored and
alecps
committed
Update cel2-migration-tool image registry

jcortejoso
authored and
alecps
committed
update op-geth to point to https://github.com/celo-org/op-geth/commits/piersy/for-use-with-migrated-celo-datadir-use-gas-limit-differentiation-rebased-celo6/

alecps
authored and
alecps
committed
add celo6 logging

alecps
authored and
alecps
committed
rename scripts to celo-migrate-state and celo-migrate-blocks

alecps
authored and
alecps
committed
first pass at combining scripts

alecps
authored and
alecps
committed

Commits on Jun 13, 2024

saving progress on testing

alecps
authored and
alecps
committed
fix lint error, use %w to fmt errors

alecps
authored and
alecps
committed

Commits on Jun 14, 2024

add updated state migration input files to testdata

alecps
authored and
alecps
committed
add ability to run block and state migration seperately or together

alecps
authored and
alecps
committed
add option for migrating only frozen blocks

alecps
authored and
alecps
committed
remove old scripts

alecps
authored and
alecps
committed

Commits on Jun 17, 2024

minor logging improvements in block migrations

alecps
authored and
alecps
committed
invert clearNonAncients flag logic --> keepNonAncients, make dry-run flag only apply to state migration

alecps
authored and
alecps
committed
adds README, improves logging

alecps
authored and
alecps
committed
fix lint err

alecps
authored and
alecps
committed

Commits on Jun 18, 2024

Fix Makefile and Dockerfile
palango
committed
move createNewDbIfNotExists

alecps
authored and
alecps
committed
rename keep-non-ancients

alecps
authored and
alecps
committed
update TODO to add more context and state changes

alecps
authored and
alecps
committed
Remove channel buffers from ancients migration

authored and
alecps
committed
bump default batch size to 100000

alecps
authored and
alecps
committed
add back extended usage string

alecps
authored and
alecps
committed
add info on state migration to README

alecps
authored and
alecps
committed

Commits on Jun 20, 2024

remove --state-dry-run flag

alecps
authored and
alecps
committed
update default batch size to 50k

alecps
authored and
alecps
committed

Commits on Jun 21, 2024

Adding building for op images
jcortejoso
committed
Setting our values for image registry and repository
jcortejoso
committed
update README

alecps
authored and
alecps
committed
fix logging when newAncients > oldAncients

alecps
authored and
alecps
committed
fix return value when skipping ancients

alecps
authored and
alecps
committed
skip transforming block bodies that have already been transformed

alecps
authored and
alecps
committed
misc. fixes to get re-runs with --keep-non-ancients working

alecps
authored and
alecps
committed
adds TODO

alecps
authored and
alecps
committed
addresses cosmetic feedback

alecps
authored and
alecps
committed
add flag for specifying a buffer

alecps
authored and
alecps
committed

Commits on Jun 24, 2024