Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

save blocks to postgres and serve via RPC get_block() #405

Draft
wants to merge 240 commits into
base: main
Choose a base branch
from

Conversation

grooviegermanikus
Copy link
Collaborator

Store and expose blocks via RPC get_block method. It uses PostgreSQL database with a rolling per-epoch schema partitioning.
The feature is optional and can be turned on by:

  • enable_postgres_block_store_importer: will run imp

Performance

Save time samples from lite-rpc service running on same machine as database. Note that this gives unrealistically low numbers!

Saving block 274506709@confirmed to postgres took 1.20ms for block and 89.53ms for 1701 transactions (4x500 chunks)
Saving block 274506710@confirmed to postgres took 1.23ms for block and 91.08ms for 1036 transactions (3x500 chunks)
Saving block 274506711@confirmed to postgres took 1.04ms for block and 68.03ms for 1042 transactions (3x500 chunks)
Saving block 274506713@confirmed to postgres took 1.45ms for block and 112.30ms for 2102 transactions (4x526 chunks)
Saving block 274506714@confirmed to postgres took 0.89ms for block and 85.43ms for 530 transactions (2x500 chunks)
Saving block 274506715@confirmed to postgres took 0.69ms for block and 59.01ms for 1274 transactions (3x500 chunks)
Saving block 274506716@confirmed to postgres took 1.23ms for block and 89.16ms for 1184 transactions (3x500 chunks)
Saving block 274506717@confirmed to postgres took 0.89ms for block and 114.35ms for 753 transactions (2x500 chunks)
Saving block 274506718@confirmed to postgres took 0.98ms for block and 121.74ms for 879 transactions (2x500 chunks)
Saving block 274506719@confirmed to postgres took 0.77ms for block and 91.44ms for 719 transactions (2x500 chunks)
Saving block 274506720@confirmed to postgres took 0.98ms for block and 119.94ms for 3158 transactions (4x790 chunks)
Saving block 274506721@confirmed to postgres took 1.03ms for block and 93.44ms for 1078 transactions (3x500 chunks)
Saving block 274506722@confirmed to postgres took 0.93ms for block and 80.33ms for 1015 transactions (3x500 chunks)
Saving block 274506723@confirmed to postgres took 1.00ms for block and 76.07ms for 1210 transactions (3x500 chunks)
Saving block 274506724@confirmed to postgres took 1.54ms for block and 169.81ms for 2487 transactions (4x622 chunks)
Saving block 274506725@confirmed to postgres took 1.02ms for block and 101.69ms for 1670 transactions (4x500 chunks)
Saving block 274506726@confirmed to postgres took 1.05ms for block and 90.57ms for 1661 transactions (4x500 chunks)

What's missing?

  • The data model is not completed and only partially compatible with Full RPC.
  • WS subscription is not there
  • use PgBouncer for read connections

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant