Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP: Trim blocks before applying ledger state #1864

Draft
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

sgillespie
Copy link
Contributor

Description

Reduces the total size of the ledger size by tampering with blocks before applying them to the ledger state. In this change, we clear out all multiassets from txouts.

To test this, I removed all ledger state snapshots and started DB sync, which forces it to replay the ledger state for genesis. I've used the time program (not the bash builtin) to measure the maximum amount of memory as reported by the OS (Resident Set Size).

Latest revision in master:

        Command being timed: "cardano-db-sync --config db-sync-config.json --socket-path /run/cardano-node/node.socket --schema-dir ../../cardano-db-sync/schema --state-dir ledger"
        User time (seconds): 39703.05
        System time (seconds): 6046.32
        Percent of CPU this job got: 137%
        Elapsed (wall clock) time (h:mm:ss or m:ss): 9:15:57
        Maximum resident set size (kbytes): 19599504
        Major (requiring I/O) page faults: 3124
        Minor (reclaiming a frame) page faults: 7021739
        Voluntary context switches: 37058762
        Involuntary context switches: 1129102
        Page size (bytes): 4096
        Exit status: 0

Using the changes in this PR:

        Command being timed: "cardano-db-sync --config db-sync-config.json --socket-path /run/cardano-node/node.socket --schema-dir ../../cardano-db-sync/schema --state-dir ledger"
        User time (seconds): 50988.88
        System time (seconds): 4944.32
        Percent of CPU this job got: 78%
        Elapsed (wall clock) time (h:mm:ss or m:ss): 19:53:47
        Maximum resident set size (kbytes): 14068792
        Major (requiring I/O) page faults: 1
        Minor (reclaiming a frame) page faults: 4904552
        Voluntary context switches: 33438678
        Involuntary context switches: 638268
        Page size (bytes): 4096
        Exit status: 0

Important: Note that this PR can't be merged until the corresponding cardano-ledger PR is integrated and released

Checklist

  • Commit sequence broadly makes sense
  • Commits have useful messages
  • New tests are added if needed and existing tests are updated
  • Any changes are noted in the changelog
  • Code is formatted with fourmolu on version 0.10.1.0 (which can be run with scripts/fourmolize.sh)
  • Self-reviewed the diff

Migrations

  • The pr causes a breaking change of type a,b or c
  • If there is a breaking change, the pr includes a database migration and/or a fix process for old values, so that upgrade is possible
  • Resyncing and running the migrations provided will result in the same database semantically

If there is a breaking change, especially a big one, please add a justification here. Please elaborate
more what the migration achieves, what it cannot achieve or why a migration is not possible.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant