Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Consider making --no-rocksdb-compaction the default #10216

Closed
mvines opened this issue May 25, 2020 · 2 comments
Closed

Consider making --no-rocksdb-compaction the default #10216

mvines opened this issue May 25, 2020 · 2 comments
Assignees
Milestone

Comments

@mvines
Copy link
Member

mvines commented May 25, 2020

As observed in #10199, the rocksdb compaction phase of Blockstore::purge_slots() on a fully loaded ledger can take 30-200 minutes. Is this even worth it?

  • If no, replace --no-rocksdb-compaction with --rocksdb-compaction
  • If yes, consider breaking out compaction from Blockstore::purge_slots() into a separate process that runs less frequently than every 512 roots (which currently effectively means it's always running, but only after the --limit-ledger-size shred limit is reached). Perhaps at the start of every epoch.
@mvines mvines added this to the v1.2.0 milestone May 25, 2020
@mvines mvines self-assigned this May 25, 2020
@mvines
Copy link
Member Author

mvines commented May 25, 2020

I have two API machines running side by size with a 1TB ledger, one with --no-rocksdb-compaction and one without.

  • --no-rocksdb-compaction present: DISK READ: 1079.34 K/s | DISK WRITE: 2.50 M/s
  • --no-rocksdb-compaction absent: DISK READ: 87.06 M/s | DISK WRITE: 89.11 M/s

@mvines
Copy link
Member Author

mvines commented Jun 5, 2020

After observing some nodes for a week, I feel that we do want to compact still by default but we don't need to compact that frequently. Once a day ought to be ample, which is what #10414 implements

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant