Add more slasher backends (redb, sqlite, postgres)

[michaelsproul]

Description

It would be great to have even more choice for slasher database backends beyond LMDB and MDBX. This could also be a good way for us to try out a couple of new DBs to see if they'd be suitable for the beacon node itself (which currently only supports LevelDB).

The candidates I'd like to try are:

• redb (Redb slasher backend impl #4529)
• postgres
• sqlite (Sqlite slasher backend impl #4666).

I think redb would be the easiest, as it's most similar to the existing backends. Then sqlite, because it can be embedded in process and we already use it in the VC via rusqlite. Most difficult would be postgres, because we'd either need to assume that a postgres server is running (yuck) or run one ourselves in a Docker container or something (also kind of yuck).

Steps to resolve

1. Add a new variant for the database to the DatabaseBackend enum.
2. Add a new file in e.g. slasher/src/database/redb_impl.rs, with types named similarly to the other existing backends, see lmdb_impl.rs
3. Add variants to the enums in interface.rs for the new backend.
4. Get the slasher tests to pass with the DEFAULT_BACKEND set to the new backend, and cargo test --release -p slasher. We can worry about the feature flag stuff later.

[eserilev]

I think I saw you mention this as a project idea in the Ethereum Protocol Fellowship repo. I plan on participating in the upcoming cohort and would be very interested in picking this up as my project.

[Gua00va]

I think I saw you mention this as a project idea in the Ethereum Protocol Fellowship repo. I plan on participating in the upcoming cohort and would be very interested in picking this up as my project.

I am also interested in taking this as my project idea. Would you mind working as a team?

[eserilev]

I am also interested in taking this as my project idea. Would you mind working as a team?

yeah! would be great to work on this as a team

[Gua00va]

I am also interested in taking this as my project idea. Would you mind working as a team?

yeah! would be great to work on this as a team

Great! Ping me in the discord server. I have the same username.

[michaelsproul]

Now that we have a couple of new backends in the works, namely Redb and Sqlite, it would be great to see a performance shoot-out. For the slasher the key metric is the time taken to process each batch, which is recorded in Prometheus as slasher_process_batch_time. We have a Grafana dashboard for this here: https://github.com/sigp/lighthouse-metrics/blob/master/dashboards/Slasher.json

A good test would be to start-up a slasher on the same machine with each of the backends -- LMDB, Redb, Sqlite -- and compare the performance. To prevent them competing for CPU and I/O it's probably best to run the slashers one at a time (not concurrently). Running each one for a few hours should be enough to get a feel for its performance. If one or both of the new backends are competitive in this test we can then look into running a longer test (2 weeks+) in which we can measure the total disk usage and the long-term performance.

I have a shared execution node on mainnet (running eleel) which I could grant access to for this experiment. If this is something you're interested in doing @eserilev, DM me your SSH pubkey on Discord.

[eserilev]

@michaelsproul this sounds great, I am sending you my SSH pubkey now. thank you!

[eserilev]

sqlite metrics: https://snapshots.raintank.io/dashboard/snapshot/TW4SxQCSehrE8534trSC0Kc9vIt2szLg?orgId=2&refresh=5s

ended up reaching 640mb in db size after about 2.5 hours

going to run lmdb and mdbx next to compare